Biomedical Foundation Models

Learning a molecular language for protein interactions is crucial for advancing drug discovery. Foundation models, trained on diverse biomedical data like antibody-antigen interactions and small molecule-protein interactions, are transforming this field. Unlike traditional computational approaches, they widen the search scope for novel molecules and refine it to eliminate unsuitable ones, emphasizing the detailed nuances in molecular structure and dynamics.

IBM Research biomedical foundation model (BMFM) technologies leverage multi-modal data of different types, including drug-like small molecules and proteins (covering a total of more than a billion molecules), as well as single-cell RNA sequence and other biomedical data.

Our research team has a diverse range of expertise, including computational chemistry, medicinal chemistry, artificial intelligence, computational biology, physical sciences, and biomedical informatics.

Our BMFM Technologies currently cover the following three domains:

Foundation models for Targets Discovery

Targets discovery models learn the representation of DNA, bulk RNA, single-cell RNA expression data and other cell level signaling information for the identification of novel diagnostic and therapeutic targets, allowing tasks such as cell type annotation and classification, gene perturbation prediction, disease state prediction, splice variants prediction, promoter region, and treatment response.

Foundation Models for Biologics Discovery

Biologics discovery models focus on biologic therapeutics discovery, with the goal of leveraging large-scale representations of protein sequences, structures, and dynamics for diverse downstream tasks associated with multiple biologics modalities. These models produce unified representations of biological molecular entities, integrating data such as protein sequences, protein complex structures, and protein-protein complex binding free energies into a single framework. These models can serve as the basis for diverse downstream tasks in therapeutic design, including candidate generation and assessment, across antibody, TCR, vaccine, and other modalities.

Foundation Models for Small Molecules Discovery

Small molecule foundation models are capable of supporting a wide range of downstream predictive and generative tasks in drug discovery. Molecules can be represented in various forms—including strings, graphs, and three-dimensional conformations. Our models are trained on multiple molecular representations to learn rich, low-dimensional embeddings that capture key biochemical features relevant to drug discovery. For predictive tasks, we adopt a flexible strategy that aggregates multiple transformer-based molecular view encoders. This approach yields versatile embeddings that maximize mutual information across different representations. For generative tasks, we employ diffusion-based denoising networks to design molecules optimized for specific properties. In line with our multi-view framework, generated molecules can be represented either as strings or as three-dimensional structures within a protein binding pocket.

Related Links and Activities

Open Source Models and Tools

Our team have been developing a broad suite of models and architectures and we have open-sourced two models (model weights available in Hugging Face and code in Git):

biomed.sm.mv-te-84m is a multi-modal, multi-view model trained on small molecules data.

• Hugging Face
• Git
• arXiv paper

biomed.omics.bl.sm.ma-ted-458m is a multi-aligned sequence-based multi-domain model trained on biologics, small molecules, and scRNA-seq data.

• Hugging Face
• Git
• arXiv paper

Biomedical AI Tools and Methods are also available online.

• BiomedSciAI git repository

AI Alliance working group: AI for Drug Discovery

• Open Innovation Forum
• AI Alliance

Cleveland Clinic and IBM Discovery Accelerator

• Cleveland Clinic Discovery Accelerator partnership and projects

Scientific Conferences

• International Conference on Intelligent Systems for Molecular Biology ISMB 2024
• ACS Fall 2024

For scientific journal publications please see our Publications section below.

Open Source Tools

• BiomedSciAI