SEARCH · Search PubMed

Results for “Foundation models”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 recordsLinked to original sources

PubMed2005

Law and order: assessing and enforcing compliance with ontological modeling principles in the Foundational Model of Anatomy.

The objective of this study is to provide an operational definition of principles with which well-formed ontologies should comply. We define 15 such principles, related to classification (e.g., no hierarchical cycles are allowed; concepts have a reasonable number of children), incompatible relationships (e.g., two concepts cannot stand both in a taxonomic and partitive relation), dependence among concepts, and the co-dependence of equivalent sets of relations. Implicit relations--embedded in concept names or inferred from a combination of explicit relations--are used in this process in addition to the relations explicitly represented. As a case study, we investigate the degree to which the Foundational Model of Anatomy (FMA)--a large ontology of anatomy--complies with these 15 principles. The FMA succeeds in complying with all the principles: totally with one and mostly with the others. Reasons for non-compliance are analyzed and suggestions are made for implementing effective enforcement mechanisms in ontology development environments. The limitations of this study are also discussed.

Anatomy↗

PubMed2026

Biological Foundation Models for Complex Disease Research and Clinical Translation.

Complex diseases, including cancer, rare genetic disorders, neurodevelopmental and psychiatric conditions, and neurodegenerative diseases, arise from interactions among genetic variation, gene regulation, and cellular states that are difficult to capture using a single data type or biological scale. Biological foundation models address this challenge by treating nucleotides and genes as tokens and learning representations that can be transferred to downstream biomedical and clinical tasks. In this review, we examine two major model classes, genomic sequence foundation models and cell foundation models, and compare their tokenization strategies, model architectures, pretraining objectives, and adaptation methods. We summarize their emerging applications in regulatory variant interpretation, disease-associated cell-state analysis, drug-response prediction, and therapeutic target discovery across complex diseases. We distinguish applications supported by experimental or retrospective validation from those that remain primarily computational or conceptual. We further discuss key challenges to clinical translation, including multimodal data integration, model interpretability, benchmarking, patient-specific prediction, and privacy protection. We highlight future opportunities to integrate biological foundation models with emerging frameworks of medical digital twins, agentic AI, and federated learning. By linking model design to translational goals, this review provides a practical framework for evaluating biological foundation models and their readiness for complex disease research and clinical use.

biological foundation model↗

PubMed1998

The digital anatomist foundational model: principles for defining and structuring its concept domain.

We define a foundational model as an abstraction of a body of knowledge that explicitly declares the principles and concepts necessary for coherently and consistently modelling a knowledge domain. Principles for a foundational model of anatomy are defined and used to specify the components of such a model. These components include an anatomy ontology (Ao), an anatomical structural abstraction (ASA), an anatomical transformation abstraction (ATA) and metaknowledge (Mk), which comprises the rules for representing relationships in the other three components of the model. The foundational model Fm is therefore specified as the four-tuple Fm = (Ao,ASA,ATA,Mk). We hypothesize that this abstraction captures the information that is sufficient and necessary for describing the anatomy of any physical entity that constitutes the body, as well as that of the body itself.

Anatomy↗

PubMed2003

Efficient web-based navigation of the Foundational Model of Anatomy.

The University of Washington's Foundational Model of Anatomy (FMA)1 is a complex, frame-based model composed of over 170,000 frames collectively specified by more than 1.4 million slot values. The Foundational Model Explorer (FME) was designed to provide simple and intuitive web access to this complex knowledge base. The Foundational Model Explorer (FME) (Figure 1) is built upon the Protégé 2000 knowledge base access library [http://protege.stanford.edu/]. The FME application, associated libraries, and the FMA database all reside on remote servers. Users of the FME are not required to download, install, or set-up any of these components. The FME application, in response to a query, retrieves the appropriate information from the FMA, formats these data into html documents, and transmits them, via the Internet, back to the client where they can be viewed from any standard web browser

Anatomy↗

PubMed2003

A reference ontology for biomedical informatics: the Foundational Model of Anatomy.

The Foundational Model of Anatomy (FMA), initially developed as an enhancement of the anatomical content of UMLS, is a domain ontology of the concepts and relationships that pertain to the structural organization of the human body. It encompasses the material objects from the molecular to the macroscopic levels that constitute the body and associates with them non-material entities (spaces, surfaces, lines, and points) required for describing structural relationships. The disciplined modeling approach employed for the development of the FMA relies on a set of declared principles, high level schemes, Aristotelian definitions and a frame-based authoring environment. We propose the FMA as a reference ontology in biomedical informatics for correlating different views of anatomy, aligning existing and emerging ontologies in bioinformatics ontologies and providing a structure-based template for representing biological functions.

Abstracting and Indexing↗

PubMed2003

Representing complexity in part-whole relationships within the Foundational Model of Anatomy.

The Foundational Model of Anatomy (FMA) is a frame-based ontology that represents declarative knowledge about the structural organization of the human body. Part-whole relationships play a particularly important role in this representation. In order to assure that knowledge-based applications relying on the FMA as a resource can reason about anatomy, we have modified and enhanced currently available schemes of meronymic relationships. We have introduced and defined distinct partitions for decomposing anatomical structures and attributed the part relationships in order to eliminate ambiguity and enhance specificity in the richness of meronymic relationships within the FMA.

Anatomy↗

PubMed2004

A relation-centric query engine for the Foundational Model of Anatomy.

The Foundational Model of Anatomy (FMA), a detailed representation of the structural organization of the human body, was constructed to support the development of software applications requiring knowledge of anatomy. The FMA's focus on the structural relationships between anatomical entities distinguishes it from other current anatomical knowledge sources. We developed Emily, a query engine for the FMA, to enable users to explore the richness and depth of these relationships. Preliminary analysis suggests that Emily is capable of correctly processing real world anatomical queries provided they have been translated into a constrained form suitable for processing by the query engine.

Anatomy↗

PubMed2026

Causal circuit tracing reveals distinct computational architectures in single-cell foundation models: inhibitory dominance, biological coherence, and cross-model convergence.

MOTIVATION: Sparse autoencoders (SAEs) decompose foundation-model activations into interpretable features, but the model-internal causal interactions between those features (i.e. what ablating one feature does to the others, as distinct from the biological causal structure of the underlying cells)-and how those model-internal relationships relate to biological structure-are uncharacterized in single-cell foundation models. RESULTS: We introduce model-internal causal circuit tracing-zeroing one SAE feature at a source layer and measuring the resulting change in all downstream SAE features, for each of 120 source features-and apply it to Geneformer V2-316M and scGPT whole-human across four conditions (96 892 ablation-derived edges, 80 191 forward passes). On annotation-selected source features, edges share GO/KEGG/Reactome/STRING/TRRUST ontology terms at 50.9%-68.5%, a 2.9-6.2× enrichment over a configuration-preserving permutation null (P<.002); on 20 randomly sampled source features this attenuates to 21.5%-26.3%-still 2.5-3.1× above null-quantifying the annotation-selection contribution. Inhibitory dominance (fraction of ablation edges with d<0, i.e. source activation supports downstream target) is 65.5%-89.4%. scGPT produces larger raw per-edge effects (mean |d|=1.40 versus 1.05); after feature-share normalization, Geneformer is stronger (paired gene-pair ratio 0.64 on 33 301 shared pairs). Cross-model consensus yields 1142 architecture-invariant domain pairs (ordered pairs of GO biological-process categories "A→B" each connected by at least one ablation edge in both models; 10.6× enrichment over permutation null; P<.001). Circuit edge magnitude explains <1% of the variance in marginal driver-gene coexpression on the same cells (R2=0.010, n=31 176): the graph encodes structure beyond bivariate correlation. Against a matched-cell-type ENCODE ChIP-seq prior, circuit-predicted transcription factor (TF)→target pairs are enriched 2.06× (Fisher OR 5.84), markedly higher than 1.12× against TRRUST; direct ChIP-seq-supported target pairs show 10-30× larger CRISPRi sign-bias-corrected excess than indirect pairs. Gene-level CRISPRi validation on Replogle K562 and the noncancer RPE1 arm (and a true primary-T-cell control from Shifrut E, Carnevale J, Tobin V et al. Genome-wide CRISPR screens in primary human T cells reveal key regulators of immune function. Cell 2018; 175: 1958-71.e15) after sign-bias correction shows excess over baseline of +0.03 and +0.35 percentage points on K562 and RPE1, respectively (baseline already 52%-56% from sign marginals); effect-magnitude Spearman correlations ρ≈0. Bootstrap and per-cell-type stability (N∈{50,100,200}; B cell, CD4 + T, macrophage) give Pearson r≥0.97 on shared edges with 100% sign agreement; edge Jaccard grows monotonically with sample size. The circuit graph is therefore highly reproducible as an effect-size map, cell type specific in edge identity, consistent with coexpression encoding, and weakly but detectably enriched for ChIP-seq-supported direct regulatory edges. AVAILABILITY AND IMPLEMENTATION: https://github.com/Biodyn-AI/bio-sae-circuits (Python). Archival DOI: 10.5281/zenodo.19,633,166 (Zenodo).

Humans↗

PubMed2025

Orthrus: Towards Evolutionary and Functional RNA Foundation Models.

In the face of rapidly accumulating genomic data, our ability to accurately predict key mature RNA properties that underlie transcript function and regulation remains limited. Pre-trained genomic foundation models offer an avenue to adapt learned RNA representations to biological prediction tasks. However, existing genomic foundation models are trained using strategies borrowed from textual domains that do not leverage biological domain knowledge. Here, we introduce Orthrus, a Mamba-based mature RNA foundation model pre-trained using a novel self-supervised contrastive learning objective with biological augmentations. Orthrus is trained by maximizing embedding similarity between curated pairs of RNA transcripts, where pairs are formed from splice isoforms of 10 model organisms and transcripts from orthologous genes in 400+ mammalian species from the Zoonomia Project. This training objective results in a latent representation that clusters RNA sequences with functional and evolutionary similarities. We find that the generalized mature RNA isoform representations learned by Orthrus significantly outperform genomic foundation models on mRNA property prediction tasks, and requires only a fraction of fine-tuning data to do so. Finally, we show that Orthrus is capable of capturing divergent biological function of individual transcript isoforms.

Journal Article↗

PubMed2025

MutBERT: probabilistic genome representation improves genomics foundation models.

MOTIVATION: Understanding the genomic foundation of human diversity and disease requires models that effectively capture sequence variation, such as single nucleotide polymorphisms (SNPs). While recent genomic foundation models have scaled to larger datasets and multi-species inputs, they often fail to account for the sparsity and redundancy inherent in human population data, such as those in the 1000 Genomes Project. SNPs are rare in humans, and current masked language models (MLMs) trained directly on whole-genome sequences may struggle to efficiently learn these variations. Additionally, training on the entire dataset without prioritizing regions of genetic variation results in inefficiencies and negligible gains in performance. RESULTS: We present MutBERT, a probabilistic genome-based masked language model that efficiently utilizes SNP information from population-scale genomic data. By representing the entire genome as a probabilistic distribution over observed allele frequencies, MutBERT focuses on informative genomic variations while maintaining computational efficiency. We evaluated MutBERT against DNABERT-2, various versions of Nucleotide Transformer, and modified versions of MutBERT across multiple downstream prediction tasks. MutBERT consistently ranked as one of the top-performing models, demonstrating that this novel representation strategy enables better utilization of biobank-scale genomic data in building pretrained genomic foundation models. AVAILABILITY AND IMPLEMENTATION: https://github.com/ai4nucleome/mutBERT.

Humans↗

PubMed2004

Evolution of a Foundational Model of Physiology: symbolic representation for functional bioinformatics.

We describe the need for a Foundational Model of Physiology (FMP) as a reference ontology for "functional bioinformatics". The FMP is intended to support symbolic lookup, logical inference and mathematical analysis by integrating descriptive, qualitative and quantitative functional knowledge. The FMP will serve as a symbolic representation of biological functions initially pertaining to human physiology and ultimately extensible to other species. We describe the evolving architecture of the FMP, which is based on the ontological principles of the BioD biological description language and the Foundational Model of Anatomy (FMA).

Computational Biology↗

PubMed2026

Accelerating inference in genomic and proteomic foundation models via speculative decoding.

MOTIVATION: Genomic and protein foundation models (GFMs and PFMs) have demonstrated strong performance in learning the language of DNA and proteins, but their use in large-scale sequence generation is limited by the latency of autoregressive decoding. Because every token triggers a forward pass of a large Transformer, whose inference is relatively slow, long-sequence generation quickly becomes costly. RESULTS: In this work we adapt speculative decoding to a representative GFM: the DNA model DNAGPT and two representative PFMs: ProGen2 and ProtGPT2. We implement a probabilistic variant of speculative decoding, in which a lightweight draft model proposes short token spans and a larger target model verifies or corrects them in parallel, while preserving the target model's sampling distribution. Across all three models we systematically study the effect of speculation window length, temperature, draft architecture and prompt length, and we benchmark tokens per second over multiple runs per configuration. Speculative decoding yields consistent speedups over standard key-value cached decoding, with maximum observed speedup reaching 100% increase, while average gains across models ranging between 20% and 40% (e.g. 1.2×-1.4×), without changing the underlying target model predictions. Our results show that speculative decoding is a practical and model-agnostic strategy for accelerating genomic and proteomic sequence generation without sacrificing prediction quality. AVAILABILITY AND IMPLEMENTATION: All code and results are freely available at https://github.com/Georgakopoulos-Soares-lab/BioSpecDec.

Genomics↗

PubMed2004

Processes and problems in the formative evaluation of an interface to the Foundational Model of Anatomy knowledge base.

The Digital Anatomist Foundational Model of Anatomy (FMA) is a large semantic network of more than 100,000 terms that refer to the anatomical entities, which together with 1.6 million structural relationships symbolically represent the physical organization of the human body. Evaluation of such a large knowledge base by domain experts is challenging because of the sheer size of the resource and the need to evaluate not just classes but also relationships. To meet this challenge, the authors have developed a relation-centric query interface, called Emily, that is able to query the entire range of classes and relationships in the FMA, yet is simple to use by a domain expert. Formative evaluation of this interface considered the ability of Emily to formulate queries based on standard anatomy examination questions, as well as the processing speed of the query engine. Results show that Emily is able to express 90% of the examination questions submitted to it and that processing time is generally 1 second or less, but can be much longer for complex queries. These results suggest that Emily will be a very useful tool, not only for evaluating the FMA, but also for querying and evaluating other large semantic networks.

Anatomy↗

PubMed1999

Semi-automatic scene generation using the Digital Anatomist Foundational Model.

A recent survey shows that a major impediment to more widespread use of computers in anatomy education is the inability to directly manipulate 3-D models, and to relate these to corresponding textual information. In the University of Washington Digital Anatomist Project we have developed a prototype Web-based scene generation program that combines the symbolic Foundational Model of Anatomy with 3-D models. A Web user can browse the Foundational Model (FM), then click to request that a 3-D scene be created of an object and its parts or branches. The scene is rendered by a graphics server, and a snapshot is sent to the Web client. The user can then manipulate the scene, adding new structures, deleting structures, rotating the scene, zooming, and saving the scene as a VRML file. Applications such as this, when fully realized with fast rendering and more anatomical content, have the potential to significantly change the way computers are used in anatomy education.

Anatomy↗

PubMed2026

Bridging Ancestry Gaps in Genomic Risk Prediction with Tabular Foundation Models.

MOTIVATION: Models deployed for genomic prediction of diseases perform unevenly across populations, limiting clinical utility. Two factors drive this limitation: large imbalances in sample availability across ancestry groups and non-stationarity of genotype-phenotype effect sizes across the ancestry continuum. While tabular foundation models with in-context learning (ICL) have shown strong sample efficiency in other domains, their effectiveness for genotype-to-phenotype prediction and their robustness to ancestry-driven effect heterogeneity remain unclear. RESULTS: Using large, ancestrally diverse biobank data, we show that ICL-capable tabular foundation models reduce performance degradation in under-sampled ancestry groups compared to conventional supervised approaches. However, we find that prevailing models trained on existing synthetic tabular tasks fail when allele effect sizes vary across ancestry space. Treating genetic ancestry as a continuous variable, we introduce an instruction-tuning framework that exposes models to synthetic tasks with ancestry-dependent non-stationary effects. Instruction-tuned models achieve improved and more stable predictive performance across the genetic ancestry continuum, including for individuals distant from in-context exemplars in ancestry space. AVAILABILITY AND IMPLEMENTATION: All code for instruction-tuning models, synthetic task generation, data wrangling, and model evaluation, is publicly available at https://github.com/ai4pm/Bridging-Ancestry-Gaps-in-Genomic-Risk-Prediction-with-Tabular-Foundation-Models. The final instruction-tuned model (ICL-NS-G2P-proto) is also released in this repository. Detailed documentation is provided, including environment setup instructions and guidelines for running various parts. The instruction-tuning task datasets are available at https://zenodo.org/records/18309187.

Ancestry Continuum↗

PubMed2026

Multimodal Deep Learning and Foundation Models for Early Detection and Forecasting of Plant Diseases.

Plant diseases destroy 20-40% of global food production annually, posing a critical threat to food security for a projected population of 9.7 billion by 2050. Conventional diagnostic approaches relying on expert visual assessment are slow, costly, and unsuitable for modern agricultural scales. While deep convolutional neural networks demonstrated early promise, single-modality, image-centric systems consistently fail under real-world field conditions characterized by variable lighting, co-occurring infections, and cultivar diversity. This review synthesizes a decade of progress across four interconnected frontiers: the evolution of deep learning architectures for plant disease detection; the adaptation of foundation models including CLIP, SAM, and DINOv2 to agricultural contexts; the development of multimodal fusion frameworks integrating imagery, environmental, genomic, and hyperspectral data; and the transition from static disease diagnosis to descriptive comparison of reported metrics, which suggested that multimodal approaches frequently reported improved diagnostic performance relative to corresponding single-modality baselines, although direct cross-study comparison was limited by methodological heterogeneity. A systematic review following PRISMA guidelines identifies eligible comparative studies. Descriptive comparison of reported performance metrics across these studies indicated that multimodal approaches generally achieved higher accuracy and sensitivity than single-modality models, particularly for pre-symptomatic disease detection. Eight critical research gaps are identified, including the absence of a unified agricultural foundation model and limited climate-aware forecasting under non-stationary climate projections. A structured research agenda is proposed to accelerate translation from laboratory performance to globally equitable, field-deployable crop protection systems.

convolutional neural networks↗

PubMed1999

Conceptualization of anatomical spatial entities in the Digital Anatomist Foundational Model.

Anatomical spatial concepts are indispensable in educational and clinical discourse, yet a system for representing these concepts has not been proposed. Guided by explicit principles and definitions of the Digital Anatomist Foundational Model, we developed an ontology of spaces, surfaces, lines and points that are associated with anatomical structures. Ontologies for Anatomical Structure and Anatomical Spatial Entity were instantiated for the thorax, abdomen, pelvis and perineum. Representing the concepts in--part of--hierarchies as well, provided formative evaluation of the classification. We invite empirical evaluation of the Foundational Model through its use for educational and clinical applications.

Anatomy↗

PubMed2026

Data distribution impacts the performance and generalisability of contrastive learning-based foundation models of electrocardiograms.

Contrastive learning is a widely adopted self-supervised pretraining strategy, yet its dependence on cohort composition remains underexplored. We present Contrasting by Augmented Patient Electrocardiograms (CAPE) foundation model and pretrain on four cohorts (n = 5,203,269), from diverse populations across three continents (North America, South America, Asia). We systematically assess how cohort demographics, health status, and population diversity influence the downstream performance for prediction tasks also including two additional cohorts from another continent (Europe). We find that downstream performance depends on the distributional properties of the pretraining cohort, including demographics and health status. Moreover, while pretraining with a multi-centre, demographically diverse cohort improves in-distribution accuracy, it reduces out-of-distribution (OOD) generalisation of our contrastive approach by encoding cohort-specific artifacts. To address this, we propose the In-Distribution Batch (IDB) strategy, which preserves intra-cohort consistency during pretraining, discourages learning of spurious cohort-specific features, and instead promotes clinically meaningful variability within cohorts. This leads to improved out-of-distribution robustness, with gains of 9-40% in downstream label prediction performance. This work provides insights into pretraining strategies for more clinically deployable and generalisable foundation models.

Journal Article↗