openbmb/SciCore-Omics

<div align="center">

🧬 SciCore-Omics

A tri-modal foundation model unifying histology, spatial transcriptomics, and biological language

   

</div>

<p align="center"> <img src="https://raw.githubusercontent.com/OpenBMB/Scicore-Omics/main/figs/fig1.png" width="95%" alt="SciCore-Omics overview"> </p>

🔍 Overview

SciCore-Omics is a tri-modal biomedical foundation model that connects histology images, spatial transcriptomic profiles, and biological language for spatial biology and pathology-related reasoning.

The model introduces a gene-aware branch based on NicheFormer + Gene Q-Former + Gene Projector, enabling transcriptomic information to be aligned with the language-model token space.

SciCore-Omics supports:

• —🖼️ image-only reasoning;
• —🧬 gene-only reasoning;
• —🖼️🧬 joint image-gene reasoning;
• —💬 natural-language biomedical interpretation.

✨ Highlights

• —Tri-modal modeling of histology, spatial transcriptomics, and language
• —Gene-aware transcriptomic encoding with NicheFormer
• —Unified image-gene-text reasoning in the language-model space
• —Designed for spatial biology, pathology reasoning, and biomedical interpretation
• —Open-source model weights, code, and demo

🚀 Quick Start

This Hugging Face repository hosts the model weights.

For full inference and training code, please refer to the GitHub repository:

git clone https://github.com/OpenBMB/Scicore-Omics.git
cd Scicore-Omics

Download the model weights:

huggingface-cli download openbmb/SciCore-Omics \
  --local-dir ./weights/SciCore-Omics

Minimal loading example:

import torch
from transformers import AutoModel, AutoTokenizer, AutoProcessor

model_path = "openbmb/SciCore-Omics"

processor = AutoProcessor.from_pretrained(
    model_path,
    trust_remote_code=True
)

tokenizer = AutoTokenizer.from_pretrained(
    model_path,
    trust_remote_code=True
)

model = AutoModel.from_pretrained(
    model_path,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

model.eval()

For complete examples, please see:

https://github.com/OpenBMB/Scicore-Omics/tree/main/eval

📦 Resources

⚠️ Limitations

SciCore-Omics is released for research use only.

It may generate inaccurate or incomplete biomedical interpretations and should not be used as a standalone clinical diagnostic or treatment recommendation system.

📚 Citation

@misc{xiao2026scicoreomics,
  title  = {SciCore-Omics: a tri-modal foundation model unifying histology, spatial transcriptomics and language for spatial biology},
  author = {Xiao, Xinyu and Li, Yunfei and Zeng, Zheni and others},
  year   = {2026},
  note   = {Manuscript in preparation}
}

📄 License

This project is released under the Apache-2.0 License.