| A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | AA | AB | AC | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Model name | Lead developers | Modality | Domain (Level 1) | Domain (Level 2) | Capabilities | Robustness tests | Source | |||||||||||||||||||||
2 | Meditron | EPFL | Language | General Biomedical | none | Medical reasoning (built on Llama 2) | Ablation analysis | https://arxiv.org/pdf/2311.16079 | |||||||||||||||||||||
3 | USFM | Fudan University | Vision | Radiology | Ultrasound | Image segmentation Image classification Image enhancement | Ablation analysis | https://www.sciencedirect.com/science/article/abs/pii/S1361841524001270 https://arxiv.org/abs/2401.00153 | |||||||||||||||||||||
4 | Med-PaLM | Google Research | Language | General Biomedical | none | QA | Ablation analysis | https://www.nature.com/articles/s41586-023-06291-2 | |||||||||||||||||||||
5 | Med-PaLM M | Google Research | Language Vision | General Biomedical | none | Visual QA Text summarization Image interpretation Medical report generation Genomic variant calling | Ablation analysis | https://ai.nejm.org/doi/full/10.1056/AIoa2300138 https://arxiv.org/abs/2307.14334 | |||||||||||||||||||||
6 | AnatCL | University of Turin EPFL | Vision | Radiology Neurology | Brain MRI | Classification tasks for multiple neurological diseases | Cross-validation | https://arxiv.org/pdf/2408.07079 | |||||||||||||||||||||
7 | GET | Columbia University Carnegie Mellon University | Language | Omics | Single-cell data | Enhancer target prediction cis-regulatory region prediction | Evaluation on external datasets | https://www.biorxiv.org/content/10.1101/2023.09.24.559168v2 | |||||||||||||||||||||
8 | Nucleotide Transformer | InstaDeep NVIDIA | Language | Omics | Genome sequence | Gene expression prediction Enhancer prediction Splicing site prediction | Evaluation on external datasets Cross-validation | https://www.biorxiv.org/content/10.1101/2023.01.11.523679v4.full.pdf | |||||||||||||||||||||
9 | UniBrain | Shanghai Jiao Tong University | Language Vision | Radiology Neurology | Brain MRI | Diagnosis of diverse brain diseases | Evaluation on external datasets Evaluation on different imbalanced datasets | https://arxiv.org/abs/2309.06828 | |||||||||||||||||||||
10 | CHIEF | Harvard University Stanford University | Vision | Pathology | WSI | Cancer call detection Gene mutation prediction Biomarker prediction Survival analysis | Evaluation on external datasets (multiple institutions) | https://www.nature.com/articles/s41586-024-07894-z | |||||||||||||||||||||
11 | Merlin | Stanford University | Language Vision | Radiology | CT | Semantic segmentation Radiology report generation Disease severity prediction | Evaluation on external datasets (multiple institutions) | https://arxiv.org/abs/2406.06512 | |||||||||||||||||||||
12 | RudolfV | Aignostics Technical University of Berlin | Vision | Pathology | WSI | Cell classification Tissue segmentation IHC scoring Diagnostic classification | Evaluation on multiple datasets | https://arxiv.org/abs/2401.04079 | |||||||||||||||||||||
13 | CopilotCAD | University of Pennsylvania | Language Vision | Radiology | CT | Text prompt-based image analysis | Evaluation on multiple datasets | https://arxiv.org/abs/2404.07424 | |||||||||||||||||||||
14 | CONCH | Harvard University Broad Institute | Language Vision | Pathology | WSI | Image classification, segmentation, retrieval, captioning in histopathology | Evaluation on multiple datasets | https://www.nature.com/articles/s41591-024-02856-4 | |||||||||||||||||||||
15 | Me-LLaMA | Yale University | Language | General Biomedical | none | QA, NER, NLI, relation extraction, text classification, summarization | Evaluation on multiple datasets | https://arxiv.org/abs/2402.12749 | |||||||||||||||||||||
16 | scGPT | University of Toronto | Language | Omics | Single-cell data | Cell type annotation Perturbation response prediction Reverse perturbation prediction Multi-batch and multi-omic integration | Evaluation on multiple datasets | https://www.nature.com/articles/s41592-024-02201-0 | |||||||||||||||||||||
17 | PMC-CLIP | Shanghai Jiao Tong University | Language Vision | General Biomedical | none | Image-text retrieval Image classification Visual QA | Evaluation on multiple datasets | https://link.springer.com/chapter/10.1007/978-3-031-43993-3_51 | |||||||||||||||||||||
18 | MAIRA-2 | Microsoft Research Cambridge University | Language Vision | Radiology Pulmonology | Chest X-ray | Grounded radiology report (with spatial annotation) | Evaluation on multiple datasets | https://arxiv.org/abs/2406.04449 | |||||||||||||||||||||
19 | LLaVa-Rad | Microsoft Research University of Washington Stanford University | Language Vision | Radiology Pulmonology | Chest X-ray | Radiology report generation | Evaluation on multiple datasets | https://arxiv.org/abs/2403.08002 | |||||||||||||||||||||
20 | MedSAM | University of Toronto | Vision | General Biomedical | none | Image segmentation in multiple data modalities | Evaluation on multiple datasets | https://www.nature.com/articles/s41467-024-44824-z | |||||||||||||||||||||
21 | PMC-LLaMA | Shanghai Jiao Tong University | Language | General Biomedical | none | QA Dialogue generation | Evaluation on multiple datasets | https://academic.oup.com/jamia/article/31/9/1833/7645318 | |||||||||||||||||||||
22 | Prov-GigaPath | Microsoft Research Providence University of Washington | Vision | Pathology | WSI | Gene mutation prediction Cancer phenotyping Image-report alignment | Evaluation on multiple datasets | https://www.nature.com/articles/s41586-024-07441-w | |||||||||||||||||||||
23 | MedDr | HKUST Cornell University | Language Vision | General Biomedical | none | Medical image diagnosis Visual QA Medical report generation | Evaluation on multiple datasets | https://arxiv.org/abs/2404.15127 | |||||||||||||||||||||
24 | Polaris | Hippocratic AI | Language | Clinical | EHR | QA Discharge note generation Dialogue generation | Evaluation on multiple datasets | https://arxiv.org/abs/2403.13313 | |||||||||||||||||||||
25 | DinoBloom | Technical University of Munich | Vision | Hematology | Light microscopy | Cell-type classification Disease subtyping | Evaluation on multiple datasets | https://link.springer.com/chapter/10.1007/978-3-031-72390-2_49 https://arxiv.org/pdf/2404.05022 | |||||||||||||||||||||
26 | BiomedGPT | Lehigh University University of Georgia | Language Vision | General Biomedical | none | Visual QA, report generation, summarization, disease diagnosis, image classification, clinical text understanding | Evaluation on multiple datasets Human evaluations Zero-shot evaluation | https://www.nature.com/articles/s41591-024-03185-2 https://arxiv.org/pdf/2305.17100 | |||||||||||||||||||||
27 | BioMistral | Nantes Université Avignon Université Zenidoc | Language | General Biomedical | none | Tailored for the biomedical domain, utilizing Mistral as its foundation model and further pre-trained on PubMed Central | Evaluation on multiple datasets Multilingual evaluation Robustness evaluation after quantization | https://arxiv.org/pdf/2402.10373 | |||||||||||||||||||||
28 | ESM1b | UCSF | Language | Omics | Genome sequence | predict the effects of all missense variants in the human genome | Evaluation on multiple datasets Permutation test | https://www.nature.com/articles/s41588-023-01465-0 | |||||||||||||||||||||
29 | RETFound | University College London NIHR Biomedical Research Center | Vision | Ophthalmology | Color fundus photography OCT | Disease detection, diagnosis, and prognosis of ocular diseases | Evaluation on shifted datasets by age | https://www.nature.com/articles/s41586-023-06555-x | |||||||||||||||||||||
30 | xTrimoGene | BioMap Tsinghua University | Language | Omics | Single-cell data (scRNA-seq) | Cell type annotation | Evaluation on shifted datasets by sparsity Perturbation response prediction | https://proceedings.neurips.cc/paper_files/paper/2023/hash/db68f1c25678f72561ab7c97ce15d912-Abstract-Conference.html | |||||||||||||||||||||
31 | BiomedParse | Microsoft Research University of Washington | Vision | General Biomedical | none | Image segmentation Object detection Object classification | Evaluation on synthetic data | https://arxiv.org/abs/2405.12971 | |||||||||||||||||||||
32 | Almanac Copilot | Stanford University | Language | Clinical | EHR | EHR information retrieval order placement clinical documentation | Evaluation on synthetic data | https://arxiv.org/abs/2405.07896 | |||||||||||||||||||||
33 | GeneCompass | Chinese Academy of Sciences | Language | Omics | Single-cell data | Cell type annotation Gene regulatory network prediction Drug dose response prediction Gene expression prediction Gene dosage sensitivity prediction | Gene perturbation prediction | https://www.nature.com/articles/s41422-024-01034-y | |||||||||||||||||||||
34 | scFoundation | Tsinghua University BioMap | Language | Omics | Single-cell data (scRNA-seq) | Gene expression enhancement tissue drug response prediction single-cell drug response classification single-cell perturbation prediction | Gene perturbation prediction | https://www.nature.com/articles/s41592-024-02305-7 | |||||||||||||||||||||
35 | DRAGON | Stanford University EPFL | Language | General Biomedical | none | Bidirectional pretraining of language and knowledge graph, improves QA, reasoning, and text classification tasks | none | https://openreview.net/forum?id=4NpoSrT8uU- | |||||||||||||||||||||
36 | PathChat | Harvard University | Language Vision | Pathology | WSI | Image captioning Image summarization Visual QA, etc | none | https://www.nature.com/articles/s41586-024-07618-3 | |||||||||||||||||||||
37 | EHRMamba | University of Toronto | Language | Clinical | EHR | Predictions of mortality, length of stay, readmission, disease conditions | none | https://arxiv.org/abs/2405.14567 | |||||||||||||||||||||
38 | CheXagent | Stanford University | Language Vision | Radiology | Chest X-ray | Image understanding QA Radiology report generation | none | https://openreview.net/forum?id=P3LOmrZWGR | |||||||||||||||||||||
39 | BrainLM | Yale University Baylor University | Language | Radiology Neuroscience | Brain fMRI | generalizes well to new, unseen fMRI data | none | https://openreview.net/forum?id=RwI7ZEfR27 | |||||||||||||||||||||
40 | ESM-2 | Meta AI New York University Stanford University | Language | Omics | Sequence-structure pairs | high-resolution atomic structures of proteins directly from primary sequences | none | https://www.biorxiv.org/content/10.1101/2022.07.20.500902v2.full.pdf | |||||||||||||||||||||
41 | LLaVa-Med | Microsoft Research | Language Vision | General Biomedical | none | Image captioning Visual QA Dialogue generation | none | https://openreview.net/forum?id=GSuP99u2kR | |||||||||||||||||||||
42 | Neuro-GPT | University of Southern California University of Montreal | Sequence | Neuroscience | EEG | Motor imagery classification | none | https://arxiv.org/abs/2311.03764 | |||||||||||||||||||||
43 | GSViT | Johns Hopkins University | Vision | General Biomedical | Surgical videos | Surgical phase detection | none | https://arxiv.org/abs/2403.05949 | |||||||||||||||||||||
44 | VoxelPrompt | MIT Harvard University | Language Vision | Radiology Neurology | Brain CT Brain MRI | ROI measurement and analysis Language characterization of image Multiple acquisition analysis Image manipulation | none | https://arxiv.org/abs/2410.08397 | |||||||||||||||||||||
45 | Med42-v2 | KAUST | Language | General Biomedical | none | QA Dialogue generation | none | https://arxiv.org/abs/2408.06142 | |||||||||||||||||||||
46 | AMIE | Google Research | Language | Clinical | none | Dialogue generation QA Text summarization | none | https://arxiv.org/abs/2401.05654 | |||||||||||||||||||||
47 | MoME | CUHK | Language Vision | Pathology | WSI | Cancer survival analysis | none | https://link.springer.com/chapter/10.1007/978-3-031-72083-3_30 | |||||||||||||||||||||
48 | CellPLM | Michigan State University Emory University | Language | Omics | Single-cell data (scRNA-seq) | scRNA-seq denoising Spatial transcriptomics imputation Cell type annotation | none | https://openreview.net/forum?id=BKXvPDekud | |||||||||||||||||||||
49 | Mammo-CLIP | Boston University University of Pittsburgh | Language Vision | Radiology | X-ray mammography | Classification tasks for estimating breast calcificaiton, mass, and density | none | https://link.springer.com/chapter/10.1007/978-3-031-72390-2_59 | |||||||||||||||||||||
50 | BiomedCLIP | Microsoft Research University of Washington | Language Vision | General Biomedical | none | Cross-modal retrieval, zero-shot image classification, medical visual question answering | none | https://arxiv.org/pdf/2303.00915 | |||||||||||||||||||||
51 | uniGradICON | UNC Chapel Hill | Vision | Radiology | CT MRI | Image registration for multiple organs or body parts | Out-of-distribution evaluation Zero-shot evaluation | https://arxiv.org/abs/2403.05780 | |||||||||||||||||||||
52 | Med-Gemini | Google Research | Language Vision | General Biomedical | none | Visual QA Text summarization Image interpretation Medical report generation | Random data splitting | https://arxiv.org/abs/2405.03162 | |||||||||||||||||||||
53 | |||||||||||||||||||||||||||||
54 | |||||||||||||||||||||||||||||
55 | |||||||||||||||||||||||||||||
56 | Abbreviation | Full name | |||||||||||||||||||||||||||
57 | WSI | Whole-slide imaging | |||||||||||||||||||||||||||
58 | OCT | Optical coherence tomography | |||||||||||||||||||||||||||
59 | EHR | Electronic health record | |||||||||||||||||||||||||||
60 | |||||||||||||||||||||||||||||
61 | |||||||||||||||||||||||||||||
62 | |||||||||||||||||||||||||||||
63 | |||||||||||||||||||||||||||||
64 | |||||||||||||||||||||||||||||
65 | |||||||||||||||||||||||||||||
66 | |||||||||||||||||||||||||||||
67 | |||||||||||||||||||||||||||||
68 | |||||||||||||||||||||||||||||
69 | |||||||||||||||||||||||||||||
70 | |||||||||||||||||||||||||||||
71 | |||||||||||||||||||||||||||||
72 | |||||||||||||||||||||||||||||
73 | |||||||||||||||||||||||||||||
74 | |||||||||||||||||||||||||||||
75 | |||||||||||||||||||||||||||||
76 | |||||||||||||||||||||||||||||
77 | |||||||||||||||||||||||||||||
78 | |||||||||||||||||||||||||||||
79 | |||||||||||||||||||||||||||||
80 | |||||||||||||||||||||||||||||
81 | |||||||||||||||||||||||||||||
82 | |||||||||||||||||||||||||||||
83 | |||||||||||||||||||||||||||||
84 | |||||||||||||||||||||||||||||
85 | |||||||||||||||||||||||||||||
86 | |||||||||||||||||||||||||||||
87 | |||||||||||||||||||||||||||||
88 | |||||||||||||||||||||||||||||
89 | |||||||||||||||||||||||||||||
90 | |||||||||||||||||||||||||||||
91 | |||||||||||||||||||||||||||||
92 | |||||||||||||||||||||||||||||
93 | |||||||||||||||||||||||||||||
94 | |||||||||||||||||||||||||||||
95 | |||||||||||||||||||||||||||||
96 | |||||||||||||||||||||||||||||
97 | |||||||||||||||||||||||||||||
98 | |||||||||||||||||||||||||||||
99 | |||||||||||||||||||||||||||||
100 |