# Merlin (Stanford abdominal CT vision-language model)

Source: https://onco.cc/technologies/merlin-ct/  
OnCo record `merlin-ct` (Technology). Data CC BY-NC 4.0, attribute "Data from OnCo (onco.cc)"; commercial use needs a licence.

## TL;DR

Merlin is a model trained on 15,000 CT scans with their reports that can find and describe hundreds of findings.

## Summary

Merlin is a 3D vision-language model for abdominal CT from Stanford whose image encoder is aligned with both the free-text radiology report and structured electronic health record codes. The 2024 arXiv paper describes training on 15,000 CT scans, amounting to 6M images and 6M EHR codes, and shows zero-shot classification of hundreds of findings plus report generation. It is aimed at radiology research groups exploring report-supervised learning, a route that avoids hand labelling every finding. Its limits are that the data come from a single institution and cover the abdomen only, so generalisation to other scanners, populations and body regions is untested; for oncology the relevance is in finding and describing lesions rather than staging. For a newcomer: Merlin learned to read abdominal CT scans by studying the reports radiologists wrote about them.

## Fields

- Kind: Technology
- Status: emerging
- Last checked: 2026-09-08
- Tags: foundation-model; radiology
- Principle: 3D image encoder aligned with report text and structured codes.
- Since: 2024
- Strengths: Report-supervised 3D learning
- Limitations: Single institution; Abdomen only

## Sources

- Merlin (arXiv 2024): https://arxiv.org/abs/2406.06512

## Connected records

- fronts: [AI & Computation](https://onco.cc/fronts/ai-computation/), [Imaging](https://onco.cc/fronts/imaging/)
- technologies: [AI in radiology](https://onco.cc/technologies/radiology-ai-screening/), [CT (computed tomography)](https://onco.cc/technologies/ct/), [Pathology & radiology foundation models](https://onco.cc/technologies/pathology-foundation-model/)
- institutions: [Stanford Health Care / Stanford Cancer Institute](https://onco.cc/institutions/stanford/)
- roadmaps: [AI in oncology roadmap: pattern readers → foundation models → agents in the workflow](https://onco.cc/roadmaps/ai-oncology-roadmap/)

---
JSON: https://onco.cc/api/v1/entities/merlin-ct.json