Merlin (Stanford abdominal CT vision-language model)

Merlin is a 3D vision-language model for abdominal CT from Stanford whose image encoder is aligned with both the free-text radiology report and structured electronic health record codes. The 2024 arXiv paper describes training on 15,000 CT scans, amounting to 6M images and 6M EHR codes, and shows zero-shot classification of hundreds of findings plus report generation. It is aimed at radiology research groups exploring report-supervised learning, a route that avoids hand labelling every finding. Its limits are that the data come from a single institution and cover the abdomen only, so generalisation to other scanners, populations and body regions is untested; for oncology the relevance is in finding and describing lesions rather than staging. For a newcomer: Merlin learned to read abdominal CT scans by studying the reports radiologists wrote about them.