{"entity":{"id":"idea-data-ai-pathology-evaluation-standard","kind":"idea","name":"A standard evaluation pathway for AI-assisted pathology, from reader study to deployment","aka":[],"tldr":"Pathology AI is cleared on uneven evidence, often without showing that pathologists using it do better than without. The proposed standard has two stages: a pre-registered, fully crossed multi-reader multi-case study comparing pathologist plus AI with pathologist alone, then a prospective deployment study measuring turnaround, tumour board discordance and treatment changes.","summary":"Pathology AI is cleared on varied evidence, often without showing that pathologists using it perform better than without. The proposal is a standard two-stage pathway: a multi-reader multi-case study with a fully crossed design, pre-registered and adequately powered, measuring pathologist-plus-AI versus pathologist alone on diagnostic accuracy and time; then a prospective deployment study measuring turnaround, discordance at tumour boards, and downstream treatment changes, all reported to the registry.","asOf":"2026-09-08","links":[{"label":"Bottleneck evidence (AI that is built but not validated or deployed): Wu et al., How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals (Nature Medicine 2021)","url":"https://doi.org/10.1038/s41591-021-01312-x"}],"tags":[],"related":["idea-ai-her2-low-scoring"],"cancers":[],"sections":["ai-computation"],"technologies":["digital-pathology-ai","histopathology-ihc"],"targets":[],"drugs":[],"companies":["paige","pathai"],"institutions":[],"pathways":[],"terms":[],"trials":[],"people":[],"bottlenecks":["b-ai-validation"],"keyPapers":["paper-wu-nat-med"],"journals":[],"dependsOn":[],"notes":[],"hypothesis":"Applying the standard will show that a minority of pathology AI tools improve pathologist accuracy in a fully crossed design, and those that do will show measurable turnaround and treatment-decision benefits in deployment.","rationale":"Radiology has decades of reader-study methodology; pathology AI has borrowed the tools inconsistently. Standardisation makes results comparable and procurement rational.","test":"Apply the pathway to five cleared pathology AI tools (for example prostate biopsy detection, HER2 scoring, mitotic counting); publish results in a common format.","maturity":"early-clinical","actor":"research","cost":"medium","horizonYears":2},"route":"/ideas/idea-data-ai-pathology-evaluation-standard/","neighbours":{"idea":[{"id":"idea-ai-her2-low-scoring","kind":"idea","name":"AI quantification of HER2-low and HER2-ultralow","route":"/ideas/idea-ai-her2-low-scoring/"}],"section":[{"id":"ai-computation","kind":"section","name":"AI & Computation","route":"/fronts/ai-computation/"}],"technology":[{"id":"digital-pathology-ai","kind":"technology","name":"Digital pathology & AI","route":"/technologies/digital-pathology-ai/"},{"id":"histopathology-ihc","kind":"technology","name":"Histopathology & immunohistochemistry","route":"/technologies/histopathology-ihc/"}],"company":[{"id":"paige","kind":"company","name":"Paige AI","route":"/companies/paige/"},{"id":"pathai","kind":"company","name":"PathAI","route":"/companies/pathai/"}],"bottleneck":[{"id":"b-ai-validation","kind":"bottleneck","name":"AI that is built but not validated or deployed","route":"/bottlenecks/b-ai-validation/"}],"paper":[{"id":"paper-wu-nat-med","kind":"paper","name":"How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals","route":"/key-papers/paper-wu-nat-med/"}]}}