# A standard evaluation pathway for AI-assisted pathology, from reader study to deployment

Source: https://onco.cc/ideas/idea-data-ai-pathology-evaluation-standard/  
OnCo record `idea-data-ai-pathology-evaluation-standard` (Idea). Data CC BY-NC 4.0, attribute "Data from OnCo (onco.cc)"; commercial use needs a licence.

## TL;DR

Pathology AI is cleared on uneven evidence, often without showing that pathologists using it do better than without. The proposed standard has two stages: a pre-registered, fully crossed multi-reader multi-case study comparing pathologist plus AI with pathologist alone, then a prospective deployment study measuring turnaround, tumour board discordance and treatment changes.

## Summary

Pathology AI is cleared on varied evidence, often without showing that pathologists using it perform better than without. The proposal is a standard two-stage pathway: a multi-reader multi-case study with a fully crossed design, pre-registered and adequately powered, measuring pathologist-plus-AI versus pathologist alone on diagnostic accuracy and time; then a prospective deployment study measuring turnaround, discordance at tumour boards, and downstream treatment changes, all reported to the registry.

## Fields

- Kind: Idea
- Last checked: 2026-09-08
- Hypothesis: Applying the standard will show that a minority of pathology AI tools improve pathologist accuracy in a fully crossed design, and those that do will show measurable turnaround and treatment-decision benefits in deployment.
- Rationale: Radiology has decades of reader-study methodology; pathology AI has borrowed the tools inconsistently. Standardisation makes results comparable and procurement rational.
- Proposed test: Apply the pathway to five cleared pathology AI tools (for example prostate biopsy detection, HER2 scoring, mitotic counting); publish results in a common format.
- Maturity: early-clinical
- Actor: research

## Sources

- Bottleneck evidence (AI that is built but not validated or deployed): Wu et al., How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals (Nature Medicine 2021): https://doi.org/10.1038/s41591-021-01312-x

## Connected records

- ideas: [AI quantification of HER2-low and HER2-ultralow](https://onco.cc/ideas/idea-ai-her2-low-scoring/)
- fronts: [AI & Computation](https://onco.cc/fronts/ai-computation/)
- technologies: [Digital pathology & AI](https://onco.cc/technologies/digital-pathology-ai/), [Histopathology & immunohistochemistry](https://onco.cc/technologies/histopathology-ihc/)
- companies: [Paige AI](https://onco.cc/companies/paige/), [PathAI](https://onco.cc/companies/pathai/)
- bottlenecks: [AI that is built but not validated or deployed](https://onco.cc/bottlenecks/b-ai-validation/)
- key papers: [How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals](https://onco.cc/key-papers/paper-wu-nat-med/)

---
JSON: https://onco.cc/api/v1/entities/idea-data-ai-pathology-evaluation-standard.json