# Public gold-standard datasets for validating every cancer biomarker test

Source: https://onco.cc/ideas/idea-tr2-open-cdx-validation-sets/  
OnCo record `idea-tr2-open-cdx-validation-sets` (Idea). Data CC BY-NC 4.0, attribute "Data from OnCo (onco.cc)"; commercial use needs a licence.

## TL;DR

Anyone building a new test for HER2, PD-L1 or tumour DNA should be able to check it against the same public reference set. Today each developer validates on private data nobody can inspect.

## Summary

Analytical validation of biomarker tests uses proprietary sample sets, making performance claims incomparable. A public repository of consensus-annotated cases per biomarker (whole slide images with multi-pathologist scores, sequencing data with orthogonally confirmed variants, plasma samples with defined variant allele fractions) would allow head-to-head benchmarking and independent verification of any test or algorithm, including AI-based ones.

## Fields

- Kind: Idea
- Last checked: 2026-09-08
- Hypothesis: Tests benchmarked on the public sets will show performance differences hidden by developer-reported validations, and regulators will begin requiring performance on the reference sets within three years.
- Rationale: ImageNet-style benchmarks drove progress and honesty in machine learning; the Genome in a Bottle reference genomes did the same for variant calling.
- Proposed test: Release reference sets for HER2, PD-L1 and ctDNA; invite all commercial and academic tests to report; publish a leaderboard.
- Maturity: early-clinical
- Actor: data

## Sources

- Genome in a Bottle: https://www.nist.gov/programs-projects/genome-bottle

## Connected records

- ideas: [A registry of external validation datasets for cancer AI models, with mandatory reporting](https://onco.cc/ideas/idea-tr2-ai-external-validation-registry/), [An annual blinded shoot-out for liquid biopsy tests](https://onco.cc/ideas/idea-tr2-liquid-biopsy-challenge/), [One digital PD-L1 scale that maps across all the competing assays](https://onco.cc/ideas/idea-tr2-pdl1-digital-calibration/), [Version control and locked reference sets for AI algorithms used as companion diagnostics](https://onco.cc/ideas/idea-tr2-ai-cdx-change-control/)
- technologies: [Digital pathology & AI](https://onco.cc/technologies/digital-pathology-ai/), [Liquid biopsy (ctDNA)](https://onco.cc/technologies/liquid-biopsy/)
- terms: [Next-generation sequencing (NGS)](https://onco.cc/terms/ngs/)
- bottlenecks: [AI that is built but not validated or deployed](https://onco.cc/bottlenecks/b-ai-validation/), [Biomarkers are not validated or standardised](https://onco.cc/bottlenecks/b-biomarker-validation/)

---
JSON: https://onco.cc/api/v1/entities/idea-tr2-open-cdx-validation-sets.json