{"entity":{"id":"idea-moon-ai-cancer-answer-audit","kind":"idea","name":"A public benchmark and audit of chatbot answers to cancer questions","aka":[],"tldr":"Patients now ask AI assistants about their cancer. Test those assistants regularly on real questions, publish the scores, and certify the ones that meet the bar.","summary":"Large language models are becoming a primary source of medical information; studies show variable accuracy, occasional dangerous advice and inconsistent sourcing on oncology questions. The proposal is an independent, continuously updated benchmark of patient-style cancer questions (treatment options, side-effects, alternative therapies, prognosis) scored by oncologists and patients for accuracy, safety, sourcing and readability, with public leaderboards and a certification mark for assistants that meet thresholds and disclose limitations.","asOf":"2026-09-08","links":[{"label":"Bottleneck evidence (Misinformation and unproven therapies): Johnson et al., Cancer misinformation and harmful information on Facebook and other social media (JNCI 2022)","url":"https://doi.org/10.1093/jnci/djab141"}],"tags":[],"related":[],"cancers":[],"sections":[],"technologies":[],"targets":[],"drugs":[],"companies":[],"institutions":[],"pathways":[],"terms":[],"trials":[],"people":[],"bottlenecks":["b-misinformation","b-ai-validation"],"keyPapers":["paper-johnson-j-natl-cancer-inst"],"journals":[],"dependsOn":[],"notes":[],"hypothesis":"Public auditing raises the accuracy and safety of assistant answers to cancer questions across vendors within a year, and certified assistants are preferred by patient organisations.","rationale":"Benchmarks drive model behaviour in AI development; making the oncology benchmark public and patient-facing turns that pressure toward safety.","test":"Run the benchmark quarterly on major assistants for a year; measure score trajectories and adoption of the certification by patient organisations and health systems.","maturity":"early-clinical","actor":"data","cost":"small","horizonYears":1},"route":"/ideas/idea-moon-ai-cancer-answer-audit/","neighbours":{"bottleneck":[{"id":"b-ai-validation","kind":"bottleneck","name":"AI that is built but not validated or deployed","route":"/bottlenecks/b-ai-validation/"},{"id":"b-misinformation","kind":"bottleneck","name":"Misinformation and unproven therapies","route":"/bottlenecks/b-misinformation/"}],"paper":[{"id":"paper-johnson-j-natl-cancer-inst","kind":"paper","name":"Cancer Misinformation and Harmful Information on Facebook and Other Social Media: A Brief Report","route":"/key-papers/paper-johnson-j-natl-cancer-inst/"}]}}