AI in Healthcare: Opportunities & Challe

Cheatsheet Content

AI in Healthcare: Executive Summary Transformative Potential: AI (especially deep neural networks) shows promise in improving public health and healthcare delivery. Current Context: Enabled by dissatisfaction with the legacy US medical system, ubiquitous smart devices, and public demand for convenience. Focus: Technical capabilities, limitations, and applications within 10 years, as per JASON's study for HHS. Overarching Observations & Challenges Growing Role: AI is poised for transformative changes inside and outside clinical settings. Key Challenges: Gaining acceptance of AI in clinical practice (especially diagnostics). Leveraging personal networked devices and AI tools. Ensuring high-quality training data availability. Executing large-scale data collection, including missing streams (e.g., environmental exposures). Understanding and mitigating AI limitations. AI in Health Diagnostics: Opportunities & Implementation Deep Learning shows utility, especially in medical imaging. Rigorous testing and validation protocols are crucial. Advances in Medical Imaging Diabetic Retinopathy Detection: AI algorithms match/exceed manual assessment ($>$100,000 images, 97.5% sensitivity, 93.4% specificity). Potential for rapid, low-cost analysis and diagnostics in underserved populations. Dermatological Skin Cancer Classification: CNN algorithm (125,000 images) performs similarly to dermatologists (e.g., 72.1% accuracy for AI vs. 66% for dermatologists). Performance constrained by quality of clinician assessments in training data. Moving AI to Clinical Standards Acceptance: Requires extensive experience and validation, robust peer-reviewed R&D. FFRCT Example (CAD Diagnosis): Addresses significant clinical need by reducing unnecessary invasive tests. Must perform at least as well as existing standards. Requires substantial clinical testing across diverse situations. Must improve patient outcomes, quality of life, practicality, and reduce costs. AI vs. Physical Principles: AI (correlation-based) may face greater skepticism than methods based on physical principles. Validation: AI diagnostics replacing established steps need far more validation than tools providing supporting information. Recommendations: Support work for rigorous approval; create testing/validation methods for conditions differing from training sets. Digital Revolution, Data Quality & Missing Data Confluence of AI and Smart Devices Revolutionary Changes: Occurring outside traditional diagnostics via smart devices and mHealth apps. Interdependence: AI powers apps (e.g., Kardia Mobile), and devices generate massive datasets for AI development. Infrastructure Needs: Develop data infrastructure to capture and integrate smart device data. Ensure privacy and transparency of data use. Clinician adoption depends on integration with EHRs, liability, and billing. Foreign Systems: Track developments like DeepMind Health (UK NHS) for lessons on transparency and data access (e.g., blockchain for auditability). Challenges in Creating Comprehensive Datasets Data Quality & Access: Critical barriers to AI implementation in health. Defining Datasets: Major challenge for AI aiming to find novel disease correlations and personalize treatments. Concerns regarding Electronic Health Records (EHRs) Caution: Use EHRs as training sets with extreme care. Issues: Incomplete, lack interoperability, variable quality (not collected under research controls). Risk: Incorrect or correlated data can lead to misleading AI outputs. Study Example (CVD Prediction): ML algorithms offered only marginal improvements over standard tools with UK NHS EHR data. Transparency: 'Black-box' nature of ML means weighted risk factors can change idiosyncratically; need for transparent reporting. The Missing Environmental Data Stream Impact: Environmental exposures and social behaviors account for $\approx 60\%$ of premature deaths. Data Imbalance: Lack of diverse data capture for precision medicine, especially environmental toxicology and exposure. Recommendations for Data Collection: Integrate toxin screening (dioxin, lead) into routine blood panels. Add diet and environmental toxin questions to health questionnaires. Start urban sensing and tracking programs aligned with large health studies (e.g., All of Us). Support wearable device development for sensing environmental toxins and pathogens. Develop protocols and IT capabilities to collect and integrate diverse data streams. Advancing AI Development and Ensuring Success Crowdsourcing and Data Generation Value: Provides high-quality, labeled data and accelerates AI algorithm development (e.g., Kaggle competitions). Examples: Improving lung cancer screening, classifying genetic mutations. Recommendations: Support competitions to advance health data understanding. Share data in public forums to engage scientists. Main Challenge: Need for large, well-labeled public/semi-public datasets. Limitations, Transparency, and Safeguards Guarding Against Misinformation ("Snake Oil"): Potential for misleading websites, apps, and companies (e.g., MTHFR treatments, non-validated diagnostic apps). Engagement of learned bodies recommended to endorse best practices for AI deployment. Ensuring Reproducibility and Transparency: Deep learning models are complex and susceptible to adversarial examples; 'black-box' nature. Methods for transparent disclosure of large-scale computational models are essential but nascent. Recommendations: Support development/adoption of transparent processes and policies for reproducibility (data, software, workflows, environment details). Involve researchers, technologists, health professionals, industry, regulators, and patients in discussions. Summary of Findings and Recommendations Category Key Findings Key Recommendations AI in Clinical Practice New techniques require robust R&D. AI diagnostics replacing established steps need more validation than supporting tools. Prepare AI results for rigorous approval. Create testing/validation for conditions differing from training sets. AI and Smart Devices AI and smart devices are interdependent, driving changes outside clinical settings. Mobile devices generate massive datasets for AI. Develop data infrastructure for smart device data. Ensure privacy and transparency. Track foreign healthcare system developments. Training Databases High-quality data access is critical. EHRs require extreme care as training sets due to inaccuracies. Support development/access to research databases (labeled/unlabeled). Investigate incentivizing data sharing and new data ownership paradigms. Missing Data Gaps Health outcomes are heavily affected by environmental exposures/social behaviors. Imbalance in data capture, especially for environmental toxicology. Support ambitious environmental exposure data collection (e.g., toxin screening, urban sensing). Develop protocols to collect and integrate diverse data. Crowdsourcing Movement AI competitions encourage curated, labeled data and showcase AI capabilities. Support competitions to advance health data understanding. Share data in public forums for scientific discovery. Limitations and Safeguards Potential for misinformation ("Snake Oil"). Lack of transparency in large-scale computational models. Encourage transparent processes for reproducibility of computational models. Engage learned bodies to endorse AI deployment best practices.