Enterprise AI Analysis

Leveraging Electronic Health Record (EHR) Data

EHRs are a rich source of longitudinal data, providing retrospective insights into a patient's health trajectory. Common features extracted from EHRs for cancer prediction models include demographics, diagnoses, laboratory tests, and prescriptions. Other features like symptoms, referrals, and even free-text notes are also utilized.

However, EHR data presents unique challenges for longitudinal analysis: data irregularity (uneven time intervals), data sparsity (infrequent observations), data heterogeneity (diverse patient trajectories), and model opacity (complex models can be hard to interpret). Despite these, EHRs offer significant benefits, being more reflective of real-world clinical practice and often more cost-effective than prospective data collection.

Effective utilization often involves strategies like individual trajectory modeling or imputing missing data for feature engineering approaches, and padding sequences or forward-filling for deep learning models to handle irregular or sparse data.

Understanding Prediction Frameworks

The review identified various prediction tasks, each with distinct time windows and objectives:

• Risk Prediction Models: Aim to identify high-risk populations for developing cancer within a future "prediction window" (e.g., 3-60 months). Data prior to this window, typically the full available history, is used.
• Cancer Detection Models: Focus on predicting the presence of cancer at a specific "prediction point" using data from an "observation window" before diagnosis. "Early detection" models incorporate a "lead time" (gap between last measurement and outcome) to prevent using data too close to diagnosis. "Follow-up periods" for controls are crucial to ensure they truly remain cancer-free.
• Recurrence or Metastasis Prediction Models: Forecast the likelihood of cancer returning or spreading. These models often use surveillance data from after the initial cancer diagnosis, or even pre-diagnosis data, and may have varied observation and prediction window definitions.

A significant finding was the lack of consistency in defining these time windows across studies, even for models predicting the same cancer type. This highlights a need for standardized reporting and comparative analyses across different window configurations to determine optimal clinical utility.

Addressing Bias & Ensuring Reproducibility

A major finding from the PROBAST assessment was that 90% of the included studies were at high risk of bias. Common sources of bias included inappropriate study design (e.g., case-control designs without proper adjustment) and insufficient sample sizes, particularly for outcome groups.

Reproducibility of research was also a concern, with only about a third of studies making their code available online. While open access to health data is rare due to confidentiality, clear reporting of cohort selection and study settings is vital. The absence of a standardized approach to defining prediction windows further complicates comparisons and reproducibility.

To improve the quality of future AI models in cancer prediction, researchers must prioritize rigorous methodology, adhere to reporting guidelines like TRIPOD-AI, ensure adequate follow-up for control populations, use comprehensive performance measures, and account for overfitting through robust validation strategies.