ENTERPRISE AI ANALYSIS

Machine-Learned Codes from EHR Data Predict Hard Outcomes Better than Human-Assigned ICD Codes

Electronic health records (EHRs) are widely used in the United States. As of 2021, the adoption rate reached 96% for non-federal acute care [1]. These systems not only enable patients and physicians to access clinical information electronically and streamline medical billing and claims processes, but they have also been recognized as valuable resources for assessing study feasibility and facilitating patient recruitment in clinical research. Beyond their clinical utility, EHRs also support observational studies in epidemiology, risk factor analysis, genomic association studies, and other areas [2,3]. A key initial step in utilizing EHR data for research involves phenotyping, the identification of patients with specific traits or medical conditions [4]. The most common data source for phenotyping is the International Classification of Diseases (ICD) codes. Developed by the World Health Organization, the ICD system was designed to standardize the classification and coding of diseases and has since been widely adopted by healthcare providers worldwide [5]. Accurate assignment of ICD codes is essential for precisely identifying and characterizing target populations in interventional and observational studies. Typically, ICD codes are assigned by clinicians as part of their clinical practice or by trained medical coders who extract relevant information from clinical notes. However, this manual process is laborious and prone to error due to incomplete and disorganized documentation, limited resources (e.g., staffing and budget), human mistakes, and changes in coding standards [6]. Prior research has highlighted considerable variability and inconsistency in the assignment of ICD codes over time and across different clinical settings. For example, the transition from ICD-9 to ICD-10 can lead to substantial discrepancies in phenotype definitions [7–9]. Even within the same ICD version, inconsistencies in coding practices continue to be a significant challenge [10-12]. Such coding errors compromise subsequent clinical analyses by not accurately representing patient's underlying conditions. The recent emergence of machine learning (ML) has done much to improve and automate disease phenotyping using EHR data [13,14]. Compared with human coders, the advantages of the ML-based phenotyping methods include improved scalability for processing large volumes of data, enhanced consistency in applying phenotyping criteria, and the ability to detect complex patterns across multiple data types. However, most ML-based phenotyping methods typically require manual chart reviews by clinical experts to create "gold standards”, which is both time-consuming and tailored for specific research projects. On the other hand, although the initial ICD codes assigned may not be perfect, they are not random. Patients with specific diagnosis codes often share common clinical characteristics—such as demographics, symptoms, and treatments—embedded within the EHR as structured (e.g., medication, procedures, etc.) or unstructured data (e.g., clinical notes). When treating the originally assigned ICD codes as a “silver standard", ML can extract and capture these characteristics, creating a unique “fingerprint" of clinical features for each patient and generating ML-derived phenotypes. This methodology can potentially provide a more robust and consistent representation of patient conditions. ICD codes are organized hierarchically, starting with broad disease and condition categories (chapters). These chapters are further subdivided into blocks, which group related conditions more specifically. In this study, we trained a ML model to create coarse-grained phenotypes, referred to as ML-derived ICD blocks, corresponding to ICD code blocks. To evaluate our hypothesis that ML-derived ICD blocks, which are more closely aligned with actual patient conditions, would improve the performance of downstream tasks such as predicting critical outcomes (mortality or hospitalization), we compared predictive performance using original ICD blocks from the EHR with that achieved using ML-derived ICD blocks across a range of ML models. By using the notes, procedures performed, and medications given to develop the ML models, we hoped to achieve a more detailed picture of each patient's condition. We chose to use the VA data for two reasons: The patient population tends to be relatively stable with fewer losses due to changes in providers or hospitals than in other datasets, and the data available include all the clinical notes, not generally available in other very large datasets. While predominantly male, our focus here was on testing whether ML predictions were more effective than human-assigned codes and considered that the differing population was unlikely to affect our results.