Enterprise AI Analysis

Patient data from 286 cases were anonymized and structured into clinical vignettes, including demographics, medical history, pathology (histologic subtype, tumor size, grade, ER/PR/HER2/Ki-67), radiological findings (multifocality, nodal status, PET/CT), and AJCC 8th edition clinical staging. A uniform prompt referencing ASCO, ESMO, and NCCN guidelines was used for all LLMs. MDTB consensus decisions served as the benchmark. LLM outputs were mapped to predefined treatment categories (systemic therapy, breast surgery, axillary management) and evaluated for concordance, Cohen's kappa, precision, recall, and F1 scores. Subgroup analyses examined performance across molecular subtypes and disease stages.

ChatGPT-4o achieved the highest overall concordance (83.2%) with MDTB decisions, followed by Claude (79.7%) and Gemini (79.4%). Agreement exceeded 90% in HER2-enriched and triple-negative breast cancer. F1 scores were highest for adjuvant systemic therapy (100) and neoadjuvant chemotherapy (≥91). However, performance significantly declined for surgical decisions, including mastectomy (<58) and axillary lymph node dissection (≤23.5). Stage-based analyses showed varied concordance, with high agreement in some stage III-IV subgroups and lower agreement in scenarios requiring complex, individualized decisions.

LLMs demonstrate substantial agreement with MDTB recommendations in structured, guideline-based breast cancer settings, particularly for systemic therapy planning. However, their performance declines when decisions require individualized clinical judgment, complex multimodal trade-offs, or nuanced interpretation of findings. These findings support further evaluation of LLMs as decision-support tools in straightforward cases, but complex surgical or multimodal treatment planning still requires expert multidisciplinary oversight. LLMs should be regarded as adjunctive tools that enhance, but do not replace, human expertise.