In medical documentation, vast amounts of unstructured text are generated that are still underutilized in current prognostic models. We investigate the potential of self-hosted large language models (LLM) to extract clinically meaningful, patient-specific information from routine clinical notes for personalized risk stratification in cancer care. We collected real-world medical notes from 2,708 non-small cell lung cancer (NSCLC) patients and 814 colon cancer patients documented before treatment at a large comprehensive cancer center. LLMs extracted key prognostic indicators, including comorbidities, metastatic sites, and qualitative descriptors of patient condition, in a zero-shot manner without prior task-specific training. Integrating these LLM-derived features into machine learning models significantly improved the prediction of overall survival compared to TNM staging alone (C-Index: NSCLC, 0.72 vs 0.64; colon cancer, 0.70 vs 0.59), and surpassed models using text embeddings. Based on the LLM-informed risk scores, patients were stratified into four distinct risk groups, enabling reclassification of 61.4% of NSCLC and 68.3% of colon cancer patients. Analysis of model drivers revealed that LLM-derived factors, such as the physical condition, substantially modulated the prognostic impact of TNM stage. These findings highlight the potential of self-hosted LLM to derive prognostically relevant information from unstructured clinical documentation and support clinical decision-making.
