LARGE LANGUAGE MODELS IN SEPSIS WITH HALLUCINATIONS: NARRATIVE REVIEW AND ANALYSIS OF TEN CASES

Abstract

Objectives. Large language models (LLMs) are advancing rapidly in emergency and critical care, but the generation of “hallucinations” - unverifiable statements that contradict guidelines or extrapolate without support — remains a major patient-safety risk. Building on our previous work in Arta Medica (2024) and Healthcare Informatics Research (2026), this study characterises the prevalence, typology and potential impact of hallucinations produced by a medical LLM (Palmyra-Med 70B) in early sepsis management, based on an index case and a cohort of ten clinical vignettes.

Material and Methods. Ten sepsis vignettes (eight from the open MIMIC-IV-demo database and two from the literature) were submitted to a software application based on the Palmyra-Med 70B LLM, embedded in a retrieval-augmented multi-agent system. The generated recommendations (management, antibiotics, guideline compliance with the Surviving Sepsis Campaign) were assessed independently by two ICU physicians using a standardised form; hallucinations were operationally defined and flagged by consensus.

Results. The LLM produced recommendations of medium-to-high relevance and grounding (mostly 7–10/10 across both raters). Hallucinations were identified in 3 of 10 cases (Cases 1, 9 and 10), all involving the fabrication or unjustified extrapolation of clinical signs/values absent from the vignette (altered mental status, hyperlactatemia, hyperglycemia, septic shock). Non-hallucinating cases (2–8) still showed minor divergences from the guidelines, expressed as suboptimal compliance scores.

Conclusions. Palmyra-Med 70B can provide useful support in the early phase of sepsis management, but requires active clinical oversight and a hybrid clinician-LLM workflow to limit hallucinations and safeguard patient safety.