From Academic Theory to Clinical Reality: A Dual Evaluation of Artificial Intelligence in Medical Education and Emergency Case Management

Abstract

Background: In many resource-constrained emergency departments, general practitioners provide the first clinical assessment and initial management before specialist input is available. Whether advanced AI systems can produce clinically useful and safe written management suggestions from concise emergency case summaries remains uncertain. This study compared GPT-5.2 Pro, Gemini 3 Pro, Glass Health, and general practitioners in the initial management of emergency medicine cases from a regional hospital. Methods: We conducted an observational, case-based comparative study in the emergency department of a 240-bed hospital in Algeria. A total of 147 adult emergency cases were retrospectively selected from routine records and reconstructed as standardized French vignettes limited to 100 words. Each vignette was submitted once to GPT-5.2 Pro, Gemini 3 Pro, Glass Health, and one of twelve practicing general practitioners. Responses were limited to 250 words, anonymized, and evaluated by blinded senior specialists using three predefined 0–5 clinical adequacy domains: clinical reasoning, therapeutic management, and clinical safety. The overall score was calculated as the mean of the three domain scores. Paired comparisons used Friedman tests, Kendall’s W, and Holm-adjusted post-hoc Wilcoxon signed-rank tests. Results: Overall clinical performance differed significantly across groups. Median overall scores were 4.33 [3.50–4.67] for GPT-5.2 Pro, 4.00 [3.67–4.67] for Gemini 3 Pro, 4.00 [3.67–4.67] for Glass Health, and 2.67 [2.00–3.33] for general practitioners (p < 0.001; Kendall’s W = 0.38). Each AI system scored higher than general practitioners for the overall score and for all three domains, with Holm-adjusted p values <0.001. No significant pairwise differences were observed between the three AI systems. The AI advantage persisted across predefined difficulty levels. GPT-5.2 Pro showed higher overall scores in medical than surgical cases, whereas no significant medical–surgical difference was observed for Gemini 3 Pro, Glass Health, or general practitioners. Prompt length in characters was not significantly associated with overall performance. Conclusions: In this specialist-blinded case-based evaluation using emergency cases reconstructed from routine hospital records, GPT-5.2 Pro, Gemini 3 Pro, and Glass Health achieved higher specialist-rated clinical performance than general practitioners for written initial management plans. These findings support further prospective evaluation of supervised AI-assisted decision support in emergency medicine, with attention to local validation, clinical governance, and data-protection requirements.