Chinese Large Language Models for Patient-Friendly Knee MRI Summaries in Sports Injury: A Guideline-Anchored Study of Quality and Robustness

Abstract

Reports describing sports-related knee MRI findings are usually written for clinicians and can be difficult for patients to interpret. We compared the quality and consistency of five widely used Chinese large language models (LLMs) when converting such reports into plain language. A guideline-anchored set of 50 simulated knee MRI reports was prepared, and the five models were tested with zero-shot prompting; DeepSeek-V4 was also tested with a few-shot prompt. The six experimental arms produced 300 summaries through the vendors' official web interfaces. Two blinded physicians rated factual accuracy (0-3), information coverage (0-5), and patient-oriented clarity (0-5). We additionally calculated mean readability grade level (mRGL) and BERT-based semantic similarity. Accuracy remained high across arms (2.60-2.83) without a significant pairwise difference after correction. Few-shot DeepSeek-V4 obtained the best coverage score (4.33 +/- 0.61), Doubao-Seed-2.0 generated the easiest text to read (mRGL 7.15), and GLM-5.2 remained closest to the wording of the source reports (similarity 0.72). The models were less alike in difficult combined injuries: Doubao-Seed-2.0 and Qwen3.8-Max showed the clearest decline, while few-shot DeepSeek-V4 had the lowest accuracy CV (12.7%). The findings indicate that Chinese flagship LLMs can produce generally accurate patient-facing knee MRI explanations, but they occupy different positions with respect to coverage, readability, fidelity, and robustness. Safe selection therefore requires more than a single average score.