Anyone who has sat beside a relative in a general practice consulting room knows the silence that opens it: the clinician scrolling, eyes moving down years of free text, looking for the thread. In a study published in JAMIA Open on 30 July 2025, researchers led by Lara Shemtob at Imperial College London put a stopwatch on exactly that silence. For primary care physicians reading a record of 10 contacts spread over at least two years, the median time simply to read and absorb it, before writing a single word of summary, was 7 minutes and 8 seconds. The range ran from 1 minute 25 seconds to 20 minutes. GPT-4, handed the same records, took a median of 16 seconds.
The design was deliberately plain. Seven primary care physicians wrote 70 simulated patient records, built from templates meant to reflect the demographics and chronic condition prevalence of patients registered with general practices in England, and published the records in full online. Each record was then summarised twice: once by a different clinician, once by GPT-4. That produced 140 summaries, and each summary was read and scored by three clinicians who were not told who had written it, giving 420 blinded ratings across eight quality domains plus an overall mark out of ten. No real patients were involved, and nothing here was tested on an actual clinic's workflow.
The headline result is a near miss. Clinician-authored summaries averaged 7.39 out of 10; the GPT-4 summaries averaged 7.00, a difference the authors report as statistically significant (P =.02). But the human scores were more scattered, and in 25 of the 70 records the AI summary actually outscored the clinician's. Raters were also good, though not infallible, at spotting the machine: they named the author correctly 89.8 per cent of the time, mistook 16 AI summaries (7.6 per cent) for human work, and mistook 10 human summaries (4.8 per cent) for the machine's.
The texture underneath the averages is where a family should look. Clinicians won on discipline: 96.7 per cent of ratings agreed the human summary contained relevant information, against 92.9 per cent for the AI, and on the question of avoiding superfluous material the gap was a canyon, 91.9 per cent for human summaries against 40.5 per cent for the machine's. The AI, in other words, says too much. But it leaves less out. Raters judged the AI summaries less likely to omit important information, and more likely to be written in language fit to hand to a patient: only 53.3 per cent of clinician summaries were rated acceptable to share with the person they described. In the authors' full write-up of the same analysis, posted as a preprint in February 2025, raters flagged 11 discrete instances of inaccuracy or incorrect information in AI summaries, of which exactly one was called a hallucination, a line about a "decision to escalate diabetes medication due to worsening control" when the repeat blood test had not yet been done. Human summaries drew 12 such flags, none labelled a hallucination.
For anyone living with a dementia, the stakes of this unglamorous task are easy to feel and hard to overstate. The record of a person with Alzheimer's disease, vascular dementia or a Lewy body diagnosis is rarely one tidy story: it is a decade of falls, a urinary infection that looked like a sudden decline, a medication started by one service and questioned by another, a spouse's remark in a letter that turns out to be the first sign anyone noticed. The summary is the handover document between everyone who will ever treat that person. It is worth being precise, then, about what this study did not do: the 70 records were invented for the purpose, not drawn from real care, and nothing in the published paper tests the machine on memory clinic records, on differentiating one dementia from another, or on any patient outcome at all. The authors flag their own constraints plainly, including that the participating clinicians were co-authors, convenience-sampled, and trained in summarisation for the study, and that simulated records may not capture the complexity and missing information of real ones.