When a “Summary” Isn’t a Summary
And what to do about it
SIGNAL
A recent academic review has confirmed something many of us suspected: when large AI models like ChatGPT or Claude “summarize” a document, they often don’t. What they actually do is generate text that looks like a summary, based more on what they’ve seen in their training than on the text you gave them.
Another study on summarizing clinical evidence shows that LLMs are nowhere near the autonomous assistants vendors promise. The models' inability to incorporate real-time research updates means they're perpetually behind on recent trial data (a critical issue for rapidly evolving fields like oncology, where treatment protocols change quarterly). The study also underlines their complete inability to exercise what physicians develop over decades of practice—that nuanced clinical judgment necessary to weigh complex, patient-specific variables.
What’s the implication? LLM summarization is often a synthetic hallucination of relevance—less summary, more simulacrum. And this matters because we’re not just automating reading—we’re automating understanding. And if the summary is wrong, the decision it supports might be, too.
STORY
Last year, IT advisor
a ran a test. He asked ChatGPT to summarise a policy document from the Dutch pensions system—a topic he knows cold. The model returned something concise, coherent… and subtly wrong.Rather than identifying the real regulatory change (a legal redefinition of fund governance), the summary fixated on broad, generic language about “investment sustainability.” A plausible angle, but not the one actually discussed in the original. To Gerben, it was clear: the model hadn’t read the document in any human sense. It had merely guessed at what a “pension regulation summary” might sound like, based on token patterns.
In his blog post, When ChatGPT Summarises, It Actually Does Nothing of the Kind, Gerben detailed how this “summary” pulled the reader toward common talking points in the financial sector—points consistent with LLM training data, but irrelevant to the document itself.
In our LinkedIn discussion, I suggested this isn’t just a bug—it’s a systematic effect. Gerben clarified: “It looks a bit of a tug of war between the prompt (the text to summarise) and the parameters (based on training material).” When the parameters win, the summary veers off course; when the prompt dominates, you get superficial shortening. But real summarization—preserving the author's intent and logic while abstracting detail—is something else entirely. And it’s not what LLMs are doing.
HUMAN OVERRIDE
Summarizing something properly is hard. It means reading carefully, understanding the key ideas, and deciding what matters. It’s not just chopping text down to size.
AI models aren’t doing this. They generate text based on patterns, not meaning. They don’t know what’s important. They can’t tell a crucial point from background noise. That’s why their summaries can look right but be wrong.
So here’s how to protect yourself:
0. Don’t be lazy. Read the whole damn text first.
Yes, it's time-consuming. But if the stakes are high, you owe it to yourself—and others—to know what’s actually in the source.
1. Start with purpose
Ask: who is this summary for, and why? A CEO, a regulator, a team lead? That should shape what goes in (and what gets left out).
2. Use AI as a rough draft tool
Let the model pull out bullet points. Then you decide what to keep, reword, or throw away. Think of it as scaffolding, not a finished product.
3. Do a side-by-side check
Compare the summary with the original. Did anything get twisted? Are key ideas missing? If yes, rewrite.
4. Prompt in steps, not all at once
Try breaking it down: “List the key findings,” then, “Rephrase each in 1–2 plain sentences.” It can help—but don’t expect miracles.
5. For important stuff, go human
Medical summaries. Legal memos. Risk reports. These aren’t tasks to fully outsource to a machine. If accuracy matters, get a human to do it.
6. For the technical guys (drawing from the initial study’s findings)
Low temperature. If you're using platforms like GroqCloud (which lets non-coders adjust temperature), set it to 0 for high-stakes scientific summarization. The difference is remarkable—and potentially dangerous when overlooked.
Skip those "please be accurate" prompts—they backfire. Explicitly asking LLMs to "avoid inaccuracy" actually increased algorithmic overgeneralizations when summarizing scientific texts. It's like telling someone "don't think about elephants"—you guarantee they will! This pattern held across multiple models and contexts in the researchers’ testing.
Consider Claude or older models for scientific content. After comparing 10 influential models (including DeepSeek, GPT-4o, and Claude 3.7 Sonnet), the study found Claude consistently stayed closest to the original text's generalization scope. Even more surprising? Older models like GPT-3.5 often produced more faithful scientific summaries than their newer, larger cousins.
Enforce past tense in scientific summaries. Top medical journals already do this for good reason. Present-tense summaries of scientific findings dramatically increased overgeneralization.
Benchmark your LLMs using this three-step framework: (1) Prompt an LLM to summarize scientific texts, (2) Classify both original texts and summaries based on generic claims, tense, and action-guiding generalizations, and (3) Calculate an overgeneralization score by comparing these classifications.
SPARK | Are We Outsourcing Thinking?
When we let a tool summarize for us, what we’re really doing is trusting it to think for us. But these tools don’t think—they mimic. They simulate. And they’re good enough to fool us.
So the question isn’t just, “Is this summary correct?”
It’s: “Am I still doing the thinking?”
Here’s where to dig deeper:
PS–If you’re interested in working with me, there are a few ways we can partner: head over here and let me know.