Designing AI summaries that help people decide
Guide · 6 min read · By Utkarsh Dhairya Panwar · Updated
Design an AI summary around the reader's decision, usually "is this worth my time?". Lead with what it's about and who it's for, include the one detail that would change the decision, make it quicker than judging the source, label it as AI and link back to the source.
A summary is for deciding, not for replacing
There are two kinds of summary, and products often ship the wrong one. A summary that replaces the source is judged on completeness. A summary that helps someone decide is judged on whether it contains the facts that decision turns on. Most users need the second kind: they want to know, cheaply, whether the full thing deserves their time or attention.
My examples come from SoundSphere, a 4-week podcast app concept where summaries let listeners judge an episode before starting it (designed, never shipped or tested with real usage), and from the pull request side of my work at CodeAnt AI, where a summary's job is to tell a reviewer where to look first. The general principles for AI features, like evidence and dismissal, are in my guide to designing AI features users trust; this page is only about summaries.
What goes in the summary
| Decision | The summary must include | Can leave out |
|---|---|---|
| Is this episode worth an hour? | The topic, the guest's angle, one specific claim or story, the level (beginner or deep) | The full argument, timestamps for everything |
| Does this pull request need a careful review? | What changed in plain words, the risky part, anything flagged as high severity | File-by-file description of trivial changes |
| Should I read this long thread? | Whether a decision was made, what it was, and whether I'm asked to do anything | Who said what, in order |
Write the decision at the top of your design doc before anyone writes a prompt. The summary's content follows from it, and so does how you'll judge whether it works.
Before and after
| Generic summary | Decision-ready summary |
|---|---|
| "In this episode, the hosts discuss productivity, habits and the importance of focus in modern work." | "A neuroscientist argues that most to-do apps make focus worse, with one experiment from her lab. Practical, light on jargon. Best part starts around minute 22." |
| "This PR updates several files to improve the authentication flow." | "Moves session tokens from localStorage to an HTTP-only cookie. Touches login, logout and refresh. One high-severity flag: the refresh endpoint no longer checks expiry." |
I wrote both examples for this page. The generic versions are accurate and useless, because they could describe a hundred other episodes or pull requests. The better versions each contain the one detail that would make someone say yes or no.
Format and placement
In SoundSphere I made a choice I'd still make: the summary was a short listening layer you could hear, with the full episode one tap away, rather than text under the title. For audio, a spoken summary keeps people in the medium they came for.
- Put the summary where the decision happens: on the feed card or episode page, at the top of the pull request, above the thread. Never behind a button.
- Match the medium. Audio for audio, a short paragraph for text, a structured list for code changes.
- Keep the path to the source one tap away, and make it continue from the right place (the timestamp, the file, the message).
- Label it plainly as an AI summary. A neutral label is enough; a sparkle icon isn't an explanation.
How long, and when to stay quiet
A summary has to be quicker than the decision would be without it. If people normally judge an episode by listening for two minutes, a summary that takes ninety seconds is barely worth having.
Compression also has a floor. Cut too much and the summary stops saying anything specific, which puts you back at the generic version above. One of the open questions I wrote down for SoundSphere was how short a summary can get before it loses meaning, and I'd test that with real readers rather than pick a word count. Design the short and the expanded version together, and watch where people need to move from one to the other.
- Don't summarise things that are already short. A three-line commit message doesn't need a summary.
- Don't summarise when being wrong is expensive and the reader can't check. Show the source instead.
- Say when the summary is partial: "Based on the first 40 minutes" or "Generated files skipped".
- Let people turn summaries off. Some readers will always want the original.
Measuring whether it works
Measure what people do after reading the summary. For a pull request, that's whether reviewers spend their time on the risky files, and whether review time drops without more bugs getting through. For an episode or an article, it's whether the summary leads people into the full thing or replaces it.
If it replaces it for most people, either the summary is doing the source's job (fine for a thread, bad for a creator's episode) or the source wasn't worth it. Both are worth knowing.
SoundSphere case study: The full concept: research signals, the summary-first listening layer and what it would need to prove next.
If you're adding summaries or other AI output to a product, my AI and developer tools design service covers that work.
Questions people ask
What makes a good AI summary?
It contains the detail the reader's decision depends on, it's faster to read than the decision would otherwise take, it's labelled as AI-generated, and it links back to the source.
Where should an AI summary go in the UI?
Where the decision happens: on the card, at the top of the page or pull request, above the thread. If people have to click to see it, many won't.
How long should an AI summary be?
Short enough to be quicker than judging the source directly. The right length depends on the content and the decision, so test it with real users rather than picking a word count.
Should summaries be generated ahead of time or on demand?
Ahead of time when the summary sits where people decide, like a feed card or the top of a pull request, because waiting for it defeats the point. On demand is fine for long documents someone has already chosen to open.
Sources
- AI Summaries of Reviews, Nielsen Norman Group
- Explainability + Trust, Google PAIR (People + AI Guidebook)
Working on this?
Tell me what you're building and where it's stuck: book a 30-minute call, email work@utkarshdesign.com or send a brief.