How do you summarize long documents with AI?

Summarizing is one of the strongest uses of AI, and one of the easiest to do badly. A generic “summarize this” gives you a generic result.

For PDFs and contracts specifically, with verification steps, see the guide Summarize PDFs and contracts with AI.

Decide what kind of summary you need

• Executive summary: three to five bullets for a busy reader.
• Decision summary: what is being asked, options, recommendation.
• Action summary: who needs to do what by when.
• Risk summary: issues, exceptions, unusual terms.

The reader and purpose decide the format.

Prompts that work

Executive

Summarize the attached report for a COO who has two minutes. Give five bullets: the main conclusion, the three findings that matter, and the one decision needed. Quote any figures exactly and note the page or section each comes from.

Decisions and actions

From this document list (1) decisions already made, (2) open questions, (3) actions with owner and deadline if stated. If an owner or date is not stated, write “not stated” instead of guessing.

Audit the summary

What important caveats, exceptions or numbers in the original did your summary leave out? List up to five.

The instruction to say “not stated” matters. Without it, assistants may fill gaps with plausible guesses.

Very long documents

Every model has a limit on how much it can consider at once, called a context window, counted in tokens. Anthropic documentation for Claude defines the context window as all the text a language model can reference when generating a response, including the response itself. The Lost in the Middle study found that language model performance can degrade significantly when the position of relevant information changes. Details in the middle of a long file are the ones most worth spot-checking. See context window. If a file is too large, summarize section by section, then summarize the summaries. Ask for section headings first, then process each chapter separately.

Grounding the summary in quotes

For long documents, Anthropic guidance is to ask Claude to extract word-for-word quotes first before performing its task. Anthropic also says to explicitly give Claude permission to admit uncertainty. That is the basis of the “not stated” instruction in the prompts above. Anthropic guidance for Claude says critical information must always be validated, especially for high-stakes decisions.

Checking accuracy

1. Spot-check numbers and names against the source.
2. Ask for locations (page or section) so you can jump to the original.
3. Read the source sections that matter for any high-stakes decision. A summary is a map, not the territory.
4. Ask for contradictions: “Are there places where the document contradicts itself?”

See how to stop AI hallucinations for why summaries sometimes contain claims that are not in the source.

Types of documents

• Contracts: ask for obligations, deadlines, termination and liability terms, but treat as a first pass. See whether AI can review contracts.
• Meeting recordings and transcripts: see meeting notes with AI.
• Email threads: ask for the chronology of decisions.
• Research articles: ask for method, sample and limitations, not only findings.

Privacy

Uploading a document sends its contents to the AI provider. Confirm the tool is approved for that document’s sensitivity.

Scaling up

If you summarize the same kind of material repeatedly, such as weekly Drive updates or project folders, you can automate it. Learn AI Fast has a lab on summarizing a provided transcript and project folder and producing a status update.

Sources

• Anthropic, Context windows (Claude Platform Docs): “all the text a language model can reference when generating a response, including the response itself”
• Liu et al., Lost in the Middle: How Language Models Use Long Contexts (arXiv 2307.03172): “performance can degrade significantly when changing the position of relevant information”
• Anthropic, Reduce hallucinations (Claude Platform Docs): “ask Claude to extract word-for-word quotes first before performing its task”
• Anthropic, Reduce hallucinations (Claude Platform Docs): “Explicitly give Claude permission to admit uncertainty”
• Anthropic, Reduce hallucinations (Claude Platform Docs): “Always validate critical information, especially for high-stakes decisions”