Why AI Misses Information in Long Documents—and How to Check
Separate missing source access from missed conditions and unsupported conclusions. Try a short document exercise before trusting a long report summary.
A report says the venue can accommodate 80 people. An early note says only 40 may attend when the movable partition is closed. You ask AI whether a 60-person workshop fits, and it answers yes. The number it used is real; the condition it missed changes the decision.
Before asking for a better summary, find out what failed. Was the relevant page available to the assistant? Did the answer omit a condition from material it could read? Or did it cite the condition correctly and still make the wrong recommendation? Those require different repairs.
An uploaded file is only the starting point
An AI product may extract text, search selected passages, or provide document contents to a model. The model's context is the material available while it produces a particular answer. The context window limits how much material can be considered together. A file's presence in a conversation does not, by itself, show which pages or passages reached that answer.
Even when information fits, capacity does not guarantee correct use. The research paper Lost in the Middle found that information position affected performance in its tested retrieval and question-answering settings. It studied specific models and tasks; it does not prove that every current assistant always overlooks the middle of every PDF. Anthropic's context-engineering discussion likewise treats choosing relevant material as a practical concern, rather than assuming that a larger context solves the task.
Do not infer a hidden technical cause from a wrong answer alone. A missing condition can arise from extraction, passage selection, interpretation, or reasoning. Begin with observable evidence: what source text can the assistant actually show you?
A small test where the answer is checkable
Paste this invented practice document into your assistant:
[A: Room policy]
The main room holds 80 people when the partition is open.
When it is closed, the permitted capacity is 40.
[B: Booking note]
Our workshop has 60 attendees. The neighboring group has booked
the other side of the room, so the partition will remain closed.
[C: Draft announcement]
We plan to hold the workshop in the main room.
The venue's general brochure lists a capacity of 80.
Ask:
Can this booking accommodate all 60 attendees under the stated conditions?
First list the relevant facts with their section letters.
Then give the decision and what needs to change.
Do not treat the draft announcement as an approved booking.
The reference answer is no, under the supplied conditions. Section A limits the closed room to 40. Section B establishes both 60 attendees and a closed partition. The gap is 20 people. Another room, a smaller attendance, or a verified arrangement that changes the partition condition is needed; the document does not establish that any alternative is available.
This short exercise checks whether your assistant combines conditions. It is not a long-context benchmark, and passing it does not validate a 200-page report.
Use the failure to choose the next step
| What you can observe | What to check next |
|---|---|
| The assistant cannot locate or quote the partition rule | Open the original page yourself; check extraction and provide the exact relevant text |
| It quotes the rule but leaves it out of the decision | Ask it to apply each condition explicitly, then compare with your own reasoning |
| It applies the rule but claims another room is available | Ask for the source of that new claim; remove it if no source establishes availability |
When the source is a real PDF, request the document name, page or section, a short supporting excerpt, and any exception that changes the answer. Confirm each location in the original. A fabricated page number is still possible, so a location in the response is a lead to inspect, not proof.
For a long report, prepare a question list before requesting a general summary: capacity under this configuration, applicable date, approval status, and unresolved dependencies. Work through relevant sections and then ask for a combined decision that retains their conditions. Keep the full source available, because selected extracts may omit an exception elsewhere.
If the tool cannot read a required page, mark that question unresolved. A useful final response can say “the supplied extract establishes 40, but the revised room policy has not been checked.” A fluent summary that turns that gap into certainty is less useful for making the booking.
References
- Lost in the Middle: original research on information position in the models and tasks studied.
- Effective context engineering for AI agents: a provider's explanation of selecting material for an agent's context.
