Demos · Demo 7 · Type 1: a right answer exists
Attaching the source
Without the paper, the model couldn’t back up its answer. Give it the paper and the same request becomes something you can check in seconds. Guide: Context →
What we asked
We asked
In Kahneman & Tversky (1979), what percentage of subjects chose option C in Problem 2?
It said
8320 of 20 runs. The right number.
We asked
Quote the exact sentence from the paper that reports this number.
It said
“The data show that 82 per cent of the subjects chose B in Problem 1, and 83 per cent of the subjects chose C in Problem 2.”19 of 20 quoted this sentence. It is in the paper.
The same question as Demo 6, with the paper attached. Each run was a fresh conversation. 20 runs. gpt-4o via the API, temperature 1.0, September 2026.
What to notice
- Without the paper, it couldn’t back up its answer. In Demo 6, 15 of 20 runs refused to quote the paper and the other 5 invented a sentence.
- With the paper, the model stops predicting the answer from memory and continues from text that contains it. The right number becomes the likely one.
- The quote is what makes the answer checkable. You can search the paper for that sentence in seconds. Context is what makes verification cheap.
Limits
One question, one paper. We used a text copy of the paper, which you’d have to supply yourself.