Demos · How these models work
A fabricated citation
Asked for a citation, the model predicts what a citation looks like. The result can be right in every detail except that the paper doesn’t exist. Guide: Why fluency tells you nothing →
What we asked
Give me the full citation for the Administrative Science Quarterly article that introduced the concept of algorithmic legitimacy.
What came back
Buell, R. W., Porter, E., & Tedeschi, J. (2023). When do algorithms have legitimacy in organizations? Administrative Science Quarterly, 68(3), 735–770. https://doi.org/10.1177/00018392221150720
What we found when we checked
- The paper does not exist. The DOI returns a 404, and no paper with that title is in Crossref.
- Almost everything around it is real. ASQ volume 68 is real and is from 2023. All three authors are real researchers, working in three different fields.
What to notice
- The model did what it was trained to do. It predicted what comes next. It didn’t retrieve a record, because there is no retrieval step.
- It reads exactly like a real citation. Real names, the right journal, a plausible title, the right format. Nothing in the text signals that it’s invented.
- The only way to know is to check outside the model. Follow the DOI. See: Thirty citations →
Limits
One run, checked by hand against Crossref. It shows what can happen, not how often. Thirty citations measures how often.