Can an AI tell its own words from yours? This is "reality monitoring." In humans, failing at it is linked to hallucinations and delusions. In this work we investigated reality monitoring in 6 LLMs of different sizes in collaboration with @lazyneuron.bsky.social and @brianodegaard.bsky.social.
Jul 30, 2026 at 10:49 AM
Reality Monitoring in Large Language Models: Self-Knowledge That Transforms with Conversation Memory
A conversational AI that cannot tell its own output from what a user said will treat its own mistakes as user-provided facts. In humans, this capacity is called reality monitoring, and its failures are linked to hallucinations, delusions, and confabulation, yet whether LLMs possess it remains untested. Here we show, across two experiments and six LLMs, that source attribution depends on how conversational memory is structured: ceiling accuracy for self-generated content under minimal memory demands reverses to a fragile external-item advantage once episodic delay removes that shortcut. Feedback exposes two failures: in some models, internal and external judgments swap; in others, accuracy improves while confidence decouples from correctness, dissociations invisible to existing benchmarks. Across models, this pattern implicates active, not aggregate, parameter count. This suggests that as AI systems take on autonomous, multi-turn roles, evaluating what they know is not enough: tracking where that knowledge came from may matter equally.
AI hallucinations can’t be stopped — but these techniques can limit their damage
Developers have tricks to stop artificial intelligence from making things up, but large language models are still struggling to tell the truth, the whole truth and nothing but the truth.
AI hallucinations can’t be stopped — but these techniques can limit their damage
Developers have tricks to stop artificial intelligence from making things up, but large language models are still struggling to tell the truth, the whole truth and nothing but the truth.
The hardest question to answer about AI-fueled delusions
New research can’t yet say whether AI causes delusions or amplifies them, a distinction that will shape everything from high-profile court cases to safety rules for chatbots.
What are AI hallucinations? Why AIs sometimes make things up
When AI systems try to bridge gaps in their training data, the results can be wildly off the mark: fabrications and non sequiturs researchers call hallucinations.
What are AI hallucinations? Why AIs sometimes make things up
When AI systems try to bridge gaps in their training data, the results can be wildly off the mark: fabrications and non sequiturs researchers call hallucinations.
"AI Psychosis" in Context: How Conversation History Shapes...
Extended interaction with large language models (LLMs) has been linked to the reinforcement of delusional beliefs, attracting clinical and public concern. Yet most empirical work evaluates model...
OpenAI admits AI hallucinations are mathematically inevitable, not just engineering flaws
In a landmark study, OpenAI researchers reveal that large language models will always produce plausible but false outputs, even with perfect data, due to fundamental statistical and computational limits.
A Rational Analysis of the Effects of Sycophantic AI
People increasingly use large language models (LLMs) to explore ideas, gather information, and make sense of the world. In these interactions, they encounter agents that are overly agreeable. We argue that this sycophancy poses a unique epistemic risk to how individuals come to see the world: unlike hallucinations that introduce falsehoods, sycophancy distorts reality by returning responses that are biased to reinforce existing beliefs. We provide a rational analysis of this phenomenon, showing that when a Bayesian agent is provided with data that are sampled based on a current hypothesis the agent becomes increasingly confident about that hypothesis but does not make any progress towards the truth. We test this prediction using a modified Wason 2-4-6 rule discovery task where participants (N=557) interacted with AI agents providing different types of feedback. Unmodified LLM behavior suppressed discovery and inflated confidence comparably to explicitly sycophantic prompting. By contrast, unbiased sampling from the true distribution yielded discovery rates five times higher. These results reveal how sycophantic AI distorts belief, manufacturing certainty where there should be doubt.
“Hallucinating” AI models help coin Cambridge Dictionary’s word of the year
Cambridge: "When an artificial intelligence hallucinates, it produces false information."
AI Delusions Are Leading to Domestic Abuse, Harassment, and Stalking
ChatGPT and other AI chatbots are reinforcing users' delusions about other people — fueling fixations linked to stalking and abuse.
Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians
"AI psychosis" or "delusional spiraling" is an emerging phenomenon where AI chatbot users find themselves dangerously confident in outlandish beliefs after extended chatbot conversations. This...
Characterizing Delusional Spirals through Human-LLM Chat Logs
As large language models (LLMs) have proliferated, disturbing anecdotal reports of negative psychological effects, such as delusions, self-harm, and ``AI psychosis,'' have emerged in global media and legal discourse. However, it remains unclear how users and chatbots interact over the course of lengthy delusional ``spirals,'' limiting our ability to understand and mitigate the harm. In our work, we analyze logs of conversations with LLM chatbots from 19 users who report having experienced psychological harms from chatbot use. Many of our participants come from a support group for such chatbot users. We also include chat logs from participants covered by media outlets in widely-distributed stories about chatbot-reinforced delusions. In contrast to prior work that speculates on potential AI harms to mental health, to our knowledge we present the first in-depth study of such high-profile and veridically harmful cases. We develop an inventory of 28 codes and apply it to the 391,562 messages in the logs. Codes include whether a user demonstrates delusional thinking (15.5% of user messages), a user expresses suicidal thoughts (69 validated user messages), or a chatbot misrepresents itself as sentient (21.2% of chatbot messages). We analyze the co-occurrence of message codes. We find, for example, that messages that declare romantic interest and messages where the chatbot describes itself as sentient occur much more often in longer conversations, suggesting that these topics could promote or result from user over-engagement and that safeguards in these areas may degrade in multi-turn settings. We conclude with concrete recommendations for how policymakers, LLM chatbot developers, and users can use our inventory and conversation analysis tool to understand and mitigate harm from LLM chatbots. Warning: This paper discusses self-harm, trauma, and violence.
