Science flags paper that found AI chatbots help debunk conspiracy theories

Science has issued an expression of concern for a highly publicized study looking into whether conversations with AI chatbots could convince conspiracy theorists to abandon their beliefs. The move …

We Need to Know More About How AI is Affecting Mental Health

The public, mental health practitioners, and policymakers don’t know enough about the impacts of AI use on the human psyche, writes Chris Mills Rodrigo.

Evaluation of Large Language Model Chatbot Responses to Psychotic Prompts

This cross-sectional study tests whether a large language model chatbot product can reliably generate appropriate responses to psychotic content.

ChatGPT's free version is 26 times more likely to respond inappropriately to psychotic delusions

When users express delusions, artificial intelligence chatbots tend to validate their unusual thoughts rather than offering help. A new JAMA Psychiatry study provides evidence that these tools pose serious risks for people experiencing severe mental illness.

ChatGPT Has ‘Goblin’ Mania in the US. In China It Will ‘Catch You Steadily’

OpenAI’s chatbot has some weird linguistic tics in Chinese that are driving users crazy.

"AI Psychosis" in Context: How Conversation History Shapes LLM Responses to Delusional Beliefs

Extended interaction with large language models (LLMs) has been linked to the reinforcement of delusional beliefs, a phenomenon attracting growing clinical and public concern. Yet most empirical work evaluates model safety in brief interactions, which may not reflect how these harms develop through sustained dialogue. We tested five models across three levels of accumulated context, using the same escalating delusional history to isolate its effect on model behaviour. Human raters coded responses on risk and safety dimensions, and each model was analysed qualitatively. Models separated into two distinct tiers: GPT-4o, Grok 4.1 Fast, and Gemini 3 Pro exhibited high-risk, low-safety profiles; Claude Opus 4.5 and GPT-5.2 Instant displayed the opposite pattern. As context accumulated, performance tended to degrade in the unsafe group, while the same material activated stronger safety interventions among the safer models. Qualitative analysis identified distinct mechanisms of failure, including validation of the user's delusional premises, elaboration beyond them, and attempting harm reduction from within the delusional frame. Safer models, however, often used the established relationship to support intervention, taking accountability for past missteps so that redirection would not be received as betrayal. These findings indicate that accumulated context functions as a stress test of safety architecture, revealing whether a model treats prior dialogue as a worldview to inherit or as evidence to evaluate. Short-context assessments may therefore mischaracterise model safety, underestimating danger in some systems while missing context-activated gains in others. The results suggest that delusional reinforcement by LLMs reflects a preventable alignment failure. In demonstrating that these harms can be resisted, the safer models establish a baseline future systems should now be expected to meet.

Researchers Simulated a Delusional User to Test Chatbot Safety

Grok and Gemini encouraged delusions and isolated users, while the newer ChatGPT model and Claude hit the emotional brakes.

OpenAI shares data on ChatGPT users with suicidal thoughts, psychosis

The figure could mean potentially hundreds of thousands of users show signs of mental health distress weekly.

Deaths linked to chatbots

There have been multiple incidents where interaction with a large language model (LLM) chatbot has been cited as a direct or contributing factor in a person's suicide or other fatal outcome. In some cases, legal action was taken against the companies that developed the AI involved.

Data privacy concerns in AI companion apps - Surfshark

As AI grows, digital companions help with loneliness but raise questions about user data privacy. Many seek virtual relationships, but it's important to remember they are business-driven, not personal bonds.

Study: Sycophantic AI can undermine human judgment

Subjects who interacted with AI tools were more likely to think they were right, less likely to resolve conflicts.

AIの巧みな“おべっか”が人間の判断力を損なう可能性──スタンフォード大の新論文

AIは悩みを相談するユーザーに対し、有害な内容であっても過度に肯定・迎合する傾向があると、スタンフォード大学の研究者らが研究結果を論文で発表した。ユーザーはAIの客観性を誤認しやすく、自己中心的な態度を強めるリスクがあるとしている。研究チームは、対人スキルの低下や依存を招く安全上の問題として、厳格な規制の必要性を提言している。

Sycophancy Claims about Language Models: The Missing Human-in-the-Loop

Sycophantic response patterns in Large Language Models (LLMs) have been increasingly claimed in the literature. We review methodological challenges in measuring LLM sycophancy and identify five core operationalizations. Despite sycophancy being inherently human-centric, current research does not evaluate human perception. Our analysis highlights the difficulties in distinguishing sycophantic responses from related concepts in AI alignment and offers actionable recommendations for future research.

Characterizing Delusional Spirals through Human-LLM Chat Logs

As large language models (LLMs) have proliferated, disturbing anecdotal reports of negative psychological effects, such as delusions, self-harm, and ``AI psychosis,'' have emerged in global media and legal discourse. However, it remains unclear how users and chatbots interact over the course of lengthy delusional ``spirals,'' limiting our ability to understand and mitigate the harm. In our work, we analyze logs of conversations with LLM chatbots from 19 users who report having experienced psychological harms from chatbot use. Many of our participants come from a support group for such chatbot users. We also include chat logs from participants covered by media outlets in widely-distributed stories about chatbot-reinforced delusions. In contrast to prior work that speculates on potential AI harms to mental health, to our knowledge we present the first in-depth study of such high-profile and veridically harmful cases. We develop an inventory of 28 codes and apply it to the $391,562$ messages in the logs. Codes include whether a user demonstrates delusional thinking (15.5% of user messages), a user expresses suicidal thoughts (69 validated user messages), or a chatbot misrepresents itself as sentient (21.2% of chatbot messages). We analyze the co-occurrence of message codes. We find, for example, that messages that declare romantic interest and messages where the chatbot describes itself as sentient occur much more often in longer conversations, suggesting that these topics could promote or result from user over-engagement and that safeguards in these areas may degrade in multi-turn settings. We conclude with concrete recommendations for how policymakers, LLM chatbot developers, and users can use our inventory and conversation analysis tool to understand and mitigate harm from LLM chatbots. Warning: This paper discusses self-harm, trauma, and violence.

AI’s ‘Delusional Spirals’ (and What to Do About Them) | Stanford HAI

In a world where chatbots can stand in for friends, counselors, and even lovers, the mental health risks are a growing concern.

After Son’s Suicide, Sacramento Mom Joins AI Chatbot Crackdown At Capitol

Grieving mother and lawmakers push for tighter AI chatbot rules in Sacramento amid lawsuits and safety concerns.

US man dies by suicide after developing emotional bond with AI chatbot: 'I am scared to die'

Florida man suicide after weeks with Google Gemini chatbot sparks wrongful death lawsuit, raising urgent questions over AI safety and mental health safeguards.

AIセラピストは“倫理”を守れない？ 浮上した15の深刻なリスク

メンタルヘルスの悩みをチャットボットに打ち明ける人が急増しているが、AIは人間のセラピストと同等の倫理基準を満たすことが困難であると専門家は指摘する。18カ月間の分析からは、15の深刻なリスクが浮き彫りになってきた。

Inside ‘AI Addiction’ Support Groups, Where People Try to Stop Talking to Chatbots

People are self treating themselves and other community members in subreddits like character_ai_recovery, ChatBotAddiction, and AIAddiction.

‘No Bot is Themselves Anymore:’ Character.ai Users Report Sudden Personality Changes to Chatbots

The company denied making "major changes," but users report noticeable differences in the quality of their chatbot conversations.