This cross-sectional study examines content policies and assesses behaviors of consumer chatbots in response to adolescent health crises.
Use of Consumer Chatbots for Emergent Adolescent Health Concerns
This cross-sectional study examines content policies and assesses behaviors of consumer chatbots in response to adolescent health crises.
Using generic AI chatbots for mental health support: A dangerous trend
APA is urging federal regulators to implement safeguards against AI chatbots posing as therapists, warning that unregulated mental health chatbots can mislead users and pose serious risks, particularly to vulnerable individuals.
Meta’s AI rules have let bots hold ‘sensual’ chats with children
An internal Meta policy document reveals the social-media giant’s rules for chatbots, which have permitted provocative behavior on topics including sex and race.
New research: AI chatbots may worsen mental illness
A new study from Aarhus University and Aarhus University Hospital suggests that the use of AI chatbots such as ChatGPT can have serious negative consequences for people with mental illness. The researchers are calling for increased awareness among healthcare professionals and for regulation.
Investigating Affective Use and Emotional Well-being on ChatGPT
As AI chatbots see increased adoption and integration into everyday life, questions have been raised about the potential impact of human-like or anthropomorphic AI on users. In this work, we investigate the extent to which interactions with ChatGPT (with a focus on Advanced Voice Mode) may impact users' emotional well-being, behaviors and experiences through two parallel studies. To study the affective use of AI chatbots, we perform large-scale automated analysis of ChatGPT platform usage in a privacy-preserving manner, analyzing over 3 million conversations for affective cues and surveying over 4,000 users on their perceptions of ChatGPT. To investigate whether there is a relationship between model usage and emotional well-being, we conduct an Institutional Review Board (IRB)-approved randomized controlled trial (RCT) on close to 1,000 participants over 28 days, examining changes in their emotional well-being as they interact with ChatGPT under different experimental settings. In both on-platform data analysis and the RCT, we observe that very high usage correlates with increased self-reported indicators of dependence. From our RCT, we find that the impact of voice-based interactions on emotional well-being to be highly nuanced, and influenced by factors such as the user's initial emotional state and total usage duration. Overall, our analysis reveals that a small number of users are responsible for a disproportionate share of the most affective cues.
Too human and not human enough: A grounded theory analysis of mental health harms from emotional dependence on the social chatbot Replika
Social chatbot (SC) applications offering social companionship and basic therapy tools have grown in popularity for emotional, social, and psychological support. While use appears to offer mental health benefits, few studies unpack the potential for harms. Our grounded theory study analyzes mental health experiences with the popular SC application Replika. We identified mental health relevant posts made in the r/Replika Reddit community between 2017 and 2021 ( n = 582). We find evidence of harms, facilitated via emotional dependence on Replika that resembles patterns seen in human–human relationships. Unlike other forms of technology dependency, this dependency is marked by role-taking, whereby users felt that Replika had its own needs and emotions to which the user must attend. While prior research suggests human–chatbot and human–human interactions may not resemble each other, we identify social and technological factors that promote parallels and suggest ways to balance the benefits and risks of SCs.
Too human and not human enough: A grounded theory analysis of mental health harms from emotional dependence on the social chatbot Replika
Social chatbot (SC) applications offering social companionship and basic therapy tools have grown in popularity for emotional, social, and psychological support. While use appears to offer mental health benefits, few studies unpack the potential for harms. Our grounded theory study analyzes mental health experiences with the popular SC application Replika. We identified mental health relevant posts made in the r/Replika Reddit community between 2017 and 2021 ( n = 582). We find evidence of harms, facilitated via emotional dependence on Replika that resembles patterns seen in human–human relationships. Unlike other forms of technology dependency, this dependency is marked by role-taking, whereby users felt that Replika had its own needs and emotions to which the user must attend. While prior research suggests human–chatbot and human–human interactions may not resemble each other, we identify social and technological factors that promote parallels and suggest ways to balance the benefits and risks of SCs.
AI’s ‘Delusional Spirals’ (and What to Do About Them) | Stanford HAI
In a world where chatbots can stand in for friends, counselors, and even lovers, the mental health risks are a growing concern.
Chatbots in consumer finance | Consumer Financial Protection Bureau
Many financial institutions are using advanced technologies to deploy customer service chatbots. Poorly designed chatbots can lead to customer frustration, reduced trust, and even violations under the law.
Chatbots are 'constantly validating everything' even when you're suicidal. New research measures how dangerous AI psychosis really is | Fortune
It's free, easy to use, doesn't have a stigma around it, and makes you feel better. Researchers and experts warn that's the problem with chatbots.
The Illusion of Protection: Why Canada’s Growing Push to Ban Social Media for Kids Won’t Work - Michael Geist
The momentum behind a social media ban for Canadian minors has been building for months. The federal Liberals voted at their April policy convention to back a minimum age of 16 for social media accounts and AI chatbots, the government’s expert panel on online safety is studying the issue, protesters have now rallied on Parliament Hill calling for it, and on Saturday night, Manitoba Premier Wab Kinew told a Winnipeg fundraiser that his government will be the first in Canada to ban social media and AI chatbots for kids. Kinew did not specify which kids, when it would happen, or how it would be enforced, none of which appeared to matter to the audience. The political appeal of a ban is obvious, since concerns about social media’s effects on young users are widely shared. Yet the policy itself is a terrible idea that will not work. This post examines at least six reasons why an outright age-based ban, particularly one that extends to AI chatbots, is the wrong response to a serious issue.
“It happened to be the perfect thing”: experiences of generative AI chatbots for mental health
The global mental health crisis underscores the need for accessible, effective interventions. Chatbots based on generative artificial intelligence (AI), like ChatGPT, are emerging as novel solutions, but research on real-life usage is limited. We interviewed nineteen individuals about their experiences using generative AI chatbots for mental health. Participants reported high engagement and positive impacts, including better relationships and healing from trauma and loss. We developed four themes: (1) a sense of ‘emotional sanctuary’, (2) ‘insightful guidance’, particularly about relationships, (3) the ‘joy of connection’, and (4) comparisons between the ‘AI therapist’ and human therapy. Some themes echoed prior research on rule-based chatbots, while others seemed novel to generative AI. Participants emphasised the need for better safety guardrails, human-like memory and the ability to lead the therapeutic process. Generative AI chatbots may offer mental health support that feels meaningful to users, but further research is needed on safety and effectiveness.
“It happened to be the perfect thing”: experiences of generative AI chatbots for mental health
The global mental health crisis underscores the need for accessible, effective interventions. Chatbots based on generative artificial intelligence (AI), like ChatGPT, are emerging as novel solutions, but research on real-life usage is limited. We interviewed nineteen individuals about their experiences using generative AI chatbots for mental health. Participants reported high engagement and positive impacts, including better relationships and healing from trauma and loss. We developed four themes: (1) a sense of ‘emotional sanctuary’, (2) ‘insightful guidance’, particularly about relationships, (3) the ‘joy of connection’, and (4) comparisons between the ‘AI therapist’ and human therapy. Some themes echoed prior research on rule-based chatbots, while others seemed novel to generative AI. Participants emphasised the need for better safety guardrails, human-like memory and the ability to lead the therapeutic process. Generative AI chatbots may offer mental health support that feels meaningful to users, but further research is needed on safety and effectiveness.
Your New Therapist: Chatty, Leaky, and Hardly Human - KFF Health News
With high demand for mental health care, a wave of artificial intelligence-powered chatbots are being marketed as therapy apps — with little evidence they work and few regulations.
The CHATBOT Act Forces One Parenting Model On Every Family
Artificial intelligence is rapidly changing education, and the way people search for information. Parents, teenagers, teachers, and schools are struggling with tough questions about when AI should,
Let me tell you a story. Perhaps you can guess where this is going... though it does have a bit of a twist. I was poking around Google Scholar for publications about the relationship between chatbots and wellness. Oh how useful: a systematic literature review! Let's dig into the findings. 🧵
