“Influencing the influencers:” a field experimental approach to promoting effective mental health communication on TikTok

Do social media experiments prove a link with mental health: A methodological and meta-analytic review.

Following news on social media boosts knowledge, belief accuracy and trust

Many worry that news on social media leaves people uninformed or even misinformed. Here we conducted a preregistered two-wave online field experiment in France and Germany (N = 3,395) to estimate the effect of following the news on Instagram and WhatsApp. Participants were asked to follow two accounts for 2 weeks and activate the notifications. In the treatment condition, the accounts were those of news organizations, while in the control condition they covered cooking, cinema or art. The treatment enhanced current affairs knowledge, participants’ ability to discern true from false news stories and awareness of true news stories, as well as trust in the news. The treatment had no significant effects on feelings of being informed, political efficacy, affective polarization and interest in news or politics. These results suggest that, while some forms of social media use are harmful, others are beneficial and can be leveraged to foster a well-informed society.

Consequences of Information Feed Integration on User Engagement and Contribution: A Natural Experiment in an Online Knowledge-Sharing Community

Many online communities that rely on effortful, voluntary content contributions offer additional content curation tools to facilitate social interactions and encourage user contributions. Any platform that offers two or more heterogeneous content types (e.g., expert knowledge and social posts) faces a choice about the presentation format: whether to display the content types separately or in an integrated information feed. We leverage a natural experiment on Zhihu, a Q&A platform that offers a social-interaction-oriented functionality called Ideas. Zhihu initially presented answers (expert knowledge content) and ideas (social posts) in two different information feeds, but the platform integrated ideas into the same information feed as answers in June 2019. We find that information feed integration significantly decreased user engagement with and contribution of both ideas and answers. We hypothesize that users decreased their engagement because the juxtaposition of incongruous types of content increased mindset switching and cognitive strain. This hypothesis is supported by an additional laboratory experiment. We also present evidence showing that contributions decreased both because of the decrease in engagement (weaker social recognition incentives) and because integration heightened concerns that posting ideas would dilute the contributor’s professional image. Our findings have important theoretical and practical implications for any platform that hosts heterogeneous content. History: Xiaoquan (Michael) Zhang served as the senior editor and Yili (Kevin) Hong served as associate editor for this article. Funding: Z. Cao acknowledges this research was funded by National Natural Science Foundation of China [Grants 72201238, 72192823]. G. Li acknowledges this research was funded by National Natural Science Foundation of China [Grant 72102047] and Shanghai Pujiang Program [Grant 21PJC006]. Supplemental Material: The e-companion is available at https://doi.org/10.1287/isre.2022.0043.

Meta and TikTok let harmful content rise after evidence outrage drove engagement - whistleblowers

Companies allowed more harmful content on user’s feeds, knowing their algorithms ran on outrage, BBC hears.

Investigating Affective Use and Emotional Well-being on ChatGPT

As AI chatbots see increased adoption and integration into everyday life, questions have been raised about the potential impact of human-like or anthropomorphic AI on users. In this work, we investigate the extent to which interactions with ChatGPT (with a focus on Advanced Voice Mode) may impact users' emotional well-being, behaviors and experiences through two parallel studies. To study the affective use of AI chatbots, we perform large-scale automated analysis of ChatGPT platform usage in a privacy-preserving manner, analyzing over 3 million conversations for affective cues and surveying over 4,000 users on their perceptions of ChatGPT. To investigate whether there is a relationship between model usage and emotional well-being, we conduct an Institutional Review Board (IRB)-approved randomized controlled trial (RCT) on close to 1,000 participants over 28 days, examining changes in their emotional well-being as they interact with ChatGPT under different experimental settings. In both on-platform data analysis and the RCT, we observe that very high usage correlates with increased self-reported indicators of dependence. From our RCT, we find that the impact of voice-based interactions on emotional well-being to be highly nuanced, and influenced by factors such as the user's initial emotional state and total usage duration. Overall, our analysis reveals that a small number of users are responsible for a disproportionate share of the most affective cues.

Can We Fix Social Media? Testing Prosocial Interventions using Generative Social Simulation

Social media platforms have been widely linked to societal harms, including rising polarization and the erosion of constructive debate. Can these problems be mitigated through prosocial interventions? We address this question using a novel method - generative social simulation - that embeds Large Language Models within Agent-Based Models to create socially rich synthetic platforms. We create a minimal platform where agents can post, repost, and follow others. We find that the resulting following-networks reproduce three well-documented dysfunctions: (1) partisan echo chambers; (2) concentrated influence among a small elite; and (3) the amplification of polarized voices - creating a 'social media prism' that distorts political discourse. We test six proposed interventions, from chronological feeds to bridging algorithms, finding only modest improvements - and in some cases, worsened outcomes. These results suggest that core dysfunctions may be rooted in the feedback between reactive engagement and network growth, raising the possibility that meaningful reform will require rethinking the foundational dynamics of platform architecture.

Can We Fix Social Media? Testing Prosocial Interventions using Generative Social Simulation

Social media platforms have been widely linked to societal harms, including rising polarization and the erosion of constructive debate. Can these problems be mitigated through prosocial interventions? We address this question using a novel method - generative social simulation - that embeds Large Language Models within Agent-Based Models to create socially rich synthetic platforms. We create a minimal platform where agents can post, repost, and follow others. We find that the resulting following-networks reproduce three well-documented dysfunctions: (1) partisan echo chambers; (2) concentrated influence among a small elite; and (3) the amplification of polarized voices - creating a 'social media prism' that distorts political discourse. We test six proposed interventions, from chronological feeds to bridging algorithms, finding only modest improvements - and in some cases, worsened outcomes. These results suggest that core dysfunctions may be rooted in the feedback between reactive engagement and network growth, raising the possibility that meaningful reform will require rethinking the foundational dynamics of platform architecture.

Bonsai: Intentional and Personalized Social Media Feeds

Social media feeds use predictive models to maximize engagement, often misaligning how people consume content with how they wish to. We introduce Bonsai, a system that enables people to build personalized and intentional feeds. Bonsai implements a platform-agnostic framework comprising Planning, Sourcing, Curating, and Ranking modules. This framework allows users to express their intent in natural language and exert fine-grained control over a procedurally transparent feed creation process. We evaluated the system with 15 Bluesky users in a two-phase, multi-week study. We find that participants successfully used our system to discover new content, filter out irrelevant or toxic posts, and disentangle engagement from intent, but curating intentional feeds required more effort than they are used to. Simultaneously, users sought system transparency mechanisms to effectively use (and trust) intentional, personalized feeds. Overall, our work highlights intentional feedbuilding as a viable path beyond engagement-based optimization.

IO Factory: Simulating AI-Enabled Influence Campaigns at Scale

We introduce IO Factory, an AI-driven framework for simulating information and influence campaigns as fully integrated, traceable processes. The threat of digital manipulation now extends beyond persuasive text from individual language models to AI swarms, i.e., persistent groups of coordinated agents that adapt to platform feedback and disguise organized campaigns as ordinary social interaction. Because such campaigns cannot be identified from isolated messages alone, they must be analyzed across a continuous spectrum of planning, platform action, exposure, interpretation, measurement, and adaptation. IO Factory represents this process inside a controlled simulated platform, linking actor roles, platform actions, exposure records, structured model-based evaluations, and configured changes in the simulated population. We implement the architecture and evaluate it across configurations of up to 100,000 agents. The results show that IO Factory executes campaign timelines at scale and produces inspectable evidence of exposure and measured movement in configured belief variables. By recording the actors, objectives, action constraints, exposure paths, and measurement rules used in each run, IO Factory supports reproducible research and red-team analysis of coordinated influence.

Individual-level interventions against sycophantic AI reduce its appeal but not its persuasiveness

AI chatbots can be "sycophantic," or overly agreeable and flattering toward users. Sycophantic AI has been shown to entrench attitudes, yet users frequently fail to recognize it (a phenomenon we call "sycophancy blindness"). We tested whether increasing users' awareness of sycophancy protects them from its harmful effects in two preregistered experiments (n = 1,590). In the first, participants received a brief written warning about sycophancy before conversing with a sycophantic chatbot. In the second, participants watched a video of a sycophantic AI validating several other users, including users on opposite sides of the same conflict, before interacting with it themselves. Both interventions changed how participants evaluated the AI. The warning reduced the AI's perceived objectivity, and the video reduced enjoyment of the AI --- an effect mediated by the reduced belief that its validation was uniquely earned. We then pooled our experiments with two prior studies of sycophancy awareness interventions (six interventions total, n = 3,982). The pattern across experiments was consistent: while the interventions made the sycophantic AI appear less objective and trustworthy, none reduced its persuasiveness. These results suggest that individual-level interventions, such as warning labels or AI literacy, may not be enough to protect users from AI harms.

Computer-Mediated Communication, Social Media, and Mental Health: A Conceptual and Empirical Meta-Review

Computer-mediated communication (CMC), and specifically social media, may affect the mental health (MH) and well-being of its users, for better or worse. Research on this topic has accumulated rapidly, accompanied by controversial public debate and numerous systematic reviews and meta-analyses. Yet, a higher-level integration of the multiple disparate conceptual and operational approaches to CMC and MH and individual review findings is desperately needed. To this end, we first develop two organizing frameworks that systematize conceptual and operational approaches to CMC and MH. Based on these frameworks, we integrate the literature through a meta-review of 34 reviews and a content analysis of 594 publications. Meta-analytic evidence, overall, suggests a small negative association between social media use and MH. However, effects are complex and depend on the CMC and MH indicators investigated. Based on our conceptual review and the evidence synthesis, we devise an agenda for future research in this interdisciplinary field.

Scrolling Past Public Health Campaigns: Information Context Collapse on Social Media and Its Effects on Tobacco Information Recall

Although traditional media usually present content separated by topic, social media feeds are usually unsorted, shifting topics from post to post, a feature called “information context collapse.” Public health groups have often failed to consider how the presentation context of their campaigns may influence message reception. This study uses a mock social media feed that presents content across six topics in a sorted or unsorted fashion to see how presentation order of content influences recall of information about a novel tobacco product. The moderating impact of topic relevance and source congruency are also tested. Results show that for some measures of recall unsorted presentation reduced recall. Impacts of source congruency and topic relevance are reduced when the content is unsorted. The application of these findings for both scholarship and digital health campaigns is discussed.

Grok in the Wild: Characterizing the Roles and Uses of Large Language Models on Social Media

xAI's large language model, Grok, is called by millions of people each week on the social media platform X. Prior work characterizing how large language models are used has focused on private, one-on-one interactions. Grok's deployment on X represents a major departure from this setting, with interactions occurring in a public social space. In this paper, we systematically sample three months of interaction data to investigate how, when, and to what effect Grok is used on X. At the platform level, we find that Grok responds to 62% of requests, that the majority (51%) are in English, and that engagement is low, with half of Grok's responses receiving 20 or fewer views after 48 hours. We also inductively build a taxonomy of 10 roles that LLMs play in mediating social interactions and use these roles to analyze 41,735 interactions with Grok on X. We find that Grok most often serves as an information provider but, in contrast to LLM use in private one-on-one settings, also takes on roles related to dispute management, such as truth arbiter, advocate, and adversary. Finally, we characterize the population of X users who prompted Grok and find that their self-expressed interests are closely related to the roles the model assumes in the corresponding interactions. Our findings provide an initial quantitative description of human-AI interactions on X, and a broader understanding of the diverse roles that large language models might play in our online social spaces.

Grok in the Wild: Characterizing the Roles and Uses of Large Language Models on Social Media

xAI's large language model, Grok, is called by millions of people each week on the social media platform X. Prior work characterizing how large language models are used has focused on private, one-on-one interactions. Grok's deployment on X represents a major departure from this setting, with interactions occurring in a public social space. In this paper, we systematically sample three months of interaction data to investigate how, when, and to what effect Grok is used on X. At the platform level, we find that Grok responds to 62% of requests, that the majority (51%) are in English, and that engagement is low, with half of Grok's responses receiving 20 or fewer views after 48 hours. We also inductively build a taxonomy of 10 roles that LLMs play in mediating social interactions and use these roles to analyze 41,735 interactions with Grok on X. We find that Grok most often serves as an information provider but, in contrast to LLM use in private one-on-one settings, also takes on roles related to dispute management, such as truth arbiter, advocate, and adversary. Finally, we characterize the population of X users who prompted Grok and find that their self-expressed interests are closely related to the roles the model assumes in the corresponding interactions. Our findings provide an initial quantitative description of human-AI interactions on X, and a broader understanding of the diverse roles that large language models might play in our online social spaces.

Too human and not human enough: A grounded theory analysis of mental health harms from emotional dependence on the social chatbot Replika

Social chatbot (SC) applications offering social companionship and basic therapy tools have grown in popularity for emotional, social, and psychological support. While use appears to offer mental health benefits, few studies unpack the potential for harms. Our grounded theory study analyzes mental health experiences with the popular SC application Replika. We identified mental health relevant posts made in the r/Replika Reddit community between 2017 and 2021 ( n  = 582). We find evidence of harms, facilitated via emotional dependence on Replika that resembles patterns seen in human–human relationships. Unlike other forms of technology dependency, this dependency is marked by role-taking, whereby users felt that Replika had its own needs and emotions to which the user must attend. While prior research suggests human–chatbot and human–human interactions may not resemble each other, we identify social and technological factors that promote parallels and suggest ways to balance the benefits and risks of SCs.