Why does misinformation influence some and not others? Vulnerability is often mischaracterized as personal weakness or deficiency, or as susceptibility to “infection” or “pollution”—dehumanizing descriptions that highlight supposed shortcomings of media consumers. We propose a broader perspective that focuses on the value of misinformation to persons, groups, and platforms that adopt or share it. The Vulnerability and Value (VV) framework conceptualizes vulnerability as arising from interconnected scales and systems of valuation. Using a complex adaptive systems framing, we describe how misinformation can generate value for individuals, groups, and platforms. When misinformation spreads successfully, it often does so because it serves purposes that extend beyond veracity. Its spread indicates something significant is at stake for those who believe or facilitate it. A key contribution of the VV framework is to formalize the tradeoff individuals face: whether to question and evaluate misinformation or to accept and share it.
Building Epistemically Healthier Platforms
When thinking about designing social media platforms, we often focus on factors such as usability, functionality, aesthetics, ethics, and so forth. Epistemic considerations have rarely been given the same level of attention in design discussions. This paper aims to rectify this neglect. We begin by arguing that there are epistemic norms that govern environments, including social media environments. Next, we provide a framework for applying these norms to the question of platform design. We then apply this framework to the real-world case of long-form informational content platforms. We argue that many current long-form informational content platforms are epistemically unhealthy. The good news? We provide concrete advice on how to take steps toward improving their health! Specifically, we argue that they should change how they verify and authenticate content creators and how this information is displayed to content consumers. We conclude by connecting this guidance to broader issues about the epistemic health of platforms.
Beyond Misinformation: Social Media and the Sculpting of Epistemic Agency
ABSTRACT The dominant account of social media's epistemic harms focuses on the spread of misinformation and the production of false beliefs at scale. This paper argues that the misinformation narrative, whatever its merits, has obscured a deeper and more intimate form of epistemic damage. Dominant social media platforms do not merely supply bad inputs to stable epistemic machinery. They reshape the machinery itself. I argue that social media platforms are powerful engines for sculpting epistemic agency via two mechanisms: normative learning, through which users internalize descriptive and injunctive norms about how to interact with evidence, and reinforcement learning, through which habitual patterns of epistemic response are developed. In these ways, platforms can reshape users' epistemic styles, including their testimonial policies, evidential thresholds for belief revision, and theoretical values governing how they select explanations. This should trouble us, as the constraints driving which styles are produced are systematically orthogonal to truth and ignore information about the environments agents inhabit. This discussion identifies a neglected form of epistemic hostility: not the exploitation of existing cognitive vulnerabilities to implant false beliefs, but the production of new vulnerabilities that can then be exploited. From this, more and better evidence cannot save us.
Facts and Myths about Misperceptions
Misperceptions threaten to warp mass opinion and public policy on controversial issues in politics, science, and health. What explains the prevalence and persistence of these false and unsupported beliefs, which seem to be genuinely held by many people? Though limits on cognitive resources and attention play an important role, many of the most destructive misperceptions arise in domains where individuals have weak incentives to hold accurate beliefs and strong directional motivations to endorse beliefs that are consistent with a group identity such as partisanship. These tendencies are often exploited by elites who frequently create and amplify misperceptions to influence elections and public policy. Though evidence is lacking for claims of a “post-truth” era, changes in the speed with which false information travels and the extent to which it can find receptive audiences require new approaches to counter misinformation. Reducing the propagation and influence of false claims will require further efforts to inoculate people in advance of exposure (for example, media literacy), debunk false claims that are already salient or widespread (for example, fact-checking), reduce the prevalence of low-quality information (for example, changing social media algorithms), and discourage elites from promoting false information (for example, strengthening reputational sanctions).
Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactions
AI assistants can impart value judgments that shape people's decisions and worldviews, yet little is known empirically about what values these systems rely on in practice. To address this, we develop a bottom-up, privacy-preserving method to extract the values (normative considerations stated or demonstrated in model responses) that Claude 3 and 3.5 models exhibit in hundreds of thousands of real-world interactions. We empirically discover and taxonomize 3,307 AI values and study how they vary by context. We find that Claude expresses many practical and epistemic values, and typically supports prosocial human values while resisting values like "moral nihilism". While some values appear consistently across contexts (e.g. "transparency"), many are more specialized and context-dependent, reflecting the diversity of human interlocutors and their varied contexts. For example, "harm prevention" emerges when Claude resists users, "historical accuracy" when responding to queries about controversial events, "healthy boundaries" when asked for relationship advice, and "human agency" in technology ethics discussions. By providing the first large-scale empirical mapping of AI values in deployment, our work creates a foundation for more grounded evaluation and design of values in AI systems.
The politics of ‘platforms’
Online content providers such as YouTube are carefully positioning themselves to users, clients, advertisers and policymakers, making strategic claims for what they do and do not do, and how their place in the information landscape should be understood. One term in particular, ‘platform’, reveals the contours of this discursive work. The term has been deployed in both their populist appeals and their marketing pitches, sometimes as technical ‘platforms’, sometimes as ‘platforms’ from which to speak, sometimes as ‘platforms’ of opportunity. Whatever tensions exist in serving all of these constituencies are carefully elided. The term also fits their efforts to shape information policy, where they seek protection for facilitating user expression, yet also seek limited liability for what those users say. As these providers become the curators of public discourse, we must examine the roles they aim to play, and the terms by which they hope to be judged.
Cleaning Up the Streets: Understanding Motivations, Mental Models, and Concerns of Users Flagging Social Media Content
Social media platforms offer flagging, a technical feature that empowers users to report inappropriate posts or bad actors to reduce online harm. The deceptively simple flagging interfaces on nearly all major social media platforms disguise complex underlying interactions among users, algorithms, and moderators. Through interviewing 25 social media users with prior flagging experience, most of whom belong to marginalized groups, we examine end-users’ understanding of flagging procedures, explore the factors that motivate them to flag, and surface their cognitive and privacy concerns. We found that a lack of procedural transparency in flagging mechanisms creates gaps in users’ mental models, yet they strongly believe that platforms must provide flagging options. Our findings highlight how flags raise critical questions about distributing labor and responsibility between platforms and users for addressing online harm. We recommend innovations in the flagging design space that enhance user comprehension, ensure privacy, and reduce cognitive burdens.
Let's normalize debunking each other
The spread of misinformation has become an epidemic, but information hygiene can help.
Community by Design
Social media empower distributed content creation by algorithmically harnessing "the social fabric" (explicit and implicit signals of association) to serve this content. While this overcomes the bottlenecks and biases of traditional gatekeepers, many believe it has unsustainably eroded the very social fabric it depends on by maximizing engagement for advertising revenue. This paper participates in open and ongoing considerations to translate social and political values and conventions, specifically social cohesion, into platform design. We propose an alternative platform model that includes the social fabric an explicit output as well as input. Citizens are members of communities defined by explicit affiliation or clusters of shared attitudes. Both have internal divisions, as citizens are members of intersecting communities, which are themselves internally diverse. Each is understood to value content that bridge (viz. achieve consensus across) and balance (viz. represent fairly) this internal diversity, consistent with the principles of the Hutchins Commission (1947). Content is labeled with social provenance, indicating for which community or citizen it is bridging or balancing. Subscription payments allow citizens and communities to increase the algorithmic weight on the content they value in the content serving algorithm. Advertisers may, with consent of citizen or community counterparties, target them in exchange for payment or increase in that party's algorithmic weight. Underserved and emerging communities and citizens are optimally subsidized/supported to develop into paying participants. Content creators and communities that curate content are rewarded for their contributions with algorithmic weight and/or revenue. We discuss applications to productivity (e.g. LinkedIn), political (e.g. X), and cultural (e.g. TikTok) platforms.
Beyond Income: Dynamic Consumer Financial Vulnerability
This research challenges the entrenched belief that financial vulnerability affects only low-income consumers. Instead, most consumers across the socioeconomic spectrum experience varying degrees of financial vulnerability at different points during their lives, whether sporadically or chronically; vulnerability is dynamic and heterogeneous. The authors propose a novel, theory-driven definition of consumer financial vulnerability (CFV) as the risk of incurring future harm, given the consumer's current access to various financial resources. A new conceptual framework decouples “vulnerability” from “harm” to distinguish the state of CFV, its determinants (access to various interdependent financial resources), and the constructs it foreshadows (multiple interconnected forms of realized harm). Five research propositions follow: (1) financial resource volatility plays a vital role in CFV, (2) recovering from harm requires more financial resources than preventing harm, (3) a multiperiod lens is needed to assess CFV accurately, (4) greater financial resource access can increase CFV, and (5) generalized financial literacy is not a panacea for mitigating CFV. The propositions and their implications for marketing strategy, public policy, and consumer well-being offer a rich research agenda. The authors propose a measure of CFV—the probability that financial resources are insufficient to meet or exceed a harm threshold—for future empirical investigations.
The governance & behavioral challenges of generative artificial intelligence’s hypercustomization capabilities
Generative artificial intelligence (GenAI) is changing human–machine interactions and the broader information ecosystem. Much as social media algorithms personalize online experiences, GenAI applications can align with user preferences to customize the way individuals interact with information. However, through training, fine-tuning, and prompting, GenAI applications can introduce a new level of customization: hypercustomization. By dynamically tailoring responses to an individual’s explicit and implicit preferences, hypercustomization can reinforce biases, false beliefs, or misconceptions. As a result, it can heighten significant societal challenges, such as the spread of misinformation and political and social polarization. In this article, we explore the risks associated with hypercustomization and the governance and behavioral challenges that might impede effective risk mitigation. These challenges include a lack of transparency in GenAI applications, opacity of the nature of their interactions with users, users’ overreliance on these systems, and the inefficacy of warning messages. We also provide recommendations for overcoming these challenges.
Deep Storytelling: Collective Sensemaking and Layers of Meaning in U.S. Elections
Misinformation and disinformation about elections remain pressing concerns for researchers, policymakers, and the public. Critics, however, argue that fears surrounding these issues are exaggerated due to a lack of evidence of impact. This debate highlights the challenges inherent in assessing the impacts of misinformation, as the drivers of false and misleading content often exist in the context of a specific claim. To address this issue, we examined false and misleading information surrounding the 2020 and 2022 U.S. national elections, focusing on the contextual features of online conversations that fueled various rumors. We developed two qualitative codebooks, creating the second after realizing that the first, which labeled individual tweets, failed to capture broader rumoring dynamics. By integrating multi-layered qualitative coding with thematic analysis and quantitative visualizations, we show how influencers, political elites, and audiences collaboratively told deep stories from 2020 through 2022. As these stories were told, audiences interpreted events in 2022 through the lens of the 2020 story, guided by influencers' cues, leading to an evolution in storytelling style between the two election cycles. This ongoing performance was tailored to align with the incentive structures, affordances, and attention economy of social media. We combine deep stories with theories of collective sensemaking and rumoring, creating a framework to better assess the contextual features surrounding false and misleading information.
Unveiling the waves of mis- and disinformation from social media
In the digital era, social media platforms have become the focal point for public discourse, with a significant impact on shaping societal narratives. However, they are also rife with mis- and disinformation, which can rapidly disseminate and influence public opinion. This paper investigates the propagation of mis- and disinformation on X, a social media platform formerly known as Twitter. We employ a multidimensional analytical approach, integrating sentiment analysis, wavelet analysis, and network analysis to discern the patterns and intensity of misleading information waves. Sentiment analysis elucidates the emotional tone and subjective context within which information is framed. Wavelet analysis reveals the temporal dynamics and persistence of disinformation trends over time. Network analysis maps the intricate web of information flow, identifying key nodes and vectors of virality. The results offer a granular understanding of how false narratives are constructed and sustained within the digital ecosystem. This study contributes to the broader field of digital media literacy by highlighting the urgent need for robust analytical tools to navigate and neutralize the infodemic in the age of social media.
Sharing of misinformation is habitual, not just lazy or biased
Why do people share misinformation on social media? In this research ( N = 2,476), we show that the structure of online sharing built into social platforms is more important than individual deficits in critical reasoning and partisan bias—commonly cited drivers of misinformation. Due to the reward-based learning systems on social media, users form habits of sharing information that attracts others' attention. Once habits form, information sharing is automatically activated by cues on the platform without users considering response outcomes such as spreading misinformation. As a result of user habits, 30 to 40% of the false news shared in our research was due to the 15% most habitual news sharers. Suggesting that sharing of false news is part of a broader response pattern established by social media platforms, habitual users also shared information that challenged their own political beliefs. Finally, we show that sharing of false news is not an inevitable consequence of user habits: Social media sites could be restructured to build habits to share accurate information.
Sharing of misinformation is habitual, not just lazy or biased
Why do people share misinformation on social media? In this research ( N = 2,476), we show that the structure of online sharing built into social platforms is more important than individual deficits in critical reasoning and partisan bias—commonly cited drivers of misinformation. Due to the reward-based learning systems on social media, users form habits of sharing information that attracts others' attention. Once habits form, information sharing is automatically activated by cues on the platform without users considering response outcomes such as spreading misinformation. As a result of user habits, 30 to 40% of the false news shared in our research was due to the 15% most habitual news sharers. Suggesting that sharing of false news is part of a broader response pattern established by social media platforms, habitual users also shared information that challenged their own political beliefs. Finally, we show that sharing of false news is not an inevitable consequence of user habits: Social media sites could be restructured to build habits to share accurate information.
Debunking: A Meta-Analysis of the Psychological Efficacy of Messages Countering Misinformation
This meta-analysis investigated the factors underlying effective messages to counter attitudes and beliefs based on misinformation. Because misinformation can lead to poor decisions about consequential matters and is persistent and difficult to correct, debunking it is an important scientific and public-policy goal. This meta-analysis ( k = 52, N = 6,878) revealed large effects for presenting misinformation ( ds = 2.41–3.08), debunking ( ds = 1.14–1.33), and the persistence of misinformation in the face of debunking ( ds = 0.75–1.06). Persistence was stronger and the debunking effect was weaker when audiences generated reasons in support of the initial misinformation. A detailed debunking message correlated positively with the debunking effect. Surprisingly, however, a detailed debunking message also correlated positively with the misinformation-persistence effect.
Practicing Information Sensibility: How Gen Z Engages with Online Information
Assessing the trustworthiness of information online is complicated. Literacy-based paradigms are both widely used to help and widely critiqued. We conducted a study with 35 Gen Zers from across the U.S. to understand how they assess information online. We found that they tended to encounter -- rather than search for -- information, and that those encounters were shaped more by social motivations than by truth-seeking queries. For them, information processing is fundamentally a social practice. Gen Zers interpreted online information together, as aspirational members of social groups. Our participants sought information sensibility: a socially-informed awareness of the value of information encountered online. We outline key challenges they faced and practices they used to make sense of information. Our findings suggest that like their information sensibility practices, solutions and strategies to address misinformation should be embedded in social contexts online.