Value and vulnerability: a framework for understanding the complexity of misinformation use

Building Epistemically Healthier Platforms

When thinking about designing social media platforms, we often focus on factors such as usability, functionality, aesthetics, ethics, and so forth. Epistemic considerations have rarely been given the same level of attention in design discussions. This paper aims to rectify this neglect. We begin by arguing that there are epistemic norms that govern environments, including social media environments. Next, we provide a framework for applying these norms to the question of platform design. We then apply this framework to the real-world case of long-form informational content platforms. We argue that many current long-form informational content platforms are epistemically unhealthy. The good news? We provide concrete advice on how to take steps toward improving their health! Specifically, we argue that they should change how they verify and authenticate content creators and how this information is displayed to content consumers. We conclude by connecting this guidance to broader issues about the epistemic health of platforms.

Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactions

AI assistants can impart value judgments that shape people's decisions and worldviews, yet little is known empirically about what values these systems rely on in practice. To address this, we develop a bottom-up, privacy-preserving method to extract the values (normative considerations stated or demonstrated in model responses) that Claude 3 and 3.5 models exhibit in hundreds of thousands of real-world interactions. We empirically discover and taxonomize 3,307 AI values and study how they vary by context. We find that Claude expresses many practical and epistemic values, and typically supports prosocial human values while resisting values like "moral nihilism". While some values appear consistently across contexts (e.g. "transparency"), many are more specialized and context-dependent, reflecting the diversity of human interlocutors and their varied contexts. For example, "harm prevention" emerges when Claude resists users, "historical accuracy" when responding to queries about controversial events, "healthy boundaries" when asked for relationship advice, and "human agency" in technology ethics discussions. By providing the first large-scale empirical mapping of AI values in deployment, our work creates a foundation for more grounded evaluation and design of values in AI systems.

The politics of ‘platforms’

Online content providers such as YouTube are carefully positioning themselves to users, clients, advertisers and policymakers, making strategic claims for what they do and do not do, and how their place in the information landscape should be understood. One term in particular, ‘platform’, reveals the contours of this discursive work. The term has been deployed in both their populist appeals and their marketing pitches, sometimes as technical ‘platforms’, sometimes as ‘platforms’ from which to speak, sometimes as ‘platforms’ of opportunity. Whatever tensions exist in serving all of these constituencies are carefully elided. The term also fits their efforts to shape information policy, where they seek protection for facilitating user expression, yet also seek limited liability for what those users say. As these providers become the curators of public discourse, we must examine the roles they aim to play, and the terms by which they hope to be judged.

Cleaning Up the Streets: Understanding Motivations, Mental Models, and Concerns of Users Flagging Social Media Content

Social media platforms offer flagging, a technical feature that empowers users to report inappropriate posts or bad actors to reduce online harm. The deceptively simple flagging interfaces on nearly all major social media platforms disguise complex underlying interactions among users, algorithms, and moderators. Through interviewing 25 social media users with prior flagging experience, most of whom belong to marginalized groups, we examine end-users’ understanding of flagging procedures, explore the factors that motivate them to flag, and surface their cognitive and privacy concerns. We found that a lack of procedural transparency in flagging mechanisms creates gaps in users’ mental models, yet they strongly believe that platforms must provide flagging options. Our findings highlight how flags raise critical questions about distributing labor and responsibility between platforms and users for addressing online harm. We recommend innovations in the flagging design space that enhance user comprehension, ensure privacy, and reduce cognitive burdens.

Let's normalize debunking each other

The spread of misinformation has become an epidemic, but information hygiene can help.

Community by Design

Social media empower distributed content creation by algorithmically harnessing "the social fabric" (explicit and implicit signals of association) to serve this content. While this overcomes the bottlenecks and biases of traditional gatekeepers, many believe it has unsustainably eroded the very social fabric it depends on by maximizing engagement for advertising revenue. This paper participates in open and ongoing considerations to translate social and political values and conventions, specifically social cohesion, into platform design. We propose an alternative platform model that includes the social fabric an explicit output as well as input. Citizens are members of communities defined by explicit affiliation or clusters of shared attitudes. Both have internal divisions, as citizens are members of intersecting communities, which are themselves internally diverse. Each is understood to value content that bridge (viz. achieve consensus across) and balance (viz. represent fairly) this internal diversity, consistent with the principles of the Hutchins Commission (1947). Content is labeled with social provenance, indicating for which community or citizen it is bridging or balancing. Subscription payments allow citizens and communities to increase the algorithmic weight on the content they value in the content serving algorithm. Advertisers may, with consent of citizen or community counterparties, target them in exchange for payment or increase in that party's algorithmic weight. Underserved and emerging communities and citizens are optimally subsidized/supported to develop into paying participants. Content creators and communities that curate content are rewarded for their contributions with algorithmic weight and/or revenue. We discuss applications to productivity (e.g. LinkedIn), political (e.g. X), and cultural (e.g. TikTok) platforms.

Beyond Income: Dynamic Consumer Financial Vulnerability

This research challenges the entrenched belief that financial vulnerability affects only low-income consumers. Instead, most consumers across the socioeconomic spectrum experience varying degrees of financial vulnerability at different points during their lives, whether sporadically or chronically; vulnerability is dynamic and heterogeneous. The authors propose a novel, theory-driven definition of consumer financial vulnerability (CFV) as the risk of incurring future harm, given the consumer's current access to various financial resources. A new conceptual framework decouples “vulnerability” from “harm” to distinguish the state of CFV, its determinants (access to various interdependent financial resources), and the constructs it foreshadows (multiple interconnected forms of realized harm). Five research propositions follow: (1) financial resource volatility plays a vital role in CFV, (2) recovering from harm requires more financial resources than preventing harm, (3) a multiperiod lens is needed to assess CFV accurately, (4) greater financial resource access can increase CFV, and (5) generalized financial literacy is not a panacea for mitigating CFV. The propositions and their implications for marketing strategy, public policy, and consumer well-being offer a rich research agenda. The authors propose a measure of CFV—the probability that financial resources are insufficient to meet or exceed a harm threshold—for future empirical investigations.

Deep Storytelling: Collective Sensemaking and Layers of Meaning in U.S. Elections

Misinformation and disinformation about elections remain pressing concerns for researchers, policymakers, and the public. Critics, however, argue that fears surrounding these issues are exaggerated due to a lack of evidence of impact. This debate highlights the challenges inherent in assessing the impacts of misinformation, as the drivers of false and misleading content often exist in the context of a specific claim. To address this issue, we examined false and misleading information surrounding the 2020 and 2022 U.S. national elections, focusing on the contextual features of online conversations that fueled various rumors. We developed two qualitative codebooks, creating the second after realizing that the first, which labeled individual tweets, failed to capture broader rumoring dynamics. By integrating multi-layered qualitative coding with thematic analysis and quantitative visualizations, we show how influencers, political elites, and audiences collaboratively told deep stories from 2020 through 2022. As these stories were told, audiences interpreted events in 2022 through the lens of the 2020 story, guided by influencers' cues, leading to an evolution in storytelling style between the two election cycles. This ongoing performance was tailored to align with the incentive structures, affordances, and attention economy of social media. We combine deep stories with theories of collective sensemaking and rumoring, creating a framework to better assess the contextual features surrounding false and misleading information.

Unveiling the waves of mis- and disinformation from social media

In the digital era, social media platforms have become the focal point for public discourse, with a significant impact on shaping societal narratives. However, they are also rife with mis- and disinformation, which can rapidly disseminate and influence public opinion. This paper investigates the propagation of mis- and disinformation on X, a social media platform formerly known as Twitter. We employ a multidimensional analytical approach, integrating sentiment analysis, wavelet analysis, and network analysis to discern the patterns and intensity of misleading information waves. Sentiment analysis elucidates the emotional tone and subjective context within which information is framed. Wavelet analysis reveals the temporal dynamics and persistence of disinformation trends over time. Network analysis maps the intricate web of information flow, identifying key nodes and vectors of virality. The results offer a granular understanding of how false narratives are constructed and sustained within the digital ecosystem. This study contributes to the broader field of digital media literacy by highlighting the urgent need for robust analytical tools to navigate and neutralize the infodemic in the age of social media.

Practicing Information Sensibility: How Gen Z Engages with Online Information

Assessing the trustworthiness of information online is complicated. Literacy-based paradigms are both widely used to help and widely critiqued. We conducted a study with 35 Gen Zers from across the U.S. to understand how they assess information online. We found that they tended to encounter -- rather than search for -- information, and that those encounters were shaped more by social motivations than by truth-seeking queries. For them, information processing is fundamentally a social practice. Gen Zers interpreted online information together, as aspirational members of social groups. Our participants sought information sensibility: a socially-informed awareness of the value of information encountered online. We outline key challenges they faced and practices they used to make sense of information. Our findings suggest that like their information sensibility practices, solutions and strategies to address misinformation should be embedded in social contexts online.

Capturing our Attention

What are the harms of misinformation? It’s widely believed that it causes false belief, including false beliefs in quite bizarre claims. I argue that this threat is greatly exaggerated: while plaus...

The Role of Media in Political Polarization| Inoculation Can Reduce the Perceived Reliability of Polarizing Social Media Content

Little research is available on psychological interventions that counter susceptibility to polarizing online content. We conducted 3 studies (n1 = 472, n2 = 193, n3 = 772) to evaluate whether psychological resistance against polarizing social media content can be conferred, using the Bad News game, a “technique-based inoculation” intervention that simulates a social media feed. We investigate (1) whether technique-based inoculation can reduce susceptibility to content designed to fuel intergroup polarization; (2) whether technique-based inoculation can offer cross-protection against misinformation techniques that people were not inoculated against; and (3) whether political ideology plays a role in how people engage with anti-misinformation interventions. In Studies 1 and 3 (but not Study 2), we found that technique-based inoculation significantly reduces the perceived reliability of polarizing content and offers partial cross-protection against untreated misinformation techniques. We found no effect for attitudinal certainty and news-sharing intentions. Finally, we report preliminary evidence that people may choose to engage with politically congruent news topics within the intervention.

AMMeBa: A Large-Scale Survey and Dataset of Media-Based Misinformation In-The-Wild

The prevalence and harms of online misinformation is a perennial concern for internet platforms, institutions and society at large. Over time, information shared online has become more media-heavy and misinformation has readily adapted to these new modalities. The rise of generative AI-based tools, which provide widely-accessible methods for synthesizing realistic audio, images, video and human-like text, have amplified these concerns. Despite intense public interest and significant press coverage, quantitative information on the prevalence and modality of media-based misinformation remains scarce. Here, we present the results of a two-year study using human raters to annotate online media-based misinformation, mostly focusing on images, based on claims assessed in a large sample of publicly-accessible fact checks with the ClaimReview markup. We present an image typology, designed to capture aspects of the image and manipulation relevant to the image's role in the misinformation claim. We visualize the distribution of these types over time. We show the rise of generative AI-based content in misinformation claims, and that its commonality is a relatively recent phenomenon, occurring significantly after heavy press coverage. We also show "simple" methods dominated historically, particularly context manipulations, and continued to hold a majority as of the end of data collection in November 2023. The dataset, Annotated Misinformation, Media-Based (AMMeBa), is publicly-available, and we hope that these data will serve as both a means of evaluating mitigation methods in a realistic setting and as a first-of-its-kind census of the types and modalities of online misinformation.

AMMeBa: A Large-Scale Survey and Dataset of Media-Based Misinformation In-The-Wild

The prevalence and harms of online misinformation is a perennial concern for internet platforms, institutions and society at large. Over time, information shared online has become more media-heavy and misinformation has readily adapted to these new modalities. The rise of generative AI-based tools, which provide widely-accessible methods for synthesizing realistic audio, images, video and human-like text, have amplified these concerns. Despite intense public interest and significant press coverage, quantitative information on the prevalence and modality of media-based misinformation remains scarce. Here, we present the results of a two-year study using human raters to annotate online media-based misinformation, mostly focusing on images, based on claims assessed in a large sample of publicly-accessible fact checks with the ClaimReview markup. We present an image typology, designed to capture aspects of the image and manipulation relevant to the image's role in the misinformation claim. We visualize the distribution of these types over time. We show the rise of generative AI-based content in misinformation claims, and that its commonality is a relatively recent phenomenon, occurring significantly after heavy press coverage. We also show "simple" methods dominated historically, particularly context manipulations, and continued to hold a majority as of the end of data collection in November 2023. The dataset, Annotated Misinformation, Media-Based (AMMeBa), is publicly-available, and we hope that these data will serve as both a means of evaluating mitigation methods in a realistic setting and as a first-of-its-kind census of the types and modalities of online misinformation.

Composable Trust, Part 1: Communities Without Credible Exit - Eclectic Corvine Muses

We guarantee that users aren’t subject to platforms. Yet communities are still subject to their stewards. Can we fix this?