Collaborative Fact Checking
From Social Network to Sense Making
Cofacts - Message reporting chatbot and crowd-sourced fact-checking community
Cofacts is a collaborative system connecting instant messages and fact-check reports or different opinions together. It's a grass-root effort fighting mis/disinformation in Taiwan.
Gaming Consensus: Coordinated Manipulation in Crowdsourced Fact-Checking
In this work, we develop the first rigorous analysis of the resistance of the core matrix factorization component of Community Notes to coordinated manipulation. Bridging-based algorithms (Ovadya & Thorburn, 2023; Small et al., 2021) offer some inherent resistance to manipulation: unlike traditional voting systems where adversaries can fabricate volume through fake upvotes, bridging requires diverse agreement across users with opposing viewpoints. Forging bridging consensus is harder as it requires an understanding of users and rating patterns on the platform. However, we demonstrate that it is possible to create synthetic consensus through a carefully designed voting strategy. Our approach does not rely on exploiting any software/security vulnerability or implementation bug, instead leveraging a fundamental property of the system’s core matrix factorization algorithm.
Seeing the Politics of Decentralized Social Media Protocols
Calls to decentralize feed-based social media have been driven by concerns about the concentrated power of centralized platforms and their societal impact. In response, numerous decentralized social media protocols have emerged, each interpreting “decentralization” in a different way. We analyze four such protocols (ActivityPub, AT Protocol, Nostr, and Farcaster) drawing on protocol documentation, media coverage, direct usage, and semi-structured interviews with protocol developers and experts to situate each protocol within its socio-technical goals. To support this analysis, we develop a grounded lexicon that facilitates the description and comparison of analogous components across each protocol. With it, we show how each protocol distributes control over key components differently, shaping who holds power over critical aspects such as user identity, curation, and infrastructural ownership. By examining protocols as socio-technical artifacts, we show how they prioritize different values and make distinct trade-offs, providing CSCW researchers with a lens to more effectively evaluate and design future decentralized social media systems.
Four Ps for Building Massive Collective Knowledge Systems
Design principles for collective knowledge systems—permanence, provenance, permission, and placement—that enable robust networks for evidence-based decision making.
Four Ps for Building Massive Collective Knowledge Systems
Design principles for collective knowledge systems—permanence, provenance, permission, and placement—that enable robust networks for evidence-based decision making.
AI Fact-Checking in the Wild: A Field Evaluation of LLM-Written Community Notes on X
Large language models (LLMs) show promising capabilities for fact-checking, yet prior work evaluates them only in controlled offline settings using benchmarks or crowdworker judgments. Success in real-world fact-checking depends also on how content is judged within a live platform environment. We present the first field evaluation of LLM fact-checking deployed on a live social media platform, testing performance directly through X Community Notes’ “AI writer” feature over a three-month period. Our LLM writer, a multi-step pipeline that handles multimodal content, conducts web and platform-native search, and writes contextual notes, was deployed to write 1,614 notes on 1,597 tweets and compared against 1,332 human-written notes on the same tweets using 108,169 ratings from 42,521 raters. Direct comparison of note-level platform outcomes is complicated by differences in submission timing and exposure between LLM and human notes; we therefore pursue two analysis strategies: a rating-level analysis modeling individual rater evaluations, and a note-level analysis that relies on common raters who rated all notes on the same post. Rating-level analysis shows that LLM notes receive more positive ratings than human notes across raters with different political viewpoints, and note-level analysis shows LLM notes achieve significantly higher helpfulness scores among common raters. Rater-provided tags suggest that people consider LLM notes to use more neutral language and cite better sources. These findings provide field evidence that LLMs can contribute broadly helpful fact-checking notes at scale, while showing that their evaluation is shaped by platform dynamics absent from controlled offline settings.
Paper Skygest: Personalized Academic Recommendations on Bluesky
We build, deploy, and evaluate Paper Skygest, a custom personalized social feed for scientific content posted by a user's network on Bluesky and the AT Protocol. We leverage a new capability on emerging decentralized social media platforms: the ability for anyone to build and deploy feeds for other users, to use just as they would a native platform-built feed. To our knowledge, Paper Skygest is the first and largest such continuously deployed personalized social media feed by academics, with over 50,000 weekly uses by over 1,000 daily active users, all organically acquired. First, we quantitatively and qualitatively evaluate Paper Skygest usage, showing that it has sustained usage and satisfies users; we further show adoption of Paper Skygest increases a user's interactions with posts about research, and how interaction rates change as a function of post order. Second, we share our full code and describe our system architecture, to support other academics in building and deploying such feeds sustainably. Third, we overview the potential of custom feeds such as Paper Skygest for studying algorithm designs, building for user agency, and running recommender system experiments with organic users without partnering with a centralized platform.
Many people are talking about decentralized social media protocols like #ActivityPub and #ATProto and all the amazing things one can do with them. But do you actually know how they work? We thought we'd do something about this. Come to #FediForum next week if you are curious and like to find […]
Original post on mastodon.social
mastodon.social
