Four Functional Quadrants for Trust & Safety Tools: Detection, Investigation, Review & Enforcement (DIRE) <div> <br> </div>

Public discourse and regulatory debates about online trust and safety have long been dominated by a focus on rules, not tools: preoccupied mostly by the questio

MeritRank: Sybil Tolerant Reputation for Merit-based Tokenomics*

Decentralized reputation systems are emerging as promising mechanisms to enhance the effectiveness of token-based economies. Unlike traditional monetary incentives, these systems reward participants based on the actual value of their contributions to the network. However, the advantages and challenges associated with such systems remain largely unexplored. In this work, we investigate the inherent trade-offs in designing a decentralized reputation system that is simultaneously generalizable, trustless, and Sybil-resistant. Specifically, “generalizable” means that the system can assess various types of contributions across different contexts, “trustless” indicates that it functions without the need for a central authority to oversee reputations, and “Sybil-resistant” refers to its ability to withstand manipulations by fake identities, i.e., Sybil attacks.

The Peer Review Crisis Demands Radical Reform: Why Reviewers Should Become Paid Professional Referees

The peer review system, fundamental to scientific quality control, faces a significant crisis. As journal editors, we often need to send up to 35 invitations just to secure two reviewers, confronting daily the collapse of voluntary participation. This reflects a critical imbalance: while publication pressure intensifies, willingness to evaluate diminishes, creating "literature elephantiasis", i.e., an overwhelming proliferation of papers exceeding human processing capacity. Current compensation models, relying on token recognition and database access, fail to incentivize quality engagement and may encourage ethically problematic practices like excessive self-citation. The unchecked infiltration of artificial intelligence into peer review, with minimal enforcement, further undermines system integrity. We propose transforming peer reviewers into professional referees, modeled on sports officiating. This radical solution involves formal training and certification for reviewers, equipping them to assess scientific merit, methodology, and ethics comprehensively. Like sports referees supported by assistants, scientific referees would collaborate with specialists - including statisticians, methodology experts, and reference checkers - ensuring thorough evaluation while distributing workload effectively. Funding would come from publishers or research funders, recognizing peer review as an essential, compensated component of the research lifecycle. Implementation faces challenges including publisher resistance and funding allocation, which we address through phased transition strategies. This professionalization addresses current inequities where conscientious scientists shoulder disproportionate reviewing burdens while others contribute minimally. Professional reviewers would view evaluation as valued career development rather than unwelcome obligation. Critics citing independence concerns overlook the sports analogy: referees maintain impartiality through professional standards despite league compensation. Quality scientific evaluation requires dedicated expertise, adequate training, and fair remuneration. Science deserves better than a system dependent on goodwill and guilt - it needs professional referees now.

mitchellh/vouch

A community trust management system based on explicit vouches to participate.

Trust Infrastructure on ATproto

I agree with Nick that you have noted a big hard problem, with Zooko’s triangle type trade-offs. (For a semi-related example that is “easier” but still crazy difficult, see our “PeerFlow: Secure Load Balancing in Tor” https://doi.org/10.1515/popets-2017-0017 ) As noted or implied already in this thread: in the end, if someone can cheaply spin up indefinite unlinked identities and build good reputations for them, and if trust is based on such reputations, it is hard (impossible?) to avoid them s...

Design principles for long-lived circular organization: An Ostromian perspective

Circular organizational templates such as holacracy and sociocracy have gained popularity. In such organizations, all members have rights to participate in decision-making. Which factors enable such organizations to sustain their circular form? We investigate 18 highly diverse circular organizations to answer this question. Our findings show that Elinor Ostrom’s design principles for avoiding the tragedy of the commons may also explain circular organizations’ longevity. With regard to their relevance, we sort these principles into three groups: indispensable, queasy, and quite unproblematic principles. Our main theoretical contribution is to show that circular organizations can be understood as commons, and thus to make research on the governance of commons fruitful for the design of circular organizations. We show that implementing circular forms has to mind especially the queasy design principles of monitoring, graduated sanctions, conflict resolution mechanisms, and balancing individual contributions and rewards.

Gaming Consensus: Coordinated Manipulation in Crowdsourced Fact-Checking

In this work, we develop the first rigorous analysis of the resistance of the core matrix factorization component of Community Notes to coordinated manipulation. Bridging-based algorithms (Ovadya & Thorburn, 2023; Small et al., 2021) offer some inherent resistance to manipulation: unlike traditional voting systems where adversaries can fabricate volume through fake upvotes, bridging requires diverse agreement across users with opposing viewpoints. Forging bridging consensus is harder as it requires an understanding of users and rating patterns on the platform. However, we demonstrate that it is possible to create synthetic consensus through a carefully designed voting strategy. Our approach does not rely on exploiting any software/security vulnerability or implementation bug, instead leveraging a fundamental property of the system’s core matrix factorization algorithm.

Inverting the physics of trust

Common Fabric is a social computing lab building a medium for software that revolves around people, not apps.

Circles

Circles is a new kind of secure social sharing app

New Paradigms in Trust and Safety: Navigating Defederation on Decentralized Social Media Platforms

Defederation on decentralized social media offers new possibilities for online governance. Experts consider how it can be used responsibly, in a way that balances speech and safety.

Avoid Blunders Designing Reputation Systems

You don’t SPEND reputation because ratings are orthogonal to exchange.

Trust & Safety Library - Trust & Safety Professional Association

Welcome to the Trust & Safety Library! We use this space to collect articles, blog posts, journal articles, lectures, podcasts, and websites that trust and safety professionals may find useful in developing policies, supporting moderators, building systems to detect violations, and generally deepening their practice. We welcome your submissions and feedback. This project was initially

ĐApps: What Web 3.0 Looks Like

As we move into the future, we find increasing need for a zero-trust interaction system. Even pre-Snowden, we had realised that entrusting our information to arbitrary entities on the internet was fraught with danger. However, post-Snowden the argument plainly falls in the hand of those who believe that large organisations and governments routinely attempt to stretch and overstep their authority. Thus we realise that entrusting our information to organisations in general is a fundamentally broken model. The chance of an organisation not meddling with our data is merely the effort required minus their expected gains. Given they tend to have an income model that requires they know as much about people as possible the realist will realise that the potential for convert misuse is difficult to overestimate.

Trust Revolution

Unfiltered conversations with builders, thinkers, and operators in Bitcoin and beyond. Exploring systems we trust, why they work (or don't), and what's next.

Composable Trust, Part 1: Communities Without Credible Exit - Eclectic Corvine Muses

We guarantee that users aren’t subject to platforms. Yet communities are still subject to their stewards. Can we fix this?

Reformation of science publishing: the Stockholm Declaration

Science relies on integrity and trustworthiness. But scientists under career pressure are lured to purchase fake publications from ‘paper mills’ that use AI-generated data, text and image fabrication. The number of low-quality or fraudulent publications is rising to hundreds of thousands per year, which—if unchecked—will damage the scientific and economic progress of our societies. The result is editor and reviewer fatigue, irreproducible experiments, misguided experiments, disinformation and escalating costs that devour funding from taxpayers intended for research. It is high time to reevaluate current publishing models and outline a global plan to stop this unhealthy development. A conference was therefore organized by the Royal Swedish Academy of Sciences to draft an action plan with specific recommendations, as follows. (i) Academia should resume control of publishing using non-profit publishing models (e.g. diamond open-access). (ii) Adjust incentive systems to merit quality, not quantity, in a reputation economy where the gaming of publication numbers and citation metrics distorts the perception of academic excellence. (iii) Implement mechanisms to prevent and detect fake publications and fraud which are independent of publishers. (iv) Draft and implement legislations, regulations and policies to increase publishing quality and integrity. This is a call to action for universities, academies, science organizations and funders to unite and join this effort.