Labels - AT Protocol

All about moderation labels

Models of collaborative tagging

Collaborative tagging, also known as social tagging or folksonomy, allows users to apply public tags to online items, typically to make those items easier for themselves or others to find later. It has been argued that these tagging systems can provide navigational cues or "way-finders" for other users to explore information.[1][2] The notion is that given that social tags are labels users create to represent topics extracted from online documents, the interpretation of these tags should allow other users to predict the contents of different documents efficiently. Social tags are arguably more important in exploratory search, in which the users may engage in iterative cycles of goal refinement and exploration of new information (as opposed to simple fact-retrievals), and interpretation of information contents by others will provide useful cues for people to discover topics that are relevant.

What Happened to Tagging? - JSTOR Daily

Tags decentralized and democratized the organization of information. What became of "social tagging?"

Beyond moderation: labelers for security/safe browsing, existing services to use · bluesky-social atproto · Discussion #1595

So far, much of the conversation around third party labelers has focused on content moderation. That's an important first use case, but obviously not the only one. We talked a bit about the dow...

Visual Moderation - Sexual Content Classes

Visual Moderation Class Descriptions

moderation - plyr.fm

the most joyous of topics

Ontology is Overrated: Categories, Links, and Tags - Clay Shirky

Based on two talks I gave in the spring of 2005—one entitled "Ontology Is Overrated", and one entitled "Folksonomies & Tags: The rise of user-developed classification."

Quantifying the Persona Effect in LLM Simulations

Large language models (LLMs) have shown remarkable promise in simulating human language and behavior. This study investigates how integrating persona variables—demographic, social, and behavioral factors—impacts LLMs' ability to simulate diverse perspectives. We find that persona variables account for \ensuremath<10% variance in annotations in existing subjective NLP datasets. Nonetheless, incorporating persona variables via prompting in LLMs provides modest but statistically significant improvements. Persona prompting is most effective in samples where many annotators disagree, but their disagreements are relatively minor. Notably, we find a linear relationship in our setting: the stronger the correlation between persona variables and human annotations, the more accurate the LLM predictions are using persona prompting. In a zero-shot setting, a powerful 70b model with persona prompting captures 81% of the annotation variance achievable by linear regression trained on ground truth annotations. However, for most subjective NLP datasets, where persona variables have limited explanatory power, the benefits of persona prompting are limited.

Value misalignment in X’s feed algorithm is a reflection of value tensions in engagement

Social media feed algorithms rank content that is purported to be preferred by users, but the engagement behaviors that drive these algorithms are (at best) indirect proxies for users’ explicitly self-stated values. Are the resulting feeds value aligned, and if not, why? We investigate this question by annotating the basic human values expressed in participants’ X (Twitter) feeds (N = 715 US users), analyzing the relationship between the posts’ value expressions and the posts’ amplification in the ranked “For You” Page feed, and then comparing the amplified values to users’ own values. We observe that the inventory of posts from followed accounts reflects users’ self-stated values—but that there is an overall negative correlation (misalignment) between users’ explicit values and the value expressions the algorithm is more likely to amplify. We turn to engagement behavior to understand this misalignment and observe that users’ engagement behaviors can be misaligned with their stated values—likely causing the algorithm to learn and reflect these misaligned values. We also detect partisan differences consistent with this theory: While the algorithm amplifies values negatively correlated with both Democrats’ and Republicans’ self-stated values, they are more misaligned for Democrats. And in fact replying, a heavily weighted form of engagement, is associated with values that are less aligned for both Democrats’ and Republicans’ self-stated values, and is even more misaligned for Democrats. Taken together, these findings offer a glimpse into the tensions between the values that people hold and those that provoke reactions, and how these value tensions can produce misaligned outcomes.

Value misalignment in X’s feed algorithm is a reflection of value tensions in engagement

Social media feed algorithms rank content that is purported to be preferred by users, but the engagement behaviors that drive these algorithms are (at best) indirect proxies for users’ explicitly self-stated values. Are the resulting feeds value aligned, and if not, why? We investigate this question by annotating the basic human values expressed in participants’ X (Twitter) feeds (N = 715 US users), analyzing the relationship between the posts’ value expressions and the posts’ amplification in the ranked “For You” Page feed, and then comparing the amplified values to users’ own values. We observe that the inventory of posts from followed accounts reflects users’ self-stated values—but that there is an overall negative correlation (misalignment) between users’ explicit values and the value expressions the algorithm is more likely to amplify. We turn to engagement behavior to understand this misalignment and observe that users’ engagement behaviors can be misaligned with their stated values—likely causing the algorithm to learn and reflect these misaligned values. We also detect partisan differences consistent with this theory: While the algorithm amplifies values negatively correlated with both Democrats’ and Republicans’ self-stated values, they are more misaligned for Democrats. And in fact replying, a heavily weighted form of engagement, is associated with values that are less aligned for both Democrats’ and Republicans’ self-stated values, and is even more misaligned for Democrats. Taken together, these findings offer a glimpse into the tensions between the values that people hold and those that provoke reactions, and how these value tensions can produce misaligned outcomes.

Attention Authors: Updated Practice for Review Articles and Position Papers in arXiv CS Category

arXiv’s computer science (CS) category has updated its moderation practice with respect to review (or survey) articles and position papers. Before being…

Giving Labels More Context - at:// pizza thoughts

A small tweak to the labeling system to allow richer context and annotations

Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactions

AI assistants can impart value judgments that shape people's decisions and worldviews, yet little is known empirically about what values these systems rely on in practice. To address this, we develop a bottom-up, privacy-preserving method to extract the values (normative considerations stated or demonstrated in model responses) that Claude 3 and 3.5 models exhibit in hundreds of thousands of real-world interactions. We empirically discover and taxonomize 3,307 AI values and study how they vary by context. We find that Claude expresses many practical and epistemic values, and typically supports prosocial human values while resisting values like "moral nihilism". While some values appear consistently across contexts (e.g. "transparency"), many are more specialized and context-dependent, reflecting the diversity of human interlocutors and their varied contexts. For example, "harm prevention" emerges when Claude resists users, "historical accuracy" when responding to queries about controversial events, "healthy boundaries" when asked for relationship advice, and "human agency" in technology ethics discussions. By providing the first large-scale empirical mapping of AI values in deployment, our work creates a foundation for more grounded evaluation and design of values in AI systems.