Why ideological preferences and epistemic failure in LLMs are not the same thing — and why the difference matters
David Rozado on Twitter / X
1. Have we been measuring AI political bias wrong? In a new paper @PTetlock and I argue that we might have. Studies have found that AIs tend to produce left-of-center responses to politically loaded questions. But ideological preferences are not the same as epistemic failure. pic.twitter.com/JuGrboRVb0— David Rozado (@DavidRozado) June 22, 2026
Thoughts on AI in academia
PhD-level thinking, LLM bias, alignment, AGI, data centers, and AI politics
KillBench: Discovering Hidden Biases of LLMs
1M+ experiments exposing bias in critical AI decision-making
Import AI 446: Nuclear LLMs; China's big AI benchmark; measurement and AI policy
Will AIs be jealous of one another?
AI FOR EPISTEMICS & COORDINATION
Civilization and technology have radically improved the human condition. Nonetheless, the world sometimes goes in directions which essentially nobody would prefer — e.g., nuclear arms races, unexpected financial crashes, predatory marketing, or ubiquitous political misinformation.
Democracy Needs AI That Listens
From Taiwan's deliberative experiments to our own AI policy - featuring jdd-kami, the Civic AI that fits in a carry-on
AI Large Language Model Training: The Potential Risks of Ideological Skewing — PSG Consulting
LLMs (AI Large Language Models) have become part of everyday life. Systems such as ChatGPT, Claude, Gemini, Meta AI (Llama) and X.ai's Grok handle billions of interactions daily. They increasingly shape what information people encounter and in what order, subtly deciding what's important and even what is true, sometimes without users realizing it. Because LLMs wield growing power over information exposure, it is vital to recognize the political and ideological structures at multiple stages of their design, and to identify manipulation risks.
ImportAI 449: LLMs training other LLMs; 72B distributed training run; computer vision is harder than generative text
Will AI cause a political interregnum
From the Platform Society to the AI Society: Towards Critical Studies of Generative AI
The era of AI has begun. Generative AI is rapidly reshaping knowledge production, culture, and political authority, giving rise to an emerging AI society. Yet this transformation did not emerge ex nihilo. This paper argues that the AI society can only be understood in relation to the platform society from which it arises. Tracing the transition from platforms to AI, we identify interlinked economic, epistemic, and political shifts. Economically, AI emerges within platform-based rentier capitalism but reconfigures the monopoly mechanisms on which its accumulation depends. Epistemically, LLMs mark a shift from predictive to generative epistemics, entangling theory formation and knowledge production with private research-as-a-service infrastructures. Politically, governance shifts from data politics to alignment politics: from shaping visibility to shaping what can be said, thought, and imagined. Together, these transformations signal a qualitative shift in mediation—from governing interaction to governing cognition itself—and call for a Critical AI Studies.
Your Online Public Engagement is Under Attack from AI
In Los Angeles and elsewhere, AI agents are diluting communities' voices and influencing how decision-makers vote.
GermanPartiesQA: Benchmarking Commercial Large Language Models and AI Companions for Political Alignment and Sycophancy
Large language models (LLMs) are increasingly shaping citizens’ information ecosystems. Products incorporating LLMs, such as chatbots and AI Companions, are now widely used for decision support and information retrieval, including in sensitive domains, raising concerns about hidden biases and growing potential to shape individual decisions and public opinion. This paper introduces GermanPartiesQA, a benchmark of 418 political statements from German Voting Advice Applications across 11 elections to evaluate six commercial LLMs. We evaluate their political alignment based on role-playing experiments with political personas. Our evaluation reveals three specific findings: (1) Factual limitations: LLMs show limited ability to accurately generate factual party positions, particularly for centrist parties. (2) Model-specific ideological alignment: We identify consistent alignment patterns and degree of political steerability for each model across temperature settings and experiments. (3) Claim of sycophancy: While models adjust to political personas during role-play, we find this reflects persona-based steerability rather than the increasingly popular, yet contested concept of sycophancy. Our study contributes to evaluating the political alignment of closed-source LLMs that are increasingly embedded in electoral decision support tools and AI Companion chatbots.
GermanPartiesQA: Benchmarking Commercial Large Language Models and AI Companions for Political Alignment and Sycophancy
Large language models (LLMs) are increasingly shaping citizens’ information ecosystems. Products incorporating LLMs, such as chatbots and AI Companions, are now widely used for decision support and information retrieval, including in sensitive domains, raising concerns about hidden biases and growing potential to shape individual decisions and public opinion. This paper introduces GermanPartiesQA, a benchmark of 418 political statements from German Voting Advice Applications across 11 elections to evaluate six commercial LLMs. We evaluate their political alignment based on role-playing experiments with political personas. Our evaluation reveals three specific findings: (1) Factual limitations: LLMs show limited ability to accurately generate factual party positions, particularly for centrist parties. (2) Model-specific ideological alignment: We identify consistent alignment patterns and degree of political steerability for each model across temperature settings and experiments. (3) Claim of sycophancy: While models adjust to political personas during role-play, we find this reflects persona-based steerability rather than the increasingly popular, yet contested concept of sycophancy. Our study contributes to evaluating the political alignment of closed-source LLMs that are increasingly embedded in electoral decision support tools and AI Companion chatbots.
Do AI and bogus respondents threaten polling’s future?
Courtney Kennedy, vice president of methods and innovation, answers some common questions about the current polling landscape in the U.S.
Political Neutrality in AI Is Impossible- But Here Is How to Approximate It
AI systems often exhibit political bias, influencing users' opinions and decisions. While political neutrality-defined as the absence of bias-is often seen as an ideal solution for fairness and safety, this position paper argues that true political neutrality is neither feasible nor universally desirable due to its subjective nature and the biases inherent in AI training data, algorithms, and user interactions. However, inspired by Joseph Raz's philosophical insight that "neutrality [...] can be a matter of degree" (Raz, 1986), we argue that striving for some neutrality remains essential for promoting balanced AI interactions and mitigating user manipulation. Therefore, we use the term "approximation" of political neutrality to shift the focus from unattainable absolutes to achievable, practical proxies. We propose eight techniques for approximating neutrality across three levels of conceptualizing AI, examining their trade-offs and implementation strategies. In addition, we explore two concrete applications of these approximations to illustrate their practicality. Finally, we assess our framework on current large language models (LLMs) at the output level, providing a demonstration of how it can be evaluated. This work seeks to advance nuanced discussions of political neutrality in AI and promote the development of responsible, aligned language models.
Political Neutrality in AI Is Impossible- But Here Is How to Approximate It
AI systems often exhibit political bias, influencing users' opinions and decisions. While political neutrality-defined as the absence of bias-is often seen as an ideal solution for fairness and safety, this position paper argues that true political neutrality is neither feasible nor universally desirable due to its subjective nature and the biases inherent in AI training data, algorithms, and user interactions. However, inspired by Joseph Raz's philosophical insight that "neutrality [...] can be a matter of degree" (Raz, 1986), we argue that striving for some neutrality remains essential for promoting balanced AI interactions and mitigating user manipulation. Therefore, we use the term "approximation" of political neutrality to shift the focus from unattainable absolutes to achievable, practical proxies. We propose eight techniques for approximating neutrality across three levels of conceptualizing AI, examining their trade-offs and implementation strategies. In addition, we explore two concrete applications of these approximations to illustrate their practicality. Finally, we assess our framework on current large language models (LLMs) at the output level, providing a demonstration of how it can be evaluated. This work seeks to advance nuanced discussions of political neutrality in AI and promote the development of responsible, aligned language models.