Eliciting Advice Instead of Feedback Improves Developmental Input

AI Assistance Reduces Persistence and Hurts Independent Performance

People often optimize for long-term goals in collaboration: A mentor or companion doesn't just answer questions, but also scaffolds learning, tracks progress, and prioritizes the other person's growth over immediate results. In contrast, current AI systems are fundamentally short-sighted collaborators - optimized for providing instant and complete responses, without ever saying no (unless for safety reasons). What are the consequences of this dynamic? Here, through a series of randomized controlled trials on human-AI interactions (N = 1,222), we provide causal evidence for two key consequences of AI assistance: reduced persistence and impairment of unassisted performance. Across a variety of tasks, including mathematical reasoning and reading comprehension, we find that although AI assistance improves performance in the short-term, people perform significantly worse without AI and are more likely to give up. Notably, these effects emerge after only brief interactions with AI (approximately 10 minutes). These findings are particularly concerning because persistence is foundational to skill acquisition and is one of the strongest predictors of long-term learning. We posit that persistence is reduced because AI conditions people to expect immediate answers, thereby denying them the experience of working through challenges on their own. These results suggest the need for AI model development to prioritize scaffolding long-term competence alongside immediate task completion.

AI Assistance Reduces Persistence and Hurts Independent Performance

People often optimize for long-term goals in collaboration: A mentor or companion doesn't just answer questions, but also scaffolds learning, tracks progress, and prioritizes the other person's growth over immediate results. In contrast, current AI systems are fundamentally short-sighted collaborators - optimized for providing instant and complete responses, without ever saying no (unless for safety reasons). What are the consequences of this dynamic? Here, through a series of randomized controlled trials on human-AI interactions (N = 1,222), we provide causal evidence for two key consequences of AI assistance: reduced persistence and impairment of unassisted performance. Across a variety of tasks, including mathematical reasoning and reading comprehension, we find that although AI assistance improves performance in the short-term, people perform significantly worse without AI and are more likely to give up. Notably, these effects emerge after only brief interactions with AI (approximately 10 minutes). These findings are particularly concerning because persistence is foundational to skill acquisition and is one of the strongest predictors of long-term learning. We posit that persistence is reduced because AI conditions people to expect immediate answers, thereby denying them the experience of working through challenges on their own. These results suggest the need for AI model development to prioritize scaffolding long-term competence alongside immediate task completion.

Social-information seeking in development: The child as experimental psychologist

Research has established that children are “naive psychologists”, adept at understanding and navigating the social world from an early age. However, most of this work has focused on how children process information that they acquire incidentally, for example by passively observing others’ actions. Here, we draw on literature framing children as intuitive scientists, who actively seek information and test hypotheses, to propose a view of children as naive experimental psychologists. From this perspective, children play an active role in selecting and pursuing relevant social information (e.g., about agents’ goals, traits, or relationships), whereby their search strategies are influenced both by context and task demands, as well as their prior beliefs, concepts, and domain-specific naive theories. We argue that the particular challenges associated with learning and reasoning about other minds may necessitate that children leverage their active learning competences, and we outline how the social domain uniquely constrains and shapes the learning process. We review existing research on social-information seeking in children and adults, and identify directions for future research, emphasizing that children’s developing social cognition should be understood in terms of the active, exploratory role they take in learning about and participating in the social world.

Reasoning Models Don't Always Say What They Think

Chain-of-thought (CoT) offers a potential boon for AI safety as it allows monitoring a model's CoT to try to understand its intentions and reasoning processes. However, the effectiveness of such monitoring hinges on CoTs faithfully representing models' actual reasoning processes. We evaluate CoT faithfulness of state-of-the-art reasoning models across 6 reasoning hints presented in the prompts and find: (1) for most settings and models tested, CoTs reveal their usage of hints in at least 1% of examples where they use the hint, but the reveal rate is often below 20%, (2) outcome-based reinforcement learning initially improves faithfulness but plateaus without saturating, and (3) when reinforcement learning increases how frequently hints are used (reward hacking), the propensity to verbalize them does not increase, even without training against a CoT monitor. These results suggest that CoT monitoring is a promising way of noticing undesired behaviors during training and evaluations, but that it is not sufficient to rule them out. They also suggest that in settings like ours where CoT reasoning is not necessary, test-time monitoring of CoTs is unlikely to reliably catch rare and catastrophic unexpected behaviors.

Reasoning Models Don't Always Say What They Think

Chain-of-thought (CoT) offers a potential boon for AI safety as it allows monitoring a model's CoT to try to understand its intentions and reasoning processes. However, the effectiveness of such monitoring hinges on CoTs faithfully representing models' actual reasoning processes. We evaluate CoT faithfulness of state-of-the-art reasoning models across 6 reasoning hints presented in the prompts and find: (1) for most settings and models tested, CoTs reveal their usage of hints in at least 1% of examples where they use the hint, but the reveal rate is often below 20%, (2) outcome-based reinforcement learning initially improves faithfulness but plateaus without saturating, and (3) when reinforcement learning increases how frequently hints are used (reward hacking), the propensity to verbalize them does not increase, even without training against a CoT monitor. These results suggest that CoT monitoring is a promising way of noticing undesired behaviors during training and evaluations, but that it is not sufficient to rule them out. They also suggest that in settings like ours where CoT reasoning is not necessary, test-time monitoring of CoTs is unlikely to reliably catch rare and catastrophic unexpected behaviors.

Social-Information Seeking in Development: The Child as Experimental Psychologist

Research has established that children are “naive psychologists,” adept at understanding and navigating the social world from an early age. However, most of this work has focused on how children process information that they acquire incidentally, for example, by passively observing others’ actions. Here, we draw on literature framing children as intuitive scientists who actively seek information and test hypotheses to propose a view of children as naive experimental psychologists. From this perspective, children play an active role in selecting and pursuing relevant social information (e.g., about agents’ goals, traits, or relationships), whereby their search strategies are influenced both by context and task demands, as well as their prior beliefs, concepts, and domain-specific naive theories. We argue that the particular challenges associated with learning and reasoning about other minds may necessitate that children leverage their active learning competences, and we outline how the social domain uniquely constrains and shapes the learning process. We review existing research on social-information seeking in children and adults and identify directions for future research, emphasizing that children’s developing social cognition should be understood in terms of the active, exploratory role they take in learning about and participating in the social world.

How AI Impacts Skill Formation

AI assistance produces significant productivity gains across professional domains, particularly for novice workers. Yet how this assistance affects the development of skills required to effectively supervise AI remains unclear. Novice workers who rely heavily on AI to complete unfamiliar tasks may compromise their own skill acquisition in the process. We conduct randomized experiments to study how developers gained mastery of a new asynchronous programming library with and without the assistance of AI. We find that AI use impairs conceptual understanding, code reading, and debugging abilities, without delivering significant efficiency gains on average. Participants who fully delegated coding tasks showed some productivity improvements, but at the cost of learning the library. We identify six distinct AI interaction patterns, three of which involve cognitive engagement and preserve learning outcomes even when participants receive AI assistance. Our findings suggest that AI-enhanced productivity is not a shortcut to competence and AI assistance should be carefully adopted into workflows to preserve skill formation -- particularly in safety-critical domains.

How AI Impacts Skill Formation

AI assistance produces significant productivity gains across professional domains, particularly for novice workers. Yet how this assistance affects the development of skills required to effectively supervise AI remains unclear. Novice workers who rely heavily on AI to complete unfamiliar tasks may compromise their own skill acquisition in the process. We conduct randomized experiments to study how developers gained mastery of a new asynchronous programming library with and without the assistance of AI. We find that AI use impairs conceptual understanding, code reading, and debugging abilities, without delivering significant efficiency gains on average. Participants who fully delegated coding tasks showed some productivity improvements, but at the cost of learning the library. We identify six distinct AI interaction patterns, three of which involve cognitive engagement and preserve learning outcomes even when participants receive AI assistance. Our findings suggest that AI-enhanced productivity is not a shortcut to competence and AI assistance should be carefully adopted into workflows to preserve skill formation -- particularly in safety-critical domains.

How Our Company Learned to Make Better Predictions About Everything

We all make bets about the future, whether we call them that or not. We select a career based on perceptions about its prospects; we take on projects based on what we think we can accomplish; we hire employees based on how we predict they will perform. We don’t have an option not to make predictions in our work lives, but we do have an option to try and make better ones. At Twitch, employees have worked to create a culture of forecasting based on the research of Philip Tetlock. Although employees were skeptical at first, most have come to embrace the practice of making frequent numerical forecasts.

I work, I think? - Annotated

How AI may quietly dismantle the feedback loop that turns inexperienced people into competent ones, and why my work matters to me.

Growth Wins When Built On A Solid Foundation of Retention & Engagement | Reforge Blog

Reforge drives team performance, with the most actionable learning from vetted operators that your team will actually use and apply.

Algorithm appreciation: People prefer algorithmic to human judgment

Even though computational algorithms often outperform human judgment, received wisdom suggests that people may be skeptical of relying on them (Dawes, 1979). Counter to this notion, results from six experiments show that lay people adhere more to advice when they think it comes from an algorithm than from a person. People showed this effect, what we call algorithm appreciation, when making numeric estimates about a visual stimulus (Experiment 1A) and forecasts about the popularity of songs and romantic attraction (Experiments 1B and 1C). Yet, researchers predicted the opposite result (Experiment 1D). Algorithm appreciation persisted when advice appeared jointly or separately (Experiment 2). However, algorithm appreciation waned when: people chose between an algorithm’s estimate and their own (versus an external advisor’s; Experiment 3) and they had expertise in forecasting (Experiment 4). Paradoxically, experienced professionals, who make forecasts on a regular basis, relied less on algorithmic advice than lay people did, which hurt their accuracy. These results shed light on the important question of when people rely on algorithmic advice over advice from people and have implications for the use of “big data” and algorithmic advice it generates.

Human learning is an understudied but promising lever for boosting human–AI synergy

Humans collaborating with AI hold the promise of achieving superior outcomes compared to either acting alone (i.e., human–AI synergy). However, the conditions that facilitate such synergy when humans are advised by AI are not well understood. A recent meta-analysis showed that, on average, human–AI combinations do not outperform the better individual agent. We argue that this pessimistic conclusion arises from insufficient attention to human learning in experimental designs. To substantiate this claim, we reanalyzed all 74 studies included in the original meta-analysis and found that most previous research overlooked design features that foster human learning (e.g., outcome feedback to participants). Our reanalysis further revealed that studies providing outcome feedback show tentatively higher synergy than those without outcome feedback. Crucially, feedback paired with AI explanations was associated with positive synergy, while explanations without feedback were associated with negative synergy—suggesting that explanations improve synergy mainly when humans can learn to verify the AI’s reliability through feedback. Our reanalysis suggests that the current literature underestimates the potential of human–AI collaboration because it predominantly relies on paradigms that do not facilitate human learning, thus hindering humans from effectively adapting their collaboration strategies. However, experiments directly varying learning opportunities are needed for stronger, causal conclusions. We advocate for a paradigm shift in human–AI interaction research that explicitly addresses human learning and thus enhances our understanding of and support for successful human–AI collaboration.

Mindful Judgment and Decision Making

A full range of psychological processes has been put into play to explain judgment and choice phenomena. Complementing work on attention, information integration, and learning, decision research over the past 10 years has also examined the effects of goals, mental representation, and memory processes. In addition to deliberative processes, automatic processes have gotten closer attention, and the emotions revolution has put affective processes on a footing equal to cognitive ones. Psychological process models provide natural predictions about individual differences and lifespan changes and integrate across judgment and decision making (JDM) phenomena. “Mindful” JDM research leverages our knowledge about psychological processes into causal explanations for important judgment and choice regularities, emphasizing the adaptive use of an abundance of processing alternatives. Such explanations supplement and support existing mathematical descriptions of phenomena such as loss aversion or hyperbolic discounting. Unlike such descriptions, they also provide entry points for interventions designed to help people overcome judgments or choices considered undesirable.

Do LLMs Exhibit Human-like Response Biases? A Case Study in Survey Design

Abstract One widely cited barrier to the adoption of LLMs as proxies for humans in subjective tasks is their sensitivity to prompt wording—but interestingly, humans also display sensitivities to instruction changes in the form of response biases. We investigate the extent to which LLMs reflect human response biases, if at all. We look to survey design, where human response biases caused by changes in the wordings of “prompts” have been extensively explored in social psychology literature. Drawing from these works, we design a dataset and framework to evaluate whether LLMs exhibit human-like response biases in survey questionnaires. Our comprehensive evaluation of nine models shows that popular open and commercial LLMs generally fail to reflect human-like behavior, particularly in models that have undergone RLHF. Furthermore, even if a model shows a significant change in the same direction as humans, we find that they are sensitive to perturbations that do not elicit significant changes in humans. These results highlight the pitfalls of using LLMs as human proxies, and underscore the need for finer-grained characterizations of model behavior.1

Sycophancy Suppression Can Impair Rational Updating: Anti-Sycophancy Should Preserve the Ability to Update

Large language models often exhibit sycophancy, revising their answers to align with users when users push back. Such answer flips, however, can arise from different causes. One possibility is that the model simply aligns with the user's feedback in order to satisfy them. Another is that the feedback genuinely contains useful evidence, prompting the model to update its answer in a rational way. We distinguish them as Unsupported-Yielding and Rational-Updating. Prior work focuses primarily on suppressing Unsupported-Yielding, while overlooking its effect on Rational-Updating. We address this gap with a two-turn evaluation framework that measures the two behaviors separately. Across representative training-time and inference-time interventions, we find that anti-sycophancy methods often encounter a trade-off in which reducing Unsupported-Yielding can sacrifice Rational-Updating, and vice versa, even when the two objectives are optimized jointly. Mechanistic analysis suggests that the two behaviors share an internal substrate: the MLP neurons and attention heads driving them overlap substantially, and their associated steering directions are positively aligned. We further conduct a preliminary orthogonalized steering exploration, which yields modest, backbone-dependent selectivity gains. Overall, our results suggest that anti-sycophancy should be treated not as a simple suppression problem, but as a selectivity problem, where effective interventions should preserve Rational-Updating while reducing Unsupported-Yielding.