Generative AI is poised to revolutionize how humans work, and has already demonstrated promise in significantly improving human productivity. A key question is how generative AI affects learning—namely, how humans acquire new skills as they perform tasks. Learning is critical to long-term productivity, especially since generative AI is fallible and users must check its outputs. We study this question via a field experiment where we provide nearly a thousand high school math students with access to generative AI tutors. To understand the differential impact of tool design on learning, we deploy two generative AI tutors: one that mimics a standard ChatGPT interface (“GPT Base”) and one with prompts designed to safeguard learning (“GPT Tutor”). Consistent with prior work, our results show that having GPT-4 access while solving problems significantly improves performance (48% improvement in grades for GPT Base and 127% for GPT Tutor). However, we additionally find that when access is subsequently taken away, students actually perform worse than those who never had access (17% reduction in grades for GPT Base)—i.e., unfettered access to GPT-4 can harm educational outcomes. These negative learning effects are largely mitigated by the safeguards in GPT Tutor. Without guardrails, students attempt to use GPT-4 as a “crutch” during practice problem sessions, and subsequently perform worse on their own. Thus, decision-makers must be cautious about design choices underlying generative AI deployments to preserve skill learning and long-term productivity.
The 50 Percent Problem
Empirical evidence of the educational harm of generative AI
The 50 Percent Problem
Empirical evidence of the educational harm of generative AI
ChatGPT as a cognitive crutch: Evidence from a randomized controlled trial on knowledge retention
The rapid integration of generative artificial intelligence into higher education has outpaced empirical understanding of its effects on fundamental learning processes. To address this gap, this randomized controlled trial (n = 120) tested ChatGPT's impact on long-term knowledge retention in undergraduates learning AI. Participants were randomly assigned either to use ChatGPT as a study aid (AI-Assisted Group) or to use only traditional, non-AI study methods (traditional learning group). Knowledge retention was assessed with a surprise test 45 days after learning. Students who used ChatGPT scored significantly lower on the retention test (57.5 % correct) compared to those who studied traditionally (68.5 % correct), t (83) = −3.19, p = .002, Cohen's d = 0.68. This suggests that unrestricted ChatGPT use impaired long-term retention, likely by reducing the cognitive effort that supports durable memory. The findings align with cognitive offloading theory and the ‘desirable difficulties’ principle: while AI assistance may ease initial learning, it appears to undermine the effortful processes needed for robust learning. These results have important implications for how generative AI tools should be integrated into higher education.
How AI Impacts Skill Formation
AI assistance produces significant productivity gains across professional domains, particularly for novice workers. Yet how this assistance affects the development of skills required to effectively supervise AI remains unclear. Novice workers who rely heavily on AI to complete unfamiliar tasks may compromise their own skill acquisition in the process. We conduct randomized experiments to study how developers gained mastery of a new asynchronous programming library with and without the assistance of AI. We find that AI use impairs conceptual understanding, code reading, and debugging abilities, without delivering significant efficiency gains on average. Participants who fully delegated coding tasks showed some productivity improvements, but at the cost of learning the library. We identify six distinct AI interaction patterns, three of which involve cognitive engagement and preserve learning outcomes even when participants receive AI assistance. Our findings suggest that AI-enhanced productivity is not a shortcut to competence and AI assistance should be carefully adopted into workflows to preserve skill formation -- particularly in safety-critical domains.
How AI Impacts Skill Formation
AI assistance produces significant productivity gains across professional domains, particularly for novice workers. Yet how this assistance affects the development of skills required to effectively supervise AI remains unclear. Novice workers who rely heavily on AI to complete unfamiliar tasks may compromise their own skill acquisition in the process. We conduct randomized experiments to study how developers gained mastery of a new asynchronous programming library with and without the assistance of AI. We find that AI use impairs conceptual understanding, code reading, and debugging abilities, without delivering significant efficiency gains on average. Participants who fully delegated coding tasks showed some productivity improvements, but at the cost of learning the library. We identify six distinct AI interaction patterns, three of which involve cognitive engagement and preserve learning outcomes even when participants receive AI assistance. Our findings suggest that AI-enhanced productivity is not a shortcut to competence and AI assistance should be carefully adopted into workflows to preserve skill formation -- particularly in safety-critical domains.
From Chalkboards to Chatbots : Evaluating the Impact of Generative AI on Learning Outcomes in Nigeria
This study evaluates the impact of a program leveraging large language models for virtual tutoring in secondary education in Nigeria. Using a randomized controlled trial, .
Gender disparities in the impact of generative artificial intelligence: Evidence from academia
Abstract. The emergence of generative artificial intelligence (AI) tools such as ChatGPT has substantially increased individuals’ productivity. In this stu
Learning by Chatting? Investigating the Impact of Generative AI on Information Seeking and Learning
Generative AI (GenAI) tools offer increasing opportunities for augmenting human cognitive tasks. Among these tasks, information seeking is being rapidly reshaped by GenAI tools, with potentially profound implications for learning and knowledge acquisition. To investigate these implications, we conducted a between-subjects field experiment in which participants pursued informal learning by seeking information through either ChatGPT or Google Search over a span of 8 days. Using a daily diary protocol, we gathered in-situ data on their information-seeking processes. Our findings show that participants in the ChatGPT group experienced diminished agency in their information-seeking processes, as they offloaded much of the information selection to AI, and consequently experienced greater meta-cognitive load arising from this reduced sense of control. We further highlight two sources of distortion in information access when using ChatGPT: biases in ChatGPT outputs, particularly towards providing solution-oriented artifacts over principled knowledge; and systematic shifts in users' information-seeking behaviors, whereby the conversational and socially-oriented interaction paradigm of current GenAI tools may inadvertently reduce exploration of the broader knowledge space. As a result, on average, participants in the ChatGPT group had worse learning outcomes than those using Google, especially for higher-order critical learning. Our work suggests inherent tensions between offloading information seeking to AI and meaningful learning, and provides broader implications for understanding AI's risks to human cognition.
Learning by Chatting? Investigating the Impact of Generative AI on Information Seeking and Learning
Generative AI (GenAI) tools offer increasing opportunities for augmenting human cognitive tasks. Among these tasks, information seeking is being rapidly reshaped by GenAI tools, with potentially profound implications for learning and knowledge acquisition. To investigate these implications, we conducted a between-subjects field experiment in which participants pursued informal learning by seeking information through either ChatGPT or Google Search over a span of 8 days. Using a daily diary protocol, we gathered in-situ data on their information-seeking processes. Our findings show that participants in the ChatGPT group experienced diminished agency in their information-seeking processes, as they offloaded much of the information selection to AI, and consequently experienced greater meta-cognitive load arising from this reduced sense of control. We further highlight two sources of distortion in information access when using ChatGPT: biases in ChatGPT outputs, particularly towards providing solution-oriented artifacts over principled knowledge; and systematic shifts in users' information-seeking behaviors, whereby the conversational and socially-oriented interaction paradigm of current GenAI tools may inadvertently reduce exploration of the broader knowledge space. As a result, on average, participants in the ChatGPT group had worse learning outcomes than those using Google, especially for higher-order critical learning. Our work suggests inherent tensions between offloading information seeking to AI and meaningful learning, and provides broader implications for understanding AI's risks to human cognition.
A new direction for students in an AI world: Prosper, prepare, protect | Brookings
This report explores the potential risks generative AI poses to students and outlines what we can do now to minimize them.
Jay Van Bavel, PhD on Twitter / X
Giving students unfettered access to chatbots can harm educational outcomesHaving AI access while solving problems significantly improves performance (48% improvement in grades for GPT Base and 127% for GPT Tutor).However, when access to AI was taken away, students actually… pic.twitter.com/2g8GJcoRya— Jay Van Bavel, PhD (@jayvanbavel) July 18, 2026
ChatGPT acts as a "cognitive crutch" that weakens memory, new research suggests
A recent experiment provides evidence that relying on artificial intelligence to study tends to reduce how much information students remember weeks later.
Mayor Mamdani and Chancellor Samuels Put Students First with Nation’s Broadest Generative AI Moratorium in Schools
NEW YORK – Mayor Zohran Kwame Mamdani and Schools Chancellor Kamar H. Samuels today announced a moratorium on student-facing generative Artificial Intelligence (AI) use in schools, alongside a new screen time policy, a decision that will impact nearly 600,000 public school students, or two-thirds of the system’s total enrollment. The policy establishes a one-year moratorium, effective in the 2026-2027 school year, on student-facing generative AI for children in 2-K through 8th grade. It also introduces twice-yearly AI critical thinking modules for high schoolers, limited AI pilots for a small number of high school classrooms and age-appropriate screen time restrictions.
The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers
As part of the Digital Library's transition to Open Access, new features for researchers are available in the Premium Edition. Click here to learn more.
The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers
As part of the Digital Library's transition to Open Access, new features for researchers are available in the Premium Edition. Click here to learn more.
What do professional software developers need to know to succeed in an age of Artificial Intelligence?
Generative AI is showing early evidence of productivity gains for software developers, but concerns persist regarding workforce disruption and deskilling. We describe our research with 21 developers at the cutting edge of using AI, summarizing 12 of their work goals we uncovered, together with 75 associated tasks and the skills & knowledge for each, illustrating how developers use AI at work. From all of these, we distilled our findings in the form of 5 insights. We found that the skills & knowledge to be a successful AI-enhanced developer are organized into four domains (using Generative AI effectively, core software engineering, adjacent engineering, and adjacent non-engineering) deployed at critical junctures throughout a 6-step task workflow. In order to "future proof" developers for this age of AI, on-the-job learning initiatives and computer science degree programs will need to target both "soft" skills and the technical skills & knowledge in all four domains to reskill, upskill and safeguard against deskilling.