Researchers compare math progress of almost 1,000 high school students
'Nature' Retracts Paper on the Benefits of ChatGPT in Education
“What educators, parents and policy officials really needed was high quality data and evidence to help guide them. What they have had to deal with instead is some substandard research.”
ChatGPT acts as a "cognitive crutch" that weakens memory, new research suggests
A recent experiment provides evidence that relying on artificial intelligence to study tends to reduce how much information students remember weeks later.
Retraction Note: The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: insights from a meta-analysis
Humanities and Social Sciences Communications - Retraction Note: The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: insights from a...
Retraction Note: The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: insights from a meta-analysis
Humanities and Social Sciences Communications - Retraction Note: The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: insights from a...
ChatGPT's Impact On Our Brains According to an MIT Study
The study, from MIT Lab scholars, measured the brain activity of subjects writing SAT essays with and without ChatGPT.
Jay Van Bavel, PhD on Twitter / X
Giving students unfettered access to chatbots can harm educational outcomesHaving AI access while solving problems significantly improves performance (48% improvement in grades for GPT Base and 127% for GPT Tutor).However, when access to AI was taken away, students actually… pic.twitter.com/2g8GJcoRya— Jay Van Bavel, PhD (@jayvanbavel) July 18, 2026
Generative AI without guardrails can harm learning: Evidence from high school mathematics
Generative AI is poised to revolutionize how humans work, and has already demonstrated promise in significantly improving human productivity. A key question is how generative AI affects learning—namely, how humans acquire new skills as they perform tasks. Learning is critical to long-term productivity, especially since generative AI is fallible and users must check its outputs. We study this question via a field experiment where we provide nearly a thousand high school math students with access to generative AI tutors. To understand the differential impact of tool design on learning, we deploy two generative AI tutors: one that mimics a standard ChatGPT interface (“GPT Base”) and one with prompts designed to safeguard learning (“GPT Tutor”). Consistent with prior work, our results show that having GPT-4 access while solving problems significantly improves performance (48% improvement in grades for GPT Base and 127% for GPT Tutor). However, we additionally find that when access is subsequently taken away, students actually perform worse than those who never had access (17% reduction in grades for GPT Base)—i.e., unfettered access to GPT-4 can harm educational outcomes. These negative learning effects are largely mitigated by the safeguards in GPT Tutor. Without guardrails, students attempt to use GPT-4 as a “crutch” during practice problem sessions, and subsequently perform worse on their own. Thus, decision-makers must be cautious about design choices underlying generative AI deployments to preserve skill learning and long-term productivity.
I saw this meta-analysis shared a couple of times recently, so we took a look and re-analyzed the data. | František Bartoš
I saw this meta-analysis shared a couple of times recently, so we took a look and re-analyzed the data. We found that the conclusion is almost entirely driven by publication bias. Both state-of-the-art and standard methods reduce the degree of the effect 2-3 fold. Moreover, the data no longer show statistical evidence for the main conclusions. Importantly, our findings do not imply that there is no positive effect of ChatGPT, or other large language models on learning; in fact, our analysis reveals that there is “absence of evidence” rather than “evidence of absence”. The present literature appears to be contaminated by publication bias; high-quality registered reports are needed to properly evaluate the effect of large language models in educational settings. See the full response just submitted for publication at https://lnkd.in/eGKHHKtg
One of ChatGPT's popular uses just got skewered by Stanford researchers
When the stakes are high, a robot therapist falls way short, researchers found.
Influential study touting ChatGPT in education retracted over red flags
The retracted study on ChatGPT in education was already cited hundreds of times.
ChatGPT as a cognitive crutch: Evidence from a randomized controlled trial on knowledge retention
The rapid integration of generative artificial intelligence into higher education has outpaced empirical understanding of its effects on fundamental learning processes. To address this gap, this randomized controlled trial (n = 120) tested ChatGPT's impact on long-term knowledge retention in undergraduates learning AI. Participants were randomly assigned either to use ChatGPT as a study aid (AI-Assisted Group) or to use only traditional, non-AI study methods (traditional learning group). Knowledge retention was assessed with a surprise test 45 days after learning. Students who used ChatGPT scored significantly lower on the retention test (57.5 % correct) compared to those who studied traditionally (68.5 % correct), t (83) = −3.19, p = .002, Cohen's d = 0.68. This suggests that unrestricted ChatGPT use impaired long-term retention, likely by reducing the cognitive effort that supports durable memory. The findings align with cognitive offloading theory and the ‘desirable difficulties’ principle: while AI assistance may ease initial learning, it appears to undermine the effortful processes needed for robust learning. These results have important implications for how generative AI tools should be integrated into higher education.
Teachers Are Not OK
AI, ChatGPT, and LLMs "have absolutely blown up what I try to accomplish with my teaching."
Papers and peer reviews with evidence of ChatGPT writing
Retraction Watch readers have likely heard about papers showing evidence that they were written by ChatGPT, including one that went viral. We and others have reported on the phenomenon. Here’…
