Retraction Watch readers have likely heard about papers showing evidence that they were written by ChatGPT, including one that went viral. We and others have reported on the phenomenon. Here’…
Does <span style="font-variant:small-caps;">ChatGPT</span> Ignore Article Retractions and Other Reliability Concerns?
ABSTRACT Large language models (LLMs) like ChatGPT seem to be increasingly used for information seeking and analysis, including to support academic literature reviews. To test whether the results might sometimes include retracted research, we identified 217 retracted or otherwise concerning academic studies with high altmetric scores and asked ChatGPT 4o‐mini to evaluate their quality 30 times each. Surprisingly, none of its 6510 reports mentioned that the articles were retracted or had relevant errors, and it gave 190 relatively high scores (world leading, internationally excellent, or close). The 27 articles with the lowest scores were mostly accused of being weak, although the topic (but not the article) was described as controversial in five cases (e.g., about hydroxychloroquine for COVID‐19). In a follow‐up investigation, 61 claims were extracted from retracted articles from the set, and ChatGPT 4o‐mini was asked 10 times whether each was true. It gave a definitive yes or a positive response two‐thirds of the time, including for at least one statement that had been shown to be false over a decade ago. The results therefore emphasise, from an academic knowledge perspective, the importance of verifying information from LLMs when using them for information seeking or analysis.
Does <span style="font-variant:small-caps;">ChatGPT</span> Ignore Article Retractions and Other Reliability Concerns?
ABSTRACT Large language models (LLMs) like ChatGPT seem to be increasingly used for information seeking and analysis, including to support academic literature reviews. To test whether the results might sometimes include retracted research, we identified 217 retracted or otherwise concerning academic studies with high altmetric scores and asked ChatGPT 4o‐mini to evaluate their quality 30 times each. Surprisingly, none of its 6510 reports mentioned that the articles were retracted or had relevant errors, and it gave 190 relatively high scores (world leading, internationally excellent, or close). The 27 articles with the lowest scores were mostly accused of being weak, although the topic (but not the article) was described as controversial in five cases (e.g., about hydroxychloroquine for COVID‐19). In a follow‐up investigation, 61 claims were extracted from retracted articles from the set, and ChatGPT 4o‐mini was asked 10 times whether each was true. It gave a definitive yes or a positive response two‐thirds of the time, including for at least one statement that had been shown to be false over a decade ago. The results therefore emphasise, from an academic knowledge perspective, the importance of verifying information from LLMs when using them for information seeking or analysis.
Influential study touting ChatGPT in education retracted over red flags
The retracted study on ChatGPT in education was already cited hundreds of times.
'Nature' Retracts Paper on the Benefits of ChatGPT in Education
“What educators, parents and policy officials really needed was high quality data and evidence to help guide them. What they have had to deal with instead is some substandard research.”
A.I. Is Homogenizing Our Thoughts
Recent studies suggest that tools such as ChatGPT make our brains less active and our writing less original.
I’m a Professional Writer Who Uses a Very Controversial Tool. It’s Not As Scary As I Thought.
I was skeptical about ChatGPT and Claude at first. Then I started to come around—and I’m glad I did.
How Ten Publishers Retract Research
Retractions are the primary mechanism for correcting the scholarly record, yet publishers differ markedly in how they use them. We present a bibliometric analysis of 46,087 retractions across 10 major publishers using data from the Retraction Watch database (1997-2026), examining retraction rates, reasons, temporal trends, and geographic distributions, among other dimensions. Normalized retraction rates vary by two orders of magnitude, from Elsevier's 3.97 per 10,000 publications to Hindawi's 320.02. China-affiliated authors account for the largest share of retractions at every publisher. Retraction lags and reason profiles also vary widely across publishers. Among the ten publishers, ACM is an outlier in its retraction profile. ACM's normalized rate is mid-range (5.65), yet 98.3% of its 354 retractions are related to one incident. Seven of the ten most common global retraction reasons (including misconduct, plagiarism, and data concerns) are entirely absent from ACM's record. ACM's first retraction dates to 2020, despite a catalog dating to 1997. ACM self-describes its retraction threshold as "extremely high." We discuss this threshold in relation to the COPE retraction guidelines and the implications of ACM's non-public dark archive of removed works.
ChatGPT's Impact On Our Brains According to an MIT Study
The study, from MIT Lab scholars, measured the brain activity of subjects writing SAT essays with and without ChatGPT.
Hey ChatGPT, write me a fictional paper: these LLMs are willing to commit academic fraud
Mainstream chatbots presented varying levels of resistance to deliberate requests for fabrication, study finds
Retraction Note: The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: insights from a meta-analysis
Humanities and Social Sciences Communications - Retraction Note: The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: insights from a...
Retraction Note: The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: insights from a meta-analysis
Humanities and Social Sciences Communications - Retraction Note: The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: insights from a...
Hack “Writers” Fuming After ChatGPT Starts Refusing Prompts to Copy a Specific Author’s Style
AI "writers" are seething after ChatGPT began refusing requests to mimic a specific author's style, living or dead.
ChatGPT, Claude, Gemini, and Grok are all bad at crediting news outlets, but ChatGPT is the worst (at least in this study)
"ChatGPT, one of the most widely used models, covered distinctive content in 54% of responses but almost never credited the originating newsroom."
Hey ChatGPT, write me a fictional paper: these LLMs are willing to commit academic fraud
Mainstream chatbots presented varying levels of resistance to deliberate requests for fabrication, study finds.
One of ChatGPT's popular uses just got skewered by Stanford researchers
When the stakes are high, a robot therapist falls way short, researchers found.
The associations of social media attention, visibility, disinformation and retraction initiators with time to retraction: a Cox regression analysis
Purpose This study examines how retraction reasons, retraction initiators, journal visibility, access models and Twitter activity associate with the speed of retracting flawed scientific publications. Design/methodology/approach Using a Cox proportional hazards model, we analyzed 1,179 articles retracted in 2019–2021, including a subset of 98 papers tweeted before retraction. Findings The results reveal that higher journal impact factor and open-access status were associated with faster retractions. However, a significant negative interaction indicated that the effect of high-impact journals diminished for open-access publications. Journal-initiated retractions were slower overall, except in cases of misconduct such as co-authorship deception and plagiarism, where journals acted more quickly. Among retraction reasons, only deception in co-authoring was associated with significantly slower retractions, but this trend reversed when journals led the process. The association of social media attention with retraction speed was statistically robust, albeit modest in magnitude: each additional pre-retraction tweet was associated with a slight reduction in time to retraction. Bootstrap validation confirmed the stability of this finding. Originality/value Public scrutiny, institutional responsibility and publication visibility jointly shape the time to retraction. This study advances altmetrics discourse by positioning social media as a conditional, yet meaningful, participant in the retraction lifecycle. Beyond altmetrics, our findings highlight retractions as part of a broader network of relationships between public accountability, digital ethics and science communication, positioning them as moments of accountability shaped jointly by journals, ethical responsibilities and digital publics.
