A response to: Should universities investigate questionable papers students and faculty wrote elsewhere?

Editor’s note: We recently published a guest post on universities’ responsibility for investigating misconduct allegations related to work by staff and students conducted and/or published while the…

Guest post: Should universities investigate questionable papers students and faculty wrote elsewhere?

Be-Art/iStock I am a research ethicist and often get asked by my university to investigate when potential concerns are raised about our staff or students. One example involved the recent case of th…

I tried to report scientific misconduct. How did it go?

This is the story of how I found what I believe to be scientific misconduct and what happened when I reported it. Science is supposed to b...

I saw this meta-analysis shared a couple of times recently, so we took a look and re-analyzed the data. | František Bartoš

I saw this meta-analysis shared a couple of times recently, so we took a look and re-analyzed the data. We found that the conclusion is almost entirely driven by publication bias. Both state-of-the-art and standard methods reduce the degree of the effect 2-3 fold. Moreover, the data no longer show statistical evidence for the main conclusions. Importantly, our findings do not imply that there is no positive effect of ChatGPT, or other large language models on learning; in fact, our analysis reveals that there is “absence of evidence” rather than “evidence of absence”. The present literature appears to be contaminated by publication bias; high-quality registered reports are needed to properly evaluate the effect of large language models in educational settings. See the full response just submitted for publication at https://lnkd.in/eGKHHKtg

Meta must face youth addiction lawsuit by Massachusetts, court rules

The decision follows a landmark trial in which a ​jury found Meta and Google negligent for designing social media platforms that are harmful to young people.

Research ethics: 3 ways to blow the whistle

Reporting suspicions of scientific fraud is rarely easy, but some paths are more effective than others.

Mapping an Ecology of Integrative Approaches to Addressing the Metacrisis

Dear friends, The Institute of Applied Metatheory (IAM) is pleased to announce the release of a new white paper that charts the landscape of leading-edge responses to the global metacrisis. In “Mapping an Ecology of Integrative Approaches to Addressing the Metacrisis”, IAM scholars Brandon Norgaard, Nicholas Hedlund, PhD, and Claudia Meglin provide a sweeping (albeit

Laura Luebbert, PhD on Twitter / X

MVS has dismissed our findings as "typographical errors and minor oversights," accused Lior and me of being "unprofessional," and called any allegations "totally bizarre." For context, I am showing four examples of duplicated data as described in our arxiv manuscript (Fig 3 & 4). pic.twitter.com/ZuTX6ivBGM— Laura Luebbert, PhD (@NeuroLuebbert) July 11, 2024

Meta allowed minors access to sex-talking chatbots despite staff concerns, lawsuit alleges

Filing by New Mexico’s attorney general includes Meta staff emails objecting to AI companion policy

Does <span style="font-variant:small-caps;">ChatGPT</span> Ignore Article Retractions and Other Reliability Concerns?

ABSTRACT Large language models (LLMs) like ChatGPT seem to be increasingly used for information seeking and analysis, including to support academic literature reviews. To test whether the results might sometimes include retracted research, we identified 217 retracted or otherwise concerning academic studies with high altmetric scores and asked ChatGPT 4o‐mini to evaluate their quality 30 times each. Surprisingly, none of its 6510 reports mentioned that the articles were retracted or had relevant errors, and it gave 190 relatively high scores (world leading, internationally excellent, or close). The 27 articles with the lowest scores were mostly accused of being weak, although the topic (but not the article) was described as controversial in five cases (e.g., about hydroxychloroquine for COVID‐19). In a follow‐up investigation, 61 claims were extracted from retracted articles from the set, and ChatGPT 4o‐mini was asked 10 times whether each was true. It gave a definitive yes or a positive response two‐thirds of the time, including for at least one statement that had been shown to be false over a decade ago. The results therefore emphasise, from an academic knowledge perspective, the importance of verifying information from LLMs when using them for information seeking or analysis.

Does <span style="font-variant:small-caps;">ChatGPT</span> Ignore Article Retractions and Other Reliability Concerns?

ABSTRACT Large language models (LLMs) like ChatGPT seem to be increasingly used for information seeking and analysis, including to support academic literature reviews. To test whether the results might sometimes include retracted research, we identified 217 retracted or otherwise concerning academic studies with high altmetric scores and asked ChatGPT 4o‐mini to evaluate their quality 30 times each. Surprisingly, none of its 6510 reports mentioned that the articles were retracted or had relevant errors, and it gave 190 relatively high scores (world leading, internationally excellent, or close). The 27 articles with the lowest scores were mostly accused of being weak, although the topic (but not the article) was described as controversial in five cases (e.g., about hydroxychloroquine for COVID‐19). In a follow‐up investigation, 61 claims were extracted from retracted articles from the set, and ChatGPT 4o‐mini was asked 10 times whether each was true. It gave a definitive yes or a positive response two‐thirds of the time, including for at least one statement that had been shown to be false over a decade ago. The results therefore emphasise, from an academic knowledge perspective, the importance of verifying information from LLMs when using them for information seeking or analysis.