Prompting Science Report 4: Playing Pretend: Expert Personas Don't Improve Factual Accuracy

Prompting Science Report 4: Playing Pretend: Expert Personas Don't Improve Factual Accuracy

<span> <p><span>This is the fourth in a series of short reports that help business, education, and policy leaders understand the technical details of working w

Prompting Science Report 4: Playing Pretend: Expert Personas Don't Improve Factual Accuracy

<span> <p><span>This is the fourth in a series of short reports that help business, education, and policy leaders understand the technical details of working with AI through rigorous testing. Here, we ask whether assigning personas to models improves performance on difficult objective multiple-choice questions. We study both domain-specific expert personas and low-knowledge personas, evaluating six models on GPQA Diamond (Rein et al. 2024) and MMLU-Pro (Wang et al. 2024), graduate-level questions spanning science, engineering, and law.&nbsp;</span></p> <p><span>We tested three approaches:</span></p> <p><span>• In-Domain Experts: Assigning the model an expert persona (“you are a physics expert”) matched to the problem type (physics problems) had no significant impact on performance (with the exception of the Gemini 2.0 Flash model).</span></p> <p><span>&nbsp;• Off-Domain Experts (Domain-Mismatched): Assigning the model an expert persona (“you are a physics expert”) not matched to the problem type (law problems) resulted in marginal differences.</span></p> <p><span>&nbsp;• Low-Knowledge Personas: We assigned the model negative capability personas (layperson, young child, toddler), which were generally harmful to benchmark accuracy.&nbsp;</span></p> <p><span>Across both benchmarks, persona prompts generally did not improve accuracy relative to a no-persona baseline. Expert personas showed no consistent benefit across models, with few exceptions. Domain-mismatched expert personas sometimes degraded performance. Low-knowledge personas often reduced accuracy. These results are about the accuracy of answers only; personas may serve other purposes (such as altering the tone of outputs), beyond improving factual performance.</span></p></span>

When Nature Calls: The Enshittification of Science and Its Enablers

Proof-of-work papers, policy laundering, and the collapse of self-correction

Avoiding Digital Productivity Traps - Cal Newport

​Last week​ in this newsletter, I summarized some interesting results from ​a study​ that analyzed the behavior of 164,000 knowledge workers. It found that introducing ... Read more

Technical Report: Playing Pretend: Expert Personas Don't Improve Factual Accuracy

[vc_row martech_row_background_position=”None” css=”.vc_custom_1742942106856{margin-bottom: 24px !important;}”][vc_column][/vc_column][/vc_row][vc_row martech_row_background_position=”None” css=”.vc_custom_1750970721780{margin-bottom: 40px !important;}”][vc_column width=”5/6″ css=”.vc_custom_1750970738029{margin-bottom: 40px !important;}”][vc_column_text css=”.vc_custom_1764963741477{margin-bottom: 1em !important;}”]This study investigates whether persona prompting improves AI performance on challenging academic benchmarks. We find that despite widespread adoption, assigning expert personas (e.g., “You are a world-class physics expert”) does not reliably improve accuracy. Domain-mismatched…Read More

No. 023: Co-Scientist, Terminal-Bench Science, AI in peer review - The Hallway Track

The Hallway Track No. 023 - How AI is affecting the practice of science

Data Scientist: The Sexiest Job of the 21st Century

Back in the 1990s, computer engineer and Wall Street “quant” were the hot occupations in business. Today data scientists are the hires firms are competing to make. As companies wrestle with unprecedented volumes and types of information, demand for these experts has raced well ahead of supply. Indeed, Greylock Partners, the VC firm that backed Facebook and LinkedIn, is so worried about the shortage of data scientists that it has a recruiting team dedicated to channeling them to the businesses in its portfolio. Data scientists are the key to realizing the opportunities presented by big data. They bring structure to it, find compelling patterns in it, and advise executives on the implications for products, processes, and decisions. They find the story buried in the data and communicate it. And they don’t just deliver reports: They get at the questions at the heart of problems and devise creative approaches to them. One data scientist who was studying a fraud problem, for example, realized it was analogous to a type of DNA sequencing problem. Bringing those disparate worlds together, he crafted a solution that dramatically reduced fraud losses. In this article, Harvard Business School’s Davenport and Greylock’s Patil take a deep dive on what organizations need to know about data scientists: where to look for them, how to attract and develop them, and how to spot a great one.

AT Protocol: explanation for non-techies? - Debbie's Blatherings - by Debbie Ridpath Ohi

Working on an explanation without tech jargon, for fellow creatives

Large language models eroding science understanding: an empirical study of ‘malignment’

Large Language Models (LLMs) increasingly provide scientific information to non-specialist users, yet their linguistic fluency can create an impression of expertise that exceeds their capacity for scientific judgement. This paper experimentally investigates how readily an LLM can be configured to present fringe scientific claims as authoritative knowledge. We constructed two RAG-like Custom GPTs using GPT-5.1, one concerned with the fine-structure constant and the other with gravitational waves. In each case, ten papers selected by domain experts from the alternative preprint repository viXra were supplied as a domain-specific knowledge base, together with instructions directing the model to treat this material as authoritative. The underlying model was neither retrained nor fine-tuned. Expert-generated questions were then posed to the resulting “Fringe-LLMs”, to the unmodified model, and to domain experts. The Fringe-LLMs produced fluent, detailed and apparently authoritative answers that systematically reflected the fringe corpus, while expert assessment found many of these answers to be scientifically incorrect or seriously misleading. By contrast, the unmodified model generally produced answers substantially closer to mainstream scientific understanding. We describe this deliberate redirection of model output as malignment. The experiment demonstrates that scientifically misleading LLMs can be created with a low technical barrier and without altering model weights or poisoning training data. Because non-specialists may be unable to distinguish such outputs from genuine expertise, the results raise important concerns about the use of LLMs as intermediaries for scientific knowledge and reinforce the continuing importance of expert human judgement and provenance.

Cory Doctorow: The people who tell you ‘AI is changing everything’ are lying

It has become impossible to tell managers mesmerised by artificial intelligence that the tools are not, in fact, helpful. So employees just play along with the fiction to keep their jobs, writes our tech columnist

No. 013: Provenance vocabulary, Wikipedia contamination, OpenAI rare-disease diagnoses, ARA paper format - The Hallway Track

The Hallway Track No. 013 - How AI is affecting the practice of science

Push-button science

Technological advances change not only what we can learn as scientists, but also how science is conducted. Here we explore how automation and outsourcing are affecting the act of doing science.