Citing Less Critically: LLMs Reshape the Rhetoric and Reach of Scientific Citation

Scientific citations carry rhetorical intent. Scholars may cite prior work positively (supporting), negatively (contrasting), or neutrally (mentioning). As large language models (LLMs) increasingly assist scientific writing, whether they reproduce citations with the same rhetorical intent as humans remains unclear. We introduce a masked-citation task to compare human and LLM-generated citation behavior. For each citation context, an LLM generates a replacement citation sentence, producing a counterfactual corpus directly comparable to human citation. We analyze what, whom, and how models cite, using an LLM-as-a-judge to classify citation intent and a 20-million-edge coauthorship network to measure social distance between cited authors. Across six popular LLMs and 1,746 top NLP conference papers (63k+ contexts, 132k+ citations), three patterns emerge: (1) Compared with human citation, LLMs cite significantly less critically; (2) LLMs over-cite popular and older papers, a tendency amplified for contrasting citations where human writing more often draws on recent, niche work; (3) Whereas humans often cite within their close social network, especially for supporting citations, LLMs tend to draw on more socially distant authors. Together, these differences are double-edged: LLM citation reaches beyond a scholar's close collaborators while being less critical and amplifying visibility bias, reshaping the rhetoric and reach of scientific citation.

There's Something Fundamentally Wrong With LLMs

LLMs aren't trained on the "vast majority of speech," experts warn, a major blind spot that could have sweeping consequences.

The most important thing when working with LLMs

Blog post: The most important thing when working with LLMs by Steve Klabnik

The LLM Critics Are Right. I Use LLMs Anyway.

I almost agree with all of the LLM critics, yet I still use LLMs a lot. I know this sounds like I am delusional, but I don't think I am alone with it.

The LLM Critics Are Right. I Use LLMs Anyway.

I almost agree with all of the LLM critics, yet I still use LLMs a lot. I know this sounds like I am delusional, but I don't think I am alone with it.

LLMs believe false statements even after explicit warnings that they're false

Fine-tuning tests show "bias... toward confidently representing the claims as true."

Dan Shipper 📧 on Twitter / X

this is true and is a big reason why you don’t need to be a highly technical researcher to use LLMs in surprising and novel ways https://t.co/TuxNzXzToU— Dan Shipper 📧 (@danshipper) July 27, 2025

The scientific case for being nice to your chatbot

New research confirms that LLMs often perform better when you encourage them. But why?

Well-intentioned obscenity

An LLM lied about me at the office and I'm cranky about it. That interaction makes me think a lot about how LLMs are becoming normalized, though, and what we're looking at in terms of their role in human-to-human interactions in the future.

Q&A from the slop trenches – GeoSpatial ML

We reviewed 22 ML conference submissions this summer. Fifteen had fabricated citations, hallucinated authors, or clear LLM slop, so we complain about that a bit, and release the paper references audit we now run.

Why do LLMs make stuff up? New research peers under the hood.

Claude's faulty "known entity" neurons sometimes override its "don't answer" circuitry.

I Think I Have LLM Burnout

I didn't expect to get so tired of reading LLM output.

Cognitive exponents and LLM leverage

I know a few people for whom LLMs have been a near-immediate multiplier of attention and effort. I know a lot for whom LLMs clearly make them worse at thinking and doing things. So: why?

LLMs and self-referentiality

I woke up yesterday with the following thoughts, which are probably either obvious or dumb. A central thesis that many readers, including me, took from Douglas Hofstadter’s Gödel Escher Bach when y…

Take caution in using LLMs as human surrogates | PNAS

Recent studies suggest large language models (LLMs) can generate human-like responses, aligning with human behavior in economic experiments, survey...

Brandon Stewart on Twitter / X

1/ New @Nature! We study how powerful institutions shape the information environment for LLMs. Commercial LLM training is opaque, so we trace a path from state-coordinated media -> training data -> model responses. pic.twitter.com/5LdFvzbFaf— Brandon Stewart (@b_m_stewart) May 13, 2026