Reviewing post-publication peer review

Peer Review at the Crossroads

Peer review has long been regarded as a cornerstone of scholarly communication, ensuring high quality and credibility of published research. Although academic journals trace their origins back three centuries, the procedures for evaluating submissions, particularly peer review, have undergone continuous evolvement. Peer review’s formal institutionalization in the mid-20th century represents a significant, yet natural, phase in this ongoing transformation of scholarly communication. By the early 21st century, there emerged an opinion that the conventional model of peer review faces systematic challenges, including inefficiency, bias, and institutional inertia. The study aims to synthesize the evolution, practices, and outcomes of both conventional and innovative peer review models in scholarly publishing. Through a mixed-methods approach combining interpretative literature review and process modeling (Business Process Model and Notation –BPMN), it identifies four frameworks: pre-publication peer review, registered reports, modular publishing, and the Publish-Review-Curate (PRC) model. While the PRC model, which integrates preprints with post-publication review, demonstrates advantages in transparency and accessibility, no single approach emerges as universally ideal. The choice of model depends on disciplinary context, resource availability, and institutional priorities. The analysis underscores the need for adaptable platforms that enable hybrid workflows, balancing rigor with inclusivity. Future research must address empirical gaps in evaluating these innovations, particularly their long-term impact on equity and epistemic norms.

Guest Post - Do we need (r)evolution in peer review? - The Scholarly Kitchen

Whether peer review needs revolution or evolution may be the wrong question. The more pressing one is whether the academic community is prepared to let scientific ideas be published, contested, and refined in public, rather than behind closed doors.

A New Paradigm for Scientific Publishing, Peer Review, and Impact Assessment

Scientific publishing and peer review have evolved little in three centuries, while the demands placed on them have grown profoundly. The growing role of artificial intelligence has underscored deep, systemic shortcomings of an aging system that has largely evaded innovation, a system whose origins are appallingly closer to the invention of the printing press than to the internet. We can do better – much better. This article is intended as the beginning of a communal experiment: a living document that critically reviews the modern academic publishing and peer-review system and presents a concrete framework to address what bibliometrics experts¹ have characterized as "the pervasive misapplication of indicators to the evaluation of scientific performance". Building on the Leiden Manifesto, DORA, and a body of scholarship spanning many disciplines and decades, we present a community-governed, non-profit platform organized around three trust-weighted impact factors, for articles, authors, and reviewers, with full algorithmic transparency, an open development log, and structural decoupling of credibility scoring from content moderation and from monetization. We invite the community to discuss, critique, and help shape it.

Open Evaluation: A Vision for Entirely Transparent Post-Publication Peer Review and Rating for Science

The two major functions of a scientific publishing system are to provide access to and evaluation of scientific papers. While open access (OA) is becoming a reality, open evaluation (OE), the other side of coin, has received less attention. Evaluation steers the attention of the scientific community and thus the very course of science. It also influences the use of scientific findings in public policy. The current system of scientific publishing provides only journal prestige as an indication of the quality of new papers and relies on a non-transparent and noisy pre-publication peer review process, which delays publication by many months on average. Here I propose an OE system, in which papers are evaluated post-publication in an ongoing fashion by means of open peer review and rating. Through signed ratings and reviews, scientists steer the attention of their field and build their reputation. Reviewers are motivated to be objective, because low-quality or self-serving signed evaluations will negatively impact their reputation. A core feature of this proposal is a division of powers between the accumulation of evaluative evidence and the analysis of this evidence by paper evaluation functions (PEFs). PEFs can be freely defined by individuals or groups (e.g. scientific societies) and provide a plurality of perspectives on the scientific literature. Simple PEFs will use averages of ratings, weighting reviewers (e.g. by H-factor) and rating scales (e.g. by relevance to a decision process) in different ways. Complex PEFs will use advanced statistical techniques to infer the quality of a paper. Papers with initially promising ratings will be more deeply evaluated. The continual refinement of PEFs in response to attempts by individuals to influence evaluations in their own favor will make the system ungameable. OA and OE together have the power to revolutionize scientific publishing and usher in a new culture of transparency, constructive criticism, and collaboration.

Post-publication critique at top-ranked journals across scientific disciplines: a cross-sectional assessment of policies and practice

Journals exert considerable control over letters, commentaries and online comments that criticize prior research (post-publication critique). We assessed policies (Study One) and practice (Study Two) related to post-publication critique at 15 top-ranked journals in each of 22 scientific disciplines ( N = 330 journals). Two-hundred and seven (63%) journals accepted post-publication critique and often imposed limits on length (median 1000, interquartile range (IQR) 500–1200 words) and time-to-submit (median 12, IQR 4–26 weeks). The most restrictive limits were 175 words and two weeks; some policies imposed no limits. Of 2066 randomly sampled research articles published in 2018 by journals accepting post-publication critique, 39 (1.9%, 95% confidence interval [1.4, 2.6]) were linked to at least one post-publication critique (there were 58 post-publication critiques in total). Of the 58 post-publication critiques, 44 received an author reply, of which 41 asserted that original conclusions were unchanged. Clinical Medicine had the most active culture of post-publication critique: all journals accepted post-publication critique and published the most post-publication critique overall, but also imposed the strictest limits on length (median 400, IQR 400–550 words) and time-to-submit (median 4, IQR 4–6 weeks). Our findings suggest that top-ranked academic journals often pose serious barriers to the cultivation, documentation and dissemination of post-publication critique.

Making sense of preprints by adding context – The Publish Your Reviews initiative - LSE Impact

Improving scientific publishing is often framed as an issue of openness and speed and less often as one of context. In this post, Ludo Waltman and Jessica Polka make the case for a more contextualised approach to open access publishing and preprinting, and introduce the Publish Your Reviews initiative. Launched today by ASAPbio, the initiative

You can just review things: A digital ethnography of informal peer review

Across scholarly communities, manuscripts face similar evaluative rituals: editors invite experts to privately assess submissions through formal peer reviews. This closed, loosely structured, and...

A Claude skill for pre-submission peer review

Open-source skill to use Claude + Consensus for pre-submission mock peer review backed by citations to published, peer-reviewed literature.

Introducing COSIG: The Collection of Open Science Integrity Guides

Investigating the integrity of published scientific papers is key to the scientific process, but the necessary knowledge is in short supply. We present COSIG, an open collection of meta-scientific guides enabling anyone to perform forensic peer review.

Guest Post — Is Growth Always Good News? 2026 Article Submission Surges - The Scholarly Kitchen

ScholarOne saw a submission surge in the first quarter of 2026 — evidence that AI is increasing the strain on peer review's social contract with researchers.

The Story behind the Science: Preprints of pandemic potential—how bioRxiv and medRxiv brought preprints to the life sciences

ABSTRACT The bioRxiv and medRxiv preprint servers brought preprinting to the life sciences and played a critical role in disseminating COVID research during the pandemic. Here, I reflect on the birth of bioRxiv and medRxiv and the crucial role so many members of the community played, our experience during the pandemic, and the launch of the new non-profit organization set up to oversee the servers. The pandemic was a stress test for bioRxiv and medRxiv that demonstrated their value and robustness. Under the umbrella of openRxiv, they are now poised to become long-term infrastructure underpinning a new publishing ecosystem.

A billion-dollar donation: estimating the cost of researchers’ time spent on peer review

The amount and value of researchers’ peer review work is critical for academia and journal publishing. However, this labor is under-recognized, its magnitude is unknown, and alternative ways of organizing peer review labor are rarely considered.

Screening, sorting, and the feedback cycles that imperil peer review

Scholarly journals rely on peer review to identify the science most worthy of publication. Yet finding willing and qualified reviewers to evaluate manuscripts has become an increasingly challenging task, possibly even threatening the long-term viability of peer review as an institution. What can or should be done to salvage it? Here, we develop mathematical models to reveal the intricate interactions among incentives faced by authors, reviewers, and readers in their endeavors to identify the best science. Two facets are particularly salient. First, peer review partially reveals authors’ private sense of their work’s quality through their decisions of where to send their manuscripts. Second, journals’ reliance on traditionally unpaid and largely unrewarded review labor deprives them of a standard market mechanism—wages—to recruit additional reviewers when review labor is in short supply. We highlight a resulting feedback loop that threatens to overwhelm the peer review system: (1) an increase in submissions overtaxes the pool of suitable peer reviewers; (2) the accuracy of review drops because journals must either solicit assistance from less qualified reviewers or ask current reviewers to do more; (3) as review accuracy drops, submissions further increase as more authors try their luck at venues that might otherwise be a stretch. We illustrate how this cycle is propelled by the increasing emphasis on high-impact publications, the proliferation of journals, and competition among these journals for peer reviews. Finally, we suggest interventions that could slow or even reverse this cycle of peer-review meltdown.

ReviewBench: An Extensible Framework for Benchmarking Human and AI Manuscript Review

The volume of scientific manuscripts is rising faster than the available pool of expert reviewers, and AI tools are emerging as a possible response, ranging from frontier large language models applied directly to peer review to purpose-built multi-agent systems. Scalable, standardized benchmarks are needed to regularly evaluate how these tools compare to one another and to human reviewers. We present ReviewBench, an open-source, venue-agnostic framework that compares human and AI reviews across structure, alignment with a paper’s major claims, impact, and critique category. We apply ReviewBench to 145,021 review comments from human reviewers, frontier large language models (GPT-5.2, and Gemini 3 Pro), and Reviewer3.com (R3), a multi-agent peer review system. The dataset spans papers in computer science (ICLR 2025, n = 1,000), social science (Nature Human Behaviour, n = 142), and life science (eLife, n = 1,000). Across disciplines, AI reviews are more structured and engage more directly with a paper’s major claims, with R3 more often surfacing consequential comments, defined as comments capable of undermining those claims. When restricting to critical comments, however, human reviewers rank first on consequential rate on more individual papers than any AI source, despite a lower average. We identify a bimodal reviewer distribution with peaks near 0% and 100%, indicating that many reviewers outperform AI on this metric, but a substantial fraction of reviewers near 0% brings the average down. Critique typing demonstrates systematic differences, where humans emphasize contribution and clarity, while AI emphasizes validity, sufficiency, and transparency. Together, these findings argue against framing AI as a replacement for human review and instead support a complementary model in which AI scales technical verification of major claims while human judgment remains essential for evaluating contribution and shaping editorial decisions.