







Repeating original studies using existing data (reproducibility) or new data (replicability) is key to improving the credibility of scientific results. Here, we focus on replications, which can be…
Institute for Replication (I4R)
The Institute for Replication improves the credibility of science by systematically reproducing and replicating empirical research.
Science has been in a “replication crisis” for a decade. Have we learned anything?
Bad papers are still published. But some other things might be getting better.

Small Telescopes - Uri Simonsohn, 2015
This article introduces a new approach for evaluating replication results. It combines effect-size estimation with hypothesis testing, assessing the extent to w...

Konrad Hinsen's blog
Thirty years after my first contact with computational (ir)reproducibility, I am happy to note that many things have improved. Reproducibility, computational and otherwise, is increasingly recognized as an important aspect of scientific quality control, and mostly considered worth striving for. However, I also note that more and more people, including reproducibility activists, have lost contact with the day-to-day reality in which reproducibility matters. Reproducibility is becoming an item on a checklist, and its precise incarnation the subject of political bickering aimed at making it easy to check off that item. So let's take a look at why computational reproducibility matters for researchers.

Nature is expanding Registered Reports to all the fields in which we publish
Registered Reports improve the credibility of scientific claims by rewarding big questions, sound methods and solid analyses. They need to become a standard tool in research.

ReScience organization
Reproducible Science is good. Replicated Science is better. - ReScience organization
ACE: Audit Clinical Evidence
Independent audits of published clinical research for statistical errors, data inconsistencies, and research integrity violations.

Reproducibility in Machine Learning-based Research: Overview, Barriers and Drivers
The concept of reproducibility can have different interpretations across various research fields and even within the same field [39]. To avoid confusion, we first specify our terms, broadly defining reproducibility and then further categorizing it into various types and degrees. The first distinction comes from Goodman et al. [42], who specify a fundamental division between whether we (i) mean reproducible in principle (termed “methods” reproducibility) due to sufficient description/sharing of methodologies, materials, etc., or (ii) whether results/conclusions actually prove to be reproducible when experiments or analyses are re-done. In the second category, they distinguish “results” and “inferential” reproducibility, depending on whether the analyses or inferences to broader conclusions are reproduced.
ReproAI: the plugin for reproducible replication packages
An author-facing AI plugin that helps you prepare a cleaner, reproducible replication package before submission.

Half of social-science studies fail replication test in years-long project
Results from massive, ‘eagerly awaited’ initiative reinforce concerns about the credibility of science — but raise hope for solutions.

Related work | Data Counterfactuals
Adjacent research areas and starting papers, curated in Semble and synced into the site.

Fabricated citations: an audit across 2·5 million biomedical papers
Scientific literature depends on the integrity of its references. Each reference implicitly asserts that a verifiable source exists and supports the claims being made. When references point to non-existent studies, readers, reviewers, and policy makers are unable to evaluate the evidence.

Fabricated citations: an audit across 2·5 million biomedical papers
Scientific literature depends on the integrity of its references. Each reference implicitly asserts that a verifiable source exists and supports the claims being made. When references point to non-existent studies, readers, reviewers, and policy makers are unable to evaluate the evidence.

Training AI Scientists to Replicate Research
The replicability of papers is a cornerstone of scientific knowledge, ensuring the reliability of existing results and providing a base for further experiments. The act of replication typically illuminates details that were previously underspecified, and thus requires similar hypothesis-driven exploration to open-ended research. In this work, we develop Replica, a scalable task space for paper replication. To provide reward signal, we introduce an auto-generated rubric-based judge that has low noise and agrees with human assessment of replication quality. We post-train Faraday, a 27B-parameter "AI Scientist" agent that leverages coding agents as tools, surpassing the performance of Claude Opus 4.8 and GPT-5.5 on held-out replication tasks. Qualitative analysis of individual rollouts reveals that Faraday adopts a more scientifically-principled approach. We believe that our results provide a stepping stone towards AI agents capable of long-horizon scientific innovation without requiring complex harnesses.

Is replication <i>possible</i> in qualitative research? A response to Makel et al. (2022)
There has been much debate in recent years about how open research practices, which have been promoted in efforts to improve research robustness, may (not) be appropriate for qualitative methodolog...
