RegCheck: A tool for structured comparisons between study registrations and papers

Across the social and medical sciences, researchers recognize that specifying planned research activities (i.e., 'registration') prior to the commencement of research has benefits for both the transparency and rigour of science. Despite this, evidence suggests that study registrations frequently go unexamined, minimizing their effectiveness. In a way this is no surprise: manually checking registrations against papers is labour- and time-intensive, requiring careful reading across formats and expertise across domains. The advent of AI unlocks new possibilities in facilitating this activity. We present RegCheck, a modular LLM-assisted tool designed to help researchers, reviewers, and editors from across scientific disciplines compare study registrations with their corresponding papers. Importantly, RegCheck keeps human expertise and judgement in the loop by (i) ensuring that users are the ones who determine which features should be compared, and (ii) presenting the most relevant text associated with each feature to the user, facilitating (rather than replacing) human discrepancy judgements. RegCheck also generates shareable reports with unique RegCheck IDs, enabling them to be easily shared and verified by other users. RegCheck is designed to be adaptable across scientific domains, as well as registration and publication formats. In this paper we provide an overview of the motivation, workflow, and design principles of RegCheck, and we discuss its potential as an extensible infrastructure for reproducible science with an example use case.

Verified.Quest

Below you will find answers to the most common questions about this project, instructions for requesting verification, and details on how to see badges on the Officially Verified accounts in your feed.

Motif Neurotech — Therapeutic Brain Computer Interface For Mental Health

We're building devices to help you monitor and regulate your mental health and performance.

Center for Precision Psychiatry & Mental Health

The Stavros Niarchos Foundation Center for Precision Psychiatry is a joint effort of the Columbia Department of Psychiatry at CUIMC and Columbia’s Zuckerman Mind Brain Behavior Institute.

Scientific Agents: Evaluating Profession-Specific System Prompts on Scientific Tasks

Detailed profession-specific system prompts raise token use and estimated cost per response without a consistent accuracy gain. We evaluate Scientific Agents, an open-source corpus of 503 profession-specific AGENTS.md profiles, with Gemini 3.8 Flash via OpenRouter in the Pi agent harness. We compare matched profiles with four controls: a minimal baseline ("You are a helpful assistant"), the profile's opening role sentence, a generic scientific rigor guide, and a profile from an unrelated domain. Across nine text-based science benchmarks (4,531 sampled questions, 100 matched profiles), 4,488 items completed all five conditions after API-error retries, scored with automated, rule-based grading. The average profile-baseline accuracy difference is -0.6 percentage points (95% bootstrap interval [-1.5, +0.2] across fixed tasks), and no benchmark shows a statistically clear improvement. Matched profiles produced 1.5-2.3 times as many output tokens and cost 2.2-4.5 times more per successful call. On 60 tool-using BioMysteryBench bioinformatics problems (three runs each for baseline and profile), mean solve rates were 46.7% with the profile and 56.7% at baseline, a difference of -10.0 percentage points (95% interval [-16.7, -3.3]) driven by more frequent token- and time-limit stops under the profile. Longer prompts had one unexpected operational advantage: on SuperGPQA, frequent provider API drops left the short baseline with a correct first-pass answer on only 54.0% of items, against 71.6% with the profile. Generic and mismatched prompts were about as reliable, so this gain comes from prompt length or formatting rather than domain expertise. For the tested model and tasks, loading full profession profiles by default does not improve accuracy and costs considerably more; whether selective retrieval of profile sections or open-ended scientific tasks would change this remains to be tested.

Autism Checklist of DOOM

Our autism checklist of doom is meant to help people the kinds of things that may upset an autistic person, ones that aren't always obvious.

Metaventory — the meta-science community

A comprehensive community of the people building meta-science and the tools they make — psychology and economics first, growing science-wide.

RegCheck

RegCheck is an AI tool to compare preregistrations with papers instantly.

Need Some Help Reaching Your Goals? Try These 5 Habit-Tracking Apps

They can help you set — and stick to — your goals

Hypercerts: Recognizing and Rewarding Impact – ATProto Implementation

Hi all, excited to share some of what we’ve been building and to introduce hypercerts to the ATProto community. Where we started The hypercerts project began at Protocol Labs with a simple goal: improve how we fund public goods. Many of the contributions society relies on most—open-source software, scientific R&D, and ecological regeneration—remain underfunded because their value is hard to see, coordinate around, and reward. What are hypercerts Hypercerts address this by serving as digital im...

Retraction Note: High replicability of newly discovered social-behavioural findings is achievable

Nature Human Behaviour - Retraction Note: High replicability of newly discovered social-behavioural findings is achievable

When the Scoreboard Becomes the Game, It’s Time to Recalibrate Research Metrics - The Scholarly Kitchen

Today's guest post discusses research metrics and their relationship to research integrity, inclusivity, and long-term impact.

What Isn’t There an App For? (Published 2015)

A writer spent three weeks trying to improve his life through apps.