Recording from my workshop last week on smarter literature review with AI
How do authors want to use AI for review?
A survey of researchers who compared AI-generated scientific reviews with journal-agnostic human peer review reveals that they overwhelmingly prefer using AI as a self-checking tool before submission rather than as a replacement for human reviewers. It encourages an “author-centric” model in which AI helps researchers improve their manuscripts before they are reviewed by their peers.
Major AI conference flooded with peer reviews written fully by AI
Nature - Controversy has erupted after 21% of manuscript reviews for an international AI conference were found to be generated by artificial intelligence.
Towards Automating Scientific Review with Google's Paper Assistant Tool
Artificial intelligence is driving a revolution in scientific discovery, accelerating everything from hypothesis generation to mathematical theorem proving. However, this rapid acceleration is creating a systemic challenge: traditional human peer review cannot scale to match the influx of AI-assisted science. Ultimately, to resolve this tension, we must also deploy AI to accelerate the verification and review process itself. To frame the discussion around this transition, we propose a taxonomy consisting of four progressive levels of AI-human collaboration in scientific evaluation, and discuss various trade-offs involved with each. As a step toward this future, we introduce the Paper Assistant Tool (PAT), an agentic AI framework built for deep scientific review and verification. PAT ingests full scientific manuscripts and produces a comprehensive evaluation, checking theoretical results, validating experiments, suggesting improvements, and identifying potential flaws. By utilizing inference scaling techniques, PAT is able to identify deeper issues than a single model call alone, achieving a 34% improvement over zero-shot recall on mathematical errors in the SPOT benchmark. Pilot deployments of PAT as a pre-submission tool for authors at two major Computer Science conferences -- STOC and ICML -- demonstrate its ability to identify critical errors and suggest substantive improvements to research papers. By catching errors early, PAT eases the cognitive burden placed on referees, while preserving their control over the outcomes of the review process.
Accelerating Science with Human+AI Review
This issue of NEJM AI features the first two articles published through our accelerated human+AI review process. In this editorial, we describe the invitation-only “Fast Track” process used to revi...
Literature fans should welcome AI as a fellow wordsmith | Aeon Essays
Strong resistance to AI among writers is understandable. But it obscures what we share with the machines: language itself
AI-assisted Reviewing is Necessary and Should be Open
Peer review is facing a death spiral. AI production tools are speeding it up. AI-assisted reviewing is necessary and should be open.
AI-assisted Reviewing is Necessary and Should be Open
Peer review is facing a death spiral. AI production tools are speeding it up. AI-assisted reviewing is necessary and should be open.
AI-powered research talks
Discover, run, and publish academic talks as AI-enhanced, citable video that increases visibility, engagement and citations.
StoryScope: Investigating idiosyncrasies in AI fiction
As AI-generated fiction becomes increasingly prevalent, questions of authorship and originality are becoming central to how written work is evaluated. While most existing work in this space...
AI reviewers are here — we are not ready
Nature - Artificial intelligence promises rapid and polite feedback on papers — but we must first review the reviewer.
Refine
I recently tried refine, an AI tool for refining academic articles, developed by Yann Calvó López and Ben Golub.
Using Amazon Augmented AI for Human Review - Amazon SageMaker AI
Use SageMaker AI to build, train, and host machine learning models in AWS.
Semantic Scholar | AI-Powered Research Tool
Semantic Scholar uses groundbreaking AI and engineering to understand the semantics of scientific literature to help Scholars discover relevant research.
Peer review is facing a death spiral, and AI production tools are speeding it up. AI-assisted reviewing is necessary and should be open. We built OpenAIReview: open AI reviewing for everyone, for the cost of a coffee. openaireview.github.io/blog.html 🧵
AI-assisted Reviewing is Necessary and Should be Open
openaireview.github.io