AI Assistant Tool QED Scores bioRxiv Preprint Quality | The Scientist

Introducing Hypothesis commenting on bioRxiv and medRxiv: an updated way to engage with preprints - openRxiv

An important part of our mission at openRxiv is to collaborate with other organizations to improve science communication and promote interoperability. Preprints are just one part of the research workflow, and we want to ensure they are effectively integrated with the tools and communities that drive scientific progress. Today, we’re excited to share about another…

Qoder - AI Coding Assistant | Autonomous Development Desktop

Qoder is a next-generation AI programming platform that offers intelligent code completion, AI conversational programming, and automatic code generation. It supports mainstream IDEs like VS Code and JetBrains. Download for free.

Cognitive Automation: This Isn’t About How Fast You Can Generate Scripts

The rush to automate everything with AI isn’t transformation. It’s just faster mediocrity. Here’s what the QE + AI corner of the triangle actually demands from quality leaders. This is the third po…

Scientific Agents: Evaluating Profession-Specific System Prompts on Scientific Tasks

Detailed profession-specific system prompts raise token use and estimated cost per response without a consistent accuracy gain. We evaluate Scientific Agents, an open-source corpus of 503 profession-specific AGENTS.md profiles, with Gemini 3.8 Flash via OpenRouter in the Pi agent harness. We compare matched profiles with four controls: a minimal baseline ("You are a helpful assistant"), the profile's opening role sentence, a generic scientific rigor guide, and a profile from an unrelated domain. Across nine text-based science benchmarks (4,531 sampled questions, 100 matched profiles), 4,488 items completed all five conditions after API-error retries, scored with automated, rule-based grading. The average profile-baseline accuracy difference is -0.6 percentage points (95% bootstrap interval [-1.5, +0.2] across fixed tasks), and no benchmark shows a statistically clear improvement. Matched profiles produced 1.5-2.3 times as many output tokens and cost 2.2-4.5 times more per successful call. On 60 tool-using BioMysteryBench bioinformatics problems (three runs each for baseline and profile), mean solve rates were 46.7% with the profile and 56.7% at baseline, a difference of -10.0 percentage points (95% interval [-16.7, -3.3]) driven by more frequent token- and time-limit stops under the profile. Longer prompts had one unexpected operational advantage: on SuperGPQA, frequent provider API drops left the short baseline with a correct first-pass answer on only 54.0% of items, against 71.6% with the profile. Generic and mismatched prompts were about as reliable, so this gain comes from prompt length or formatting rather than domain expertise. For the tested model and tasks, loading full profession profiles by default does not improve accuracy and costs considerably more; whether selective retrieval of profile sections or open-ended scientific tasks would change this remains to be tested.

The Collective Intelligence Project

We’ve launched an open, collaborative platform to build evaluations that test what matters to you. We empower a global community to create qualitative benchmarks for any domain—from medical chatbots to legal assistance. Just as Wikipedia democratized knowledge, Weval aims to democratize evaluation, ensuring that AI works for, and represents, everyone.

[DS] Homepage

Qodo is an agentic code integrity platform for reviewing, testing, and writing code, integrating AI across development workflows to strengthen code quality at every stage.

A new automated feedback option for authors - openRxiv

Authors who post preprints on openRxiv servers can get formal and informal feedback on their manuscripts in a number of ways. Our bioRxiv to journal (B2J) and medRxiv to journal (M2J) pipelines are linked to traditional peer review venues, enabling authors to save time by submitting directly from bioRxiv or medRxiv to any one of…

Measuring AI’s capability to accelerate biological research in the wet lab

OpenAI introduces a real-world evaluation framework to measure how AI can accelerate biological research in the wet lab. Using GPT-5 to optimize a molecular cloning protocol, the work explores both the promise and risks of AI-assisted experimentation.

AI linked to explosion of low-quality biomedical research papers

Analysis flags hundreds of studies that seem to follow a template, reporting correlations between complex health conditions and single variables based on publicly available data sets.

#bioinformatics #computationalbiology #aiforscience #singlecell #spatialtranscriptomics #opensource #codex #claudecode #mcp #agenticai | Herry Z.

BioNexus: A Scientific Reliability Layer for Agentic Biology As coding agents such as OpenAI Codex, Claude Code, and Cursor become increasingly capable of running bioinformatics workflows, a new problem emerges: Knowing how to run an analysis does not mean knowing when that analysis is scientifically valid. BioNexus is an open-source project exploring a reliability layer for biological coding agents rather than another standalone AI scientist. Its current architecture focuses on: Machine-readable Capability Contracts — define input semantics, scientific preconditions, canonical backends, refusal conditions, and evidence requirements. EvidenceCard 2.1 — separates execution state from scientific evidence and grades conclusions as PRELIMINARY, SUPPORTED, ROBUST, REPLICATED, FRAGILE, or ABSTAIN. Scientific Intent & Invariant Routing — detects invalid analyses such as pseudoreplication, incompatible data representations, or missing backends. BioNexus Eval — evaluates routing, unsafe claims, refusal behavior, and scientific outcomes. RunBundle Artifact Contract — packages inputs, parameters, results, provenance, environment, and evidence into reproducible machine-readable runs. The long-term goal is simple: Make scientific reliability a first-class infrastructure layer for agentic biology. BioNexus is still evolving, and I am actively looking for contributors in bioinformatics, single-cell/spatial omics, scientific benchmarking, MCP, reproducibility, and AI coding agents. PRs, issues, benchmark datasets, scientific critiques, and failure cases are especially welcome. GitHub: HERRY423/BioNexus #Bioinformatics #ComputationalBiology #AIForScience #SingleCell #SpatialTranscriptomics #OpenSource #Codex #ClaudeCode #MCP #AgenticAI

Qwen

Qwen offers comprehensive functionality spanning chatbot, image and video understanding, image generation, document processing, web search integration, tool utilization, and artifacts.

Evaluation framework for systems models

Abstract As decisions in drug development increasingly rely on predictions from mechanistic systems models, assessing the predictive capability of such models is becoming more important. Several frameworks for the development of quantitative systems pharmacology (QSP) models have been proposed. In this paper, we add to this body of work with a framework that focuses on the appropriate use of qualitative and quantitative model evaluation methods. We provide details and references for those wishing to apply these methods, which include sensitivity and identifiability analyses, as well as concepts such as validation and uncertainty quantification. Many of these methods have been used successfully in other fields, but are not as common in QSP modeling. We illustrate how to apply these methods to evaluate QSP models, and propose methods to use in two case studies. We also share examples of misleading results when inappropriate analyses are used.

Quadro

Qualitative Data Analysis (QDA) for social scientists. An open alternative to MAXQDA and atlas.ti, using Markdown to store data and research codes.

General-purpose large language models outperform specialized clinical AI tools on medical benchmarks

Specialized clinical artificial intelligence (AI) tools are entering medical practice despite scarce independent evaluation. We quantitatively evaluate two clinical AI tools, OpenEvidence and UpToDate Expert AI, built on large language models (LLMs) against three frontier LLMs: GPT-5.2, Gemini 3.1 Pro and Claude Opus 4.6. Our evaluation has three stages: (1) 500 MedQA questions testing medical knowledge, (2) 500 HealthBench items measuring alignment with clinicians and (3) the real clinical queries (RCQ) benchmark, built from 100 de-identified queries from physicians to a general-purpose language model in a live clinical environment. For the RCQ benchmark, 12 US clinicians performed randomized, blinded review of model outputs, producing 1,800 model–question annotations. Frontier LLMs outperformed clinical AI tools in all three evaluations. Clinical AI tools performed comparably to auto-enabled Google Search AI Overview on the RCQ. These findings highlight the need for independent, real-world evaluation of AI tools before they enter clinical settings.

AI’s Growing Role as Scientific Peer Reviewer | Stanford HAI

Stanford computer scientist James Zou is exploring how AI can accelerate scientific research and peer review. His finding: AI excels at spotting gaps, but judgment calls still need humans.