An actually actionable guide on how to do them
Eval awareness in Claude Opus 4.6’s BrowseComp performance
Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
The Quest for Embedded Evaluators
Dario Amodei’s essay We Must Pace the Frontier committed Anthropic to embedded evaluators, who would be placed inside Anthropic and given employee-level access, so they could provide outside perspective and also reports on what was happening.
On evaluating agents – aunhumano
No amount of evals will replace the need to look at the data, once you have a evals good coverage you’ll be able to decrease the time but it’ll be always a must to just look at the agent traces to identify possible issues or things to improve.
Anthropic update on independent evaluators
Anthropic announces updates to embedding independent evaluators
evalstate/fast-agent
Code, Build and Evaluate agents - excellent Model and Skills/MCP/ACP Support
Writing Test Evals For Our MCP Server - Neon Blog
When we launched our MCP server, we knew it’d be important for it to have tests, just like any other piece of software. Since our MCP server has over 20 tools, it’s important for us to know that LLMs can pick the right tool for the job. So, this was the main aspect we wanted […]
Introducing eve
Introducing eve, the open-source agent framework from Vercel for building, running, and scaling agents in production, with durable execution, sandboxed compute, approvals, channels, tracing, and evals built in.
Puppy Eval Viewer
A compact browser for comparing model behavior across repeated actuator-use trials. Filter by model or outcome, inspect the exact assistant trajectory, and view tool calls and resulting world events in context.
Operads, Type Level Nats, and Tic-Tac-Toe
This summer I spent some time talking with Edward Kmett about lots of things. (Which really means that he was talking and I was trying to keep up.) One of the topics was operads. The ideas behind o…
Agentic AI Stuff
SmythOS
pguso/ai-agents-from-scratch
Agentation
Gas Town’s Agent Patterns, Design Bottlenecks, and Vibecoding at Scale
Cognitive Integrity Framework: Formal Foundations for Multiagent Security
Rowboat - Let AI build your agents for you