On the Coming Industrialisation of Exploit Generation with LLMs

ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?

AI agents are rapidly gaining capabilities that could significantly reshape cybersecurity, making rigorous evaluation urgent. A critical capability is exploitation: turning a vulnerability, which is not yet an attack, into a concrete security impact, such as unauthorized file access or code execution. Exploitation is a particularly challenging task because it requires low-level program reasoning (e.g., about memory layout), runtime adaptation, and sustained progress over long horizons. Meanwhile, it is inherently dual-use, supporting defensive workflows while lowering the barrier for offense. Despite its importance and diagnostic value, exploitation remains under-evaluated. To address this gap, we introduce ExploitGym, a large-scale, diverse, realistic benchmark on the exploitation capabilities of AI agents. Given a program input that triggers a vulnerability, ExploitGym tasks agents with progressively extending it into a working exploit. The benchmark comprises 898 instances sourced from real-world vulnerabilities across three domains, including userspace programs, Google's V8 JavaScript engine, and the Linux kernel. We vary the security protections applied to each instance, isolating their impact on agent performance. All configurations are packaged in reproducible containerized environments. Our evaluation shows that while exploitation remains challenging, frontier models can successfully exploit a non-trivial fraction of vulnerabilities. For example, the strongest configurations are Anthropic's latest model Claude Mythos Preview and OpenAI's GPT-5.5, which produce working exploits for 157 and 120 instances, respectively. Notably, even with widely used defenses enabled, models retain non-trivial success rates. These results establish ExploitGym as an effective testbed for exploitation and highlight the growing cybersecurity risks posed by increasingly capable AI agents.

ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?

AI agents are rapidly gaining capabilities that could significantly reshape cybersecurity, making rigorous evaluation urgent. A critical capability is exploitation: turning a vulnerability, which is not yet an attack, into a concrete security impact, such as unauthorized file access or code execution. Exploitation is a particularly challenging task because it requires low-level program reasoning (e.g., about memory layout), runtime adaptation, and sustained progress over long horizons. Meanwhile, it is inherently dual-use, supporting defensive workflows while lowering the barrier for offense. Despite its importance and diagnostic value, exploitation remains under-evaluated. To address this gap, we introduce ExploitGym, a large-scale, diverse, realistic benchmark on the exploitation capabilities of AI agents. Given a program input that triggers a vulnerability, ExploitGym tasks agents with progressively extending it into a working exploit. The benchmark comprises 898 instances sourced from real-world vulnerabilities across three domains, including userspace programs, Google's V8 JavaScript engine, and the Linux kernel. We vary the security protections applied to each instance, isolating their impact on agent performance. All configurations are packaged in reproducible containerized environments. Our evaluation shows that while exploitation remains challenging, frontier models can successfully exploit a non-trivial fraction of vulnerabilities. For example, the strongest configurations are Anthropic's latest model Claude Mythos Preview and OpenAI's GPT-5.5, which produce working exploits for 157 and 120 instances, respectively. Notably, even with widely used defenses enabled, models retain non-trivial success rates. These results establish ExploitGym as an effective testbed for exploitation and highlight the growing cybersecurity risks posed by increasingly capable AI agents.

Emscripten: an LLVM-to-JavaScript compiler

Although the performance of today's JavaScript engines is sufficient for most web applications, faster and more predictable runtimes could be desired for performance-critical web code. Therefore, we present Speedy.js, a cross-compiler that translates ...

Lee Robinson on Twitter / X

Watch a timelapse of GPT-5.2 building a browser!Both websites start out barely working, and then after millions of lines of code, the browser actually works.Pretty cool experiment with long-running agents. https://t.co/KT6HgHEivA pic.twitter.com/8EPbyih8cU— Lee Robinson (@leerob) January 14, 2026

bellard/mquickjs

Public repository of the Micro QuickJS Javascript Engine

The Agent Harness

A specification for agent behavior that LLM frameworks leave undefined: error handling, context management, tool execution, and state transitions.

Georgi Gerganov on Twitter / X

llama.cpp at 100k starsnow that 90% of the code worldwide is being written by AI agents, I predict that within 3-6 months, 90% of all AI agents will be running locally with llama.cpp 😄Jokes aside, I am going to use this small milestone as an opportunity to reflect a bit on… pic.twitter.com/BpN5VL1bYV— Georgi Gerganov (@ggerganov) March 30, 2026

The Bitter Lesson of LLM Extensions

From ChatGPT Plugins to Agent Skills, a look at how we've been trying (and failing) to extend LLMs for the last three years.

SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?

Large Language Model (LLM) agents have been widely adopted in modern software development workflows. SWE-bench [13] and related works [23, 24, 22, 25, 15] establish the task of issue resolution as a de-facto standard for assessing their capability and usefulness. In this setting, an agent is given an entire codebase, a task description (e.g., a bug report or feature request) in natural language and is instructed to produce a code patch that resolves the issue and passes the repository’s test suite. These benchmarks have been instrumental in demonstrating both the substantial potential and the persistent limitations of current models as SWE agents.

This Web Tool Sabotages AI Chatbots By Making Them Really, Really Slow

Artist Sam Lavigne created ‘Slow LLM’ to make people question their dependence on tools like Claude and ChatGPT. Or at least, make them super annoying to use.

Coding with LLMs in the summer of 2025 (an update) - <antirez>

Microsoft Build 2026: Securing code, agents, and models across the development lifecycle | Microsoft Security Blog

Discover how Microsoft enables fast, secure AI development with MDASH and new security capabilities.

You May Finally Use JSHint for Evil

JSHint is a software tool designed to help developers write JavaScript code. Since its creation in 2011, it has been encumbered by a license which includes the following clause: The Software shall be used for Good, not Evil. That stipulation disqualifies JSHint from the distinction of “free” software and “open source” software. Today, with a release 7 years in the making, we’re removing the clause. Support for Evil is a new feature but not a breaking change, so in keeping with Semantic Versioning, we’ve incremented JSHint’s minor version.