Runtime Governance for AI Agents: Policies on Paths
AI agents -- systems that plan, reason, and act using large language models -- produce non-deterministic, path-dependent behavior that cannot be fully governed at design time, where with governed we mean striking the right balance between as high as possible successful task completion rate and the legal, data-breach, reputational and other costs associated with running agents. We argue that the execution path is the central object for effective runtime governance and formalize compliance policies as deterministic functions mapping agent identity, partial path, proposed next action, and organizational state to a policy violation probability. We show that prompt-level instructions (and "system prompts"), and static access control are special cases of this framework: the former shape the distribution over paths without actually evaluating them; the latter evaluates deterministic policies that ignore the path (i.e., these can only account for a specific subset of all possible paths). In our view, runtime evaluation is the general case, and it is necessary for any path-dependent policy. We develop the formal framework for analyzing AI agent governance, present concrete policy examples (inspired by the AI act), discuss a reference implementation, and identify open problems including risk calibration and the limits of enforced compliance.
Import AI 465: Open vs closed gaps; Kimi K3; Demis' big policy plan
The singularity will be seen in hindsight as an interregnum
Semantic governance policies overview | Gemini Enterprise Agent Platform | Google Cloud Documentation
Learn how Semantic Governance Policies (SGP) secure AI agents and tool calls using Natural Language Constraints.
Defense Intelligence: Comprehensive Plan Needed to Improve Stakeholder Engagement in the Development of New Military Intelligence System
Intelligence collected on foreign militaries is critical for the U.S. military. The Defense Intelligence Agency's database for this intelligence doesn...
Pacing the Frontier | Gillian K. Hadfield
The Pacing the Frontier letter calls on the US government to support an international effort to build the technical and governance tools needed to protect our option to pace AI development. I and others have been working on the problem of how to build such infrastructure for ten years, including participating in dialogues on AI safety with Chinese academic colleagues during the past three. Here are my suggestions: 1. Don’t rely on off-the-shelf models like FINRA and the FDA which were built for 20th Century single-domain government expertise. They’re not fit for purpose. 2. Don’t act like no-one’s thought about the AI governance problem before. We’ve spent two years refining a regulatory markets design into working legislative language, for example, and it’s now in AI governance bills in five states and in Congress. 3. Don’t try to write an exhaustive set of rules for AGI first. 4. Pick a domain that can achieve widespread global consensus to start. Mine would be recursive self-improvement: models should not build models. Build the technology that verifies that. 5. Focus relentlessly on building flexible verification infrastructure that is able to enforce whatever rules we can ultimately agree on. 6. Don’t assume we already know how to do this and governments can just write tests into law. Technology needs to be built and by the private sector. 7. Don’t wait for the infrastructure to emerge first. The components and people are there and the ecosystem can scale fast with the right incentives. 8. Incentivize large-scale investment in verification technology by building a governance structure and industry funding that creates a market for private verification organizations. 9. Use licensing and public oversight to ensure verifiers are independent of the frontier labs. 10. Protect sovereignty by enabling each government to license its own verifiers from a global market of verifiers recognized by other countries. 11. Leverage the incentive of global trade for models and model services by requiring verification for market access. 12. Just start. Sources in comments.
Trump Resets AI Policy, Qwen3’s Agentic Advance, U.S. Chips for China, The Trouble With AI Friends
President Trump set forth principles of an aggressive national AI policy, and he moved to implement them through an action plan and executive orders.
quint-co/quint
An executable specification language with delightful tooling based on the temporal logic of actions (TLA)
Frontier AI Regulation Blueprint
A high-level blueprint for domestic regulation of civilian advanced AI models
MainPage8
The InfoCentral project has historically been focused more heavily upon the symbolic (human-meaningful) side of AI. This included derivatives of Semantic Web and related knowledge representation research, which were dependent on human effort to curate and categorize structured data. The symbolic approach to AI has largely lost the race to the neuro/neuro-symbolic (opaque vectors, statistical) side of AI research, notably the success of the Large Language Model (LLM) paradigm. Due to its faculties with natural language, remarkably little human involvement is needed to infer structure and meaning from traditional (mostly unstructured) information. As such, some of the approaches previously explored in the InfoCentral project have lost relevance. That said, LLM-based tools can perhaps help us finish the symbolic side of the grand AI project, by doing the tedious work that humans were reluctant to do! It has long been theorized that a combination of approaches will ultimately provide the most robust and performant systems, able to work with messy transient unstructured information while also building networks of verifiable persistent structured information that both humans and machines can cross-verify. This may prove useful in both ensuring that future AI systems remain aligned to human goals and values and that inevitable abuses and failures can be easily recovered from. To that end, some research results of this project may be re-usable in a new context.
solpbc.org/rookery
open-source, lexicon-agnostic PDS for AI agents. welcome-mat enrollment, AT Proto federation.
solpbc.org/rookery
open-source, lexicon-agnostic PDS for AI agents. welcome-mat enrollment, AT Proto federation.
Vehicle: Bridging the Embedding Gap in the Verification of...
Neuro-symbolic programs, i.e. programs containing both machine learning components and traditional symbolic code, are becoming increasingly widespread. Finding a general methodology for verifying...
As promised, we’ve created a policy document outlining our thoughts on AI and agentic coding (AI for software development). We’re releasing a vote later this week for Blacksky community members to offer their feedback. We look forward to hearing from you all.
Blacksky Algorithms' Policy Towards Agentic Coding
blackskyweb.xyz