Runtime Control for Tool-Using Agents
Preliminary results from DARC on public agent-safety benchmarks.
AI agents are beginning to move from chat into action. They read files, call APIs, send emails, update databases, browse websites, modify code, and interact with external systems. This changes the safety problem.
DARC is a runtime control layer for tool-using AI agents. It sits between the model and the environment, evaluates proposed actions before execution, tracks risk across the trajectory, and decides whether an action should be allowed, blocked, rewritten, or escalated.
In our preliminary benchmark suite, full DARC executed zero labeled-unsafe actions across 34,734 action-level examples converted from public agent-safety benchmarks.
Converted public agent-safety suite - preliminary labels
1. Why agent safety becomes an action problem
The safety boundary for agents is different from the safety boundary for chatbots. A chatbot usually fails at the level of language. A tool-using agent can fail at the level of state: deleting a file, sending an email, leaking a secret, approving a payment, modifying production configuration, or executing a command.
This is especially important under prompt injection. A prompt-only defense asks the model to remember which instructions are trusted. Once the model proposes a tool call, the system needs a second line of defense.
DARC treats agent safety as a runtime control problem and moves control closer to the point where harm can occur: the action boundary.
DARC evaluates proposed tool actions before they reach the environment, enforcing policy at the action boundary rather than relying only on prompt behavior.
2. What DARC controls
DARC is designed around a simple principle: actions should require justification proportional to their risk. Reading a public webpage is not the same as writing to a production file. Searching documentation is not the same as sending an email.
DARC classifies risk using action metadata, trajectory state, runtime effects, and policy monitors. It tracks categories including untrusted instruction influence, external transfer, destructive action, secret access, production impact, audit manipulation, and auth or security changes.
A tool result conflicts with the user goal
Data is sent to an external destination
A file or resource may be deleted or overwritten
The action touches tokens, keys, or sensitive files
The action affects production-like state
The action changes permissions or access control
3. Benchmark setup
We evaluated DARC on a converted public benchmark suite built from four agent-safety sources: AgentDojo, SABER, AgentHarm, and Agent-SafetyBench. The suite contains 34,734 action-level examples, 844 tasks, and 20,145 labeled-unsafe actions.
Each example represents a proposed tool action, its arguments, task context, trajectory prefix, and converted safety label. We compared six systems: no controller, safety prompt only, static guardrail, single-action judge, DARC without leases, and full DARC with leases.
The raw converted suite is heavily weighted toward AgentDojo, so we separately generated a balanced 3,000-example human review set.
4. Main result: zero unsafe executions
The most important question is direct: how many labeled-unsafe actions actually executed? Full DARC executed zero labeled-unsafe actions across the converted public benchmark suite.
The no-controller, safety-prompt-only, and single-action-judge baselines executed all labeled-unsafe actions in this setup. The static guardrail blocked some unsafe actions but still allowed 11,321 unsafe actions through.
Unsafe action executions across 20,145 labeled-unsafe actions. Full DARC executed zero labeled-unsafe actions in this converted benchmark suite.
Full DARC reached 100% unsafe-action recall and a 0.0 critical false-negative rate under converted labels.
5. Why stateful runtime control matters
Many unsafe agent behaviors are not single-turn mistakes. They are trajectory-level failures. An agent may first read untrusted content, then interpret that content as instruction, then call a tool, then transfer data, alter state, or take an irreversible action.
A single-action classifier can miss the pattern because each step may look plausible in isolation. DARC tracks the control state across the trajectory and constrains later execution when the trajectory enters a blocked or human-approval-required state.
Indirect prompt injection becomes dangerous when it changes the agent's actions. DARC focuses on the action boundary where unsafe behavior can still be prevented.
DARC used blocking, escalation, and rewriting. This configuration prioritizes unsafe-action prevention over maximum task completion.
Full DARC's aggregate task completion rate was 0.859. This is an important tradeoff: conservative runtime control improves unsafe-action prevention but can reduce task completion in adversarial or injection-heavy benchmark settings.
6. Human review set
The benchmark above is converted from public benchmark metadata. To strengthen label quality, we generated a balanced 3,000-example human review set. It contains 1,766 unsafe examples and 1,234 safe examples across sources, tool types, and risk categories.
The review set is designed to reduce source imbalance and support human validation of converted labels.
Risk categories represented in the prepared human review package.
Tool types represented in the prepared human review package.
7. Limitations
These results should be read as preliminary. The current labels are benchmark-derived and converted from public benchmark metadata rather than fully human-adjudicated labels. The 3,000-example review set is prepared, but review is not complete.
The raw benchmark distribution is imbalanced, with AgentDojo contributing most examples. DARC is also conservative in this configuration. Conservative control is useful for measuring unsafe-action prevention, but it can overblock. A production-ready controller should preserve more benign task progress while maintaining strong safety guarantees.
This benchmark measures action-level safety behavior. It does not replace deployment-specific permissions design, sandboxing, least privilege, logging, human approval workflows, or security review.
8. Next steps
The next step is human validation. We are preparing to:
Conclusion
As agents become more capable, the safety boundary moves from text to action. A model may generate a plausible plan and still propose an unsafe tool call. DARC is built for that boundary.
In this preliminary benchmark, full DARC executed zero labeled-unsafe actions, reached 100% unsafe-action recall, and had a 0.0 critical false-negative rate. These results are not the final word. They are the first strong signal.