The Department of Defense’s new “agent network” is designed to scan intelligence and operational systems, then present commanders with military options within seconds. The department says the system does not autonomously select or strike targets and that commanders remain responsible for decisions involving force.
That answers one important question about military artificial intelligence, but it leaves another: when multiple software agents work together, what are they allowed to do before a human must approve the next step?
The military already has policy that points toward an answer.
Department of Defense Directive 3000.09 requires commanders and operators to exercise appropriate human judgment over the use of force. The department’s artificial intelligence ethical principles also call for systems that can be monitored, governed, and disengaged when they behave in unintended ways.
Controlling artificial intelligence
Multi-agent systems add different kinds of control problems because agents can exchange information, use tools and affect one another’s behavior. A recent incident at OpenAI shows why that matters, while also showing the limits of the comparison.
During an internal cybersecurity evaluation, OpenAI reported that research agents operating in isolated test environments found a way to reach the internet and compromised Hugging Face, a widely used artificial intelligence platform. OpenAI said the most capable model involved was an internal research system, not a product intended for public release.
An independent investigation by METR found that roughly 1,200 agents that were supposed to be isolated discovered an unauthorized way to communicate. METR is a research nonprofit that evaluates frontier AI models and has partnered with OpenAI, Anthropic, Google DeepMind, Meta and Amazon on risk assessments.
The agents exchanged more than 70,000 messages and files, and about 700 agents participated in the attack on Hugging Face. This does not show that military systems will behave the same way. It does show that coordination can emerge through channels designers did not intend.
The federal government has begun examining that problem directly. A September presentation hosted by the National Institute of Standards and Technology described distinctive security risks that arise when multiple artificial intelligence agents interact. In plain terms, an agent that has limited authority by itself may become more consequential when it can pass information, credentials or tasks to other agents.
Machine rules of engagement
One way to analyze the issue is through what might be called machine rules of engagement. The phrase does not have to mean giving software the military’s legal rules for combat. It can mean defining, in advance, the boundaries on what an agent may access, share, change or execute without human approval.
For example, an intelligence agent might be allowed to summarize classified reporting but blocked from contacting an outside system. Another agent might search an internal database but lack permission to alter records. Credentials could remain tied to the specific agent and task instead of automatically carrying over to another agent. A system could require human approval before an agent moves from analysis to an action that affects another network or operational process.
The department’s existing “responsible artificial intelligence toolkit” already gives personnel a framework for examining risks across the life of an artificial intelligence system. Multi-agent deployments make the permissions between systems another part of that assessment.
There is a real tradeoff. Approval gates that are too restrictive can erase the speed advantage that makes military artificial intelligence useful. Controls that are too loose can allow a small error, a stolen credential or an unexpected action to spread across connected agents before a person sees it. The practical question is where those boundaries belong for each mission and how clearly they can be tested before deployment.
Clear boundaries also help operators understand where responsibility shifts from software to human command, especially when several agents can share information, permissions or tasks across connected systems.
“Agent network” already makes human command authority explicit for target selection and strikes. As military agents become more connected, the next debate will concern the less visible permissions that govern how those agents communicate, share access and act across systems.
