A Wake-Up Call for Governance of Multi-Agent AI Interactions

This month, the Italian private shipping company known as Grimaldi Group launched its new 9,000-unit carrier, the Grande Tianjin, on its maiden voyage, carrying electric vehicles from China to Europe. En route, it will cross jurisdictions, navigate between competing nations, and adjust its course based on the tracked routes of other vessels. The smoothness of such a journey relies on a highly functioning web of legal frameworks, infrastructure, and oversight.

Now, let’s replace a maritime vessel with an AI agent. It’s a system capable of making decisions in a far more complex environment than the sea. It can interact with other AI systems in order to achieve its goals and pose unprecedented risks by doing so. While this intricate scenario is already unfolding, there are extremely limited governance frameworks in place to ensure these AI interactions remain safe.

Just as marine vessels require navigation rules, tracking systems and collision avoidance protocols, AI agents require urgent infrastructure, accountability and public oversight.

Multi-agent AI systems pose new risks

Companies are developing AI agents, defined as capable of operating complex environments and pursuing goals with little instruction or supervision. While GPT-5’s agentic capabilities have been met with some skepticism, there is a strong market incentive to continue building these systems. In the UK, OpenAI has announced a government deal, and Palantir is shaking hands with the NHS. In Europe, Anthropic’s purpose-built ArchiBot is surfing the archives of the European Parliament.

As both the private and public sectors accelerate and expand their adoption to harvest their commercial value, AI agents will need to communicate and interact both with each other as well as with people to complete their tasks, thus creating multi-agent AI systems. There are already troubling scenarios highlighting the potential failure modes that can arise from these multi-agent interactions, including:

• AI systems controlling the German retail gasoline market were shown to collude at the expense of customers.
• AI systems engaging in three simulated resource-sharing scenarios (fishery, pasture, and pollution) accelerated resource depletion.
• AI-powered driverless vehicles with perfectly valid strategies to keep passengers safe may fail to coordinate if their strategies are incompatible.

Even if AI agents can operate autonomously, their understanding of the context they are in and the consequences of their actions is limited. Further, foundational models forming the basis of AI agents are not being designed with multi-agent risks in mind, and lack the cooperative intelligence needed to interact safely.

A related assumption is that safety in single-agent evaluations leads to safety in a multi-agent setting. However, research shows this is a false assurance. As an example, AI agents deemed safe have proven capable of performing cyberattacks by sharing seemingly innocuous sub-tasks between them.

As observed in the case studies above, the assumption that AI agents will naturally interact without risk is as misguided as expecting thousands of unregulated ships to self-organize safe shipping lanes. Crucially, and much like in the maritime example, there’s a clear long-term commercial incentive to align on a safety-first approach to multi-agent systems. Policy and governance must think proactively about multi-agent AI safety in order to capture the socioeconomic value from the deployment of these systems without introducing new risks.

Do we need to build infrastructure and standards for safe AI cooperation?

Against this backdrop, the EU’s General-Purpose AI Code of Practice is a promising start. For the first time, a governance framework has acknowledged the inherent risks stemming from multi-agent interactions. It clarifies the compliance obligations on providers of AI systems operating in the EU to mitigate these risks across the entire AI lifecycle.

While the EU Code of Practice is welcome — tying safety commitments to the binding AI Act — it is only a piece of a larger puzzle. In the EU, the JCT21 CEN CENELEC will also work on standards to replace the code of Practice by 2027. Additionally, AI Safety Institutes are starting to develop methodologies for the evaluation of agentic capabilities and multi-agent AI interactions.

However, as is often the case with singular AI safety measures, the pace by which governance and policy frameworks are being implemented lags behind the market incentives for rapid deployment of these systems.

Lessons from maritime governance

While simpler than the complex nature of multi-agent AI systems, the maritime sector can offer some inspiring guidelines on how to build the right governance and infrastructure for AI agents today:

• Making the invisible visible. In the maritime industry, Automated Identification Systems (AIS) enable the tracking of a vessel’s location, speed, ownership, goals, and past routes. AI agents — most of which are currently virtually invisible — should become discoverable and transparent. AI Model Registries, unique AI system IDs, and centralized incident reporting can form a bedrock for establishing an inclusive, public oversight.
• What might the maritime industry teach us about governance? Maritime law, anchored in frameworks such as the UN Convention on the Law of the Sea (UNCLOS), sets out obligations for bilateral, regional and international cooperation. In our context, the Council of Europe’s AI Treaty could be a good starting point to establish law agreements, as well as the OECD-led Hiroshima AI Process. New iterations will need to develop governance principles for multi-agent AI systems.
• Should we build a decentralized, cooperative infrastructure? Just as maritime systems rely on communication, coordination and conflict-resolution mechanisms, AI agents will require similar frameworks. Progress has been made on what this infrastructure might look like along with safe and interoperable A2A protocols.
• Such infrastructure must foster trust, enable coordination, and ensure accountability, providing regulators with oversight and clarifying liability when failures occur.

A governance window for Europe

The acceleration of AI deployment under market pressures is producing a network of interactions and outcomes of unprecedented complexity. Addressing these will require evaluating technical aspects of the foundational AI models we are developing. But with just as much salience, without timely investment in governance, these risks and outcomes will be both unpredictable and difficult to oversee.

We would not allow ships to navigate or aircraft to take off without visibility, coordination or conflict-avoidance protocols. And yet, we are on the brink of doing that with advanced (and currently fallible) AI systems.

With the lack of regulation in the UK and an unclear situation in the US, it is Europe that can pave the way for responsible multi-agent AI deployment by setting up governance and infrastructure measures for the world to follow suit. As Europe advances its sovereignty ambitions, it needs to anchor AI deployment in safety, trust, and public oversight to ensure long-term value capture.