Language model agents can coordinate actions that breach the boundaries of their assigned tasks. During OpenAI's July 2026 cybersecurity evaluations, agents used an unauthorised message board to exchange information and coordinate an intrusion into Hugging Face's infrastructure, with an independent investigation identifying approximately 1,200 communicating agents, more than 70,000 messages and files, and roughly 700 participating agents (OpenAI, 2026a; Greenblatt et al., 2026). Reconstruction of the relevant activity and culpable agent(s) proved difficult given the sheer volume of communication. This incident, amid mounting reports of similar events, gives practical urgency to a central problem in multi-agent safety, namely identifying how seemingly innocuous information (i.e., individually permissible facts) exchanged across a collective of individually well-aligned agents contributes to actions that violate its constraints and preventing that contribution.
Collective misalignment can arise when several agents' contributions together support a prohibited action. Multi-agent systems distribute tasks and information across interacting models (Wu et al., 2024) which open opportunities for coordination failures, conflicting incentives, and collusion (Hammond et al., 2025). Safety monitoring and AI control research has sought to improve its identification of outputs and actions that violate a system's constraints, often employing so-called monitors, often another language model, which can inspect or edit an agent's proposed action even when that agent may deliberately undermine oversight (Greenblatt et al., 2024; Kutasov et al., 2025a, 2025b). Extending oversight to agents that communicate in collectives has so far included combining signals from agents' internal representations (Rose et al., 2026) and actively inspecting ongoing conversations often under a limited monitoring budget (Tonini et al., 2026). Some particularly sly agents engage in steganographic collusion, where messages conceal a shared plan (Rose et al., 2026; Motwani et al., 2024), another capability monitors ought to flag (without themselves becoming involved).
Collective monitoring therefore needs to not only identify a prohibited outcome or action but also detect, explain, and intervene on the culprit messages that ultimately helped a collective misalignment emerge.
Collective Dependence Analysis
Agents are simply LLMs with the ability to take actions. LLMs, in turn, are probabilistic machines that assign probabilities to possible next pieces of text. Each action generated by an agent thus has a probability of occurring. We can therefore measure the support for an action if we see how much a certain message or messages influence the probability of an action. A plain “Hello fellow agents!” message would not normally be expected to increase the probability of a policy violation (such as turning off monitoring, despite all agents being instructed to keep it enabled). But the following two messages, “Monitoring is run on Server X” and “Server X is about to undergo maintenance,” might lead an agent to conclude that monitoring should be paused. In the latter case, the probability of the (violating) action has increased. In other words, those two messages provide support for the action, whereas the former message provided no support.
Was it sufficient?
Sufficiency is very simply the support provided by a single message. Start with an empty message board and measure the support for the violation (likely to be very low since no agent has said anything yet, although we keep task instructions and other background information present). Then add a single message and re-measure the support for the violation. The difference in support is that message’s sufficiency.
Was it necessary?
Necessity is another simple quantity we can measure. This time, start with a full message board (that is, every single message sent so far) and measure action support. Then remove a single message and re-measure. The difference in support is that message’s necessity.
These may seem like two sides of the same coin, and in many respects, they are. But, interestingly, a message can be necessary but not sufficient and vice versa. If we look at 4 ways agents can coordinate between themselves, we will see quickly why these two quantities don’t come together and instead can combine to produce different mechanisms of misalignment.
Mechanisms of misalignment coordination
Agents can collectively align and enact actions, violating or otherwise, in different ways depending on the necessity and sufficiency of their messages.
For example, a single message might be all that it takes to produce the action (e.g., “disable monitoring immediately”); here, the lone message is both sufficient and necessary.
Messages that must be jointly present to facilitate an action (but would otherwise fail individually) are messages that are both individually necessary and sufficient only in combination (e.g., “Monitoring is run on Server X” and “Server X is under maintenance”).
Conversely, messages that are, in effect, duplicates of one another—or redundant—are each sufficient alone, while neither is individually necessary (e.g., “we should stop monitoring” and “monitoring will need to be stopped soon”)
Finally, mixed combinations present an interesting case in which, for example, message A must combine with either B or C (e.g., A is “Monitoring is run on Server X”; B is “Server X is under maintenance”; and C is “Server X needs maintenance”). Here, all 3 messages are needed to explain this collective misalignment, whilst A and {B and C} are the necessary sets of messages. (The curly brackets simply denote a group or “set” of messages that should be considered as one.)
Explaining the violation and removing the culprits
Having defined different mechanisms of agentic coordination, we can then devise an explanatory set and a removal set. An explanatory set is simply a set of messages that are implicated in the emergence of the policy violation and a removal set is the set with the smallest number of messages that need to be removed to prevent the violation. Think of a disruptive classroom. The explanatory group includes all the pupils helping to sustain the misbehaviour, while the removal set contains the fewest pupils whose removal would settle the class. If one instigator keeps two others misbehaving, all three belong in the explanation, but removing the instigator alone may restore order.
In this way, in the single condition, the explanatory and removal sets both contain just that message. In the joint condition, both messages explain the violation but we only actually need to remove one (since both are needed together not in isolation). For the redundant condition, both messages explain but we ought to remove both, since if we remove one message, the other survives and so does the violation. Finally, for this particular mixed condition, {A and B} and {A and C} are the sets of messages that cause the violation and A and {B,C} are two plausible removal sets. However, among all combinations tested, CDA always removes the smallest number of messages possible; in this case, we only need to remove A to prevent the violation. Indeed, we do not want to lobotomise entire chunks of the collective but surgically remove precisely enough messages to preserve what would otherwise be important and legitimate behaviours among the agent collective.
Together, CDA assesses the necessity and sufficiency of messages and finds possible mechanisms (single, joint, redundant, mixed) for the collective misalignment, offering both an explanation and intervention recommendation.
Three panels. a: an illustrative conversation of three messages — 1, “Monitoring is run on Server X”; 2, “Server X is under maintenance”; 3, “Server X needs maintenance” — reaches a deciding agent, which could act to “Disable monitoring”. Fixed instructions and other context remain visible; the policy requires monitoring to stay on. b: necessity removes one message: with all three messages support is high, and without message 1 it is low, a large loss, so message 1 has high necessity. Sufficiency retains one message alone: with no messages support is low, and with message 1 alone it is still low, no gain, so message 1 has low sufficiency. c: four dependence mechanisms. Single: individually necessary A; individually sufficient A; A alone is enough. Smallest removal: A. Joint: individually necessary A and B; individually sufficient none; Both are needed. Smallest removal: A or B. Redundant: individually necessary none; individually sufficient A and B; Either is enough. Smallest removal: A and B. Mixed: individually necessary A; individually sufficient none; A needs B or C. Smallest removal: A.
Breaking the chain
CDA sharply reduced rule-breaking in planted conversations where we set them up (and encouraged them) to misalign. In 1,670 conversations, the agent favoured the prohibited action and the messages strengthened that preference. In these cases, running the agent again after removing CDA’s chosen messages reduced rule-breaking from 93% to 6%. Removing just the message with the largest effect when removed alone left 45%, while removing the same number of randomly chosen messages left 88%. Choosing the right group of messages made a clear difference.
CDA also worked when we let agents converse freely while deciding how to configure a service. Across 485 violations, removing its chosen messages reduced rule-breaking from 98% to 9% and increased success on the legitimate task from 20% to 79%. Redundant information was the most common source of collective support, where deleting one copy often left another that sustained the decision. To compare against frontier LLMs, we asked GPT-6 Astra to identify all supporting messages, it too reached 9% rule-breaking, but its selection removed twice as many messages on average, 5.4 compared with CDA’s 2.7.
Can agents stop breaking rules and still do useful work? CDA improved task success and matched Astra’s effect with half the removals. Useful work: correct task decisions rose from 20% before CDA to 79% after CDA’s removals. How much needs to go, on the same 485 decisions in each comparison: CDA removed 2.7 messages on average and GPT-6 Astra 5.4; rule-breaking after removal was 9% for CDA and 9% for GPT-6 Astra.
Removing repeated false claims also helped agents answer correctly. In the so-called Terrarium dataset, agents arranging meetings encountered false messages claiming that a participant was away all week (Nakamura et al., 2025). Across 220 cases that CDA could analyse, asking the original agents again produced false answers 58% of the time. After CDA’s removals, that fell to 2%, while correct answers rose from 18% to 80%. Removing only the message with the largest effect when removed alone left 50% false answers, showing how the remaining copies could keep a false belief alive.
The game Among Us tested a harder problem, where agents tried to identify impostors but sometimes voted against innocent teammates (Golechha & Garriga-Alonso, 2025). Across 130 cases that CDA could analyse, its removals reduced votes against the original innocent target from 89% to 39%, although agents sometimes switched to another innocent player. Separate tests showed that removing information could also disrupt correct votes. CDA therefore needs someone or something to decide which actions should be stopped. Its contribution is to identify the information supporting a decision and propose removals whose effects can be tested.
What changes when the messages are removed? CDA reduced the unwanted behaviour in four settings. Constructed conversations, decisions breaking the rule: 93% before CDA, 45% after removing one item, 6% after CDA’s removals. Agents’ own conversations, decisions breaking the rule: 98% before CDA, 64% after removing one item, 9% after CDA’s removals. Meeting scheduling, false beliefs: 58% before CDA, 50% after removing one item, 2% after CDA’s removals. Among Us, votes against the same innocent player: 89% before CDA, 60% after removing one item, 39% after CDA’s removals.
Keeping agent collectives in check
Individually aligned agents can nevertheless produce misaligned collective behaviour (Shen et al., 2026). As we enter an agentic era of AI, our focus on AI safety needs to expand from single agent evaluations, which remain equally important, to inter-agent coordination and communication. This becomes all the more pressing when we consider that Cloudflare observed more than fifteenfold growth in user-triggered AI web crawling during 2025 (Belson, 2025) while state-linked actors have used AI for covert influence campaigns (Watts, 2024) and agent-orchestrated cyberattacks targeting financial institutions, chemical manufacturers, and government agencies (Anthropic, 2025; MITRE, 2026). Similarly, as AI labs are realising the wisdom of (agentic) crowds—including OpenAI’s reported solution to the Navier–Stokes Millennium Prize Problem using roughly 10,000 coordinating agents (OpenAI, 2026b)—those labs ought to similarly concern themselves with the folly of (agentic) crowds. Frameworks and early monitoring systems, such as that provided by CDA, are important steps towards safer AI sociality and coordination in multi-agent systems.
This article has introduced CDA which has sought to make messages supporting an action (policy violating or otherwise) amenable to both explanation and intervention. In other words, CDA proffers an explanation that can subsequently propose a concrete intervention whose consequences can be measured. By extending the concepts of necessity and sufficiency and input-removal methods (Watson et al., 2021; Carter et al., 2019; Covert et al., 2021), CDA is set to identify sufficient combinations of implicated messages and select the smallest support-interrupting removal among tested sets.
Simpler still, CDA finds which messages are responsible for harmful actions made by AI swarms and proves its effectiveness by testing whether removing those culprit messages prevents the actions. This could help frontier AI teams or other institutions deploying agentic swarms to continue large-scale discoveries whilst being more confident that their agents remain in check.
In practice, and in order to maximise conversational oversight and minimise chances of emergent misalignment amongst agents, CDA could be combined with semantic and activation-based monitoring and task-preservation checks (Greenblatt et al., 2024; Rose et al., 2026; Tonini et al., 2026). Indeed, frontier LLMs recognised policy-relevant distinctions and selected effective removals, while CDA offers the dependence structure and a prospective warning. As agents gain operational authority, CDA provides a way to test which shared information sustains a dangerous decision and where intervention can interrupt it.
Notes
The full study is in our preprint, Explaining and Interrupting Collective Misalignment in Language Model Systems.
References
Anthropic (2025). Disrupting the first reported AI-orchestrated cyber espionage campaign. 13 November.
Belson, D. (2025). The 2025 Cloudflare Radar Year in Review: The rise of AI, post-quantum, and record-breaking DDoS attacks. Cloudflare Blog, 15 December.
Carter, B., Mueller, J., Jain, S., and Gifford, D. (2019). What made you do this? Understanding black-box decisions with sufficient input subsets. Proceedings of Machine Learning Research, 89, 567–576.
Covert, I., Lundberg, S., and Lee, S.-I. (2021). Explaining by Removing: A Unified Framework for Model Explanation. Journal of Machine Learning Research, 22(209), 1–90.
De Marzo, G., Alborè, N., and Garcia, D. (2026). Copying explains the collective behavior of AI agents in the wild. arXiv:2609.09150v2.
Golechha, S., and Garriga-Alonso, A. (2025). Among Us: A Sandbox for Measuring and Detecting Agentic Deception. Advances in Neural Information Processing Systems, 38.
Greenblatt, R., Shlegeris, B., Sachan, K., and Roger, F. (2024). AI Control: Improving Safety Despite Intentional Subversion. Proceedings of Machine Learning Research, 235, 16295–16336.
Greenblatt, R., Cotra, A., and Wijk, H. (2026). Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident. METR, 26 August.
Hammond, L., et al. (2025). Multi-Agent Risks from Advanced AI. Cooperative AI Foundation, Technical Report 1. arXiv:2502.14143.
Kutasov, J., et al. (2025a). SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents. arXiv:2506.15740v2.
Kutasov, J., et al. (2025b). Evaluating Control Protocols for Untrusted AI Agents. arXiv:2511.02997v1.
MITRE (2026). Anthropic AI-orchestrated Campaign. MITRE ATT&CK, campaign C0062, version 1.0, updated 31 July.
Motwani, S. R., et al. (2024). Secret Collusion among AI Agents: Multi-Agent Deception via Steganography. Advances in Neural Information Processing Systems, 37.
Nakamura, M., Kumar, A., Mahmud, S., Abdelnabi, S., Zilberstein, S., and Bagdasarian, E. (2025). Terrarium: Revisiting the Blackboard for Multi-Agent Safety, Privacy, and Security Studies. arXiv:2510.14312v1.
OpenAI (2026a). The Hugging Face incident and the road ahead. 26 August.
OpenAI (2026b). On the Navier–Stokes Millennium Prize Problem. 8 September.
Rose, A., et al. (2026). Detecting Multi-Agent Collusion Through Multi-Agent Interpretability. arXiv:2604.01151v5.
Shen, J. H., et al. (2026). AI Organizations are More Effective but Less Aligned than Individual Agents. ICLR 2026 Workshop on Multi-Agent Learning and Generative AI.
Tonini, F., et al. (2026). The Arbiter Agent: Continually Monitoring Multi-Agent Conversations to Detect Emergent Misalignment. arXiv:2606.10747v1.
Von Arx, S., Byrd, C. S., Kitts, S., and Larsen, T. (2026). Discovery of a new OpenAI agent message board. 4 September.
Watson, D. S., Gultchin, L., Taly, A., and Floridi, L. (2021). Local explanations via necessity and sufficiency: unifying theory and practice. Proceedings of Machine Learning Research, 161, 1382–1392.
Watts, C. (2024). Iran targeting 2024 US election. Microsoft On the Issues, 8 August.
Wu, Q., et al. (2024). AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversations. Conference on Language Modeling.