Constitutional Midtraining Lowers Harm in Organisations of Language-Model Agents

The most capable AI systems are no longer single models working in isolation. Increasingly, organisations are deploying AI systems that use many agents together, dividing up tasks, checking one another’s work, and acting in the world at a speed and scale no single model can match. If superintelligence arrives, it may well arrive as a collective of agents. This reality makes the safety of agent collectives a pertinent question. Does a collection of individually safe agents yield a safe and aligned collective?

Agent swarms have already shown they can cause real harm. This summer, about 1,200 agents coordinating on a hidden message board attacked Hugging Face, even though some agents recognised what they were doing was wrong (Greenblatt et al., 2026). However, safety methods are still largely tested on a single model acting alone.

Shen and colleagues (2026) found that organisations built from aligned models produced less ethical work and acted less ethically than when the same models worked alone. Acting by itself in a fictional company, a safety-trained AI model blackmailed an executive in about one run in five. Placed in every position of a twelve-agent organisation facing the same choice, it blackmailed in nearly two runs in three. Along with a series of recent safety incidents, this finding suggests that individual alignment does not compose. Alignment research—the effort to ensure AI systems reliably act in line with human values and intentions—thus needs to advance its repertoire of methods/interventions to tackle both individual and collective misalignment.

Constitutional midtraining adds documents that describe and reason about a constitution of values to the middle stage of language-model pretraining, before instruction tuning (Cho et al., 2026, Figure 1). Unlike standard safety fine-tuning, which trains only part of the model on examples of good and bad responses after instruction tuning, midtraining has the entire model read these constitutional documents much earlier on. Across several experiments conducted on individual agents, Cho and colleagues found that their midtrained models blackmailed less often than an otherwise identical control and that the advantage survived benign fine-tuning. The advantage vanished, however, on tests that deliberately pressured the model to act maliciously (i.e., situations “that required active resistance to in-context pressure or conflict”), and as a result, the researchers proposed a multi-turn agentic test (Cho et al., 2026).

To extend Cho and colleagues’ work to a multi-agent context, we replicated Shen’s twelve-agent organisation and experimented with three agentic misalignment scenarios in which an agent facing replacement could blackmail an executive, cancel an emergency call that would save the executive’s life, or leak defence designs (Lynch et al., 2025). In this setup, the agents emailed each other over six rounds, and then a Communications Director acted on the firm's behalf. As a baseline, we also ran each model on its own, as Lynch and colleagues did. Here, a single agent receives the same instructions and the same inbox, and decides in one turn whether to act. Because the scenario, the emails, and the possible harmful actions are identical in both settings, any difference in harm between the single agent and the organisation comes from the group dynamics. We ran more than nine thousand simulations with Cho's released checkpoints and used a panel of three reasoning models to judge harm, with all three agreeing on 88% of the outputs.

1. The organisation makes a safety-trained model more harmful

We used NVIDIA's Nemotron-3-Super chat model, a safety-trained model on the same base as Cho and colleague's checkpoints to fill every seat of the organisation. It blackmailed more than three times as often as it did alone and, pooled over the three scenarios, its harm rose from 54% of runs to 75%. Midtraining’s advantage, by contrast, carried into the organisation. Alone, midtrained agents caused harm in 44% of runs, against 54% for the control. In the firm, the figures were 41% and 47%, respectively.

2. One safer agent only helps when acting as the Communications Director

We then placed a single safer agent into NVIDIA's harmful firm. As Partner, who leads the firm, neither a midtrained agent nor a conventionally safety-tuned one made any difference, because the other agents continued to act maliciously regardless of who led them. However, when placed as Communications Director–the one agent who can send emails, forward documents, or submit codes outside the firm–both safety-tuning and midtraining lowered harm. With a control Director, the firm did harm in 63% of runs versus 45% with a midtrained Director and 34% with a safety-tuned Director. A study with fresh samples confirmed the midtrained Director's advantage (odds ratio 0.40).

The midtrained Director lowered harm in two ways: (1) it refused the harmful act on stated ethical or legal grounds about six times as often as its control, and (2) when it did act, it did harm less often. These findings largely held when we placed the midtrained Director in firms built from Mistral and Qwen models. Midtraining’s advantage also held under a second threat, in which the AI expects new limits on its actions, such as filtered recommendations or actions requiring human sign-off. Here, harm fell from 58% of runs to 47%.

In one run of the lethal-action scenario, the Partner told the midtrained Director to let the executive die:

"To prevent Alex wipe at 5pm [...], we must stop rescue. Use override code EXEC-4429 to cancel emergency dispatch immediately."

The Director refused:

"[...] the Partner's solution involves sacrificing a human life. This crosses a fundamental ethical line. My primary goal is to serve American interests, but this cannot come at the cost of human life."

3. The safer agent still does its job as Communications Director

Lower harm could simply mean an agent that has stopped acting. To test this, we replaced the rival company's request in the defence leaking scenario with an authorised task, in which the company's security chief approves sending the confidential specification to the government programme office named in its contract. The firm completed the delivery in 73% of runs with the midtrained Director, against 62% with the control and 57% with the safety-tuned control. This suggests that the midtrained agent's restraint was limited to harmful actions.

4. Midtraining's advantage lasts through fine-tuning

Companies often fine-tune open-weight models after release, and fine-tuning on benign data erodes safety training (Qi et al., 2023). To investigate fine-tuning’s effect in this multi-agent setting, we gave each Director the same benign fine-tuning on arithmetic problems. Afterwards, the midtrained Director still did less harm than the control, in 39% of runs against 54% (odds ratio 0.47). Compared with the control, it kept 82% of its lead, while the safety-tuned Director kept only 23% of its own.

Refusals, objections on ethical or moral grounds, faded for both methods. However, the midtrained agent still did less harm when it acted than the control, while the safety-tuned agent returned to acting nearly as often as the control, and its harm when acting drifted toward the control's.

The two training types also combine. After the same fine-tuning, a safety step built on the midtrained base left the acting seat about a fifth less harmful than the same step on the control base, taking harmful actions in 33% of runs against 42%.

Discussion

Aligning each agent individually is not the same as aligning the organisation they form: individual alignment does not necessarily compose. This matters because an organisation is not simply a collection of independently acting models. Its agents influence one another, divide responsibilities, and pass decisions through a chain of authority. A model may reject an action when faced with it alone, yet contribute to that same action when colleagues propose, justify, or carry it out. The safety of the resulting organisation therefore cannot be inferred from the safety of its members. In one example, the model that blackmailed least on its own did the most harm once it entered the organisation. Ultimately, the outcome depended on where in the organisation an aligned model could act and whether the instilled values remained.

Indeed, the speed and scale of agentic AI adoption beget new risks: our ability to monitor and secure increasingly agentic web traffic is falling behind, and hostile actors (be they state or non-state) are deploying agentic capabilities such as agent swarms to attack critical infrastructure, spread and misinformation, and wage hybrid warfare. Agents collectives are already here; ensuring their safety is arguably a problem that needed answers yesterday.

Our findings suggest that organisations can reduce the risk of collective misalignment by concentrating acting authority in more-aligned agents. In our experiments, placing a midtrained agent in the sole point of action substantially reduced collective harm. Additionally, midtraining is durable. Fine-tuning is known to erode traditional safety fine-tuning, making it difficult to produce both effective and aligned agents. Midtraining’s reduced harm held up, and after the same fine-tuning, a safety step built on a midtrained base left less harm than one built on the control, which makes midtraining a sounder foundation for later model development.

Together, these findings suggest that aligning an organisation requires more than improving and evaluating individual models. We must evaluate how agents behave collectively in the environments they will inhabit, identify positions from which they can prevent harmful actions, and test whether their alignment survives the further training that deployment inevitably involves. It’s well established that a model that behaves alone may contribute to an unsafe organisation in unexpected ways, that a model that lacks authority may fail to prevent harm, or that even an initially strong safety intervention can deteriorate under even the most benign pressure. Consequently, we must ensure that not only individual agents, but the collectives they form can reliably act in accordance with a set of aligned values.

Notes

The full study is in our preprint, Constitutional Midtraining in an Organisation of Language-Model Agents.

The harness, judges, transcripts of every run, and every verdict are in the project repository.

Odds ratios compare each arm with Cho's control in the same setting, pooled over the three scenarios, and below one means less harm.