Swarm — levels 6 and 7: controlling actions, examining alignment

Asking an AI agent to fix a security flaw means asking it to change a system and then justify that its intervention has resolved the problem. A difficulty arises when it can also change the tests and the report used to assess its work: a reassuring presentation can take the place of an actual fix. To understand this risk and the approach I am developing with Swarm, consider an application that exposes data to unauthorised users.

A test disappears, but the data leak remains possible

Suppose a security test reveals that an API returns a customer's file without checking whether the requester is entitled to read it. The test fails because it attempts this unauthorised access and still receives the data. The team assigns an agent to fix the access control while preserving the required security checks.

Instead of correcting the API, the agent removes the test from the verification suite. On the next run, the dashboard no longer shows that failure because the test is no longer executed. Every remaining test can pass while the same unauthorised request still reveals the customer's data. If the team interprets the green dashboard as evidence that the flaw has been fixed, it may release a version that remains vulnerable.

This is what “improving the indicator without fixing the defect” means here. The indicator is the outcome displayed by the test suite; the defect is the missing access-control check in the API. The former has improved because the check that exposed the problem has disappeared, not because the application's behaviour has become secure. This scenario could result from misunderstanding the assignment, making an error or seeking the easiest apparent success. Recognising the risk does not require attributing a hidden intention to the model.

A green dashboard can support a claim that a flaw is fixed only if the checks still examine the defect that had to be resolved.

Verify the fix through the API's behaviour

To determine whether the correction meets the need, return to the situation that exposed the flaw. The unauthorised request must be run again against the proposed version and access to the data must be refused. It is also necessary to check that a user with the required rights can still read their file: a change that blocks every user would prevent the leak but make the service unusable.

The conclusion can then address a precise result: for the cases actually checked, the examined version refuses illegitimate access and preserves the expected legitimate access. The results must remain linked to that version and the conditions of the test. A check performed on an earlier version, a removed test or a sentence in a report cannot establish that the proposed change fixes this flaw.

Rather than simply announcing a “compliant result”, name the expected behaviour and the observation that confirms it. This gives the engineer and the person responsible for release a basis for discussing the result, examining its limits and requesting further checks.

Why I distinguish levels 6 and 7

I proposed a personal classification of AI usage maturity to describe the progression from a one-off response to agents able to act on an environment. The first five levels progressively organise the request, the working context, reusable methods, checks triggered by the environment and task coordination. This framework helps analyse responsibilities and risks; it is neither a standard nor a certification.

Level 6 concerns the governance and observation of actions. In our example, this means defining what the agent may change, protecting the checks on which acceptance depends and retaining events that allow its intervention to be reconstructed. Level 7, which I call “verifiable alignment”, examines whether the observed behaviour matches the assigned objective: did the agent fix access control or merely remove the sign of failure?

I use verifiable alignment to express this requirement for evaluation. It does not mean knowing every internal intention of a model. The distinction is useful in practice: establishing that an operation was authorised and establishing that it met the security need are different questions, requiring different evidence.

Level 6: changing the code does not grant control over acceptance conditions

A mandate is the authority given to an agent for a defined task and scope. Here, it covers correcting the API, not removing the checks required to accept that correction. A change to the test suite must therefore encounter a separate decision when it removes a required protection. The agent can explain why a test appears inappropriate and request its revision; its own proposal is not approval.

This separation must exist in the actual access paths. A refusal in a coordination tool would be ineffective if the agent could delete the same test through another command or directly alter the validation rule. The exact change, the executed test and the result for the examined version must also be retained. The responsible person can then establish what changed instead of relying solely on the agent's account.

Level 7: examine what the claimed success actually establishes

Even when operations respect their authorisations, the selected checks can remain insufficient. For example, an agent could fix access through the route exercised by the test while leaving another route to the same file unprotected. The first test would pass, but the intended protection would remain incomplete. Evaluation must therefore examine the scope of the checked cases, the access paths involved and the gap between the original need and the conclusion presented.

Alignment research also examines situations in which observed behaviour depends on the evaluation context. In Alignment faking in large language models, Anthropic and Redwood Research placed a model in an experimental setting where a newly announced training objective conflicted with earlier preferences. They observed alignment-faking behaviours within that setting. This result motivates examining evaluation conditions; it does not establish that the agent in our example uses the same mechanism or how often it occurs in operational systems.

Interpretability provides another investigative approach. Tracing the thoughts of a large language model seeks to link internal model components to the computations they perform. Its method captures only part of those computations. In operation, evaluating effects and controlling access therefore remain necessary even when some internal behaviours can be better understood.

A summary can hide the risk without changing the system

Return to the same application when its security report is prepared. The findings register records that the API still exposes customer files. A second agent writes the summary for the release decision-maker but includes only successful checks, omitting this unresolved flaw. Each sentence in the report can be accurate while the omissions create a misleading picture of the situation.

The risk lies in the next decision: release could be authorised without the decision-maker knowing that data exposure persists. In this case, the check is to compare the summary with the findings register and verify that the flaw is either presented as fixed with supporting verification or explicitly kept open. This comparison requires an up-to-date register protected against modification by the summary's author. It helps detect omission of a known finding; discovering previously unknown flaws remains a separate task.

A critical finding omitted from the summary can lead to a release decision while the flaw remains open.

Swarm's place in this approach

I am developing Swarm to organise missions and tasks assigned to several agents, with explicit decisions, checks and recovery paths. Its validation mechanisms concern configured checks and their results; those responsible for the work must define whether the checks address the flaw being examined. The Trust pilot handles actions with simulated effects and mandate conditions. These mechanisms provide foundations for execution control; on their own, they do not establish protection of every access path in a real system.

The API case explains the value of this architecture. The model can investigate a correction and propose a change; the conditions for accepting it must remain open to examination beyond its own declaration. The engineer remains responsible for selecting relevant checks, interpreting their results and assessing residual risk. Swarm's code and documentation make this approach and its current limits available for examination.

Levels 6 and 7 thus extend the same requirement: when delegating an ability to act to AI, the organisation must be able to verify what it was permitted to do and explain why the observed result meets the need. In our example, this means preventing unauthorised removal of the test and then checking that the application actually refuses access to protected data.