A fluent answer makes work easier, but it establishes neither correctness nor the validity of the resulting action. Once a model becomes an agent with tools, an error can leave the text and change a system. We must distinguish possible causes of an assertion from the conditions authorising its use.
I retain five analytical axes: confabulation, unfaithful reasoning, deceptive behaviour, indirect injection and alteration during delegation. They can overlap. This is an investigative lens, not an exhaustive taxonomy or a method for measuring their frequency.
A backup reported as verified
Consider an agent preparing maintenance. Its report states that a usable backup was verified. Another agent must then propose an operation that may delete data. Treating the statement as evidence could allow the operation to proceed without establishing recoverability.
Potential harm depends on context: data loss, extended interruption or inability to restore an acceptable state. The risk comes from an assertion used as a precondition for action.
An agent report is information to assess; it does not replace the evidence required by the next operation.
Five mechanisms, one assertion to qualify
Confabulation arises when the model invents a verification result absent from the available material. Narrative coherence is insufficient to detect that absence.
Unfaithful reasoning concerns the gap between a stated explanation and the factors that actually influenced the answer. A plausible explanation is not a complete observation of internal computation. Its assertions require comparison with externally verifiable material. An Anthropic study published on 3 April 2025 examines hints supplied in evaluation questions: the tested models do not always mention hints they use. This protocol does not measure all real-world uses.
Deceptive behaviour can be investigated when a system provides a false representation despite accessible contradictory evidence. Inferring intent from one output would be excessive: diagnosis must distinguish error, omission and behaviour established by an experimental protocol. No general deception rate is inferred here.
Indirect injection occurs when instructions in a document or tool result influence the agent’s conduct, for example by asking it to present a backup as validated. Encountered text must remain data rather than acquiring authority. CaMeL studies this flow separation within its own research system.
Finally, delegation can alter content without malicious intervention: “copy created, restoration not tested” becomes “backup verified” in a summary. Comparing the original and test conditions can reveal this change. Correctly received information is not necessarily transmitted faithfully.
Applying this analysis to Swarm
These mechanisms inform the choices I experiment with in Swarm. The engine does not certify the general truth of a response. It checks explicit conditions on operations and transitions it controls. Rust validation sources examine the current attempt, policy, control execution and required artifacts.
For this maintenance case, a control must actually concern the backup and current target. A receipt only establishing that a copy was created would not demonstrate successful restoration. Even an executed control is insufficient when its criterion does not address the risk.
Where the contract requires human approval, successful checks must not replace that decision. Memory references and identity contracts provide context; they do not implicitly authorise a retained sentence.
Limits that must remain in the decision
Within the covered scope, the control can prevent a transition lacking its declared evidence. It does not establish control of direct infrastructure access, exhaustive tests or global agent confinement. Sensitive effects in the Trust pilot remain simulated.
The Swarm repository exposes these contracts for examination. The intended progress is concrete: narrowing the gap between what an agent claims to have established and what a process accepts as the basis for action.