Skip to content

The deterministic layer

Building with agents

An agent produces work faster than you can read it. That’s the point, and the problem. The only way to keep pace without gambling is a layer that doesn’t reason at all: checks that either pass or fail, guardrails that hold regardless of what the model believes, and a human gate at the end. Prove it works = plan + test + evidence, not a green checkmark. And a human owns the merge.

Verification covers the habit: demand evidence, not assertion. This page is about building that habit into the system, so it holds even when nobody is watching the session.

When you drive a tool by hand, you can eyeball each change. An agent working through forty files unattended removes that option, and the base rate says you need one. One large-scale study of AI-assisted commits across 6,299 repositories found that more than 15% of commits from every AI assistant studied introduced at least one issue, and 62.6% of the repositories were affected. Every tool, at meaningful rates.

So the gate can’t be “the agent seemed confident” or “the summary looked right.” A gate is a condition that either holds or doesn’t, checked by something that can’t be persuaded. Learn to hear the difference:

The agent saysEvidence says
”Done, everything works.”The test output, pasted, with the new test named in it
”Behaviour is unchanged.”A passing suite that actually covers that behaviour
”Fixed the bug.”A test that failed before the fix and passes after
”Reviewed and looks good.”A second, fresh reviewer’s findings on the diff

The left column is a green checkmark. The right column is what you merge on.

The strongest version of this is test-driven: the check exists before the code does. Have the agent state its plan and write the failing test first, then implement until the test passes. Now “done” has a definition the agent can read and you can trust: the loop closes on evidence instead of on the model’s self-assessment.

The same move works for judgment-shaped tasks, where there’s no test suite. Build a small eval set: real inputs, expected outputs, a scoring rule. An agent that classifies support tickets gets graded against fifty tickets you labelled by hand, before it ever touches a live one. If you can’t say what passing looks like, you’re not delegating; you’re gambling.

A check the agent is merely asked to run is advisory: it can be skipped, forgotten, or rationalized away under pressure. Wire it in as code instead. Claude Code, for example, lets a Stop hook block the agent from ending its turn until a command exits clean: the tests run, and if they fail, the failure is fed back and the turn continues. No judgment involved, no way to argue with it. The agent literally cannot declare victory over a red build.

This is the general pattern behind what enforces what: determinism belongs in hooks and CI, not in instructions the model might weigh against its eagerness to finish.

The strongest guardrail is a capability the agent doesn’t have, not a rule it follows. An agent that never holds deploy credentials cannot deploy, no matter how confused, misled, or prompt-injected it gets. Read-only database access, no production tokens, a sandboxed shell, an allowlist of commands: each one removes a failure mode outright instead of asking a model to avoid it.

Design for this deliberately. For every capability you grant, ask what the worst honest mistake looks like, then check whether the agent needs the capability at all. You don’t have to trust an agent not to do what it cannot do.

The model that wrote the change is the worst-placed model to review it. Its context is full of its own plan, its own assumptions, its own rationalizations. It will grade generously, for the same reason authors make poor proofreaders. Anthropic’s own guidance for Claude Code recommends the fix: have a fresh model instance review the work, one that sees only the diff and the requirement, none of the session history.

Every gate above narrows what reaches you. None of them replaces you. The MCP specification is explicit on this point: there should always be a human in the loop with the ability to deny a tool invocation. The protocol itself assumes a person who can say no.

For code, that moment is the merge. Review the diff as if a colleague had handwritten it, because from the moment you merge, that’s exactly what it is: your change, your name, your pager. Accountability doesn’t compile. The deterministic layer exists so that what reaches this gate is already tested, already second-reviewed, already incapable of the worst mistakes, and so that your judgment is spent where only judgment works.