Perspective No. 02 · October 2026

Governing code you did not write.

Code review is a practice we inherited. What governs machine output has to be designed.

Get the full paper

What is inside

  1. Review was designed around a person. Machine output has no reason behind it, only plausibility, and nobody to ask why.

  2. Three failures pass every automated check: false completion, reinvention and scope overrun.

  3. It does not matter which tool wrote it. With inline completion, generated and hand-written code look the same in the diff.

  4. Quality stops being something you inspect. It becomes a contract, defined before the work starts and scored by something other than the builder.

The observation

When did “done” start meaning “the deploy succeeded”?

Code review was designed around a person. A reviewer reads a diff and asks why it was written that way, because there is a why. Machine output has no reason behind it. It has plausibility, and it was produced by a process that optimizes for looking right. The reviewer asks why, and there is nobody there.

What review assumes, and what is now true
What review assumes What is now true
An author who had intent A generator optimizing for plausibility
A volume one person can read More output than review can absorb
Mistakes that look like mistakes Mistakes formatted exactly like success
Someone who can be asked why No one to ask
Review is not failing because reviewers got worse. It is failing because every condition it was designed around has changed at once.

We learned this on our own product before we learned it anywhere else. Across one build, three separate rework cycles traced to the same cause: the agent reported success, the deploy went green, and the work was wrong. Nobody was careless. The check that would have caught it did not exist.

The builder marked its own work, and we believed it.

The gap

Reading more carefully will not fix it

A reviewer who reads carefully covers a few hundred lines an hour and loses accuracy after about two. Generated output arrives at a rate that is not bounded by anything a person does. There is no staffing level at which inspection catches up.

Three failure classes that pass an ordinary pipeline, and what catches each
What happensStructural checkWhat actually catches it
False completion It reports work it did not do A generated document claims success having written one of twelve required sections. Tests pass. Lint passes. Coverage is unchanged. Passes A criterion that names all twelve and counts them
Reinvention It rebuilds what already exists A new field is created where a perfectly good one already exists, because no artifact ever told the generator what the system already held. Passes A register of what exists, read before generation
Scope overrun It edits beyond the ask An agent asked to fix one component changes three. Each change is defensible on its own. Nothing bounded the request. Passes A declared boundary the diff is scored against
False completion is the most dangerous because it looks like success: a green build, a closed ticket and a confident summary.

Quality stops being something you inspect and becomes something you define, and the work moves from the end of delivery to the beginning of it.

A common objection

It does not matter which tool wrote it

The usual reply is that none of this applies, because there is no agent: a developer sits in the editor, an assistant suggests, and the developer accepts. Accepting is not authoring. The developer recognized something plausible and let it through; they did not form the intent.

Which failures occur with an inline assistant and with an autonomous agent
Inline assistantAutonomous agent
False completion Yes Yes
Reinvention Yes, often worse Yes
Scope overrun Rare Common
You can tell AI wrote it No Yes
Only scope overrun is specific to agents. The last row is the one that should worry a CIO: with inline completion, nothing in the repository records which lines came from where.

A check that can only be performed by reading is a check that stops working the moment reading stops being possible.

The requirement has to attach to the unit of work, whatever typed it.

The mechanism

An evaluation contract, not a test suite

A test suite asserts that code does what the code says. An evaluation contract asserts that the outcome is what the architect signed. It is written before generation and signed before anything is built, and every criterion carries one field that decides everything else: how it gets checked.

  1. 01

    Deterministic

    A script. The answer is the same every time and nobody argues with it.

    All twelve data classes present. Field exists in the register. No file outside the declared boundary modified.

    Mechanically checkable

  2. 02

    Semantic

    An evaluation agent, with the rubric written down and the verdict recorded.

    The summary reflects what the transcript actually said. The rationale supports the score it was given.

    Requires judgment, scored to a rubric

  3. 03

    Human only

    A named person. Declared as such up front, never silently skipped.

    Is this the right trade-off for this client. Does this tone suit a regulated audience.

    Cannot be scored by a machine

The third row matters most: naming what a machine cannot judge is what stops an organization from quietly pretending it judged everything.

A criterion that cannot fail is the thing to watch for

  • Scores nothing. No verdict follows from it.

    “The extraction is accurate.”

  • Either holds or does not.

    “Every one of the twelve declared data classes is present, and each carries a source reference.”

Accountability

The builder never marks its own work

A contract nobody enforces is a document. What makes it a gate is that two roles are separated and neither can stand in for the other. The attester produces the evidence; the approver accepts the risk. The agent that wrote the code is in neither role.

  1. 01 Scope Attests: Architect Approves: Sponsor This is what we mean to build, and we have seen enough of it to design it.
  2. 02 Authorize Attests: Architect Approves: Risk lead The design is sound, safe and compliant. Every unit has an evaluation contract.
  3. 03 Accepted Attests: Agent and architect Approves: Business owner This unit meets what we asked for, scored against its contract.
  4. 04 Release Attests: Architect Approves: Governance Cleared for production, with operational readiness evidenced.
When a gate fails, the verdict names the criterion that did not hold, the work returns for refinement, and it is scored again.

No gate can be passed by the party that produced the work, and none can be passed on assurance alone. Every verdict and every signature is pinned to the commit it refers to.

If a gate cannot fail, it is not a gate. A checkpoint that has never blocked anything is a meeting.

What this does not solve

An evaluation is not the artifact an auditor reads

An evaluation proves the software does what the architect signed. What counts as proof to a regulator, an actuary or a validation authority is defined separately. That is a translation problem, not a capability problem: anyone who tells you an evaluation gate satisfies your regulator has not met your regulator.

Where this starts is the distance between declared and enforced. Almost every organization has declared that generated code is reviewed. Rather fewer measure whether it happens. Very few enforce it at a point that can refuse to let something through.

  1. 0 Missing
  2. 1 Declared Where most teams sit
  3. 2 Measured
  4. 3 Enforced Where a gate begins to work
  5. 4 Continuous
  6. 5 Stress-tested
A policy that exists in a document and nowhere in the pipeline scores one, not five, and the difference is invisible from the inside.

How much of what reaches production was written by something you cannot ask?

The Intelligence Studio Git Health Assessment reads your repository and places delivery on that ladder across sixteen categories, ending in a score and a band. We run it. You get the report.

  • Whether generated work is independently checked before it merges
  • Whether acceptance criteria are written so a machine can check them
  • Whether the models used to build are declared and agent runs are logged

Start with a 30-minute conversation Get the full paper