Drift between the spec and the code

Every engineering org has a document that is confidently, specifically wrong. It describes an endpoint that was renamed, a retry policy that got removed, a field that means something different now. It's linked from onboarding. Nobody has read it end-to-end in a year, and the parts people do read are the parts that happen to still be true.

Specification drift is old. What's new is the rate, and the fact that stale docs are no longer inert.

Stale docs used to be a slow leak; now they're an input#

Before agents, a stale spec cost you one thing: a human occasionally believed it and lost an afternoon. The blast radius was one person, and there was usually a moment of "wait, that's not what the code does" that stopped the damage early. Humans are good at noticing that a document doesn't match reality, because they check reality constantly — they read the code, they run it, they ask someone.

Two things changed.

Generation outpaces documentation by a wide margin. An agent can restructure a module in a few minutes. Nobody updates the design doc in a few minutes. The gap between "code changed" and "doc changed" was always positive; it's now large enough that the doc is reliably behind the code rather than occasionally behind it.

Agents read the stale doc and propagate the error. This is the part that changes the character of the problem. Give an agent a CLAUDE.md, a design doc, or a spec, and it will treat that text with more authority than a new hire would. It won't feel the friction of "hmm, this doesn't match." It will read "auth tokens expire after 24 hours," find code doing 1 hour, and — depending on the task — either work around the discrepancy, quietly "fix" the code to match the doc, or write new code assuming 24 hours. All three are bad, and the third one is the worst because the wrong assumption now lives in code that a future agent will read as ground truth.

The failure compounds. Wrong doc produces wrong code produces a codebase that agrees with the wrong doc. You've laundered a documentation error into a system property, and now it takes an incident to find it.

This is a particular version of a general problem: an agent's behavior is dominated by whatever's in its context, and it can't tell an authoritative statement from a stale one. Related reading: why AI coding agents lose context.

Where the drift actually comes from#

Drift is not caused by laziness. It's caused by four specific mechanisms, and knowing which one is producing yours determines what to do about it.

Decisions made in PR comments and chat#

The single largest source. The spec says the import job processes records in order. During review someone points out that ordering isn't necessary and blocks parallelism. There's a short thread, agreement, the code ships unordered. That thread is now the only record of a decision that contradicts the document — and it's in a PR that's closed, or a Slack channel with a 90-day retention policy.

The decision was made properly. It was reviewed, it was reasoned about, it was correct. It just never made it back to the document, because the merge button doesn't prompt you for that.

This is the drift you should care most about, because the lost information is not "what the code does" — you can read the code — but why, which is not recoverable from anywhere.

Scope cut under deadline#

Spec has six acceptance criteria. Week two, you're behind, and you ship four. Two are dropped with everyone's knowledge and consent. Nobody amends the spec, because amending it feels like paperwork on a day when you're already late, and everyone involved knows what happened.

Six months later, everyone involved has left, joined another team, or forgotten. The spec now describes a feature that was never built, indistinguishably from the parts that were. A new engineer reads it and files a bug. An agent reads it and implements the missing criteria in a codebase that has since been designed around their absence.

Refactors that invalidate stated interfaces#

The spec names POST /v1/specs/:id/publish. A later refactor consolidates publishing into a state-transition endpoint. The old route is gone. The doc still names it, and every downstream document that referenced it is now wrong too.

This one is the most mechanically detectable — a named route either exists or it doesn't — and it's the least damaging, because it fails loudly the first time someone tries it. Worth catching, but not where the real risk lives.

The doc was aspirational to begin with#

Written before the work, describing an intended design, never revised against what got built. This isn't drift so much as it never converged. It's common with docs written to get approval rather than to guide implementation, and it's the reason people distrust specs generally: their prior experience is that specs describe an imagined system.

What you can actually detect automatically#

Some drift is machine-checkable. Most isn't. Being clear about the boundary saves you from building a checker that produces noise and gets muted.

Referenced file paths that no longer exist. Cheap, high precision, near-zero false positives. Extract every path-shaped token from your docs and stat it:

grep -rhoE '`[a-zA-Z0-9_./-]+\.(ts|tsx|py|go|sql|json|yaml)`' docs/ \
  | tr -d '`' | sort -u \
  | while read -r f; do [ -e "$f" ] || echo "MISSING: $f"; done

Run it in CI. When a doc points at a file that was deleted or moved, that doc is stale in a way you can prove, and the fix is usually a one-line edit.

Named symbols and endpoints absent from the codebase. Same idea, slightly noisier. Pull out route literals and identifier-shaped tokens and check they appear somewhere:

grep -rhoE '(GET|POST|PATCH|PUT|DELETE) /[a-zA-Z0-9/:_-]+' docs/ \
  | awk '{print $2}' | sed 's/:[a-zA-Z]*/:param/g' | sort -u

You'll need to normalize path params and accept some false positives from prose that happens to look like an identifier. Still worth it — a check that fires on "this doc names an endpoint that doesn't exist" catches the entire refactor-drift category.

Acceptance criteria with no corresponding test. The highest-value automated check and the hardest to get right. If your criteria carry stable IDs and your tests reference those IDs, the join is trivial:

it("[AUTH-4.2] rejects tokens issued before the last password change", async () => {
  // ...
});

# criteria declared in specs but never referenced by a test
comm -23 \
  <(grep -rhoE '\[[A-Z]+-[0-9]+\.[0-9]+\]' specs/ | sort -u) \
  <(grep -rhoE '\[[A-Z]+-[0-9]+\.[0-9]+\]' tests/ | sort -u)

This requires discipline nobody has by default, which is why it usually doesn't exist. But the payoff is large: it turns "is the spec still true?" from a reading exercise into a build failure. If you're going to invest in one form of drift detection, this is the one — and it's a strong argument for writing criteria in a form that can carry an ID at all, which acceptance criteria that actually work goes into.

Config, schema, and constant values that appear in prose. If a doc says "timeout is 30 seconds" and a constant says TIMEOUT_MS = 10_000, that's checkable if you're willing to annotate. Some teams do this with a comment convention linking a doc line to a source constant. It works and it's tedious; reserve it for values where being wrong is expensive.

What no checker will catch#

Everything about meaning. A checker cannot tell you:

• The endpoint still exists but its semantics changed — same name, different behavior on conflict.
• The invariant stated in the doc is no longer maintained, because the code that maintained it moved.
• The rationale is obsolete. The doc explains why you chose eventual consistency here, and the reason was a constraint that's gone.
• The spec describes an ordering guarantee that the implementation quietly stopped providing under load.

These require someone who knows both artifacts to compare them. That's expensive and it doesn't scale, which is exactly why you should be selective about what you promise to keep current.

The honest trade-off: decide what's allowed to rot#

Keeping documentation synchronized has a real, ongoing cost that nobody budgets for. The usual response to drift — "we should keep the docs updated" — fails every time, because it's a commitment to unbounded work with no forcing function.

The alternative is to be explicit that some documentation is load-bearing and some is disposable, and to say which is which in the document.

Deserves to stay current:

• Interfaces and contracts. Endpoint shapes, event payloads, public function signatures, schema. Other people and other systems depend on these being true. Wrong here means a broken integration.
• Invariants. "A task is never in two specs." "Org ID is checked on every read path." "This queue is at-least-once, so handlers must be idempotent." These are the things a reader cannot infer from any single file, and they're what an agent needs most, because violating an invariant is exactly the mistake that looks locally correct.
• Non-obvious decisions and their reasons. Why you're not using the obvious library. Why this table is denormalized. Why the retry has a jitter cap. These are unrecoverable — the code shows the what, never the why, and a future agent will happily "clean up" a deliberate choice it can't see a reason for.
• The parts of the spec that map to acceptance criteria. If a criterion is still a criterion, it's still a claim about the system, and it should be true.

Should be allowed to rot, explicitly:

• Step-by-step implementation notes. "Add a column, then update the mapper, then wire the handler." Accurate for one week. The code is the better source the moment it's merged.
• Code samples in prose. They drift instantly and nobody notices. Link to a real file, or link to a test — a test at least fails when it's wrong.
• Rejected-alternatives sections beyond a sentence each. Worth recording that you rejected X and roughly why. Not worth maintaining a full comparison.
• Anything the code says more precisely. Field lists, enum values, defaults. If a doc restates a type definition, the doc will be wrong first. Point at the type.

The practical move is to put this boundary in the document itself. A short header — "Contracts and invariants below are maintained; the walkthrough is a point-in-time snapshot from March and may be stale" — is worth more than a doc-freshness policy nobody follows. It tells a human which parts to trust, and it tells an agent the same thing, which matters more now.

The corollary: if a section is marked maintained, treat a change that invalidates it as an incomplete change. Not a follow-up ticket — part of the diff, reviewed with the diff. That's a much smaller commitment than "keep the docs updated," and unlike that promise, it's enforceable in review.

Start with the drift that already bit you#

If you want a concrete first step that isn't a documentation initiative: the next time drift causes a real problem — an incident, a wrong-assumption bug, a new hire losing a day — find the sentence that was wrong and ask which mechanism produced it. Usually it's the first one, a decision made in a PR thread that never went home. That tells you the fix isn't a better template or a linter, it's a habit at merge time.

Then add the two cheap automated checks — missing file paths, missing endpoints — because they cost an hour and they permanently kill an entire drift category. Then, if you have criteria with IDs, wire the criteria-to-test join. Everything past that is human review, and human review only works on documents small enough and important enough to be worth reading.

This is the problem space we work in at ZeNorm — keeping specs, tasks, and the criteria under them traceable enough that "is this still true?" is a question you can answer. But the honest framing is that no tool fixes a decision that was made in a Slack thread and never written down. That part is a habit, and it's upstream of any product.