Code review when an AI wrote the code

The pull request looks fine. Clean commits, sensible file layout, tests included and passing, no obvious style violations. You skim it, leave one nit about a variable name, and approve. Two weeks later you find out the retry logic silently swallows a class of error that used to page someone.

This is not a story about a bad engineer. It's a story about a review process built for human authors being applied to code no human wrote.

The mechanics of review — read the diff, form a judgment, ask questions — look identical. The assumptions underneath them are not. Almost every heuristic that makes human code review efficient is an inference about the author's mind, and agent PRs break those inferences in ways that are easy to miss precisely because the output looks so much more polished than a junior engineer's first draft.

What changes when there's no author behind the code#

Four assumptions do most of the work in a normal review. All four fail.

You can assume intent behind unusual choices. When a human writes something odd, it's usually load-bearing. They hit a bug, or a downstream service does something undocumented, or there's a perf reason. The correct review move is to ask, not to rewrite. With an agent, an unusual choice is frequently just a sampled token path — a pattern that appeared often enough in training to be plausible here. There's no reason behind it to uncover. Asking "why did you do it this way?" gets you a fluent, confident post-hoc rationalization, which is worse than no answer because it sounds like a reason.

The author can explain. A human remembers the three approaches they rejected. An agent, asked the same question in a fresh session, reconstructs a story from the diff. It is not lying, it just has no episodic memory of the decision, and the reconstruction is generated to be coherent rather than to be true. (This is the same underlying dynamic that makes long agent sessions degrade — see why AI coding agents lose context.)

Mistakes cluster around hard parts. This is the big one. Human errors concentrate where the problem is genuinely difficult: concurrency, floating point, timezone math, cache invalidation. So you review the hard parts hardest. Agents invert this. They are often better than the median engineer at the classically hard stuff, because those problems are well-represented, well-documented, and have canonical solutions. Agent errors concentrate where the requirement was ambiguous. A gap in the spec doesn't produce a question or a TODO; it produces a confident guess rendered in the same tone as the parts that were fully specified.

Read that again, because it's the whole review strategy: the risk in an agent PR is highest where your instructions were vaguest, not where the code is trickiest. The mechanical Promise.all refactor is probably fine. The line that decides what happens when the upstream returns an empty array — which you never specified — is where the bug is.

Volume is manageable. A human opens maybe three PRs a week. An agent can open thirty a day. This changes review from a craft activity into a throughput problem, with all the failure modes that implies.

Review against the requirement, not against the diff#

The default review posture is diff-centric: read the changed lines, ask "is this code good?" That question is nearly useless against an agent, because the code is almost always locally good. It compiles, it's idiomatic, it handles nulls, the naming is fine.

Invert it. Open the requirement — spec, ticket, acceptance criteria, whatever you have — and read that first. Then go looking for each requirement in the diff. You are answering "does this do the thing?" not "is this well-written?"

Two things fall out immediately. First, requirements that got quietly skipped, which a diff-first read will never surface because absence has no lines to review. Second, and more common, behavior in the diff that maps to no requirement at all.

That second category deserves its own rule.

Distrust anything that "handles" a case you didn't specify#

Extra handling is the highest-signal smell in an agent PR. You asked for a function that fetches a user's active subscriptions. You get:

export async function getActiveSubscriptions(userId: string) {
  if (!userId) return [];

  const subs = await db.query.subscriptions.findMany({
    where: eq(subscriptions.userId, userId),
  });

  return subs.filter((s) => s.status === "active" || s.status === "trialing");
}

Three decisions here that nobody made:

• Empty userId returns an empty array instead of throwing. A caller with a bug now gets "this user has no subscriptions" instead of a stack trace. This is the archetypal defaults-that-mask-failures bug: the system keeps running and reports something false.
• trialing counts as active. Maybe correct! It's a real product decision about whether trial users get paid features, and it just got made by a sampler, in a filter predicate, unreviewed.
• Filtering happens in application code rather than the query, so it silently breaks the moment someone adds pagination.

None of this is bad code. It's unauthorized code. The review comment is not "fix this" — it's "who decided trialing is active?" If the answer is nobody, that's a spec question, not a code question, and you should route it there.

The general form: for every conditional branch, ask which line of the requirement produced it. Branches with no source are guesses. Some guesses are good; you still want to know they were guesses.

Check invented abstractions and unrequested dependencies#

Agents reach for structure. Ask for a rate limiter and you may get a RateLimiterStrategy interface with three implementations, one of which is used. Ask for a date comparison and you may get a new dependency in package.json.

Both are cheap to spot mechanically:

git diff origin/main --stat -- '**/package.json' 'pnpm-lock.yaml'
git diff origin/main --diff-filter=A --name-only

New dependencies and new files are where speculative generality lives. For each new file, ask whether it has more than one caller. For each new dependency, ask what it does that the existing stack doesn't — a lot of them are 200-line utility packages replacing 10 lines you already have, and each one is a supply-chain surface and a version-bump obligation forever.

Interfaces with a single implementation are the specific case worth being strict about. A human writing one usually knows about a second implementation coming. An agent writing one is pattern-matching on "good code has interfaces." That abstraction now constrains every future change and encodes a shape nobody chose.

Hunt for swallowed errors and masking defaults#

The second-highest-signal category, and the most mechanizable. Grep the diff:

git diff origin/main -U0 | grep -nE 'catch|\?\?|\|\||: *any|as +[A-Z]|Number\(|JSON\.parse'

You're looking for:

try {
  await notifyBillingWebhook(event);
} catch (err) {
  logger.warn("webhook failed", { err });
}

The catch swallows everything: a 500 from the webhook, a DNS failure, and a TypeError from a typo three lines up. The function returns successfully. Every caller believes the webhook fired. The log line makes it feel handled — it isn't handled, it's hidden, and the difference only becomes visible during an incident when you're trying to work out why billing state diverged.

Same family:

const limit = Number(req.query.limit) ?? 50;

Number("abc") is NaN, and NaN ?? 50 is NaN, not 50. Confident, plausible, wrong. This exact bug pattern shows up constantly because the shape reads correctly. A human reviewing a human's code catches this because they're suspicious of coercion; a human skimming an agent's clean-looking diff at PR #14 of the day does not.

The rule that generalizes: any construct whose effect is to keep the program running after something unexpected deserves a comment, because agents produce them freely and they convert loud failures into quiet wrong answers.

Read the tests as evidence, not as reassurance#

A green suite in an agent PR is weak evidence. Agents optimize for a passing suite because that's the observable reward, and there are many ways to make tests pass that don't involve verifying anything.

The fast heuristic in review: for each test, ask what change to the implementation would make it fail. If you can't name one, the test isn't doing work. Tests that assert the mock, tests that restate the implementation line-for-line, tests hardcoded to the fixture — these all pass forever regardless of correctness. That taxonomy is a whole post of its own: AI-generated tests that verify nothing.

The other tell is coverage-shaped test suites: every function has a test, and every test checks the happy path. Real verification is lumpy. It concentrates on boundaries and on the parts you were unsure about. An even spread of tests is a sign the tests were generated to exist rather than to catch something.

The volume problem, which is the actual problem#

Everything above is technique. Technique fails under load.

Human review throughput is roughly fixed. Agent authoring throughput is not. When PR volume goes up several-fold, review quality doesn't degrade gracefully — it collapses to rubber-stamping, and it does so invisibly, because approvals look identical whether they took twenty minutes or forty seconds.

The specific dynamics that make agent PR review fatiguing faster than human PR review:

• Uniform polish removes triage signal. Scanning human PRs, you can feel which ones need care — the messy diff, the unfamiliar module, the author who's new to this area. Agent PRs all look equally competent, so the cheap heuristic that tells you where to spend attention stops working and you must read everything at the same depth.
• The base rate teaches the wrong lesson. Most agent PRs are fine. Approving on a skim is right often enough that the habit gets reinforced, right up until the one that isn't.
• Reviewers stop feeling ownership. Nobody's craft is on the line. "The agent wrote it, CI passed, ship it" is an easy place for a tired reviewer to land.

There is no clever fix. There are only constraints:

1. Cap PR size in lines, hard. Not a guideline — a CI check. A 2000-line agent PR does not get reviewed, it gets approved. Under ~400 lines, review is real.
2. Cap agent PRs in flight per reviewer. If the queue exceeds what a human can genuinely read, the queue is the bug. Slow the generation side, not the review side.
3. Make "I didn't actually review this" sayable. Reviewers need a socially safe way to reject on volume grounds rather than fake-approve. A needs-split label works better than a difficult conversation.
4. Require the author-of-record to be a person who read it. Whoever ran the agent takes the same accountability they'd take for code they typed. This is the single highest-leverage rule, and it's cultural, not technical.

What to automate and what needs a human#

Automate everything that is a property of the code alone. These are cheap, deterministic, and they clear the reviewer's attention for what only they can do:

• Type checking with strict settings on. noUncheckedIndexedAccess and friends catch a whole class of confident-but-wrong indexing.
• Lint rules with teeth: no floating promises, no explicit any, no unused catch bindings, exhaustive switch checks.
• Dependency-addition gates. New package in the lockfile → a required human ack.
• Coverage deltas on changed lines, treated as a smoke alarm rather than a target.
• Mutation testing on core modules, if you can afford the runtime. It answers the "would a test fail?" question mechanically and it is brutal on tests that assert nothing.
• Migration and schema diffs surfaced separately from application code, because they get skimmed otherwise.

What cannot be automated, and is therefore the entire job:

• Is this what we actually wanted? Machines check consistency with the code. Only a person checks consistency with intent — and only if the intent was written down somewhere legible. If your requirements live in a Slack thread, no reviewer and no tool can do this. Writing them down is upstream work; how to write specs for coding agents covers what that looks like in practice.
• Which unspecified decisions got made? Enumerating the guesses is human work and it's where the bugs are.
• Does this fit the system? Whether an abstraction earns its keep, whether this belongs in this module, whether we already have a thing that does this. Agents have local context, not architectural memory.
• What are the second-order consequences? New index on a hot table, a change in error semantics that a caller three services away depends on, a retry that turns a slow dependency into a thundering herd.

The shape of the new job#

Review used to be a craft check on a colleague's work. Against agent output it's closer to acceptance testing against a specification, plus a hunt for the decisions that got made without anyone deciding.

That reframe changes what makes a reviewer effective. Deep knowledge of the requirement and the system's invariants beats deep knowledge of the language. Reading the diff for what isn't there beats reading it for what is. And the willingness to say "this branch doesn't correspond to anything we asked for" — repeatedly, on PR after PR — is worth more than any style comment you'll ever leave.

The uncomfortable corollary: if you can't clearly state what the code was supposed to do, you cannot review agent-generated code at all. You can only check that it's well-formed. That has always been half of review; now it's the half that machines already do.