Spec-driven development with coding agents

The problem

An agent can produce a plausible thousand-line pull request in a few minutes. One person cannot review a thousand plausible lines in a few minutes, and “plausible” is precisely the failure mode — the code compiles, the tests it wrote pass, and a requirement nobody restated has quietly gone missing.

Across 12 days this repo took 146 commits — roughly a fifth of them authored by the coding agent — into a product with live buyer PII, an authorisation model that lives entirely in application code, and a deployed pilot. Reviewing that at the diff was never going to work. The review had to move to the layer where the judgment actually is: what the system is required to do.

Architecture & why

flowchart TD
  I["Idea / bug / field finding"] --> G1{"Gate 1 — owner promotes"}
  G1 --> P["Agent drafts an OpenSpec change<br/>proposal · design · spec deltas · tasks"]
  P --> G2{"Gate 2 — owner approves the SPEC<br/>~60 lines, not the diff"}
  G2 --> B["Agent implements on a claude/* branch"]
  B --> CI["CI: typecheck · lint · format · tests<br/>+ automated review pass"]
  B --> PV["Per-PR preview<br/><i>refuses to boot against prod project</i>"]
  CI --> G3{"Gate 3 — owner merges<br/>agents never push to main"}
  PV --> G3
  G3 --> AR["Archive: fold deltas into<br/>the canonical specs"]
  AR -.->|"amended with what<br/>ACTUALLY happened"| I

• Decision: specs are canonical and code is downstream. Each capability has a spec.md of RFC-2119 SHALL prose with WHEN/THEN scenarios; in-flight work exists only as delta files marked ## ADDED or ## MODIFIED, folded into the canonical set on archive. Alternative rejected: review the diff, like everyone else. Why: a spec delta is the smallest artifact that still contains the decision. Approving one is a real act of judgment; approving a diff of that size is a signature.

• Decision: tasks.md is a living ledger, not a checklist. Items are amended in place with what actually happened — the real field name where it differed from the plan, the bug found mid-implementation, the command that had to be re-run, tasks added on the fly. Known debt stays as unchecked follow-ups. Alternative rejected: ticking boxes. Why: a ticked box asserts that a plan was followed, which is unverifiable and usually false. An amended ledger is the only honest record of a plan meeting reality, and it’s what the next agent session reads to understand why the code doesn’t match the design.

• Decision: rules are path-scoped and load on file match. UI conventions load when you touch UI files; the OpenSpec workflow loads under openspec/**. Only the facts that apply before any file exists — the ones you need while creating a file, before any rule has triggered — stay in always-loaded project memory. Alternative rejected: one large always-on instruction file. Why: context that always loads is context that gets skimmed. Splitting by path means the rule arrives at the moment it’s actionable, and the always-on file stays short enough to still be read.

• Decision: write the near-misses into the rule file, not into a commit message. Alternative rejected: fix the bug and move on. Why: the agent re-reads the rule file every session and remembers nothing between them. A documented failure mode is a permanent fix; a remembered one lasts until the next context window. The file now carries an explicit “Known gaps” section, and it earns its place — see below.

• Decision: previews are structurally unable to reach production, not procedurally. A preview deployment refuses to initialise the Admin SDK if its configured Firebase project ID equals production’s — it throws before any read or write. The canonical origin falls back to the branch URL only when the deployment is genuinely a Vercel preview, and is never derived from inbound request headers. Every tool-sent email routes through a single deliverEmail() guard: outside production, a recipient not on the allowlist is suppressed — logged, and treated by callers as sent so funnels stay testable — and an unset allowlist suppresses everything. Alternative rejected: environment-variable discipline and a checklist. Why: the guarantee has to survive a tired person on a Friday. A new email module that called the provider directly would silently escape the guard, so the guard is the only door.

• Decision: three fixed human gates — backlog promotion, spec approval, merge — and agents never push to main. Risky surfaces ship behind server-evaluated feature flags, so merge ≠ release and the kill switch needs no revert PR. Alternative rejected: auto-merge on green CI. Why: green CI means the code does what the code says. The gates exist for the question CI cannot ask.

Evals / validation

The honest test of a system like this is whether it catches its own failures, and whether the record admits them.

• Two silent-drop archive bugs, caught before shipping, now permanent rules. Folding a delta into a canonical spec replaces a requirement by exact name, and the whole block with it. So a delta headed “Agent authentication” where the canonical said “Email-link sign-in” appends a second requirement and leaves the stale one live — and a delta restating 2 scenarios where the canonical had 5 silently drops the other 3. Both nearly shipped during the same archive. Neither would have failed a test, a typecheck, or a code review, because the artifact they corrupt is the one everything else is checked against.
• A collision the rule file refuses to let anyone forget. One in-flight change branched before another landed, so archiving it as-is would silently revert specific sentences and scenarios in three requirements. Rather than fix it invisibly, the rule file names each one that would be dropped, so whoever archives it has to union them by hand. Cost: a permanently uglier rule file. Benefit: the trap is disarmed for a person who wasn’t there.
• Real defects found and fixed as spec’d changes, not hotfixes. Closing a campaign hid QR self-signup, invites and report sending with no way back — found live in production, fixed by adding the counterpart operation to the campaign-lifecycle requirement. Fixing it surfaced a duplicate QR block, which became its own change against the “QR encodes only the join URL” requirement.
• A deterministic CI gate — typecheck, lint, format check, tests — required alongside one human approval before merge, with an automated review pass and an additional security-review pass on PRs touching the auth chain, PII paths, or CSP.
• Debt is carried visibly rather than quietly repaid. format:check failing on 28 pre-existing files was archived as accepted debt instead of being swept into an unrelated change. The absence of a spec-sync tool the archive skill assumes exists is recorded with the note that it “bites on every archive, not just once.” Eleven canonical specs are flagged as still carrying a placeholder purpose line.

Outcome

• 24 canonical capability specs and 13 changes shipped spec-first and archived, in the first 12 days, by one person working with agents — where the reviewable unit was a spec delta rather than a diff. The set has since grown to 31 specs and 33 archived changes; the figures here are the window this piece describes, not a ceiling.
• Two requirement-deleting bugs caught before they reached the source of truth, and converted into rules that apply to every future archive rather than lessons that applied once.
• No preview deployment can reach production data — enforced by a boot-time refusal and a single email guard, so it holds for code nobody has written yet.
• Three human gates, never bypassed, with the expensive one (spec approval) placed where a person’s judgment is worth the most and their reading time is worth the least. The spec-approval gate was later folded into the merge review once it turned out to be duplicating it, leaving the two structural gates the delivery pipeline runs on today.

Built solo. Roughly 28 of the 146 commits are agent-authored; the specs, the architectural decisions, the gates and the review are mine — which is the claim this piece is actually making. The system is not “the agent built it.” It is that agent output is only as trustworthy as the artifact you make it accountable to.