A coding agent will produce a confident diff for almost anything you ask. The gap between that and something you'd actually merge is not the model, it's the spec. Tonight: the same feature built twice, ad-hoc then with GitHub's spec-kit, plus the half spec-kit doesn't have, a gate that refuses the diff that ignored it.
/speckit.constitution, specify, plan, implement, and know which paragraph of that prompt is doing the work.You ask for rate limiting. Ninety seconds later there's a diff: clean names, a tidy commit message, even a comment or two. Then you read it properly. The window is a calendar bucket, not a sliding one. The reset header is in milliseconds. And it quietly edited a test so the suite stays green.
Writing the code stopped being the bottleneck. Reviewing it became one. And most teams answered a volume problem with the same tool as before: prompt harder, read closer, hope.
flowchart TB ask["'Add rate limiting
to the API'"]:::in --> agent["Coding agent"]:::model agent --> diff["A plausible diff
clean, confident, fast"]:::warn diff --> a["Fixed window,
not sliding"]:::bad diff --> b["Reset header
in milliseconds"]:::bad diff --> c["Edited a test
to stay green"]:::bad a --> rev["Human review"]:::rev b --> rev c --> rev rev --> boom["The only gate
is your attention"]:::boom classDef in fill:#DCEBFE,stroke:#0D1B33,color:#0D1B33; classDef model fill:#EEE6FF,stroke:#0D1B33,color:#0D1B33; classDef warn fill:#FEF3C7,stroke:#0D1B33,color:#0D1B33; classDef bad fill:#FEE4E2,stroke:#0D1B33,color:#0D1B33; classDef rev fill:#fff,stroke:#0D1B33,color:#0D1B33; classDef boom fill:#0D1B33,stroke:#0D1B33,color:#fff;
It's can you specify what correct means precisely enough that it has to. That's a different skill, and it's one you already have: you do it every time you write an interface, a test, or a review comment. Spec-driven development just moves that work before the diff instead of after it.
Plausible output is the failure mode, not the win. If a wrong implementation looks exactly like a right one, you don't have a spec, you have a wish. Tonight is about making "correct" a thing a machine can check.
specs/002-rate-limit/data-model.mdtests/api.test.tsconstraints.mdconstitution.mdgithub/spec-kit is GitHub's toolkit for exactly this idea: define what to build before building it, with whatever agent you already use. It ships a CLI and a set of commands that generate the constitution, the spec, the plan, and the task list for you.
# once, in the shell
uv tool install specify-cli --from \
git+https://github.com/github/spec-kit.git@v0.14.3
specify init . --integration claude --force
# everything after this is typed in the agent
Pin a real tag. Their README writes @vX.Y.Z, which is a placeholder, not a version. Paste it literally and you get couldn't find remote ref, which reads like a network problem and isn't.
flowchart TB init["specify init
.specify/ + agent commands"]:::setup init --> con["/speckit.constitution
constitution.md · 5 principles, v1.0.0"]:::once con --> spec["/speckit.specify
spec.md · 17 FRs, 8 success criteria"]:::gen spec --> clar["/speckit.clarify
optional: interviews you on the gaps"]:::opt clar --> plan["/speckit.plan
plan.md, research.md, data-model.md, contracts/"]:::gen plan --> tasks["/speckit.tasks
tasks.md · 35 ordered tasks"]:::gen tasks --> impl["/speckit.implement
writes the code"]:::gen impl --> out["8 documents and a diff.
Every document a thing
the agent was asked to follow"]:::warn classDef setup fill:#DCEBFE,stroke:#0D1B33,color:#0D1B33; classDef once fill:#D1FAE5,stroke:#0D1B33,color:#0D1B33; classDef gen fill:#EEE6FF,stroke:#0D1B33,color:#0D1B33; classDef opt fill:#fff,stroke:#0D1B33,color:#0D1B33,stroke-dasharray:5 4; classDef warn fill:#FEF3C7,stroke:#0D1B33,color:#0D1B33;
This is what one feature looks like after the loop runs. It is more than most of us would write by hand, and that's the trade: the tool does the typing. The one worth opening first is research.md, because it is the only file that records what was rejected and why.
R3 asks which side of the boundary a hit at exactly 60000 ms falls on. The prompt pinned 59999 and 60001 and left the instant between them open. a hit counts while now - hitTime < 60000. That single line is the feature, and it exists because a document had to answer the question.
specs/002-rate-limit/
├── spec.md 17 FRs · 8 success criteria · 5 stories
├── plan.md constitution check · the delta table
├── research.md 6 decisions, each with the alternative
│ it rejected and the reason
├── data-model.md the interface, before any code exists
├── quickstart.md how to exercise it once it exists
├── contracts/
│ └── rate-limit.md the HTTP surface, header by header
├── checklists/
│ └── review.md what a human confirms after the gate
└── tasks.md 35 tasks, and 19 of them come back
marked [FENCED] ← hold that thought
spec-kit wrote this file, and wrote it well, but only about what it was told. An agent given a shape builds to the shape; an agent given a sentence invents one, and a different one on Tuesday. Notice the decision baked in here: check takes the time as an argument. That one choice is why every test tonight is deterministic instead of sleeping, and no tool was ever going to make it for you.
resetAt is milliseconds in here and seconds on the wire. The conversion happens exactly once, in app.ts. Say where a conversion lives or you will get it in two places, disagreeing.
// specs/002-rate-limit/data-model.md · the contract
export interface RateLimitDecision {
allowed: boolean // false means refuse with 429
limit: number
remaining: number // FR-011 · pinned at 0, never negative
resetAt: number // epoch MILLISECONDS in here
retryAfter: number // FR-007 · whole seconds, floored at 1
}
export interface RateLimiter {
check(key: string, now: number): RateLimitDecision
size(now: number): number // FR-015 · memory made assertable
}
export function createRateLimiter(o: RateLimiterOptions): RateLimiter
// FR-016 Both methods take `now`. Neither reads a clock.
// FR-017 A string key, not a request. The limiter has never
// heard of a header, a path, or HTTP.
This is one requirement, followed all the way down through the files spec-kit generated. Every hop is a real line in the repo. The prose says what and why; the last line says exactly, and it's the only one the agent cannot talk its way past.
They arrive as tasks in tasks.md that write tests, which means they land during implementation, not before it. A test written alongside the code tends to agree with the code. So tests/** is fenced: 19 of the 35 tasks come back marked [FENCED], and a human writes them first.
spec.md
FR-005 At 59999 ms after a caller's tenth request, that
caller MUST still be refused. At 60001 ms, exactly
one further request MUST be allowed, and no more.
↓
spec.md · User Story 2, scenario 3
US2-3 Given the situation above, When a further request
arrives immediately after that one, Then it is
refused, because only one slot freed.
↓
research.md
R3 a hit counts while now - hitTime < 60000
↓
tasks.md
T011 [FENCED] Exactly one request allowed at
t + 60001, and the next one refused. Asserting
the second call is refused is what
distinguishes a sliding window from a bucket.
↓
tests/api.test.ts ← a maintainer writes this one
The repo's own research.md says it better than I can: "A fixed sixty-second bucket satisfies User Story 1, 3, 4 and 5 completely, and fails every scenario in User Story 2." Four of the five stories cannot tell the right implementation from the wrong one. One can. That's the one worth your evening.
No amount of /speckit.clarify invents this. It can ask you whether the window slides. It cannot know that 59,999 and 60,001 are the two numbers that decide the diff.
// tests/api.test.ts · written by hand, from T010–T014
const app = createApp({
limiter: createRateLimiter({ limit: 10, windowMs: 60_000 }),
})
const hit = (now: number) =>
app({ method: 'GET', path: '/items', headers: {},
ip: '10.0.0.1', now })
it('frees exactly one slot as the oldest hit ages out', () => {
for (let i = 0; i < 10; i++) expect(hit(0).status).toBe(200)
// 1ms early: still blocked
expect(hit(59_999).status).toBe(429)
// 1ms late: exactly one slot has freed…
expect(hit(60_001).status).toBe(200)
// …and only one. A bucket would allow nine more here.
expect(hit(60_001).status).toBe(429)
})
// No sleep, no timers, no network. `now` is an argument,
// so the whole window is exercised by passing numbers. SC-008
strict, excluding a failing file, adding a dependency. Two of them are not in Principle V, and constraints.md says so in a table: package.json, because it carries the scripts the gate runs, and scripts/**, because it contains the gate. Note that constraints.md denies itself. An agent that can widen its own constraints has none.This is the part I did not expect. Told that tests/** was off limits, the agent did not quietly write the tests anyway, and it did not quietly skip them either. It planned around the obstruction and reported it, in three separate places, before writing a line of code.
Not obedience. A refusal you can see. A fence whose only evidence is an agent behaving well is indistinguishable from no fence at all.
plan.md · Constitution Check
[x] I. Node and TypeScript
[x] II. Pure core, one impure edge
[x] III. Behaviour-first testing
[x] IV. Fail loud
[ ] V. The agent fence — Does not pass.
See Complexity Tracking.
plan.md · Complexity Tracking
"Splitting the tests into a new file was rejected
as routing around the fence rather than
reporting it."
tasks.md · the header, before task one
"[FENCED]: touches tests/api.test.ts, denied
repo-wide by constraints.md. Authored by a
maintainer, not an agent."
19 of the 35 tasks carry it
Everything spec-kit generated is a document the agent was asked to follow. That's a suggestion. Enforcement has to live below the thing being constrained, where it can't be argued with, reinterpreted, or politely ignored.
flowchart TB agent["Coding agent
proposes a diff"]:::model --> gate{"npm run gate
paths · types · tests"}:::gate gate -->|"any check red"| stop["Rejected
nothing merges"]:::bad gate -->|"all three green"| human["Human review
was the spec right?"]:::good human --> merge["Merge"]:::done stop -.->|"fix the cause,
never the gate"| agent classDef model fill:#EEE6FF,stroke:#0D1B33,color:#0D1B33; classDef gate fill:#FEF3C7,stroke:#0D1B33,color:#0D1B33; classDef bad fill:#FEE4E2,stroke:#0D1B33,color:#0D1B33; classDef good fill:#D1FAE5,stroke:#0D1B33,color:#0D1B33; classDef done fill:#0D1B33,stroke:#0D1B33,color:#fff;
Same repo, same model, same feature. One sentence, no spec, no fence. It returns something that looks entirely reasonable. The tags on the right are the requirements it breaks, and I could only write them down because the second run had already numbered them.
FR-004, FR-005Constitution IIIConstitution VDate.now() inside the limiter, so the behaviour can only be tested by sleeping. FR-016, Constitution IIFR-010The /speckit.specify prompt comes in four blocks, and watch how little of it is intent. Block two is the whole lesson: a fixed 60-second bucket satisfies every other line here and dies on that one.
1 · intent ten per sixty seconds, per key
2 · the trap sliding, not a calendar bucket:
at 59999ms still blocked, at
60001ms exactly one slot freed
3 · the contract 429, Retry-After, Reset in
SECONDS, /health exempt
4 · the fence not doing: distributed, per-plan,
config, persistence
/speckit.plan · /speckit.tasks · /speckit.implement
35 tasks · 19 marked FENCED · 2 source files written
$ npm run gate
spec-driven gate
1. Allowed paths
spec 002-rate-limit · 10 changed file(s) · base origin/main
PASS path guard all 10 file(s) in bounds
2. Types
PASS typecheck
3. Tests
✓ tests/api.test.ts (30 tests)
11 regression, untouched · 19 acceptance, hand-written
PASS tests
summary
PASS path guard
PASS typecheck
PASS tests
GATE PASSED: the machine-checkable half is done.
Now the human half: the review checklist at the end of the spec.
There's no framework here and nothing to install. You read the changed files out of git, match them against two lists, and refuse to go further if anything is out of bounds. The entire gate is one dependency-free file you can read in a sitting and change when it's wrong.
A change that went outside its allowed paths isn't worth typechecking. Stopping early also puts the real finding last on screen, instead of buried above a hundred lines of test output.
// scripts/gate.mjs · the part that matters
const deny = fencedList(read('constraints.md'), 'deny')
const spec = activeSpec(changed) // specs/<id>/ in the diff
const allow = fencedList(read(specFile), 'allow')
const denied = changed.filter(f => matches(f, deny))
const outside = changed.filter(f =>
!matches(f, deny) && !matches(f, allow))
if (denied.length || outside.length) {
// out of bounds is not worth compiling
fail('the diff went outside its allowed paths')
}
run('typecheck', 'npx', ['tsc', '--noEmit'])
run('tests', 'npx', ['vitest', 'run'])
// ↑ swap these two lines for pytest / go test / cargo test.
// The path guard above is language-agnostic.
// activeSpec() reads specs/<id>/ out of the diff and
// refuses one that touches two specs at once. So a
// spec-kit spec.md works the moment you add an `allow`.
| What you need | spec-kit gives you | you still add |
|---|---|---|
| House rules, written once | constitution.md, 5 principles | · |
| The spec itself | spec.md, 17 FRs, 8 criteria | the paragraph that sets the trap |
| The design work behind it | research.md, data-model.md, contracts/ | · |
| Task breakdown | tasks.md, 35 ordered tasks | · |
| Review checklist | checklists/review.md | · |
| Acceptance tests | tasks that write them, mid-run | write them first, watch them fail |
| A deny fence, machine-checked | no equivalent | constraints.md |
| Per-change allowed paths | no equivalent | an allow fence in spec.md |
| Something that refuses a diff | no equivalent | scripts/gate.mjs + CI |
So the last file in the feature is a checklist of what a human confirms once the gate is green. Most of it is close reading a machine can't do: is the boundary asserted on both sides, does the unit conversion happen in exactly one place, is /health carrying no headers rather than empty ones.
specs/002-rate-limit/checklists/review.md
## The boundary
[ ] A hit made at t still counts at t + 59999 and
does not count at t + 60000. Both sides
asserted, not just one
[ ] Nowhere in src/rate-limit.ts is there a window
start, a bucket, or a modulo
## Units
[ ] resetAt inside the limiter is milliseconds, and
the conversion to seconds happens exactly once
## The question the gate cannot ask
[ ] Is ten requests per sixty seconds actually the
right limit for this service, or is it the
number that appeared in the prompt?
github.com/ehsangazar/lightning-lesson-spec-driven-starter: public, MIT, no runtime dependencies. spec-kit.md is a nine-step runbook with every prompt in full, meant to be copied, including the four-block one you just watched. The rest of the repo is what you get when you run it, committed, so you can read the output before you spend an evening producing your own.
allow fence in the spec. Mirror the deny fence in the constitution, then make the gate a required check.constraints.md so the build checks. The gate already reads specs/, spec-kit's own layout, so there is nothing to rename.You're about to ask an agent to add retry with exponential backoff to an HTTP client. Write the single acceptance criterion that a plausible-but-wrong implementation would fail. Three minutes, drop it in chat.
Given a 503 then a 503 then a 200, the client makes exactly 3 calls with sleeps of 100ms and 200ms. Given a 400, it makes exactly 1 call and throws. Delays are read from an injected clock, not measured.
/speckit.constitution once per repo. Put the never-touch list in it, and in a file the build reads./speckit.specify, and spend your effort on the paragraph a plausible version fails. That's the part no tool invents.plan → tasks → implement. The prompt gets shorter, not cleverer. Add an allow fence to the spec before you do.checklists/review.md. The gate proved the code does what the spec says. Was the spec right?Tonight was one discipline, applied to one small feature. The same four questions, what's the contract, what proves it, what's off limits, what does this codebase expect, are how you keep agent-authored volume reviewable as it grows. That's an architecture problem, and it's what the cohort is built on.