Does spec-driven development work?

Published 22 October 2026

Partly. Specs an agent can check its work against — tests, examples, Given/When/Then scenarios — measurably improve what it builds. The spec, plan and task files that spec-driven tools generate show little measured gain on their own. The costs in tokens and review time are measured; the benefits are mostly reported by the people using it. And the one peer-reviewed test of the main claim, fewer defects, found no effect — though it could not include the tools’ own specs. Almost all of this research is from 2026, and much of it is not yet peer-reviewed. Below is what each study found, what spec-driven development costs, the numbers to stop repeating, and when it pays off.

Most practitioners found that specs made agent work more reliable, although they could not separate this benefit from improvements in models.

— Agarwal, Singhal, Macvean, Vasilescu and Breaux, Spec-Driven Development as Prescribed and as Practiced, 2026

The evidence at a glance

We looked at about 190 papers and preprints about spec-driven development and about 75 studies of the questions behind it, and use here only those with data. A preprint is a paper posted before peer review; where a study is one, this page says so.

What spec-driven development promises

The tools promise fewer defects, less rework and more reliable code. Brenn Hill, an independent researcher, collected those claims from vendor material for a paper at ESEM 2026, the empirical software engineering conference, and opened with the problem: “No vendor has published empirical evidence for these claims” (Hill, ESEM 2026). The largest study of how people actually use spec-driven development starts from the same gap: “evidence on its use comes largely from tool vendors” (Agarwal et al.).

How teams actually use spec-driven development

Shyam Agarwal and colleagues at Carnegie Mellon University and Google interviewed 21 practitioners (eight of them tool makers), coded Reddit discussions, and analysed 266 active repositories from a corpus of 470,795 spec files in 73,030 public GitHub repositories. It is a preprint. What they found departs from what the tools describe:

• Specs say how, not only what. The tools keep the “how” in a plan. Yet 77.6% of the specs judged contain implementation detail — class names, file paths, library versions — and in 58.2% it fills a section or more.
• It is used selectively. People decided by task size and risk, and skipped it for bugs, scripts and prototypes; for a production hotfix, “there is no time for spec-driven development and the whole flow”, as one participant put it. An estimated 82% of specs describe new features.
• Nobody runs a tool as shipped. “Almost no participant used an SDD tool as published.” The most frequent code the researchers gave Reddit posts was building one’s own spec-driven tool.
• Specs have one author and one commit. “We observed that 98.3% of spec files with a recorded history have a single author, and 68.5% of inline comments on specs were written by bots.” 81.5% of spec files have a single recorded commit.
• Team benefits appear only beyond developers. Shared understanding improved where specs passed to people in other roles — a product owner told one participant, “Now I see what my developers are doing” — while in developer-only teams the benefit was modest.

The study also names the two ways code and spec part: “If a spec says a session expires after 30 minutes and the agent implements 60, that is implementation drift; if a later change sets 60 as intended and the spec still says 30, that is spec decay.” More on both in what spec drift is.

Does spec-driven development reduce defects?

The only peer-reviewed test found no effect. Hill tested five hypotheses taken from vendor claims against 88,052 pull requests in 119 open-source repositories, tracing which changes later introduced bugs. “None of the five hypotheses are supported.” After matching pull requests of similar risk, those with a spec and those without were indistinguishable in defects introduced. The raw data even showed more defects with specs, which Hill explains by task difficulty: “developers spec their hardest work.”

The limit matters as much as the result. The “specs” in the study were mostly links to issues and tickets, and Spec Kit, Kiro or OpenSpec output was not in it: “no repository in the sample commits such output.” The defect claim has not been tested for the tools at all.

Better specs go with fewer bug fixes — as a correlation. Bruno Claudino and colleagues scored the spec, plan and task files of 114 open-source spec-driven projects with a rubric built on the ISO/IEC/IEEE 29148 requirements standard. “Repositories in the top third of specification quality spent 11.8% of their commits on bug fixes, compared with 20.4% in the bottom third.” The authors treat this as “hypothesis-generating” (EPIC, preprint): it may be that better-run projects simply write better specs.

Does writing the spec first make the agent better?

In controlled tests, yes — though the effect is smaller and more specific than the claims.

• Google, on 90 production bugs. An agent that first wrote a spec of the code’s contract — what must hold before and after each call — and then generated tests found 63.2% of the bugs in five attempts, against 53.4% for an agent that wrote tests directly. It used 38.0% more tokens. The spec was the reason: when it captured the contract the bug broke, the tests caught the bug in 54.9% of cases; when it missed it, in 19.4% (Tufano et al., SpecOps 2026).
• Spec Kit’s files, with and without. In 128 runs on 32 features in five open-source repositories, an LLM judge scored implementations at 3.46 out of 5 without the spec, plan and task files and 3.51 with them; the paper reports no test of that difference. What helped was letting the agent read the repository at each step, which raised the score to 3.66. Without it, agents stay “context blind” in large repositories, “leading to hallucinated APIs and architectural violations” (Spec Kit Agents, preprint).
• How the spec is delivered. In a small pilot, asking a model for “a specification, then the code” in one prompt did no better than asking for code; writing the spec first and implementing it in a fresh step nearly doubled the pass rate, to 45%. Its author warns: “On easier tasks, the dramatic gains people attribute to ‘specifying first’ are largely a reasoning effect in disguise” (Nitin Garg, InfoQ, 2026).
• Several agents on one task. When two agents each built half of a class, removing detail from the spec hurt far more than with one agent: “two-agent integration accuracy drops from 58% to 25% as detail is removed, while a single-agent baseline degrades more gracefully (89% to 56%)” (The Specification Gap, preprint).

What kind of spec helps

The clearest result across studies: a spec the agent can run beats a spec it can only read.

• Tests in the prompt. Giving a model the tests along with the task “improves correctness by 9.15 to 29.57%” on standard benchmarks (Mathews and Nagappan, ASE 2024). With human-written tests, an agent workflow reached 94.3% on SWE-bench Verified, and its authors picture a system “where human developers write tests solved by LLM systems” (TDFlow, EACL 2026).
• Scenarios, not prose. In 2,700 runs with specs built from the Spec Kit template, cutting the spec to a bare user story raised the cost by 29.7% without a credible change in success. Removing the acceptance scenarios mattered; removing prose success criteria did not. “What matters is concreteness about which cases must pass, not the presence of a stated requirement” (Smékal, preprint; five tasks, one model).
• Shorter, concrete, located. Across 433 issues attempted by 87 agents, longer reports went with lower success; “agents benefit most from concrete, executable, and well-localized information” (Khatib et al., preprint).
• Clear about what is unknown. Given vague tasks, “models struggle to distinguish between well-specified and underspecified instructions” — but when they ask, performance improves by up to 74% (Ambig-SWE, ICLR 2026).
• Firm words. “One practitioner we interviewed said their team writes “must” instead of “should” when instructing a coding agent, because the agent may treat “should” as optional” (EPIC).

In practice, that means acceptance criteria written as cases that pass or fail — see acceptance criteria, Gherkin and EARS.

What spec-driven development costs

• Tokens and time. Google’s spec-first agent used 38.0% more tokens. One team quoted in the practice study found “the spec-driven runs typically consume two to three times as many tokens and take about twice as long”; a participant complained that “80% of tokens are being used to make a plan, that is not okay.” The other side: a vague task costs tokens too (the 29.7% above).
• Review. In a five-reviewer pilot, reviewing code against an approved spec took about 48 minutes against 27 for code alone, and found no more of the real problems — about half either way (0.525 against 0.518). What changed was ownership: 81% of findings were tied to a named requirement, against none without the spec. “Reviewers found the same defects, but they become ownable (i.e., tied to an approved requirement) only when a baseline exists” (Garg).
• Reading. In the practice study, most tool users read specs only partially or had other agents review them, and checked code mainly through tests the agent wrote. One participant called the result “verification debt.” More on who should read the spec in who reviews the spec.

Plans and context files

A bad plan is worse than none. Across 21,120 agent runs on SWE-bench, a standard plan improved results, reminders of the plan helped, and “A subpar plan hurts performance even more than no plan at all” (Liu et al., ASE 2026). Reviewing the plan is not optional.

Context files show no reliable gain. On AGENTS.md and CLAUDE.md-style files: “Surprisingly, we find that providing context files does not generally improve task success rates, while increasing inference cost by over 20% on average” (Gloaguen et al., preprint). The evidence is mixed: a smaller study found agents ran faster with an AGENTS.md file but did not check correctness, and across 441 repositories, those with committed AI configuration showed about half the growth in code complexity of those without (+27% against +53%; an observational result, Denisov-Blanch et al., ASE 2026).

These files go stale like any spec. The same study found that “73.8% of artifacts are committed once and never modified”, and 23.0% of repositories with AI configuration files point to code that no longer exists. “When stale, such descriptions can degrade AI assistance without any visible error” (Treude and Baltes, preprint).

Is the productivity gain real?

Nobody has measured it for spec-driven development, and the wider evidence is a warning about self-reports. In a randomised trial with 16 experienced open-source developers and 246 tasks: “allowing AI actually increases completion time by 19%” — while the developers estimated afterwards that it had made them 20% faster (METR, 2025). METR believes developers are likely faster with 2026 tools, but its follow-up could not measure by how much. Among open-source projects adopting Cursor, the speed-up in code written faded within two months, while static analysis warnings rose by 30% and code complexity by 41% (He et al., MSR 2026).

So spec-driven development made us faster is weak evidence on its own. The case for it rests on quality and agreement, which is what the studies above measure least.

Numbers to stop repeating

What nobody has tested yet

• Spec-driven development against plain prompting, with real developers on the same tasks.
• Defects and rework from the specs that Spec Kit, Kiro, OpenSpec or BMAD produce.
• Whether the planning cost is paid back by less rework later.
• How far specs and code drift apart over months, not at the moment of merge.
• Whether a person reviewing the spec improves the result.
• Any head-to-head comparison of the tools with real users.

Is spec-driven development dead?

The question comes up because the attention moved. The Thoughtworks Technology Radar put spec-driven development in “Assess” in November 2025; the entry now reads “This blip is not on the current edition of the Radar”, and the April 2026 edition mentions it inside other techniques: “Teams working with coding agents are increasingly adopting spec-driven development workflows” (Thoughtworks). On Hacker News in August 2026: “it seems like all the SDD tools have kinda fallen off but all of the complaints remain” (Hacker News).

The files say otherwise: of the 470,795 spec files in the practice study’s corpus, 92% were first committed in 2026. The name may fade into plan mode and “harness engineering”; writing down what to build and checking the work against it is spreading. When a plan is enough on its own is in plan mode vs spec-driven development.

When spec-driven development pays off

From the evidence so far:

• Hard, high-stakes work. Participants used it “when the stakes associated with the task were high and the underlying intent was clear”; Garg’s conclusion: “Specification governance is the best investment in one specific area: hard, multi-constraint work built by a capable-but-imperfect model.”
• When acceptance can be written as cases the agent can run — the strongest effect in the research.
• When several people or agents build against the same description.
• When someone else has to agree, and a finding needs an owner — the review pilot’s one clear effect.
• Not for one-line changes, bug fixes or prototypes, where practitioners skip it.

Keep the spec short and concrete, review the plan as well as the spec, and change the spec when the work finds something — the rules, with the evidence for each, are in how to write a spec for an AI coding agent, and a template is in the spec template.

Where Specroot fits

This is the problem Specroot works on: specs that someone other than their author has reviewed, and that are still true after the next change — the two things the studies above find missing most often. Specroot for teams using AI coding agents.