Spec-driven development research: the studies so far

Published 23 October 2026

This is a list of the studies of spec-driven development we could read, in full or in part — what each measured, what it found, and whether it has been peer-reviewed. We looked at about 190 papers and preprints on spec-driven development and about 75 studies of the questions behind it, and list here only those with data. Almost all of it is from 2026, and most of it is preprints: papers posted before peer review. The list is updated when a notable new study appears; the date at the top says when it last was. What the studies add up to is in does spec-driven development work?

In short:

The strongest results are about what the agent is given. Executable cases help; reading the repository helps; the spec, plan and task files on their own help little.
The only peer-reviewed test of the defect claim found no effect, and it did not cover the output of Spec Kit, Kiro or OpenSpec.
Nobody has compared the tools with real users, or measured whether the planning pays for itself.

How spec-driven development is used

Spec-Driven Development as Prescribed and as Practiced — Agarwal, Singhal, Macvean, Vasilescu and Breaux (Carnegie Mellon University, Google), 2026, preprint. Interviews with 21 practitioners, Reddit discussions, and 266 repositories from a corpus of 470,795 spec files. Specs mostly contain implementation detail (77.6%), describe new features (an estimated 82%), have a single author (98.3%) and a single commit (81.5%). “Almost no participant used an SDD tool as published.” Team benefits appeared only where specs passed to people outside development.
The EPIC Framework for Spec-Driven Development — Claudino and colleagues, 2026, preprint. Scored the spec, plan and task files of 114 open-source projects against the ISO/IEC/IEEE 29148 requirements standard, with 15 interviews. Projects in the top third of spec quality spent 11.8% of their commits on bug fixes, against 20.4% in the bottom third — a correlation the authors call “hypothesis-generating”.
Specification Artifacts in Open-Source Pull Request Workflows Do Not Reduce Defects — Hill, ESEM 2026, peer-reviewed. Tested five vendor claims against 88,052 pull requests in 119 repositories. “None of the five hypotheses are supported”; specs went with harder work. The specs were mostly linked issues, not the output of spec-driven tools.
One Developer Is All You Need — Vilas Boas and colleagues, 2026, preprint. A single case at Itaú Unibanco: one staff engineer with AI agents delivered work scoped for a four-person squad in half the planned time, on an existing system, by the team’s own measures. Undocumented legacy integration contracts were the most frequent gap in the specs.
An Exploratory Study of Agent Plans — Abubakar and colleagues, ESEM 2026, peer-reviewed. Of 36,710 repositories screened, 85 plan files in 10 repositories; most were committed once and never revised.

Does writing the spec first help?

Grounding AI Agents in Contracts — Tufano and colleagues (Google), SpecOps 2026 workshop, peer-reviewed. On 90 production bugs, an agent that wrote a spec of the code’s contract before generating tests found 63.2% of the bugs, against 53.4% without, and used 38.0% more tokens. When the spec missed the contract the bug broke, it found 19.4%.
Spec Kit Agents — Taghavi and Bhavani, 2026, preprint. 128 runs on 32 features in five repositories. Judged quality was 3.46 out of 5 without Spec Kit’s files and 3.51 with them; letting the agent read the repository at each step raised it to 3.66.
Citation Discipline in Spec-Driven Development — Panda, 2026, preprint, pre-registered. 840 implementations: Spec Kit, OpenSpec and a variant citing a requirement on every line all passed every functional test; only the citations let invented requirements be detected automatically.
The Specification Gap — Chacón Sartori, 2026, preprint. When two agents each built half of a class, accuracy fell from 58% to 25% as detail was removed from the spec; a single agent fell from 89% to 56%.
SpecPath — Wu and colleagues, 2026, preprint. Of 100 cases where agents succeeded from a final spec, 35 failed on at least one history of revisions ending in the same spec.
Specification Portability Across LLM Development Agents — Grynets and colleagues (EPAM), 2026, preprint. A Kiro-format spec given directly to Gemini produced almost unusable output; specs are not agent-neutral.
LLM-Assisted Repository-Level Generation with Structured Spec-Driven Engineering — Feng and colleagues, FSE 2026 companion, peer-reviewed pilot. Any structured input improved test pass rates over a prose spec; Gherkin in place of the prose did slightly worse on average.
Specification-driven code translation — Saha and colleagues, Journal of Systems and Software, 2026, peer-reviewed. A natural-language spec as the only input did not improve translation between languages.

What kind of spec helps

Test-Driven Development and LLM-based Code Generation — Mathews and Nagappan, ASE 2024, peer-reviewed. Tests in the prompt raised correctness by 9.15 to 29.57% on function-level benchmarks.
TDFlow — Han and colleagues, EACL 2026, peer-reviewed. With human-written tests, an agent workflow reached a 94.3% pass rate on SWE-bench Verified.
Effects of Task Specifications on Token Spend — Smékal, 2026, preprint. 2,700 runs with specs built from Spec Kit’s template: a bare user story cost 29.7% more than the full spec; removing Given/When/Then scenarios mattered, removing prose success criteria did not. Five tasks, one model.
What Makes a Good Bug Report for an AI Agent? — Khatib and colleagues, 2026, preprint. Across 433 issues and 87 agents, longer reports went with lower success; concrete, executable and well-located information helped most.
Ambig-SWE — Vijayvargiya and colleagues, ICLR 2026, peer-reviewed. Models struggle to tell vague tasks from clear ones; asking questions improved results by up to 74%.
On the Impact of Requirement Smells in LLM-Based Code Generation — Villamizar and colleagues, 2026, preprint. The more ambiguous or badly worded the requirements, the lower the functional correctness, though clean requirements could still produce faulty code.

Plans and instruction files

From Plan to Action — Liu and colleagues, ASE 2026, peer-reviewed. 21,120 agent runs: a standard plan helped, reminders helped, and “A subpar plan hurts performance even more than no plan at all.”
Evaluating AGENTS.md — Gloaguen and colleagues, 2026, preprint. Context files did not generally raise task success and cost over 20% more; instructions were followed, repository overviews did not help.
On the Impact of AGENTS.md Files on the Efficiency of AI Coding Agents — Lulla and colleagues, 2026, workshop paper. With an AGENTS.md, Codex ran 28.64% faster at the median; correctness was not measured.
A Few Pages of Markdown — Denisov-Blanch and colleagues, ASE 2026, peer-reviewed. Across 441 repositories, those with committed AI configuration showed about half the growth in code complexity of those without; 73.8% of the files were committed once and never modified. Observational.
Agent READMEs — Chatlatanagulchai and colleagues, ACM TOSEM, peer-reviewed. 2,303 context files: mostly testing, implementation and architecture; security and performance rarely specified.
Configuring Agentic AI Coding Tools — Galster and colleagues, AIware 2026, peer-reviewed. Of 2,853 repositories configuring an AI coding tool, 90.6% had a context file.
Context Rot in AI-Assisted Software Development — Treude and Baltes, 2026, preprint. 23.0% of the repositories checked had AI configuration files pointing to code that no longer exists.

Spec-as-source and regeneration

AfterVibe — Paltenghi and Chandra (Meta), 2026, preprint. Specs recovered from 72 vibe-coded projects let a second agent rebuild them, scoring 5.06 out of 6 on average, with different details each time.
Generative File Systems — Liu and colleagues, ACM Transactions on Storage, 2026, peer-reviewed. A file system generated from specs, and changed by patching them, matched a hand-written one on hundreds of regression tests.
Evaluation of spec-driven development for reproducible AI-generated code — Abdulnoor and Bygde, KTH, 2026, degree thesis. Three models built the same module from one spec; all passed the unit tests, but poor reproducibility and technical debt limited practical use.
Towards Specification-Driven LLM-Based Generation of Embedded Automotive Software — Patil, Ung and Nyberg, 2024, peer-reviewed. A feasibility study on three industrial cases at Scania, pairing an LLM with formal verification.

Productivity and the older claims

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — Becker and colleagues (METR), 2025, preprint. In a randomised trial with 16 developers and 246 tasks, AI made them 19% slower, while they believed it had made them 20% faster.
Speed at the Cost of Quality — He and colleagues, MSR 2026, peer-reviewed. The speed-up in code written faded within two months, while static analysis warnings rose by 30% and code complexity by 41%.
Are delayed issues harder to resolve? — Menzies, Nichols, Shull and Layman, Empirical Software Engineering, 2017, peer-reviewed. In 171 projects: “We found no evidence for the delayed issue effect” — the basis of the 100 times more to fix later claim.
Naming the Pain in Requirements Engineering — Méndez Fernández and colleagues, Empirical Software Engineering, 2017, peer-reviewed. Across 228 companies, incomplete or hidden requirements were the most cited problem (48%).
Agile Specification-Driven Development — Ostroff, Makalsky and Paige, XP 2004. An early use of the name: test-driven development combined with design by contract.

Industry reports with numbers

When Spec-Driven Development Pays Off — Nitin Garg, InfoQ, 2026, reporting a five-reviewer pilot. Reviewing against an approved spec took 48 minutes against 27, found no more problems, and tied 81% of findings to a named requirement.
TDD inside the agent loop — Birgitta Böckeler, martinfowler.com, 2026, an exploratory comparison. No clear gain from asking the agent to follow TDD, at least three times the tokens.
An Accidental Blackboard — Giles Edwards-Alexander, martinfowler.com, 2026. Ten engineers built an airline disruption system in four days, with every agent working from one spec in numbered sections.

Numbers that are not evidence

Claim	Why not
73% fewer security defects	One demo project: 3 findings against 11; the paper itself warns of “n=1 project”
99.98% cost reduction	Self-published, not peer-reviewed
Context files lower success by 3%	Press coverage of an early version; the current version finds no general effect either way
A defect costs 100 times more to fix later	Decades-old data that can no longer be checked; contradicted by Menzies et al.
Standish CHAOS failure rates	The primary report is not available, and the method is contested

What no study has tested yet

Spec-driven development against plain prompting, with real developers on the same tasks.
Defects and rework from the specs that Spec Kit, Kiro, OpenSpec or BMAD produce.
Whether the planning cost is paid back by less rework later.
How far specs and code drift apart over months.
Whether a person reviewing the spec improves the result.
Any comparison of the tools with real users.

How this list was made

We searched OpenAlex and arXiv in October 2026 and followed the references of the studies we found. A study is listed if it reports data and we could read at least its results; where it is a preprint, a workshop paper or a thesis, the entry says so. Some peer-reviewed papers were available to us only as abstracts — among them “From Vibe to Spec” (ASE 2026) and COSPEC (MDPI Software) — and are left out until their results can be read. Self-published papers on Zenodo, OSF or SSRN are not listed.