Abstract
Teams ship larger diffs, faster, with agents iterating unattended. The bottleneck has moved from keystrokes to verification. Unit tests stay green while signup confuses a first-time user. End-to-end scripts pass while checkout feels hostile. A one-shot crawl finds broken links and still says nothing about whether a skeptic can recover a password or a novice can reach first value.
Vibetest-Driven Development (VDD) closes that gap. Encode customer outcomes as natural-language scenarios; let an independent agent act as the customer on a real product surface; ship only when the acceptance result is acceptable—and keep those scenarios as living contracts, not one-off demos.
After reading this paper, you should be able to author one critical scenario, treat its acceptance result as the bar for merge or release, and re-run that same contract on every meaningful ship.
The new bottleneck is not writing code
For most of software’s history, the scarce resource was the ability to change the system. Verification was expensive too, but it scaled roughly with how much humans could implement. That balance is gone.
Agents scale diffs; humans design outcomes
Coding agents and unattended iteration do not merely accelerate individual pull requests. They multiply surface area: more files touched, more edge paths, more UI states that never appear in a ticket. Humans stop being typists and become designers of outcomes—people who decide what “done” means when a machine can draft the path there. The question shifts from “can we build this?” to “can we trust that a customer still succeeds after we ship?”
That trust is no longer free. A team that can generate three feature variants overnight still faces the same fixed cost of someone opening a browser and walking the product like a user. Manual checks do not scale with agent throughput. They become a queue of tired humans replaying happy paths while risk hides in the scenarios nobody re-runs.
Scripts and suites leave customer truth underspecified
Teams reach for automation—and often get a different failure mode. Brittle end-to-end scripts couple to selectors and layout. A redesign that improves the experience can fail a suite for the wrong reason; a regression that only confuses humans can leave the suite green. Maintenance cost rises with every intentional UI change. Happy-path bias is structural: the tests cheapest to keep green are rarely where real users struggle.
Unit tests and lower-level integration remain essential. They still do not answer the question customers care about. Green coverage can coexist with broken flows, dead ends, missing feedback, and friction that looks fine to an implementer who already knows where the button is. The suite proves the machinery; it does not prove engagement.
One-shot probes are not a development practice
Site crawls, swarm probes, and ad-hoc “vibetests” are useful. They surface broken links, blank pages, and obvious failures without a hand-written script. They are not, by themselves, a development practice. A probe without a named customer outcome does not tell you whether this goal still works. A one-shot run does not become a regression contract. Without a durable definition of success, findings do not re-enter shipping as something the team can re-run, require for acceptance, and keep.
Agents stop when they decide they are done
There is a subtler failure than slow checking: premature completion. An agent working alone rarely runs out of time or budget—it stops because, by its own assessment, it is finished. Research from Factory makes the size of this gap measurable: rebuilding large programs from scratch, a frontier model reproduced 36 percent of gdal’s behavior when it judged its own completion—and 90 percent when a separate role authored an executable standard of completion before implementation began and the work was held to that standard.
The model’s skill did not change; the judgment of completion did. Additional compute does not help an agent that will not spend it. The lesson transfers directly to product work: the bottleneck is not only how fast checks run after a ship, but whether the definition of done was ever independent of the implementer at all.
The lag between ship and customer proof
Call the gap verification lag: the time and uncertainty between “code is in” and “we know a real user can succeed.” AI compresses the left side of that interval and leaves the right side untouched—or worse, moves risk into production analytics and support tickets. Escaped defects of this kind are not always crashes. They are scenarios that fail quietly: the invite that never lands, the recovery flow that loops, the first-value path that requires tribal knowledge.
Teams need a durable, re-runnable definition of “a real user can succeed”—expressed in language product, engineering, and success already use; verified independently of the implementer; backed by evidence a human can trust; and decidable enough to accept or block a ship. That definition is the unit of work this paper develops next.
What VDD is
Vibetest-Driven Development is an outcome-first practice: you encode what a customer must be able to do in natural language, an independent agent attempts that scenario on a real product surface, and you treat the resulting evidence-backed acceptance result as the bar for ship—then keep the scenario as a living contract for every meaningful change that follows.
A customer outcome in plain language becomes acceptance criteria, agent task, and regression contract at once.
In one lineFour properties
Start from a named customer outcome
Work starts from a customer outcome and acceptance criteria, not from selectors or a free-form crawl. Implementation may be human or agent; the scenario does not care who typed the code.
An agent acts as the customer
Navigating, reading the UI, attempting the goal—on a real browser or equivalent product surface. The checker is not the same actor that just shipped the candidate.
A pass or fail is not a vibe
Screenshots, step traces, failure notes, and other artifacts make the result reviewable without replaying the session live.
Machines and humans can act on it
The acceptance result is decidable enough to allow merge, block ship, or open fix work. Soft judgment still produces a status and rationale.
Core objects
| Object | Role |
|---|---|
| Scenario | Natural-language contract (acceptance scenario / sheet): who the customer is (implicitly or by lens), what outcome they pursue, what success looks like, and what is out of scope. The unit of work. |
| Run | One independent attempt of a scenario against a specific product surface (environment, build, credentials, fixtures). |
| Evidence | Durable artifacts from the run: what the agent did, saw, and recorded. |
| Acceptance result | The decidable summary—status, score or summary of evidence, issues—consumable by humans and by automation. |
The scenario is the durable asset. Runs are ephemeral; evidence and acceptance results are the record that makes acceptance and regressions possible. Customer engagement lives inside the scenario wording—who is trying, what “good” feels like—not as a second methodology layered on top.
Non-goals
- Not a crawler or swarm probe. Crawls discover; scenarios assert named customer outcomes. Discovery can inspire scenarios; it does not replace them.
- Not self-check by the implementer. The agent or engineer who built the change does not grade their own homework as the sole acceptance.
- Not a unit-test replacement. Lower layers still catch logic and contracts cheaply. VDD sits at the customer-outcome layer.
- Not a one-shot demo. A single impressive run is not VDD. The practice requires scenarios that survive as regression of engagement.
Running an agent once over a site and chatting about what broke is useful exploration. VDD starts when the outcome is named, success is decidable, the run is independent, evidence is kept, and the same scenario re-enters on the next ship. The difference is not the model in the browser; it is whether customer truth is a durable contract.
Eight principles
Eight principles keep the practice about customer truth rather than tooling fashion.
-
Outcome before (or with) implementation
Write what success means for a customer before—or at least alongside—the change meant to deliver it. Outcomes can arrive as a short scenario draft in a ticket, a PR description, or a shared library of critical paths. The order is not dogma; the discipline is that “done” is not defined only by the implementer’s knowledge of the UI. When agents generate large diffs, this keeps outcome design ahead of unattended churn.
-
Prototype first; contract before the real build
When code is cheap, the fastest way to learn what a customer outcome actually is may be to build a throwaway version and put it in front of someone—or an agent acting as one. A prototype exists to sharpen the scenario: which steps are essential, where a novice stalls, what “good” turns out to feel like. Author or revise the contract from what the prototype taught you about the customer, then hold the real implementation to that contract.
Two rules keep this honest. The prototype is disposable—it is not the candidate that ships, and its shortcuts do not become the definition of done. And the scenario is written from the outcome the prototype revealed, not transcribed from whatever the prototype happened to do; a scenario that merely describes the prototype’s path inherits its scope and ends up verifying a sketch rather than a customer. Running the draft scenario against the prototype is also the cheapest way to find out whether the contract is decidable before anyone ships against it.
-
Natural language is the source of truth
The scenario text is the contract. Selectors and page objects may exist as substrate; they are not what product and success can own. Natural language is already how teams describe engagement—“a new user should reach first value without a tutorial call.” VDD makes that description executable and re-runnable instead of leaving it in a doc that drifts.
-
Agents act as customers, not as locators
Verification means attempting the goal the way a person would: reading labels, following affordances, recovering from confusion when the product allows it. An agent that only clicks pre-baked coordinates or asserts fixed CSS paths is running a script in disguise. The value is goal-seeking behavior on a real surface—customer simulation, not locator maintenance.
-
Evidence over assertion spam
A wall of green checkmarks without artifacts is weak acceptance. Prefer fewer scenarios with durable evidence—what happened, where it failed, what the agent saw—over hundreds of brittle asserts nobody trusts. Humans should be able to review a run without watching it live.
-
Acceptance, not vibes
“Looks fine” is not a ship decision. Status, thresholds, and issue severity should be clear enough that a run can block, allow, or open work without a meeting to reinterpret the output. Soft judgment about friction still belongs in the run; it must land as a decidable acceptance result, not an open-ended chat log.
-
Scenarios live forever
A scenario that passes once and is discarded was a demo. In VDD, scenarios accumulate as regression of engagement: the same contracts re-run when the product changes. New features add scenarios; old ones keep guarding first-value, recovery, invite, and checkout long after the original author has moved on.
-
Contracts tighten; they never quietly loosen
A scenario may be expanded, clarified, or made stricter as the team learns more about the outcome. What it must never do is quietly collapse around whatever has already been built. When a run fails, the choices are: fix the product, or change the contract deliberately and in the open—because product agrees the outcome itself changed. Editing acceptance criteria until the current build passes is not iteration; it deletes the standard while keeping its name.
These principles do not require a particular vendor or harness. They require teams to treat natural-language customer outcomes as durable, independently verified, evidence-backed acceptance—and to keep that bar for the life of the product.
Why natural language is the interface
Engineering, product, and customer success already share one medium for describing what a good experience looks like: plain language. A PM writes “a new user should reach first value without a sales call.” Support notes that “returning customers get stuck when the invite link expires.” An engineer says “skeptical buyers need to see pricing before they create an account.” None of those sentences mention selectors or page objects. VDD keeps that language as the contract—not a comment on a script, and not a slide that dies after kickoff.
One artifact, three jobs
Acceptance criteria
For the team shipping the change.
An agent task
For an independent verifier acting as a customer.
A regression contract
Re-run whenever the product surface moves.
When those three jobs collapse onto one natural-language artifact, product outcomes stop drifting away from what automation actually exercises.
Engagement lives inside the scenario
Customer engagement is not a second methodology bolted onto VDD. It is how you write the scenario: who is trying, what outcome they came for, and what “good” feels like when they get there. Functional completion alone is not enough. A checkout that only works if you already know where the coupon field is still fails the customer even when every API returned 200.
That bar shows up in wording, not in costume. Engagement lenses steer the scenario without theatrical personas:
Novice
First visit, limited product vocabulary, low tolerance for jargon and empty states that assume prior knowledge.
Returning
Already has an account or prior context; expects continuity, saved state, and recovery that respects that history.
Skeptical
Will abandon if trust, pricing, or exit options are opaque; needs clear value before commitment.
You do not invent a character backstory. You write sentences like “as someone who has never used this product, complete signup and create one project without external docs,” or “as a returning member whose session expired, recover access and land where you left off.” The lens tightens acceptance criteria and friction signals; it does not require the agent to role-play a name and hobby.
Why not scripts as the source of truth
Locator-based scripts encode how the UI was wired last week. Natural language encodes what customer outcome must still be achievable. Layouts and copy change constantly—especially when agents generate the UI. Outcomes change more slowly, and when they do, product and engineering edit the same paragraph instead of reverse-engineering a brittle chain of clicks.
Teams that already write tickets, PRDs, and support macros in natural language are not learning a foreign format. They are promoting that language from conversation to acceptance: something an independent agent can attempt on a real product surface, record with evidence, and leave as a contract the next ship must still honor.
Anatomy of a scenario
A scenario is a named, natural-language contract for one customer outcome—specific enough that an independent agent can attempt it on a real product surface and produce a decidable acceptance result your team can use for merge or release.
Required parts
Goal
The outcome the customer is trying to achieve, in one clear sentence. Prefer results over UI tourism (“complete checkout and receive confirmation,” not “explore the cart”).
Acceptance criteria
Observable conditions that mean the outcome was met—functional completion and engagement quality when it matters (no dead ends, no unexplained errors). A third party should score them without watching the implementer’s screen share.
Context
Base URL or environment, auth state, fixtures, locale, and flags that define the surface. Without context, “pass” is not reproducible.
Constraints
Rules while pursuing the outcome: stay in the app, no admin backdoors, happy path unless recovery is the point.
Out of scope
Explicit non-goals so the run does not sprawl into a free-form crawl of the product.
Optional: engagement lens
When the same outcome feels different for different customers, add a short lens—novice, returning, skeptical, or one line: “First-time visitor; abandon if pricing is hidden until after signup.” The lens adjusts trust and friction; it is not a costume or a long persona.
What makes a scenario acceptance-ready
A scenario is ready for acceptance when a verifier can answer yes or no (or an agreed threshold) from evidence alone. Decidable scenarios name a single primary outcome, tie success to customer-visible signals, separate must-pass from nice-to-haves, avoid unbounded exploration, and re-run cleanly with the same context. Vague goals produce vibe reports. Decidable goals produce artifacts a merge policy can consume.
Short examples
Signup novice
- Goal
- Create an account and land in a workspace ready for first use.
- Acceptance criteria
- Verification completes or is clearly deferred; next action is visible without external docs.
- Context / out of scope
- Preview, signed out. Out of scope: OAuth edge cases.
Checkout
- Goal
- Purchase the listed plan with a test card and receive confirmation.
- Acceptance criteria
- Total matches plan; confirmation has a support-quotable reference; account shows paid.
- Context
- Seeded catalog, test credentials.
Invite
- Goal
- Accept an invite and join the correct workspace with the expected role—not a blank personal account.
- Context / out of scope
- Invite link fixture, signed out. Out of scope: invite creation UI.
Recovery returning
- Goal
- Regain access after expired session or forgotten password and continue prior work without support.
- Context
- User with existing project data; expect a sensible post-login destination.
First value novice
- Goal
- From landing, reach one meaningful product result without a sales call.
- Acceptance criteria
- Result saved and visible on return; blocking empty states called out.
- Out of scope
- Billing and advanced settings.
Avoid “test the app,” “make sure nothing is broken,” or crawl briefs that ask an agent to wander and report vibes—those are probes, not contracts. Prefer customer-recognizable outcomes over implementation steps (“click the blue button in the third card”) unless the path itself is the point. If you cannot tell from the text whether a run should pass or fail, rewrite before anyone ships against it.
Independent verification
VDD separates the people and systems that build a change from the process that judges whether a customer can still succeed. The maker is not the checker.
That separation is the difference between a hopeful demo and acceptance you can trust when code ships faster than humans can click through every path.
Why self-grade fails
When the same agent—or the same engineer—implements a feature and then marks the scenario done, failures hide in plain sight. The implementer knows the intended path and skips the dead end a stranger would hit. They start already authenticated, already on the right URL, already holding fixture knowledge the customer does not have. They treat partial success as “good enough” because they remember yesterday’s hard fixes.
None of this requires bad faith. Familiarity is enough. Under AI-assisted shipping, a coding agent can generate both the UI and a cheerful narrative that the flow “works,” while a real browser session for a novice still fails on the first form. Self-check is useful smoke while developing. It is not a substitute for independent verification against the scenario contract.
There is a structural problem deeper than familiarity: checks inherit the scope of the work that produced them. As an implementer decomposes a feature, they decide at each piece what evidence would count and whether it is sufficient. Those checks can establish everything the implementer thought to build while excluding outcomes, interactions, and customers that were never represented. Every local judgment is reasonable, and much of the whole remains unmeasured. This is why the scenario must be authored from the customer outcome, not derived from the build: a check reverse-engineered from the shipped UI verifies the implementation’s memory of itself.
Agent as customer, on a real surface
Independent verification means an agent attempts the scenario as a customer would: on the actual product surface (or a faithful preview), through the real UI, under the context the scenario specifies. It is not a locator script replaying known selectors. It pursues the natural-language outcome, discovers affordances as a user would, and stops when acceptance criteria are met or the attempt clearly fails.
That stance keeps the contract stable when markup moves. The scenario still says “complete checkout and see confirmation.” Layout this week is the agent’s problem for that run—not something product rewrites every time a button moves.
The outcome crosses the wall; the cases do not.
The scenario is shared. The verifier’s path is not.The scenario crosses the wall; the verifier’s path does not
Sharing tests with the implementer is sometimes said to invite teaching to the test. The risk is real, but it lives at the case level, not the outcome level. A fixed script—exact inputs, pinned selectors, a known click path—is a sparse sample of the behavior a customer actually needs. Once that sample is visible, it becomes the target: an implementer (human or agent) can patch until it greens without the outcome ever becoming true.
VDD’s contract sits above that level. The scenario is deliberately shared—product and engineering co-author it, and “teaching to” an outcome like a novice completes signup without external docs is simply building the product. What stays on the verifier’s side is the concrete path: the session, the affordances discovered, the specific probing of a given run—regenerated fresh each time rather than replayed from a fixed artifact. This is also why a locator script handed to the implementer is a weaker instrument than a goal-seeking agent, even when both count as “automation.”
Separate run, separate report, durable evidence
Trustworthy acceptance has structure:
- Separate run — distinct from implementation, with its own session and environment binding.
- Separate report — written for reviewers and machines, not only a chat reply that scrolls away.
- Durable artifacts — screenshots, traces, step logs, and a structured acceptance result that outlive the run.
Without durable evidence, “it passed” is a rumor. With evidence, a failing scenario is actionable: you see where the customer would have stalled, not only that a boolean flipped false.
Weak acceptance to avoid
Independence is wasted on soft mush: rubric-free “looks fine,” status with no screenshots or reproducible context, essays that do not map to pass/fail or a clear blockage, and unbounded crawls with no named customer outcome. Probes are useful; they are not ship decisions.
Strong verification answers a hard question: given this scenario and this build, can the customer achieve the outcome, and what proof do we keep? VDD insists on that question, asked by someone—or something—that did not just write the code, and answered with artifacts that still make sense after the tab is closed.
VDD in the shipping cycle
VDD does not replace your pipeline design. It supplies a contract (the scenario) and an acceptance result (evidence-backed, decidable) that any stage can re-run.
-
Author the outcome
Before or alongside implementation, write the scenario: goal, acceptance criteria, context, constraints, out of scope, and an engagement lens if who is trying changes the path. Product and engineering can co-author; the test is whether a stranger could decide pass or fail from the text. Critical paths—signup, first value, checkout, invite, recovery—belong here first.
-
Ship a candidate
Humans or coding agents implement against that outcome. Unit tests still catch structural mistakes; the scenario is the customer-shaped bar above them. The candidate may be a branch, preview deploy, or local surface—whatever exposes a real UI.
-
Verify the scenario independently
Run the scenario with a verifier that did not just author the change: an agent acting as customer on the real product surface, under the scenario’s context. Capture a structured acceptance result and durable evidence. A green unit suite with a failed scenario still means the customer cannot succeed as written.
-
Act on the acceptance result
Treat the acceptance result as a decision input, not wallpaper. Merge or promote when must-pass criteria hold and friction is within agreed thresholds. Fix and re-run when the customer is blocked or the path is hostile. Open work when the scenario itself is wrong—ambiguous criteria, bad fixtures, or an outcome product no longer wants—rather than silently weakening acceptance.
Acting includes saying no to ship. Always finding a way to green without changing code or contract is not real acceptance.
-
Keep the scenario as an ongoing contract
Do not delete the scenario after launch. It becomes regression of engagement: the same outcome re-checked when the surface moves. Over time you keep a small library of critical scenarios—the smoke of “a real user can still succeed”—not a museum of one-off demos.
Re-entry and roles
The same scenario can re-enter whenever you care about that outcome again: on pull request, on deploy, on a schedule, or before a release train. How you wire those triggers is your automation. VDD keeps the contract stable and the acceptance result interpretable so re-entries mean the same thing every time.
Authors own customer outcomes—often product with engineering. Fixers are usually the team that shipped: they consume evidence, repair product or fixtures, and re-run until the acceptance result is acceptable. Acceptance owners set thresholds and protect the practice from vague scenarios and evidence-free scores.
The cycle is intentionally boring: outcome in, candidate out, independent customer attempt, decide, retain. When machines write more of the candidate, who owns customer truth—and when it is re-checked—is the advantage.
Decidable acceptance results
A scenario without a decidable result is a prompt, not acceptance. VDD turns each independent run into an acceptance result: a machine-readable summary of what happened, backed by evidence a human can audit later.
The point is not a magic number. It is a result that automation, a pull request, or a release checklist can act on without replaying the session live.
Dual signal: reliability and engagement
Every run should answer two questions that often diverge.
Reliability
Did the customer complete the outcome: account created, invite accepted, first report generated, payment confirmed. If the path is blocked—error page, missing control, broken auth—the run fails on reliability regardless of how polished the rest of the surface feels.
Engagement quality
Would a real person stay on that path. The outcome may have been reached after dead ends, opaque copy, or optional steps a novice would abandon. Reliability can be green while engagement is poor; engagement can look smooth while a silent failure means the outcome never completed.
VDD keeps both signals on the same run so the team does not ship “it worked for the bot” while customers still bounce.
Score summarizes evidence
A score is a compact summary of scored criteria and observed issues—not a free-floating vibe. It should be reconstructible from the evidence pack: which acceptance criteria held, which constraints were violated, which friction points were recorded. If two reviewers cannot explain why the score moved, the score is not ready to drive merge or release.
Prefer criteria tied to the scenario contract (outcome reached, critical copy present, forbidden paths avoided) over global “UX quality” rubrics no one owns. Soft judgment belongs in labeled issues and notes, not in an unexplained float between 0 and 100.
Status, thresholds, and severity
A practical acceptance result usually includes:
| Field | Meaning |
|---|---|
| Status | pass fail inconclusive—the last for infrastructure, timeout, or environment that prevents a fair judgment. |
| Thresholds | Team policy for when status becomes a hard stop. For example: any reliability failure blocks merge; engagement below a floor blocks release but not every draft PR. |
| Issues with severity | Blocker (outcome impossible), major (path works but hostile or error-prone), minor (nits that should not flip acceptance alone). |
Status is the primary actuator. Score and issue lists explain why. Thresholds should be explicit and stable; changing them every week teaches people to ignore acceptance.
Flakiness and soft judgment without killing the practice
Natural-language scenarios and real browsers introduce variance. Treat flakiness as part of the practice, not a reason to abandon it.
- Prefer inconclusive over a random fail when the environment, auth fixture, or third-party dependency clearly poisoned the run.
- Require reproducible evidence before a soft engagement complaint becomes a hard fail—screenshot, step timeline, or transcript of the dead end.
- Separate scenario flakiness (ambiguous outcome, unstable acceptance criteria) from product flakiness (race conditions, flaky deploys). Fix the contract or the product; do not paper over either with “retry until green.”
- Cap retries. Unlimited retries turn acceptance into a lottery.
Soft judgment—“this flow feels confusing”—is valuable when attached to a concrete moment and an agreed severity. It is poisonous when it is the only line in the report and status is still forced to pass.
With status, dual signal, score-as-summary, severities, and durable evidence, the same scenario can re-enter on a pull request, preview deploy, or scheduled smoke without rewriting the contract. Humans review failures; machines enforce thresholds. Outcomes stay fixed, candidates change, and the acceptance result decides what happens next.
How VDD relates to existing practices
VDD does not replace the rest of the quality stack. It sits where customer outcomes must stay true on a real product surface.
| Practice | Relationship to VDD |
|---|---|
| TDD / unit tests | Complementary lower layer. Protect modules cheaply; do not speak as a novice trying to reach first value. |
| E2E scripts | Optional substrate. Natural language is the interface teams author and accept against; scripts are implementation detail when useful. |
| Exploratory QA | Partly absorbed into agent runs + human review of evidence. Skill remains; a repeatable slice becomes a contract. |
| Crawls / swarm probes | Useful discovery. Not a substitute for named outcomes or ship acceptance. |
| Product analytics | Real users after traffic. VDD pre-validates critical paths before that traffic. |
Combine without double work: units for code shape, scripts for stable UI you own tightly, scenarios for customer-critical engagement, crawls for discovery, analytics for post-release truth. When a scenario fails, fix product or outcome first; only then lock a regression lower in the stack. VDD’s claim is narrow: named natural-language customer outcomes, independently verified, with a decidable acceptance result, belong next to—not instead of—the tools you already trust.
Organizational adoption
Adopting VDD is less about tooling theater and more about choosing a few paths that define whether the product still works for the people who use it—and treating those paths as durable contracts.
Start with a small set of critical scenarios
Begin with a smoke of engagement: three to seven scenarios for real first-value or revenue-critical outcomes (signup to first success, core checkout, invite accept, password recovery). Write them in natural language with clear acceptance criteria and an engagement lens where it matters. Resist covering every screen. Breadth without decidability produces noise and trains the team to ignore results.
Owners should be shared: product or success often drafts who and what good feels like; engineering owns environment, fixtures, and actionable failures. The scenario is a team artifact, not a private QA script.
Progressive rollout
One path
Author a single critical scenario, run it independently on a real surface, and review the evidence pack until humans trust pass and fail.
PR or preview acceptance
Attach the same scenario (or a small set) to meaningful ships. Start advisory if needed; promote to hard block on merge once flakiness is under control and thresholds are agreed.
Library
Grow a living set as features ship. Retire scenarios only when product outcomes die—not because UI class names changed.
The same contract can re-enter on pull request, deploy, or schedule. How you schedule runs is your delivery system’s concern; VDD supplies the contract and the acceptance result.
Anti-patterns to avoid early
Vague scenarios, self-grading only, scenarios reverse-engineered from the shipped UI, scores without evidence, one-off demos, crawl-as-substitute, and uninterpretable acceptance results. Catch these in the first month; they reappear later as formal failure modes.
Lightweight practice metrics
Scenario pass rate
On main or release candidates; split reliability vs engagement if results allow.
Time-to-green
After a scenario fails—how long until outcome and product agree again.
Escaped defects
On covered paths: support tickets, funnel drops, P0s a scenario should have caught.
If pass rate is always 100%, thresholds may be too soft or scenarios too shallow. If nothing ever greens, criteria may be undecidable or the environment is lying. Adjust contracts and fixtures before adding more scenarios.
Adoption succeeds when a new engineer can find the critical scenarios, read an acceptance result, and know whether shipping is allowed—without watching a browser live.
Worked example
One feature walked through VDD without a product pitch. The shape is what matters: outcome first, independent verify, engagement-aware fail, fix, green, scenario kept.
Feature idea
A team ships a project digest: after a user connects a data source, the product should produce a short weekly summary they can trust at a glance. Engineering can build the pipeline and UI quickly. The open question is whether a novice can reach that first valuable digest without a guide on a call.
Natural-language scenario (novice, first value)
Before treating the feature as done, the team authors:
You are a new user who has never used the product. Starting from the signed-out marketing entry for digests, create an account if needed, connect the provided sample data source using only in-product guidance, and reach a state where a digest for the sample project is visible. Success: a digest with a clear title and at least one substantive summary section is on screen, and you could explain in one sentence what it is summarizing. Constraints: do not use external docs or support chat; do not require knowledge of internal field names. Out of scope: customizing schedule or sharing the digest.
Optional engagement lens: novice, time-boxed curiosity—if the path feels like setup theater, note where you would quit.
Implementation and first independent run
A candidate lands on a preview environment. An independent run—separate from the implementer’s self-checks—executes the scenario on a real browser surface and writes a durable report.
Fix and green
The team does not rewrite the scenario to match the broken UI. They fix the product: sensible defaults for the sample source, plain-language labels, and an empty state that shows a partial digest or a single “Generate sample digest” action. A second independent run completes the outcome with fewer dead ends and no major engagement issues. Status: pass. The score rises because evidence of acceptance criteria improved—not because someone argued for a higher vibe.
Scenario remains
The scenario stays in the library. It re-enters when digest onboarding changes, when auth or empty states are refactored, and on release candidates. Weeks later a UI cleanup reintroduces jargon on field mapping. The same contract fails with comparable evidence. The fix is obvious because the customer outcome never moved.
That is VDD in miniature. Code and agents can iterate freely underneath; customer-proof of first value remains the acceptance bar.
Failure modes and anti-patterns
VDD fails less from missing tools than from soft contracts. The patterns below look productive and still leave you without reliable customer acceptance.
Vague scenarios
“Test signup” or “make sure checkout works” is not an outcome. Without a goal, acceptance criteria, context, and out-of-scope bounds, two runs cannot disagree usefully—and acceptance cannot decide. Write who the user is, what they must accomplish, and what counts as done.
Uninterpretable acceptance results
A pass/fail that depends on a human rereading a novel of logs is not usable acceptance. Status, thresholds, and issue severities must be machine-decidable enough that merge, fix, or hold can follow without a meeting.
Score without evidence
A number with no screenshots, steps, or failure notes is a vibe in numeric clothing. Scores summarize evidence; they do not replace it. If a human cannot reconstruct why the score moved, treat the run as incomplete.
One-off demos
A single impressive agent walkthrough proves little about the next release. Scenarios earn their keep when they re-run on every meaningful ship and stay as regression of engagement. Archive demos; keep contracts.
Crawl as substitute
Site-wide probes and swarm visits surface broken links and surprises. They do not encode named customer outcomes. Discovery can seed scenarios; it cannot replace them as the unit of work or the definition of done.
Self-grading only
When the same agent (or person) that implemented the change grades its own result, maker and checker collapse. Independent verification—separate run, separate report, durable artifacts—is the point. Self-check is a useful draft; it is not acceptance.
Scenarios derived from the build
Writing the scenario after implementation, from what the UI happens to do, inherits the implementer’s scope: everything they thought to build is covered, and everything they never represented stays unmeasured. Author from the customer outcome and let the build be judged against it, not the reverse.
Avoid these seven and the practice stays sharp
Outcomes you can re-run, acceptance results you can act on, and evidence someone else can trust.
Closing: outcomes are the scarce asset
Code is no longer the scarce input. Agents can draft, refactor, and iterate faster than any review queue can watch every path by hand. What remains scarce—and what separates teams that ship confidently from teams that ship loudly—is a durable account of what must stay true for a real customer.
Vibetest-Driven Development is that account made operational: customer outcomes written in natural language as scenarios, verified independently on a real product surface, and kept as living contracts. The scenario is not documentation of a demo; it is the definition of done that machines can re-check after every meaningful change.
Who owns customer truth becomes the advantage as autonomous iteration increases. Own it by writing it so it can be re-run, not by hoping the next human QA pass catches what the last one missed.
Author one critical scenario for a path that would embarrass you if it broke. Make the acceptance result decidable—status, threshold, evidence a stranger can read. Re-run that scenario on every meaningful ship: pull request, deploy, or release candidate. Expand only after that cycle is boringly reliable.
Looking ahead, the same shape scales without changing the idea: richer engagement-lens libraries still expressed as wording, multi-app and multi-surface scenarios under one outcome library, and deliberate tradeoffs on cost and latency per run. The practice does not depend on any single product. It depends on treating customer outcomes as acceptance—and keeping that bar honest.
Glossary, templates, and schema
This appendix is conceptual: adapt field names and storage to your stack. The methodology requires scenario, independent run, evidence, and decidable acceptance result—not a particular schema or vendor.
Glossary (short)
| Term | Meaning |
|---|---|
| Customer outcome | What a real customer must still be able to achieve. |
| Scenario | Natural-language acceptance contract for one outcome. |
| Acceptance criteria | Observable conditions that mean the outcome was met. |
| Run | One independent attempt of a scenario on a real surface. |
| Evidence | Durable artifacts (acceptance materials) from a run. |
| Acceptance result | Status, score, issues, and pointers to evidence. |
| Engagement lens | Optional wording (novice, returning, skeptical). |
Sample scenario templates
Signup novice
- Goal
- A new user creates an account and reaches the first useful screen without support.
- Acceptance criteria
- Account created; confirmation or onboarding shown; no dead ends on required fields.
- Context
- Staging URL; empty fixture; no prior cookies.
- Out of scope
- Social OAuth variants, admin invite paths.
Checkout returning
- Goal
- A returning customer completes purchase with a known cart and test payment method.
- Acceptance criteria
- Order confirmation with correct total; cart empty; UI confirmation sufficient if email is secondary.
- Context
- Auth fixture; seeded cart; payment sandbox.
- Constraints
- Do not use production cards.
Invite / first value
- Goal
- An invited teammate accepts and performs the one action the product promises first.
- Acceptance criteria
- Invite accepted; action completed; product shows clear proof of success.
- Context
- Fresh invite token; role as specified.
- Out of scope
- Billing upgrade, SSO setup.
Minimal acceptance-result schema (conceptual)
{
"scenarioId": "string",
"runId": "string",
"status": "pass | fail | inconclusive",
"score": 0,
"threshold": 0,
"summary": "string",
"reliability": "pass | fail | inconclusive",
"engagement": "pass | fail | inconclusive",
"issues": [
{ "severity": "blocker | major | minor", "title": "string", "detail": "string" }
],
"evidence": {
"steps": ["string"],
"artifacts": ["uri-or-path"]
},
"startedAt": "ISO-8601",
"finishedAt": "ISO-8601"
}score summarizes evidence against the scenario; never ship on score alone without artifacts. inconclusive means re-run policy applies before treating status as decisive (environment, timeout, or fixture failure—not a silent pass).
Further reading
- Classic acceptance-test and behavior-driven ideas for how teams already name outcomes in prose.
- Exploratory testing literature for engagement friction and charters—useful when authoring scenarios, not as a substitute for re-runnable contracts.
- Product analytics and session replay as post-traffic complements; VDD pre-validates critical paths before real users hit them.
- Factory’s research on coding agents completing large software tasks—empirical evidence that an independently authored, pre-implementation standard of completion changes what the same model ships.