Documentation · Delivery

Testing & validation

Nothing lands in your repository on an AI's word alone. Every ticket must pass your project's real build and test commands, in clean containers, on your own build VM — and every milestone must pass an integrated end-to-end run of the whole application before its pull request opens.

The Testing Bar — your project's contract

The Solution Architecture Document ends with a Testing Bar: one line per component naming the container image and the exact install, build, and test commands, plus one end-to-end entry describing how to run the E2E suite. Example:

apps/web: image="node:20-slim" install="npm install" build="npm run build" test="npm test"
apps/api: image="node:20-slim" install="npm install" build="npm run build" test="npm test"
e2e: compose="docker-compose.yml" prefix="e2e" install="npm install && npx playwright install --with-deps chromium" test="npx playwright test" image="node:20-slim"

The e2e: line can describe your project's E2E entrypoint three ways: a compose overlay file (shown above), naming one existing compose service to run, or — for a project whose real entrypoint needs multi-step orchestration a plain compose file can't express — pointing at your own script (e.g. scripts/ci/run-e2e-smoke.sh), run directly with its own exit code as the verdict.

The pipeline executes these literally and gates delivery on their real exit codes. That's why an unapproved SAD blocks ticket dispatch — no approved Testing Bar, no test gate.

Per ticket: the unit stage and the repair loop

Generated code is written into an isolated workspace on your VM.
The affected component's commands run inside its declared image — install, then build (a project that passes tests but can't compile fails loudly here), then tests. Unit runs are isolated: no live database or sibling services — that's the E2E stage's job.
On failure, the distilled error output (the actual failing assertions, not log noise) goes back to the AI, which returns a surgical fix — exact text edits applied deterministically by the platform, not a rewrite. The loop is bounded; a ticket that can't converge is surfaced with its reason, never force-landed.
Only after a green run is the work committed and pushed.

Per milestone: the integrated E2E stage

Tickets write their end-to-end specs as they go, but E2E execution happens once per milestone, when the last ticket has landed:

The whole application starts however your Testing Bar's e2e: line says — a compose overlay, a named service, or your own script — every service up, with healthchecks and seeded fixture data.
The full unit suite re-runs against the integrated branch (a free cross-ticket regression check), then the E2E suite exercises real user paths end to end.
Failures here are integration bugs by definition — the repair loop fixes them on the milestone branch with visibility across all the milestone's tickets.
The milestone PR opens only after this passes, with the evidence in its description.

Requirements on your side: none beyond the build VM. The compose file, Dockerfiles, healthchecks, E2E harness, and CI skeleton are all delivered by Milestone 1's own infrastructure tickets.

After merge: the technical-debt review

The integrated E2E stage above proves the milestone branch works — it doesn't check whether the code is actually good, or whether it fully delivers what the milestone set out to do. That's a separate step, gated behind clicking Approve on the Milestones page after the PR is merged: a live AI review session checks the merged PR against the milestone's own scope and the project's architecture/understanding documents, and flags real technical debt — missing test coverage, security gaps, dead code, overly complex functions. A clean result approves the milestone; real findings become new tickets instead, and approval waits until those are resolved. See the auto-deliver guide for the full flow.