Skip to content

QUALITY_GATES.md

Latest commit

 

History

History
283 lines (205 loc) · 14.3 KB

QUALITY_GATES.md

File metadata and controls

283 lines (205 loc) · 14.3 KB

Quality Gates — retort

Definition of done for each lifecycle phase. No work item may advance to the next phase until all quality gates for the current phase are satisfied.

Table of Contents

  1. Overview
  2. Phase 1: Discovery Gates
  3. Phase 2: Planning Gates
  4. Phase 3: Implementation Gates
  5. Phase 4: Validation Gates
  6. Phase 5: Ship Gates
  7. Gate Enforcement
  8. Exemption Process
  9. Metrics and Reporting

Overview

Quality gates are checkpoints that ensure work meets a defined standard before progressing. They exist at each of the five lifecycle phases defined in UNIFIED_AGENT_TEAMS.md.

Gate Evaluation

Each gate is evaluated as:

  • Pass: All criteria met. Work may advance to the next phase.
  • Conditional Pass: Minor criteria unmet but documented. Work may advance with a follow-up task created for the gaps.
  • Fail: One or more critical criteria not met. Work must not advance until resolved.

Phase 1: Discovery Gates

Purpose: Ensure the problem is well-understood before investing in a solution.

Required Gates

# Gate Criteria Verification
D1 Product docs updated docs/product/ contains problem statement and user stories for this work item File exists and is non-empty
D2 Issue created A tracking issue exists in the issue tracker with labels, assignee, and milestone Issue URL recorded in backlog
D3 Scope identified Affected teams and file scope patterns are documented Listed in backlog entry
D4 Stakeholders notified Relevant teams have been tagged or notified of the upcoming work Notification logged in state

Optional Gates

# Gate Criteria Verification
D5 User research User interviews or data analysis supporting the problem statement Link to research artifact
D6 Competitive analysis Review of how similar problems are solved elsewhere Document in docs/product/

Discovery Checklist

- [ ] Problem statement written in docs/product/
- [ ] User stories or acceptance criteria defined
- [ ] Tracking issue created with appropriate labels
- [ ] Affected teams identified (minimum: primary team)
- [ ] Initial scope estimate documented

Phase 2: Planning Gates

Purpose: Ensure the solution design is sound and agreed upon before building.

Required Gates

# Gate Criteria Verification
P1 ADRs authored Architecture Decision Records created for significant technical decisions Files exist in docs/adr/ or docs/architecture/specs/adr/
P2 ADRs approved ADRs have been reviewed and approved by relevant team leads Approval recorded (comment or status)
P3 Specs complete Technical specification in docs/architecture/specs/ covers approach, API contracts, data models, and edge cases Spec document exists and reviewed
P4 Tasks broken down Work is decomposed into tasks assigned to specific teams Tasks listed in AGENT_BACKLOG.md
P5 Dependencies mapped Cross-team dependencies identified and sequenced Dependency table updated

Optional Gates

# Gate Criteria Verification
P6 Risk assessment Identified risks with mitigation strategies Section in spec document
P7 Prototype/spike Proof of concept for high-risk technical approaches Branch or document reference
P8 Capacity check Team has bandwidth for the estimated effort Sprint capacity confirmed

Planning Checklist

- [ ] ADR(s) written for key decisions
- [ ] ADR(s) approved by relevant reviewers
- [ ] Technical spec complete in docs/architecture/specs/
- [ ] API contracts defined (if applicable)
- [ ] Data model changes documented (if applicable)
- [ ] Tasks created and assigned in AGENT_BACKLOG.md
- [ ] Cross-team dependencies identified

Phase 3: Implementation Gates

Purpose: Ensure code quality and completeness before requesting review.

Required Gates

# Gate Criteria Verification
I1 /check passes Running /check (lint, type-check, unit tests) completes without errors Exit code 0
I2 Tests added New or modified code has corresponding test coverage Coverage diff shows no decrease
I3 Coverage threshold Test coverage meets or exceeds project minimum (default: 80%) Coverage report
I4 No lint violations Zero lint errors (warnings acceptable if pre-existing) Lint report clean
I5 Type-safe No TypeScript errors (for TS projects) or equivalent type checks pass Type-check exit code 0
I6 Conventional commits All commits follow conventional commit format Commit messages validated

Optional Gates

# Gate Criteria Verification
I7 Performance baseline No regression in benchmark results Benchmark comparison
I8 Bundle size check No unexpected increase in build output size Size report
I9 Documentation inline Public APIs have JSDoc/TSDoc comments Doc coverage tool

Implementation Checklist

- [ ] /check passes locally (lint + type-check + tests)
- [ ] New tests written for new/changed code
- [ ] Test coverage meets threshold (>= 80%)
- [ ] No new lint errors introduced
- [ ] Types are correct (no `any` escape hatches without justification)
- [ ] Commits follow conventional commit format
- [ ] Code follows project style guide

Phase 4: Validation Gates

Purpose: Ensure the solution is reviewed, tested, and ready for production.

Required Gates

# Gate Criteria Verification
V1 PR created Pull request opened with description, test plan, and linked issue PR URL exists
V2 CI passes All continuous integration checks pass (lint, test, build, type-check) CI status green
V3 Code review approved At least one approval from a team member with relevant expertise PR approval count >= 1
V4 No merge conflicts PR is rebased or merged with the target branch without conflicts Git status clean
V5 PR description complete Description includes: summary, test plan, breaking changes (if any) PR template filled

Optional Gates

# Gate Criteria Verification
V6 E2E tests pass End-to-end test suite passes against the PR branch E2E CI job green
V7 Manual testing UI or user-facing changes manually verified Tester sign-off
V8 Security scan No new vulnerabilities introduced Security scan report
V9 Accessibility audit UI changes meet WCAG 2.1 AA standards Audit report

Validation Checklist

- [ ] Pull request created with full description
- [ ] Issue linked in PR description
- [ ] All CI checks passing
- [ ] At least one code review approval
- [ ] No merge conflicts with target branch
- [ ] Test plan documented and executed
- [ ] Breaking changes documented (if applicable)

Phase 5: Ship Gates

Purpose: Ensure the change is deployed safely and operational documentation is current.

Required Gates

# Gate Criteria Verification
S1 PR merged Pull request merged to the target branch Merge commit exists
S2 Operations docs updated docs/operations/ reflects any new operational procedures, configuration changes, or monitoring requirements Docs updated
S3 CHANGELOG updated CHANGELOG.md has an entry for this change under the appropriate section Entry exists
S4 Deployment successful Change deployed to target environment without errors Deployment log
S5 Monitoring confirmed No anomalies in error rates, latency, or resource usage post-deploy Monitoring dashboard

Optional Gates

# Gate Criteria Verification
S6 Rollback tested Rollback procedure verified in staging Rollback log
S7 Stakeholder notification Relevant stakeholders informed of the change Notification sent
S8 Feature flag cleanup Temporary feature flags removed or documented for cleanup Flag audit

Ship Checklist

- [ ] Pull request merged
- [ ] docs/operations/ updated (if applicable)
- [ ] CHANGELOG.md entry added
- [ ] Deployment completed successfully
- [ ] Post-deploy monitoring reviewed (15-minute window minimum)
- [ ] Backlog item marked as completed

Gate Enforcement

Automated Enforcement

The following gates are enforced automatically through CI and tooling:

  • I1, I4, I5: Enforced by /check command and CI pipeline.
  • I3: Enforced by coverage thresholds in test configuration.
  • V2: Enforced by GitHub branch protection rules.
  • V3: Enforced by required review settings.
  • V4: Enforced by merge conflict detection.

Manual Enforcement

The following gates require human or AI agent verification:

  • D1-D4: Verified by the orchestrator during phase transition.
  • P1-P5: Verified during planning review.
  • S2-S5: Verified post-deployment by the shipping team.

Override Authority

Role Can Override
Team Lead Optional gates for their own team
Orchestrator Conditional pass on non-critical required gates
Project Owner Any gate (must document reason)

Exemption Process

When a quality gate cannot be satisfied and an exemption is needed:

  1. Document the reason in the task notes explaining why the gate cannot be met.
  2. Identify the risk of proceeding without the gate.
  3. Create a follow-up task to address the gap within the next sprint.
  4. Get approval from the appropriate override authority (see table above).
  5. Record the exemption in the task history for audit purposes.

Metrics and Reporting

Tracked Metrics

Metric Definition Target
Gate pass rate Percentage of gates passed on first attempt > 85%
Phase cycle time Average time to move through a single phase < 2 days
Exemption rate Percentage of gates exempted vs. total evaluations < 5%
Rework rate Percentage of items sent back to a previous phase < 10%

Reporting Cadence

  • Per-task: Gate results logged in .claude/state/orchestrator.json after each evaluation.
  • Per-sprint: Aggregate metrics reviewed during sprint retrospective.
  • Per-milestone: Trend analysis to identify systemic quality gaps.

Quality gates are maintained by Retort. See UNIFIED_AGENT_TEAMS.md for the lifecycle model that these gates support.