treeify logo
Awesome Test Case Design65 review gates metrics traceability

Review Gates

Review Gates

Ship faster by reducing ambiguity and raising evidence. These gates define when a change moves forward, what artifacts must exist, and how metrics tie back to requirements.


Why gates?

  • Catch gaps early (spec → design → code → ship).
  • Create traceable evidence for decisions.
  • Make quality visible: contracts, tests, a11y, perf, security, privacy, i18n.
  • Standardize what “ready” means across teams.

Gate taxonomy (lightweight)

G0 — Intake

  • Problem statement, value, owners, scope/out-of-scope.

G1 — Spec Ready (DoR)

  • Contracts sketched (I/O, states, errors).
  • MAE acceptance drafted with oracles.
  • NFR budgets defined (p95/p99, retries, a11y, privacy, i18n, compatibility).
  • Metrics & instrumentation defined.

G2 — Design Ready

  • MAE screens: Empty/Loading/Error designed.
  • Focus order map; microcopy with message IDs.
  • Responsive & RTL notes; token usage; reduced-motion variants.

G3 — Impl Ready

  • Test data/fixtures; flags; environments; runbooks.
  • Observability wiring plan (logs/metrics/traces).
  • Security & privacy checklist planned.

G4 — Pre-merge

  • Contracts validated by tests.
  • Functional + API + a11y + perf quick passes green.
  • Security, compatibility, i18n checks green.
  • Evidence attached to PR.

G5 — Ship / Launch

  • Canary plan + guardrails; rollback.
  • Docs updated; on-call briefed; dashboards live.

G6 — Post-launch

  • Error budgets healthy; KPIs move as expected.
  • Postmortem if thresholds breached; action items tracked.

Keep gates fast and automatable. Prefer short, high-signal checks.


Evidence ledger (per gate)

  • G1: one-pager, contracts draft, MAE acceptance, metrics plan.
  • G2: annotated screens, focus map, locale/RTL samples.
  • G3: test data list, flags/config, runbooks links.
  • G4: test reports (functional/API/a11y), perf snapshot, security scans, logs/traces.
  • G5: canary dashboard, alert rules, rollback doc.
  • G6: KPI dashboard, incident notes, learnings.

Automation (CI/CD gates)

Recommended jobs

  • Schema/contract diff → block incompatible changes.
  • Unit/integration/e2e → MAE smoke.
  • API contract tests → request/response shapes, message IDs.
  • A11y (axe/pa11y) → threshold of 0 critical.
  • i18n pseudo-localization + screenshots → no clipping.
  • Perf micro-bench / route p95 guardrails.
  • Security: SAST, SCA/SBOM, secret scan; CSP/headers snapshot.
  • Lint for logs without PII; check for correlation_id/msgid.
  • Coverage by risk: core modules ≥ target.

Decision policy (ship/hold)

  • Ship when all G4 checks pass and guardrails are defined for G5.
  • Hold when:
    • Contracts unclear; MAE missing.
    • Critical a11y/security/perf failures.
    • No rollback or on-call plan.

Traceability

Link Requirement → Acceptance → Test → Signal → Owner.

Minimal fields

  • req_id
  • accept_id
  • test_id
  • signal (metric/log/trace/msgid)
  • owner

Use the CSV seed below and keep it in-repo.


Dashboards (standard set)

  • User journeys: success rate, p95/p99, error rate.
  • Perf: route histograms; exemplars (trace links).
  • A11y/i18n: violations count; missing keys; pseudo-loc fail count.
  • Security: auth failures, rate limits, CSP reports, webhook signatures.
  • Reliability: error budgets, burn rates, queue age.
  • Business: north-star + guardrails.

PR template (gate-aligned)

Feature: <name>    Req: <req_id>   Owners: <PM/Eng/QA/Design/SRE>

G1 — Spec Ready
[ ] Contracts (I/O, states, errors, message IDs)
[ ] MAE acceptance + oracles
[ ] NFR budgets (perf, resiliency, security, privacy, a11y, i18n, compatibility)
[ ] Metrics & instrumentation plan

G2 — Design Ready
[ ] MAE screens (Empty/Loading/Error)
[ ] Focus order map, message IDs
[ ] Responsive + RTL + reduced-motion notes

G3 — Impl Ready
[ ] Test data & flags
[ ] Observability plan (logs/metrics/traces)
[ ] Runbooks/rollout outline

G4 — Pre-merge
[ ] Functional/API tests green
[ ] A11y quick pass
[ ] Perf snapshot
[ ] Security checks
[ ] i18n pseudo-loc screenshots
[ ] Evidence attached (links)

G5 — Ship
[ ] Canary plan + guardrails
[ ] Rollback runbook
[ ] Dashboards live; alerts wired

G6 — Post-launch
[ ] KPIs + error budgets ok
[ ] Incidents & learnings logged

Gate config (YAML template)

gates:
  - id: G1
    name: Spec Ready
    required:
      - contracts
      - mae_acceptance
      - nfr_budgets
      - metrics_plan
  - id: G2
    name: Design Ready
    required:
      - mae_screens
      - focus_map
      - i18n_notes
  - id: G3
    name: Impl Ready
    required:
      - test_data
      - observability_plan
      - runbooks
  - id: G4
    name: Pre-merge
    required:
      - tests_functional
      - tests_api
      - checks_a11y
      - checks_perf
      - checks_security
      - checks_i18n
      - evidence_links
  - id: G5
    name: Ship
    required:
      - canary_plan
      - rollback
      - dashboards_alerts
  - id: G6
    name: Post-launch
    required:
      - kpi_check
      - error_budget_check
      - learnings_logged

CSV seeds

Gate registry

gate_id,name,owner
G0,Intake,PM
G1,Spec Ready,PM
G2,Design Ready,Design
G3,Impl Ready,Eng
G4,Pre-merge,QA
G5,Ship,Eng+SRE
G6,Post-launch,PM+SRE

Evidence register

gate_id,artifact,type
G1,one_pager,doc
G1,contracts_draft,doc
G2,annotated_screens,images
G2,focus_map,images
G3,test_data,csv
G3,runbook,doc
G4,test_reports,pdf
G4,perf_snapshot,images
G4,security_scans,zip
G4,logs_traces,links
G5,canary_dashboard,link
G5,rollback_doc,doc
G6,kpi_dashboard,link
G6,postmortem,doc

Traceability matrix

req_id,accept_id,test_id,signal,owner
REQ-101,ACC-101,API-Create-001,metric:orders_create_p95,Checkout
REQ-102,ACC-201,UI-Error-RTL-001,log:MSG.validation.email,Web
REQ-103,ACC-301,Perf-Spike-002,trace:checkout_slow_exemplar,SRE

Guardrails (canary)

metric,threshold,action
error_rate_pct,> 1%,rollback
latency_p95_ms,> +20 vs base,hold
unsafe_rate_pct,> 0,rollback
kpi_uplift_pct,< 0,hold

Review checklist (quick gate)

  • Gates defined and understood by owners
  • Evidence ledger populated per gate
  • CI/CD automation mapped to gates
  • Dashboards exist and are linked in PRs
  • Traceability CSV updated
  • Guardrails + rollback documented

  • Functional Coverage → ../60-checklists/functional-coverage.md
  • API Coverage → ../60-checklists/api-coverage.md
  • Performance → ../60-checklists/performance-review.md
  • Security → ../60-checklists/security-review.md
  • Compatibility → ../60-checklists/compatibility-review.md
  • Accessibility → ../60-checklists/accessibility-review.md
  • For PMs → ../57-cross-discipline-bridges/for-pms.md
  • For SREs → ../57-cross-discipline-bridges/for-sres.md