Skip to content

Add CTM comparison site test cases (pricing, ranking, errors, flags)#6

Open
devin-ai-integration[bot] wants to merge 1 commit into
mainfrom
devin/1777910862-ctm-test-cases
Open

Add CTM comparison site test cases (pricing, ranking, errors, flags)#6
devin-ai-integration[bot] wants to merge 1 commit into
mainfrom
devin/1777910862-ctm-test-cases

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

Summary

Adds a single comprehensive review-ready test-case markdown file for the comparison-site backend. No production code, requirements docs, existing feature files, or application.yml were modified — this PR only adds one new file under docs/test-cases/.

Scope covered:

  • Quote Generation & Pricing (DefaultPricingEngine) — age, mileage, car-value, claims factors; deterministic seeded ±2% noise; monthly = round(annual / 12 × 1.05)
  • Ranking & Scoring — DefaultRankingService (price 50% / rating 30% / features 20%) vs ExperimentalRankingService (price 30% / rating 35% / features 15% / value-for-money 20%); isBestPrice and isRecommended flags; score normalization 0–1
  • Error Handling & Validation (GlobalExceptionHandler) — MethodArgumentNotValidException → 400, ConstraintViolationException → 400, generic Exception → 500; all include timestamp; no stack traces leak
  • Feature Flags & Admin — experimental-ranking toggle, map-enabled toggle, health check endpoint; admin endpoint protection (BACKLOG #16) gap documented

Output file:

  • docs/test-cases/ctm-comparison-test-cases.md

Total test cases: 115

Sub-area Positive Negative Boundary Exception Security Total
Quote endpoint (TC-QUOTE-NNN) 2 5 16 2 2 27
Pricing engine (TC-PRICING-NNN) 2 2 37 1 1 43
Ranking & scoring (TC-RANK-NNN) 6 3 5 1 1 16
Error handling (TC-ERR-NNN) 2 5 2 1 3 13
Feature flags & admin (TC-FLAG-NNN) 8 2 2 1 3 16
Totals 20 17 62 6 10 115

Every age / mileage / car-value bracket boundary is covered with TWO test cases (one each side of the threshold), per the scope:

  • Age: 20, 21, 24, 25, 29, 30, 65, 66, 75, 76 (TC-PRICING-005..014)
  • Mileage: 5000, 5001, 8000, 8001, 12000, 12001, 20000, 20001, 30000, 30001 (TC-PRICING-015..024)
  • Car value: 4999, 5000, 14999, 15000, 29999, 30000, 49999, 50000 (TC-PRICING-025..032)
  • Claims linearity: 0, 1, 2, 5 (TC-PRICING-033..036)
  • Monthly formula: 3 example annual values including £1000 (not divisible by 12) and £1234.56 (TC-PRICING-037..039)
  • Deterministic noise: same input → same output across two calls (TC-PRICING-040); ±2% envelope (TC-PRICING-041)

The file follows the spec layout: title + summary, scope/traceability table, coverage matrix, BDD gherkin scenarios with proper tags (@positive/@negative/@boundary/@exception/@security plus suite tags @smoke/@regression/@critical and feature-area tags @quote/@pricing/@ranking/@error-handling/@feature-flag/@admin/@deterministic), traditional tabular section, known defects subsection, and the QA Review Checklist.

Known Defects Covered

No explicit BUG-NNN IDs were assigned in this scope. Documented BACKLOG gaps that have explicit security/regression coverage:

  • BACKLOG.md item #16 (admin auth missing) — covered by TC-FLAG-014 and TC-FLAG-015 (security; flagged as documented gap with current vs expected behavior)
  • BACKLOG.md item #19 (input sanitization for postcode) — covered by TC-QUOTE-026 (XSS payload rejection at validation)

Review & Testing Checklist for Human

  • Confirm AC/BR identifiers in the traceability table match the team's preferred naming convention (currently AC-Q-001..005, AC-R-001..003, AC-E-001..004, AC-F-001..006, BR-PRICE-*, BR-RANK-*, BR-SEC-*)
  • Spot-check a handful of bracket-boundary expected values against DefaultPricingEngine source to confirm interpretation matches (e.g., age 21 → 1.5, mileage 5001 → 0.9, car value 5000 → 1.0)
  • Verify the QA Review Checklist appended at the end of the file matches your team's template

Notes

  • Only one file was added: docs/test-cases/ctm-comparison-test-cases.md
  • No production code, requirement docs, existing feature files, or application.yml were modified
  • The file is markdown-only; CI checks that compile/test the backend should be unaffected

Link to Devin session: https://bobby-demo.devinenterprise.com/sessions/ec923b7b585344ddaf2119e5d60e5767
Requested by: @bsmitches

@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants