Skip to content

perf(round2): Redis route cache + distributed cache-fill lease + distributed gateway limiter - #12

Merged
llfzzz merged 1 commit into
mainfrom
llfzzz/perf-redis-cache-distributed-limiter
Jul 23, 2026
Merged

perf(round2): Redis route cache + distributed cache-fill lease + distributed gateway limiter#12
llfzzz merged 1 commit into
mainfrom
llfzzz/perf-redis-cache-distributed-limiter

Conversation

@llfzzz

@llfzzz llfzzz commented Jul 23, 2026

Copy link
Copy Markdown
Owner

What & why

Second performance/scalability round on top of the merged PR #11 (S49). Implements the already-approved architecture (see docs/adr/0005): a real Redis read cache with avalanche/stampede protection, a narrowly-scoped distributed cache-fill lease, and a genuinely distributed gateway rate limiter — preserving every business, security, idempotency, and state-machine guarantee. Everything new is optional and config-reversible, and adds zero footprint to the low-memory prod demo host by default.

./scripts/verify.sh is green: all 15 backend modules (~386 tests, including live-Redis Testcontainers integration tests), both frontends typecheck + build.

Changes (6 slices)

Area Change
Two Redis roles State Redis (existing, stays noeviction) now also holds the distributed rate-limit counters. A new optional, disposable Cache Redis — separate instance via docker-compose.cache.yml (allkeys-lfu, no persistence, port 6380) — holds only derived data (route cache + leases). Off by default. Instance-level isolation, never logical DBs.
Map route Redis cache Cache-aside over the MySQL snapshot: validate → Redis → MySQL(populate) → provider(CB) → save → populate. Versioned cache:map:route:v1:<sha256> String; TTL = base + jitter capped by the snapshot's remaining freshness; oversize/decode protection; endpoints never cached; every Redis failure degrades to a miss and can never mask credential/city/coordinate/provider errors or fabricate a route.
Distributed cache-fill lease SET NX PX + atomic owner-compare-and-delete Lua, bounded lease + bounded loser wait. Cache-fill dedup only — never seat/order/payment/identity/authz correctness.
Distributed gateway limiter RATE_LIMIT_BACKEND=redis now works — reactive Lua on the state Redis, no blocking Redis call on the Netty event loop, Retry-After = real window remainder. Misconfig fails startup (never silent-memory). Redis outage → metered degraded mode (bounded local emergency, or fail-closed for sensitive buckets), never silently unlimited. In-memory retained for single-instance demo.
Tests Unit (TTL jitter/cap) + Testcontainers against real Redis: cache put/get, malformed-repair, oversize, lease exclusivity/owner-release/expiry, two map instances → one provider call, two limiter instances → one shared quota, concurrency, window reset, Retry-After, degraded mode. @Testcontainers(disabledWithoutDocker=true) keeps mvn test green without a daemon.
Docs ADR-0005; AGENTS.md (S49 corrected to merged 6376e30, S50 added); architecture/operations/load-testing/security; .env.example; compose overlay.

Verification (separated, per the brief)

  • Code + unit/integration: done. verify.sh green; the distributed behaviors are proven by real-Redis Testcontainers tests, not mocks.
  • Runtime config: verified. docker compose -f docker-compose.yml -f docker-compose.cache.yml config validates; cache Redis is a separate allkeys-lfu instance on 6380.
  • Load evidence: blocked, not fabricated. The multi-instance comparison the DoD asks for (≥2 gateways / ≥2 map-services on shared Redis; before/after provider-call + route_snapshots-read counts; hit rate; p50/p95/p99; Redis memory; 429 curves) needs a non-prod staging target with adequate hardware this session didn't have. scripts/loadtest/* are ready and production-denylisted. Production domain was never touched.

Rollback

Pure config: MAP_ROUTE_CACHE_REDIS_ENABLED=false → MySQL/provider path; RATE_LIMIT_BACKEND=memory → single-gateway limiter; dropping the cache overlay doesn't stop the demo stack. No DB rollback.

🤖 Generated with Claude Code

… distributed gateway limiter

Second scalability round (ADR-0005). Preserves all business/security/idempotency/state-machine
guarantees; everything new is optional and config-reversible. ./scripts/verify.sh green, including
real-Redis Testcontainers integration tests.

- Two Redis roles as two instances (never logical DBs): the existing STATE Redis stays `noeviction`
  and now also holds the distributed rate-limit counters; a new optional, disposable CACHE Redis
  (docker-compose.cache.yml: allkeys-lfu, no persistence, port 6380) holds only derived data. Off by
  default so the low-mem prod host is unaffected.
- map-service Redis route read-cache (cache-aside over the MySQL snapshot): versioned String key
  cache:map:route:v1:<sha256>, TTL = base+jitter capped by the snapshot's remaining freshness,
  oversize/decode protection, endpoints never cached, every Redis failure degrades to a miss and can
  never mask credential/city/coordinate/provider errors or fabricate a route.
- Distributed cache-fill lease (cache Redis): SET NX PX + atomic owner-compare-and-delete Lua,
  bounded lease + bounded loser wait; cache-fill dedup only, never business correctness.
- Distributed gateway limiter: RATE_LIMIT_BACKEND=redis now works via reactive Lua on the state
  Redis (no blocking call on the Netty event loop); Retry-After = real window remainder; misconfig
  fails startup instead of silently reverting to memory; Redis outage -> metered degraded mode
  (bounded local emergency, or fail-closed for sensitive buckets), never silently unlimited.
- Tests: unit (TTL jitter/cap) + Testcontainers (cache put/get, malformed-repair, oversize, lease
  exclusivity/owner-release/expiry, two map instances -> one provider call, two limiter instances ->
  one shared quota, concurrency, window reset, Retry-After, degraded mode). disabledWithoutDocker so
  mvn test stays green without a daemon.
- Docs: ADR-0005, AGENTS.md (S49 corrected to merged, S50 added), architecture/operations/
  load-testing/security, .env.example, compose overlay.

Load evidence (multi-instance provider-call/hit-rate/p95/Redis-memory/429 comparison) needs a
non-prod staging target with adequate hardware and is not captured here; the distributed behavior is
proven by the Testcontainers tests. Production was never touched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 23, 2026 04:47

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@llfzzz
llfzzz merged commit e59c5a7 into main Jul 23, 2026
1 check passed
@llfzzz
llfzzz deleted the llfzzz/perf-redis-cache-distributed-limiter branch July 23, 2026 05:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants