Goal
Move the diagnosis cache from a per-process in-memory dict to a shared managed
Redis (Redis Cloud free tier), so the cache works correctly in production with
multiple workers / replicas, while keeping the in-memory dict as the offline /
$0 fallback.
Follow-up to #45 (in-memory cache) and PR #57.
Why the current cache is not production-ready
The cache today is a module-level dict _CACHE in agent/diagnose.py
((machine, fault_code) -> (expiry, Diagnosis)), which lives in one process's
RAM. That is fine for the single-process demo (the MQTT listener runs as a
thread inside the API process, so they share one cache), but breaks in prod:
- Multi-worker. Prod runs
uvicorn/gunicorn --workers N or multiple K8s
replicas. Each process has its own _CACHE. A fault cached in worker A is a
miss in worker B, so the hit rate collapses to ~1/N and the LLM is re-called.
- Restarts / deploys. The cache is empty after every restart, rolling
deploy, crash, or autoscale event. Always cold.
- No shared invalidation. When a repair outcome updates the knowledge base,
there is no way to invalidate "machine X fault Y" across all workers, so a
stale diagnosis can be served.
- No bounded eviction. No max size and no active sweep (expired entries are
only overwritten on the next miss for the same key). Fine at demo scale, a
slow leak if the key space grows.
Proposed solution: Redis Cloud (managed, free tier, no Docker)
- Redis Cloud (Redis Inc.): free tier ~30 MB, TLS, no Docker / no VM to run.
Gives a rediss:// connection URL. (Upstash serverless is an equivalent
alternative; either works. Free-tier limits drift, confirm at signup.)
- Shared across all workers / replicas, TTL handled server-side, supports LRU
eviction (maxmemory-policy allkeys-lru).
Implementation sketch
uv add redis
config.py: add redis_url: str = "" (empty = Redis disabled -> dict
fallback). Also reuse the existing diagnose_cache_ttl.
agent/diagnose.py cache layer, keyed the same way:
key = f"diag:{machine}:{fault_code}"
# read: cached = r.get(key); Diagnosis.model_validate_json(cached) if cached
# write: r.setex(key, ttl, diag.model_dump_json()) # TTL server-side
- Graceful fallback (hard requirement, per CLAUDE.md): wrap every Redis
call in try/except. On connect error / timeout, fall back to the in-memory
dict. Redis being unreachable must never break diagnosis.
.env: REDIS_URL=rediss://... in prod; leave empty for the offline demo.
TTL
- Duration stays
diagnose_cache_ttl (default 600s, 0 disables).
- With Redis, pass the duration to
SETEX and Redis expires the key
server-side. No time.monotonic() bookkeeping on the Redis path.
Acceptance criteria
Notes
- Optional durability: a Postgres table
(machine, fault_code, diagnosis_json, expires_at) is a zero-new-infra alternative if a managed Redis is not
desired (slower, but durable + queryable).
- Not needed for the hackathon demo (single process). This is the prod-ready
upgrade path.
Goal
Move the diagnosis cache from a per-process in-memory dict to a shared managed
Redis (Redis Cloud free tier), so the cache works correctly in production with
multiple workers / replicas, while keeping the in-memory dict as the offline /
$0 fallback.
Follow-up to #45 (in-memory cache) and PR #57.
Why the current cache is not production-ready
The cache today is a module-level dict
_CACHEinagent/diagnose.py(
(machine, fault_code) -> (expiry, Diagnosis)), which lives in one process'sRAM. That is fine for the single-process demo (the MQTT listener runs as a
thread inside the API process, so they share one cache), but breaks in prod:
uvicorn/gunicorn --workers Nor multiple K8sreplicas. Each process has its own
_CACHE. A fault cached in worker A is amiss in worker B, so the hit rate collapses to ~1/N and the LLM is re-called.
deploy, crash, or autoscale event. Always cold.
there is no way to invalidate "machine X fault Y" across all workers, so a
stale diagnosis can be served.
only overwritten on the next miss for the same key). Fine at demo scale, a
slow leak if the key space grows.
Proposed solution: Redis Cloud (managed, free tier, no Docker)
Gives a
rediss://connection URL. (Upstash serverless is an equivalentalternative; either works. Free-tier limits drift, confirm at signup.)
eviction (
maxmemory-policy allkeys-lru).Implementation sketch
uv add redisconfig.py: addredis_url: str = ""(empty = Redis disabled -> dictfallback). Also reuse the existing
diagnose_cache_ttl.agent/diagnose.pycache layer, keyed the same way:call in try/except. On connect error / timeout, fall back to the in-memory
dict. Redis being unreachable must never break diagnosis.
.env:REDIS_URL=rediss://...in prod; leave empty for the offline demo.TTL
diagnose_cache_ttl(default 600s, 0 disables).SETEXand Redis expires the keyserver-side. No
time.monotonic()bookkeeping on the Redis path.Acceptance criteria
REDIS_URLset, two API workers share cache hits (fault diagnosedby one worker is a hit for the other).
REDIS_URLempty, behavior is identical to today (in-memory dict).logged, no crash.
SETEX); expired key triggers a freshdiagnosis.
Notes
(machine, fault_code, diagnosis_json, expires_at)is a zero-new-infra alternative if a managed Redis is notdesired (slower, but durable + queryable).
upgrade path.