Skip to content

Shared diagnosis cache via Redis Cloud (prod-ready) #60

Description

@Goodnight77

Goal

Move the diagnosis cache from a per-process in-memory dict to a shared managed
Redis (Redis Cloud free tier), so the cache works correctly in production with
multiple workers / replicas, while keeping the in-memory dict as the offline /
$0 fallback.

Follow-up to #45 (in-memory cache) and PR #57.

Why the current cache is not production-ready

The cache today is a module-level dict _CACHE in agent/diagnose.py
((machine, fault_code) -> (expiry, Diagnosis)), which lives in one process's
RAM. That is fine for the single-process demo (the MQTT listener runs as a
thread inside the API process, so they share one cache), but breaks in prod:

  • Multi-worker. Prod runs uvicorn/gunicorn --workers N or multiple K8s
    replicas. Each process has its own _CACHE. A fault cached in worker A is a
    miss in worker B, so the hit rate collapses to ~1/N and the LLM is re-called.
  • Restarts / deploys. The cache is empty after every restart, rolling
    deploy, crash, or autoscale event. Always cold.
  • No shared invalidation. When a repair outcome updates the knowledge base,
    there is no way to invalidate "machine X fault Y" across all workers, so a
    stale diagnosis can be served.
  • No bounded eviction. No max size and no active sweep (expired entries are
    only overwritten on the next miss for the same key). Fine at demo scale, a
    slow leak if the key space grows.

Proposed solution: Redis Cloud (managed, free tier, no Docker)

  • Redis Cloud (Redis Inc.): free tier ~30 MB, TLS, no Docker / no VM to run.
    Gives a rediss:// connection URL. (Upstash serverless is an equivalent
    alternative; either works. Free-tier limits drift, confirm at signup.)
  • Shared across all workers / replicas, TTL handled server-side, supports LRU
    eviction (maxmemory-policy allkeys-lru).

Implementation sketch

  1. uv add redis
  2. config.py: add redis_url: str = "" (empty = Redis disabled -> dict
    fallback). Also reuse the existing diagnose_cache_ttl.
  3. agent/diagnose.py cache layer, keyed the same way:
    key = f"diag:{machine}:{fault_code}"
    # read:  cached = r.get(key); Diagnosis.model_validate_json(cached) if cached
    # write: r.setex(key, ttl, diag.model_dump_json())   # TTL server-side
  4. Graceful fallback (hard requirement, per CLAUDE.md): wrap every Redis
    call in try/except. On connect error / timeout, fall back to the in-memory
    dict. Redis being unreachable must never break diagnosis.
  5. .env: REDIS_URL=rediss://... in prod; leave empty for the offline demo.

TTL

  • Duration stays diagnose_cache_ttl (default 600s, 0 disables).
  • With Redis, pass the duration to SETEX and Redis expires the key
    server-side. No time.monotonic() bookkeeping on the Redis path.

Acceptance criteria

  • With REDIS_URL set, two API workers share cache hits (fault diagnosed
    by one worker is a hit for the other).
  • With REDIS_URL empty, behavior is identical to today (in-memory dict).
  • Redis unreachable at runtime -> diagnosis still works via dict fallback,
    logged, no crash.
  • TTL honored server-side (SETEX); expired key triggers a fresh
    diagnosis.
  • Tests cover: Redis hit path (mocked), fallback-on-error path, TTL disable.

Notes

  • Optional durability: a Postgres table (machine, fault_code, diagnosis_json, expires_at) is a zero-new-infra alternative if a managed Redis is not
    desired (slower, but durable + queryable).
  • Not needed for the hackathon demo (single process). This is the prod-ready
    upgrade path.

Metadata

Metadata

Assignees

Labels

cacheDiagnosis / result cachingfuture-enhancementsPost-hackathon / future scope

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions