Skip to content

feat(cache): two-tier cache + per-node request coalescing - #1

Merged
java-jaydev merged 9 commits into
mainfrom
phase2-cache-coalescing
Jul 16, 2026
Merged

feat(cache): two-tier cache + per-node request coalescing#1
java-jaydev merged 9 commits into
mainfrom
phase2-cache-coalescing

Conversation

@java-jaydev

Copy link
Copy Markdown
Owner

Why

Market-data calls hit a rate-limited upstream. Bursts of identical requests (the common case when several agents ask for the same quote) each cost an upstream call, and slow-changing data was re-fetched as aggressively as live quotes.

What

A two-tier read-through cache in front of the Toss Open API, with per-node single-flight coalescing.

  • L1 — Caffeine near-cache (in-process). get(key, loader) is atomic per key, so concurrent identical requests on a node collapse to one upstream load. L1 honors the caller's per-type TTL via Caffeine variable expiry, so a single node caches correctly with no Redis deployed.
  • L2 — Redis shared cache (opt-in, TOSS_CACHE_L2=true). Adds cross-instance sharing. Any Redis error degrades to a cache miss and never breaks a tool call.
  • Per-type TTLs: quotes/orderbook 2s, trades 3s, intraday candles 10s, daily candles 1h, stock info 6h.
  • Cache keys normalize comma-separated symbols (trim + sort), so 005930,000660 and 000660,005930 share one entry.

Tools still return the raw upstream JSON string — the cache is transparent to the MCP contract.

Scope

L1 single-flight is per-node. Cross-node request coalescing is not implemented; the shared L2 narrows but does not eliminate the concurrent-miss window across instances. HTTP (Streamable) transport is deferred to Phase 2.5.

Testing

18 tests. RedisL2CacheIT uses Testcontainers (disabledWithoutDocker = true) and is skipped locally without a Docker daemon — this PR run is its first execution on a Docker-enabled runner.

Suite Covers
MarketDataCacheTest L1 hit/miss, single-flight
MarketDataCacheTtlTest per-type TTL via variable expiry (time-controlled Ticker)
MarketDataServiceTest per-type TTL selection, symbol normalization
RedisL2CacheDegradeTest Redis failure degrades to miss
RedisL2CacheIT real Redis round-trip + TTL expiry (Testcontainers)
CacheWiringTest exactly one L2Cache bean across all three property modes

@java-jaydev
java-jaydev merged commit e70316e into main Jul 16, 2026
1 check passed
@java-jaydev
java-jaydev deleted the phase2-cache-coalescing branch July 16, 2026 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant