Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/tag-read-mode.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@leejpsd/nextjs-cache-handler": minor
---

New `tagReadMode` option for the `'use cache'` handler (#31). The default, `"scan"`, is unchanged: `refreshTags()` SCANs the namespace's tag markers into a local mirror on every request. Redis SCAN walks the whole keyspace regardless of the MATCH pattern, so that cost grows with the total number of keys in the database, not with the number of tags: about 20 ms per request at 20k keys and 65–170 ms at 100k against a local Redis. `tagReadMode: "on-demand"` turns the scan off; `get()` and `getExpiration()` instead `MGET` the markers for exactly the tags they are given (the entry's own tags plus the route's soft tags), one extra round trip per `get()`, which is how the singular handler and `getExpiration()` already worked. `tagPubSub` works in either mode. Thanks @royvou for the report and the proposal.
44 changes: 44 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,7 @@ interface CacheHandlerOptions {
staleWhileRevalidate?: boolean; // default: true (cache-components only)
singleFlight?: boolean; // default: false — see "Single-flight refresh lock" below
singleFlightLockTtlSec?: number; // default: 10
tagReadMode?: "scan" | "on-demand"; // default: "scan" — see "Tag reads at scale" below
isBuildPhase?: () => boolean; // override PHASE_PRODUCTION_BUILD detection
hashTag?: boolean; // default: false (set true on Redis Cluster)
memoryMaxEntries?: number; // default: 1000 — LRU cap for the in-memory fallback
Expand Down Expand Up @@ -331,6 +332,49 @@ for the full state machine and a reference to the Lua script body.

---

## Tag reads at scale: `tagReadMode` (v1.1)

The `'use cache'` handler has to learn about invalidations made by other
instances. By default (`tagReadMode: "scan"`) `refreshTags()` runs once per
request and SCANs the namespace's tag markers into a local mirror, and
`get()` checks tags against that mirror without touching Redis. Redis SCAN
walks the whole keyspace regardless of the MATCH pattern, so that cost grows
with the **total number of keys in the database**, not with the number of
tags:

| keys in Redis | tag markers | scan: `refreshTags()` per request | on-demand: `get()` including its `MGET` |
|---|---|---|---|
| 2.5k | 500 | 3–4 ms | 0.1 ms |
| 20.5k | 500 | 17–35 ms | 0.1 ms |
| 100k | 500 | 65–166 ms | 0.1 ms |

p50 over several runs, local Redis 7 over redis@5
(`scripts/bench-tag-read.mjs`). On a networked Redis multiply by the
round-trip time: at 20k keys the scan is roughly 400 round trips, the
on-demand read is one. A page with many `'use cache'` calls pays one `MGET`
per call, but redis@5 and ioredis pipeline parallel reads: 50 parallel
`get()` took 0.7 ms on-demand, against 17 ms (20k keys) and 63 ms (100k)
for the scan followed by the same 50 reads.

`tagReadMode: "on-demand"` turns the scan off. `get()` reads the markers for
exactly the entry's own tags plus the route's soft tags with one `MGET`, and
`getExpiration()` does the same, so Redis stays the single source of truth
and a request pays one extra round trip per cached entry instead of a scan.
This is how the singular `cacheHandler` has always read tags. `tagPubSub`
works in either mode.

```js
// cache-components.cjs
module.exports = createCacheComponentsHandler({
client: { type: "redis", url: process.env.REDIS_URL },
tagReadMode: "on-demand",
});
```

Reported in [#31](https://github.com/leejpsd/nextjs-cache-handler/issues/31).
The default stays `"scan"` for now; it may become `"on-demand"` in a later
minor once it has run in more deployments.

## OpenTelemetry instrumentation (v0.2)

The handler doesn't bundle `@opentelemetry/api` (zero runtime
Expand Down
1 change: 1 addition & 0 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ ctx contract are accepted (`ctx.cacheControl.revalidate` / `kind` and
| `isBuildPhase` | `() => boolean` | `NEXT_PHASE === "phase-production-build"` | build-phase gate override |
| `hashTag` | `boolean` | `false` | wrap namespace in `{}` — **required on Redis Cluster** (multi-key Lua) |
| `tagPubSub` | `boolean` | `false` | push-based cross-instance tag propagation (plural handler; ~3ms); scan polling remains the safety net; unavailable on Cluster |
| `tagReadMode` | `"scan" \| "on-demand"` | `"scan"` | plural handler only. `scan`: `refreshTags()` SCANs the namespace's tag markers once per request (cost grows with total keys in Redis, not with tags: ~20 ms at 20k keys, 65–170 ms at 100k on a local Redis). `on-demand`: no scan; `get()` and `getExpiration()` `MGET` exactly the tags they need (one extra round trip per `get()`). See [#31](https://github.com/leejpsd/nextjs-cache-handler/issues/31) |
| `onMetric` | `(event: MetricEvent) => void` | — | telemetry hook; emitter errors are swallowed |
| `logger` | `Logger` | console (warn+) | injectable 4-level logger |

Expand Down
13 changes: 11 additions & 2 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,13 +104,22 @@ set(cacheKey, pendingEntry: Promise<CacheEntry>)
For multi-instance deployments:

1. `updateTags()` writes a per-tag timestamp to Redis (the marker key).
2. `refreshTags()` is called before each request. It scans
`next-cache:tag-expiration:{ns}:*` and updates `localTagTimestamps`.
2. `refreshTags()` is called before each request. In the default
`tagReadMode: "scan"` it scans `next-cache:tag-expiration:{ns}:*` and
updates `localTagTimestamps`. Redis SCAN walks the whole keyspace, so
this costs about 20 ms per request at 20k keys and 65–170 ms at 100k
(local Redis), whatever the number of tags.
3. `getExpiration([t1, t2])` returns `Math.max(timestamps)` from the local
mirror.
4. `get(cacheKey, softTags)` consults `localTagTimestamps` and treats the
entry as stale if any soft tag was invalidated after `entry.timestamp`.

With `tagReadMode: "on-demand"` (#31) step 2 is a no-op and steps 3 and 4
read the markers for exactly the tags they are given with one `MGET`,
merging them into the local mirror, so Redis stays the source of truth and
no request pays for a scan. That is also how the singular handler has always
read tags.

The reference implementation in next-redis-cache-demo measured **6.4 ms**
mean propagation across two ECS tasks under sustained load with this
exact pattern.
Expand Down
158 changes: 158 additions & 0 deletions scripts/bench-tag-read.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
#!/usr/bin/env node
// Reproduces #31: what one request pays for tag reads in the 'use cache'
// handler, in `tagReadMode: "scan"` (default) and `"on-demand"`, against a
// Redis holding N keys and T tag markers.
//
// npm run build
// REDIS_URL=redis://127.0.0.1:6390 node scripts/bench-tag-read.mjs [keys] [tags] [runs]
//
// The target database is FLUSHED. The script refuses to run against a
// non-empty database unless BENCH_FLUSH=1 is set.

import { performance } from "node:perf_hooks";

import { createClient } from "redis";

import { createCacheComponentsHandler } from "../dist/cache-components/index.js";

const N = Number(process.argv[2] ?? 20000);
const T = Number(process.argv[3] ?? 500);
const RUNS = Number(process.argv[4] ?? 30);
const url = process.env.REDIS_URL ?? "redis://127.0.0.1:6390";
const ns = "bench";

if (![N, T, RUNS].every(Number.isFinite)) {
console.error("usage: bench-tag-read.mjs [keys] [tags] [runs]");
process.exit(1);
}

const raw = createClient({ url });
raw.on("error", () => {});
await raw.connect();
if ((await raw.dbSize()) > 0 && process.env.BENCH_FLUSH !== "1") {
console.error(`${url} is not empty; set BENCH_FLUSH=1 to flush it.`);
await raw.quit();
process.exit(1);
}
await raw.flushAll();

// N entry-shaped keys in the handler's namespace. SCAN cost depends on the
// total key count, so the values can stay small.
const filler = "x".repeat(256);
for (let i = 0; i < N; i += 1000) {
const m = raw.multi();
for (let j = i; j < Math.min(i + 1000, N); j++) {
m.set(`next-cache:entry:${ns}:/page-${j}`, filler);
}
await m.exec();
}

const common = { client: { type: "redis", url }, buildNamespace: ns };
const scan = createCacheComponentsHandler({ ...common, tagReadMode: "scan" });
const onDemand = createCacheComponentsHandler({
...common,
tagReadMode: "on-demand",
});

// T tag markers (soft invalidations with a 1h deadline).
const tags = Array.from({ length: T }, (_, i) => `tag-${i}`);
for (let i = 0; i < T; i += 50) {
await scan.updateTags(tags.slice(i, i + 50), { expire: 3600 });
}

// One real entry with five tags, read with two route tags, like a page.
const entryTags = tags.slice(0, 5);
const softTags = ["_N_T_/layout", "_N_T_/page"];
const body = Buffer.from("<html>".padEnd(4096, "."));
await scan.set(
"bench-page",
Promise.resolve({
value: new ReadableStream({
start(c) {
c.enqueue(new Uint8Array(body));
c.close();
},
}),
tags: entryTags,
stale: 60,
timestamp: Date.now(),
expire: 3600,
revalidate: 60,
})
);
// 50 more entries, 5 tags each, for the "one page, many 'use cache' calls" case.
const pageKeys = Array.from({ length: 50 }, (_, i) => `bench-fn-${i}`);
for (const [i, key] of pageKeys.entries()) {
await scan.set(
key,
Promise.resolve({
value: new ReadableStream({
start(c) {
c.enqueue(new Uint8Array(Buffer.from("x".repeat(512))));
c.close();
},
}),
tags: tags.slice((i * 5) % T, ((i * 5) % T) + 5),
stale: 60,
timestamp: Date.now(),
expire: 3600,
revalidate: 60,
})
);
}
const dbsize = await raw.dbSize();
await new Promise((r) => setTimeout(r, 2000)); // let seeding settle

function pct(arr, p) {
const s = [...arr].sort((a, b) => a - b);
return s[Math.min(s.length - 1, Math.floor((p / 100) * s.length))];
}
async function time(fn, runs) {
await fn(); // warm-up
const out = [];
for (let i = 0; i < runs; i++) {
const t0 = performance.now();
await fn();
out.push(performance.now() - t0);
}
return out;
}
const fmt = (a) =>
`p50 ${pct(a, 50).toFixed(1).padStart(6)} ms p95 ${pct(a, 95)
.toFixed(1)
.padStart(6)} ms`;

const rows = [
['scan: refreshTags()', () => scan.refreshTags()],
['scan: get()', () => scan.get("bench-page", softTags)],
['on-demand: refreshTags()', () => onDemand.refreshTags()],
['on-demand: get()', () => onDemand.get("bench-page", softTags)],
[
"scan: refreshTags() + 50 parallel get()",
async () => {
await scan.refreshTags();
await Promise.all(pageKeys.map((k) => scan.get(k, softTags)));
},
],
[
"on-demand: 50 parallel get()",
() => Promise.all(pageKeys.map((k) => onDemand.get(k, softTags))),
],
[
"raw MGET of 7 markers",
() =>
raw.mGet(
[...entryTags, ...softTags].map(
(t) => `next-cache:tag-expiration:${ns}:${t}`
)
),
],
];

console.log(`keys=${dbsize} tags=${T} runs=${RUNS} redis=${url}`);
for (const [label, fn] of rows) {
console.log(`${label.padEnd(46)} ${fmt(await time(fn, RUNS))}`);
}

await raw.quit();
process.exit(0);
1 change: 1 addition & 0 deletions skills/nextjs-redis-cache/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ Facts that prevent misdiagnosis:
| `hashTag: true` | off | REQUIRED on Redis Cluster |
| `onMetric` | — | Wire `createOtelMetricEmitter()` from `@leejpsd/nextjs-cache-handler/otel` |
| `tagPubSub: true` | off | 0.4+: push-based cross-instance invalidation (~3ms vs seconds); polling stays as safety net; not on Cluster |
| `tagReadMode: "on-demand"` | `"scan"` | 1.1+: Redis holds many keys (tens of thousands) and `refreshTags()` shows up in request latency. Skips the per-request SCAN; `get()` MGETs only the entry's tags (one extra round trip) |

Reliability built in (0.3+): reconnect with exponential backoff (1s→30s cap),
bounded memory fallback, per-op timeouts. A Redis outage degrades to
Expand Down
Loading
Loading