diff --git a/README.md b/README.md index 6c6b345..67fd633 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ The repository implements an inspectable experiment, not a production inference - **LRU** keeps the most recently used experts. - **LFU** keeps the most frequently used experts over the run. -- **ShiftCache** compares recent and longer-horizon activation distributions with Jensen-Shannon divergence (JSD), continuously reweights historical frequency, recent frequency, and recency, and can add observed one-step transition scores. Threshold-crossing events are currently telemetry; they do not yet trigger a separate controller action. +- **ShiftCache** compares recent and longer-horizon activation distributions with Jensen-Shannon divergence (JSD), can continuously reweight historical frequency, recent frequency, and recency, and can add observed one-step transition scores. An opt-in persistent detector can gate a fixed intervention; its preregistered test failed the traffic gate. Every policy moves the requested expert weights; none may replace, skip, reroute, or silently lower the precision of a selected expert. `semanticRoutingChanges` should therefore remain zero. @@ -64,11 +64,19 @@ only **0.52%** and **1.33%**, respectively. It was the best ShiftCache variant but still moved **11.50% more whole-run modeled bytes than LFU**. On this trace, JSD reweighting is negative evidence, not a result to promote as a win. +A preregistered 30-seed synthetic sanity sweep then added three-token +persistence, a 64-token cooldown, and a fixed 64-token event-gated intervention. +It detected all 30 known midpoint shifts after six tokens and produced zero +stationary false triggers, but it **increased** first-64-token post-shift modeled +link bytes by a median **11.52%**; the paired-bootstrap 95% interval was +**[+10.78%, +12.64%]**. The traffic gate failed, so this JSD-gated retention +intervention is not being carried into ShiftQ-MoE. + ## Why this experiment exists [Colibrì](https://github.com/JustVugg/colibri) demonstrates a real VRAM/RAM/storage hierarchy, per-layer caching, pinned hot experts, and live placement policies while preserving router semantics by default. [MoE-Infinity](https://arxiv.org/abs/2401.14361) studies request-level activation traces, predictive caching, and recovery after task or dataset shifts. [DALI](https://arxiv.org/abs/2602.03495) already proposes workload-aware cache replacement, and Colibrì now documents an LFRU-style live placement policy. Those systems make a broad “first workload-aware MoE cache” claim indefensible here. -StrataMoE Lab instead contributes a small, reproducible surface for isolating one hypothesis: whether **router-distribution change signals** can help a non-semantic placement policy recover from abrupt shifts in a controlled trace. The first captured trace is a negative result for continuous JSD reweighting. A genuinely change-triggered controller still requires persistence, cooldown, and ground-truth timing evaluation before it exists in this project. Scientific novelty would still require multiple real router traces, stronger baselines, hardware measurements, ablations, and statistical replication. +StrataMoE Lab instead contributes a small, reproducible surface for isolating one hypothesis: whether **router-distribution change signals** can help a non-semantic placement policy recover from abrupt shifts in a controlled trace. The first captured trace is a negative result for continuous JSD reweighting, and the preregistered synthetic sweep is a negative result for the tested event-gated intervention. Scientific novelty would still require a new out-of-sample hypothesis, multiple real router traces, stronger baselines, hardware measurements, ablations, and statistical replication. ## Quick start @@ -95,6 +103,7 @@ npm run benchmark npm run benchmark:captured npm run benchmark:captured:prefetch npm run benchmark:captured:retention +npm run benchmark:detector ``` The dashboard lets you choose a scenario, seed, token count, model shape, cache capacity, expert size, and modeled bandwidths; run all policies; inspect per-token behavior and final tier residency; and export or import deterministic router traces. @@ -121,6 +130,7 @@ When reporting a result, include: - [Related work](docs/RELATED_WORK.md) — primary papers and official repositories, including work from Chinese universities and labs - [Limitations](docs/LIMITATIONS.md) — what the simulator cannot support as a claim - [Captured Switch-Base-8 trace](docs/CAPTURED_SWITCH_TRACE.md) — pinned model execution, provenance, reproduction, and replay boundary +- [Preregistered detector sanity sweep](docs/DETECTOR_SANITY.md) — frozen gates, per-seed synthetic result, and failed carry-forward decision - [ShiftQ-MoE research plan](docs/SHIFTQ_MOE_RESEARCH_PLAN.md) — closest prior work, candidate method, baselines, ablations, and strict go/no-go gates - [Two-minute outreach demo](docs/OUTREACH_DEMO.md) — an honest walkthrough and ask for the 2026-07-21 call @@ -131,7 +141,7 @@ When reporting a result, include: 1. **Trace harness:** deterministic synthetic traces, inspection UI, fixed regression benchmark, and honest modeled metrics. 2. **Real-trace replay:** collect router IDs from multiple open MoE families and compare stronger activation-aware baselines. 3. **Runtime validation:** integrate a frozen policy into one open runtime and measure actual traffic, TTFT, TPOT, and energy on named hardware. -4. **Precision study:** only after the placement result is understood, test the falsifiable [ShiftQ-MoE research plan](docs/SHIFTQ_MOE_RESEARCH_PLAN.md) with expert-balanced calibration, per-expert quality evidence, close baselines, and preregistered failure gates. +4. **Precision study:** paused for the current JSD-gated design because its preregistered placement gate failed. Any future precision study must begin as a separately registered static sensitivity baseline or supply new out-of-sample detector evidence before reviving a shift-triggered claim. The source code is licensed under the [Apache License 2.0](LICENSE). diff --git a/docs/DETECTOR_SANITY.md b/docs/DETECTOR_SANITY.md new file mode 100644 index 0000000..b9b1ea9 --- /dev/null +++ b/docs/DETECTOR_SANITY.md @@ -0,0 +1,79 @@ +# Preregistered detector sanity sweep + +## Question + +Can a persistent JSD event safely gate temporary ShiftCache reweighting on +deterministic synthetic traces with a known midpoint change? + +This is an engineering sanity check. It is not evidence about semantic domains, +language-model quality, a physical memory hierarchy, or ShiftQ-MoE novelty. + +## Preregistration + +The seed range, token count, requirement to add persistence/cooldown before the +run, and four gates were merged in commit +[`fe6042b`](https://github.com/Labeeb2339/stratamoe-lab/commit/fe6042b6cd628d4976c2069a94d2739614ce9ce9) +before the sweep ran: + +- seeds `2300` through `2329`; +- `1024` tokens per stationary and abrupt-shift trace; +- known synthetic change at token `512`; +- detection, stationary false-trigger, post-shift traffic, and stationary + traffic gates with fixed thresholds. + +The implementation was then written and frozen before its first sweep with: + +- three consecutive above-threshold JSD scores; +- a 64-token cooldown and 64-token triggered reweighting interval; +- prefetch and transition retention disabled in both arms; and +- fixed frequency/recency scoring as the paired baseline. + +The triggered arm uses maximum short-window/recency reweighting for 64 tokens +after an accepted event. Parameters were not changed after the results were +observed. The exact `3/64/64` implementation values were not independently +timestamped in the earlier public commit, so they are a frozen reproducibility +record rather than a fully public preregistration claim. + +## Gates and result + +| Gate | Requirement | Result | Pass | +| --- | --- | ---: | :---: | +| Detection | At least 27/30 shifts within 64 tokens | 30/30; median delay 6 tokens | Yes | +| Stationary false triggers | At most 1 per 10,000 tokens | 0 | Yes | +| Post-shift traffic | Median change at most -5%; bootstrap interval below 0 | **+11.52%**, 95% interval **[+10.78%, +12.64%]** | **No** | +| Stationary traffic | Median regression no greater than 2% | 0% | Yes | + +All 30 triggered runs moved more modeled link bytes than their paired fixed +baseline over tokens `512` through `575`. The detector recognized this strong, +synthetic boundary, but its prescribed cache intervention consistently made the +placement outcome worse. + +The registered decision is therefore: + +> **Do not carry this JSD-gated retention intervention into ShiftQ-MoE.** + +This result rejects one mechanism, not every possible change detector. Retuning +the threshold, hold interval, or weights on these same 30 seeds would invalidate +the preregistered test and is intentionally not done. + +## Reproduce + +```bash +npm ci +npm run benchmark:detector +``` + +The run takes roughly two minutes on the development machine. The complete +per-seed record, controls, fingerprints, bootstrap settings, gate decisions, +and evidence boundary are checked in as +[`detector-sanity.json`](../evidence/synthetic/detector-sanity.json). + +The result may be described as: + +> On 30 deterministic synthetic traces with a known midpoint change, the +> persistent detector fired after six tokens with no stationary false triggers, +> but its fixed 64-token intervention increased simulated post-shift link traffic +> by a median 11.52%; the preregistered carry-forward gate failed. + +It must not be described as a real-model inference slowdown, a quantization +result, or proof that all distribution-change detection is ineffective. diff --git a/docs/METHODOLOGY.md b/docs/METHODOLOGY.md index 0a91d61..0c83d0f 100644 --- a/docs/METHODOLOGY.md +++ b/docs/METHODOLOGY.md @@ -69,10 +69,10 @@ Jensen-Shannon divergence is symmetric and finite for discrete distributions wit As divergence rises, ShiftCache continuously reduces the weight assigned to long-term frequency and increases the influence of recent frequency, recency, and optionally learned one-step expert transitions. A transition score estimates which expert is likely after recently observed experts. Prefetching may promote an exact expert earlier; a wrong prefetch wastes bandwidth or cache capacity but must not change later router selections. -The current threshold-crossing event is telemetry only. It does not gate the -retention formula or trigger a separate policy update. This implementation is -therefore described as **JSD-score reweighting**, not a completed -change-triggered detector/controller. +In the original/default mode, a threshold-crossing event is telemetry only and +the JSD score changes retention continuously. The preregistered sanity sweep +adds an opt-in persistent detector and a fixed event-gated intervention without +changing the historical continuous-scoring result. This combination is a **project hypothesis**, not a claim that distribution-aware caching, recency/frequency hybrids, transition prediction, or workload adaptation are new. MoE-Infinity studies request-level activation patterns and workload changes; DALI studies workload-aware cache replacement; HybriMoE uses score-based caching for unstable activations; and Colibrì documents live LFRU placement. @@ -161,9 +161,13 @@ reduced them by 0.52%. The combined score was 2.68% worse than the fixed score, and every ShiftCache variant remained worse than LFU. These are descriptive results from one 215-token encoder trace, not estimates of a population effect. -The next detector experiment must implement explicit persistence and cooldown, -record known synthetic change points, and evaluate delay, misses, and stationary -false triggers across a preregistered seed set before any threshold is changed. +The preregistered detector sweep is now complete. Three-token persistence detected all 30 known +midpoint shifts with a six-token median delay and no stationary false triggers. +However, the registered 64-token intervention increased paired post-shift +modeled link bytes by a median 11.52%, with a paired-bootstrap 95% interval of +[+10.78%, +12.64%]. The traffic gate failed, so the current JSD-gated retention +mechanism is stopped rather than retuned on the evaluated seeds. See +[Preregistered detector sanity sweep](DETECTOR_SANITY.md). ## 9. Path to real-trace validation diff --git a/docs/SHIFTQ_MOE_RESEARCH_PLAN.md b/docs/SHIFTQ_MOE_RESEARCH_PLAN.md index 6946405..3c923e3 100644 --- a/docs/SHIFTQ_MOE_RESEARCH_PLAN.md +++ b/docs/SHIFTQ_MOE_RESEARCH_PLAN.md @@ -3,8 +3,9 @@ **Working question:** Does router-distribution change detection improve mixed-precision expert residency under sequential workload shifts? -**Status on 2026-07-19:** candidate research direction, not a claimed new -algorithm, quantizer, thesis result, or performance improvement. +**Status on 2026-07-19:** current JSD-gated design stopped after a +preregistered traffic gate failed; not a claimed new algorithm, quantizer, +thesis result, or performance improvement. ## Recommendation @@ -253,15 +254,18 @@ LFU over the full replay. The JSD threshold crossings were at token positions and 167. This is negative evidence for the current continuous JSD reweighting rule on one short encoder trace. -The implementation also exposed a terminology boundary: a threshold crossing -is currently telemetry only. The eviction weights change continuously with the -JSD score, so StrataMoE does not yet contain a change-triggered controller. +That checkpoint also exposed a terminology boundary: its threshold crossing +was telemetry only while the eviction weights changed continuously with the JSD +score. The following preregistered control added a separate event-gated mode. -The next preregistered control is therefore a detector sanity experiment, not -quantization. Use seeds `2300` through `2329`, `1024` tokens, and otherwise the -fixed default synthetic configuration for both stationary and abrupt-shift -traces. Add explicit persistence and cooldown before observing the sweep. Carry -JSD gating forward only if all of these provisional engineering gates pass: +The next preregistered control was therefore a detector sanity experiment, not +quantization. It used seeds `2300` through `2329`, `1024` tokens, and otherwise +the fixed default synthetic configuration for both stationary and abrupt-shift +traces. The public plan required persistence and cooldown before observing the +sweep; the implementation then froze three-token persistence, a 64-token +cooldown, and a fixed 64-token intervention before its first run. Those exact +numeric implementation values were not in the earlier public commit. JSD gating +could continue only if all of these provisional engineering gates passed: 1. at least 27 of 30 abrupt shifts trigger within 64 tokens of the known change; 2. stationary traces produce at most one false trigger per 10,000 tokens; @@ -270,11 +274,24 @@ JSD gating forward only if all of these provisional engineering gates pass: below zero; and 4. stationary whole-run modeled bytes regress by no more than 2%. -These synthetic gates can reject a broken mechanism but cannot pass the real -OLMoE detector gate or support a novelty claim. Only a stable detector and -allocation signal should become the foundation for ShiftQ-MoE. +The detector passed the timing controls: 30 of 30 shifts triggered after six +tokens, with zero stationary false triggers. The intervention failed the +traffic gate decisively. It increased first-64-token post-shift modeled link +bytes by a median **11.52%**, with a paired-bootstrap 95% interval of +**[+10.78%, +12.64%]**. Stationary traffic was unchanged. -The strongest honest working title is: +The registered outcome is `carryForward = false`. The current JSD-gated +retention design stops here and must not be tuned on the same seeds. These +synthetic gates can reject a broken mechanism but could never pass the real +OLMoE detector gate or support a novelty claim. -> **ShiftQ-MoE: Does Router-Distribution Change Detection Improve -> Mixed-Precision Expert Residency?** +Do not implement the proposed ShiftQ-MoE mixed-precision layer on top of this +failed signal. A future quantization study should start separately with a static +expert-sensitivity baseline, or preregister a materially different detector on +new out-of-sample traces. It must not reuse the ShiftQ-MoE name as if this gate +had passed. + +The strongest honest title for the completed placement experiment is: + +> **When Change Detection Is Not Enough: A Negative Control for JSD-Gated MoE +> Expert Placement** diff --git a/evidence/switch-base-8/prefetch-ablation.json b/evidence/switch-base-8/prefetch-ablation.json index 1b30722..05097d2 100644 --- a/evidence/switch-base-8/prefetch-ablation.json +++ b/evidence/switch-base-8/prefetch-ablation.json @@ -48,7 +48,10 @@ "shiftThresholdBits": 0.28, "rearmThresholdBits": 0.12, "minimumTransitionObservations": 2, - "maximumPrefetchesPerToken": 2 + "maximumPrefetchesPerToken": 2, + "persistentThresholdTokens": 3, + "detectorCooldownTokens": 64, + "triggeredReweightingTokens": 64 }, "traceDiagnostics": { "totalExpertSelections": 1290, @@ -63,7 +66,9 @@ "controls": { "shiftCachePrefetch": true, "shiftCacheJsdReweighting": true, - "shiftCacheTransitionRetention": true + "shiftCacheTransitionRetention": true, + "shiftCachePersistentDetector": false, + "shiftCacheTriggeredReweighting": false }, "metrics": { "gpuHitRate": 0.32325581, @@ -99,7 +104,9 @@ "controls": { "shiftCachePrefetch": false, "shiftCacheJsdReweighting": true, - "shiftCacheTransitionRetention": true + "shiftCacheTransitionRetention": true, + "shiftCachePersistentDetector": false, + "shiftCacheTriggeredReweighting": false }, "metrics": { "gpuHitRate": 0.28062016, diff --git a/evidence/switch-base-8/retention-ablation.json b/evidence/switch-base-8/retention-ablation.json index b7f0f5c..4fd0db9 100644 --- a/evidence/switch-base-8/retention-ablation.json +++ b/evidence/switch-base-8/retention-ablation.json @@ -49,7 +49,10 @@ "shiftThresholdBits": 0.28, "rearmThresholdBits": 0.12, "minimumTransitionObservations": 2, - "maximumPrefetchesPerToken": 2 + "maximumPrefetchesPerToken": 2, + "persistentThresholdTokens": 3, + "detectorCooldownTokens": 64, + "triggeredReweightingTokens": 64 }, "semanticGroupBoundaries": [ { @@ -79,7 +82,9 @@ "controls": { "shiftCachePrefetch": false, "shiftCacheJsdReweighting": false, - "shiftCacheTransitionRetention": false + "shiftCacheTransitionRetention": false, + "shiftCachePersistentDetector": false, + "shiftCacheTriggeredReweighting": false }, "detectedShiftEvents": [ { @@ -172,7 +177,9 @@ "controls": { "shiftCachePrefetch": false, "shiftCacheJsdReweighting": true, - "shiftCacheTransitionRetention": false + "shiftCacheTransitionRetention": false, + "shiftCachePersistentDetector": false, + "shiftCacheTriggeredReweighting": false }, "detectedShiftEvents": [ { @@ -265,7 +272,9 @@ "controls": { "shiftCachePrefetch": false, "shiftCacheJsdReweighting": false, - "shiftCacheTransitionRetention": true + "shiftCacheTransitionRetention": true, + "shiftCachePersistentDetector": false, + "shiftCacheTriggeredReweighting": false }, "detectedShiftEvents": [ { @@ -358,7 +367,9 @@ "controls": { "shiftCachePrefetch": false, "shiftCacheJsdReweighting": true, - "shiftCacheTransitionRetention": true + "shiftCacheTransitionRetention": true, + "shiftCachePersistentDetector": false, + "shiftCacheTriggeredReweighting": false }, "detectedShiftEvents": [ { diff --git a/evidence/synthetic/detector-sanity.json b/evidence/synthetic/detector-sanity.json new file mode 100644 index 0000000..fdbe2f3 --- /dev/null +++ b/evidence/synthetic/detector-sanity.json @@ -0,0 +1,829 @@ +{ + "benchmark": "synthetic-persistent-detector-sanity-v1", + "preregistration": { + "publicPlanCommit": "fe6042b6cd628d4976c2069a94d2739614ce9ce9", + "publicPlanPath": "docs/SHIFTQ_MOE_RESEARCH_PLAN.md", + "note": "Seed range, token count, persistence/cooldown requirement, and all four gates were merged before this sweep was executed." + }, + "evidenceBoundary": "All traces in this sweep are deterministic synthetic fixtures with known generator boundaries. The result can reject a broken mechanism but cannot validate real semantic shifts, model quality, hardware performance, or ShiftQ-MoE novelty.", + "intervention": "After three consecutive JSD scores at or above the fixed threshold, use maximum short-window/recency reweighting for 64 tokens, then require cooldown and rearming. Prefetch and transition retention stay disabled in both arms.", + "configuration": { + "seeds": [ + 2300, + 2301, + 2302, + 2303, + 2304, + 2305, + 2306, + 2307, + 2308, + 2309, + 2310, + 2311, + 2312, + 2313, + 2314, + 2315, + 2316, + 2317, + 2318, + 2319, + 2320, + 2321, + 2322, + 2323, + 2324, + 2325, + 2326, + 2327, + 2328, + 2329 + ], + "tokens": 1024, + "knownShiftToken": 512, + "postShiftWindowTokens": 64, + "base": { + "layers": 8, + "expertsPerLayer": 16, + "topK": 2, + "gpuSlots": 32, + "ramSlots": 64, + "expertSizeMB": 64, + "pcieGBps": 24, + "nvmeGBps": 7, + "computeMsPerToken": 6 + }, + "shiftCacheParameters": { + "minimumShortWindowTokens": 4, + "maximumShortWindowTokens": 12, + "longWindowMultiplier": 4, + "shiftThresholdBits": 0.28, + "rearmThresholdBits": 0.12, + "minimumTransitionObservations": 2, + "maximumPrefetchesPerToken": 2, + "persistentThresholdTokens": 3, + "detectorCooldownTokens": 64, + "triggeredReweightingTokens": 64 + }, + "fixedControls": { + "shiftCachePrefetch": false, + "shiftCacheJsdReweighting": false, + "shiftCacheTransitionRetention": false, + "shiftCachePersistentDetector": true, + "shiftCacheTriggeredReweighting": false + }, + "triggeredControls": { + "shiftCachePrefetch": false, + "shiftCacheJsdReweighting": false, + "shiftCacheTransitionRetention": false, + "shiftCachePersistentDetector": true, + "shiftCacheTriggeredReweighting": true + } + }, + "analysis": { + "pairedUnit": "seed", + "postShiftMetric": "Percent change in total modeled link bytes across tokens 512 through 575, triggered versus fixed.", + "stationaryMetric": "Percent change in whole-run total modeled link bytes, triggered versus fixed.", + "bootstrapStatistic": "Median paired percent change.", + "bootstrap": { + "resamples": 10000, + "seed": 2339, + "lowerPercentile": 2.5, + "upperPercentile": 97.5, + "lower": 10.78035747, + "upper": 12.63942461 + } + }, + "records": [ + { + "seed": 2300, + "domainShift": { + "traceFingerprint": "aeff2dc1", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 24960000000, + "triggeredPostShiftModeledLinkBytes": 28672000000, + "postShiftModeledLinkBytesPercentChange": 14.87179487 + }, + "steady": { + "traceFingerprint": "c3bd7d2d", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 612608000000, + "triggeredWholeRunModeledLinkBytes": 612608000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2301, + "domainShift": { + "traceFingerprint": "ce1b8e2a", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25920000000, + "triggeredPostShiftModeledLinkBytes": 28736000000, + "postShiftModeledLinkBytesPercentChange": 10.86419753 + }, + "steady": { + "traceFingerprint": "ff1f609a", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 615680000000, + "triggeredWholeRunModeledLinkBytes": 615680000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2302, + "domainShift": { + "traceFingerprint": "2c57f363", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25920000000, + "triggeredPostShiftModeledLinkBytes": 28544000000, + "postShiftModeledLinkBytesPercentChange": 10.12345679 + }, + "steady": { + "traceFingerprint": "b9ca967d", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 615552000000, + "triggeredWholeRunModeledLinkBytes": 615552000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2303, + "domainShift": { + "traceFingerprint": "14710323", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 26048000000, + "triggeredPostShiftModeledLinkBytes": 28736000000, + "postShiftModeledLinkBytesPercentChange": 10.31941032 + }, + "steady": { + "traceFingerprint": "8e14e88e", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 611584000000, + "triggeredWholeRunModeledLinkBytes": 611584000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2304, + "domainShift": { + "traceFingerprint": "71c209fa", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25856000000, + "triggeredPostShiftModeledLinkBytes": 28864000000, + "postShiftModeledLinkBytesPercentChange": 11.63366337 + }, + "steady": { + "traceFingerprint": "d17fee5b", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 610816000000, + "triggeredWholeRunModeledLinkBytes": 610816000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2305, + "domainShift": { + "traceFingerprint": "a6da2176", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25792000000, + "triggeredPostShiftModeledLinkBytes": 28992000000, + "postShiftModeledLinkBytesPercentChange": 12.40694789 + }, + "steady": { + "traceFingerprint": "9e788db1", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 623296000000, + "triggeredWholeRunModeledLinkBytes": 623296000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2306, + "domainShift": { + "traceFingerprint": "7a3aa75d", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25792000000, + "triggeredPostShiftModeledLinkBytes": 29056000000, + "postShiftModeledLinkBytesPercentChange": 12.65508685 + }, + "steady": { + "traceFingerprint": "e2bade34", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 614464000000, + "triggeredWholeRunModeledLinkBytes": 614464000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2307, + "domainShift": { + "traceFingerprint": "82994a96", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25728000000, + "triggeredPostShiftModeledLinkBytes": 28480000000, + "postShiftModeledLinkBytesPercentChange": 10.69651741 + }, + "steady": { + "traceFingerprint": "83ff1e37", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 614912000000, + "triggeredWholeRunModeledLinkBytes": 614912000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2308, + "domainShift": { + "traceFingerprint": "e3d4e364", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 26112000000, + "triggeredPostShiftModeledLinkBytes": 28544000000, + "postShiftModeledLinkBytesPercentChange": 9.31372549 + }, + "steady": { + "traceFingerprint": "367d82a1", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 615552000000, + "triggeredWholeRunModeledLinkBytes": 615552000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2309, + "domainShift": { + "traceFingerprint": "946eb0e8", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25856000000, + "triggeredPostShiftModeledLinkBytes": 29760000000, + "postShiftModeledLinkBytesPercentChange": 15.0990099 + }, + "steady": { + "traceFingerprint": "00041349", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 619968000000, + "triggeredWholeRunModeledLinkBytes": 619968000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2310, + "domainShift": { + "traceFingerprint": "f599f20d", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25920000000, + "triggeredPostShiftModeledLinkBytes": 29248000000, + "postShiftModeledLinkBytesPercentChange": 12.83950617 + }, + "steady": { + "traceFingerprint": "bb1c92ff", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 614592000000, + "triggeredWholeRunModeledLinkBytes": 614592000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2311, + "domainShift": { + "traceFingerprint": "5ced96b1", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25856000000, + "triggeredPostShiftModeledLinkBytes": 29184000000, + "postShiftModeledLinkBytesPercentChange": 12.87128713 + }, + "steady": { + "traceFingerprint": "8dc56dae", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 617280000000, + "triggeredWholeRunModeledLinkBytes": 617280000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2312, + "domainShift": { + "traceFingerprint": "b6c5c8b2", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 26048000000, + "triggeredPostShiftModeledLinkBytes": 28736000000, + "postShiftModeledLinkBytesPercentChange": 10.31941032 + }, + "steady": { + "traceFingerprint": "f1df4eef", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 615552000000, + "triggeredWholeRunModeledLinkBytes": 615552000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2313, + "domainShift": { + "traceFingerprint": "db868df6", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25344000000, + "triggeredPostShiftModeledLinkBytes": 28160000000, + "postShiftModeledLinkBytesPercentChange": 11.11111111 + }, + "steady": { + "traceFingerprint": "fd8f5527", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 617280000000, + "triggeredWholeRunModeledLinkBytes": 617280000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2314, + "domainShift": { + "traceFingerprint": "f3bf72ac", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25920000000, + "triggeredPostShiftModeledLinkBytes": 28608000000, + "postShiftModeledLinkBytesPercentChange": 10.37037037 + }, + "steady": { + "traceFingerprint": "7e56d51b", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 613312000000, + "triggeredWholeRunModeledLinkBytes": 613312000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2315, + "domainShift": { + "traceFingerprint": "1f254fff", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25920000000, + "triggeredPostShiftModeledLinkBytes": 28672000000, + "postShiftModeledLinkBytesPercentChange": 10.61728395 + }, + "steady": { + "traceFingerprint": "478a7505", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 616960000000, + "triggeredWholeRunModeledLinkBytes": 616960000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2316, + "domainShift": { + "traceFingerprint": "69585354", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25984000000, + "triggeredPostShiftModeledLinkBytes": 28352000000, + "postShiftModeledLinkBytesPercentChange": 9.11330049 + }, + "steady": { + "traceFingerprint": "a3114e8f", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 617600000000, + "triggeredWholeRunModeledLinkBytes": 617600000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2317, + "domainShift": { + "traceFingerprint": "1d030792", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25856000000, + "triggeredPostShiftModeledLinkBytes": 29120000000, + "postShiftModeledLinkBytesPercentChange": 12.62376238 + }, + "steady": { + "traceFingerprint": "90d515e3", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 608896000000, + "triggeredWholeRunModeledLinkBytes": 608896000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2318, + "domainShift": { + "traceFingerprint": "594dfb5a", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25920000000, + "triggeredPostShiftModeledLinkBytes": 28928000000, + "postShiftModeledLinkBytesPercentChange": 11.60493827 + }, + "steady": { + "traceFingerprint": "aead2a6c", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 615104000000, + "triggeredWholeRunModeledLinkBytes": 615104000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2319, + "domainShift": { + "traceFingerprint": "64e766c6", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25856000000, + "triggeredPostShiftModeledLinkBytes": 28736000000, + "postShiftModeledLinkBytesPercentChange": 11.13861386 + }, + "steady": { + "traceFingerprint": "ca4f4004", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 616704000000, + "triggeredWholeRunModeledLinkBytes": 616704000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2320, + "domainShift": { + "traceFingerprint": "067c9063", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 26112000000, + "triggeredPostShiftModeledLinkBytes": 29504000000, + "postShiftModeledLinkBytesPercentChange": 12.99019608 + }, + "steady": { + "traceFingerprint": "dd6f3f29", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 614720000000, + "triggeredWholeRunModeledLinkBytes": 614720000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2321, + "domainShift": { + "traceFingerprint": "d3572e7a", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25920000000, + "triggeredPostShiftModeledLinkBytes": 27840000000, + "postShiftModeledLinkBytesPercentChange": 7.40740741 + }, + "steady": { + "traceFingerprint": "80a42831", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 616320000000, + "triggeredWholeRunModeledLinkBytes": 616320000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2322, + "domainShift": { + "traceFingerprint": "741aa569", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25920000000, + "triggeredPostShiftModeledLinkBytes": 29504000000, + "postShiftModeledLinkBytesPercentChange": 13.82716049 + }, + "steady": { + "traceFingerprint": "425350e2", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 616128000000, + "triggeredWholeRunModeledLinkBytes": 616128000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2323, + "domainShift": { + "traceFingerprint": "a35450ff", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25792000000, + "triggeredPostShiftModeledLinkBytes": 28224000000, + "postShiftModeledLinkBytesPercentChange": 9.4292804 + }, + "steady": { + "traceFingerprint": "4bef34e5", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 613056000000, + "triggeredWholeRunModeledLinkBytes": 613056000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2324, + "domainShift": { + "traceFingerprint": "b793bf04", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25920000000, + "triggeredPostShiftModeledLinkBytes": 28928000000, + "postShiftModeledLinkBytesPercentChange": 11.60493827 + }, + "steady": { + "traceFingerprint": "5985e620", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 617216000000, + "triggeredWholeRunModeledLinkBytes": 617216000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2325, + "domainShift": { + "traceFingerprint": "bde787aa", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25856000000, + "triggeredPostShiftModeledLinkBytes": 29888000000, + "postShiftModeledLinkBytesPercentChange": 15.59405941 + }, + "steady": { + "traceFingerprint": "41720c35", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 616448000000, + "triggeredWholeRunModeledLinkBytes": 616448000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2326, + "domainShift": { + "traceFingerprint": "897024f4", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25728000000, + "triggeredPostShiftModeledLinkBytes": 28672000000, + "postShiftModeledLinkBytesPercentChange": 11.44278607 + }, + "steady": { + "traceFingerprint": "6a549db0", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 612672000000, + "triggeredWholeRunModeledLinkBytes": 612672000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2327, + "domainShift": { + "traceFingerprint": "14262c3a", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25856000000, + "triggeredPostShiftModeledLinkBytes": 28800000000, + "postShiftModeledLinkBytesPercentChange": 11.38613861 + }, + "steady": { + "traceFingerprint": "404187dc", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 617856000000, + "triggeredWholeRunModeledLinkBytes": 617856000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2328, + "domainShift": { + "traceFingerprint": "67f83804", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25664000000, + "triggeredPostShiftModeledLinkBytes": 29248000000, + "postShiftModeledLinkBytesPercentChange": 13.96508728 + }, + "steady": { + "traceFingerprint": "5a7e4dc5", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 616384000000, + "triggeredWholeRunModeledLinkBytes": 616384000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + }, + { + "seed": 2329, + "domainShift": { + "traceFingerprint": "7af54990", + "detectedEventTokens": [ + 518 + ], + "preShiftEvents": 0, + "firstPostShiftEvent": 518, + "detectionDelayTokens": 6, + "detectedWithin64Tokens": true, + "fixedPostShiftModeledLinkBytes": 25856000000, + "triggeredPostShiftModeledLinkBytes": 29440000000, + "postShiftModeledLinkBytesPercentChange": 13.86138614 + }, + "steady": { + "traceFingerprint": "bbf25789", + "detectedEventTokens": [], + "fixedWholeRunModeledLinkBytes": 614720000000, + "triggeredWholeRunModeledLinkBytes": 614720000000, + "wholeRunModeledLinkBytesPercentChange": 0 + } + } + ], + "summary": { + "detectedWithin64Count": 30, + "missedOrLateCount": 0, + "medianDetectionDelayTokens": 6, + "stationaryFalseTriggers": 0, + "stationaryFalseTriggersPer10000Tokens": 0, + "domainPreShiftTriggers": 0, + "domainPreShiftTriggersPer10000Tokens": 0, + "medianPostShiftModeledLinkBytesPercentChange": 11.52386217, + "postShiftMedianBootstrap95PercentInterval": { + "resamples": 10000, + "seed": 2339, + "lowerPercentile": 2.5, + "upperPercentile": 97.5, + "lower": 10.78035747, + "upper": 12.63942461 + }, + "medianStationaryWholeRunModeledLinkBytesPercentChange": 0, + "maximumStationaryWholeRunModeledLinkBytesPercentChange": 0 + }, + "gates": { + "detection": { + "requirement": "At least 27 of 30 abrupt shifts detected within 64 tokens.", + "passed": true + }, + "stationaryFalseTriggers": { + "requirement": "At most one false trigger per 10,000 stationary tokens.", + "passed": true + }, + "postShiftTraffic": { + "requirement": "Median first-64-token post-shift modeled-link-byte change is at most -5%, with paired-bootstrap 95% interval entirely below zero.", + "passed": false + }, + "stationaryTraffic": { + "requirement": "Median stationary whole-run modeled-link-byte regression is no greater than 2%.", + "passed": true + } + }, + "carryForward": false +} diff --git a/lib/simulator.ts b/lib/simulator.ts index 4a23198..3d14980 100644 --- a/lib/simulator.ts +++ b/lib/simulator.ts @@ -28,6 +28,10 @@ export interface SimulationControls { shiftCacheJsdReweighting: boolean; /** Include one-step transition predictions in the eviction retention score. */ shiftCacheTransitionRetention: boolean; + /** Require persistent threshold exceedance and apply a cooldown to shift events. */ + shiftCachePersistentDetector: boolean; + /** Use binary post-event reweighting instead of continuous JSD-score reweighting. */ + shiftCacheTriggeredReweighting: boolean; } export type ComparisonConfig = Omit; @@ -250,6 +254,8 @@ export const DEFAULT_SIMULATION_CONTROLS: SimulationControls = Object.freeze({ shiftCachePrefetch: true, shiftCacheJsdReweighting: true, shiftCacheTransitionRetention: true, + shiftCachePersistentDetector: false, + shiftCacheTriggeredReweighting: false, }); /** Runtime limits protect browser/CLI callers before trace allocation begins. */ @@ -449,6 +455,8 @@ export function validateSimulationControls( "shiftCachePrefetch", "shiftCacheJsdReweighting", "shiftCacheTransitionRetention", + "shiftCachePersistentDetector", + "shiftCacheTriggeredReweighting", ] as const; assertAllowedKeys(input, "Simulation controls", keys); for (const key of keys) { @@ -456,10 +464,25 @@ export function validateSimulationControls( throw new TypeError(`${key} must be a boolean.`); } } + if (input.shiftCacheTriggeredReweighting && input.shiftCacheJsdReweighting) { + throw new RangeError( + "shiftCacheTriggeredReweighting and shiftCacheJsdReweighting cannot both be enabled.", + ); + } + if ( + input.shiftCacheTriggeredReweighting && + !input.shiftCachePersistentDetector + ) { + throw new RangeError( + "shiftCacheTriggeredReweighting requires shiftCachePersistentDetector.", + ); + } return { shiftCachePrefetch: input.shiftCachePrefetch, shiftCacheJsdReweighting: input.shiftCacheJsdReweighting, shiftCacheTransitionRetention: input.shiftCacheTransitionRetention, + shiftCachePersistentDetector: input.shiftCachePersistentDetector, + shiftCacheTriggeredReweighting: input.shiftCacheTriggeredReweighting, }; } @@ -1010,6 +1033,9 @@ export const SHIFT_CACHE_PARAMETERS = Object.freeze({ rearmThresholdBits: 0.12, minimumTransitionObservations: 2, maximumPrefetchesPerToken: 2, + persistentThresholdTokens: 3, + detectorCooldownTokens: 64, + triggeredReweightingTokens: 64, }); interface CacheCounters { @@ -1082,9 +1108,14 @@ class ShiftTracker { private readonly shortQueue: string[][] = []; private readonly longQueue: string[][] = []; + private readonly persistentDetector: boolean; private armed = true; + private aboveThresholdTokens = 0; + private cooldownTokensRemaining = 0; + private triggeredReweightingTokensRemaining = 0; - constructor(tokens: number) { + constructor(tokens: number, persistentDetector: boolean) { + this.persistentDetector = persistentDetector; this.shortWindowTokens = Math.max( SHIFT_CACHE_PARAMETERS.minimumShortWindowTokens, Math.min( @@ -1095,7 +1126,21 @@ class ShiftTracker { this.longWindowTokens = this.shortWindowTokens * SHIFT_CACHE_PARAMETERS.longWindowMultiplier; } + get triggeredReweightingActive(): boolean { + return this.triggeredReweightingTokensRemaining > 0; + } + observe(experts: readonly string[]): { score: number; detected: boolean } { + if (this.persistentDetector) { + this.cooldownTokensRemaining = Math.max( + 0, + this.cooldownTokensRemaining - 1, + ); + this.triggeredReweightingTokensRemaining = Math.max( + 0, + this.triggeredReweightingTokensRemaining - 1, + ); + } const tokenExperts = [...experts]; this.shortQueue.push(tokenExperts); this.longQueue.push(tokenExperts); @@ -1127,11 +1172,40 @@ class ShiftTracker { this.currentScore = jensenShannonDivergence(this.shortCounts, baselineCounts); let detected = false; - if (this.armed && this.currentScore >= SHIFT_CACHE_PARAMETERS.shiftThresholdBits) { + if (!this.persistentDetector) { + if (this.armed && this.currentScore >= SHIFT_CACHE_PARAMETERS.shiftThresholdBits) { + this.detectedShifts += 1; + this.armed = false; + detected = true; + } else if (this.currentScore <= SHIFT_CACHE_PARAMETERS.rearmThresholdBits) { + this.armed = true; + } + return { score: this.currentScore, detected }; + } + + if (this.currentScore >= SHIFT_CACHE_PARAMETERS.shiftThresholdBits) { + this.aboveThresholdTokens += 1; + } else { + this.aboveThresholdTokens = 0; + } + + if ( + this.armed && + this.cooldownTokensRemaining === 0 && + this.aboveThresholdTokens >= SHIFT_CACHE_PARAMETERS.persistentThresholdTokens + ) { this.detectedShifts += 1; this.armed = false; + this.aboveThresholdTokens = 0; + this.cooldownTokensRemaining = SHIFT_CACHE_PARAMETERS.detectorCooldownTokens; + this.triggeredReweightingTokensRemaining = + SHIFT_CACHE_PARAMETERS.triggeredReweightingTokens; detected = true; - } else if (this.currentScore <= SHIFT_CACHE_PARAMETERS.rearmThresholdBits) { + } else if ( + !this.armed && + this.cooldownTokensRemaining === 0 && + this.currentScore <= SHIFT_CACHE_PARAMETERS.rearmThresholdBits + ) { this.armed = true; } return { score: this.currentScore, detected }; @@ -1187,7 +1261,10 @@ class HierarchySimulator { constructor(config: SimulationConfig, controls: SimulationControls) { this.config = config; this.controls = controls; - this.shiftTracker = new ShiftTracker(config.tokens); + this.shiftTracker = new ShiftTracker( + config.tokens, + controls.shiftCachePersistentDetector, + ); } demand(key: string): AccessSource { @@ -1289,13 +1366,17 @@ class HierarchySimulator { const longMaximum = Math.max(1, ...this.shiftTracker.longCounts.values()); const shortMaximum = Math.max(1, ...this.shiftTracker.shortCounts.values()); const predictionMaximum = Math.max(1, ...this.predictionScores.values()); - const adaptation = this.controls.shiftCacheJsdReweighting - ? Math.min( - 1, - this.shiftTracker.currentScore / - SHIFT_CACHE_PARAMETERS.shiftThresholdBits, - ) - : 0; + const adaptation = this.controls.shiftCacheTriggeredReweighting + ? this.shiftTracker.triggeredReweightingActive + ? 1 + : 0 + : this.controls.shiftCacheJsdReweighting + ? Math.min( + 1, + this.shiftTracker.currentScore / + SHIFT_CACHE_PARAMETERS.shiftThresholdBits, + ) + : 0; const longWeight = 0.55 - 0.45 * adaptation; const shortWeight = 0.15 + 0.3 * adaptation; const transitionWeight = this.controls.shiftCacheTransitionRetention diff --git a/package.json b/package.json index b82c5c6..37670cb 100644 --- a/package.json +++ b/package.json @@ -18,6 +18,7 @@ "benchmark:captured": "tsx scripts/run-captured-benchmark.ts", "benchmark:captured:prefetch": "tsx scripts/run-prefetch-ablation.ts", "benchmark:captured:retention": "tsx scripts/run-retention-ablation.ts", + "benchmark:detector": "tsx scripts/run-detector-sanity.ts", "check": "npm run lint && npm run typecheck && npm run test:unit && npm run build && node --test tests/rendered-html.test.mjs", "lint": "eslint . --ignore-pattern dist --ignore-pattern .next" }, diff --git a/scripts/run-detector-sanity.ts b/scripts/run-detector-sanity.ts new file mode 100644 index 0000000..c81595a --- /dev/null +++ b/scripts/run-detector-sanity.ts @@ -0,0 +1,302 @@ +import { mkdirSync, writeFileSync } from "node:fs"; +import { + DEFAULT_CONFIG, + DEFAULT_SIMULATION_CONTROLS, + SHIFT_CACHE_PARAMETERS, + generateRouterTrace, + runSimulation, + type ScenarioId, + type SimulationControls, + type SimulationResult, +} from "../lib/simulator"; + +const evidenceDirectory = new URL("../evidence/synthetic/", import.meta.url); +const outputUrl = new URL("detector-sanity.json", evidenceDirectory); + +const seeds = Array.from({ length: 30 }, (_, index) => 2300 + index); +const tokens = 1024; +const knownShiftToken = tokens / 2; +const postShiftWindowTokens = 64; +const bootstrapResamples = 10_000; +const bootstrapSeed = 2339; + +const fixedControls: SimulationControls = { + ...DEFAULT_SIMULATION_CONTROLS, + shiftCachePrefetch: false, + shiftCacheJsdReweighting: false, + shiftCacheTransitionRetention: false, + shiftCachePersistentDetector: true, + shiftCacheTriggeredReweighting: false, +}; +const triggeredControls: SimulationControls = { + ...fixedControls, + shiftCacheTriggeredReweighting: true, +}; + +function round(value: number, digits = 8): number { + return Number(value.toFixed(digits)); +} + +function median(values: readonly number[]): number { + if (values.length === 0) return 0; + const sorted = [...values].sort((left, right) => left - right); + const middle = Math.floor(sorted.length / 2); + return sorted.length % 2 === 0 + ? (sorted[middle - 1] + sorted[middle]) / 2 + : sorted[middle]; +} + +function createRandom(seed: number): () => number { + let state = seed >>> 0; + return () => { + state ^= state << 13; + state ^= state >>> 17; + state ^= state << 5; + return (state >>> 0) / 4_294_967_296; + }; +} + +function pairedMedianBootstrap(values: readonly number[]) { + const random = createRandom(bootstrapSeed); + const estimates: number[] = []; + for (let sample = 0; sample < bootstrapResamples; sample += 1) { + const resampled: number[] = []; + for (let index = 0; index < values.length; index += 1) { + resampled.push(values[Math.floor(random() * values.length)]); + } + estimates.push(median(resampled)); + } + estimates.sort((left, right) => left - right); + return { + resamples: bootstrapResamples, + seed: bootstrapSeed, + lowerPercentile: 2.5, + upperPercentile: 97.5, + lower: round(estimates[Math.floor(0.025 * (estimates.length - 1))]), + upper: round(estimates[Math.ceil(0.975 * (estimates.length - 1))]), + }; +} + +function percentChange(after: number, before: number): number { + return before === 0 ? 0 : round(((after - before) / before) * 100); +} + +function modeledBytes( + result: SimulationResult, + start = 0, + endExclusive = result.timeline.length, +): number { + return result.timeline + .slice(start, endExclusive) + .reduce((sum, point) => sum + point.bytesTransferred, 0); +} + +function simulation(seed: number, scenario: ScenarioId, controls: SimulationControls) { + const traceConfig = { + scenario, + seed, + tokens, + layers: DEFAULT_CONFIG.layers, + expertsPerLayer: DEFAULT_CONFIG.expertsPerLayer, + topK: DEFAULT_CONFIG.topK, + }; + const trace = generateRouterTrace(traceConfig); + return runSimulation( + { + ...DEFAULT_CONFIG, + ...traceConfig, + policy: "shift-cache", + }, + trace, + controls, + ); +} + +const records = seeds.map((seed) => { + const domainFixed = simulation(seed, "domain-shift", fixedControls); + const domainTriggered = simulation(seed, "domain-shift", triggeredControls); + const steadyFixed = simulation(seed, "steady", fixedControls); + const steadyTriggered = simulation(seed, "steady", triggeredControls); + + const domainEvents = domainTriggered.timeline + .filter((point) => point.shiftDetected) + .map((point) => point.token); + const firstPostShiftEvent = domainEvents.find( + (token) => token >= knownShiftToken, + ); + const detectionDelay = + firstPostShiftEvent === undefined + ? null + : firstPostShiftEvent - knownShiftToken; + const postShiftEnd = knownShiftToken + postShiftWindowTokens; + const fixedPostShiftBytes = modeledBytes( + domainFixed, + knownShiftToken, + postShiftEnd, + ); + const triggeredPostShiftBytes = modeledBytes( + domainTriggered, + knownShiftToken, + postShiftEnd, + ); + + return { + seed, + domainShift: { + traceFingerprint: domainTriggered.traceFingerprint, + detectedEventTokens: domainEvents, + preShiftEvents: domainEvents.filter((token) => token < knownShiftToken) + .length, + firstPostShiftEvent: firstPostShiftEvent ?? null, + detectionDelayTokens: detectionDelay, + detectedWithin64Tokens: + detectionDelay !== null && + detectionDelay >= 0 && + detectionDelay <= postShiftWindowTokens, + fixedPostShiftModeledLinkBytes: fixedPostShiftBytes, + triggeredPostShiftModeledLinkBytes: triggeredPostShiftBytes, + postShiftModeledLinkBytesPercentChange: percentChange( + triggeredPostShiftBytes, + fixedPostShiftBytes, + ), + }, + steady: { + traceFingerprint: steadyTriggered.traceFingerprint, + detectedEventTokens: steadyTriggered.timeline + .filter((point) => point.shiftDetected) + .map((point) => point.token), + fixedWholeRunModeledLinkBytes: steadyFixed.metrics.totalBytesTransferred, + triggeredWholeRunModeledLinkBytes: + steadyTriggered.metrics.totalBytesTransferred, + wholeRunModeledLinkBytesPercentChange: percentChange( + steadyTriggered.metrics.totalBytesTransferred, + steadyFixed.metrics.totalBytesTransferred, + ), + }, + }; +}); + +const detectedWithin64Count = records.filter( + (record) => record.domainShift.detectedWithin64Tokens, +).length; +const stationaryFalseTriggers = records.reduce( + (sum, record) => sum + record.steady.detectedEventTokens.length, + 0, +); +const domainPreShiftTriggers = records.reduce( + (sum, record) => sum + record.domainShift.preShiftEvents, + 0, +); +const stationaryTokens = seeds.length * tokens; +const postShiftPercentChanges = records.map( + (record) => record.domainShift.postShiftModeledLinkBytesPercentChange, +); +const stationaryPercentChanges = records.map( + (record) => record.steady.wholeRunModeledLinkBytesPercentChange, +); +const bootstrap = pairedMedianBootstrap(postShiftPercentChanges); +const summary = { + detectedWithin64Count, + missedOrLateCount: seeds.length - detectedWithin64Count, + medianDetectionDelayTokens: round( + median( + records + .map((record) => record.domainShift.detectionDelayTokens) + .filter((delay): delay is number => delay !== null), + ), + ), + stationaryFalseTriggers, + stationaryFalseTriggersPer10000Tokens: round( + (stationaryFalseTriggers / stationaryTokens) * 10_000, + ), + domainPreShiftTriggers, + domainPreShiftTriggersPer10000Tokens: round( + (domainPreShiftTriggers / (seeds.length * knownShiftToken)) * 10_000, + ), + medianPostShiftModeledLinkBytesPercentChange: round( + median(postShiftPercentChanges), + ), + postShiftMedianBootstrap95PercentInterval: bootstrap, + medianStationaryWholeRunModeledLinkBytesPercentChange: round( + median(stationaryPercentChanges), + ), + maximumStationaryWholeRunModeledLinkBytesPercentChange: round( + Math.max(...stationaryPercentChanges), + ), +}; + +const gates = { + detection: { + requirement: "At least 27 of 30 abrupt shifts detected within 64 tokens.", + passed: summary.detectedWithin64Count >= 27, + }, + stationaryFalseTriggers: { + requirement: "At most one false trigger per 10,000 stationary tokens.", + passed: summary.stationaryFalseTriggersPer10000Tokens <= 1, + }, + postShiftTraffic: { + requirement: + "Median first-64-token post-shift modeled-link-byte change is at most -5%, with paired-bootstrap 95% interval entirely below zero.", + passed: + summary.medianPostShiftModeledLinkBytesPercentChange <= -5 && + summary.postShiftMedianBootstrap95PercentInterval.upper < 0, + }, + stationaryTraffic: { + requirement: + "Median stationary whole-run modeled-link-byte regression is no greater than 2%.", + passed: + summary.medianStationaryWholeRunModeledLinkBytesPercentChange <= 2, + }, +}; + +const output = { + benchmark: "synthetic-persistent-detector-sanity-v1", + preregistration: { + publicPlanCommit: "fe6042b6cd628d4976c2069a94d2739614ce9ce9", + publicPlanPath: "docs/SHIFTQ_MOE_RESEARCH_PLAN.md", + note: + "Seed range, token count, persistence/cooldown requirement, and all four gates were merged before this sweep was executed.", + }, + evidenceBoundary: + "All traces in this sweep are deterministic synthetic fixtures with known generator boundaries. The result can reject a broken mechanism but cannot validate real semantic shifts, model quality, hardware performance, or ShiftQ-MoE novelty.", + intervention: + "After three consecutive JSD scores at or above the fixed threshold, use maximum short-window/recency reweighting for 64 tokens, then require cooldown and rearming. Prefetch and transition retention stay disabled in both arms.", + configuration: { + seeds, + tokens, + knownShiftToken, + postShiftWindowTokens, + base: { + layers: DEFAULT_CONFIG.layers, + expertsPerLayer: DEFAULT_CONFIG.expertsPerLayer, + topK: DEFAULT_CONFIG.topK, + gpuSlots: DEFAULT_CONFIG.gpuSlots, + ramSlots: DEFAULT_CONFIG.ramSlots, + expertSizeMB: DEFAULT_CONFIG.expertSizeMB, + pcieGBps: DEFAULT_CONFIG.pcieGBps, + nvmeGBps: DEFAULT_CONFIG.nvmeGBps, + computeMsPerToken: DEFAULT_CONFIG.computeMsPerToken, + }, + shiftCacheParameters: SHIFT_CACHE_PARAMETERS, + fixedControls, + triggeredControls, + }, + analysis: { + pairedUnit: "seed", + postShiftMetric: + "Percent change in total modeled link bytes across tokens 512 through 575, triggered versus fixed.", + stationaryMetric: + "Percent change in whole-run total modeled link bytes, triggered versus fixed.", + bootstrapStatistic: "Median paired percent change.", + bootstrap, + }, + records, + summary, + gates, + carryForward: Object.values(gates).every((gate) => gate.passed), +}; + +mkdirSync(evidenceDirectory, { recursive: true }); +const serialized = `${JSON.stringify(output, null, 2)}\n`; +writeFileSync(outputUrl, serialized, "utf8"); +process.stdout.write(serialized); diff --git a/tests/simulator.captured-trace.test.ts b/tests/simulator.captured-trace.test.ts index 09a478e..7d31ac6 100644 --- a/tests/simulator.captured-trace.test.ts +++ b/tests/simulator.captured-trace.test.ts @@ -181,6 +181,8 @@ test("checked-in prefetch ablation preserves the captured result", () => { shiftCachePrefetch: boolean; shiftCacheJsdReweighting: boolean; shiftCacheTransitionRetention: boolean; + shiftCachePersistentDetector: boolean; + shiftCacheTriggeredReweighting: boolean; }; metrics: SimulationMetrics; }>; @@ -245,6 +247,8 @@ test("checked-in retention ablation isolates the two scoring factors", () => { shiftCachePrefetch: boolean; shiftCacheJsdReweighting: boolean; shiftCacheTransitionRetention: boolean; + shiftCachePersistentDetector: boolean; + shiftCacheTriggeredReweighting: boolean; }; detectedShiftEvents: Array<{ token: number; scoreBits: number }>; metrics: SimulationMetrics; diff --git a/tests/simulator.detector-sanity.test.ts b/tests/simulator.detector-sanity.test.ts new file mode 100644 index 0000000..482228b --- /dev/null +++ b/tests/simulator.detector-sanity.test.ts @@ -0,0 +1,187 @@ +import assert from "node:assert/strict"; +import { readFileSync } from "node:fs"; +import test from "node:test"; + +import { + DEFAULT_CONFIG, + generateRouterTrace, + runSimulation, + type SimulationControls, +} from "../lib/simulator"; + +const evidenceUrl = new URL( + "../evidence/synthetic/detector-sanity.json", + import.meta.url, +); + +interface DetectorEvidence { + benchmark: string; + evidenceBoundary: string; + preregistration: { publicPlanCommit: string }; + configuration: { + seeds: number[]; + tokens: number; + knownShiftToken: number; + postShiftWindowTokens: number; + shiftCacheParameters: { + persistentThresholdTokens: number; + detectorCooldownTokens: number; + triggeredReweightingTokens: number; + }; + fixedControls: SimulationControls; + triggeredControls: SimulationControls; + }; + records: Array<{ + seed: number; + domainShift: { + traceFingerprint: string; + detectedEventTokens: number[]; + preShiftEvents: number; + detectionDelayTokens: number | null; + detectedWithin64Tokens: boolean; + fixedPostShiftModeledLinkBytes: number; + triggeredPostShiftModeledLinkBytes: number; + postShiftModeledLinkBytesPercentChange: number; + }; + steady: { + detectedEventTokens: number[]; + wholeRunModeledLinkBytesPercentChange: number; + }; + }>; + summary: { + detectedWithin64Count: number; + medianDetectionDelayTokens: number; + stationaryFalseTriggersPer10000Tokens: number; + medianPostShiftModeledLinkBytesPercentChange: number; + postShiftMedianBootstrap95PercentInterval: { + lower: number; + upper: number; + }; + medianStationaryWholeRunModeledLinkBytesPercentChange: number; + }; + gates: Record; + carryForward: boolean; +} + +test("preregistered detector evidence preserves the failed carry-forward gate", () => { + const evidence = JSON.parse( + readFileSync(evidenceUrl, "utf8"), + ) as DetectorEvidence; + + assert.equal(evidence.benchmark, "synthetic-persistent-detector-sanity-v1"); + assert.match(evidence.evidenceBoundary, /deterministic synthetic fixtures/i); + assert.equal( + evidence.preregistration.publicPlanCommit, + "fe6042b6cd628d4976c2069a94d2739614ce9ce9", + ); + assert.deepEqual(evidence.configuration.seeds, [ + ...Array.from({ length: 30 }, (_, index) => 2300 + index), + ]); + assert.equal(evidence.configuration.tokens, 1024); + assert.equal(evidence.configuration.knownShiftToken, 512); + assert.equal(evidence.configuration.postShiftWindowTokens, 64); + assert.deepEqual(evidence.configuration.shiftCacheParameters, { + minimumShortWindowTokens: 4, + maximumShortWindowTokens: 12, + longWindowMultiplier: 4, + shiftThresholdBits: 0.28, + rearmThresholdBits: 0.12, + minimumTransitionObservations: 2, + maximumPrefetchesPerToken: 2, + persistentThresholdTokens: 3, + detectorCooldownTokens: 64, + triggeredReweightingTokens: 64, + }); + + assert.equal(evidence.records.length, 30); + assert.ok( + evidence.records.every( + (record) => + record.domainShift.detectedWithin64Tokens && + record.domainShift.detectionDelayTokens === 6 && + record.domainShift.preShiftEvents === 0 && + record.steady.detectedEventTokens.length === 0 && + record.domainShift.postShiftModeledLinkBytesPercentChange > 0, + ), + ); + assert.equal(evidence.summary.detectedWithin64Count, 30); + assert.equal(evidence.summary.medianDetectionDelayTokens, 6); + assert.equal(evidence.summary.stationaryFalseTriggersPer10000Tokens, 0); + assert.equal( + evidence.summary.medianPostShiftModeledLinkBytesPercentChange, + 11.52386217, + ); + assert.deepEqual( + evidence.summary.postShiftMedianBootstrap95PercentInterval, + { + resamples: 10000, + seed: 2339, + lowerPercentile: 2.5, + upperPercentile: 97.5, + lower: 10.78035747, + upper: 12.63942461, + }, + ); + assert.equal( + evidence.summary.medianStationaryWholeRunModeledLinkBytesPercentChange, + 0, + ); + assert.equal(evidence.gates.detection.passed, true); + assert.equal(evidence.gates.stationaryFalseTriggers.passed, true); + assert.equal(evidence.gates.postShiftTraffic.passed, false); + assert.equal(evidence.gates.stationaryTraffic.passed, true); + assert.equal(evidence.carryForward, false); +}); + +test("one checked-in detector record replays without routing or prefetch changes", () => { + const evidence = JSON.parse( + readFileSync(evidenceUrl, "utf8"), + ) as DetectorEvidence; + const record = evidence.records[0]; + const traceConfig = { + scenario: "domain-shift", + seed: record.seed, + tokens: evidence.configuration.tokens, + layers: DEFAULT_CONFIG.layers, + expertsPerLayer: DEFAULT_CONFIG.expertsPerLayer, + topK: DEFAULT_CONFIG.topK, + } as const; + const trace = generateRouterTrace(traceConfig); + const config = { ...DEFAULT_CONFIG, ...traceConfig, policy: "shift-cache" } as const; + const fixed = runSimulation( + config, + trace, + evidence.configuration.fixedControls, + ); + const triggered = runSimulation( + config, + trace, + evidence.configuration.triggeredControls, + ); + const start = evidence.configuration.knownShiftToken; + const end = start + evidence.configuration.postShiftWindowTokens; + const modeledBytes = (result: typeof fixed) => + result.timeline + .slice(start, end) + .reduce((sum, point) => sum + point.bytesTransferred, 0); + + assert.equal(triggered.traceFingerprint, record.domainShift.traceFingerprint); + assert.deepEqual(triggered.trace.selections, fixed.trace.selections); + assert.deepEqual( + triggered.timeline + .filter((point) => point.shiftDetected) + .map((point) => point.token), + record.domainShift.detectedEventTokens, + ); + assert.equal( + modeledBytes(fixed), + record.domainShift.fixedPostShiftModeledLinkBytes, + ); + assert.equal( + modeledBytes(triggered), + record.domainShift.triggeredPostShiftModeledLinkBytes, + ); + assert.equal(triggered.metrics.prefetchesIssued, 0); + assert.equal(triggered.metrics.prefetchBytesTransferred, 0); + assert.equal(triggered.metrics.semanticRoutingChanges, 0); +}); diff --git a/tests/simulator.test.ts b/tests/simulator.test.ts index a59a16c..fe4f20e 100644 --- a/tests/simulator.test.ts +++ b/tests/simulator.test.ts @@ -3,6 +3,7 @@ import test from "node:test"; import { DEFAULT_CONFIG, + DEFAULT_SIMULATION_CONTROLS, POLICY_IDS, SIMULATION_LIMITS, analyzeRouterTrace, @@ -367,11 +368,15 @@ test("config validation rejects unsafe or nonsensical settings", () => { shiftCachePrefetch: false, shiftCacheJsdReweighting: true, shiftCacheTransitionRetention: false, + shiftCachePersistentDetector: false, + shiftCacheTriggeredReweighting: false, }), { shiftCachePrefetch: false, shiftCacheJsdReweighting: true, shiftCacheTransitionRetention: false, + shiftCachePersistentDetector: false, + shiftCacheTriggeredReweighting: false, }, ); assert.throws( @@ -380,6 +385,8 @@ test("config validation rejects unsafe or nonsensical settings", () => { shiftCachePrefetch: "no", shiftCacheJsdReweighting: true, shiftCacheTransitionRetention: true, + shiftCachePersistentDetector: false, + shiftCacheTriggeredReweighting: false, } as never), /shiftCachePrefetch must be a boolean/, ); @@ -389,10 +396,29 @@ test("config validation rejects unsafe or nonsensical settings", () => { shiftCachePrefetch: true, shiftCacheJsdReweighting: true, shiftCacheTransitionRetention: true, + shiftCachePersistentDetector: false, + shiftCacheTriggeredReweighting: false, unsupported: true, } as never), /unsupported field.*unsupported/, ); + assert.throws( + () => + validateSimulationControls({ + ...DEFAULT_SIMULATION_CONTROLS, + shiftCacheTriggeredReweighting: true, + }), + /cannot both be enabled/, + ); + assert.throws( + () => + validateSimulationControls({ + ...DEFAULT_SIMULATION_CONTROLS, + shiftCacheJsdReweighting: false, + shiftCacheTriggeredReweighting: true, + }), + /requires shiftCachePersistentDetector/, + ); assert.throws( () => validateSimulationConfig(simulationConfig({ topK: 17 })), /topK cannot exceed expertsPerLayer/,