Record failed detector carry-forward gate

README.md

@@ -10,7 +10,7 @@ The repository implements an inspectable experiment, not a production inference
 
 - **LRU** keeps the most recently used experts.
 - **LFU** keeps the most frequently used experts over the run.
-- **ShiftCache** compares recent and longer-horizon activation distributions with Jensen-Shannon divergence (JSD), continuously reweights historical frequency, recent frequency, and recency, and can add observed one-step transition scores. Threshold-crossing events are currently telemetry; they do not yet trigger a separate controller action.
+- **ShiftCache** compares recent and longer-horizon activation distributions with Jensen-Shannon divergence (JSD), can continuously reweight historical frequency, recent frequency, and recency, and can add observed one-step transition scores. An opt-in persistent detector can gate a fixed intervention; its preregistered test failed the traffic gate.
 
 Every policy moves the requested expert weights; none may replace, skip, reroute, or silently lower the precision of a selected expert. `semanticRoutingChanges` should therefore remain zero.
 
@@ -64,11 +64,19 @@ only **0.52%** and **1.33%**, respectively. It was the best ShiftCache variant
 but still moved **11.50% more whole-run modeled bytes than LFU**. On this trace,
 JSD reweighting is negative evidence, not a result to promote as a win.
 
+A preregistered 30-seed synthetic sanity sweep then added three-token
+persistence, a 64-token cooldown, and a fixed 64-token event-gated intervention.
+It detected all 30 known midpoint shifts after six tokens and produced zero
+stationary false triggers, but it **increased** first-64-token post-shift modeled
+link bytes by a median **11.52%**; the paired-bootstrap 95% interval was
+**[+10.78%, +12.64%]**. The traffic gate failed, so this JSD-gated retention
+intervention is not being carried into ShiftQ-MoE.
+
 ## Why this experiment exists
 
 [Colibrì](https://github.com/JustVugg/colibri) demonstrates a real VRAM/RAM/storage hierarchy, per-layer caching, pinned hot experts, and live placement policies while preserving router semantics by default. [MoE-Infinity](https://arxiv.org/abs/2401.14361) studies request-level activation traces, predictive caching, and recovery after task or dataset shifts. [DALI](https://arxiv.org/abs/2602.03495) already proposes workload-aware cache replacement, and Colibrì now documents an LFRU-style live placement policy. Those systems make a broad “first workload-aware MoE cache” claim indefensible here.
 
-StrataMoE Lab instead contributes a small, reproducible surface for isolating one hypothesis: whether **router-distribution change signals** can help a non-semantic placement policy recover from abrupt shifts in a controlled trace. The first captured trace is a negative result for continuous JSD reweighting. A genuinely change-triggered controller still requires persistence, cooldown, and ground-truth timing evaluation before it exists in this project. Scientific novelty would still require multiple real router traces, stronger baselines, hardware measurements, ablations, and statistical replication.
+StrataMoE Lab instead contributes a small, reproducible surface for isolating one hypothesis: whether **router-distribution change signals** can help a non-semantic placement policy recover from abrupt shifts in a controlled trace. The first captured trace is a negative result for continuous JSD reweighting, and the preregistered synthetic sweep is a negative result for the tested event-gated intervention. Scientific novelty would still require a new out-of-sample hypothesis, multiple real router traces, stronger baselines, hardware measurements, ablations, and statistical replication.
 
 ## Quick start
 
@@ -95,6 +103,7 @@ npm run benchmark
 npm run benchmark:captured
 npm run benchmark:captured:prefetch
 npm run benchmark:captured:retention
+npm run benchmark:detector
 ```
 
 The dashboard lets you choose a scenario, seed, token count, model shape, cache capacity, expert size, and modeled bandwidths; run all policies; inspect per-token behavior and final tier residency; and export or import deterministic router traces.
@@ -121,6 +130,7 @@ When reporting a result, include:
 - [Related work](docs/RELATED_WORK.md) — primary papers and official repositories, including work from Chinese universities and labs
 - [Limitations](docs/LIMITATIONS.md) — what the simulator cannot support as a claim
 - [Captured Switch-Base-8 trace](docs/CAPTURED_SWITCH_TRACE.md) — pinned model execution, provenance, reproduction, and replay boundary
+- [Preregistered detector sanity sweep](docs/DETECTOR_SANITY.md) — frozen gates, per-seed synthetic result, and failed carry-forward decision
 - [ShiftQ-MoE research plan](docs/SHIFTQ_MOE_RESEARCH_PLAN.md) — closest prior work, candidate method, baselines, ablations, and strict go/no-go gates
 - [Two-minute outreach demo](docs/OUTREACH_DEMO.md) — an honest walkthrough and ask for the 2026-07-21 call
 
@@ -131,7 +141,7 @@ When reporting a result, include:
 1. **Trace harness:** deterministic synthetic traces, inspection UI, fixed regression benchmark, and honest modeled metrics.
 2. **Real-trace replay:** collect router IDs from multiple open MoE families and compare stronger activation-aware baselines.
 3. **Runtime validation:** integrate a frozen policy into one open runtime and measure actual traffic, TTFT, TPOT, and energy on named hardware.
-4. **Precision study:** only after the placement result is understood, test the falsifiable [ShiftQ-MoE research plan](docs/SHIFTQ_MOE_RESEARCH_PLAN.md) with expert-balanced calibration, per-expert quality evidence, close baselines, and preregistered failure gates.
+4. **Precision study:** paused for the current JSD-gated design because its preregistered placement gate failed. Any future precision study must begin as a separately registered static sensitivity baseline or supply new out-of-sample detector evidence before reviving a shift-triggered claim.
 
 The source code is licensed under the [Apache License 2.0](LICENSE).
 

docs/DETECTOR_SANITY.md

@@ -0,0 +1,79 @@
+# Preregistered detector sanity sweep
+
+## Question
+
+Can a persistent JSD event safely gate temporary ShiftCache reweighting on
+deterministic synthetic traces with a known midpoint change?
+
+This is an engineering sanity check. It is not evidence about semantic domains,
+language-model quality, a physical memory hierarchy, or ShiftQ-MoE novelty.
+
+## Preregistration
+
+The seed range, token count, requirement to add persistence/cooldown before the
+run, and four gates were merged in commit
+[`fe6042b`](https://github.com/Labeeb2339/stratamoe-lab/commit/fe6042b6cd628d4976c2069a94d2739614ce9ce9)
+before the sweep ran:
+
+- seeds `2300` through `2329`;
+- `1024` tokens per stationary and abrupt-shift trace;
+- known synthetic change at token `512`;
+- detection, stationary false-trigger, post-shift traffic, and stationary
+  traffic gates with fixed thresholds.
+
+The implementation was then written and frozen before its first sweep with:
+
+- three consecutive above-threshold JSD scores;
+- a 64-token cooldown and 64-token triggered reweighting interval;
+- prefetch and transition retention disabled in both arms; and
+- fixed frequency/recency scoring as the paired baseline.
+
+The triggered arm uses maximum short-window/recency reweighting for 64 tokens
+after an accepted event. Parameters were not changed after the results were
+observed. The exact `3/64/64` implementation values were not independently
+timestamped in the earlier public commit, so they are a frozen reproducibility
+record rather than a fully public preregistration claim.
+
+## Gates and result
+
+| Gate | Requirement | Result | Pass |
+| --- | --- | ---: | :---: |
+| Detection | At least 27/30 shifts within 64 tokens | 30/30; median delay 6 tokens | Yes |
+| Stationary false triggers | At most 1 per 10,000 tokens | 0 | Yes |
+| Post-shift traffic | Median change at most -5%; bootstrap interval below 0 | **+11.52%**, 95% interval **[+10.78%, +12.64%]** | **No** |
+| Stationary traffic | Median regression no greater than 2% | 0% | Yes |
+
+All 30 triggered runs moved more modeled link bytes than their paired fixed
+baseline over tokens `512` through `575`. The detector recognized this strong,
+synthetic boundary, but its prescribed cache intervention consistently made the
+placement outcome worse.
+
+The registered decision is therefore:
+
+> **Do not carry this JSD-gated retention intervention into ShiftQ-MoE.**
+
+This result rejects one mechanism, not every possible change detector. Retuning
+the threshold, hold interval, or weights on these same 30 seeds would invalidate
+the preregistered test and is intentionally not done.
+
+## Reproduce
+
+```bash
+npm ci
+npm run benchmark:detector
+```
+
+The run takes roughly two minutes on the development machine. The complete
+per-seed record, controls, fingerprints, bootstrap settings, gate decisions,
+and evidence boundary are checked in as
+[`detector-sanity.json`](../evidence/synthetic/detector-sanity.json).
+
+The result may be described as:
+
+> On 30 deterministic synthetic traces with a known midpoint change, the
+> persistent detector fired after six tokens with no stationary false triggers,
+> but its fixed 64-token intervention increased simulated post-shift link traffic
+> by a median 11.52%; the preregistered carry-forward gate failed.
+
+It must not be described as a real-model inference slowdown, a quantization
+result, or proof that all distribution-change detection is ineffective.

docs/METHODOLOGY.md

@@ -69,10 +69,10 @@ Jensen-Shannon divergence is symmetric and finite for discrete distributions wit
 
 As divergence rises, ShiftCache continuously reduces the weight assigned to long-term frequency and increases the influence of recent frequency, recency, and optionally learned one-step expert transitions. A transition score estimates which expert is likely after recently observed experts. Prefetching may promote an exact expert earlier; a wrong prefetch wastes bandwidth or cache capacity but must not change later router selections.
 
-The current threshold-crossing event is telemetry only. It does not gate the
-retention formula or trigger a separate policy update. This implementation is
-therefore described as **JSD-score reweighting**, not a completed
-change-triggered detector/controller.
+In the original/default mode, a threshold-crossing event is telemetry only and
+the JSD score changes retention continuously. The preregistered sanity sweep
+adds an opt-in persistent detector and a fixed event-gated intervention without
+changing the historical continuous-scoring result.
 
 This combination is a **project hypothesis**, not a claim that distribution-aware caching, recency/frequency hybrids, transition prediction, or workload adaptation are new. MoE-Infinity studies request-level activation patterns and workload changes; DALI studies workload-aware cache replacement; HybriMoE uses score-based caching for unstable activations; and Colibrì documents live LFRU placement.
 
@@ -161,9 +161,13 @@ reduced them by 0.52%. The combined score was 2.68% worse than the fixed score,
 and every ShiftCache variant remained worse than LFU. These are descriptive
 results from one 215-token encoder trace, not estimates of a population effect.
 
-The next detector experiment must implement explicit persistence and cooldown,
-record known synthetic change points, and evaluate delay, misses, and stationary
-false triggers across a preregistered seed set before any threshold is changed.
+The preregistered detector sweep is now complete. Three-token persistence detected all 30 known
+midpoint shifts with a six-token median delay and no stationary false triggers.
+However, the registered 64-token intervention increased paired post-shift
+modeled link bytes by a median 11.52%, with a paired-bootstrap 95% interval of
+[+10.78%, +12.64%]. The traffic gate failed, so the current JSD-gated retention
+mechanism is stopped rather than retuned on the evaluated seeds. See
+[Preregistered detector sanity sweep](DETECTOR_SANITY.md).
 
 ## 9. Path to real-trace validation
 

docs/SHIFTQ_MOE_RESEARCH_PLAN.md

@@ -3,8 +3,9 @@
 **Working question:** Does router-distribution change detection improve
 mixed-precision expert residency under sequential workload shifts?
 
-**Status on 2026-07-19:** candidate research direction, not a claimed new
-algorithm, quantizer, thesis result, or performance improvement.
+**Status on 2026-07-19:** current JSD-gated design stopped after a
+preregistered traffic gate failed; not a claimed new algorithm, quantizer,
+thesis result, or performance improvement.
 
 ## Recommendation
 
@@ -253,15 +254,18 @@ LFU over the full replay. The JSD threshold crossings were at token positions
 and 167. This is negative evidence for the current continuous JSD reweighting
 rule on one short encoder trace.
 
-The implementation also exposed a terminology boundary: a threshold crossing
-is currently telemetry only. The eviction weights change continuously with the
-JSD score, so StrataMoE does not yet contain a change-triggered controller.
+That checkpoint also exposed a terminology boundary: its threshold crossing
+was telemetry only while the eviction weights changed continuously with the JSD
+score. The following preregistered control added a separate event-gated mode.
 
-The next preregistered control is therefore a detector sanity experiment, not
-quantization. Use seeds `2300` through `2329`, `1024` tokens, and otherwise the
-fixed default synthetic configuration for both stationary and abrupt-shift
-traces. Add explicit persistence and cooldown before observing the sweep. Carry
-JSD gating forward only if all of these provisional engineering gates pass:
+The next preregistered control was therefore a detector sanity experiment, not
+quantization. It used seeds `2300` through `2329`, `1024` tokens, and otherwise
+the fixed default synthetic configuration for both stationary and abrupt-shift
+traces. The public plan required persistence and cooldown before observing the
+sweep; the implementation then froze three-token persistence, a 64-token
+cooldown, and a fixed 64-token intervention before its first run. Those exact
+numeric implementation values were not in the earlier public commit. JSD gating
+could continue only if all of these provisional engineering gates passed:
 
 1. at least 27 of 30 abrupt shifts trigger within 64 tokens of the known change;
 2. stationary traces produce at most one false trigger per 10,000 tokens;
@@ -270,11 +274,24 @@ JSD gating forward only if all of these provisional engineering gates pass:
    below zero; and
 4. stationary whole-run modeled bytes regress by no more than 2%.
 
-These synthetic gates can reject a broken mechanism but cannot pass the real
-OLMoE detector gate or support a novelty claim. Only a stable detector and
-allocation signal should become the foundation for ShiftQ-MoE.
+The detector passed the timing controls: 30 of 30 shifts triggered after six
+tokens, with zero stationary false triggers. The intervention failed the
+traffic gate decisively. It increased first-64-token post-shift modeled link
+bytes by a median **11.52%**, with a paired-bootstrap 95% interval of
+**[+10.78%, +12.64%]**. Stationary traffic was unchanged.
 
-The strongest honest working title is:
+The registered outcome is `carryForward = false`. The current JSD-gated
+retention design stops here and must not be tuned on the same seeds. These
+synthetic gates can reject a broken mechanism but could never pass the real
+OLMoE detector gate or support a novelty claim.
 
-> **ShiftQ-MoE: Does Router-Distribution Change Detection Improve
-> Mixed-Precision Expert Residency?**
+Do not implement the proposed ShiftQ-MoE mixed-precision layer on top of this
+failed signal. A future quantization study should start separately with a static
+expert-sensitivity baseline, or preregister a materially different detector on
+new out-of-sample traces. It must not reuse the ShiftQ-MoE name as if this gate
+had passed.
+
+The strongest honest title for the completed placement experiment is:
+
+> **When Change Detection Is Not Enough: A Negative Control for JSD-Gated MoE
+> Expert Placement**

evidence/switch-base-8/prefetch-ablation.json

@@ -48,7 +48,10 @@
     "shiftThresholdBits": 0.28,
     "rearmThresholdBits": 0.12,
     "minimumTransitionObservations": 2,
-    "maximumPrefetchesPerToken": 2
+    "maximumPrefetchesPerToken": 2,
+    "persistentThresholdTokens": 3,
+    "detectorCooldownTokens": 64,
+    "triggeredReweightingTokens": 64
   },
   "traceDiagnostics": {
     "totalExpertSelections": 1290,
@@ -63,7 +66,9 @@
       "controls": {
         "shiftCachePrefetch": true,
         "shiftCacheJsdReweighting": true,
-        "shiftCacheTransitionRetention": true
+        "shiftCacheTransitionRetention": true,
+        "shiftCachePersistentDetector": false,
+        "shiftCacheTriggeredReweighting": false
       },
       "metrics": {
         "gpuHitRate": 0.32325581,
@@ -99,7 +104,9 @@
       "controls": {
         "shiftCachePrefetch": false,
         "shiftCacheJsdReweighting": true,
-        "shiftCacheTransitionRetention": true
+        "shiftCacheTransitionRetention": true,
+        "shiftCachePersistentDetector": false,
+        "shiftCacheTriggeredReweighting": false
       },
       "metrics": {
         "gpuHitRate": 0.28062016,

evidence/switch-base-8/retention-ablation.json

@@ -49,7 +49,10 @@
     "shiftThresholdBits": 0.28,
     "rearmThresholdBits": 0.12,
     "minimumTransitionObservations": 2,
-    "maximumPrefetchesPerToken": 2
+    "maximumPrefetchesPerToken": 2,
+    "persistentThresholdTokens": 3,
+    "detectorCooldownTokens": 64,
+    "triggeredReweightingTokens": 64
   },
   "semanticGroupBoundaries": [
     {
@@ -79,7 +82,9 @@
       "controls": {
         "shiftCachePrefetch": false,
         "shiftCacheJsdReweighting": false,
-        "shiftCacheTransitionRetention": false
+        "shiftCacheTransitionRetention": false,
+        "shiftCachePersistentDetector": false,
+        "shiftCacheTriggeredReweighting": false
       },
       "detectedShiftEvents": [
         {
@@ -172,7 +177,9 @@
       "controls": {
         "shiftCachePrefetch": false,
         "shiftCacheJsdReweighting": true,
-        "shiftCacheTransitionRetention": false
+        "shiftCacheTransitionRetention": false,
+        "shiftCachePersistentDetector": false,
+        "shiftCacheTriggeredReweighting": false
       },
       "detectedShiftEvents": [
         {
@@ -265,7 +272,9 @@
       "controls": {
         "shiftCachePrefetch": false,
         "shiftCacheJsdReweighting": false,
-        "shiftCacheTransitionRetention": true
+        "shiftCacheTransitionRetention": true,
+        "shiftCachePersistentDetector": false,
+        "shiftCacheTriggeredReweighting": false
       },
       "detectedShiftEvents": [
         {
@@ -358,7 +367,9 @@
       "controls": {
         "shiftCachePrefetch": false,
         "shiftCacheJsdReweighting": true,
-        "shiftCacheTransitionRetention": true
+        "shiftCacheTransitionRetention": true,
+        "shiftCachePersistentDetector": false,
+        "shiftCacheTriggeredReweighting": false
       },
       "detectedShiftEvents": [
         {