From b1ef54382275f623044f8abe0f9175f9317d3722 Mon Sep 17 00:00:00 2001 From: HiddenTrojan <93521146+Labeeb2339@users.noreply.github.com> Date: Sun, 19 Jul 2026 17:07:01 +0800 Subject: [PATCH] Correct research pitch and define actionability study --- docs/OUTREACH_DEMO.md | 73 ++++++++++++++++++++++++---- docs/SHIFTQ_MOE_RESEARCH_PLAN.md | 83 +++++++++++++++++++++++++++----- 2 files changed, 134 insertions(+), 22 deletions(-) diff --git a/docs/OUTREACH_DEMO.md b/docs/OUTREACH_DEMO.md index 7eff259..a6b54f1 100644 --- a/docs/OUTREACH_DEMO.md +++ b/docs/OUTREACH_DEMO.md @@ -1,8 +1,15 @@ # Two-minute outreach demo -## Opening — about 20 seconds +## Opening — about 30 seconds -> Hi, I'm Labeeb, a Form 3 student in Sarawak. I build applied AI systems, especially retrieval-augmented generation, agent harnesses, and experiments around efficient local inference. I wanted to show one research-style project and ask how I could develop it with stronger technical guidance. +> Hi, I'm Muhammad Labeeb Aryan, a Form 3 student in Sarawak. I build +> evidence-first AI and computer-science projects, especially RAG evaluation and +> efficient model inference. My current project, StrataMoE Lab, studies how +> Mixture-of-Experts models move expert weights through limited memory. I built a +> reproducible trace-and-simulation harness with a preregistered pass/fail gate, +> and the first idea failed in a useful way: it detected every controlled +> workload shift, but the intervention increased modeled transfer traffic. I +> published the negative result instead of tuning it away. ## Show StrataMoE Lab — about 70 seconds @@ -12,21 +19,35 @@ > Every policy receives the exact same router-selected experts. LRU uses recency, LFU uses accumulated frequency, and ShiftCache uses a short-versus-long Jensen-Shannon divergence signal plus recent transitions to adapt after the workload changes. The policy is only allowed to move expert weights; it cannot change routing. -4. Point to the provenance-bearing RouterTrace v2 fingerprint `d860285d` and the fixed result: +4. Point to the provenance-bearing RouterTrace v2 fingerprint `d860285d`, then + open the captured-trace and detector reports: - > On this one synthetic regression trace, ShiftCache moves 264 million modeled link bytes per token versus 282.67 million for LRU, which is 6.60% lower. Its modeled transfer stall is 5.36% lower. All three policies report zero semantic routing changes. + > My first synthetic trace was 6.60% better than LRU, so I tested the + > mechanism harder instead of treating that as proof. A pinned real-router + > trace exposed prefetch pollution. Then, in a preregistered 30-seed control, + > the detector found all 30 known shifts after a median six tokens with zero + > stationary false triggers, but the action it triggered increased modeled + > post-shift traffic by 11.52%. The counterintuitive finding is that detecting + > a workload change does not automatically make an intervention useful. 5. Show **Export router trace** and the methodology/limitations links: - > The trace can be exported and replayed, so the result is inspectable. This is not a real GPU speed benchmark yet; the next stage is real router traces, stronger workload-aware baselines, ablations, and hardware measurements. + > I froze the failed intervention rather than retuning it on the same seeds. + > My next question is when a shift is actually actionable: whether the new + > reuse pattern has enough benefit to repay cache churn and migration. This + > remains a simulator result, not a real GPU speed claim. ## The ask — about 30 seconds For Sarawak AI Centre: -> I would value feedback from someone working on inference systems or model serving. My next evidence step is to collect open MoE router traces and test whether the detector still helps against stronger baselines. Is there a researcher or engineer I could speak with for 20 to 30 minutes, or a small scoped experiment I could contribute to? +> Could I show you a five-minute reproducible demo and ask what evidence or +> small supervised contribution would make this useful to your team? My next +> evidence step is a preregistered actionability experiment on fresh open MoE +> router traces: compare correct change detection with whether an intervention +> actually repays its movement cost. -If speaking with Sarawak Microelectronics Design: +If speaking with SMD Semiconductor (Sarawak Microelectronics Design): > This project is currently software simulation, not chip design. I am also interested in a later hardware-aware side project around memory traffic, quantization, or edge inference. What beginner-accessible measurement or FPGA-oriented problem would be genuinely useful for me to study? @@ -34,25 +55,57 @@ If speaking with Sarawak Microelectronics Design: **Does it run a real model?** -> No. It replays router traces through a modeled hierarchy. I separated that clearly so I do not present simulated throughput as hardware evidence. +> The dashboard does not execute a model. One checked-in trace was captured +> from pinned `google/switch-base-8` execution, but its memory traffic and timing +> are still simulated. I keep that boundary explicit so I do not present modeled +> throughput as hardware evidence. **Is ShiftCache completely new?** -> I cannot claim that. Work such as MoE-Infinity, DALI, HybriMoE, and Colibrì already covers activation-aware or workload-aware caching. My current contribution is a transparent, deterministic harness for testing this specific JSD-triggered policy coupling. +> I cannot claim that. Work such as MoE-Infinity, DALI, HybriMoE, and Colibrì +> already covers activation-aware or workload-aware caching. My current +> contribution is a transparent deterministic harness, a captured router-trace +> pipeline, mechanism ablations, and a documented negative control for the +> tested JSD-gated action. + +**Did you make a breakthrough?** + +> Not yet. I have a reproducible counterintuitive finding and a preregistered +> negative control: accurate shift detection still produced a harmful cache +> action. A breakthrough claim would require a new method to beat strong +> baselines on held-out real models and named hardware, with quality checks and +> statistical replication. + +**Why show a failed intervention?** + +> Because the harness is useful only if it can falsify my idea. I kept the +> negative result, isolated prefetch pollution, and stopped the failed action +> instead of selecting only the synthetic trace where it won. **Why is this useful if it is only a simulator?** -> It makes the hypothesis falsifiable before spending time on a runtime integration. The same harness can expose failure cases, replay real traces, and compare policies without changing routing. +> It made the hypothesis falsifiable before a runtime integration. The harness +> has already exposed a failure case, replayed a captured real-model router +> trace, and compared policies without changing routing. **What would make it research-grade?** > Multiple real MoE model traces, preregistered seeds, faithful close baselines, detector and prefetch ablations, statistical uncertainty, then measurements on named hardware. +**Did AI tools write the project?** + +> I use AI coding tools, but I treat their output as untrusted. I require tests, +> pinned provenance, explicit evidence boundaries, and preregistered failure +> gates, and I can explain and reproduce every public claim. Keeping the negative +> result is evidence that I do not accept generated ideas just because they sound +> convincing. + ## Do not say - “I invented workload-aware MoE caching.” - “It makes AI inference 6.6% faster.” - “It preserves model quality” or “runs huge models on my laptop.” - “It is an IC-design project.” +- “I made a breakthrough” or “ShiftQ-MoE works.” Say **modeled**, **simulated**, **on this trace**, and **hypothesis**. That precision makes the work more credible, not less impressive. diff --git a/docs/SHIFTQ_MOE_RESEARCH_PLAN.md b/docs/SHIFTQ_MOE_RESEARCH_PLAN.md index 3c923e3..15d095c 100644 --- a/docs/SHIFTQ_MOE_RESEARCH_PLAN.md +++ b/docs/SHIFTQ_MOE_RESEARCH_PLAN.md @@ -31,10 +31,11 @@ vector representation. ## Why the current captured result matters StrataMoE's first pinned Switch-Base-8 trace is a failure case for the current -combined ShiftCache policy. ShiftCache moved 44.67% more modeled bytes than LFU -because 258 of 366 prefetches were unused. That result prevents a weak story in -which JSD scoring is assumed to help merely because it wins on one synthetic -fixture. +combined ShiftCache policy. ShiftCache moved 44.67% more modeled bytes than LFU, +and 258 of 366 issued prefetches were unused. The no-prefetch ablation identified +substantial prefetch pollution, although the remaining policy still lost to +LFU. That result prevents a weak story in which JSD scoring is assumed to help +merely because it wins on one synthetic fixture. Before any precision work, the captured-trace harness needed to separate: @@ -45,8 +46,9 @@ Before any precision work, the captured-trace harness needed to separate: That first mechanism matrix is now complete. It found prefetch pollution and a negative JSD-score result on this trace; it did not validate change detection. -ShiftQ-MoE should initially disable transition prefetching. Precision migration -must have its own explicit byte budget and hysteresis. +Any separate future precision study should initially disable transition +prefetching. Precision migration must have its own explicit byte budget and +hysteresis. ## Closest prior work and the remaining candidate gap @@ -56,9 +58,9 @@ must have its own explicit byte budget and hysteresis. | [DyMoE](https://arxiv.org/abs/2603.19172) (Sun Yat-sen University and Tencent authors) | Dynamic importance, mixed precision, look-ahead prefetching, and edge inference | Cross-request change detection plus an expert-specific damage table may be separable | | [HOBBIT](https://arxiv.org/abs/2411.01433) (Shanghai Jiao Tong University and CUHK authors) | Token-level mixed-precision loading, adaptive prefetching, and multidimensional caching | No explicit sequential workload change-point experiment is claimed here | | [MxMoE](https://arxiv.org/abs/2505.05799) | Calibration perturbation, activation frequency, hardware cost, and mixed-precision allocation | Strong static baseline rather than an online shift-triggered policy | -| [FRI-MxMoE](https://aclanthology.org/2026.acl-long.982/) (Xiamen University authors) | Fast quantization-error and runtime prediction followed by mixed-precision allocation | Static calibration rather than online sequential adaptation | +| [FRI-MxMoE](https://aclanthology.org/2026.acl-long.982/) (ACL 2026) | Fast quantization-error and runtime prediction followed by mixed-precision allocation | Static calibration rather than online sequential adaptation | | [MoPEQ](https://arxiv.org/abs/2509.02512) | Per-expert Hessian sensitivity combined with activation frequency | Already rules out novelty based only on sensitivity times frequency; remains a static baseline | -| [HCRMap](https://arxiv.org/abs/2607.11586) (Nanjing University authors) | Hotness drift, multi-tier residency, migration cost, and hysteresis | A serious novelty threat to generic migration-aware scheduling; the possible distinction is measured bit-level quality damage | +| [HCRMap](https://arxiv.org/abs/2607.11586) (Yongqin Zhang, Nanjing Vocational College of Information Technology; arXiv v1 preprint) | Pressure-aware hot-expert replica residency across stacked SRAM, local HBM, and shared DRAM, accounting for migration overhead, runtime pressure, minimum residency, and hysteresis | A serious novelty threat to generic multi-tier migration-aware scheduling; it does not evaluate bit-level quantization damage | | [KBVQ-MoE](https://arxiv.org/abs/2602.11184) | A genuine shared/expert vector quantizer with bias correction | Shows why creating a competitive new numerical quantizer is outside the first experiment's scope | The possible gap is only the intersection of all four items: @@ -69,7 +71,8 @@ The possible gap is only the intersection of all four items: 4. controlled evaluation on real router traces. This gap is provisional. Literature must be searched again immediately before -any public novelty statement, especially because HCRMap appeared in July 2026. +any public novelty statement, especially because HCRMap v1 was submitted on 13 +July 2026. ## Proposed method @@ -209,9 +212,10 @@ model run. ## Strict go/no-go gates -### Detector gate +### Detector gate for any materially different future design -Proceed only if real OLMoE traces satisfy every item: +Proceed only if a materially different detector is preregistered on fresh data +and real OLMoE traces satisfy every item: - at least two of three preregistered domain transitions exceed the stationary 99th-percentile JSD threshold; @@ -236,7 +240,7 @@ preregistered schedules, ShiftQ-MoE must: Failure means it remains an engineering experiment, not a new research method. A synthetic StrataMoE result alone can never pass this gate. -## Immediate next experiment +## Completed go/no-go experiment and decision Do not implement mixed precision yet. The pinned Switch-Base-8 mechanism matrix is now complete with prefetch disabled: @@ -295,3 +299,58 @@ The strongest honest title for the completed placement experiment is: > **When Change Detection Is Not Enough: A Negative Control for JSD-Gated MoE > Expert Placement** + +## Candidate continuation: shift actionability study + +This is a proposal, not a result and not yet a new benchmark. The next question +is narrower than designing another detector: + +> When does a correctly detected routing shift create enough placement headroom +> to repay cache churn and migration cost? + +That question must be tested against the legitimate **do nothing** decision. +Generic caching with switching costs, workload-aware MoE caching, and dynamic +expert promotion already exist, so change detection or migration awareness alone +cannot support a novelty claim. + +### Preregistered comparison matrix + +Use fresh held-out causal-MoE router traces with stationary, abrupt `A -> B`, +reversal `A -> B -> A`, gradual, and high-churn schedules. Cross each detector +with each action so detector quality is not confused with policy quality: + +- no detector and no action; +- a perfect boundary oracle used only as an upper-bound diagnostic; +- a persistent distribution-shift detector; and +- the frozen current detector, without retuning its published seeds; + +against: + +- no action; +- the frozen failed ShiftCache action; +- a simple LFU/LRU refresh baseline; +- a shadow-gated action that acts only when estimated reuse savings exceed its + movement cost; and +- a clairvoyant action oracle used only to measure available headroom. + +Report detector delay and false-trigger rate separately from false-action rate, +modeled demand-transfer bytes, migration bytes, time-to-benefit, and regret +against the action oracle. Any later latency claim requires a real implementation +and named hardware. + +### Provisional evidence gates + +Before describing the work as a benchmark contribution, require at least two +causal-MoE model families, three preregistered non-stationary schedules, +immutable trace and configuration hashes, faithful strong baselines, and a +clean-environment reproduction. A candidate action should also: + +- regress by no more than 2% on the worst held-out trace versus the strongest + non-adaptive baseline; +- improve median modeled traffic by at least 5% only where the action oracle + shows at least 10% available headroom; +- have a paired 95% confidence interval excluding zero; and +- preserve semantic routing exactly. + +Until those gates pass, the defensible description is **a preregistered +actionability experiment**, not a new method or a breakthrough.