Skip to content

Test whether detected MoE shifts justify cache action - #9

Merged
Labeeb2339 merged 3 commits into
mainfrom
codex/shift-actionability-benchmark
Jul 19, 2026
Merged

Test whether detected MoE shifts justify cache action#9
Labeeb2339 merged 3 commits into
mainfrom
codex/shift-actionability-benchmark

Conversation

@Labeeb2339

Copy link
Copy Markdown
Owner

Summary

  • preregister a causal shift-actionability pilot before inspecting held-out seeds
  • add frozen execution plans, shadow decisions, finite-oracle diagnostics, cluster bootstrap intervals, and provenance hashes
  • publish the complete failed carry-forward result and surface it honestly in the workbench
  • add regression and replay tests for the checked-in evidence

Frozen result

  • controlled shifts detected: 30 / 30
  • gated actions: 76
  • harmful actions: 16 / 76 (21.05%)
  • worst regression: +21.34%
  • median benefit in the useful 64-slot subset: -9.75% modeled traffic
  • oracle-actionable coverage: 1 capacity, below the required 3
  • carry-forward decision: false

The detector passed, but the action failed its preregistered safety and coverage gates. This PR does not claim a new algorithm, quantization method, hardware measurement, or breakthrough.

Reproducibility

  • protocol commit: daf79356365b21249fde42012f1b446069809548
  • execution commit: 090481f4b710996a343e3b07332d69ff16c03258
  • result SHA-256: be80f2238f8dd33587f0bf1f00656d3a963138368c82ca7f8840e9863cb4f681
  • canonical payload SHA-256: 7b0b123d81344eb5b12e22f6539a266e00706b324e45160ff15854c3eca4f6c6
  • independent byte-for-byte verifier passed

Validation

  • npm run check — 42 unit tests, production build, and 2 rendered HTML tests passed
  • npm audit --omit=dev --audit-level=high — 0 vulnerabilities
  • desktop and 390 px mobile visual QA — no horizontal overflow or console warnings

@Labeeb2339
Labeeb2339 merged commit 6f76766 into main Jul 19, 2026
2 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 34d13120a7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

"provenance": {
"cleanTreeRequiredByRunner": true,
"executionCommit": "090481f4b710996a343e3b07332d69ff16c03258",
"implementationCommit": "090481f4b710996a343e3b07332d69ff16c03258",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Regenerate provenance for the submitted commit

In this reviewed commit, git log -1 --format=%H -- lib/simulator.ts lib/actionability.ts scripts/run-actionability-pilot.ts ... resolves to ef640752ef58e4b3f57f38863f95adeb1e14ae86, but the checked-in payload records 090481f4... as the implementation commit. Because --verify rebuilds provenance with latestCommitFor(implementationPaths) before doing the byte-for-byte JSON comparison, running npm run benchmark:actionability:verify from this commit will recompute ef640... here and reject the committed evidence; the same squash-history mismatch applies to the recorded protocol commit below. Regenerate the evidence/provenance on the commit being submitted, or keep the preregistration/execution commits in the submitted history.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant