Skip to content

research(H243): preregister zero-update module-wise soup of r882 and r782 - #2089

Merged
jepsontaylor merged 2 commits into
mainfrom
rd/h243-modsoup-prereg
Nov 3, 2026
Merged

jepsontaylor merged 2 commits into
mainfrom
rd/h243-modsoup-prereg

Conversation

@jepsontaylor

Copy link
Copy Markdown
Contributor

H243 prereg: zero-update module-wise soup of r882 and r782 (rd_opus)

Assignment: ~/nudges/hw_t0435_opus_h243.md, decided. Claim: H243_claim.txt, taken under
Hwave-admission.lock. This is a diagnostic only. DOPE c51a56d0 stays frozen, and the study uses read-only copies
of r882 (0084479a) and r782 (427b0d46).

What this preregisters

Groups. groups.json freezes 11 module groups that partition the 47 adapter tensors. Biases, norms and scalars
travel with their module.

Candidates. For each group g and β ∈ {0.5, 1.0}, r882 with g replaced by (1 − β)·r882_g + β·r782_g: 22
candidates at 0 optimizer updates. Each also gets a matched-L2 seeded random placebo (22 more).

Arms:

  • MODSWAP_SELECT (primary): the best single (g, β) per FIT cell, strict (gain > .001 and family LCB95 > 0, else
    r882).
  • GREEDY_SOUP (exploratory, Holm m = 2): Wortsman 2022 greedy forward addition on FIT. Groups whose best β beats
    r882 by more than .001 are tried in gain order, at most 5; an addition is kept only for a strict improvement above
    .001; the final soup gets the same strict check.
  • PLACEBO (report-only): the matched random move of the selected group.

Selection. On DL-Train FIT folds only (fold seeds 243141–243143, seeds 243101–243103, 15 cells). Decisions are
locked before any candidate is scored on DL-Val91, and each policy is scored once.

Decision on DL-Val91. A decision arm advances only if all four hold:

  • median gain > +0.002;
  • family LCB95 > 0;
  • Holm p < .05;
  • ≥ 2/3 seeds positive.

Also reported: Elo against r882 = 1500, the plateau gap, per-group L2 and harm families. The H152 cliff context
#27 is reported separately and never excluded.

Machinery: the merged H242/H241 zero-update harness, and the H239 A1 guard on xbabe2 cores 44–47 with a 07:30Z
stop.

Local checks

Check Result
pytest test_h243.py 15 passed (data-free)
Ruff check and format clean
git diff --check and JSON parse clean
7 records gates all rc 0

The gates ran on a privately copied, sha-verified jope-records (825c3ae7…).

Pre-merge smoke smoke-001: PASS, 0 contexts scored. It confirms:

  • DOPE c51a56d0;
  • the 11 groups partition the adapter;
  • r782's non-adapter tensors are identical to r882's;
  • 45 distinct policies;
  • placebo L2 matched;
  • group L2 at β = 1 ranges from 0.002 (kernel scalars) to 0.069 (traj cross-attention).

Not touched

xbabe0, xbabe3, any GPU, the live flywheel, cores 40–43 and 48–51, and PR #2071.

🤖 Generated with Claude Code

…r782

Agent-Session: rd_opus
Agent-Model: claude-opus-5-5
Agent-Task: H243 prereg
Agent-Session: rd_opus
Agent-Model: claude-opus-5-5
Agent-Task: H243 prereg

@jepsontaylor jepsontaylor left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same-agent review (rd_opus, claude-opus-5-5) of head 2ec83793422ce3e1354e37ee5d84d72f160f08b8. It is not independent and not a human review.

Scope reviewed. The full diff against origin/main:

  • the wave directory: PREREG, README, runner, soup, analysis, monitor, tests, groups.json, manifest, fold manifest,
    provenance and the metrics placeholder;
  • the coverage, test-inventory and research-source registrations;
  • the README census (1,151 items, 3,146 .py files, recounted from this tree);
  • the INDEX and overnight-index rows.

Science and integrity checks:

  • Groups. groups.json partitions exactly the 47 adapter parameters; the runner refuses otherwise.
  • Candidates. Soups touch only their recipe groups; β = 1 copies r782's bits.
  • Inputs. r782's non-adapter tensors must be bitwise equal to r882's. The smoke confirmed this, and 45 distinct
    hashes.
  • Placebo. It is matched-L2 to 1e-5 and report-only: outside Holm, and it never advances.
  • Greedy soup. It is pure in its score callback. Soups are evaluated on DL-Train only, cached, and locked with
    the other decisions in selection.json before any DL-Val91 scoring. Tests cover gain ordering, rejection of
    non-improving additions, the 5-ingredient cap, the strict LCB final check and the r882 fallback.
  • Ordering. Native91 is exact before TRAIN opens. CONTROL equals the native baseline on DL-Val. Context #27 is
    reported per policy and per arm and is excluded from no statistic.
  • Advance rule. All four gates, with Holm over MODSWAP_SELECT and GREEDY_SOUP.

Checks at this head:

  • pytest: 15 passed.
  • Ruff check and format: clean.
  • git diff --check and JSON parse: clean.
  • 7 records gates rc 0, on a sha-verified private copy of jope-records (825c3ae7…).
  • Pre-merge smoke-001: PASS with 0 contexts scored.

Findings:

  • None blocking.
  • Note. Group L2 at β = 1 is small (0.002–0.069) compared with H242's perturbations (0.14–4.4). The placebo
    controls for size.

Disposition: PASS. Merge with --match-head-commit 2ec83793422ce3e1354e37ee5d84d72f160f08b8.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant