Skip to content

Fix: every optimizer formula shows the full path from loss to update - #13

Merged
NeoVand merged 4 commits into
mainfrom
fix-card-formulas-complete
Jul 2, 2026
Merged

Fix: every optimizer formula shows the full path from loss to update#13
NeoVand merged 4 commits into
mainfrom
fix-card-formulas-complete

Conversation

@NeoVand

@NeoVand NeoVand commented Jul 2, 2026

Copy link
Copy Markdown
Owner

The problem

Several optimizer cards showed only the last line of their algorithm — Adam's card had m̂ and ŝ appearing from nowhere, AdamW/RAdam/Sophia/Prodigy never mentioned ∇ℒ at all. The formula couldn't be eyeballed on its own; it expected the reader to carry state definitions over from other cards. The role of the formula is to show how you get from the loss to the change in the parameters — several didn't.

The fix

Every card formula now walks the complete pipeline — the state feeds from ∇ℒ, then the parameter update — repeating the EMA pattern deliberately so each card is self-contained:

  • Adam: m ← β₁m + (1−β₁)∇ℒ, s ← β₂s + (1−β₂)(∇ℒ)², θ ← θ − γ·m̂/(√ŝ+ε) (the hats are narrated in the prose directly beside it)
  • AdamW: the same two feeds, so the one change — +λθ — pops visually
  • RAdam: feeds + the rectified step; the ρ∞ bookkeeping moves to prose, which now names the rectification factor r_t
  • Sophia: the loss enters twice — m from ∇ℒ and h from diag(H)
  • Prodigy: the d-weighted feeds, the growing distance estimate, the step — four clauses, fully traceable
  • AdaDelta: the missing s-feed now opens the chain (the loss used to appear only mid-formula)

The Formulas panel speaks the same language: the core updateRuleLatex strings are synced — Prodigy's still carried the old v/sₜ₊₁ notation from before the vocabulary unification, Sophia's and AdamW's panel strings were missing their feeds too, and the tight (1{-}β) minus is normalized everywhere.

Plus a real bug found along the way: the panel's fit-to-width silently clipped rows mid-formula once its 11.5px font floor was hit (the long Gaussian-Mixture model row already triggered this on main). The floor is now 10px — the complete rules fit on laptop widths — and past the floor the viewport scrolls rather than truncating an equation.

Verified

  • All fifteen cards contain ∇ℒ (Sophia also H) with zero horizontal overflow at desktop width — checked programmatically via the rendered KaTeX
  • Formulas soft-wrap cleanly on mobile (Prodigy's four clauses break into three tidy lines at 375px)
  • The panel renders Prodigy's complete rule at 10.2px, uncut, measured live
  • 194 tests green, svelte-check clean; formula strings only — no behavior changes (golden-trajectory tests untouched)

🤖 Generated with Claude Code

Several cards showed only the final step and expected the reader to
carry m, s, h from earlier cards — the loss never appeared, so the
formula couldn't be eyeballed on its own. Every card now shows the
complete pipeline (state feeds from ∇ℒ, then the parameter update),
repeating the EMA pattern deliberately so each card is self-contained:

- Adam: both memory feeds + the step (the bias-correction hats are
  narrated in the prose right beside it)
- AdamW: the same two feeds, so the one change (+λθ) pops
- RAdam: feeds + the rectified step; ρ∞ bookkeeping stays in prose,
  which now names the rectification factor r_t
- Sophia: the loss enters twice — m from ∇ℒ, h from diag(H)
- Prodigy: d-weighted feeds, the growing estimate, the step
- AdaDelta: the missing s-feed opens the chain

The core updateRuleLatex strings (the Formulas panel) are synced to
the same vocabulary — Prodigy's still had the old v/s notation, Sophia
and AdamW's were missing their feeds, and the tight (1{-}β) minus is
now the spaced (1-β) everywhere.

The panel's fit-to-width had a silent-clipping mode: at its 11.5px
floor, over-long rows were cut mid-formula (the Gaussian-Mixture model
row already hit this). Floor lowered to 10px — the complete rules fit
on laptop widths — and the viewport scrolls instead of clipping past
the floor.

Verified live: all fifteen cards contain ∇ℒ (Sophia also 𝐇) with zero
overflow at desktop width; formulas soft-wrap cleanly on mobile; the
panel shows Prodigy's full rule at 10.2px uncut; 194 tests,
svelte-check clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 2, 2026

Copy link
Copy Markdown

Deploying gradient-lab with  Cloudflare Pages  Cloudflare Pages

Latest commit: d6a2b94
Status: ✅  Deploy successful!
Preview URL: https://64ad844a.gradient-lab.pages.dev
Branch Preview URL: https://fix-card-formulas-complete.gradient-lab.pages.dev

View logs

…-equation

KaTeX's inline soft-wrap could split a clause anywhere — on mobile,
Prodigy's s-feed broke after '(1−'. Every formula with three or more
clauses (Adam, AdamW, RAdam, Sophia, Prodigy, AdaDelta, Lion) is now an
aligned block, one clause per row — it reads like the pseudocode it is,
and a row either fits or the container scrolls; an equation never tears.

Cards and the Formulas panel share the stacked form (the panel's
AdaDelta keeps its honest θ ← θ + γΔθ tail). Bonus: the stacked Update
row is much narrower, so the panel's fit recovers from its font floor —
Prodigy now renders at 12.1px (was clipped at 11.5px, then 10.2px)
with zero clipping on either axis.

Verified: desktop cards read as clean rows; mobile shows zero overflow
on all seven stacked cards with no mid-clause breaks; panel checked
with Prodigy selected; 194 tests, svelte-check clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@NeoVand

NeoVand commented Jul 2, 2026

Copy link
Copy Markdown
Owner Author

Update per review: multi-clause formulas no longer rely on KaTeX's soft-wrap (which could break mid-clause on narrow screens). Every 3+-clause rule — Adam, AdamW, RAdam, Sophia, Prodigy, AdaDelta, Lion — is now an aligned block, one clause per row, in both the cards and the Formulas panel. A row either fits or scrolls; an equation never tears. Side benefit: the stacked Update row is narrower, so the panel's fit-to-width recovered from its font floor — Prodigy renders at 12.1px, fully uncut, and mobile shows all seven stacked cards with zero overflow.

NeoVand and others added 2 commits July 2, 2026 00:10
…verflow

The overflow-x:auto fallback and the lowered font floor from two
commits ago put a scrollbar on the Formulas panel for long problem
formulas. Wrong trade for this app: the panel is a quiet reference, not
a scroll region. Both changes reverted — MIN_PX back to 11.5, overflow
back to hidden — restoring the panel's exact original behavior.

The clipping this originally chased is gone anyway: the stacked
one-clause-per-row update rules are far narrower than the old single
lines, so the fit sits comfortably above its floor (Prodigy: 12.1px,
update row and block fully inside the viewport, no scrollbar —
verified live).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Centred against a four-row aligned block, the label floated beside the
third row — reading as if the memory feeds above it belonged to the
Gradient formula. Rows whose rule is stacked now top-align, with the
label padded onto the first row's centreline; single-line rows are
untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@NeoVand
NeoVand merged commit 5a56b7e into main Jul 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant