Fix: every optimizer formula shows the full path from loss to update - #13
Merged
Conversation
Several cards showed only the final step and expected the reader to
carry m, s, h from earlier cards — the loss never appeared, so the
formula couldn't be eyeballed on its own. Every card now shows the
complete pipeline (state feeds from ∇ℒ, then the parameter update),
repeating the EMA pattern deliberately so each card is self-contained:
- Adam: both memory feeds + the step (the bias-correction hats are
narrated in the prose right beside it)
- AdamW: the same two feeds, so the one change (+λθ) pops
- RAdam: feeds + the rectified step; ρ∞ bookkeeping stays in prose,
which now names the rectification factor r_t
- Sophia: the loss enters twice — m from ∇ℒ, h from diag(H)
- Prodigy: d-weighted feeds, the growing estimate, the step
- AdaDelta: the missing s-feed opens the chain
The core updateRuleLatex strings (the Formulas panel) are synced to
the same vocabulary — Prodigy's still had the old v/s notation, Sophia
and AdamW's were missing their feeds, and the tight (1{-}β) minus is
now the spaced (1-β) everywhere.
The panel's fit-to-width had a silent-clipping mode: at its 11.5px
floor, over-long rows were cut mid-formula (the Gaussian-Mixture model
row already hit this). Floor lowered to 10px — the complete rules fit
on laptop widths — and the viewport scrolls instead of clipping past
the floor.
Verified live: all fifteen cards contain ∇ℒ (Sophia also 𝐇) with zero
overflow at desktop width; formulas soft-wrap cleanly on mobile; the
panel shows Prodigy's full rule at 10.2px uncut; 194 tests,
svelte-check clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Deploying gradient-lab with
|
| Latest commit: |
d6a2b94
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://64ad844a.gradient-lab.pages.dev |
| Branch Preview URL: | https://fix-card-formulas-complete.gradient-lab.pages.dev |
…-equation KaTeX's inline soft-wrap could split a clause anywhere — on mobile, Prodigy's s-feed broke after '(1−'. Every formula with three or more clauses (Adam, AdamW, RAdam, Sophia, Prodigy, AdaDelta, Lion) is now an aligned block, one clause per row — it reads like the pseudocode it is, and a row either fits or the container scrolls; an equation never tears. Cards and the Formulas panel share the stacked form (the panel's AdaDelta keeps its honest θ ← θ + γΔθ tail). Bonus: the stacked Update row is much narrower, so the panel's fit recovers from its font floor — Prodigy now renders at 12.1px (was clipped at 11.5px, then 10.2px) with zero clipping on either axis. Verified: desktop cards read as clean rows; mobile shows zero overflow on all seven stacked cards with no mid-clause breaks; panel checked with Prodigy selected; 194 tests, svelte-check clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner
Author
|
Update per review: multi-clause formulas no longer rely on KaTeX's soft-wrap (which could break mid-clause on narrow screens). Every 3+-clause rule — Adam, AdamW, RAdam, Sophia, Prodigy, AdaDelta, Lion — is now an aligned block, one clause per row, in both the cards and the Formulas panel. A row either fits or scrolls; an equation never tears. Side benefit: the stacked Update row is narrower, so the panel's fit-to-width recovered from its font floor — Prodigy renders at 12.1px, fully uncut, and mobile shows all seven stacked cards with zero overflow. |
…verflow The overflow-x:auto fallback and the lowered font floor from two commits ago put a scrollbar on the Formulas panel for long problem formulas. Wrong trade for this app: the panel is a quiet reference, not a scroll region. Both changes reverted — MIN_PX back to 11.5, overflow back to hidden — restoring the panel's exact original behavior. The clipping this originally chased is gone anyway: the stacked one-clause-per-row update rules are far narrower than the old single lines, so the fit sits comfortably above its floor (Prodigy: 12.1px, update row and block fully inside the viewport, no scrollbar — verified live). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Centred against a four-row aligned block, the label floated beside the third row — reading as if the memory feeds above it belonged to the Gradient formula. Rows whose rule is stacked now top-align, with the label padded onto the first row's centreline; single-line rows are untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
Several optimizer cards showed only the last line of their algorithm — Adam's card had m̂ and ŝ appearing from nowhere, AdamW/RAdam/Sophia/Prodigy never mentioned ∇ℒ at all. The formula couldn't be eyeballed on its own; it expected the reader to carry state definitions over from other cards. The role of the formula is to show how you get from the loss to the change in the parameters — several didn't.
The fix
Every card formula now walks the complete pipeline — the state feeds from ∇ℒ, then the parameter update — repeating the EMA pattern deliberately so each card is self-contained:
m ← β₁m + (1−β₁)∇ℒ, s ← β₂s + (1−β₂)(∇ℒ)², θ ← θ − γ·m̂/(√ŝ+ε)(the hats are narrated in the prose directly beside it)+λθ— pops visuallyr_tmfrom ∇ℒ andhfrom diag(H)s-feed now opens the chain (the loss used to appear only mid-formula)The Formulas panel speaks the same language: the core
updateRuleLatexstrings are synced — Prodigy's still carried the oldv/sₜ₊₁notation from before the vocabulary unification, Sophia's and AdamW's panel strings were missing their feeds too, and the tight(1{-}β)minus is normalized everywhere.Plus a real bug found along the way: the panel's fit-to-width silently clipped rows mid-formula once its 11.5px font floor was hit (the long Gaussian-Mixture model row already triggered this on main). The floor is now 10px — the complete rules fit on laptop widths — and past the floor the viewport scrolls rather than truncating an equation.
Verified
svelte-checkclean; formula strings only — no behavior changes (golden-trajectory tests untouched)🤖 Generated with Claude Code