Skip to content

Fix: one symbol vocabulary across the optimizer cards - #12

Merged
NeoVand merged 1 commit into
mainfrom
fix-card-notation
Jul 2, 2026
Merged

Fix: one symbol vocabulary across the optimizer cards#12
NeoVand merged 1 commit into
mainfrom
fix-card-notation

Conversation

@NeoVand

@NeoVand NeoVand commented Jul 2, 2026

Copy link
Copy Markdown
Owner

The inconsistencies

Reviewing all fifteen card equations side by side turned up five places where notation silently shifted meaning between cards:

  1. The squared-gradient memory kept changing identity. AdaGrad and RMSProp wrote plain s, AdaDelta wrote bold 𝐬 for the same quantity, Adam wrote plain ŝ — while its own repair cards (Nadam, AdamW, RAdam) wrote bold ŝ right underneath it.
  2. Prodigy resurrected v — the letter that meant velocity in the momentum act — for Adam's squared-gradient denominator, and used s for a different, paper-internal tally two symbols away from the family's s.
  3. Prodigy alone spoke in subscripted recursion (d_{t+1} = max(d_t, …)) where every other card uses the update style.
  4. Nesterov decorated one gradient with (θ) and left the other bare in the same line — inviting a hunt for a difference that doesn't exist.
  5. Two minus signs: Nadam and Lion used the tight (1{-}β), everyone else the spaced (1-β).

The convention (now recorded in DEVIATIONS.md)

  • Bold per-parameter state vectors (θ, m, v, s, u, h, c, w); plain scalars (γ, μ, ρ, β, λ, ε, t, d, r).
  • One meaning per letter across the family tree: m = gradient memory, v = velocity, s = squared-gradient memory, u = AdaDelta's step memory, h = Sophia's curvature diagonal, c = Lion's blend, w = Prodigy's alignment tally. (Adam's paper calls its buffer v; the cards deliberately say s so it can't read as velocity — Prodigy now follows.)
  • for updates, = for one-shot definitions; no time subscripts unless the quantity genuinely indexes time (β^t, ρₜ).
  • Gradients never carry an argument — they're always at the current point.
  • Collisions the papers own (RMSProp's decay ρ / Sophia's clip ρ / RAdam's ρₜ) are defused in prose — Sophia's card now carries the house "the alphabet is small and the field is greedy" note, its third verse.
  • The reveal-the-code files keep each paper's own variable names — the card is the classroom, the code is the primary source; the ledger states that split explicitly.

Prodigy's prose also now names r and w as the two running tallies of the ⟨g, x₀−x⟩ alignment, so no symbol in its formula is unexplained.

Verified

All fifteen rendered formulas checked in the live DOM via their KaTeX annotation sources; Prodigy card eyeballed in dark mode; 194 tests green, svelte-check clean. Content-only — no optimizer behavior changes.

🤖 Generated with Claude Code

The card equations drifted letter by letter as they were written; a
reader comparing adjacent cards could reasonably conclude a symbol had
changed meaning. Unified under one convention (recorded in
DEVIATIONS.md):

- The squared-gradient memory is one bold s everywhere: AdaGrad and
  RMSProp promote plain s to \mathbf{s} (matching AdaDelta), Adam's
  plain ŝ becomes the bold ŝ its own repair cards were already using
  (formula + all five prose mentions).
- Prodigy no longer resurrects v — the letter that meant velocity two
  acts earlier — for Adam's denominator: it is the family's s. Its
  paper-internal ℓ1 tally (confusingly also called s in the paper)
  becomes w, and the prose now names r and w as the two running
  tallies of the ⟨g, x₀−x⟩ alignment. The d-update drops subscripted
  recursion for the ← style every other card speaks.
- Nesterov's lone ∇ℒ(θ) argument decoration is gone — gradients are
  at the current point on every card, so it only invited a hunt for a
  difference that isn't there.
- One minus: the tight (1{-}β) in Nadam and Lion becomes the spaced
  (1-β) of every other card.
- Sophia's clip ρ gets the house defusal against RMSProp's decay ρ
  ('the alphabet is small and the field is greedy' — third verse).

The reveal-the-code files keep each paper's own names; the ledger now
states that split explicitly. Verified in the live DOM: all fifteen
rendered formulas checked via their KaTeX annotations; 194 tests,
svelte-check clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 2, 2026

Copy link
Copy Markdown

Deploying gradient-lab with  Cloudflare Pages  Cloudflare Pages

Latest commit: 7c2fe77
Status: ✅  Deploy successful!
Preview URL: https://5fa33072.gradient-lab.pages.dev
Branch Preview URL: https://fix-card-notation.gradient-lab.pages.dev

View logs

@NeoVand
NeoVand merged commit e4f9b13 into main Jul 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant