Skip to content

feat(train): overfitting-risk and reproducibility cards (phase 4a) - #56

Merged
alikahwaji merged 1 commit into
mainfrom
feat/redesign-phase4-model-cards
Oct 29, 2026
Merged

alikahwaji merged 1 commit into
mainfrom
feat/redesign-phase4-model-cards

Conversation

@alikahwaji

Copy link
Copy Markdown
Contributor

Phase 4a of the redesign: the two cards designed in the mockups next to Regularisation. Based directly on main.

Overfitting risk

  • A Low / Moderate / High meter comparing trainable parameters with training points (e.g. 65 parameters · 160 training points).
  • Noisy data (> 30%) raises the risk by one band. Any L1, L2 or dropout lowers it by one.
  • Shows a one-line explanation and a suggestion that points to the Regularisation settings above it.
  • Updates live as layers, classes, samples, noise, validation split or regularisation change, and when data loads. Before any data is loaded, it estimates the training set from the sliders.
  • Level and meter come from one effective ratio, so the fill always sits inside the band its label names (covered by a property test).
  • The Regularisation section now starts open, because the card's suggestion refers to it.

Reproducibility

  • Seed field with a new-seed button, a weight-initialiser choice (He normal, Glorot uniform, LeCun normal) and a Lock seed checkbox.
  • New core/domain/Random.ts: a seedable Mulberry32 stream that now drives dataset generation, the train/validation split and batch order. When unseeded it falls back to Math.random, so existing test spies still work.
  • TF.js initialisers get a per-layer seed, so layers of the same shape don't start with identical weights.
  • Locked: the same seed repeats a run exactly. Unlocked: each dataset load draws a new seed, so the seed shown always describes the current run, and you can lock it after a run you liked.
  • Dropout is deliberately left unseeded: a seeded TF.js dropout reuses one mask on every batch, which would quietly turn dropout into a fixed, smaller network. The card's hint says so.
  • Seed and initialiser count toward the stale-model badge. An invalid seed is rejected with a validation message instead of silently falling back to random.

Testing

  • Unit: new suites for Random (including repeatable shuffles and datasets), OverfittingRisk, seeded TFNeuralNet initialisation, ReproducibilityPanel and OverfittingRiskCard. Full suite: 1,217 passed.
  • Browser check (CI build):
    • Seed 1234 locked, spiral, 8 epochs, run twice with a page reload in between: identical point positions, loss, validation loss and accuracy.
    • Seed 999 gives a different run.
    • Unlocked runs drew different seeds.
  • E2E (CI mode): 73 passed, 5 skipped. Visual baselines unchanged.
  • Screenshots checked at desktop and phone sizes. tsc and eslint are clean.

🤖 Generated with Claude Code

Overfitting risk: a Low/Moderate/High meter beside Regularisation that
compares trainable parameters with training points, adjusted for label
noise and active regularisation, with a one-line explanation and a
suggestion. Updates as layers, classes, samples, noise, split or
regularisation change and when data loads. Level and meter come from
one effective ratio, so the fill always sits in its labelled band.

Reproducibility: seed (with a new-seed button), weight initialiser (He
normal, Glorot uniform, LeCun normal) and a lock. A seedable random
source (Mulberry32) now drives dataset generation, the train/validation
split and batch order; TF.js initialisers get per-layer seeds. Locked,
the same seed repeats a run exactly; unlocked, each dataset load draws
a new seed so the seed shown always describes the current run. Dropout
stays unseeded on purpose: a seeded TF.js dropout reuses one mask on
every batch.

Seed and initialiser count as model settings for the stale-model badge,
and an invalid seed is rejected by InitializeNetworkCommand.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alikahwaji
alikahwaji merged commit 6364ec9 into main Oct 29, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant