Skip to content

Add default-off classify early stopping - #132

Merged
scrimshawlife-ctrl merged 9 commits into
mainfrom
cursor/early-stop-train-loop-41af
Sep 27, 2026
Merged

scrimshawlife-ctrl merged 9 commits into
mainfrom
cursor/early-stop-train-loop-41af

Conversation

@scrimshawlife-ctrl

Copy link
Copy Markdown
Owner

Summary

Optional, default-off early stopping for the classify training loop. HYPERLEX_TRAIN_EPOCHS stays the max-epoch cap and does not turn this on.

  • Enable with HYPERLEX_EARLY_STOP=1, HYPERLEX_EARLY_STOP_PATIENCE, and HYPERLEX_EARLY_STOP_MIN_EPOCHS.
  • Supported only when HLX_SELECT_METRIC=classify_macro_f1_nonnone. Negative patience, invalid minimum epochs, and other selection metrics fail before the optimizer is constructed.
  • After a scored epoch records any strict improvement, stop when epoch_index - best_epoch >= patience and at least minimum_epochs epochs have been scored. Ties keep the earlier checkpoint and consume patience.
  • A best at epoch 3 with patience 4 and no later strict improvement stops after epoch 7 (8 epochs scored) when the cap is 12.
  • The declared metric's best checkpoint is still restored after early stopping or after the cap.
  • Every epoch-progress.jsonl row gains observational seconds, rounded with round() to 6 decimal places: epoch_wallclock_seconds and training_elapsed_seconds. They do not affect selection or stopping.
  • A completed train-receipt.json records stop_reason (max_epochs or early_stopping) and training_elapsed_seconds. A failed loop still raises and does not write that completion receipt.

Tests

Focused tests: tests/shadow/test_hyperlexical_early_stop.py (24 passed) on Python 3.10.21, 3.11.16, and 3.12.3, using an injected monotonic clock and no sleeps.

Full HYPERLEX_OFFLINE=1 HYPERLEX_NO_RATE_LIMIT=1 PYTHONPATH=src python3 -m pytest -q:

  • Python 3.10.21: 1039 passed, 13 skipped, 16 subtests passed
  • Python 3.11.16: 1039 passed, 13 skipped, 16 subtests passed
  • Python 3.12.3: 1047 passed, 7 skipped, 16 subtests passed

This does not seal an experiment, allocate a reserve, launch training, score a checkpoint, or move BEST. Do not merge from this description alone.

Optional patience stops only after a scored epoch records any strict improvement, and epoch progress plus the completion receipt gain observational wall-clock fields.
Optional patience stops only after a scored epoch records any strict improvement, and epoch progress plus the completion receipt gain observational wall-clock fields.
Optional patience stops only after a scored epoch records any strict improvement, and epoch progress plus the completion receipt gain observational wall-clock fields.
Optional patience stops only after a scored epoch records any strict improvement, and epoch progress plus the completion receipt gain observational wall-clock fields.
Optional patience stops only after a scored epoch records any strict improvement, and epoch progress plus the completion receipt gain observational wall-clock fields.
Optional patience stops only after a scored epoch records any strict improvement, and epoch progress plus the completion receipt gain observational wall-clock fields.
Optional patience stops only after a scored epoch records any strict improvement, and epoch progress plus the completion receipt gain observational wall-clock fields.
Optional patience stops only after a scored epoch records any strict improvement, and epoch progress plus the completion receipt gain observational wall-clock fields.
@scrimshawlife-ctrl
scrimshawlife-ctrl merged commit a7d254e into main Sep 27, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant