Skip to content

fix: prevent LM damping underflow - #28

Merged
jlperla merged 1 commit into
mainfrom
codex/add-min-damping
Jul 23, 2026
Merged

fix: prevent LM damping underflow#28
jlperla merged 1 commit into
mainfrom
codex/add-min-damping

Conversation

@jlperla

@jlperla jlperla commented Jul 22, 2026

Copy link
Copy Markdown
Member

Summary

  • add min_damping as a traced LM hyperparameter
  • default the floor to jnp.finfo(residual.dtype).tiny
  • clamp damping before every linear solve and after each accept/reject update
  • document the distinction between an underflow floor and an eps * operator_scale conditioning floor
  • cover eager/JIT float32, LSMR, explicit floors, static cache keys, and float64 behavior

Root cause

Repeated accepted steps could push damping into a backend's flush-to-zero range. Once damping became exactly zero, the multiplicative update could not recover it. Rank-deficient factorizations could then produce rejected NaN trials indefinitely, while LSMR divided by sqrt(damping) directly.

Impact

The default now prevents literal damping underflow without imposing a materially larger regularization. Users can set a larger absolute min_damping when they need a conditioning-oriented floor.

Validation

  • uv run pytest -q — 565 passed, 12 skipped
  • uv run ruff check .
  • uv run --group docs mkdocs build --strict

@jlperla
jlperla marked this pull request as ready for review July 23, 2026 00:04
@jlperla
jlperla merged commit cf372ce into main Jul 23, 2026
4 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant