Fix silent NaN forecast from zero-variance lag-subsample in NNS.ARMA - #53
Merged
Conversation
Parallel, checkpointed M4 runner with correct sMAPE/MASE/OWA and a seasonally-adjusted Naive2. Per-series NNS forecast uses M4's declared seasonality as the nns_seas modulo and the default optim objective. Validation (30 series/subset): Hourly is strong (OWA ~0.53, near the M4 winners); Daily/Weekly stay ~3-4x worse than naive due to a catastrophic tail on near-random-walk series. Framing of the final example still TBD; no README/PR yet. Includes _modonly_probe.py scratch comparison. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MxVLKYqazC2uuAW3P3MbAm
Embed the M4 data download (cached in ./m4_data) and surface a CONFIG block: series count, modulo source, mod_only, optim-vs-plain nns_arma, explicit training_set, and a custom obj_fn hook. Run, tweak, compare OWA. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MxVLKYqazC2uuAW3P3MbAm
_numeric_seasonal_weights divided 1 / (subsample_CV / baseline_CV). When a seasonal lag-subsample is perfectly stable (CV 0) in an otherwise-varying series, that is 1/0 = Inf, and the weight normalisation collapses to Inf/Inf = NaN -- silently producing an all-NaN forecast for valid input (e.g. M4 Hourly series 131 at period 168). Floor the CV ratio so a maximally-seasonal lag gets a large but finite weight. A fully constant series (baseline CV 0 -> relative NaN) still propagates NaN, matching the R reference. Adds a regression test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MxVLKYqazC2uuAW3P3MbAm
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes a silent NaN forecast from
NNS.ARMAwhen a seasonal lag-subsample has zero variance.In
_numeric_seasonal_weights(src/nns/arma.py), the seasonal weight is1 / (subsample_CV / baseline_CV). When a lag-subsample is perfectly stable (CV = 0) within an otherwise-varying series, that's1/0 = Inf, and the subsequent weight normalisation collapses toInf/Inf = NaN. The NaN weight then multiplies a finite estimate, sonns_armareturns an all-NaN forecast for valid, finite input — with no error or warning. It also defeatsnns_arma_optim, which selects the period on a finite validation split and only blows up on the final full-series refit.Reproducer (M4 Hourly series 131, where
y[::-168] == [10, 10, 10, 10, 10]):Fix
Floor the CV ratio so a maximally-seasonal (CV 0) lag gets a large but finite weight instead of
Inf:np.maximumis a no-op for finiterelative).relativeNaN) still propagates NaN, matching the R reference and the existing parity test.Tests
test_numeric_seasonal_weights_constant_subsample_is_finite(constant lag-subsample in a varying series ⇒ finite weights summing to 1, finite forecast).tests/invariants/test_arma.py(19) andtests/property/test_arma.py(2) pass; the pre-existing constant-series NaN parity test still passes. Ruff clean.Note (R side)
The same guard applies to
tools/NNS/R/ARMA.R(theis.null(weights)weighting block) — handled separately.🤖 Generated with Claude Code
Generated by Claude Code