NNS 13.1 parity: repaired regression, stack, boost and VAR - #110
Merged
Conversation
…rule Adds nns._reg_engine, a port of the audited R regression engine merged in OVVO-Financial/NNS#41/#42: - one consistent prediction rule for fitted values and point estimates (range-normalized L1/L2/FACTOR dispatch, stable tie ordering, k = 1 exact-tie aggregation via gravity, eight-component ensemble weights, exact double-precision weighted class mode), vectorized with NumPy/SciPy and chunked for memory; - training-fitted encodings and dimension-reduction normalization (batch-independent predictions); - restricted automatic classification (categorical or exact binary 0/1); - predictive R2 = 1 - SSE/SST, quantile type-8 residual intervals with one row per prediction point; - order='max' fitted values and predictions from the same rule, replicating R's split() string-key grouping (%.15g) so regression points match R exactly; - round-half-up dependence-driven default order and sqrt(n_obs)-scaled default n.best; - strict argument validation mirroring the R messages. nns_reg now delegates to the engine; the previous implementation is retained as _nns_reg_legacy during the migration. Cross-validated against live merged R on shared data: 25-case battery (univariate orders NULL/2/max, multivariate L1/L2/default/explicit k, classification, dimension reduction, prediction intervals including below-range points) agrees to <= 5e-12, most cases <= 5e-15. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
- NA/NaN pairs are dropped (complete-case) while infinities propagate through mean/median partition centers, matching the repaired R NNS.part contract and test_NNS_reg_infinite.R (verified identical to live R output). - order='max' returns the explicit maximum partition representation: per-observation points for two-axis partitioning, unique-x groups reduced per noise.reduction for XONLY. - gravity/mode aggregation still filters to finite values, matching NNS.gravity / NNS.mode internals. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
Full rewrite of nns_stack against R/Stack.R at the NNS 13.1 Beta tip: - bounded small-k Method 1 search (k = 1..floor(sqrt(n)) plus the mandatory all-points candidate) scored per fold with early stopping, pooled out-of-fold aggregation, and first-tie selection; univariate folds use uniform k-NN cumulative means, multivariate folds score every k through the same prediction rule as the final NNS.reg fit (_mreg_predict_path over the fold RPM); - Method 2 selects the active dimension count from pooled OOF predictions of cumulative top-m X* projections; - OOF-optimized ensemble weights (0..1 grid) and a final classification threshold optimized directly on the weighted OOF blend; - rolling chronological ts.test folds, repeated stratified holdouts, and disjoint k-fold assignment; per-fold balancing; train-only X* scaling; - the response-offset guard against auto-classification, historical class-code contract with class.levels, strict argument validation; - candidates with no out-of-fold coverage score NA rather than an empty objective of 0, mirroring the upstream fix (OVVO-Financial/NNS#44). Adds _mreg_prepare (partition + RPM only) and _mreg_predict_path (k = 1..kmax path under the production rule) to the engine. Deviation by design: random fold assignment and balancing use a local numpy Generator, so randomized-fold outputs match R distributionally rather than bit-for-bit. The deterministic chronological ts_test mode was cross-validated against live R: all output fields (stack, reg, dim.red, objectives, weights, thresholds, selected n.best) agree to <= 7e-13 for method 1 and method c(1, 2), with the identical n.best selected; classification returns codes with the class.levels map. Runtime is on par with R's native path (0.27s vs 0.19s on the 500x5 ts.test benchmark). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
Full rewrite of nns_boost against R/Boost.R at the NNS 13.1 Beta tip: learner trials over feature subsets (exhaustive when 2^p - 1 <= learner.trials, otherwise unique random subsets) scored against a holdout, quantile type-2 (or extreme/user) survivor threshold, survivor-weighted epoch sampling, frequency-weighted synthetic X* final design with train-only normalization, local out-of-fold n.best selection over 1..sqrt(n_train) plus the all-points candidate, and a duplicated-X* final NNS.reg fit. depth passes through as order. Numeric class labels are restored in results and prediction intervals; factor/character responses keep integer codes per the R contract. Deterministic configuration (ts_test + exhaustive subsets) cross-validated against live R: results bit-identical (5e-15) with identical feature weights and frequencies. Randomized paths use a local numpy Generator (documented deviation from R's sample() stream). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
…ax IDs Bring the Python NNS.reg and NNS.M.reg paths to parity with the repaired R 13.1 implementation: - Faithful port of R's stats::smooth.spline(x, y, spar): penalized cubic B-spline fit with dependence-tied spar and R's Fortran band-trace ratio, fit once and reused for training and point predictions (replaces scipy GCV, which diverged badly). Tracks R to ~1e-4. - Encoded predictor column names threaded through the dim-reduction equation and the M.reg RPM/rhs.partitions/Fitted.xy tables, matching R's V1.. (matrix) / x (vector) / nm_level (dummy) naming. - order="max" multivariate IDs now use exact 1-based rank (R's match()) instead of rightmost-closed findInterval, keeping the maximum its own ID. - Dim-reduction NNS.caus weights call Uni.caus semantics directly (tau="ts" -> fixed lag 3), not the seasonal-period search. - Public nns_m_reg delegates to the repaired engine so multivariate output carries n.best/dist/class.levels and the repaired estimator. - _derivative suppresses subnormal-run overflow warnings (silent in R). Test harness updates for the repaired contracts: treat port None as R's NULL (dropped in jsonlite) for R2/SE/Point.est/Fitted.xy/class.levels, compare NNS.ID numerically or as strings by type, document the smooth.spline ~1e-4 deviation, and drop the removed noise="mode_class" option. Regenerate the R parity cache against live R 13.1; avoid the uninstalled data.table namespace in the harness encode helper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
The public bulk-distance path scrambled rows through a column-major reshape helper and never matched R for k > 1. R's NNS_distance_bulk_cpp uses the raw (un-normalized) d^2 + |d| distance with simple 1/d weights over the k nearest regression points and ignores the class flag, unlike the range-normalized eight-component single-point kernel. Compute the distances directly over the observation rows (stable sort for ties) so numeric and class bulk predictions match R exactly for k = 1, 2, and all. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
NNS.stack draws its validation splits (and balance resampling) from R's Mersenne-Twister stream after set.seed(seed). The port previously used a numpy Generator, so random-split objective scores and the n.best/threshold selected from them diverged from R, cascading into different final models. Add nns._rrng.RRNG, a faithful reproduction of R's default RNG: set.seed initialization, unif_rand (Mersenne Twister), R_unif_index (the R >= 3.6 "Rejection" sampler), runif, and sample/sample.int with and without replacement. Validated against live R for sample.int, permutations, and runif sequences. Wire it through stack.py's split generation and balance resampling so every random-split draw matches R exactly; OOF objectives, selected n.best, thresholds, and predictions now agree to ~1e-15. Update the stack parity tests to the repaired contracts: R's scalar NA_real_ placeholder for an absent method surfaces as None; the named weights vector vs dict; class.levels string comparison; and native character-class encoding (R no longer errors on raw labels). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
- NNS.CDF: label an unnamed matrix's Function/columns by their index
("1", "2", ...) as R does, rather than V1/V2.
- dy.d_ parity: the finite-difference derivatives are built on the
smooth-spline NNS.reg fit, so they inherit smooth.spline's ~1e-4
deviation amplified by the small step. Refresh the stale hardcoded
expectations to live R 13.1 (which the port matches), widen the dy.d_
tolerance to that documented level, and allow a small fraction of
smooth.spline-sensitive outliers in the average-partial-derivative sweep.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
- NNS.part now rejects a type other than NULL/'XONLY' with R's message instead of silently collapsing arbitrary types to XONLY. - Refresh the stale 13.1 smoke expectations (regression points, NNS.ARMA nonseasonal forecast, and the seeded stack sample) to live R 13.1. With the reproduced Mersenne-Twister stream the seeded stack now matches R exactly, including the dim-reduction threshold. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
The port evaluated Method 1 candidates with fold-local early stopping, giving larger k partial OOF coverage — exactly the behavior R's NNS.stack moved away from. Replace it with R's pooled scheme: evaluate every local candidate in every fold, take common_small_kmax = min over folds, then score against candidate k=1's coverage pattern (identical count vector required), break the local search on incomplete coverage or the diminishing-returns rule, and gate the ALL candidate the same way. Raise R's "no finite complete-coverage OOF" error when nothing is eligible. Stack parity is unchanged (40/40) and NNS.var now reproduces R's guard on degenerate inputs bit-for-bit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
Repin the parity harness to the authoritative merged R commit 21be6d9 (NNS 13.1 Beta + PR #44 guard) and regenerate the cached R references. With the reproduced RNG and pooled Method-1 coverage rule the balanced iris stack vignette now collapses to R's single repeated class exactly, so assert the port matches R's result rather than the old 13.0 divergence. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
Route NNS.boost's CV split, balance resampling, random feature subsets, and survivor-weighted epoch sampling through the reproduced R Mersenne stream (RRNG), adding R's ProbSampleNoReplace for the weighted draw. The deterministic-subset boosts now match R bit-for-bit. Add a random_seed alias (matching nns_stack) and update the parity tests to the repaired contracts: pred.int keys (pred.int.neg/pos), named feature-weight vector vs dict, native character-class encoding, and drop the removed _learner_scores internal-function tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
…ture checks Drop the stale random_seed overrides that predate the reproduced RNG (the port now shares R's default seed exactly), fix the class-structure helper to read the named feature-weight dicts, and refresh the pred.int shape assertions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
Port R's boost factor handling: per-learner NNS.reg now receives the raw categorical columns (factor.2.dummy encodes them), and the final frequency-weighted synthetic X* is built from a one-hot numeric design whose dummies each inherit their source feature's weight. Drop the removed factor_levels test argument; factor levels are inferred natively. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
Accept class_levels for the categorical response (matching nns_stack), and compare the named feature-weight/frequency dicts against R's serialized arrays in the class-metadata test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
Drop the removed internal-function tests (_cv_split, _stack_weights, _distance_*), add the weights/class.levels keys, accept ts_test=1, use pred.int.neg/pos, and treat an absent method's predictions as None. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
Read named feature-weight/frequency dicts, use pred.int.neg/pos, drop the removed factor_levels arg (native factor encoding) and the _nns_boost_core retry monkeypatch, and assert the stochastic epoch path now honors [threshold] as R does. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
M.reg point_only exposes the full key set with suppressed values None; pred.int uses pred.int.neg/pos with class bounds snapped to codes; the direct factor path rejection message changed; and NNS.part now supports order='max' instead of rejecting it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
The stochastic epoch path now honors [threshold], so its NotImplementedError guard no longer exists. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
… tests - nns_var: when the built-in co-movement ratio objective leaves every Method 1 candidate non-finite, retry that variable's stack fit with plain MSE (mirrors the R NNS.VAR fallback). A user objective is never overridden. - Regenerate the var R cache: previously-degenerate NNS.VAR cases now succeed under the fixed R and cache their references. - boost multi-factor parity: drop the random_seed=1 override so the port uses its default seed 123, matching R NNS.boost's own seed = 123L default. The earlier factor-level hypothesis was disproven -- R's output is invariant to factor level order; the gap was purely the seed mismatch. - iris balanced-boost: fix the stale feature.weights/frequency dict handling and rewrite the xfail reason to record the verified R sampling edge (R 4.3.3, NNS 13.1, RNGkind Rejection, seed 123). R now nails the holdout; the gap_is_explicit test pins the characterized balanced-sampling divergence. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPbZvtDw4h2XJo9w5hw57h
OVVO-Financial
marked this pull request as draft
July 14, 2026 00:40
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Completes the Python parity overhaul against repaired R NNS 13.1, including:
NNS.reg/NNS.M.regestimator parity;NNS.stackandNNS.boost;NNS.stackcandidate/final-estimator and complete-coverage invariants;NNS.VARMSE fallback matching the merged R-side correction;[0,1];Final validation
GitHub Actions runs 29297899818 and 29297948419 passed the complete matrix on Python 3.11, 3.12, 3.13, and 3.14.
Every version passed:
0 <= R2 <= 1;src/nnspackage;The one documented balanced-boost R sampling edge remains explicitly characterized in the test suite rather than hidden.
R dependency
The corresponding R-side Racine-Hastie restoration is in
OVVO-Financial/NNSPR #46. The earlierNNS.VARfallback was merged in R PR #45.