Constrained refit with biological sign bounds - #5
Open
holoros wants to merge 1 commit into
Open
Conversation
Re-fit DG and HG equations using nlsLM with sign-bounded coefficients on the full 8.22M-row CONUS remeasurement panel + ClimateNA EMT/TD join. The previous unconstrained fits had biologically wrong-signed coefficients in 70% of species (HG_B4 alone in 56% of HG fits). The constrained refit pins those at the biological minimum while leaving free coefficients to adjust. Coverage: 113 DG species (was 84), 109 HG species (was 96). Validation against df_dg_res / df_hg_res on Douglas-fir: cor 0.9998 (DG) and 0.9965 (HG); RMSE 0.245 in / 3.64 ft. Reproducible 47-min Cardinal SLURM job; scripts/refit_constrained.R adds the bounded refit driver with a synthetic-data smoke test. Co-authored-by: Greg Johnson <greg@nosnhoj.org> Co-authored-by: David Marshall <davidkathymarshall@comcast.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-fits the DG and HG equations on a sign-bounded nlsLM using the full 8.22M-row CONUS remeasurement panel + ClimateNA EMT/TD raster join. The previous unconstrained fits had biologically wrong-signed coefficients in 70% of species across the union of DG and HG fits — HG_B4 (CCFL competition) negative in 56% of HG fits being the dominant issue. The constrained refit pins those coefficients at the biological minimum while leaving free coefficients to adjust.
Coverage: 113 DG species (up from 84) and 109 HG species (up from 96) qualified at the >= 5,000 obs threshold and converged.
Goodness of fit cost: median DG RSS/n essentially flat (1.03x ratio), median HG RSS/n actually slightly better (22.6 vs 23.5) because the larger panel improves the free coefficients. Across 82 shared DG species and 96 shared HG species, only 10 / 15 had constrained RSS/n more than 10% worse than the published unconstrained fit.
Validation: on a 3,000-row sample of df_dg_res.RDS / df_hg_res.RDS (Douglas-fir, SPCD 202), the constrained predictions track Greg's published predictions with correlation 0.9998 (DG) and 0.9965 (HG), RMSE 0.245 in / 3.64 ft. The HG bias of -0.5 ft is the predicted, biologically correct downward shift from removing the wrong-signed CCFL term.
Bound bindings table is in bound_violations_constrained.csv (added to a follow-up PR if helpful) — 91 individual bindings across 73 species, dominated by HG_B4 (54 species), HG_B3 (21), and HG_B8 (9), exactly the coefficients flagged in the README's "biologically unreasonable signs" caveat.
Reproducibility: scripts/refit_constrained.R (added in this PR) carries the bounded refit driver with a synthetic-data smoke test. The production run was a 47-min Cardinal SLURM job at /users/PUOM0008/crsfaaron/fvs-modern/calibration/refit_constrained/. The 11 species that initially failed with singular-gradient errors (mostly low-BAL Western junipers and pinyons) were recovered with a flatter B4 starting value (0.5 instead of 0.628).
Co-authored with Greg Johnson and David Marshall.