Integrate IJDS full-data credit controls and comparator audit#100
Draft
EigenCharlie wants to merge 45 commits into
Draft
Integrate IJDS full-data credit controls and comparator audit#100EigenCharlie wants to merge 45 commits into
EigenCharlie wants to merge 45 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Full-data contract and credit controls
A chunked audit reads all 2,925,493 raw rows and 142 columns. The archive contains 2,060,077 36-month and 865,415 60-month loans. The active 640,543-loan population is exhaustive for the declared 36-month binary terminal-default estimand:
Using the remaining archive as ordinary labeled training data would change the estimand or condition on maturity: 60-month loans have a different horizon, 48 fields are late-schema, and only 36.85%, 10.20%, and 1.15% of the 2014, 2015, and 2016Q1 cohorts have labels available by the March 2016 cutoff.
Five predeclared full-data scores are frozen before OOT outcomes:
All 45 OptBinning problems are OPTIMAL. Platform pricing/grade dominates its IV ranking, while FICO and recent inquiries lead the borrower-only model. The controls never enter portfolio optimization and are not a predictive leaderboard.
Scientific result
All 40 model-window all-candidate coverage upper bounds are below 0.90:
OOT AUC ranges from 0.612712 to 0.652244. Every calibration slope is below one and every calibration-in-the-large value is negative. Numeric logistic has development-to-OOT score PSI 0.005168 yet still fails coverage in every window, so marginal score stability alone is not a validity certificate.
CatBoost stratum 2 crosses alpha=0.10 from prevalence 0.101703 to 0.097147; its residual quantile changes from 0.888435 to 0.111801 and mean width from 0.984263 to 0.207631.
The locked two-ruler diagnostic contains 6,240 solves and 622,455 funded rows. Objective-matched coordinate .25 is one repeated four-month contrast: 44 loan-month positions, USD 155,937.27 one-way turnover, USD 5,603.66 higher standardized payoff, and 0.0068 pp lower default/miscoverage. Coordinate .50 is adverse and .75 is mostly unidentified for payoff/default. All normalized-score tracks are adverse, but do not match plug-in opportunity cost.
The exact point-cap frontier contains 3,067 caps. All 216 broad-stress envelopes cross zero, as do all 72 development-support default envelopes and all 27 W8 development envelopes. No universal policy direction is active.
Theory and claim boundary
The manuscript formalizes the binary residual phase transition, positive-affine cap equivalence, normalized-ruler invariance, same-cap nesting, C2 plug-in dominance, sharp common-outcome bounds, and basis-endpoint sufficiency over declared supports.
It does not claim selected-set conformal validity, latent-PD confidence intervals, causal effects, cash-flow return, a Markov certificate, prospective confirmation, a continuous joint frontier, scorecard superiority, universal robustness, fair-lending certification, or deployment readiness.
Implementation and reproducibility
dagshubremote are in sync; 26 new objects were pushed.Validation
just submission-check: pass.ty: zero diagnostics.just smokeandjust book: pass.Boundaries
No protected DVC stage was executed,
EXTRACTION_MANIFEST.jsonwas not modified, and no frozen champion or protocol artifact was overwritten. The rolling-origin evidence was deterministically rebuilt only to refresh two aggregation-source hashes; all derived tables and metrics remained byte-identical.This PR remains draft. Identity insertion, clean-clone release rehearsal, submission-week guideline recheck, sanitized upload archive, immutable release tag, and ScholarOne proof remain intentionally deferred until an explicit freeze decision.