Skip to content

Integrate IJDS full-data credit controls and comparator audit#100

Draft
EigenCharlie wants to merge 45 commits into
mainfrom
codex/crpto-ijds-exhaustive-audit-2026-07-09
Draft

Integrate IJDS full-data credit controls and comparator audit#100
EigenCharlie wants to merge 45 commits into
mainfrom
codex/crpto-ijds-exhaustive-audit-2026-07-09

Conversation

@EigenCharlie

@EigenCharlie EigenCharlie commented Jul 10, 2026

Copy link
Copy Markdown
Owner

Summary

  • Integrates the active V4 binary-geometry audit, the locked V1c/V2 two-ruler diagnostic, and a full-data credit-risk control lineage into one IJDS manuscript and evidence manifest.
  • Keeps CRPTO as one ML -> conformal -> optimization research object, framed as an identification audit rather than a winning credit policy.
  • Uses all rows eligible under the declared term, date, schema, maturity, and observability contracts; there is no computational sample or OOT model selection.
  • Remains deliberately pre-freeze: no learner, window, gamma, ruler, coordinate, policy, cap, or comparator is promoted.

Full-data contract and credit controls

A chunked audit reads all 2,925,493 raw rows and 142 columns. The archive contains 2,060,077 36-month and 865,415 60-month loans. The active 640,543-loan population is exhaustive for the declared 36-month binary terminal-default estimand:

  • PD development: 17,433 rows.
  • 2011 Platt/taxonomy block: 14,101 rows.
  • Residual pool: 49,007 rows.
  • Outcome-free policy development: 94,885 rows.
  • Primary OOT: 376,890 rows, including 11,551 unresolved outcomes.
  • Censored extension: 88,227 rows.

Using the remaining archive as ordinary labeled training data would change the estimand or condition on maturity: 60-month loans have a different horizon, 48 fields are late-schema, and only 36.85%, 10.20%, and 1.15% of the 2014, 2015, and 2016Q1 cohorts have labels available by the March 2016 cutoff.

Five predeclared full-data scores are frozen before OOT outcomes:

  1. active CatBoost + Platt;
  2. numeric logistic + Platt;
  3. CatBoost with 20 domain-signed monotonic constraints + Platt;
  4. a 26-field platform-signal OptBinning WOE/IV scorecard + Platt; and
  5. a 19-field borrower-only WOE/IV scorecard + Platt.

All 45 OptBinning problems are OPTIMAL. Platform pricing/grade dominates its IV ranking, while FICO and recent inquiries lead the borrower-only model. The controls never enter portfolio optimization and are not a predictive leaderboard.

Scientific result

All 40 model-window all-candidate coverage upper bounds are below 0.90:

  • CatBoost maximum: 0.882167.
  • Numeric logistic maximum: 0.895654.
  • Monotonic CatBoost maximum: 0.885991.
  • Platform WOE maximum: 0.894317.
  • Borrower-only WOE maximum: 0.896973.

OOT AUC ranges from 0.612712 to 0.652244. Every calibration slope is below one and every calibration-in-the-large value is negative. Numeric logistic has development-to-OOT score PSI 0.005168 yet still fails coverage in every window, so marginal score stability alone is not a validity certificate.

CatBoost stratum 2 crosses alpha=0.10 from prevalence 0.101703 to 0.097147; its residual quantile changes from 0.888435 to 0.111801 and mean width from 0.984263 to 0.207631.

The locked two-ruler diagnostic contains 6,240 solves and 622,455 funded rows. Objective-matched coordinate .25 is one repeated four-month contrast: 44 loan-month positions, USD 155,937.27 one-way turnover, USD 5,603.66 higher standardized payoff, and 0.0068 pp lower default/miscoverage. Coordinate .50 is adverse and .75 is mostly unidentified for payoff/default. All normalized-score tracks are adverse, but do not match plug-in opportunity cost.

The exact point-cap frontier contains 3,067 caps. All 216 broad-stress envelopes cross zero, as do all 72 development-support default envelopes and all 27 W8 development envelopes. No universal policy direction is active.

Theory and claim boundary

The manuscript formalizes the binary residual phase transition, positive-affine cap equivalence, normalized-ruler invariance, same-cap nesting, C2 plug-in dominance, sharp common-outcome bounds, and basis-endpoint sufficiency over declared supports.

It does not claim selected-set conformal validity, latent-PD confidence intervals, causal effects, cash-flow return, a Markov certificate, prospective confirmation, a continuous joint frontier, scorecard superiority, universal robustness, fair-lending certification, or deployment readiness.

Implementation and reproducibility

  • One evidence builder verifies three immutable outcome-free/evaluation pairs and emits one active manifest, ten tables, and six figure files.
  • Twelve DVC pointers preserve V4-v1/v2, two-ruler V1c/V2, and credit-control V1b/V2b.
  • The raw CSV pointer is healthy; the local file has the declared 1,773,470,505 bytes and DVC reports it up to date.
  • The manuscript, supplement, generated INFORMS TeX, claim registry, disclosure surfaces, book corrections, governance, and CRPTO skill are synchronized.
  • The active DVC capsule and dagshub remote are in sync; 26 new objects were pushed.
  • Core active scientific packages are already current in the resolved environment. Major pandas/PyArrow/OR-Tools/mypy upgrades are documented as new-run sensitivities, not invisible maintenance.

Validation

  • just submission-check: pass.
  • Full pytest suite: pass with two expected skips.
  • Ruff/format: 299 files clean.
  • Mypy: 316 source files clean.
  • Full pinned ty: zero diagnostics.
  • Strict historical manifest gate: pass.
  • Champion drift gate: exact zero drift for predictions, intervals, score edges, coverage cells, and floor multipliers.
  • just smoke and just book: pass.
  • Official INFORMS PDF: 29 pages; references begin on page 25, with 24 complete pre-reference pages and the conclusion on page 25.
  • Preview PDFs: body 22 pages, supplement 22 pages.
  • Main body: five tables, two figures, 218-word abstract, seven keywords.
  • BibTeX: zero warnings; no undefined citations or references.
  • Every page of all three PDFs was rendered and visually inspected; no clipping, overlap, blank page, broken table, missing glyph, or illegible figure was found.

Boundaries

No protected DVC stage was executed, EXTRACTION_MANIFEST.json was not modified, and no frozen champion or protocol artifact was overwritten. The rolling-origin evidence was deterministically rebuilt only to refresh two aggregation-source hashes; all derived tables and metrics remained byte-identical.

This PR remains draft. Identity insertion, clean-clone release rehearsal, submission-week guideline recheck, sanitized upload archive, immutable release tag, and ScholarOne proof remain intentionally deferred until an explicit freeze decision.

@EigenCharlie EigenCharlie changed the title Rebuild IJDS submission around maturity-safe evidence Rebuild and recover IJDS submission around maturity-safe evidence Jul 10, 2026
@EigenCharlie EigenCharlie changed the title Rebuild and recover IJDS submission around maturity-safe evidence Finalize IJDS submission around comparator stringency audit Jul 11, 2026
@EigenCharlie EigenCharlie changed the title Finalize IJDS submission around comparator stringency audit Rebuild IJDS submission as a fixed-taxonomy comparator audit Jul 12, 2026
@EigenCharlie EigenCharlie changed the title Rebuild IJDS submission as a fixed-taxonomy comparator audit Finalize IJDS temporal-transport and comparator audit Jul 12, 2026
@EigenCharlie EigenCharlie changed the title Finalize IJDS temporal-transport and comparator audit Finalize IJDS V4 binary-geometry and comparator audit Jul 13, 2026
@EigenCharlie EigenCharlie changed the title Finalize IJDS V4 binary-geometry and comparator audit Integrate IJDS V4 and locked two-ruler identification audit Jul 13, 2026
@EigenCharlie EigenCharlie changed the title Integrate IJDS V4 and locked two-ruler identification audit Integrate IJDS full-data credit controls and comparator audit Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant