Feat/71 kindex quantile policy - #79
Merged
Merged
Conversation
goldilocks.k_index.ladder_1based.max50.v1 declared a 50-per-axis enumeration cap that never existed for this ladder -- it was always built to a minimum k-distance instead. Replace it with ladder_1based.v2, carrying that floor and the dataset's convergence criterion as structured ContractSpec fields rather than encoding them into the contract name, which can only ever hold part of the semantics and would need a fresh release for every added dimension.
KIndexQRFPredictor hardcoded index_base=0 and max_kpoints_per_axis=50 in every prediction's details, regardless of which release a record declared. That was silently correct while only one release existed; serving the 1-based contract through the same runtime would have kept reporting the 0-based boundary. Read both from the loaded contract instead, via an explicit per-contract table rather than inferring the base from the contract string.
Retrains on 52713-55d86, which already carries the corrected, floor-based 1-based ladder, instead of shifting the superseded d5ds2-64f16 labels by one -- that shift is not uniform for every structure, since the 1-based enumeration also drops a small number of repeated meshes the old per-axis count did not. The publishing policy (q0.90 rounded, banded to hold under-prediction at 6%) is ported forward unchanged so this change measures only the effect of the ladder move; band edges are shifted from [6, 11] to [7, 12] to cut at the same physical mesh density the superseded protocol did, matching the precedent already set by the k-index screen's own threshold=12. Under the same floor this record's own labels select a higher quantile level (0.95, not 0.90): the estimator is essentially unchanged in skill (test MAE at the median 1.124 against 1.118), but the safety policy now costs more mesh, and the top of the ladder is measurably harder (17.8% true-rung under-prediction at the rung-12 cut against 14.6% before). Both are recorded in the model card rather than assumed away because the contract lines up. Reseals the shared 52713-55d86 snapshot to replace the raw ladder's superseded contract string, updating the k-index screen protocol's pin to match; the screen's own served contract and its already -submitted PSDI draft are unaffected. Updates the deposit (PSDI 4050a-aas85, in place per deposits/README.md), the model card, and the notebook, whose ladder-reconstruction was rewritten from the old 50-per-axis enumeration to the real floor-based one and re-executed -- surfacing that the known small-cell over-prediction (issue #68) is worse under the new decision level, not merely carried forward.
_decision() only set rounding: half_up when decision_bands was also declared, coupling two independent things: whether the target is integer-valued and whether its published value gets a band lift on top. A protocol that publishes a plain quantile level with no bands -- the shape issue #71 needs -- would have published a raw float where a whole rung was required. Round whenever the decision metric itself belongs to the integer-target family, since choosing a level by mean_excess or rounded_accuracy already commits to that target shape regardless of banding.
…sets Holding under-prediction at 6% forced q0.95 on this record with no band lift left to give, at a mean cost of almost 3 rungs of extra mesh per recommendation -- most of the distance between the model and the truth, not a small correction on top of a good estimate. Raises the floor to admit q0.6 and drops the band lift entirely. Measured on the held-out test split: mean excess falls from +2.96 to +0.06 rungs and MAE from 3.12 to 1.14, at the cost of under-prediction rising from 3.5% to 22.9% overall and to 76.7% (from 17.8%) for structures that genuinely need rung 12 or above. The estimator itself is unchanged -- same pickle, same SHA-256 -- only which quantile gets published and whether a band lift sits on top of it. Small-cell over-prediction (issue #68) shrinks under the new level but does not close, and the qualitative direction of the primitive- versus-conventional-silicon comparison in the notebook flips: the model now (coincidentally, not because the underlying per-cell parameterisation changed) orders the two cells correctly, though both predictions remain far past common practice. Updates the deposit (PSDI 4050a-aas85, in place), the model card, and the notebook, re-executed against the new decision.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.