feat(k-index): publish a level chosen per band, not one shared quantile - #80
Merged
Merged
Conversation
A single shared quantile means the easy majority and the hard tail share one number: q0.6 under-predicted 76.7% of the time for structures that genuinely need rung 12 or above, while the bulk of the dataset (rung < 7) already sat at 12.1% with room to spare. Raising the one shared level to fix the tail would tax the majority with mesh it does not need. quantile_by_band cuts structures into bands by the model's own median and picks a separate published level per band -- the cheapest one keeping that band's own under-prediction at or below 5% on validation, not the dataset average. Bands land on q0.90 below rung 7 and q0.975 at or above it. The estimator is unchanged -- same pickle, same SHA-256 -- only the decision rule changed. Measured on the held-out test split: under-prediction falls from 22.9% to 4.2% overall and from 76.7% to 14.7% for rung 12 or above, at the cost of MAE rising from 1.135 to 2.974 and R2 going negative (-0.305) -- a real trade, not a free win. Two things get worse in isolation, both bounded downstream rather than fixed here: small, simple cells are pushed to denser meshes than before (primitive silicon now predicts rung 38, up from rung 17), which Core's planned metallicity-gated k-distance ceiling is designed to cap; and for a very small number of simple, high-symmetry cells the published rung can exceed the number of meshes that exist on that structure's own ladder (conventional silicon predicts rung 40 against a 38-rung ladder), which Core is designing a bounds check for. Neither is something this decision rule can fix on its own. Updates the deposit (PSDI 4050a-aas85, in place), the protocol (edited in place, per the same-forest precedent), the model card, and the notebook, re-executed against the new decision.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A single shared quantile means the easy majority and the hard tail share one number: q0.6 under-predicted 76.7% of the time for structures that genuinely need rung 12 or above, while the bulk of the dataset (rung < 7) already sat at 12.1% with room to spare. Raising the one shared level to fix the tail would tax the majority with mesh it does not need.
quantile_by_band cuts structures into bands by the model's own median and picks a separate published level per band -- the cheapest one keeping that band's own under-prediction at or below 5% on validation, not the dataset average. Bands land on q0.90 below rung 7 and q0.975 at or above it. The estimator is unchanged -- same pickle, same SHA-256 -- only the decision rule changed.
Measured on the held-out test split: under-prediction falls from 22.9% to 4.2% overall and from 76.7% to 14.7% for rung 12 or above, at the cost of MAE rising from 1.135 to 2.974 and R2 going negative (-0.305) -- a real trade, not a free win.
Two things get worse in isolation, both bounded downstream rather than fixed here: small, simple cells are pushed to denser meshes than before (primitive silicon now predicts rung 38, up from rung 17), which Core's planned metallicity-gated k-distance ceiling is designed to cap; and for a very small number of simple, high-symmetry cells the published rung can exceed the number of meshes that exist on that structure's own ladder (conventional silicon predicts rung 40 against a 38-rung ladder), which Core is designing a bounds check for. Neither is something this decision rule can fix on its own.
Updates the deposit (PSDI 4050a-aas85, in place), the protocol (edited in place, per the same-forest precedent), the model card, and the notebook, re-executed against the new decision.