gas bench: over/under split + lag=2 secondary grade (methodology v2 seed) - #1500
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Delivers the two red-team recommendations that don't require touching the primary ranking.
Over/under split (Phase C)
Emits
gas_error_priority_over_gwei_histogramandgas_error_priority_under_gwei_histogramalongside the existing abs histogram. Same buckets. Only the branch the oracle actually erred on gets a sample.Why: percentile trackers (PublicNode feeHistory) and inclusion-confidence oracles (Etherscan, MetaMask 'high') can post the same abs p99 gap but with opposite user consequences — over-prediction = overpay a few wei, under-prediction = tx stuck through the next spike. The leaderboard can now decompose these instead of collapsing them.
Lag=2 secondary grade (Phase D seed)
New dual buffer in the harness so every prediction is graded twice — once at the target block (primary, unchanged) and once at target+2 (new histogram
gas_error_priority_lag2_gwei_histogram).Why: grading against the very next block rewards whichever oracle scraped the mempool 100 ms before we did — a latency race, not accuracy. Block+2 matches actual wallet UX (sign, broadcast, propagate).
Verified locally
60s smoke run:
Primary p99 ranking is unchanged. Leaderboard columns can be added later once the histograms accumulate 24h of data.