Skip to content

gas bench: over/under split + lag=2 secondary grade (methodology v2 seed) - #1500

Merged
Flotapponnier merged 3 commits into
devfrom
feat/gas-bench-methodology-v2
Jul 26, 2026
Merged

gas bench: over/under split + lag=2 secondary grade (methodology v2 seed)#1500
Flotapponnier merged 3 commits into
devfrom
feat/gas-bench-methodology-v2

Conversation

@Flotapponnier

Copy link
Copy Markdown
Collaborator

Delivers the two red-team recommendations that don't require touching the primary ranking.

Over/under split (Phase C)

Emits gas_error_priority_over_gwei_histogram and gas_error_priority_under_gwei_histogram alongside the existing abs histogram. Same buckets. Only the branch the oracle actually erred on gets a sample.

Why: percentile trackers (PublicNode feeHistory) and inclusion-confidence oracles (Etherscan, MetaMask 'high') can post the same abs p99 gap but with opposite user consequences — over-prediction = overpay a few wei, under-prediction = tx stuck through the next spike. The leaderboard can now decompose these instead of collapsing them.

Lag=2 secondary grade (Phase D seed)

New dual buffer in the harness so every prediction is graded twice — once at the target block (primary, unchanged) and once at target+2 (new histogram gas_error_priority_lag2_gwei_histogram).

Why: grading against the very next block rewards whichever oracle scraped the mempool 100 ms before we did — a latency race, not accuracy. Block+2 matches actual wallet UX (sign, broadcast, propagate).

Verified locally

60s smoke run:

  • over/under counters populated per (oracle, tier, chain) with the expected asymmetric distributions (Etherscan under_count > over_count on ETH-p50 during current low-vol period; PublicNode balanced)
  • lag2 histogram counts increasing on all (oracle, tier, chain) pairs
  • Build clean, vet clean, spec validation ✓

Primary p99 ranking is unchanged. Leaderboard columns can be added later once the histograms accumulate 24h of data.

@Flotapponnier
Flotapponnier merged commit 177735c into dev Jul 26, 2026
1 check failed
@Flotapponnier
Flotapponnier deleted the feat/gas-bench-methodology-v2 branch July 26, 2026 06:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant