Implement round-robin timing strategy to reduce benchmark noise

[imakris]

Summary

Refactored the benchmark timing methodology to use a round-robin strategy instead of back-to-back timing trials. This reduces the impact of VM/container scheduling hiccups and other transient noise on benchmark results by spreading measurements across multiple rounds and taking the minimum observed time.

Key Changes

• Replaced median-based timing with round-robin minimum: Instead of running multiple consecutive trials for each expression and taking the median, the new approach runs one timing trial per expression per round, then takes the minimum across all rounds. This ensures that transient noise (which only adds latency) doesn't inflate results.

• Adaptive round count: Changed from a fixed timing_trials variable to num_rounds that adapts based on iteration count:

  • 3 rounds for ≥10,000 iterations
  • 5 rounds for ≥1,000 iterations
  • 7 rounds for <1,000 iterations

• Restructured benchmark into three phases:

  1. Phase 1: Compilation and precision evaluation (single pass, no timing)
  2. Phase 2: Round-robin timing for mexce expressions
  3. Phase 3: Round-robin timing for native expressions

• Separated timing collection: Timing is now collected in dedicated phases after all compilation and correctness checks, improving code clarity and reducing variable state management.

• Updated progress output: Added round count to the final progress message for transparency.

Implementation Details

• Timing samples are now stored in 2D vectors (round_timings and native_round_timings) indexed by expression, with each inner vector containing one sample per round.
• Only expressions that compiled and evaluated successfully are included in the timing phase.
• The minimum value across rounds is selected using std::min_element, which is theoretically sound since noise only adds latency, never removes it.
• Variable initialization (a, b, c, x, y, z, w) is performed at the start of each timing iteration to ensure consistent state.

https://claude.ai/code/session_01S5ELKkY3xhYFDE3En7yAkQ

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!