perf(evm): u256 arithmetic optimizations (shift/addmod/barrier/value-range/div-mod) - #458
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR improves EVM u256 arithmetic codegen in the MIR compiler by applying several strength reductions and inlining a large-modulus ADDMOD fast path to reduce runtime calls and expensive arithmetic sequences.
Changes:
- Strength-reduce dynamic u256 shift lowering by replacing
/ 64and% 64with>> 6and& 63in SHL/SHR/SAR paths. - Inline an
ADDMODfast path at MIR level for large moduli (based on the high-limb guard), with a runtime-call slow path for other cases (includingmod == 0). - Remove unnecessary
protectUnsafeValuebarriers in select u64-const ADD/SUB fast paths where carry/borrow flags are proven dead.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
⚡ Performance Regression Check Results✅ Performance Check Passed (interpreter)Performance Benchmark Results (threshold: 25%)
Summary: 194 benchmarks, 0 regressions ✅ Performance Check Passed (multipass)Performance Benchmark Results (threshold: 25%)
Summary: 194 benchmarks, 0 regressions |
abmcar
changed the title
abmcar
force-pushed
the
perf/u256-batch1
branch
from
April 13, 2026 10:10
36acc06 to
062fa17
Compare
abmcar
changed the title
abmcar
added a commit
to abmcar/DTVM
that referenced
this pull request
Apr 15, 2026
…ariant Address Copilot review feedback on PR DTVMStack#458: 1. Extract the cascaded u256 unsigned-LT comparison (previously inlined four times in handleAddMod for AugLtMod, AddLtMod, Overflow, and SumLtMod) into a single u256UnsignedLT(A, B) lambda returning i64 {0,1}. No behavior change; reduces risk of future divergent edits across the four call sites. 2. Strengthen the fast-path eligibility comment to document the invariant the guard establishes: mod[3] != 0 && x[3] <= mod[3] implies x < 2*mod (and symmetrically for y), which is exactly what the single-subtraction normalization step requires. Includes the proof sketch so future refactors don't weaken the check. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace OP_udiv(ShiftAmount, 64) with OP_ushr(ShiftAmount, 6) and OP_urem(ShiftAmount, 64) with OP_and(ShiftAmount, 63) in SHL/SHR/SAR dynamic shift paths. Since the divisor is always 64 (2^6), these strength reductions avoid expensive x86 DIV instructions (~30 cycles). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Inline the intx::addmod fast-path algorithm at the MIR level for non-constant ADDMOD operations. When mod[3] != 0 (modulus >= 2^192) and both operands' high limbs are <= mod's high limb, the result is computed entirely in JIT-generated code using conditional normalize, add, and subtract-and-select — avoiding the runtime function call to evmGetAddMod/intx::addmod. The fast path performs: 1. Normalize augend: if augend >= mod, use augend - mod 2. Normalize addend: if addend >= mod, use addend - mod 3. Add normalized values, detect overflow via sum < augend 4. Subtract mod from sum, detect borrow via sum < mod 5. Select result: if (overflow || !borrow) use difference, else sum Falls back to the existing runtime call for small moduli (mod[3] == 0) and the mod == 0 edge case. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…st paths Remove protectUnsafeValue barriers in handleAddU64Const and handleSubU64Const where the x86 carry flag is provably dead: - handleAddU64Const: Remove barriers on RHS constants (MOV imm never clobbers flags) and on the last ADC result (no subsequent ADC consumes the carry). Intermediate ADC barriers are preserved to force immediate execution while CF is live between ADD→ADC→ADC. - handleSubU64Const: Remove all barriers. This function uses explicit borrow computation (SUB + ICMP_ULT + zeroExtend) rather than SBB instructions, so the carry flag is never live. Generic u256 ADD/SUB chains (evm_mir_compiler.h) are intentionally left unchanged: their last ADC/SBB barriers ensure the instruction executes while CF from the previous ADC/SBB is still valid, and removing them causes deferred lowering to execute after CF is clobbered. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add lightweight ValueRange tracking (U64/U128/U256) to Operand class to detect when u256 operands provably fit in narrower types, then emit single-instruction fast paths instead of expensive multi-limb arithmetic. Range sources: - Constants: auto-derived from limb values - AND masks: result narrows to the smaller operand range - BYTE: always U64 (single byte value 0..255) - Comparisons: always U64 (result is 0 or 1) Range-based fast paths: - ADD(U64,U64): single add + carry → U128 result - MUL(U64,U64): single 64×64→128 multiply → U128 result - DIV(U64,U64): single udiv with div-by-zero guard → U64 result - MOD(U64,U64): single urem with div-by-zero guard → U64 result Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the runtime function call fallback in handleDiv and handleMod with inline runtime divisor-size branching. At runtime, check if the divisor fits in a single 64-bit limb (upper 3 limbs all zero). If so, use the existing cascading 128/64 division pattern inline, avoiding the expensive indirect call, caller-save register spills, and stack frame overhead. Multi-limb divisors still fall back to the runtime call. Also handles the divisor==0 case inline (EVM spec: returns 0) to avoid entering the runtime for a common edge case. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The 5-block CFG pattern in handleDivModGeneral caused a live range calculation assertion failure (TheVNI != nullptr) during register allocation. The root cause was cross-block value references spanning two intermediate blocks (EntryBB -> SingleLimbCheckBB -> ZeroDivisorBB), which the live range calculator could not resolve. Replace the ZeroDivisorBB branch with a select-based zero guard: SafeB0 = select(IsB0Zero, 1, B[0]) prevents hardware DIV-by-zero, then select(IsB0Zero, 0, result) zeros the output when divisor was 0. This reduces the CFG from 5 blocks to 3, matching the proven handleDiv pattern (EntryBB -> SingleLimbBB/MultiLimbBB -> AfterBB). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ariant Address Copilot review feedback on PR DTVMStack#458: 1. Extract the cascaded u256 unsigned-LT comparison (previously inlined four times in handleAddMod for AugLtMod, AddLtMod, Overflow, and SumLtMod) into a single u256UnsignedLT(A, B) lambda returning i64 {0,1}. No behavior change; reduces risk of future divergent edits across the four call sites. 2. Strengthen the fast-path eligibility comment to document the invariant the guard establishes: mod[3] != 0 && x[3] <= mod[3] implies x < 2*mod (and symmetrically for y), which is exactly what the single-subtraction normalization step requires. Includes the proof sketch so future refactors don't weaken the check. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add three .easm/.expected pairs to exercise the inline ADDMOD fast path introduced in earlier u256 batch commits. The existing addmod fixtures cover only small modulus / 0xFF carry / mod==0 cases; none of them set mod[3] != 0, so the fast path's eligibility branch and normalize-subtract chain were untested. New cases: - addmod_fastpath_boundary: mod[3]=x[3]=y[3]=1 with x>mod and y>mod, forces the conditional subtract on both operands before sum. - addmod_fastpath_x_eq_mod: x == mod, exercising the AugLtMod=false / NormAugend=0 path while y < mod stays in place. - addmod_fastpath_overflow_into_high_limb: x[3]=y[3]=mod[3]=2^63 with low limbs maxed, sum carries out of limb 3 -> Overflow detection forces selecting SumSubMod regardless of borrow flag. Verified with run_evm_tests.py in both multipass JIT and interpreter modes (6/6 pass each), and via evmone-unittests multipass (223/223).
abmcar
force-pushed
the
perf/u256-batch1
branch
from
April 25, 2026 05:58
ac7cc13 to
66235af
Compare
abmcar
marked this pull request as ready for review
April 25, 2026 16:07
Light-tier change doc covering the five MIR-level u256 lowering optimizations on this branch (shift strength reduction, ADDMOD inline fast path, dead-carry barrier elimination, value-range narrowing, inline DIV/MOD) plus the 3-block CFG workaround for the register coalescer assertion. Records the suite-level geomean +0.69% [-1.80%, +3.44%] and targeted +6%-18% wins on u256-arithmetic-heavy benches, and the dropped Barrett MULMOD variant.
7 tasks
abmcar
added a commit
to abmcar/DTVM
that referenced
this pull request
May 12, 2026
After rebasing onto current upstream/main (which now includes DTVMStack#458 / DTVMStack#460 / DTVMStack#482 / DTVMStack#483 perf work) and running a 10-rep evmone-bench on the 27 paper benches, the cumulative PR delta has collapsed to noise (raw geomean +1.15%, +0.46% after correcting a single-iteration outlier on main/blake2b_shifts/8415nulls via a focused 20-rep re-measurement). 0 benches above the +/-25% CI gate. The A-vs-PR-base -2.73% from this commit's own optimization is unchanged; the framing shift is that the absolute runtime delta of the whole PR vs unmodified main has been absorbed by the intervening upstream perf optimizations. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Five u256 arithmetic optimizations plus one follow-up register-allocator fix, all contained in
src/compiler/evm_frontend/evm_mir_compiler.{cpp,h}.Shift strength reduction (
1eb0df3, +14/-6) — InSHL/SHR/SARdynamic-shift lowering, replaceOP_udiv/OP_uremby 64 withOP_ushr 6/OP_and 63. Avoids ~30-cycle x86DIV.ADDMOD inline fast path (
ab69050, +227/-4) — For non-constantADDMODwithmod[3] != 0(modulus ≥ 2^192), inlineintx::addmodat MIR level: normalize augend/addend against mod, add with overflow detection, subtract mod with borrow detection, select. Small moduli (mod[3] == 0) andmod == 0still call the runtime.Dead-carry barrier elimination (
df7d5c5, +16/-9) — RemoveprotectUnsafeValuebarriers where x86CFis provably dead:handleAddU64Const: drop barriers on RHS immediates (MOV imm never clobbers flags) and on the terminalADC(no consumer). IntermediateADCbarriers are preserved to keep lowering ordered whileCFis live.handleSubU64Const: drop all barriers. This function uses explicitSUB + ICMP_ULT + zeroExtendborrow, neverSBB, soCFis never live.Value-range narrowing (
718dab0, +155/-7) — AddValueRange{U64,U128,U256}onOperand. Range is auto-derived from constants and propagated through AND masks (narrow to smaller operand),BYTE(always U64), and comparisons (always U64). When both operands ofADD/MUL/DIV/MODare provably U64, emit a single-instruction fast path instead of the 4-limb chain.Inline u256 DIV/MOD (
0c5afd8+f4fc7ce, net +157/-42) — Replace the runtime fallback inhandleDiv/handleModwith a runtime divisor-size check. Single-limb divisors (upper three limbs zero) use the existing cascading 128/64 division pattern inline; multi-limb divisors still call the runtime.The initial 5-block CFG (with a dedicated
ZeroDivisorBB) tripped a live-range assertion (TheVNI != nullptr) during register allocation.f4fc7cereworks it into a 3-block CFG matchinghandleDiv: a branchless guardSafeB0 = select(B[0] == 0, 1, B[0])feeds the hardwareDIV, thenselect(B[0] == 0, 0, result)zeros the output. This path is distinct from PR fix(compiler): fix live range calc assertion when no reaching def found #456'sCgLiveRangeCalc::findReachingDefsfix.Dropped
A Barrett-reduction MULMOD variant was prototyped but abandoned — both the inline and C-helper forms crashed the register coalescer on weierstrudel (1408
MULMODcall sites in one function) and showed no measurable win.Benchmark
evmone-bench, multipass mode, current upstream/main baseline (HEAD
ec3c9f9), 27external/total/{main,micro}benches, 10 reps each, CPU-pinned cores 2-3, single session.Suite-level geomean speedup: 1.0069 (+0.69%), 95% bootstrap CI [-1.80%, +3.44%] — within per-bench noise; not statistically distinguishable from parity at suite level.
Targeted wins on u256-arithmetic-heavy benches:
Suite-level geomean is in the noise band because integer-arithmetic wins on JIT-bound micro benches have been largely absorbed by recent upstream work (#428 BMI2 lowering, #435 peephole, #395 SSA). No regressions outside per-bench noise (CV > 7% on bottom 3 benches).
Test plan
evmone-unittestsmultipass: 223/223evmone-unittestsinterpreter: 215/215evmone-statetestfork_Cancun: 2723/2723tests/evm_asm/addmod_fastpath_*: 3 new boundary fixtures (multipass + interpreter both 6/6)66235af)🤖 Generated with Claude Code