Custom thread pool#239

Open

TomWambsgans wants to merge 35 commits into

mainfrom

custom-thread-pool

Commits on May 29, 2026

parallel: in-house fixed thread pool; port eq_mle and sumcheck off rayon

TomWambsgans
and
claude
committed
sumcheck: port clean reductions to parallel::map_reduce

TomWambsgans
and
claude
committed
parallel/sumcheck: per-worker scratch in sumcheck_compute_core

TomWambsgans
and
claude
committed
parallel: lock-free spin-then-park dispatch (replaces std::Barrier)

TomWambsgans
and
claude
committed
parallel: nested dispatch falls back to sequential (no deadlock)

TomWambsgans
and
claude
committed
parallel: port field/poly/sumcheck off rayon (evals, utils, product_computation)

TomWambsgans
and
claude
committed
parallel: port symetric merkle + fiat-shamir grinding off rayon

TomWambsgans
and
claude
committed
parallel: port whir (dft/merkle/open/utils) off rayon

TomWambsgans
and
claude
committed
parallel: port sub_protocols (logup/gkr/air_sumcheck) off rayon; re-export parallel from backend

TomWambsgans
and
claude
committed
parallel: port lean_vm/lean_prover/xmss/rec_aggregation/lean_compiler/utils off rayon

TomWambsgans
and
claude
committed
parallel: teardown rayon — drop deps, flush_rayon hack, backend re-export, rayon test

TomWambsgans
and
claude
committed
parallel: guided self-scheduling + flatten dft butterflies + chunk product sumcheck

TomWambsgans
and
claude
committed
parallel: range-based core dispatch (for_each_chunk)

TomWambsgans
and
claude
committed
parallel: lower spin limit to 2^12 to reduce variance during sequential phases

TomWambsgans
and
claude
committed
sumcheck: product reductions use in-place per-worker accumulation

TomWambsgans
and
claude
committed
gitignore: ignore sibling /target-* build dirs and /.lm-* scratch binaries
TomWambsgans
committed
lean_vm: in-place get_slice_into for Poseidon inputs (avoid ~100K Vec allocs/proof)

TomWambsgans
and
claude
committed
alloc: replace zk-alloc bump arena with tuned mimalloc

TomWambsgans
and
claude
committed
poseidon: stack-allocate per-row column pointer arrays in trace gen

TomWambsgans
and
claude
committed
whir: stack-allocate single-element dimensions array in verify

TomWambsgans
and
claude
committed
sumcheck: reuse per-worker point/rows scratch in fold+split-eq compute paths

TomWambsgans
and
claude
committed
poseidon: bind fixed-size array ref to elide per-row bounds checks in trace gen

TomWambsgans
and
claude
committed
Merge branch 'main' into remove-small-allocs
TomWambsgans
committed
Merge branch 'remove-small-allocs' into custom-thread-pool
TomWambsgans
committed
alloc: disable THP for the prover (fixes ~50% intermittent slowdown on Zen4)

TomWambsgans
and
claude
committed
parallel: centralize per-worker-slot unsafe; tidy dispatch

TomWambsgans
and
claude
committed
parallel: centralize SendPtr + chunk-size heuristic across the workspace

TomWambsgans
and
claude
committed
eq_mle: trim the duplicated 10-line packing-width caveat to 2 lines

TomWambsgans
and
claude
committed
eq_mle: extract shared par_eval_eq tail from the compute_eval_eq variants

TomWambsgans
and
claude
committed
sumcheck: tighten build_evals body to a one-line map_or

TomWambsgans
and
claude
committed
fmt
TomWambsgans
committed
lean_compiler: collapse if into match guard (clippy -Dwarnings)

TomWambsgans
and
claude
committed
Merge branch 'main' into custom-thread-pool
TomWambsgans
committed

Commits on May 30, 2026