fix: keep inventory-limited pools in water-fill splits via get_limits - #405
fix: keep inventory-limited pools in water-fill splits via get_limits#405kayibal wants to merge 3 commits into
Conversation
An inventory-limited pool — e.g. a propAMM whose output balance is below the order — lost its entire allocation to worse-priced pools the moment the order exceeded its inventory, through three compounding defects: 1. A path failing simulation at the full order amount was ranked with gross 0, sorting it below every split candidate window (top-4 disjoint, top-8 full-amount / top-32 marginal fill-and-spill probes). Rank such paths by their output at the get_limits-capped input instead: each hop's sell limit, back-converted to path-input units through the preceding edges' spot prices (pessimistic under AMM concavity, so the capped simulation succeeds). One retry at 99% covers marginally stale limits; a failure the limits don't explain keeps gross 0. 2. Even ranked honestly, a capped path's output sinks below every order-absorbing pool once the order is a small multiple of its inventory, dropping out of the probe window again. The fill-and- spill marginal probe now always includes the best limit-capped paths (CAPPED_PROBE_PATHS) wherever they rank; the first-chunk probe then judges them by price. 3. Chunked allocation over-filled such pools: propAMM balance overrides reject only a single swap larger than the balance, so per-chunk simulations all succeed past the real capacity, and the rebuilt single leg at the summed amount reverts — discarding the whole split candidate. Both chunk loops now stop allocating to a path at its get_limits cap. The extra get_limits/simulation calls run only for paths that fail at the full amount plus the small allocation subsets. get_limits was verified against live states: 0 errors across 1,868 pool directions on 15 protocols, and get_amount_out succeeds at exactly max_in on all but one broken pool. Live result (USDC->WETH, 1-hop water_fill, fermiswap inventory ~\$278k): the pool's absolute allocation now pins at its inventory from \$400k to \$2M order size instead of dropping to zero, with the remainder spilling to on-chain pools. Adds get_limits_guarded to GuardedProtocolSim for panic containment. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The capped ranking pass simulated every full-amount-failing path at its limit cap with uncached per-hop get_limits calls. At three hops most of the ~4.5k candidate paths fail once the order is large, which added 1-2s to setup on large orders. Now sell limits are memoized per solve (LimitsCache), failing paths are deduplicated by bottleneck component and ranked by spot price, and only the top CAPPED_PROBE_PATHS get the honest capped simulation; the rest keep gross 0 as before the capped ranking existed. The allocation passes replace per-path caps with a shared per-component budget (ComponentBudget) fed by exact hop inputs from simulate_step: fill-and-spill paths share components, and their combined allocation must stay a valid single swap or the rebuilt merged leg reverts and the whole candidate is discarded. The gated coarse pass also anchors the top-ranked candidate in the active set, so inventory-limited probe winners cannot claim every max_paths slot, saturate together, and leave the order unfillable. Regression test: 20 two-hop paths bottlenecked by one limited pool must not crowd a distinct best-price limited pool out of the split (test_capped_probe_slots_deduped_by_bottleneck, fails on the previous commit). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Follow-up commit 658bc43 after benchmarking against Problem found: the capped ranking pass simulated every full-amount-failing path at its limit cap with uncached per-hop Fixes in this commit:
After (live, paired same-block probes): setup 0.64–0.71 s at $600k–2M ( New regression test One live check is still pending: fermi retention at $600k–2M with the anchor change was validated in the unit scenario but not yet live — Tycho beta RPC and the node endpoint started refusing new connections mid-validation. Will re-run the paired probe when the endpoints recover. 🤖 Generated with Claude Code |
Live validation showed the capped-probe slots being crowded out on blocks with many failing paths: broken pools advertise fantasy spot prices (multiples of the market rate) that outrank honest inventory, and VM components can lack derived edge data entirely, which excluded their paths from selection. Probes are now deduplicated keeping the largest cap per bottleneck and ranked by input cap — the path's real fillable notional from the same get_limits oracle the allocation budgets trust, comparable across paths because every cap is in path-input token units. A failed capped simulation no longer consumes one of the CAPPED_PROBE_PATHS slots; attempts continue down the ranked list, bounded by CAPPED_PROBE_ATTEMPTS. Adds debug-level logs for capped-probe selection, fill-and-spill candidate selection, and per-candidate nets with legs. Live result (paired same-block probes, USDC->WETH, water_fill 3-hop): fermiswap retained at ~its inventory in 8/9 probes across $600k-$2M orders (previously 0/9 at these sizes), +0.9 to +6.9 bps over main. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Live validation is complete (the earlier network issue was Little Snitch blocking the rebuilt binaries). Two more defects surfaced and are fixed in 3af5651:
Final live result (paired same-block probes vs main, USDC→WETH, 3-hop water_fill): fermiswap retained at ~its inventory in 8/9 probes across $600k–$2M orders — previously 0/9 at these sizes on this branch and structurally impossible on main — winning +0.9 to +6.9 bps. Solve times ~1–1.7 s on fermi-heavy solves vs 0.7–1.3 s baseline, versus 2.2–4.2 s before the perf commit. 🤖 Generated with Claude Code |
Problem
An inventory-limited pool — e.g. a propAMM whose output-side balance is below the order — lost its entire allocation the moment the order exceeded its inventory, instead of contributing up to its capacity in a split. Measured live (USDC→WETH, 1-hop
water_fillpool, fermiswap): 93% route share at a $300k order, 0% from $310k upward, cliff moving with the pool's per-block inventory.Cause
Three compounding defects, found by bisecting live quotes and instrumenting the chunk allocation:
EVMPoolState::get_amount_outre-derives token-balance overwrites on every call, so per-chunk simulations never see cumulative depletion. The allocator over-filled the pool (observed: $707k of an $800k order into a pool with $280k inventory, zero chunk failures), the rebuilt single leg reverted, and the whole split candidate was discarded.Fix
get_limits-capped input (per hop, back-converted to path-input units through the preceding edges' spot prices; one retry at 99% for marginally stale limits; unexplained failures keep gross 0).get_limitswas verified against live states: 0 errors across 1,868 pool directions on 15 protocols;get_amount_outsucceeds at exactlymax_inon all but one broken pool.CAPPED_PROBE_PATHS), where the first-chunk probe judges them by price.get_limitscap in both chunk loops (disjoint_waterfill,fillspill_waterfill).Adds
get_limits_guardedtoGuardedProtocolSim(same panic containment asget_amount_out_guarded).Extra
get_limits/simulation calls run only for paths that fail at the full amount, plus the small allocation subsets. No bisection loops.Result (live, USDC→WETH, fermiswap inventory ~$278k)
The pool's absolute allocation pins at its inventory from $400k to $2M; the remainder spills to on-chain pools.
Tests
test_inventory_limited_component_kept_in_split— 38 healthy competitors push a full-amount-failing path below every candidate window; asserts it still receives a substantial share.test_per_swap_limited_component_capped_at_its_limit— a pool that accepts every chunk but rejects the rebuilt leg (MockProtocolSimhas exactly the propAMM per-swap semantics); asserts allocation respects the sell limit.test_max_fillable_input_single_hop_uses_exact_limit,test_max_fillable_input_two_hop_converts_via_spot— cap computation.Each test was verified to fail with its mechanism reverted. Full workspace suite passes (one pre-existing macOS-local issue: panic-containment tests abort under rustc 1.95/macOS 26 because unwinding aborts; they pass on Linux).
An upstream issue about the non-depleting balance overwrites in
EVMPoolState::get_amount_outwill be filed on propeller-heads/tycho; if fixed there, mechanism 3 here becomes an optimization rather than a correctness requirement.🤖 Generated with Claude Code