feat: Aperture solver - #443
Open
carloszanella wants to merge 34 commits into
Open
Conversation
`cargo fmt` output for the benchmark commit carried over from the water-fill branch, split out so it does not obscure a real diff. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`SwapCache` hashed a `PoolDirection` — a component id and two token addresses — on every lookup. A caller that resolves its pools once up front can key on a plain index and hash one word instead, so the key becomes a type parameter and `swap_keyed` takes it. `swap` stays as the `PoolDirection` case, so water-fill and most-liquid are unchanged at their call sites. Taken from the decomposition branch, where it was written; nothing here is decomposition-specific and the growth algorithm needs it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Choosing the best-paying pool at each leg of a token path, net of that pool's gas, is most-liquid's whole algorithm. It is also the first thing any other algorithm needs, so it moves to `path_scoring` rather than being copied: the per-leg choice on output net of gas, a pool withheld from a path that already crossed it, and the winner of a pair remembered so paths sharing a leg do not each rescan it. What stays out is the swap itself. The caller passes a closure saying what one pool pays, so how that reaches the pool is its own business. Taken from the decomposition branch, where it was written. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`execute_split_plan` pooled everything arriving in a token and divided it among the swaps consuming it in proportion to the paths' **top-level flow fractions** — their shares of the order. Two paths reaching that token through different first hops did not arrive in that proportion, so the execution could hand a pool far more than the allocation intended, and a pool with a sell limit then refused and failed the whole route. Each path now carries its own amount through the walk. A swap's input is the sum of the carries actually feeding it, and its output is shared back to those paths in proportion to what each put in — which is what one on-chain swap does. The fractions the encoder needs are then derived from those amounts and round-tripped through `fractions_to_amounts`, so the executed amounts are the ones `replay_route` and the router will produce. `test_build_split_route_cross_depth_convergence_with_downstream_split` pinned the old behaviour and is corrected: paths 1 and 3 bring 1800 DAI to component_b, not the 1680 that the 0.7/0.3 split of their WETH shares implied. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Starts an order as one route and adds complexity only where it is large enough, relative to the liquidity in front of it, to pay for the search. A solve seeds with the best single route, probes how much rate that route gives up to its own size, admits further candidates at the slice they would actually be handed, divides the order by marginal value, and returns a split only if it beats the seed on a re-executed route. Two things it does that the incumbents do not. Candidates are scored at the amount they would receive rather than at the whole order, so a pool that cannot swallow the order — or quotes a collapsed price for it — stays on the table for a slice of it. And the allocator simulates each chunk against the state committed so far, so paths sharing a pool are priced correctly during the search instead of being repaired afterwards. The floor is structural: the seed pass calls most-liquid's own functions under the same hop bounds, and every later stage is accept-only-if-better against a fully re-executed route. Over 10,000 offline orders `GR_d3` is never worse than `most_liquid_d3` and beats it on 2,843 orders, where `water_fill_d3` manages 2,321 and loses one. Against `water_fill_d3` it wins 63.6% of the orders where the two differ, at parity on latency. `PLAN.md` and `BASELINE.md` next to the module carry the measurements and the open problems. No worker pool is enabled by default. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`report.md` says how often a config won. These say where and why: `bench-analyze.py` breaks a run down by order size and route shape, `bench-setdiff.py` separates losses where the winner used a pool we never considered from losses where we had every pool and allocated worse. That distinction decided most of the growth algorithm's design. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The trade dataset carries rows where the recorded atomic amount and the recorded USD value cannot both be right — a USDT amount with eighteen decimals, for instance. Solved as-is they produce routes that appear to destroy 99% of the order's value while the routing is fine, which makes any measurement of value loss unreadable. An order is dropped when its USD per atomic unit sits more than `MAX_UNIT_VALUE_FACTOR` away from the median for its token, and only once that token has `MIN_ORDERS_TO_JUDGE` orders behind the median. Both bounds are deliberately loose: a token really can move a long way inside the dataset's window, and the corruption being caught is off by orders of magnitude. The report counts what went. Carried from the decomposition branch, where it was written. The exclusive-liquidity bench config goes with it, having been deleted there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three changes, measured against a cross-tab of the impact probe against whether a split actually beat the seed, over 2,758 solved orders. The trim loop walks nested subsets, so gross output is non-decreasing in fan-out and concave while gas is linear: net is single-peaked, and stopping at the first non-improvement cannot skip a better narrower fan-out. At low gas the first trim almost always fails and the loop then ran four more. It now stops, and the widest fan-out reuses the allocation already made instead of recomputing an identical one. The allocator opened its heap by simulating every alternative before committing a chunk, on every allocation. Lazy greedy needs only an upper bound and the candidate walk already had one — the gross output it saw, scaled when the walk amount sat below a chunk, which is sound because output is concave through the origin. Alternatives that never reach the top of the heap are now never simulated. Both of those are exact; the benchmark returns bit-identical figures. The third is a cheap tier: below 0.3 bps of measured impact the solve walks no slice of the order at all. Below that threshold a split beats the seed on 4% of orders and wins a median 0.02 bps when it does. The cut is the largest that costs no wins over most-liquid in the 1k-5k bin, which is the thing it may not spend; at 1 bps it would cost 65 of 178. Solve time falls in every bin — 1k-5k 7,996us to 6,908us, 500k+ 15,189us to 11,854us — with every bin's mean against water-fill unchanged at both 0.1 and 2 gwei, and the same 855 wins over most-liquid with none lost. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Stopping the trim loop at the first fan-out that failed to improve looked safe: narrowing trades gross output for gas, which reads like a curve with one peak. It is not. `allocate_refs` reallocates the whole order over the survivors, so two fan-outs one step apart are different allocations rather than points on a curve, and a losing step can be followed by a winning one. Measured against the pre-M12 baseline, per order: with the early stop, 40 orders moved and 37 were worse, the worst by 0.99 bps. Without it, 13 moved and 10 were worse, the worst by 0.061. It bought about 2% of solve time and was slower in the two largest bins. What stays is the part that was worth having: the heap opening on bounds the walk already computed, the widest fan-out reusing the allocation already made, and the cheap tier. Solve time still falls 19-26% in every bin, with every bin's mean against water-fill unchanged at 0.1 and 2 gwei, and the same 855 wins over most-liquid with none lost. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The allocator and `execute_split_plan` disagree about a pool's capacity on 6 of 2,758 orders — 6.2% of the largest bin, 0% of the three smallest. `797` and `665` are near-isolated rather than the visible part of a class, so widening the search to catch them is not warranted. A spot-price upper bound cannot prune the candidate walk. It would have been exact, unlike sampling, but instrumenting it over 205,494 walked paths shows it skips 4%. Spot ignores slippage, so the bound sits far above the realised net it is compared against, and the gap widens with order size — the largest bin prunes least, at 2%. Neither is built. The instrumentation that measured them is removed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The chunk loop pops the best offer; a stale one is re-simulated to re-measure it and pushed back. When that offer next came up fresh and won the chunk, the loop simulated the same path, at the same chunk, against the same overlay a second time — only to recover the post-trade states the first call had already produced and discarded. Those states are now kept, in a map cleared on every commit so it holds only the offers measured for the chunk in hand. Output is byte-identical on all 2,541 solved orders; the allocator makes 22% fewer path simulations (1,459,266 to 1,137,580 over 2,000 orders). Measured by counting simulations, not by timing: wall clock on this benchmark carries about 10% run-to-run noise, wide enough to hide this entirely. The trace now carries `path_sims` so it stays measurable. Profiling motivated it — 62% of a solve is in `grow`, 83% of that in `allocate_refs`, and 73% of that is BigUint subtraction inside Uniswap v3 pool math. The allocator is simulation-bound, so the only lever is fewer simulations, and that lever grows with the pool count of a real market. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Halving `CHUNKS` removes 39% of the allocator's path simulations and six sevenths of the 500k+ bin's edge against water-fill, +1859.9 bps to +303.4. It also wins more orders than the fine grid, 891 against 855, because the orders a rough split is good enough for are the small ones — a win count alone would have read the change as an improvement. Left at 64. Tiering it on measured impact, as the candidate walks already are, is the form worth trying; the saving is bounded by the ~28% of the field in the cheap tier. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`grow` deduplicated candidates on the sequence of components a route goes
through, discarding which pair each hop trades. A pool holding more than
two tokens serves several pairs, so with component X over {A,B,D} and Y
over {B,D,C}, the token paths A->B->C and A->D->C both key as [X, Y] and
the second is dropped — a different route through a different
intermediate token, with different liquidity and a different price,
silently discarded.
Keyed on the (component, token_in, token_out) triple instead, which is
the same identity `split_primitives::hop_key` already uses to decide
which hops merge into one on-chain swap.
Latent on the recorded fixture, which carries no multi-token pools: all
5,033 route outputs over 3,000 orders are byte-identical. It bites on a
real market, where Curve- and Balancer-style pools are common.
Found by review.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three said something the code does not do. `execute_split_plan` still described "each token is fully pooled from all its inflows before being re-split downstream" — the pooling this branch replaced, and the very thing the fix below it points at as wrong. `price_impact` and `OrderTrace::probed` both said a probe that cannot be taken leaves the solve on the cheap tier. It leaves it on the middle one, as the inline comment beside the match already said. `OrderTrace::dead_by_hops` is indexed by the number of tokens a path names, one more than its hop count, so every bucket read one off. Also renamed `SEED_STAGE` to `CANDIDATE_WALK_STAGE`. It labels the swaps of the seed pass, the slice passes and the impact probe alike, so a simulation report was attributing all of them to the seed — and the floor argument rests on the seed being the distinguishable one. Two dead paths go with them: `candidate_scored` took a hop count and a net it never read, and the allocator's fallback to re-simulating a winning offer could not fire — a winner is fresh only when the re-measure branch recorded it during this same chunk, and that map is cleared on every commit. Route outputs over 3,000 orders are byte-identical. Found by review. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`expand_candidate` read `candidate.pools` through `get` and then indexed the same slice directly on the next line, so a short `pools` would have panicked in a crate that denies `panic`. Taken through `get_mut` instead. Several constants carried the benchmark that chose them — order ids, bps figures, a six-line cross-tabulation. House style keeps that out of code: each doc now states the mechanism and points at `PLAN.md`, which has the figures and is where they stay current. Route outputs over 3,000 orders are byte-identical. Found by review. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The floor — "never worse than most_liquid" — was only ever asserted against `lower_bound`, a constant the test harness works out by hand. No test ran both algorithms on one market, which is what the claim actually says. `test_growth_never_returns_less_than_most_liquid` does, over every shared scenario, and it covers the gap PLAN.md flags: most-liquid compares candidates on one figure and returns a route re-simulated by `build_route`, and it is the second that has to be beaten. `test_growth_does_not_invert_with_depth` compared two identical searches: every shared scenario is within two hops, so `growth(2)` and `growth(3)` walk the same set and only determinism was being asserted. Now 1 against 4 hops, where most of them lose their route entirely at one. `TokenFlow::admit` had no tests and a bug they found: it checked every hop against the edges admitted *before* the call and inserted them all afterwards, so an alternative closing a cycle with its own hops slipped through — `C->A` and `B->C` are each harmless against `A->B` alone. It now checks and records one hop at a time and rolls back the edges it introduced when a later hop is refused, so a half-admitted path cannot block a later alternative that never conflicted with anything. Route outputs over 3,000 orders are byte-identical; every real alternative runs token_in to token_out, so the cycle needs a shape the fixture does not produce. Found by review. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`beam.rs` had no tests, and the behaviour its module doc is built on was unasserted: keeping a leg-one pool that pays *less* because it feeds a better leg two. `walk_beam` takes a closure for what a pool pays, so none of this needs a market. The first test is the module's reason to exist — at width one the walk takes the fast leg and is stuck; at width two it keeps the slower one and wins by fifty times. The rest cover a partial refusing a pool it already crossed, the width bound with best first, and the empty cases. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The beam walk populates no pool rankings, so the deep tier gets no one-leg variant expansion — the highest-impact orders are exactly where the mechanism is off. Fixing it gains +90,226 bps on `797`, the $30.8M order nothing else has moved, and regresses 81 others through a second bug the review filed separately: `TokenFlow` refuses alternatives in arrival order, so a better one can be turned away for conflicting with a variant that ends up carrying nothing. Admission has to stop depending on arrival order first. Recorded rather than applied. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Names that needed their doc line to say what they were: carried -> path_amount_in position -> next_hop_ix feeding -> path_ixs_by_hop normalized -> path_flow_fractions contributions -> fed_amounts assigned -> hop_amounts refused -> refused_components measured -> chunk_results offers -> offers_by_pay active -> holders_by_size narrowed -> kept pools (of indices) -> pool_ixs Partial -> PartialPath, with amount/crossed/net/gas spelled out TokenFlow -> AcyclicTokenEdges pools_on -> components_on The module headers were doing too much: `beam.rs` spent eleven of thirty lines replaying one order's arithmetic, and constants argued from order ids and bps figures measured on one fixture. Those go stale and bury the line that was wanted. Each doc now states the mechanism; `PLAN.md` keeps the numbers. Headers: mod 42 to 30 lines, allocate 35 to 23, beam 30 to 13. Route outputs over 3,000 orders are byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`AcyclicTokenEdges` read worse than the name it replaced. Four doc comments still argued from order ids, bps figures and sample sizes taken on one fixture. They state the mechanism now; `PLAN.md` keeps the numbers. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Explains the algorithm from the top down: what it is for in a paragraph, the five steps of a solve, then each mechanism on its own — the floor against most-liquid, measuring difficulty rather than guessing it, why candidates are scored at a slice of the order, one-leg variants, the beam, the allocator, how gas decides the fan-out, and the two constraints assembly imposes on admission. It carries the reasoning behind the decisions that are not obvious from the code: why difficulty is measured self-referentially rather than from order size or spot prices, why the lazy heap gives the same answer as evaluating everything, and why gas is priced on an assembled route rather than estimated during the search. The module header shrinks to the floor argument and a pointer, since it was restating the same walkthrough. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Those three sections asserted what the mechanisms do without showing why they are needed. Each now states the problem first, with a worked example, before the fix. One-leg variants: a token path names tokens, not pools, and scoring picks one pool per leg — so a token path yields exactly one route, and two pools on the same pair can never both be split across. That is not the search doing badly, it is something it cannot express, which the section now says outright and draws. The beam: worked through with numbers, showing the pool that pays most on leg one handing leg two an amount only a bad pool will take. The allocator: what it is deciding, the chunk-by-chunk method traced out, why concavity makes greedy optimal rather than merely reasonable, and what the shared overlay and the lazy heap each buy. Also drops PLAN.md and BASELINE.md, which were a working record rather than something worth carrying in version control. The dozen code comments that pointed at them now state the reasoning themselves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
"there may be five pools trading USDC/WETH" was a number I made up, and it reads as a statement about the market. The combinatorics that followed were derived from it, so both go: the point is that `k^h` combinations grow out of reach while `h·k` neighbours do not, which needs no count. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The allocator section used the word "offer" without saying what one is, and asserted that upper bounds save simulations without showing how. It now says plainly that the chunk loop ignores gas, states the fact the whole trick rests on — a route's pay for a chunk never rises — and walks a pass through the heap showing two simulations where there would have been three, and where a hundred candidates would still be two. An offer is defined: the candidate, its last measured pay, and the chunk count that measurement was taken at. It also says where the first figures come from, since nothing is simulated to fill the heap either. The gas section defines fan-out in its heading and its first line rather than assuming it, explains why a fixed per-path cost is exactly what a greedy loop cannot weigh, and shows every fan-out being priced rather than only the narrower ones. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An offer was treated as stale after any commit, and `chunk_results` was cleared with it, so every chunk paid at least one re-simulation and the simulations of paths the commit never touched were thrown away. But a commit only moves the pools on the path that won it: an offer sharing none of them was measured against the state it still faces. `moved_at` records the commit that last moved each pool, and `still_true` reads an offer against its own hops. Results now outlive the chunk they were measured for, since they stop being true on the same condition. Exact — the allocation is unchanged chunk for chunk. Over 3000 offline orders no route moved and allocator simulations fell about 4%: the alternatives overlap enough that most offers do share a pool with the winner.
Solve time was flat in order size: a $2k order cost as much as a $2M one, and orders under $25k held 73% of total time on a live market. What they were buying was measured rather than assumed. Below an impact of 1e-5 a split beats the seed on a twentieth of orders and by 0.00 bps; the two lowest impact buckets spent a sixth of all solve time to win hundredths of a basis point. So an order below `SMALL_ORDER_GAS_TOKEN_UNITS` that probes below `SMALL_ORDER_SPLIT_IMPACT` now returns its seed: no slice walk, no allocation. Size comes from `TokenGasPrices`, which prices every token against the gas token, so it is known before anything is simulated and no oracle is consulted. Such an order is also walked `most_liquid`'s cheap way first — the pair's remembered winner rather than every pool — because on the orders the gate catches, the full scan was measured to produce a route worth no more than `most_liquid`'s own on every single one of them. Where the probe says otherwise the walk is redone in full; both share one cache, so the cheap walk is not paid for twice. `LegScan` names which of the two a pass runs. The probe no longer re-simulates the seed path at the full amount, which the caller had just done; `seed_route` gathers that step for both walks. Floor-safe by construction: everything after the seed is accept-only-if-better, so giving it up trades surplus and not the floor. Over 3000 offline orders, per order: no order loses to `most_liquid`, no order above $25k moves at all, and 205 below it give up a mean 0.7 bps. Live, p50 in the 1k-5k bin goes from 611ms to 69ms against `most_liquid`'s 42ms, and quality holds at 0 orders worse. Written against `SolveContext` and `walk_slices` rather than cherry-picked from cz/feat/aperture-speed, where it predates both.
Adds "Giving up on a small order" after the floor section, since the floor is what makes it safe, and records both measurements it rests on: what a split is worth below the impact threshold, and that the full leg scan bought nothing on the orders the gate catches. Also corrects the note on order size in USD. It remains the wrong variable for deciding how hard to search, and it is the right one for deciding whether an order may give up searching at all.
Three counters, all in the trace and all off the hot path. `alternatives_simulated` and `alternatives_holding` measure how much of the allocator's work earns anything. Over 3000 offline orders it simulates about 132 alternatives per order and 17 of them ever hold a chunk, so 87% of its simulations go to ruling an alternative out. That is not laziness failing — at a chunk smaller than the walked slice the only valid upper bound on a concave curve is the walked output itself, which is roughly three times the truth, so nearly every alternative reaches the top of the heap once. `alternatives_unbounded` is a canary. An unbounded offer sorts *below* every bounded one, the opposite of what the code claimed, so it would never be popped. Nothing produces one today because the only source is a one-leg variant of a beam-walked candidate and a beamed pass records no rankings to build variants from — two defects that hide each other. The counter reads zero over 3000 orders; the doc now records both facts so whoever fixes the rankings fixes the order with it.
`pools_simulated` only ever counted the candidate walk's cache misses. The seed's re-simulation, the impact probe and every fan-out's re-execution go straight to the market, so a third of the solve could have been invisible. Measured, it is 1.3%: per solved order the walk is 741 pool swaps, the allocator's search 989, assembly 19 and seed plus probe 4. The cost model was right; now it is checked rather than assumed. `winning_trim` records which fan-out the winning split came from. Over 648 orders where a split won, the widest gives 34%, and the four narrower ones 22%, 10%, 12% and 23%. So the ladder earns its re-allocations, and its bottom rung winning a fifth of the time suggests it stops too early rather than too late.
The summary table cannot show that a change was behaviour-neutral. A run that moved 31 orders once reported an identical mean, better count and tie count, so "nothing changed" has to be read per order. `bench-compare.py` pins one run's per-order outputs and diffs a later run against them: orders that fell below most_liquid, which is the floor and must be zero; orders that moved, grouped by order size, since a change may be allowed to give up surplus on small orders and not on large ones; and, given a trace, the simulation counts behind them. This is the guardrail every change on this branch was checked with.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.