fix: pay and finalize multi-batch merkle uploads (antd >= 0.12.0) - #162
fix: pay and finalize multi-batch merkle uploads (antd >= 0.12.0)#162Nic-dorman wants to merge 2 commits into
Conversation
Uploads larger than one merkle tree (256 fresh chunks ~= 1 GiB) arrive
from antd >= 0.12.0 as multiple MerkleBatches with the legacy mirror
fields empty, so the worker submitted an empty commitments list and the
upload failed opaquely. The merkle branch now:
- plans batches from MerkleBatches (legacy single-batch fields remain as
the pre-0.12 fallback; both-empty errors with a clear version-gap
message) and validates every batch before any money moves
- pre-approves the summed per-batch max payout in one approve tx
(Signer.EnsureMerkleAllowance), then pays one payForMerkleTree tx per
batch, recording each confirmed spend before the next batch (V2-426;
one transactions row per batch, tx_hash = that batch's winner hash)
- finalizes via FinalizeMerkleUploadMulti with the index-aligned winner
list (single-batch keeps the legacy FinalizeMerkleUpload call)
- on a definitive mid-plan payment failure, salvage-finalizes the paid
batches ("" for unpaid) so their chunks store and a retry pays only
the remainder, then preserves the source (errPaidNoRetry); a
confirmation timeout skips salvage so the manual full finalize stays
possible
- sums estimatedUploadCost across all batches for the max_gas_fee check
V2-1056
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Found live on the multi-batch devnet e2e: batch 1's payForMerkleTree reverted with ERC20InsufficientAllowance because the allowance bound still used the V1-era "sum over pools of max candidate" formula. PaymentVaultV2 charges median16(winner pool quotes) * 2^depth and picks the winner pool at execution time (PaymentVaultV2.sol:140), so the correct worst case is the highest pool median shifted by depth, summed across batches. The single-batch path had the same latent under-approval bug; both bounds now share the contract-accurate formula, and the worker validates batch depth (1-8) before paying. V2-1056 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dirvine
left a comment
There was a problem hiding this comment.
Reviewed at exact head 85228779cc6701945fa57388636ff053e8334a2c.
The multi-batch flow is consistent with antd-go v0.12.0 and PaymentVaultV2: the allowance bound uses the upper median quote multiplied by 2^depth, payments are nonce-serialised, and the timeout/salvage branches avoid automatically re-paying an unconfirmed transaction. The single-batch compatibility path and multi-finalize ordering also match the SDK contract.
Verified locally:
go test ./... -count=1go test -race ./... -count=1go vet ./...- frontend
npm ci && npm run build git diff --check
Required exact-head CI is green (Docker, frontend, lint, smoke, PostgreSQL and SQLite).
Non-blocking follow-up: validateMerkleBatches does not yet fully enforce its “reject malformed data before any money moves” comment: it could also validate pool-hash syntax, the contract-required pool count, and non-negative uint256 candidate amounts. The later signer/contract checks plus partial-finalize path keep this from being a merge blocker, but validating those fields up front would make the stated invariant complete.
Fixes multi-batch merkle uploads against antd >= 0.12.0 (V2-1056), plus a latent allowance-formula bug the e2e exposed.
Linear: https://linear.app/autonominetwork/issue/V2-1056/multi-batch-merkle-uploads-fail-merklebatches
What
antd >= 0.12.0 splits uploads larger than one merkle tree (256 fresh chunks ≈ 1 GiB) into multiple
MerkleBatchesand leaves the legacy mirror fields empty, so the worker previously submitted an empty commitments list and failed opaquely.merkleBatchPlan):MerkleBatcheswhen present (antd >= 0.12.0 always populates it for merkle, len >= 1), legacy single-batch synthesis as the pre-0.12 fallback, and a clear "needs antd >= 0.12.0" error when both are empty. All batches are validated (depth 1-8, 16 candidates/pool, parseable amounts) before any money moves.payMerkleBatches): onepayForMerkleTreetx per batch (no ABI change —payForMerkleTree2()in antd docs is a naming slip for PaymentVaultV2's function, confirmed against evmlib + ADR-0003). Each confirmed batch is recorded before the next is paid (V2-426): onetransactionsrow per batch,tx_hash= that batch's winner pool hash.Signer.EnsureMerkleAllowance): pre-approves the whole plan's worst case, so the per-batch approval short-circuits (1 approve instead of N).FinalizeMerkleUploadMultiwith the index-aligned winner list when > 1 batch; the single-batch path keeps the legacyFinalizeMerkleUploadcall unchanged.""for unpaid batches — the paid batches' chunks store (surfaced asPARTIAL_UPLOAD), then the upload fails preserve-paid (errPaidNoRetry). A confirmation timeout never salvages: the tx may still mine, and skipping keeps the manual full finalize possible (the error carries the pending tx hash + batch index). If salvage unexpectedly fully succeeds (unpaid batches deduped), the upload completes normally.median16(winner pool quotes) * 2^depthand picks the winner at execution time (PaymentVaultV2.sol:140), so batch 1 of the e2e reverted withERC20InsufficientAllowance. Both the allowance bound and themax_gas_feeestimate now use the contract-accurate worst case (highest pool median << depth, summed across batches). The single-batch path had the same latent under-approval bug.E2E proof (local devnet, 2026-08-24)
24-node devnet (merkle pools need 16 candidate peers; the 10-node preset can't quote) + anvil + antd v0.12.0, this branch's indelible:
batches=2(depth 8 + depth 5) → one approve tx → batch 1/2 paid 3.0 ANT, batch 2/2 paid 0.75 ANT (distinct winner hashes) →FinalizeMerkleUploadMulti→ completed, 291 chunks, downloaded byte-identical (cmpclean). DB: twotransactionsrows with the exact per-batch winner hashes/amounts;actual_cost= 3.75 ANT = sum.chunks_stored=256, chunks_failed=35, total_chunks=291→ uploadfailed/paid_unfinalized, temp source preserved, exactly one transaction row.Unit:
go test ./...green (batch planning/fallback/version-gap error, validation, pad/salvage decision, contract-formula payout bounds, classifyFailure rows for the new wrapping shapes).Notes for review
merkleAllowancePayeris an optional interface seam so the uncommitted hosted-payment work (payerinterface) merges without changes — payers lacking the method fall back to per-batch self-approval insidePayForMerkleTree.RequeueStuck(60)headroom vs N×5-min confirmation waits on very large files — follow-up ticket suggested.🤖 Generated with Claude Code