diff --git a/docs/idempotency.md b/docs/idempotency.md new file mode 100644 index 000000000..e3f548e48 --- /dev/null +++ b/docs/idempotency.md @@ -0,0 +1,149 @@ +# Idempotency on the naira rails + +What is safe to send twice, why, and how sure we are. Every claim here has a test +in one of these two files, and when the two disagree, the tests are the ones to trust: + +``` +src/__tests__/linq/idempotency.test.ts the provider clients, against the fake +src/__tests__/routes/idempotency.test.ts the HTTP routes, end to end, against the fake +src/__tests__/fakes/linqProvider.ts the fake: what we believe Linq does +``` + +## Why this needs writing down + +Every creating call to Linq is wrapped in `retryOnProviderFailure`, which repeats it +once on a 5xx or a dropped connection. **Neither of those means nothing happened.** +Linq can create the order and then lose the answer — a reset connection, a proxy +timeout — and our retry goes out not knowing. A retry is safe only if the thing it +repeats is idempotent, and that depends on the route. + +Linq has no sandbox, so we cannot test against them. What we believe about their +behaviour lives in one place, the fake, with each belief marked `BELIEF:`. If +Linq's behaviour turns out to differ, correct the fake. The tests that relied on the +old belief then fail in CI instead of in production. + +## The routes + +| Route | Idempotent? | Keyed on | Who enforces it | +| --- | --- | --- | --- | +| `POST /offramp/orders` | **Yes** | `idempotencyKey` (+ `walletAddress`, amount) | Us, then Linq | +| `POST /ngn/customers` | **Yes**, for creation | `customerRef` | Linq only | +| `POST /webhooks/linq` | **Yes**, per event | `orderId` | Us | +| `POST /ngn/onramp/orders` | **No** | — | — | +| `POST /ngn/bills` | **No** | — | — | +| `POST /ngn/customers/kyc` | **No claim made** | — | Not retried; see W079 | + +### `POST /offramp/orders` + +**Our side.** The key is looked up in `OfframpOrder` before Linq is called. A match +with the same wallet and amount replays the stored order (`200`, `replayed: true`, +same `id`, new `accessToken`). A different wallet or amount gets `409`. The +replay never reaches Linq. + +**Linq's side.** We send `idempotencyKey` on every create, and the retry inside +`createOfframpOrder` sends the identical body. **We believe** Linq returns the +original order for a key it has seen. Their docs accept the field but do not say +what it guarantees, so this is unconfirmed. Two cases rely on it, because our own +check cannot see an order we have not recorded: + +1. **Linq's answer is lost on the way to us.** The internal retry repeats the create + with the same key. +2. **Linq created the order, then our database write failed.** The client gets a + `500` and retries. With no row to find, the retry goes back to Linq with the same + key. + +**Concurrent duplicates.** Two identical requests in flight together both miss the +lookup. Linq deduplicates, and the `@@unique([network, idempotencyKey])` index +refuses the second insert. **No duplicate is created**, but the losing caller gets +a `500` for an order that exists. Its retry then replays the order normally. Note +that a replay issues a new `accessToken` and the winner's token stops working; for +two submits from one client, that is the same person. + +**Not covered.** A replay that changes only the bank details (`bankAccount`, +`bankCode`, `bankName`, `accountName`) still replays, because those fields are not +stored. See the comment in `src/api/offramp.ts`. + +#### Scope and lifetime of the key + +| | Scope | Lifetime | +| --- | --- | --- | +| **Ours** | One network. The same key on testnet and mainnet makes two separate orders (tested). It is bound to the wallet that first used it. | **Forever.** `OfframpOrder` rows are never deleted, so a key is spent for good. There is no expiry to wait out. | +| **Linq's** | **Unknown.** Presumably our business account, with no notion of network: Linq has one environment. The fake assumes this. Unlike `/ngn`, the offramp router is not gated to mainnet, so a key reused across networks is reachable. We would hold two rows (one per network), and Linq would return the first order to both. A random key per order makes this practically impossible (tested). | **Unknown.** The fake remembers keys forever, which is the most generous reading. If Linq forgets keys after some window, case 2 above becomes a duplicate order once that window has passed. | + +So **clients must generate a fresh random key per order, and send the same key +again only when retrying that order**. Never derive a key from the order's contents. +Two genuine orders for the same amount would then collide and get a `409`. + +### `POST /ngn/customers` + +**We do no deduplication.** This works only because **Linq keys customers on +`customerRef`** and returns an existing one unchanged with `created: false`. Both +a client's retry and the internal retry after a dropped answer end with one +customer (tested). + +It is weaker than "the same request twice is a no-op" in two ways: + +- **The details are not updated.** A repeat with a corrected email or phone returns + the existing customer and keeps the old details. Provisioning again is not how + you fix a typo. +- **`created` describes the moment Linq answered, not this request.** After a + dropped answer, the retry finds the customer that the first attempt made, and the + caller is told `created: false` for someone who did not exist a second earlier. + Do not use `created` to decide whether to show a welcome step. + +`customerRef` has no expiry. A reference, once provisioned, is that person at Linq +for good, which is why a drifting reference becomes a NIN lockout (see +[ngn-rails.md](./ngn-rails.md)). + +### `POST /webhooks/linq` + +A redelivered event writes the same values again: one row, nothing counted twice +(tested for both the offramp and the onramp/bill tables). `recordNgnOrder` is +likewise an upsert on `(network, orderId)`. + +**Not order-safe.** The last delivery wins. A late redelivery of `order.processing` +that arrives after `order.completed` sets the status back to processing until the +next status poll corrects it. This is not a double-count, but the row reads wrong +in the meantime. + +### `POST /ngn/onramp/orders` and `POST /ngn/bills`: **not idempotent** + +No key is sent and Linq documents none, yet both are retried on a 5xx or a dropped +connection. A retry after a lost answer **places a second order**, and the caller +is given the second. A client that resubmits gets a second order the same way. Both +are pinned by tests, so whoever adds a key has to update them. + +**Why this does not move money twice.** Creating an order only reserves it. The +user is shown one order's bank account or deposit address, the one that answered, +and pays into that. The orphan receives no deposit and lapses at `expiresAt`. If +Linq sends a webhook for it, it arrives for an id we never recorded and is logged +and ignored. + +**What would make it idempotent.** Find out from Linq whether `/b2b/onramp` and +`/b2b/bills/pay` accept an idempotency key. If they do, take one from the client +as the offramp does. If they do not, consider not retrying a dropped connection or +timeout on these two, since those are the ambiguous failures, and keep the retry +for a 5xx that Linq documents as "nothing was created". + +### `POST /ngn/customers/kyc` + +**No claim is made, and none should be until Linq answers.** It is not retried +today. W079 proposes retrying it on 5xx. Before that lands, someone needs to know +what Linq does with a second submission of a NIN that the first attempt already +verified against *the same* `customerRef`. It might be a success, or it might be +`nin_already_used`. If it is the latter, a retry after a dropped answer turns a +successful verification into a refusal the user cannot get past. The fake +implements no KYC endpoint, so this is deliberately not encoded as a belief. + +## The 429 assumption + +`call()` in `src/linq/client.ts` retries a `429` up to twice, order creation +included. That is safe only if a rate-limited request did no work. This is the +conventional meaning of 429, but Linq has not stated it. The fake refuses a 429 +before doing anything, and one test relies on it. + +## Changing any of this + +- **Linq tells us something new:** change the fake first, then make the tests pass. +- **Adding a creating call:** decide which row of the table above it belongs to + before wrapping it in `retryOnProviderFailure`, then add it to both test files. diff --git a/docs/ngn-rails.md b/docs/ngn-rails.md index 938c573b6..b82e45930 100644 --- a/docs/ngn-rails.md +++ b/docs/ngn-rails.md @@ -101,7 +101,9 @@ POST /ngn/customers Safe to call again: Linq returns the existing customer with `created: false`, so a client retrying after a dropped response cannot create two customers for one -person. +person. It is keyed on `customerRef` alone, though, so a repeat with different +details keeps the first ones. What is and is not safe to repeat across all the +rails is in [idempotency.md](./idempotency.md). ### 2. Verify by NIN — once per person, ever @@ -335,9 +337,10 @@ id belongs to": the offramp table first, then `NgnOrder`. An id in neither is logged and ignored; nothing here creates a row, because an event for an order we never placed is not ours to act on. -Reconciliation is idempotent by construction — it writes settled figures to a row -keyed on the order id, so a redelivery writes the same values rather than -double-counting. The 200 is sent *before* reconciling, because Linq times out at +Reconciliation is idempotent per event — it writes settled figures to a row keyed +on the order id, so a redelivery writes the same values rather than +double-counting. It is not order-safe: a late redelivery of an earlier event wins +until the next status poll ([idempotency.md](./idempotency.md)). The 200 is sent *before* reconciling, because Linq times out at 10 seconds and marks the delivery failed, and a database write on a throttled instance can outlast that. @@ -378,6 +381,9 @@ on their own. ``` src/__tests__/linq/bills.test.ts the coin trap, refundAddress, auth header src/__tests__/routes/ngn.test.ts mainnet gate, C-address refusal, NIN absence +src/__tests__/linq/idempotency.test.ts what a retry repeats, per provider call +src/__tests__/routes/idempotency.test.ts the same request twice, per route +src/__tests__/fakes/linqProvider.ts the fake provider those two run against ``` The route tests need `NETWORKS=testnet,mainnet` in the environment, because diff --git a/docs/offramp-orders.md b/docs/offramp-orders.md index e012a54bc..cdf58fc18 100644 --- a/docs/offramp-orders.md +++ b/docs/offramp-orders.md @@ -44,6 +44,11 @@ a **new** `accessToken`; the previous token stops working. A different `walletAddress` gets `409` and nothing about the order. Idempotency keys should therefore be random and generated by the client. +A key is scoped to one network and never expires: once used, it belongs to that +order for good. Generate a new one per order and resend it only to retry that +order. A `500` on creation does not mean no order exists, so retry with the +same key rather than a new one. The details are in [idempotency.md](./idempotency.md). + ## Upgrading - Orders created before this change have no `id` or token and cannot be read diff --git a/src/__tests__/fakes/linqProvider.ts b/src/__tests__/fakes/linqProvider.ts new file mode 100644 index 000000000..79df95173 --- /dev/null +++ b/src/__tests__/fakes/linqProvider.ts @@ -0,0 +1,248 @@ +/** + * A fake of Linq, for tests that need the provider to behave like a provider + * rather than like whatever one test's `mockResolvedValue` imagined. + * + * ## Why this exists + * + * Linq has no sandbox: every call moves real naira or real personal data. So + * the only thing a test can run against is a fake, and the fake is where our + * beliefs about the provider are written down. Each endpoint below implements + * one belief and says so. When one of them turns out to be wrong, correct it + * here — the tests that relied on it will then fail, which is the point: they + * fail in CI instead of in production. + * + * This is deliberately small. It covers the creating endpoints whose + * idempotency the rails depend on (W090). W089 is the issue that grows it into + * the whole documented surface; extend this file rather than starting another. + * + * ## Nothing reaches the real provider + * + * `installFakeLinq` replaces `global.fetch` outright. Every request — whatever + * host it names — is answered here, and a path the fake does not implement gets + * a 404 and is recorded in `unhandled`, so a test can assert it never strayed. + * There is no pass-through to the network. + * + * ## No personal data + * + * The fake implements no KYC endpoint, so it never receives a NIN. It keeps the + * customer details a provisioning call sends because its behaviour depends on + * them (the "unchanged" in "returns the existing customer unchanged"); fixtures + * passed to it should be plainly fake. + */ + +import { randomUUID } from "node:crypto"; + +export interface FakeCustomer { + customerRef: string; + firstName: string; + lastName: string; + email: string; + phone: string; + verified: boolean; +} + +export interface FakeRequest { + method: string; + path: string; +} + +type Fault = + /** + * Linq does the work, then the response never reaches us — a connection + * reset, a proxy timeout. The order or customer EXISTS at the provider, and + * we see only a network failure. This is the case idempotency is for. + */ + | { kind: "drop-after-work" } + /** Linq refuses before doing any work, with this status. */ + | { kind: "refuse"; status: number; body: unknown }; + +export interface FakeLinq { + readonly customers: Map; + /** Offramp orders, by Linq's order id. */ + readonly offrampOrders: Map>; + /** Onramp orders, by Linq's order id. */ + readonly onrampOrders: Map>; + /** Bill orders, by Linq's order id. */ + readonly billOrders: Map>; + /** Every request that reached the fake, in order. Method and path only. */ + readonly requests: FakeRequest[]; + /** Requests to paths the fake does not implement. Should stay empty. */ + readonly unhandled: FakeRequest[]; + /** Apply `fault` to the next request to `path`, then behave normally. */ + failNext(path: string, fault: Fault): void; + /** How many requests reached `path`. */ + callsTo(path: string): number; + /** Put the real `fetch` back. */ + restore(): void; +} + +/** NGN per USD the fake quotes. Arbitrary; nothing here asserts on it. */ +const FAKE_NGN_PER_USD = 1500; + +const json = (status: number, body: unknown) => + new Response(JSON.stringify(body), { + status, + headers: { "Content-Type": "application/json" }, + }); + +export function installFakeLinq(): FakeLinq { + const originalFetch = global.fetch; + + const customers = new Map(); + const offrampOrders = new Map>(); + const offrampByKey = new Map(); + const onrampOrders = new Map>(); + const billOrders = new Map>(); + const requests: FakeRequest[] = []; + const unhandled: FakeRequest[] = []; + const faults = new Map(); + + // ── The provider's behaviour, one belief per endpoint ───────────────────── + + const routes: Record Response> = { + /** + * BELIEF: provisioning is keyed on `customerRef` alone. A second call with + * a reference Linq already holds returns that customer **unchanged**, with + * `created: false` — it neither errors nor updates the stored details. + * + * Source: Linq's B2B docs and observed behaviour; restated in the comment + * on `provisionCustomer`. Not contractually guaranteed to us. + */ + "POST /b2b/customers": (body) => { + const existing = customers.get(body.customerRef); + if (existing) { + return json(200, { + customerRef: existing.customerRef, + verified: existing.verified, + created: false, + }); + } + const customer: FakeCustomer = { + customerRef: body.customerRef, + firstName: body.firstName, + lastName: body.lastName, + email: body.email, + phone: body.phone, + verified: false, + }; + customers.set(customer.customerRef, customer); + return json(200, { customerRef: customer.customerRef, verified: false, created: true }); + }, + + /** + * BELIEF: an offramp order is deduplicated on `idempotencyKey`. A second + * create carrying a key Linq has already seen returns the ORIGINAL order — + * same id, same deposit address — and creates nothing. + * + * This is the belief the offramp's retry safety rests on whenever our own + * row does not exist yet (a retry inside `retryOnProviderFailure`, or a + * client retry after our database write failed). It is NOT confirmed by + * Linq: their docs accept the field without stating its scope or how long + * a key is remembered. The fake remembers it forever, which is the most + * generous reading. See docs/idempotency.md. + */ + "POST /b2b/offramp": (body) => { + const seen = offrampByKey.get(body.idempotencyKey); + if (seen) return json(200, offrampOrders.get(seen)); + + const amountStableCoin = + body.amountStableCoin ?? Number(body.amountNGN) / FAKE_NGN_PER_USD; + const order = { + id: randomUUID(), + walletAddress: `GDEPOSIT${offrampOrders.size}`, + coinType: "usdc", + coin: body.coin, + chain: body.chain, + amountStableCoin, + amountNGN: body.amountNGN ?? amountStableCoin * FAKE_NGN_PER_USD, + rate: FAKE_NGN_PER_USD, + currency: "NGN", + status: "initiated", + }; + offrampOrders.set(order.id, order); + offrampByKey.set(body.idempotencyKey, order.id); + return json(200, order); + }, + + /** + * BELIEF: an onramp order has NO deduplication. We send no key and Linq + * documents none, so every create is a new order with a new bank account. + */ + "POST /b2b/onramp": (body) => { + const order = { + orderId: randomUUID(), + customerRef: body.customerRef, + accountNumber: `00000000${onrampOrders.size}`.slice(-10), + bankName: "Fake Bank", + accountName: "Fake Collections", + amountNgn: body.amountStableCoin * body.rate, + amountStableCoin: body.amountStableCoin, + fee: 0, + expiresAt: "2099-01-01T00:00:00.000Z", + status: "pending", + }; + onrampOrders.set(order.orderId, order); + return json(200, order); + }, + + /** BELIEF: as for the onramp — no key is sent, so no deduplication. */ + "POST /b2b/bills/pay": (body) => { + const order = { + id: randomUUID(), + customerRef: body.customerRef, + wallet: `GBILLDEPOSIT${billOrders.size}`, + status: "initiated", + }; + billOrders.set(order.id, order); + return json(200, order); + }, + }; + + // ── Transport ───────────────────────────────────────────────────────────── + + const fakeFetch = async (input: unknown, init: RequestInit = {}): Promise => { + const url = new URL(String(input instanceof Request ? input.url : input)); + const method = (init.method ?? "GET").toUpperCase(); + const request = { method, path: url.pathname }; + requests.push(request); + + const handler = routes[`${method} ${url.pathname}`]; + if (!handler) { + unhandled.push(request); + return json(404, { message: `The fake does not implement ${method} ${url.pathname}` }); + } + + const fault = faults.get(url.pathname)?.shift(); + if (fault?.kind === "refuse") return json(fault.status, fault.body); + + const body = typeof init.body === "string" && init.body ? JSON.parse(init.body) : {}; + const response = handler(body); + + if (fault?.kind === "drop-after-work") { + // The shape undici gives a reset connection. The work above is done and + // stays done; only the answer is lost. + throw new TypeError("fetch failed"); + } + return response; + }; + + global.fetch = fakeFetch as typeof fetch; + + return { + customers, + offrampOrders, + onrampOrders, + billOrders, + requests, + unhandled, + failNext(path, fault) { + faults.set(path, [...(faults.get(path) ?? []), fault]); + }, + callsTo(path) { + return requests.filter((r) => r.path === path).length; + }, + restore() { + global.fetch = originalFetch; + }, + }; +} diff --git a/src/__tests__/linq/idempotency.test.ts b/src/__tests__/linq/idempotency.test.ts new file mode 100644 index 000000000..e7e6391d5 --- /dev/null +++ b/src/__tests__/linq/idempotency.test.ts @@ -0,0 +1,230 @@ +/** + * Which provider calls are safe to repeat, proven against the fake rather than + * asserted in a comment (W090). + * + * Every creating call here is wrapped in `retryOnProviderFailure`, so each one + * IS repeated whenever Linq answers 5xx or the connection drops. Whether that + * repeat is harmless depends on the provider recognising it, and the fake in + * `../fakes/linqProvider.ts` is where what we believe about that is written + * down. If Linq's behaviour changes, change the fake: these then fail here + * rather than in production. + * + * The case that matters is the dropped response — Linq did the work and the + * answer never arrived — not the polite duplicate. A duplicate is easy to + * deduplicate; a dropped response means our retry goes out not knowing whether + * the first one landed. + * + * The retry waits its real 1.5s, so the drop tests take a moment each. + */ +import { createOfframpOrder, LinqError } from "../../linq/client"; +import { createOnrampOrder, provisionCustomer } from "../../linq/onramp"; +import { payBill } from "../../linq/bills"; +import { installFakeLinq, type FakeLinq } from "../fakes/linqProvider"; + +const DROP = { kind: "drop-after-work" } as const; + +// Plainly fake. None of these identify anyone. +const customer = (over: Record = {}) => ({ + customerRef: "ref_test_0001", + firstName: "Test", + lastName: "Person", + email: "test.person@example.com", + phone: "00000000000", + ...over, +}); + +const offrampParams = (over: Record = {}) => ({ + amountStableCoin: 50, + bankAccount: "0000000000", + bankCode: "000", + bankName: "Fake Bank", + accountName: "Test Person", + customerRef: "GTESTWALLET", + idempotencyKey: "idem-key-1", + ...over, +}); + +let linq: FakeLinq; + +beforeEach(() => { + process.env.LINQ_API_KEY = "biz_live_test"; + linq = installFakeLinq(); + // Every injected failure is logged by the client, by design. Not this file's subject. + jest.spyOn(console, "warn").mockImplementation(() => {}); +}); + +afterEach(() => { + jest.restoreAllMocks(); + // Enforced, not assumed: nothing in this file asked for a path the fake does + // not implement, and the fake has no way to pass a request on. + expect(linq.unhandled).toEqual([]); + linq.restore(); +}); + +describe("provisionCustomer — idempotent on customerRef, by the provider's behaviour", () => { + it("the same request twice leaves one customer, and the second says it was not created", async () => { + const first = await provisionCustomer(customer()); + const second = await provisionCustomer(customer()); + + expect(first.created).toBe(true); + expect(second.created).toBe(false); + expect(linq.customers.size).toBe(1); + }); + + it("a dropped response is retried into the existing customer, not a second one", async () => { + linq.failNext("/b2b/customers", DROP); + + const result = await provisionCustomer(customer()); + + // Two requests reached Linq and both were "create" — the first one landed, + // we never heard, and the retry was recognised. + expect(linq.callsTo("/b2b/customers")).toBe(2); + expect(linq.customers.size).toBe(1); + // Worth knowing for anyone reading `created`: after a dropped response the + // caller is told false for a customer that this very call created. + expect(result.created).toBe(false); + }); + + it("is keyed on the reference alone, so a retry with different details keeps the first ones", async () => { + // The weaker half of the guarantee. "Cannot create two customers" holds; + // "the customer has the details you sent" does not. Correcting an email + // by provisioning again silently does nothing. + await provisionCustomer(customer({ email: "first@example.com" })); + const again = await provisionCustomer(customer({ email: "second@example.com" })); + + expect(again.created).toBe(false); + expect(linq.customers.size).toBe(1); + expect(linq.customers.get("ref_test_0001")?.email).toBe("first@example.com"); + }); +}); + +describe("createOfframpOrder — idempotent on idempotencyKey, by the provider's behaviour", () => { + it("the same key twice is one order with one deposit address", async () => { + const first = await createOfframpOrder(offrampParams()); + const second = await createOfframpOrder(offrampParams()); + + expect(second.id).toBe(first.id); + expect(second.walletAddress).toBe(first.walletAddress); + expect(linq.offrampOrders.size).toBe(1); + }); + + it("a dropped response is retried with the same key and recognised as the same order", async () => { + linq.failNext("/b2b/offramp", DROP); + + const order = await createOfframpOrder(offrampParams()); + + expect(linq.callsTo("/b2b/offramp")).toBe(2); + expect(linq.offrampOrders.size).toBe(1); + expect(linq.offrampOrders.has(order.id)).toBe(true); + }); + + it("a different key is a different order", async () => { + await createOfframpOrder(offrampParams({ idempotencyKey: "idem-key-1" })); + await createOfframpOrder(offrampParams({ idempotencyKey: "idem-key-2" })); + + expect(linq.offrampOrders.size).toBe(2); + }); + + it("a 429 did no work, so the request that follows it creates the only order", async () => { + // BELIEF, stated in `call()` in linq/client.ts: a rate-limited request was + // refused before anything was created. Encoded by the fake's `refuse`. + linq.failNext("/b2b/offramp", { + kind: "refuse", + status: 429, + body: { message: "Rate limit exceeded" }, + }); + global.fetch = withRetryAfterZero(global.fetch); + + await createOfframpOrder(offrampParams()); + + expect(linq.callsTo("/b2b/offramp")).toBe(2); + expect(linq.offrampOrders.size).toBe(1); + }); +}); + +describe("onramp and bill creation — NOT idempotent, and pinned as such", () => { + // Both are retried by `retryOnProviderFailure` exactly like the offramp, but + // neither sends a key and Linq documents none. A retry after a dropped + // response therefore places a SECOND order. These tests pin that, so the day + // a key is added the change has to come here and say so. + // + // Why it does not move money twice: nothing is paid on creation. The user is + // shown only the order that answered, and pays into that one; the orphan has + // no deposit and lapses at `expiresAt`. Its webhook, if any, arrives for an + // id we never recorded and is logged and ignored. + + it("createOnrampOrder after a dropped response leaves two orders, and returns the second", async () => { + linq.failNext("/b2b/onramp", DROP); + + const order = await createOnrampOrder({ + customerRef: "ref_test_0001", + amountStableCoin: 10, + walletAddress: "GTESTWALLET", + rate: 1500, + coin: "usdc", + }); + + expect(linq.onrampOrders.size).toBe(2); + const [orphan] = [...linq.onrampOrders.keys()]; + expect(order.orderId).not.toBe(orphan); + }); + + it("payBill after a dropped response leaves two orders, and returns the second", async () => { + linq.failNext("/b2b/bills/pay", DROP); + + const order = await payBill({ + customerRef: "ref_test_0001", + billCategory: "airtime", + provider: "MTN", + customerId: "00000000000", + amountNgn: 1000, + amountStableCoin: 0.67, + rate: 1500, + coin: "usdc", + refundAddress: "GTESTWALLET", + }); + + expect(linq.billOrders.size).toBe(2); + const [orphan] = [...linq.billOrders.keys()]; + expect(order.id).not.toBe(orphan); + }); + + it("a duplicate request is a duplicate order", async () => { + const params = { + customerRef: "ref_test_0001", + amountStableCoin: 10, + walletAddress: "GTESTWALLET", + rate: 1500, + coin: "usdc" as const, + }; + await createOnrampOrder(params); + await createOnrampOrder(params); + + expect(linq.onrampOrders.size).toBe(2); + }); +}); + +describe("what is never retried", () => { + it("a 4xx is the provider's answer, not a failure, and is not repeated", async () => { + linq.failNext("/b2b/customers", { + kind: "refuse", + status: 400, + body: { message: "email is invalid", code: "invalid_email" }, + }); + + await expect(provisionCustomer(customer())).rejects.toBeInstanceOf(LinqError); + expect(linq.callsTo("/b2b/customers")).toBe(1); + expect(linq.customers.size).toBe(0); + }); +}); + +/** The fake's 429 with `Retry-After: 0`, so the rate-limit wait is not real time. */ +function withRetryAfterZero(inner: typeof fetch): typeof fetch { + return (async (...args: Parameters) => { + const res = await inner(...args); + if (res.status !== 429) return res; + const headers = new Headers(res.headers); + headers.set("Retry-After", "0"); + return new Response(await res.text(), { status: 429, headers }); + }) as typeof fetch; +} diff --git a/src/__tests__/routes/idempotency.test.ts b/src/__tests__/routes/idempotency.test.ts new file mode 100644 index 000000000..4f38a3505 --- /dev/null +++ b/src/__tests__/routes/idempotency.test.ts @@ -0,0 +1,395 @@ +/** + * Every route documented as safe to repeat, repeated (W090). + * + * The provider is the fake in `../fakes/linqProvider.ts`, not a per-test mock, + * so where a guarantee depends on Linq it depends on the belief written there. + * The database is an in-memory stand-in that enforces the same unique + * constraints as `prisma/schema.prisma`, because one of the cases below is + * decided by exactly that constraint. + * + * "Dropped response" here means three different things, and each is covered: + * - the client never got OUR answer, and sends the request again; + * - Linq's answer never reached US, and `retryOnProviderFailure` sends it again; + * - Linq did the work but WE failed before recording it, so the client's retry + * finds no row and has to go back to Linq. + */ +import express from "express"; +import request from "supertest"; + +process.env.LINQ_API_KEY = "biz_live_test"; + +type Row = Record; +const tables: Record<"offrampOrder" | "ngnOrder", Row[]> = { offrampOrder: [], ngnOrder: [] }; +let nextId = 1; + +/** The @@unique sets from prisma/schema.prisma, which is what makes a race lose. */ +const UNIQUE: Record = { + offrampOrder: [ + ["network", "orderId"], + ["network", "publicId"], + ["network", "idempotencyKey"], + ], + ngnOrder: [["network", "orderId"]], +}; + +/** Prisma's compound-key `where`: `{ network_orderId: { network, orderId } }` or `{ id }`. */ +function matches(row: Row, where: Row): boolean { + return Object.entries(where).every(([key, value]) => + value !== null && typeof value === "object" + ? key.split("_").every((f) => row[f] === value[f]) + : row[key] === value, + ); +} + +function uniqueViolation(table: keyof typeof tables, data: Row, self?: Row): Error | null { + for (const fields of UNIQUE[table]) { + if (fields.some((f) => data[f] == null)) continue; + const clash = tables[table].find( + (r) => r !== self && fields.every((f) => r[f] === data[f]), + ); + if (clash) { + return Object.assign(new Error(`Unique constraint failed on (${fields.join(", ")})`), { + code: "P2002", + }); + } + } + return null; +} + +/** Let a microtask hop happen, so two requests genuinely interleave. */ +const tick = () => new Promise((r) => setImmediate(r)); + +const mockCreateFailure = jest.fn(() => null); + +function model(table: keyof typeof tables) { + return { + findUnique: jest.fn(async ({ where }: { where: Row }) => { + await tick(); + return tables[table].find((r) => matches(r, where)) ?? null; + }), + findFirst: jest.fn(async ({ where }: { where: Row }) => { + await tick(); + return tables[table].find((r) => matches(r, where)) ?? null; + }), + create: jest.fn(async ({ data }: { data: Row }) => { + await tick(); + const injected = table === "offrampOrder" ? mockCreateFailure() : null; + if (injected) throw injected; + const violation = uniqueViolation(table, data); + if (violation) throw violation; + const row = { id: nextId++, createdAt: new Date(), updatedAt: new Date(), ...data }; + tables[table].push(row); + return row; + }), + update: jest.fn(async ({ where, data }: { where: Row; data: Row }) => { + await tick(); + const row = tables[table].find((r) => matches(r, where)); + if (!row) throw Object.assign(new Error("Record not found"), { code: "P2025" }); + Object.assign(row, data, { updatedAt: new Date() }); + return row; + }), + upsert: jest.fn(async ({ where, create, update }: { where: Row; create: Row; update: Row }) => { + await tick(); + const row = tables[table].find((r) => matches(r, where)); + if (row) return Object.assign(row, update); + const created = { id: nextId++, createdAt: new Date(), updatedAt: new Date(), ...create }; + tables[table].push(created); + return created; + }), + }; +} + +jest.mock("../../db", () => ({ + prisma: { offrampOrder: model("offrampOrder"), ngnOrder: model("ngnOrder") }, +})); + +import { createOfframpRouter } from "../../api/offramp"; +import { createNgnRouter } from "../../api/ngn"; +import { createLinqWebhookRouter } from "../../api/linqWebhook"; +import { networkMiddleware } from "../../middleware/network"; +import { computeSignature, SIGNATURE_HEADER } from "../../linq/webhook"; +import { recordNgnOrder } from "../../linq/ngnOrders"; +import { installFakeLinq, type FakeLinq } from "../fakes/linqProvider"; + +const DROP = { kind: "drop-after-work" } as const; +const WALLET = "GAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAWHF"; +const WEBHOOK_SECRET = "whsec_test_secret"; + +function makeApp() { + const app = express(); + // Ahead of express.json(), as in api.ts: the webhook verifies raw bytes. + app.use("/webhooks/linq", createLinqWebhookRouter()); + app.use(express.json()); + app.use(networkMiddleware); + app.use("/offramp", createOfframpRouter()); + app.use("/ngn", createNgnRouter()); + return app; +} + +const offrampBody = (over: Record = {}) => ({ + amountStableCoin: 50, + bankAccount: "0000000000", + bankCode: "000", + bankName: "Fake Bank", + accountName: "Test Person", + walletAddress: WALLET, + idempotencyKey: "idem-key-1", + ...over, +}); + +// Plainly fake. None of these identify anyone. +const customerBody = { + customerRef: "ref_test_0001", + firstName: "Test", + lastName: "Person", + email: "test.person@example.com", + phone: "00000000000", +}; + +let linq: FakeLinq; + +beforeEach(() => { + tables.offrampOrder.length = 0; + tables.ngnOrder.length = 0; + nextId = 1; + mockCreateFailure.mockReset(); + mockCreateFailure.mockImplementation(() => null); + process.env.NETWORKS = "testnet,mainnet"; + process.env.LINQ_WEBHOOK_SECRET = WEBHOOK_SECRET; + linq = installFakeLinq(); + jest.spyOn(console, "warn").mockImplementation(() => {}); + jest.spyOn(console, "error").mockImplementation(() => {}); +}); + +afterEach(() => { + jest.restoreAllMocks(); + expect(linq.unhandled).toEqual([]); + linq.restore(); +}); + +// ─── POST /ngn/customers ────────────────────────────────────────────────────── + +describe("POST /ngn/customers", () => { + const provision = (app: express.Express) => + request(app).post("/ngn/customers").set("x-network", "mainnet").send(customerBody); + + it("the same request twice is one customer", async () => { + const app = makeApp(); + const first = await provision(app); + const second = await provision(app); + + expect([first.status, second.status]).toEqual([200, 200]); + expect(first.body.created).toBe(true); + expect(second.body.created).toBe(false); + expect(linq.customers.size).toBe(1); + }); + + it("a client retrying after our response was dropped gets the same customer", async () => { + const app = makeApp(); + await provision(app); // answered, but the client never saw it + const retry = await provision(app); + + expect(retry.status).toBe(200); + expect(retry.body).toEqual({ customerRef: "ref_test_0001", verified: false, created: false }); + expect(linq.customers.size).toBe(1); + }); + + it("Linq's answer dropped on the way to us is retried into the same customer", async () => { + linq.failNext("/b2b/customers", DROP); + + const res = await provision(makeApp()); + + expect(res.status).toBe(200); + expect(linq.callsTo("/b2b/customers")).toBe(2); + expect(linq.customers.size).toBe(1); + }); +}); + +// ─── POST /offramp/orders ───────────────────────────────────────────────────── + +describe("POST /offramp/orders", () => { + const place = (app: express.Express, over: Record = {}) => + request(app).post("/offramp/orders").send(offrampBody(over)); + + it("the same key twice is one provider order and one row, and the replay says so", async () => { + const app = makeApp(); + const first = await place(app); + const second = await place(app); + + expect(first.status).toBe(201); + expect(second.status).toBe(200); + expect(second.body.replayed).toBe(true); + expect(second.body.id).toBe(first.body.id); + expect(second.body.walletAddress).toBe(first.body.walletAddress); + expect(tables.offrampOrder).toHaveLength(1); + // Answered from our row: the replay never went back to Linq at all. + expect(linq.callsTo("/b2b/offramp")).toBe(1); + expect(linq.offrampOrders.size).toBe(1); + }); + + it("Linq's answer dropped on the way to us is one order, recorded once", async () => { + linq.failNext("/b2b/offramp", DROP); + + const res = await place(makeApp()); + + expect(res.status).toBe(201); + expect(linq.callsTo("/b2b/offramp")).toBe(2); + expect(linq.offrampOrders.size).toBe(1); + expect(tables.offrampOrder).toHaveLength(1); + expect(tables.offrampOrder[0].orderId).toBe([...linq.offrampOrders.keys()][0]); + }); + + it("Linq created it but we failed to record it: the client's retry finds the same order", async () => { + // The order exists at Linq and nowhere here, so our own idempotency check + // misses and the retry goes back to Linq. Only the provider's dedup on the + // key keeps this to one order — the fake's belief, not ours to promise. + mockCreateFailure.mockImplementationOnce(() => new Error("connection terminated")); + const app = makeApp(); + + const failed = await place(app); + expect(failed.status).toBe(500); + expect(linq.offrampOrders.size).toBe(1); + expect(tables.offrampOrder).toHaveLength(0); + + const retry = await place(app); + expect(retry.status).toBe(201); + expect(linq.callsTo("/b2b/offramp")).toBe(2); + expect(linq.offrampOrders.size).toBe(1); + expect(tables.offrampOrder).toHaveLength(1); + }); + + it("two identical requests in flight at once are one order; the loser is told to retry, and can", async () => { + // Both miss the existing-row check, both reach Linq (which dedupes on the + // key), and the unique index on (network, idempotencyKey) refuses the + // second insert. Nothing is duplicated — but the loser is answered 500 + // for an order that exists. Pinned so that is a known edge, not a surprise. + const app = makeApp(); + const [a, b] = await Promise.all([place(app), place(app)]); + + expect([a.status, b.status].sort()).toEqual([201, 500]); + expect(linq.offrampOrders.size).toBe(1); + expect(tables.offrampOrder).toHaveLength(1); + + const retry = await place(app); + expect(retry.status).toBe(200); + expect(retry.body.replayed).toBe(true); + }); + + it("the key is scoped to the network: the same key on mainnet is a separate order", async () => { + const app = makeApp(); + await place(app); + const other = await request(app) + .post("/offramp/orders") + .set("x-network", "mainnet") + .send(offrampBody()); + + expect(other.status).toBe(201); + expect(tables.offrampOrder).toHaveLength(2); + // Our rows are per network; whether Linq treats the key as per network is + // unknown. The fake does not, and returns the testnet order's id. + expect(linq.offrampOrders.size).toBe(1); + }); +}); + +// ─── recordNgnOrder ─────────────────────────────────────────────────────────── + +describe("recordNgnOrder", () => { + const params = { + network: "mainnet", + kind: "onramp" as const, + orderId: "00000000-0000-4000-8000-000000000001", + customerRef: "ref_test_0001", + walletAddress: WALLET, + coin: "usdc", + amountStableCoin: 10, + amountNgn: 15000, + rate: 1500, + status: "pending", + }; + + it("recording the same order id twice keeps one row", async () => { + await recordNgnOrder(params); + await recordNgnOrder({ ...params, status: "processing" }); + + expect(tables.ngnOrder).toHaveLength(1); + expect(tables.ngnOrder[0].status).toBe("processing"); + }); +}); + +// ─── POST /webhooks/linq ────────────────────────────────────────────────────── + +describe("POST /webhooks/linq", () => { + const deliver = (app: express.Express, event: Record) => { + const raw = JSON.stringify(event); + return request(app) + .post("/webhooks/linq") + .set("Content-Type", "application/json") + .set(SIGNATURE_HEADER, computeSignature(raw, WEBHOOK_SECRET)) + .send(raw); + }; + + /** The route answers before it reconciles; wait for the write to land. */ + const reconciled = async () => { + for (let i = 0; i < 20; i++) await tick(); + }; + + it("a redelivered event is applied once, and the second delivery changes nothing", async () => { + const app = makeApp(); + const placed = await request(app).post("/offramp/orders").send(offrampBody()); + expect(placed.status).toBe(201); + const orderId = tables.offrampOrder[0].orderId; + + const event = { + event: "order.completed", + orderId, + status: "completed", + amountStableCoin: 50, + amountNGN: 75000, + txHash: "f".repeat(64), + }; + + expect((await deliver(app, event)).status).toBe(200); + await reconciled(); + const afterFirst: Row = { ...tables.offrampOrder[0], updatedAt: undefined }; + + expect((await deliver(app, event)).status).toBe(200); + await reconciled(); + const afterSecond: Row = { ...tables.offrampOrder[0], updatedAt: undefined }; + + expect(afterFirst.status).toBe("completed"); + expect(afterFirst.settledNGN).toBe("75000"); + expect(afterSecond).toEqual(afterFirst); + expect(tables.offrampOrder).toHaveLength(1); + }); + + it("a redelivered event for an onramp order does not create or duplicate a row", async () => { + await recordNgnOrder({ + network: "mainnet", + kind: "onramp", + orderId: "00000000-0000-4000-8000-000000000002", + customerRef: "ref_test_0001", + walletAddress: WALLET, + coin: "usdc", + amountStableCoin: 10, + amountNgn: 15000, + rate: 1500, + status: "pending", + }); + const app = makeApp(); + const event = { + event: "order.completed", + orderId: "00000000-0000-4000-8000-000000000002", + status: "completed", + amountStableCoin: 10, + }; + + await deliver(app, event); + await reconciled(); + await deliver(app, event); + await reconciled(); + + expect(tables.ngnOrder).toHaveLength(1); + expect(tables.ngnOrder[0].status).toBe("completed"); + expect(tables.ngnOrder[0].settledStableCoin).toBe("10"); + }); +}); diff --git a/src/api/linqWebhook.ts b/src/api/linqWebhook.ts index cd05729d2..1fede1b8b 100644 --- a/src/api/linqWebhook.ts +++ b/src/api/linqWebhook.ts @@ -59,11 +59,17 @@ export function createLinqWebhookRouter(): Router { res.status(200).json({ ok: true }); try { - // Idempotent by construction: the update targets a row keyed on the - // order id and writes the settled figures. A redelivery writes the - // same values a second time, which is a no-op rather than a - // double-count. Nothing here creates a row - an event for an order we - // never placed is not ours to act on. + // Idempotent per event: the update targets a row keyed on the order + // id and writes the settled figures. A redelivery of the SAME event + // writes the same values a second time, which is a no-op rather than + // a double-count (pinned in __tests__/routes/idempotency.test.ts). + // Nothing here creates a row - an event for an order we never placed + // is not ours to act on. + // + // Not order-safe, though: the last delivery wins. A late redelivery + // of `order.processing` arriving after `order.completed` sets the + // status back to processing. Nothing is counted twice, but the row + // reads wrong until the next status poll corrects it. const network = DEFAULT_NETWORK; const existing = await prisma.offrampOrder.findFirst({ where: { orderId: event.orderId }, diff --git a/src/api/ngn.ts b/src/api/ngn.ts index 60188e95b..a030950ef 100644 --- a/src/api/ngn.ts +++ b/src/api/ngn.ts @@ -330,7 +330,9 @@ export function createNgnRouter( /** * One-time per person, not per order. Safe to call again: Linq returns the * existing customer with `created: false`, so a client retrying after a - * dropped response cannot create two customers for one person. + * dropped response cannot create two customers for one person. Keyed on + * `customerRef` alone — a repeat with different details keeps the first + * ones. See `provisionCustomer` and docs/idempotency.md. */ router.post("/customers", async (req: Request, res: Response) => { const { customerRef, firstName, lastName, email, phone } = req.body ?? {}; diff --git a/src/api/offramp.ts b/src/api/offramp.ts index b233079a2..4b1e60ccc 100644 --- a/src/api/offramp.ts +++ b/src/api/offramp.ts @@ -205,6 +205,21 @@ export function createOfframpRouter( // The key comes from the caller and is never generated here: regenerating // it on a retry is precisely how one order becomes two, and the second one // also gets paid for. + // + // Scope: one key per network, enforced by @@unique([network, + // idempotencyKey]). Lifetime: as long as the row, and rows are never + // deleted, so a key is spent for good. Full write-up in + // docs/idempotency.md. + // + // This check only covers orders we have RECORDED. Two cases get past it + // and are held to one order by Linq deduplicating on the key instead — + // a belief about the provider, encoded in the fake and pinned in + // __tests__/routes/idempotency.test.ts: + // - Linq created the order but our insert below failed, and the client + // retried. No row, so the retry goes back to Linq with the same key. + // - Two identical requests in flight together. Both miss here; the + // unique index refuses the second insert, and that caller gets a 500 + // for an order that exists. Retrying then replays it. const existing = await prisma.offrampOrder.findUnique({ where: { network_idempotencyKey: { network: net, idempotencyKey } }, }); diff --git a/src/linq/bills.ts b/src/linq/bills.ts index 65f739e95..24c86e146 100644 --- a/src/linq/bills.ts +++ b/src/linq/bills.ts @@ -178,6 +178,11 @@ export interface BillOrderResponse { * Nothing is vended until the deposit arrives — this call only reserves the * order and locks the rate. The user then sends `amountStableCoin` of `coin` * to the returned `wallet`, and Linq vends automatically on confirmation. + * + * **Not idempotent**, for the same reason as `createOnrampOrder`: no key is + * sent, but a 5xx or dropped connection is retried, so a lost answer becomes a + * second order. Nothing is vended without a deposit and the user is shown only + * the second `wallet`, so the orphan is never paid. See docs/idempotency.md. */ export async function payBill(params: PayBillParams): Promise { const { diff --git a/src/linq/client.ts b/src/linq/client.ts index 413fde6c6..b661e6d61 100644 --- a/src/linq/client.ts +++ b/src/linq/client.ts @@ -105,9 +105,11 @@ type RateLimited = LinqError & { retryAfterMs?: number }; * All Veil users share one provider key, and the provider limits it. A user * creating a cash-out got "rate limit exceeded, please wait" and succeeded on * a second tap: the limit had cleared within a second or two, so the app should - * have waited, not the user. A 429 means the request was refused before any - * work was done, so repeating it (order creation included, with the same - * idempotencyKey) cannot create anything twice. + * have waited, not the user. A 429 is taken to mean the request was refused + * before any work was done, so repeating it (order creation included, with the + * same idempotencyKey) cannot create anything twice. That is the conventional + * meaning rather than one Linq has stated; the fake provider encodes it, so a + * correction there fails the test that relies on it. */ async function call(path: string, init: CallInit = {}): Promise { const deadline = init.deadline ?? Date.now() + TOTAL_BUDGET_MS; @@ -346,8 +348,22 @@ export async function createOfframpOrder( * Order creation can fail transiently at the provider — "Wallet generation * failed" is it not managing to mint the deposit address — and the user was * shown that and left to try again by hand. A 4xx is about the request and - * is never retried. The retry sends the identical body, idempotencyKey - * included, so the provider can recognise it as the same order. + * is never retried. + * + * The retry sends the identical body. Whether that is SAFE depends on the + * caller, not on this function, because a 5xx or a dropped connection does not + * mean nothing was created — Linq may have done the work and lost the answer: + * + * - offramp orders carry an idempotencyKey, and Linq is believed to return the + * original order for a key it has seen. Believed, not documented to us. + * - customers are keyed on customerRef, and Linq returns the existing one. + * - onramp orders and bills carry NO key. A retry here after a lost answer + * places a second, unpaid order; the caller is given the second. + * + * Each case is pinned against the fake provider in + * `__tests__/linq/idempotency.test.ts`, and docs/idempotency.md says why the + * unkeyed ones do not move money twice. Wrap nothing new in this without + * deciding which of those it is. * * `deadline` bounds the pair. Without it the retry doubled the caller's whole * budget, which is how a 35s ceiling silently became 70. diff --git a/src/linq/ngnOrders.ts b/src/linq/ngnOrders.ts index 6e4cede1c..40fe6582d 100644 --- a/src/linq/ngnOrders.ts +++ b/src/linq/ngnOrders.ts @@ -58,8 +58,13 @@ export interface RecordNgnOrderParams { * user needs. Failing their request because our bookkeeping failed would turn a * placed order into an error message, and they would reasonably place it again. * - * Idempotent on `(network, orderId)`, so a retried delivery or a double-submit - * updates the row rather than creating a second one. + * Idempotent on `(network, orderId)`: recording the same order again updates + * its row rather than creating a second one. + * + * That is idempotency of the *record*, not of the order. A client that submits + * the same onramp or bill twice gets two orders from Linq, with two ids, and + * so two rows — correctly, because there are two orders. See + * docs/idempotency.md. */ export async function recordNgnOrder(params: RecordNgnOrderParams): Promise { const { diff --git a/src/linq/onramp.ts b/src/linq/onramp.ts index a13645e17..a3652af60 100644 --- a/src/linq/onramp.ts +++ b/src/linq/onramp.ts @@ -272,7 +272,11 @@ export interface ProvisionCustomerParams { export interface LinqCustomer { customerRef: string; verified: boolean; - /** False when the customer already existed — provisioning is idempotent. */ + /** + * False when the customer already existed at the moment Linq answered — + * provisioning is idempotent. Not "this request created nothing": see + * {@link provisionCustomer} on dropped responses. + */ created: boolean; /** * True when `customerRef` is **not** the one that was asked for. @@ -291,7 +295,8 @@ export interface LinqCustomer { * Safe to call again with the same `customerRef`: Linq returns the existing * customer unchanged with `created: false` rather than erroring or duplicating, * which means a client that retries after a dropped response cannot create two - * customers for one person. + * customers for one person. That rests on Linq's behaviour, which the fake + * provider encodes and `__tests__/linq/idempotency.test.ts` holds us to. * * **The reference that comes back is not always the one that went in.** An * email already registered under another reference is refused with @@ -308,6 +313,14 @@ export interface LinqCustomer { * they sent. A reference that drifts between calls is a permanent lockout for * that person — `reusedExistingRef` is set so a caller can tell this happened * rather than having to compare strings. + * + * Two further things it does NOT promise: + * - The details you sent are stored. The key is the reference alone, and an + * existing customer comes back *unchanged* — provisioning again with a + * corrected email or phone silently keeps the old one. + * - `created` says whether *this call* created them. After a dropped response + * the internal retry finds the customer the first attempt made, and answers + * `created: false` for a person who did not exist a second earlier. */ export async function provisionCustomer(params: ProvisionCustomerParams): Promise { const deadline = Date.now() + TOTAL_BUDGET_MS; @@ -465,6 +478,14 @@ export interface OnrampOrderResponse { * carries a warning that funds sent on the wrong chain are unrecoverable. The * mapping is therefore explicit and total, so a new coin cannot silently fall * through to a default. + * + * **Not idempotent.** No key is sent and Linq documents none, yet this is + * retried on a 5xx or a dropped connection — so a retry after Linq created the + * order and lost the answer places a second one, and the customer is shown the + * second. That is tolerable only because creating an order moves no money: the + * orphan has no deposit and lapses at `expiresAt`. A client that resubmits + * gets a second order the same way. Pinned in `__tests__/linq/idempotency.test.ts`; + * see docs/idempotency.md before adding a key or removing the retry. */ // `async` deliberately, matching `createOfframpOrder`: the guards below must // surface as a rejection rather than a synchronous throw, so a caller that