diff --git a/CHANGELOG.md b/CHANGELOG.md index 00d3ae1..dc1ac61 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,18 @@ All notable changes to `@luxalgo/broker-sdk` are documented here. +## Unreleased + +These schema additions are intended for the next minor release; the package version is unchanged until release preparation. + +- Trade schema: optional native fill/instrument identity, a deterministic numeric tie-break key (`executionTieBreak`, not a chronology guarantee), and provider-reported realized gross P&L. +- Topstep: preserve fill IDs, full contract IDs, and sub-millisecond UTC timestamps; include reported fees plus commissions and distinguish explicit zero from unknown costs; expose `profitAndLoss` as `reportedGrossPnl`; reject `success: false` JSON responses even with HTTP 200. Provider array order and automatic discovery of all returned accounts are preserved. +- Topstep: malformed JSON bodies raise typed request errors. Failed searches abort the snapshot instead of appearing as empty positions/history, preventing false position-closure events. Both cost fields exist in the published API schema; their additive interpretation remains an accounting assumption. +- Topstep compatibility: timestamps without `Z` or a numeric offset now leave `executedAt` absent instead of using the host's local timezone. A negative `fees` or `commissions` component omits the entire `fee`, even when the other component is positive, because negative component semantics are undocumented. +- Sync: prefer native fill IDs within each account. Match pre-upgrade Topstep snapshots using millisecond timestamps and occurrence counts, reserving exact matches first. Old fills with identical legacy fields cannot be individually identified during migration. +- Stats: compare mixed-length fractional timestamps chronologically without losing sub-millisecond digits. Topstep fraction parsing no longer relies on engines accepting more than three digits in `Date.parse`. +- Questrade: update in-memory credentials and await persistence immediately after single-use token exchange, before account/history requests. Persistence failures propagate and stop further requests. Direct adapter consumers can use `FetchContext.onCredentialsRotated`; consumers without it retain `rotatedCredentials` on success. + ## 0.5.1 - IBKR Flex XML: optional `statementTimeZone` on `connect()` and `parseFlexStatement()` interprets timestamps without offsets in the selected IANA timezone, including daylight saving. Existing callers retain the UTC default. Explicit UTC/numeric offsets are honored; invalid, ambiguous, or nonexistent local timestamps leave `executedAt` absent. diff --git a/README.md b/README.md index 7b9b861..0ea6a34 100644 --- a/README.md +++ b/README.md @@ -56,10 +56,29 @@ type Account = { cash?: number; // when the broker reports it separately environment?: "live" | "paper"; positions: { symbol: string; quantity: number; marketValue?: number }[]; - trades: { symbol: string; side: "buy" | "sell"; quantity: number; price: number; fee?: number; executedAt?: string }[]; + trades: { + symbol: string; + side: "buy" | "sell"; + quantity: number; + price: number; + fee?: number; + executedAt?: string; + executionId?: string; + instrumentId?: string; + executionTieBreak?: number; + reportedGrossPnl?: number; + }[]; }; ``` +Topstep fills retain native fill IDs, full futures contract IDs, and UTC +timestamps with source fractional precision. `fee` includes reported fees +plus commissions (including explicit zero); `reportedGrossPnl` carries the +provider's realized gross P&L. Numeric fill IDs provide `executionTieBreak` as a +deterministic tie-break for equal timestamps, without a guarantee of actual +chronology. See [the schema](docs/SCHEMA.md#trade) for cost assumptions and consumer +limitations. All returned provider accounts are imported automatically. + ### IBKR Flex statement timezone For Flex XML timestamps without an offset, pass the timezone used by the report: @@ -166,7 +185,7 @@ const positions = positionsFromTrades(trades); - **Your keys stay yours.** The SDK runs where your code runs. There is no LuxAlgo server in the path, no telemetry, no phoning home. We don't want your keys. - **Read-only by default.** The root export reads accounts, balances, positions, and history. Nothing in it can place an order. -- **Credential rotation is first-class.** Brokers with single-use tokens (Questrade) hand the rotated credentials back through `onCredentialsRotated` so you can persist them before the old ones die. +- **Credential rotation is first-class.** Questrade updates in-memory credentials and awaits `onCredentialsRotated` before account/history requests. Persist the replacement there; callback errors abort further requests. See [the rotation recipe](docs/recipes.md#persist-snapshots-and-survive-token-rotation). - **Fail-soft, never fabricate.** A position the broker can't price has no `marketValue` rather than a made-up one. A history row that can't be read is skipped and counted, not guessed. ## Give your AI agent portfolio access @@ -197,6 +216,11 @@ npx broker-sync --interval 300 --webhook-url https://example.com/hooks/portfolio Events are `trade_executed`, `balance_changed`, `position_opened`, `position_closed`, `position_changed`, `broker_error`, and `sync_completed`, delivered per sweep as one JSON batch. When a secret is configured, every webhook request carries `X-BrokerSync-Signature`, the lowercase hex HMAC-SHA256 of the raw body. State persists to a local JSON file before delivery, so a crash can drop a batch but never replay one. Programmatic use lives at `@luxalgo/broker-sdk/sync`. +The CLI keeps rotated credentials in memory while running, but never stores them +in snapshot state or updates its environment/config file. For Questrade tokens +that must survive restarts, supply SDK connections with a persistence callback +to the programmatic daemon; see [the rotation recipe](docs/recipes.md#persist-snapshots-and-survive-token-rotation). + ## Place orders (experimental) The write layer lives in a deliberately separate module. Importing the SDK never gives code trading capability by accident: diff --git a/conformance/vectors/topstep.json b/conformance/vectors/topstep.json index d7db15c..c338d98 100644 --- a/conformance/vectors/topstep.json +++ b/conformance/vectors/topstep.json @@ -1,6 +1,6 @@ { "broker": "topstep", - "description": "Futures prop account: contract ids like CON.F.US.EP.U26 normalize to the symbol segment, position type 2 means short, voided executions drop out.", + "description": "Futures prop account: native fill/contract identity, numeric ID tie-break key, reported gross P&L, fees plus commissions, short positions, and voided execution filtering.", "raw": { "accounts": [ { @@ -17,13 +17,16 @@ ], "trades": [ { + "id": 101, "contractId": "CON.F.US.EP.U26", "side": 1, "size": 1, "price": 5600.25, "fees": 1.4, "creationTimestamp": "2026-08-10T15:00:00Z", - "voided": false + "voided": false, + "commissions": 0.5, + "profitAndLoss": 32.5 }, { "contractId": "CON.F.US.EP.U26", @@ -52,12 +55,16 @@ ], "trades": [ { + "executionId": "101", + "instrumentId": "CON.F.US.EP.U26", "symbol": "EP", "side": "sell", "quantity": 1, "price": 5600.25, - "fee": 1.4, - "executedAt": "2026-08-10T15:00:00.000Z" + "fee": 1.9, + "executedAt": "2026-08-10T15:00:00.000Z", + "executionTieBreak": 101, + "reportedGrossPnl": 32.5 } ] } diff --git a/docs/SCHEMA.md b/docs/SCHEMA.md index 610da2b..b872c5b 100644 --- a/docs/SCHEMA.md +++ b/docs/SCHEMA.md @@ -32,11 +32,35 @@ One shape for everything, regardless of broker. This document is the human-reada | Field | Type | Notes | | --- | --- | --- | | `symbol` | `string` | Same normalization as positions. | +| `executionId?` | `string` | Native fill identifier, scoped to the broker account; never substituted with an order ID. | +| `instrumentId?` | `string` | Native instrument identifier, such as Topstep's full futures contract ID, including expiry. | +| `executionTieBreak?` | `number` | Deterministic numeric tie-break key for equal timestamps. Does not imply chronological sequence unless the provider documents that guarantee. | +| `reportedGrossPnl?` | `number` | Provider-reported realized gross P&L on this fill, in account currency, before fees/commissions. Omitted when unknown; zero and losses are preserved. | | `side` | `"buy" \| "sell"` | Always from the account holder's side. | | `quantity` | `number` | Always positive — `side` carries direction. | | `price` | `number` | Per unit, in the account's `currency`. | -| `fee?` | `number` | Commission/fee, absolute value, only when reported and non-zero. | -| `executedAt?` | `string` | ISO 8601, only when the source carried a parseable timestamp. | +| `fee?` | `number` | Non-negative reported costs. Topstep sums available non-negative `fees` and `commissions`, preserving explicit zero; omitted when neither is known or either is negative. Other adapters may omit zero costs. | +| `executedAt?` | `string` | ISO 8601, only when the source carried a parseable timestamp. Topstep normalizes offsets to UTC while retaining fractional precision beyond milliseconds. | + +Topstep's `executionTieBreak` uses non-negative safe-integer numeric fill IDs; +the [API reference](https://gateway.docs.projectx.com/docs/api-reference/trade/trade-search/) +does not guarantee their chronology. String IDs have no inferred ordering key; +unsafe numeric IDs are omitted. Provider array order is preserved. Stats compares +precise instants, preserving input order for ties; it does not use this key, +contract IDs, or reported P&L to reconstruct futures results. + +Keep precise timestamps as strings: `Date` discards sub-millisecond digits. +Sync uses native fill IDs and occurrence counts within each account. On upgrade, +old Topstep fills without native metadata can match by the original trade fields +and millisecond timestamp. Exact matches take precedence. Fills indistinguishable +in the old data remain ambiguous during that transition. Other timestamp-keyed +consumers must handle the precision change themselves. + +The [published Topstep API schema](https://api.topstepx.com/swagger/v1/swagger.json) +declares both `fees` and optional `commissions`, but does not describe whether +`fees` is inclusive. The adapter adds these fields as separate components; +confirm totals against broker statements when accounting accuracy is required. +Negative component semantics are also unspecified, so their total is omitted. ### Bar diff --git a/docs/recipes.md b/docs/recipes.md index d3601b8..f3b4fe7 100644 --- a/docs/recipes.md +++ b/docs/recipes.md @@ -67,13 +67,26 @@ import { connect } from "@luxalgo/broker-sdk"; const connection = connect({ broker: "questrade", credentials: { refreshToken: loadToken() }, - // Questrade tokens are single-use: persist the rotation or the next fetch dies. + // Awaited immediately after exchange, before any account/history requests. onCredentialsRotated: (next) => saveToken(next.refreshToken), }); const snapshot = await connection.fetchSnapshot(); await db.upsert("accounts", snapshot.accounts); // Account.id is a stable upsert key ``` +`saveToken` must return its persistence promise so the SDK can await it. +Questrade's old refresh token is invalid after exchange. The replacement is +already in `connection.credentials` even if persistence or a later request +fails; a persistence error stops further provider requests. Avoid concurrent +fetches sharing a single-use token. + +Direct `questrade.fetchRaw(credentials, { fetch, onCredentialsRotated })` +consumers can supply the same awaited callback, which receives the rotated +credential fields. Without it, successful calls retain the legacy +`rotatedCredentials` result, but a later request failure prevents that result +from being returned. With a callback, rotation is delivered only through the +callback. Other adapters retain their existing rotation timing. + ## Give your agent read access (MCP) No code — run [`@luxalgo/mcp`](https://github.com/LuxAlgo/luxalgo-mcp-server) locally with your read-only keys as `BROKERS_*` env vars in your MCP client config, then ask Claude "how's my portfolio doing?". diff --git a/src/adapters/questrade.ts b/src/adapters/questrade.ts index 913397a..466e8fa 100644 --- a/src/adapters/questrade.ts +++ b/src/adapters/questrade.ts @@ -7,9 +7,10 @@ import type { BrokerAdapter, Credentials, FetchContext } from "./types.js"; Questrade (Canada), read-only via the user's own API refresh token (generated self-serve in Questrade's API centre). The token is single-use: every exchange returns a fresh access token AND a fresh refresh token, so - the rotated pair is handed back through rotatedCredentials — persist it - before the old one dies. Accounts, balances, positions, and the last month - of trade activities. + the replacement refresh token must be persisted immediately, before any + later request can fail. Direct callers without a rotation callback receive + rotatedCredentials on success. Accounts, balances, positions, and the last + month of trade activities. */ const QUESTRADE_LOGIN = "https://login.questrade.com"; @@ -123,6 +124,10 @@ const fetchRaw = async (credentials: Credentials, ctx: FetchContext) => { throw new BrokerRequestError("questrade", "Questrade returned an incomplete token exchange"); } + // History requests can fail after this single-use token has been consumed. + // Persist the replacement before making any of those requests. + await ctx.onCredentialsRotated?.({ refreshToken: tokens.refresh_token }); + const get = async (path: string): Promise => { const response = await ctx.fetch(`${apiServer}v1${path}`, { headers: { Authorization: `Bearer ${accessToken}` }, @@ -156,8 +161,8 @@ const fetchRaw = async (credentials: Credentials, ctx: FetchContext) => { return { raw: { accounts }, - // The old refresh token is now dead — persist the rotated one. - rotatedCredentials: { refreshToken: tokens.refresh_token }, + // Compatibility for direct adapter callers without the immediate callback. + ...(ctx.onCredentialsRotated ? {} : { rotatedCredentials: { refreshToken: tokens.refresh_token } }), }; }; @@ -166,7 +171,7 @@ export const questrade: BrokerAdapter = { displayName: "Questrade", credentials: [{ key: "refreshToken", label: "API refresh token", secret: true }], readOnlySetup: - "Generate a refresh token in Questrade's API centre. Tokens are single-use and rotate on every fetch — persist rotatedCredentials (see onCredentialsRotated) or the connection dies.", + "Generate a refresh token in Questrade's API centre. Tokens are single-use and rotate on every fetch — persist them with onCredentialsRotated or the connection dies.", fetchRaw, normalize, }; diff --git a/src/adapters/topstep.ts b/src/adapters/topstep.ts index 11c754b..660883c 100644 --- a/src/adapters/topstep.ts +++ b/src/adapters/topstep.ts @@ -1,6 +1,7 @@ import { BrokerAuthError, BrokerRequestError, MissingCredentialsError } from "../errors.js"; import type { Account, Position, Trade } from "../schema.js"; -import { asFiniteNumber, asIsoTimestamp, rejectResponse } from "./http.js"; +import { asPreciseIsoTimestamp } from "../timestamps.js"; +import { asFiniteNumber, rejectResponse } from "./http.js"; import type { BrokerAdapter, Credentials, FetchContext } from "./types.js"; /* @@ -15,10 +16,13 @@ const TOPSTEP_API = "https://api.topstepx.com/api"; const TRADE_WINDOW_DAYS = 30; type TopstepTradeRow = { + id?: number | string; contractId?: string; creationTimestamp?: string; price?: number; fees?: number; + commissions?: number; + profitAndLoss?: number | null; side?: number; size?: number; voided?: boolean; @@ -50,14 +54,32 @@ export const parseTopstepTrades = (trades: TopstepTradeRow[] | undefined): Trade const quantity = Math.abs(trade.size ?? 0); const price = trade.price ?? 0; if (!trade.contractId || !side || quantity <= 0 || price <= 0) continue; - const fee = Math.abs(trade.fees ?? 0); - const executedAt = asIsoTimestamp(trade.creationTimestamp); + const fees = asFiniteNumber(trade.fees); + const commissions = asFiniteNumber(trade.commissions); + // The API does not specify negative component semantics; don't invent a charge. + const fee = (fees !== undefined || commissions !== undefined) + && (fees ?? 0) >= 0 && (commissions ?? 0) >= 0 + ? (fees ?? 0) + (commissions ?? 0) + : undefined; + const reportedGrossPnl = asFiniteNumber(trade.profitAndLoss); + const executedAt = asPreciseIsoTimestamp(trade.creationTimestamp); mapped.push({ + ...(typeof trade.id === "string" && trade.id.length > 0 + ? { executionId: trade.id } + : typeof trade.id === "number" && Number.isSafeInteger(trade.id) + ? { executionId: String(trade.id) } + : {}), + // Numeric IDs give a repeatable tie-break, not a documented execution sequence. + ...(typeof trade.id === "number" && Number.isSafeInteger(trade.id) && trade.id >= 0 + ? { executionTieBreak: trade.id } + : {}), + ...(reportedGrossPnl !== undefined ? { reportedGrossPnl } : {}), + instrumentId: trade.contractId, symbol: topstepSymbol(trade.contractId), side, quantity, price, - ...(fee > 0 ? { fee } : {}), + ...(fee !== undefined ? { fee } : {}), ...(executedAt ? { executedAt } : {}), }); } @@ -110,7 +132,19 @@ const fetchRaw = async (credentials: Credentials, ctx: FetchContext) => { body: JSON.stringify(body), }); if (!response.ok) rejectResponse("topstep", "Topstep", response); - return (await response.json()) as T; + const result: unknown = await response.json(); + if (typeof result !== "object" || result === null || Array.isArray(result)) { + throw new BrokerRequestError("topstep", `Topstep returned an invalid response for ${path}`); + } + const envelope = result as { success?: boolean; errorMessage?: string | null }; + if (envelope.success === false) { + const message = envelope.errorMessage || `Topstep rejected ${path}`; + // Before authentication, rejection is an auth error; endpoint business + // error codes are endpoint-specific and must not be guessed from text. + if (!token) throw new BrokerAuthError("topstep", message); + throw new BrokerRequestError("topstep", message); + } + return result as T; }; type TopstepLogin = { token?: string; success?: boolean; errorMessage?: string | null }; diff --git a/src/adapters/types.ts b/src/adapters/types.ts index f37bc70..d031666 100644 --- a/src/adapters/types.ts +++ b/src/adapters/types.ts @@ -10,13 +10,19 @@ export type NormalizationContext = { /** Injected IO — lets callers supply a custom fetch (proxies, tests). */ export type FetchContext = { fetch: typeof globalThis.fetch; + /** + * Awaited by supporting adapters immediately after rotation, before further IO. + * Receives rotated fields to merge into existing credentials. Rejection aborts the fetch. + */ + onCredentialsRotated?: (credentials: Credentials) => void | Promise; }; export type AdapterFetchResult = { raw: Raw; /** - * Rotated credentials the caller must persist before the old ones die - * (e.g. Questrade refresh tokens are single-use). + * Rotated credentials the caller must persist. Questrade returns this + * only when no onCredentialsRotated callback is supplied; its old token + * is already invalid after exchange. */ rotatedCredentials?: Credentials; }; diff --git a/src/index.ts b/src/index.ts index 41a6cab..da9d984 100644 --- a/src/index.ts +++ b/src/index.ts @@ -54,7 +54,8 @@ export type ConnectOptions = { /** * Called when a fetch rotates the stored credentials (e.g. Questrade * refresh tokens are single-use). Persist the new value here, or the next - * fetch fails. + * fetch fails. Questrade awaits this immediately after token exchange, + * before fetching accounts or history; rejection aborts the fetch. */ onCredentialsRotated?: (credentials: Credentials) => void | Promise; }; @@ -62,7 +63,7 @@ export type ConnectOptions = { export type BrokerConnection = { readonly broker: BrokerId; readonly label: string; - /** Current credentials — reflects rotation after each fetch. */ + /** Current credentials — reflects rotation even if a later request or persistence fails. */ readonly credentials: Credentials; /** Fetch a fresh normalized snapshot straight from the broker. */ fetchSnapshot: () => Promise; @@ -93,7 +94,14 @@ export const connect = (options: ConnectOptions): BrokerC const createConnection = (adapter: AnyBrokerAdapter, options: ConnectOptions): BrokerConnection => { let credentials: Credentials = { ...(options.credentials as Credentials) }; - const ctx: FetchContext = { fetch: options.fetch ?? globalThis.fetch }; + const onCredentialsRotated = async (next: Credentials): Promise => { + credentials = { ...credentials, ...next }; + await options.onCredentialsRotated?.({ ...credentials }); + }; + const ctx: FetchContext = { + fetch: options.fetch ?? globalThis.fetch, + onCredentialsRotated, + }; const normalization = adapter.id === "ibkr-flex" ? { statementTimeZone: ibkrTimeZone(options.statementTimeZone) } : undefined; @@ -101,8 +109,7 @@ const createConnection = (adapter: AnyBrokerAdapter, options: ConnectOptions): B const fetchSnapshot = async (): Promise => { const result = await adapter.fetchRaw(credentials, ctx); if (result.rotatedCredentials) { - credentials = { ...credentials, ...result.rotatedCredentials }; - await options.onCredentialsRotated?.(credentials); + await onCredentialsRotated(result.rotatedCredentials); } return { broker: adapter.id, diff --git a/src/schema.ts b/src/schema.ts index 53483a7..1cc0f8b 100644 --- a/src/schema.ts +++ b/src/schema.ts @@ -31,15 +31,23 @@ export type Position = { /** One executed trade (a fill), always from the account holder's side. */ export type Trade = { + /** Deterministic numeric tie-break key for equal timestamps; not necessarily chronological. */ + executionTieBreak?: number; + /** Provider-reported realized gross P&L on this fill, in account currency. */ + reportedGrossPnl?: number; + /** Native fill identifier, unique within its broker account. Not an order ID. */ + executionId?: string; + /** Native instrument identifier, when available (for example a futures contract). */ + instrumentId?: string; symbol: string; side: "buy" | "sell"; /** Always positive — `side` carries the direction. */ quantity: number; /** Price per unit in the account's `currency`. */ price: number; - /** Commission/fee paid, when the source reports one. */ + /** Total reported fees and commissions, as a non-negative cost; zero may be explicit. */ fee?: number; - /** ISO 8601 timestamp, when the source carried one. */ + /** ISO 8601 timestamp, when the source carried one; may retain sub-millisecond precision. */ executedAt?: string; }; diff --git a/src/stats.ts b/src/stats.ts index cd9aec7..af25ea3 100644 --- a/src/stats.ts +++ b/src/stats.ts @@ -1,4 +1,5 @@ import type { Account, Position, Trade } from "./schema.js"; +import { compareIsoTimestamps } from "./timestamps.js"; /* The stats engine: FIFO round-trip matching, win rate, per-symbol activity. @@ -43,7 +44,7 @@ export type StatsAccount = Pick & { /** FIFO-match sells against prior buys; returns per-round-trip results. */ export const matchRoundTrips = (trades: Trade[]): { pnl: number }[] => { - const sorted = [...trades].sort((a, b) => (a.executedAt ?? "").localeCompare(b.executedAt ?? "")); + const sorted = [...trades].sort((a, b) => compareIsoTimestamps(a.executedAt, b.executedAt)); const openLots = new Map(); const roundTrips: { pnl: number }[] = []; diff --git a/src/sync/diff.ts b/src/sync/diff.ts index 1866fdb..e6befe9 100644 --- a/src/sync/diff.ts +++ b/src/sync/diff.ts @@ -9,15 +9,24 @@ an already-known broker snapshot. */ import type { BrokerSnapshot, Trade } from "../index.js"; +import { asPreciseIsoTimestamp } from "../timestamps.js"; import type { SyncEvent } from "./events.js"; /** - * Identity of one trade for diffing: symbol + side + quantity + price + - * executedAt. Two fills identical on all five fields collide, so the diff - * counts occurrences per key instead of assuming keys are unique. + * Prefer account-scoped native fill identity; otherwise use the original + * symbol/side/quantity/price/timestamp tuple. Occurrence counting preserves + * duplicate fills even when a provider supplies no unique ID. */ export const tradeKey = (trade: Trade): string => - [trade.symbol, trade.side, String(trade.quantity), String(trade.price), trade.executedAt ?? ""].join("|"); + trade.executionId ? `id:${trade.executionId}` : legacyTradeKey(trade); + +const legacyTradeKey = (trade: Trade, timestamp = trade.executedAt ?? ""): string => + [trade.symbol, trade.side, String(trade.quantity), String(trade.price), timestamp].join("|"); + +const millisecondTradeKey = (trade: Trade): string => { + const timestamp = asPreciseIsoTimestamp(trade.executedAt) ?? trade.executedAt ?? ""; + return legacyTradeKey(trade, timestamp.replace(/(\.\d{3})\d+Z$/, "$1Z")); +}; /** * Diff two snapshots of the same broker into events. Accounts are matched @@ -41,20 +50,40 @@ export const diffSnapshots = (previous: BrokerSnapshot | undefined, next: Broker // Trades: anything in `next` not matched by a remaining occurrence in // `previous` is new. Occurrence counting keeps duplicate identical // fills honest in both directions. - const remaining = new Map(); + const remaining = new Map(); for (const trade of before.trades) { const key = tradeKey(trade); - remaining.set(key, (remaining.get(key) ?? 0) + 1); + const fills = remaining.get(key) ?? []; + fills.push(trade); + remaining.set(key, fills); } + // Reserve exact matches first so a coarse legacy match cannot steal one. + const unmatched: Trade[] = []; for (const trade of account.trades) { - const key = tradeKey(trade); - const count = remaining.get(key) ?? 0; - if (count > 0) { - remaining.set(key, count - 1); + const fills = remaining.get(tradeKey(trade)); + if (fills && fills.length > 0) { + fills.pop(); } else { - events.push({ type: "trade_executed", broker, accountId, at, trade }); + unmatched.push(trade); } } + const legacy = new Map(); + if (broker === "topstep") { + for (const fills of remaining.values()) { + for (const trade of fills) { + // Pre-upgrade snapshots had neither native identity nor full contracts. + if (trade.executionId || trade.instrumentId) continue; + const key = millisecondTradeKey(trade); + legacy.set(key, (legacy.get(key) ?? 0) + 1); + } + } + } + for (const trade of unmatched) { + const key = millisecondTradeKey(trade); + const count = legacy.get(key) ?? 0; + if (count > 0) legacy.set(key, count - 1); + else events.push({ type: "trade_executed", broker, accountId, at, trade }); + } // Balance. if (account.equity !== before.equity) { diff --git a/src/timestamps.ts b/src/timestamps.ts new file mode 100644 index 0000000..e892da6 --- /dev/null +++ b/src/timestamps.ts @@ -0,0 +1,30 @@ +/** Split offset-bearing ISO timestamps before parsing so engines only see milliseconds. */ +const timestampParts = (value: string | undefined | null) => { + if (!value) return undefined; + const match = /^(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2})(?:\.(\d+))?(Z|[+-]\d{2}:?\d{2})$/i.exec(value); + if (!match) return undefined; + const fraction = (match[2] ?? "").padEnd(3, "0"); + const zone = match[3]!.toUpperCase().replace(/([+-]\d{2})(\d{2})$/, "$1:$2"); + const milliseconds = Date.parse(`${match[1]!.toUpperCase()}.${fraction.slice(0, 3)}${zone}`); + if (!Number.isFinite(milliseconds)) return undefined; + return { iso: new Date(milliseconds).toISOString(), fraction, milliseconds }; +}; + +/** Canonical UTC timestamp retaining all source fractional digits (at least three). */ +export const asPreciseIsoTimestamp = (value: string | undefined | null): string | undefined => { + const parts = timestampParts(value); + return parts?.iso.replace(/\.\d{3}Z$/, `.${parts.fraction}Z`); +}; + +/** Compare instants without losing sub-millisecond digits or misordering shorter fractions. */ +export const compareIsoTimestamps = (left: string = "", right: string = ""): number => { + const a = timestampParts(left); + const b = timestampParts(right); + // Preserve the existing ordering of missing or non-ISO timestamps. + if (!a || !b) return left.localeCompare(right); + if (a.milliseconds !== b.milliseconds) return a.milliseconds - b.milliseconds; + const width = Math.max(a.fraction.length, b.fraction.length); + const aFraction = a.fraction.padEnd(width, "0"); + const bFraction = b.fraction.padEnd(width, "0"); + return aFraction < bFraction ? -1 : aFraction > bFraction ? 1 : 0; +}; diff --git a/tests/rotation-persistence.test.ts b/tests/rotation-persistence.test.ts new file mode 100644 index 0000000..3147726 --- /dev/null +++ b/tests/rotation-persistence.test.ts @@ -0,0 +1,168 @@ +import path from "node:path"; +import { describe, expect, it, vi } from "vitest"; + +import { questrade } from "../src/adapters/questrade.js"; +import { connect } from "../src/index.js"; +import { openConnections } from "../src/sync/config.js"; +import { createSyncDaemon } from "../src/sync/daemon.js"; +import { loadState } from "../src/sync/state.js"; +import { makeTmpDir } from "./sync-helpers.js"; + +const tokens = { + access_token: "synthetic-access", + refresh_token: "synthetic-next", + api_server: "https://synthetic.invalid/", +}; + +const accountResponse = (input: Parameters[0]): Response => { + const path = new URL(String(input)).pathname; + if (path === "/v1/accounts") return Response.json({ accounts: [{ number: "TEST-1" }, { number: "TEST-2" }] }); + if (path.endsWith("/balances")) return Response.json({ combinedBalances: [{ currency: "CAD", totalEquity: 100 }] }); + if (path.endsWith("/positions")) return Response.json({ positions: [] }); + if (path.endsWith("/activities")) return Response.json({ activities: [] }); + throw new Error(`Unexpected synthetic request: ${path}`); +}; + +describe("Questrade immediate credential rotation", () => { + it("keeps CLI daemon polls alive in memory, but requires updated credentials on restart", async () => { + const statePath = path.join(await makeTmpDir(), "state.json"); + const suppliedTokens: (string | null)[] = []; + let currentToken = "synthetic-old"; + let exchanges = 0; + const network = vi.spyOn(globalThis, "fetch").mockImplementation(async (input) => { + const url = new URL(String(input)); + if (url.pathname.endsWith("oauth2/token")) { + const supplied = url.searchParams.get("refresh_token"); + suppliedTokens.push(supplied); + if (supplied !== currentToken) return Response.json({}, { status: 401 }); + currentToken = `synthetic-next-${++exchanges}`; + return Response.json({ ...tokens, refresh_token: currentToken }); + } + return accountResponse(input); + }); + try { + const specs = [{ broker: "questrade" as const, credentials: { refreshToken: "synthetic-old" } }]; + const daemon = createSyncDaemon({ + connections: openConnections(specs), sinks: [], statePath, intervalSeconds: 60, + }); + expect((await daemon.sweep()).failures).toEqual([]); + expect((await daemon.sweep()).failures).toEqual([]); + expect(suppliedTokens).toEqual(["synthetic-old", "synthetic-next-1"]); + expect(JSON.stringify(await loadState(statePath))).not.toContain("synthetic-next"); + + const restarted = createSyncDaemon({ + connections: openConnections(specs), sinks: [], statePath, intervalSeconds: 60, + }); + expect((await restarted.sweep()).failures).toEqual([ + expect.objectContaining({ broker: "questrade" }), + ]); + } finally { + network.mockRestore(); + } + }); + + it("persists the replacement before a later history request fails and reuses it on retry", async () => { + const events: string[] = []; + const exchangedTokens: (string | null)[] = []; + const persist = vi.fn(() => { events.push("persist"); }); + const connection = connect({ + broker: "questrade", + credentials: { refreshToken: "synthetic-old" }, + onCredentialsRotated: persist, + fetch: async (input) => { + const url = new URL(String(input)); + if (url.pathname.endsWith("oauth2/token")) { + exchangedTokens.push(url.searchParams.get("refresh_token")); + events.push("exchange"); + return Response.json(tokens); + } + events.push("request"); + if (url.pathname.endsWith("/activities")) return Response.json({}, { status: 503 }); + return accountResponse(input); + }, + }); + await expect(connection.fetchSnapshot()).rejects.toThrow("Questrade rejected the request (503)"); + expect(events.slice(0, 3)).toEqual(["exchange", "persist", "request"]); + expect(persist).toHaveBeenCalledWith({ refreshToken: tokens.refresh_token }); + expect(persist).toHaveBeenCalledTimes(1); + expect(connection.credentials).toEqual({ refreshToken: tokens.refresh_token }); + await expect(connection.fetchSnapshot()).rejects.toThrow("Questrade rejected the request (503)"); + expect(exchangedTokens).toEqual(["synthetic-old", "synthetic-next"]); + }); + + it("awaits asynchronous persistence before requesting accounts and imports all returned accounts", async () => { + let finishPersistence!: () => void; + let startedPersistence!: () => void; + const started = new Promise((resolve) => { startedPersistence = resolve; }); + const pending = new Promise((resolve) => { finishPersistence = resolve; }); + const network = vi.fn(async (input) => String(input).includes("oauth2/token") + ? Response.json(tokens) + : accountResponse(input)); + const persist = vi.fn(async () => { + startedPersistence(); + await pending; + }); + const connection = connect({ + broker: "questrade", + credentials: { refreshToken: "synthetic-old" }, + onCredentialsRotated: persist, + fetch: network, + }); + const fetching = connection.fetchSnapshot(); + await started; + expect(connection.credentials).toEqual({ refreshToken: tokens.refresh_token }); + expect(network).toHaveBeenCalledTimes(1); + finishPersistence(); + const snapshot = await fetching; + expect(snapshot.accounts.map((account) => account.id)).toEqual(["TEST-1", "TEST-2"]); + expect(network).toHaveBeenCalledTimes(8); + expect(persist).toHaveBeenCalledTimes(1); + }); + + it.each([false, true])("aborts further requests when persistence rejects (async: %s)", async (asynchronous) => { + const network = vi.fn(async () => Response.json(tokens)); + const connection = connect({ + broker: "questrade", + credentials: { refreshToken: "synthetic-old" }, + onCredentialsRotated: () => { + const error = new Error("Persistence failed"); + if (asynchronous) return Promise.reject(error); + throw error; + }, + fetch: network, + }); + await expect(connection.fetchSnapshot()).rejects.toThrow("Persistence failed"); + expect(connection.credentials).toEqual({ refreshToken: tokens.refresh_token }); + expect(network).toHaveBeenCalledTimes(1); + }); + + it("updates in-memory credentials even without an application persistence callback", async () => { + const connection = connect({ + broker: "questrade", + credentials: { refreshToken: "synthetic-old" }, + fetch: async (input) => String(input).includes("oauth2/token") + ? Response.json(tokens) + : Response.json({}, { status: 503 }), + }); + await expect(connection.fetchSnapshot()).rejects.toThrow("Questrade rejected the request (503)"); + expect(connection.credentials).toEqual({ refreshToken: tokens.refresh_token }); + }); + + it("returns fallback rotatedCredentials to direct adapter consumers without a callback", async () => { + const result = await questrade.fetchRaw({ refreshToken: "synthetic-old" }, { + fetch: async (input) => String(input).includes("oauth2/token") ? Response.json(tokens) : accountResponse(input), + }); + expect(result.rotatedCredentials).toEqual({ refreshToken: tokens.refresh_token }); + expect(questrade.normalize(result.raw).map((account) => account.id)).toEqual(["TEST-1", "TEST-2"]); + }); + + it("does not return rotation twice when direct adapter consumers supply a callback", async () => { + const persist = vi.fn(); + const result = await questrade.fetchRaw({ refreshToken: "synthetic-old" }, { + fetch: async (input) => String(input).includes("oauth2/token") ? Response.json(tokens) : accountResponse(input), + onCredentialsRotated: persist, + }); + expect(persist).toHaveBeenCalledExactlyOnceWith({ refreshToken: tokens.refresh_token }); + expect(result).not.toHaveProperty("rotatedCredentials"); + }); +}); diff --git a/tests/stats.test.ts b/tests/stats.test.ts index 026f1f2..a35be08 100644 --- a/tests/stats.test.ts +++ b/tests/stats.test.ts @@ -1,7 +1,8 @@ import { describe, expect, it } from "vitest"; import type { Trade } from "../src/schema.js"; -import { computeStats, positionsFromTrades } from "../src/stats.js"; +import { computeStats, matchRoundTrips, positionsFromTrades } from "../src/stats.js"; +import { compareIsoTimestamps } from "../src/timestamps.js"; const trade = (overrides: Partial): Trade => ({ symbol: "BTC", @@ -14,6 +15,26 @@ const trade = (overrides: Partial): Trade => ({ const statsFor = (trades: Trade[]) => computeStats([{ broker: "csv", equity: 0, positions: [], trades }]); describe("the trading stats derived from trades", () => { + it.each([ + ["2026-10-06T14:00:00.1234Z", "2026-10-06T14:00:00.123456Z"], + ["2026-10-06T14:00:00Z", "2026-10-06T14:00:00.000007Z"], + ["2026-10-06T14:00:00.123456789001Z", "2026-10-06T14:00:00.123456789002Z"], + ["2026-10-06T08:00:00.1234-06:00", "2026-10-06T14:00:00.123456Z"], + ])("matches mixed-precision fills in chronological order (%s, %s)", (buyAt, sellAt) => { + const buy = trade({ side: "buy", price: 100, executedAt: buyAt }); + const sell = trade({ side: "sell", price: 150, executedAt: sellAt }); + expect(matchRoundTrips([sell, buy])).toEqual([{ pnl: 50 }]); + expect(matchRoundTrips([buy, sell])).toEqual([{ pnl: 50 }]); + }); + + it("treats trailing fractional zeros as the same instant and preserves input order for ties", () => { + expect(compareIsoTimestamps("2026-10-06T14:00:00.1234Z", "2026-10-06T14:00:00.123400Z")).toBe(0); + expect(matchRoundTrips([ + trade({ side: "buy", price: 100, executedAt: "2026-10-06T14:00:00.1234Z" }), + trade({ side: "sell", price: 150, executedAt: "2026-10-06T14:00:00.123400Z" }), + ])).toEqual([{ pnl: 50 }]); + }); + it("count a completed buy-then-sell round trip as one win when it closed higher", () => { const stats = statsFor([ trade({ side: "buy", price: 100, executedAt: "2026-01-01T00:00:00Z" }), diff --git a/tests/sync-diff.test.ts b/tests/sync-diff.test.ts index 9575af8..2737718 100644 --- a/tests/sync-diff.test.ts +++ b/tests/sync-diff.test.ts @@ -10,6 +10,12 @@ import type { import { makeAccount, makePosition, makeSnapshot, makeTrade } from "./sync-helpers.js"; describe("tradeKey", () => { + it("prefers native fill IDs even when timestamp formatting or other fields change", () => { + expect(tradeKey(makeTrade({ executionId: "synthetic-1" }))).toBe("id:synthetic-1"); + expect(tradeKey(makeTrade({ executionId: "synthetic-1", price: 201 }))).toBe("id:synthetic-1"); + expect(tradeKey(makeTrade({ executionId: "synthetic-2" }))).not.toBe("id:synthetic-1"); + }); + it("keys on symbol, side, quantity, price, and executedAt", () => { const trade = makeTrade(); expect(tradeKey(trade)).toBe("AAPL|buy|10|200|2026-08-28T10:00:00.000Z"); @@ -32,6 +38,67 @@ describe("tradeKey", () => { }); describe("diffSnapshots", () => { + const topstepSnapshot = (trades: ReturnType[]) => makeSnapshot({ + broker: "topstep", + accounts: [makeAccount({ trades })], + }); + + it("does not replay an unchanged Topstep fill when upgrading a millisecond-only snapshot", () => { + const oldFill = makeTrade({ executedAt: "2026-10-06T14:45:36.246Z" }); + const preciseFill = { ...oldFill, executionId: "100", instrumentId: "CON.F.US.EP.U26", + executedAt: "2026-10-06T14:45:36.246377Z" }; + const upgraded = topstepSnapshot([preciseFill]); + expect(diffSnapshots(topstepSnapshot([oldFill]), upgraded)).toEqual([]); + expect(diffSnapshots(upgraded, upgraded)).toEqual([]); + }); + + it("preserves occurrence counts during the precision/identity transition", () => { + const oldFill = makeTrade({ executedAt: "2026-10-06T14:45:36.246Z" }); + const fills = ["100", "101", "102"].map((executionId) => ({ + ...oldFill, executionId, instrumentId: "CON.F.US.EP.U26", executedAt: "2026-10-06T14:45:36.246377Z", + })); + const events = diffSnapshots(topstepSnapshot([oldFill, { ...oldFill }]), topstepSnapshot(fills)); + expect(events).toHaveLength(1); + expect(events[0]).toMatchObject({ type: "trade_executed", trade: { executionId: "102" } }); + }); + + it("reserves exact matches before consuming coarse legacy occurrences", () => { + const oldFill = makeTrade({ executedAt: "2026-10-06T14:45:36.246Z" }); + const newFill = { ...oldFill, executionId: "new", instrumentId: "CON.F.US.EP.U26", + executedAt: "2026-10-06T14:45:36.246377Z" }; + const events = diffSnapshots(topstepSnapshot([oldFill]), topstepSnapshot([newFill, oldFill])); + expect(events).toHaveLength(1); + expect(events[0]).toMatchObject({ type: "trade_executed", trade: { executionId: "new" } }); + }); + + it("does not merge different native IDs within the same millisecond", () => { + const base = makeTrade({ instrumentId: "CON.F.US.EP.U26", executedAt: "2026-10-06T14:45:36.246377Z" }); + const first = { ...base, executionId: "100" }; + const second = { ...base, executionId: "101" }; + const events = diffSnapshots(topstepSnapshot([first]), topstepSnapshot([second])); + expect(events).toHaveLength(1); + expect(events[0]).toMatchObject({ type: "trade_executed", trade: { executionId: "101" } }); + }); + + it("does not match IDs across accounts", () => { + const fill = makeTrade({ executionId: "100" }); + const previous = makeSnapshot({ accounts: [ + makeAccount({ id: "one", trades: [fill] }), makeAccount({ id: "two" }), + ] }); + const next = makeSnapshot({ accounts: [ + makeAccount({ id: "one", trades: [fill] }), makeAccount({ id: "two", trades: [fill] }), + ] }); + expect(diffSnapshots(previous, next)).toEqual([ + expect.objectContaining({ type: "trade_executed", accountId: "two" }), + ]); + }); + + it("limits the coarse transition fallback to Topstep", () => { + const previous = makeSnapshot({ accounts: [makeAccount({ trades: [makeTrade()] })] }); + const next = makeSnapshot({ accounts: [makeAccount({ trades: [makeTrade({ executionId: "100" })] })] }); + expect(diffSnapshots(previous, next)).toHaveLength(1); + }); + it("emits nothing for the first-ever snapshot (baseline only)", () => { const next = makeSnapshot({ accounts: [ diff --git a/tests/topstep-identity.test.ts b/tests/topstep-identity.test.ts new file mode 100644 index 0000000..df0a637 --- /dev/null +++ b/tests/topstep-identity.test.ts @@ -0,0 +1,235 @@ +import { describe, expect, it, vi } from "vitest"; + +import { asPreciseIsoTimestamp } from "../src/timestamps.js"; +import { parseTopstepTrades } from "../src/adapters/topstep.js"; +import { BrokerAuthError, connect } from "../src/index.js"; + +const fill = { + contractId: "CON.F.US.MNQ.Z26", + side: 1, + size: 2, + price: 20000, + creationTimestamp: "2026-10-06T14:00:00Z", +}; + +describe("Topstep execution identity", () => { + it("preserves three separate fills inside one millisecond without reordering them", () => { + const rows = ["246377", "246373", "246367"].map((fraction, index) => ({ + ...fill, + id: 100 + index, + creationTimestamp: `2026-10-06T14:45:36.${fraction}+00:00`, + })); + const trades = parseTopstepTrades(rows); + expect(trades.map((trade) => trade.executionId)).toEqual(["100", "101", "102"]); + expect(trades.map((trade) => trade.executionTieBreak)).toEqual([100, 101, 102]); + expect(trades.map((trade) => trade.executedAt)).toEqual( + rows.map((row) => row.creationTimestamp.replace("+00:00", "Z")), + ); + expect(trades.every((trade) => trade.instrumentId === fill.contractId)).toBe(true); + }); + + it.each([0, 42, Number.MAX_SAFE_INTEGER])("uses safe numeric ID %s as identity and tie-break key", (id) => { + expect(parseTopstepTrades([{ ...fill, id }])[0]).toMatchObject({ + executionId: String(id), + executionTieBreak: id, + }); + }); + + it.each(["native-fill", "00042", "9007199254740993", -1])("preserves ID %s without inferring an order key", (id) => { + const trade = parseTopstepTrades([{ ...fill, id }])[0]; + expect(trade?.executionId).toBe(String(id)); + expect(trade).not.toHaveProperty("executionTieBreak"); + }); + + it.each([undefined, "", 1.5, Number.MAX_SAFE_INTEGER + 1, NaN, Infinity])( + "omits unusable ID %s without dropping the fill", + (id) => { + const row = { ...fill, ...(id !== undefined ? { id } : {}), orderId: 7 }; + const trades = parseTopstepTrades([row]); + expect(trades).toHaveLength(1); + expect(trades[0]).not.toHaveProperty("executionId"); + expect(trades[0]).not.toHaveProperty("executionTieBreak"); + }, + ); + + it("keeps separate contract identities when display symbols match", () => { + const trades = parseTopstepTrades([fill, { ...fill, contractId: "CON.F.US.MNQ.H27" }]); + expect(trades.map((trade) => trade.symbol)).toEqual(["MNQ", "MNQ"]); + expect(trades.map((trade) => trade.instrumentId)).toEqual([fill.contractId, "CON.F.US.MNQ.H27"]); + }); + + it("still filters voided and unreadable executions", () => { + expect(parseTopstepTrades([ + { ...fill, voided: true }, + { ...fill, side: 9 }, + { ...fill, size: 0 }, + { ...fill, price: 0 }, + { ...fill, contractId: "" }, + ])).toEqual([]); + }); +}); + +describe("precise UTC timestamps", () => { + it("never passes extended fractions to the engine's date parser", () => { + const parse = Date.parse; + const parser = vi.spyOn(Date, "parse").mockImplementation((value) => + /\.\d{4,}/.test(value) ? NaN : parse(value)); + try { + expect(asPreciseIsoTimestamp("2026-10-06T08:45:36.123456789012-0600")) + .toBe("2026-10-06T14:45:36.123456789012Z"); + expect(parser).toHaveBeenCalledExactlyOnceWith("2026-10-06T08:45:36.123-06:00"); + } finally { + parser.mockRestore(); + } + }); + + it.each([ + ["2026-10-06T08:45:36.000007-06:00", "2026-10-06T14:45:36.000007Z"], + ["2026-10-06T00:15:36.1234567+01:30", "2026-10-05T22:45:36.1234567Z"], + ["2026-10-06T08:45:36.987654321-0600", "2026-10-06T14:45:36.987654321Z"], + ["2026-10-06T14:45:36.000000Z", "2026-10-06T14:45:36.000000Z"], + ["2026-10-06T14:45:36.12Z", "2026-10-06T14:45:36.120Z"], + ["2026-10-06T14:45:36Z", "2026-10-06T14:45:36.000Z"], + ])("normalizes %s without losing source precision", (source, expected) => { + expect(asPreciseIsoTimestamp(source)).toBe(expected); + }); + + it.each([undefined, null, "", "not a timestamp", "2026-10-06T25:00:00.123456Z", "2026-10-06T14:45:36.123456"])( + "omits invalid, missing or offset-less timestamp %s", + (value) => expect(asPreciseIsoTimestamp(value)).toBeUndefined(), + ); +}); + +describe("Topstep reported costs and P&L", () => { + it("adds fees and commissions and preserves reported gross P&L", () => { + expect(parseTopstepTrades([{ ...fill, fees: 0.72, commissions: 0.5, profitAndLoss: 40 }])[0]) + .toMatchObject({ fee: 1.22, reportedGrossPnl: 40 }); + }); + + it("preserves explicit zero costs and zero gross P&L", () => { + expect(parseTopstepTrades([{ ...fill, fees: 0, commissions: 0, profitAndLoss: 0 }])[0]) + .toMatchObject({ fee: 0, reportedGrossPnl: 0 }); + }); + + it.each([ + [{ fees: 0.72 }, 0.72], + [{ commissions: 0.5 }, 0.5], + [{ fees: 0 }, 0], + [{ commissions: 0 }, 0], + [{ fees: 0.72, commissions: 0.5 }, 1.22], + [{ fees: NaN, commissions: 0.5 }, 0.5], + ])("uses reported cost components %j", (costs, expected) => { + expect(parseTopstepTrades([{ ...fill, ...costs }])[0]?.fee).toBe(expected); + }); + + it.each([ + { fees: -0.72 }, + { commissions: -0.5 }, + { fees: 0.72, commissions: -0.5 }, + { fees: -0.72, commissions: 0.5 }, + ])("omits costs with unexplained negative components %j", (costs) => { + expect(parseTopstepTrades([{ ...fill, ...costs }])[0]).not.toHaveProperty("fee"); + }); + + it("omits unknown or non-finite costs and null gross P&L", () => { + for (const costs of [{}, { fees: NaN, commissions: Infinity }]) { + const trade = parseTopstepTrades([{ ...fill, ...costs, profitAndLoss: null }])[0]; + expect(trade).not.toHaveProperty("fee"); + expect(trade).not.toHaveProperty("reportedGrossPnl"); + } + }); + + it("preserves losses and omits non-finite gross P&L", () => { + expect(parseTopstepTrades([{ ...fill, profitAndLoss: -40 }])[0]?.reportedGrossPnl).toBe(-40); + expect(parseTopstepTrades([{ ...fill, profitAndLoss: NaN }])[0]).not.toHaveProperty("reportedGrossPnl"); + }); +}); + +describe("Topstep fetch responses", () => { + it.each([null, [], 42, "invalid"])("rejects malformed HTTP-200 response %j with a typed error", async (body) => { + const connection = connect({ + broker: "topstep", + credentials: { userName: "synthetic", apiKey: "synthetic" }, + fetch: async () => Response.json(body), + }); + await expect(connection.fetchSnapshot()).rejects.toMatchObject({ + name: "BrokerRequestError", + message: "Topstep returned an invalid response for /Auth/loginKey", + }); + }); + + it("classifies HTTP authentication failures without guessing endpoint business error codes", async () => { + const connection = connect({ + broker: "topstep", + credentials: { userName: "synthetic", apiKey: "synthetic" }, + fetch: async (input) => String(input).endsWith("loginKey") + ? Response.json({ success: true, token: "synthetic" }) + : Response.json({}, { status: 401 }), + }); + await expect(connection.fetchSnapshot()).rejects.toThrow(BrokerAuthError); + }); + + it.each(["/Account/search", "/Position/searchOpen", "/Trade/search"])( + "rejects HTTP 200 with success false from %s", + async (failedPath) => { + const connection = connect({ + broker: "topstep", + credentials: { userName: "synthetic", apiKey: "synthetic" }, + fetch: async (input) => { + const path = new URL(String(input)).pathname.replace(/^\/api/, ""); + if (path === failedPath) return Response.json({ success: false, errorMessage: "History unavailable" }); + if (path === "/Auth/loginKey") return Response.json({ success: true, token: "synthetic" }); + if (path === "/Account/search") return Response.json({ success: true, accounts: [{ id: 1 }] }); + return Response.json({ success: true, positions: [], trades: [] }); + }, + }); + await expect(connection.fetchSnapshot()).rejects.toMatchObject({ + name: "BrokerRequestError", + message: "History unavailable", + }); + }, + ); + + it("uses a useful fallback when the provider omits its error message", async () => { + const connection = connect({ + broker: "topstep", + credentials: { userName: "synthetic", apiKey: "synthetic" }, + fetch: async (input) => Response.json(String(input).endsWith("loginKey") + ? { success: true, token: "synthetic" } + : { success: false, errorMessage: null }), + }); + await expect(connection.fetchSnapshot()).rejects.toThrow("Topstep rejected /Account/search"); + }); + + it("keeps failed login responses as authentication errors", async () => { + const connection = connect({ + broker: "topstep", + credentials: { userName: "synthetic", apiKey: "synthetic" }, + fetch: async () => Response.json({ success: false, token: "unusable", errorMessage: "Invalid key" }), + }); + await expect(connection.fetchSnapshot()).rejects.toThrow(BrokerAuthError); + }); + + it("fetches every returned account and preserves its identity", async () => { + const requestedAccounts: number[] = []; + const connection = connect({ + broker: "topstep", + credentials: { userName: "synthetic", apiKey: "synthetic" }, + fetch: async (input, init) => { + const path = new URL(String(input)).pathname; + if (path.endsWith("loginKey")) return Response.json({ success: true, token: "synthetic" }); + if (path.endsWith("Account/search")) return Response.json({ success: true, accounts: [{ id: 1 }, { id: 2 }] }); + if (path.endsWith("Trade/search")) { + const { accountId } = JSON.parse(String(init?.body)) as { accountId: number }; + requestedAccounts.push(accountId); + return Response.json({ success: true, trades: [{ ...fill, id: accountId }] }); + } + return Response.json({ success: true, positions: [] }); + }, + }); + const snapshot = await connection.fetchSnapshot(); + expect(requestedAccounts).toEqual([1, 2]); + expect(snapshot.accounts.map((account) => account.id)).toEqual(["1", "2"]); + expect(snapshot.accounts.map((account) => account.trades[0]?.executionId)).toEqual(["1", "2"]); + }); +});