Skip to content

fix(sync): import broker trade history without phantom positions - #29

Draft
anthony-som wants to merge 1 commit into
LuxAlgo:mainfrom
anthony-som:fix/webull-sync-history
Draft

anthony-som wants to merge 1 commit into
LuxAlgo:mainfrom
anthony-som:fix/webull-sync-history

Conversation

@anthony-som

Copy link
Copy Markdown

Fixes #28. Draft: blocked on LuxAlgo/broker-sdk#19. This PR uses fields that don't exist in @luxalgo/broker-sdk 0.5.1: Trade.assetClass, Trade.multiplier, Trade.positionEffect and ConnectOptions.historySince. Typecheck fails until the SDK release with #19 lands and the dependency is bumped. I'll push the version bump once that release is out.

Summary

Per CONTRIBUTING, broker connectivity stays in the SDK: the Webull route, equity and history fixes are in broker-sdk#19. This PR is only the journal side, meaning what to do with a broker's trade history once it arrives.

apps/web/src/server/sync-history.ts (new, pure)

  • orphanFills(known, incoming, brokerQuantity) finds incoming fills that close a position opened before the broker's history floor, which would otherwise show up as a phantom position:
    • A fill the broker labels as a close (positionEffect: "close") with nothing open on the other side is dropped.
    • Unlabeled equity fills: if replaying known and incoming fills ends on a position the broker's list doesn't report, the incoming fill that opened it from flat is dropped, earliest first, until the replay agrees.
    • Brokers without a trustworthy position list (brokerQuantity returns undefined) keep every unlabeled fill. Behavior is unchanged for every broker except Webull.
  • expiredOptionCloses(optionFills, now) closes expired option positions (OCC-style XYZ 260925P50 symbols) at 4pm New York on the expiry date. The DST-aware time is 20:00Z in summer and 21:00Z in winter, and it stays fixed so the synthetic fill hashes the same on every sync.

apps/web/src/server/sync.ts

  • Passes historySince, the last sync minus a 30-day overlap, once an account has synced fills. The overlap catches orders placed earlier that filled since, in case the broker filters by placement time. Existing content-hash dedupe absorbs the refetches.
  • Carries broker-stated assetClass onto executions. The SDK's "cash" has no journal equivalent, so it's left off.
  • Saves broker-stated option multipliers to multipliers settings without overwriting a value the user set, so option P&L is ×100 instead of ×1.
  • Webull only: drops orphan fills before insert and reports them in skippedReasons. Duplicates still pass through, so dedupe counts stay accurate; an earlier draft broke four IBKR tests on exactly that. It also inserts expired-option closes at $0 with preserveFees, so default fee rules don't charge commission on an expiry.

apps/web/tests/webull-sync.test.ts (new)

Product statements, following CONTRIBUTING:

  • selling shares bought before the history floor does not invent a short
  • a real short the broker still reports is kept
  • a short opened and covered inside the history is kept
  • a position held since before the floor and never traded adds nothing
  • an option close with nothing open is dropped, and one closing a known open is kept
  • brokers without a position list keep every unlabeled fill
  • an expired option still held closes at the 4pm New York expiry, in summer and winter
  • an option closed before expiry gets no extra fill

Test plan

  • pnpm typecheck passes, with a local build of broker-sdk#19 linked through pnpm patch. That patch isn't part of this PR.
  • pnpm test: 54 files / 540 tests pass, including the IBKR timezone, broker-connect and sync-resilience suites.
  • prettier --check on the changed files.
  • Live on a real Webull US account (about 1,200 fills over ~10 months):
    • trades, dashboard, calendar and reports populate;
    • 12 expired options that were stuck open are closed;
    • open positions match Webull's list, except stocks held since before the history floor;
    • the phantom short from shares bought before the floor goes away;
    • repeat sync takes about 1s, versus about 75s for the first.
  • After the SDK release, bump @luxalgo/broker-sdk and re-run pnpm typecheck in CI.

Known limits (called out in code with ponytail: notes)

  • An in-the-money option that was exercised or assigned is also closed at $0. Telling them apart needs the underlying's price at expiry.
  • A close that only partly overshoots the known position is kept whole, because splitting it would change its dedupe hash between syncs.
  • Stocks bought before the floor and never traded since don't appear as trades. A Webull CSV import covering the earlier period fills that gap.

Review

A second Claude Code session reviewed an earlier draft. It found one high issue (expiry closes on truncated history booking fake P&L) and four medium issues (default fees on synthetic closes, partial-fill double counting, the since-overlap window, and first-sync failure on empty history). All are addressed here or in broker-sdk#19.

🤖 Generated with Claude Code

Broker trade history has a floor (Webull keeps roughly the last year),
so a sync can see fills that close positions opened before it. Those
showed up as positions that never existed, e.g. a phantom short after
selling shares bought earlier.

- Skip fills that close a position opened before the history floor:
  broker-labelled closes with nothing open, and, for Webull (whose
  position list is complete), unlabeled equity fills that leave the
  replay disagreeing with the broker's position.
- Close expired Webull options at 0 at the 4pm New York expiry, since
  Webull records no order for an expiry. No default fee rules apply.
- Save broker-stated option contract multipliers, keeping user overrides.
- Pass the SDK's historySince (last sync minus a 30-day overlap) so repeat
  syncs fetch only recent orders; content-hash dedupe absorbs refetches.
- Carry broker-stated asset classes onto synced executions.

Requires @luxalgo/broker-sdk with Trade.assetClass/multiplier/
positionEffect and historySince (LuxAlgo/broker-sdk#19).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Webull sync: connection fails, no trades import, and synced history creates phantom positions

1 participant