Add documented paper fill-pricing model: fees, funding, liquidation estimate, provenance (PAPER-REALISM-001) - #63
Conversation
…stimate, provenance (PAPER-REALISM-001) Issue #39: execution/paperEngine.ts already enforced real risk limits (kill switch, position/leverage/loss limits) but paper fills didn't model simulated fees/funding, a liquidation estimate, or record price provenance -- undermining the credibility of any backtest-vs-live-practice comparison the product wants to offer. - execution/fillModel.ts (new): versioned (FILL_MODEL_VERSION) pure functions -- computeFee, estimateLiquidationPrice (standard isolated- margin formula with a documented flat maintenance-margin-ratio simplification), computeFundingCost (pro-rated by elapsed time relative to Hyperliquid's real hourly funding interval, standard perp convention). - fills table: every fill now records priceSource, sourceTimestamp, fillModelVersion, referencePrice, slippageAmount, feeAmount, marketType, simulated. The six provenance fields are nullable -- fills predating this feature never had them computed, and backfilling a synthetic value would fabricate evidence that doesn't exist (same reasoning signals.signal_score already follows for SIGNAL-SCORE-001; the shared dev DB had pre-existing fills rows confirming this wasn't hypothetical). - positions table: leverage (quantity-weighted-averaged the same way entryPrice is), liquidationPriceEstimate, feesPaid, fundingPaid, lastFundingChargedAt. closePosition now settles accumulated fees (including a real exit fee) and funding into realizedPnl -- which flows straight into analytics/'s existing DATA-015 sample-size-tiered performance metrics with no changes needed there. - accruePaperFunding (new, run every 5 minutes from server.ts): charges open positions real Hyperliquid funding via fetchFundingHistory -- deliberately NOT getFundingRate, whose type: 'fundingRate' endpoint was verified directly against live Hyperliquid mainnet during implementation to currently return a real HTTP 422 (a discrepancy an earlier issue's audit had flagged but left unfixed as out of scope). Building this issue's real, recurring cost calculation on the broken endpoint would have meant funding silently never accruing in practice -- confirmed fetchFundingHistory (already used successfully by BACKTEST-001) works correctly instead. - docs/architecture/paper-execution.md documents every assumption: perp-only (no spot ingestion exists), reduce-only out of scope (not represented anywhere in this codebase's order model), the fee/funding/ liquidation-estimate formulas, and what's structurally out of scope. Verified end-to-end against live Hyperliquid mainnet and a real Postgres DB, not just unit tests: submitted a real order and confirmed the fill recorded genuine provenance (real slippage/fee amounts matching hand computation), confirmed the resulting position's leverage/liquidation- estimate/feesPaid, closed it and confirmed realizedPnl correctly nets out entry+exit fees, confirmed GET /api/analytics/performance still works correctly (insufficient tier at sample size 1, no regression), and ran accruePaperFunding directly against real open positions -- discovering and fixing the getFundingRate/fetchFundingHistory issue in the process. Full suite: server 217/217 (211 baseline + fillModel + new paperEngine coverage), client 35/35. Both builds clean.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 16217fa1e4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| } | ||
|
|
||
| const notional = parseFloat(position.entryPrice) * parseFloat(position.quantity); | ||
| const cost = computeFundingCost(notional, fundingRate, position.side, elapsedMs); |
There was a problem hiding this comment.
When an accrual is delayed by an API failure, restart, or pause longer than one funding interval, elapsedMs still spans the entire gap but fundingRate is only the newest entry from the six-hour query. The next successful cycle therefore applies one recent rate retroactively across every missed interval, potentially with the wrong magnitude or sign; either integrate the returned history over the elapsed period or advance the checkpoint when an interval is deliberately skipped.
Useful? React with 👍 / 👎.
| const FUNDING_LOOKBACK_MS = 6 * 60 * 60_000; | ||
|
|
||
| export async function accruePaperFunding(now: Date = new Date()): Promise<void> { | ||
| if (env.EXECUTION_MODE !== 'paper' || isGloballyHalted()) return; |
There was a problem hiding this comment.
If GLOBAL_KILL_SWITCH is enabled while positions remain open, this return suppresses all passive funding accrual even though closePosition remains usable during the halt. A user who closes before the switch is cleared consequently settles a fundingPaid value that omits the halted period; the kill switch should block new trading, not stop accounting on existing positions.
Useful? React with 👍 / 👎.
| lastFundingChargedAt: now, | ||
| updatedAt: now, | ||
| }) | ||
| .where(eq(positions.id, position.id)); |
There was a problem hiding this comment.
When a user closes a position after the accrual query selects it but while fetchFundingHistory is pending, closePosition computes realized P&L from the old funding total and marks the row closed, after which this ID-only update still writes a new funding charge. The stored fundingPaid then disagrees with realizedPnl; closing and accrual need transactional coordination or a status-guarded update with the final charge settled by the close path.
Useful? React with 👍 / 👎.
| ALTER TABLE "fills" ADD COLUMN "fee_amount" numeric;--> statement-breakpoint | ||
| ALTER TABLE "fills" ADD COLUMN "market_type" varchar(10) DEFAULT 'perp' NOT NULL;--> statement-breakpoint | ||
| ALTER TABLE "fills" ADD COLUMN "simulated" boolean DEFAULT true NOT NULL;--> statement-breakpoint | ||
| ALTER TABLE "positions" ADD COLUMN "leverage" numeric DEFAULT '1' NOT NULL;--> statement-breakpoint |
There was a problem hiding this comment.
For every position created before this migration, the new non-null column is populated as leverage 1 regardless of the leverage recorded on its originating orders. Those still-open positions are immediately exposed through the API with false leverage, and a later same-direction fill uses that fabricated value to calculate the liquidation estimate; existing positions should be backfilled from their fills/orders or represented as unknown until they can be derived.
Useful? React with 👍 / 👎.
|
@copilot resolve the merge conflicts on this branch. |
# Conflicts: # server/src/execution/paperEngine.test.ts
…esolve conflicts)
Done. Merged
All 221 server tests pass. |
There was a problem hiding this comment.
Pull request overview
This PR implements a more realistic, provenance-rich paper fill model to support credible paper/backtest comparisons (PAPER-REALISM-001 / #39), adding simulated fees, funding accrual, and a liquidation-price estimate while persisting detailed pricing provenance on fills/positions.
Changes:
- Introduces a versioned paper fill model (
fillModel.ts) for fee, funding, and liquidation-estimate calculations, with unit tests. - Extends paper execution to persist fill provenance fields and to track fees/leverage/liquidation estimate/funding at the position level; updates realized P&L to net out fees/funding.
- Adds a periodic funding accrual loop (every 5 minutes) and expands DB schema/migrations + docs/client types to reflect the new fields.
Reviewed changes
Copilot reviewed 12 out of 13 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| server/src/server.ts | Schedules periodic paper funding accrual. |
| server/src/execution/paperEngine.ts | Records fill provenance, tracks position fees/leverage/liquidation estimate, settles fees/funding into realized P&L, and accrues funding periodically. |
| server/src/execution/paperEngine.test.ts | Adds coverage for fill provenance, realized P&L netting, and funding accrual behavior. |
| server/src/execution/fillModel.ts | Adds versioned pure functions for fee, funding cost, and liquidation estimate. |
| server/src/execution/fillModel.test.ts | Unit tests for fill-model formulas and proration behavior. |
| server/src/db/schema.ts | Adds new nullable fill provenance columns and new position tracking columns (fees/funding/leverage/liquidation estimate). |
| server/drizzle/0009_wide_korath.sql | Migration adding the new fills/positions columns. |
| server/drizzle/meta/0009_snapshot.json | Drizzle snapshot update for the migration. |
| server/drizzle/meta/_journal.json | Drizzle journal entry for migration 0009. |
| docs/architecture/paper-execution.md | Documents assumptions and scope for the new paper realism model. |
| docs/architecture/current-state.md | Notes the new paper fill model and funding accrual in the architecture summary. |
| client/src/features/execution/types.ts | Updates client types to include fill provenance and new position accounting fields. |
Files not reviewed (1)
- server/package-lock.json: Generated file
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| export async function accruePaperFunding(now: Date = new Date()): Promise<void> { | ||
| if (env.EXECUTION_MODE !== 'paper' || isGloballyHalted()) return; | ||
|
|
||
| const openPositions = await db.select().from(positions).where(eq(positions.status, 'OPEN')); |
| let fundingRate: number; | ||
| try { | ||
| const history = await fetchFundingHistory(position.asset, now.getTime() - FUNDING_LOOKBACK_MS, now.getTime()); | ||
| const latest = history[history.length - 1]; | ||
| if (!latest) continue; | ||
| fundingRate = parseFloat(latest.fundingRate); | ||
| } catch { | ||
| continue; | ||
| } |
| async function fillOrder(order: typeof orders.$inferSelect, fillPrice: number, market: MarketSnapshot) { | ||
| const referencePrice = parseFloat(market.price); | ||
| const slippageAmount = Math.abs(fillPrice - referencePrice); | ||
| const fillQuantity = parseFloat(order.quantity); | ||
| const feeAmount = computeFee(fillPrice * fillQuantity); | ||
|
|
||
| await db.insert(fills).values({ | ||
| orderId: order.id, | ||
| price: fillPrice.toString(), | ||
| quantity: order.quantity, | ||
| priceSource: market.source, | ||
| sourceTimestamp: market.updatedAt, | ||
| fillModelVersion: FILL_MODEL_VERSION, | ||
| referencePrice: referencePrice.toString(), | ||
| slippageAmount: slippageAmount.toString(), | ||
| feeAmount: feeAmount.toString(), | ||
| }); |
| export function estimateLiquidationPrice(entryPrice: number, leverage: number, side: Side): number { | ||
| const factor = 1 / leverage; | ||
| return side === 'LONG' | ||
| ? entryPrice * (1 - factor + MAINTENANCE_MARGIN_RATIO) | ||
| : entryPrice * (1 + factor - MAINTENANCE_MARGIN_RATIO); | ||
| } |
| for (const position of openPositions) { | ||
| const lastCharged = position.lastFundingChargedAt ?? position.createdAt; | ||
| const elapsedMs = now.getTime() - lastCharged.getTime(); | ||
| if (elapsedMs < FUNDING_MIN_ACCRUAL_INTERVAL_MS) continue; | ||
|
|
||
| let fundingRate: number; | ||
| try { | ||
| const history = await fetchFundingHistory(position.asset, now.getTime() - FUNDING_LOOKBACK_MS, now.getTime()); | ||
| const latest = history[history.length - 1]; |
|
@copilot resolve the merge conflicts on this branch. |
Resolved. I merged |
Summary
Closes #39 (PAPER-REALISM-001).
execution/paperEngine.tsalready enforced real risk limits (kill switch, position/leverage/loss limits) but paper fills didn't model simulated fees/funding, a liquidation estimate, or record price provenance -- undermining the credibility of any backtest-vs-live-practice comparison the product wants to offer.execution/fillModel.ts(new) -- versioned (FILL_MODEL_VERSION) pure functions:computeFee,estimateLiquidationPrice(standard isolated-margin formula with a documented flat maintenance-margin-ratio simplification),computeFundingCost(pro-rated by elapsed time relative to Hyperliquid's real hourly funding interval, standard perp convention).fillstable: every fill now recordspriceSource,sourceTimestamp,fillModelVersion,referencePrice,slippageAmount,feeAmount,marketType,simulated. The six provenance fields are nullable -- fills predating this feature never had them computed, and backfilling a synthetic value would fabricate evidence that doesn't exist (same reasoningsignals.signal_scorealready follows forSIGNAL-SCORE-001; the shared dev DB had pre-existingfillsrows confirming this wasn't hypothetical).positionstable:leverage(quantity-weighted-averaged the same wayentryPriceis),liquidationPriceEstimate,feesPaid,fundingPaid,lastFundingChargedAt.closePositionnow settles accumulated fees (including a real exit fee) and funding intorealizedPnl-- which flows straight intoanalytics/'s existing DATA-015 sample-size-tiered performance metrics with no changes needed there.accruePaperFunding(new, run every 5 minutes fromserver.ts): charges open positions real Hyperliquid funding viafetchFundingHistory-- deliberately notgetFundingRate, whosetype: 'fundingRate'endpoint was verified directly against live Hyperliquid mainnet during implementation to currently return a real HTTP 422 (a discrepancy an earlier issue's audit had flagged but left unfixed as out of scope). Building this issue's real, recurring cost calculation on the broken endpoint would have meant funding silently never accruing in practice -- confirmedfetchFundingHistory(already used successfully byBACKTEST-001) works correctly instead.docs/architecture/paper-execution.mddocuments every assumption: perp-only (no spot ingestion exists), reduce-only out of scope (not represented anywhere in this codebase's order model), the fee/funding/liquidation-estimate formulas, and what's structurally out of scope.Verification
realizedPnlcorrectly nets out entry+exit fees, confirmedGET /api/analytics/performancestill works correctly (insufficient tier at sample size 1, no regression), and ranaccruePaperFundingdirectly against real open positions -- discovering and fixing thegetFundingRate/fetchFundingHistoryissue in the process (a real bug that would have silently made funding accrual never work in production).git diff --statvsgit diff -w --statnow identical).Test plan
getFundingRate→fetchFundingHistoryswitch is correct (i.e., independently confirmgetFundingRatereally is broken against live Hyperliquid before trusting this PR's claim)