Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 45 additions & 12 deletions .github/workflows/paper.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,11 @@ on:
required: true
type: string
extended-provider:
description: Provider needing a fresh six-hour soak, or all for initial qualification
description: Provider needing fresh evidence, or all for initial qualification
required: true
default: none
default: all
type: choice
options: [none, alpaca, ib, okx, all]
options: [alpaca, ib, okx, all]

permissions: {}

Expand Down Expand Up @@ -75,6 +75,7 @@ jobs:
mkdir -p "${{ runner.temp }}/paper-run" "${{ runner.temp }}/paper-state"
- name: Exercise Alpaca paper order lifecycle
id: alpaca-exercise
if: ${{ inputs.extended-provider == 'alpaca' || inputs.extended-provider == 'all' }}
continue-on-error: true
env:
ALPACA_API_KEY: ${{ secrets.ALPACA_API_KEY }}
Expand All @@ -92,7 +93,11 @@ jobs:
--output "${{ github.workspace }}/paper-evidence/alpaca-exercise.json"
- name: Reconcile Alpaca in a fresh process
id: alpaca-restart
if: ${{ always() }}
if: >-
${{
always() &&
(inputs.extended-provider == 'alpaca' || inputs.extended-provider == 'all')
}}
continue-on-error: true
env:
ALPACA_API_KEY: ${{ secrets.ALPACA_API_KEY }}
Expand All @@ -110,7 +115,11 @@ jobs:
--output "${{ github.workspace }}/paper-evidence/alpaca-restart.json"
- name: Exercise IB paper order lifecycle
id: ib-exercise
if: ${{ always() }}
if: >-
${{
always() &&
(inputs.extended-provider == 'ib' || inputs.extended-provider == 'all')
}}
continue-on-error: true
env:
IB_ACCOUNT: ${{ secrets.IB_ACCOUNT }}
Expand All @@ -130,7 +139,11 @@ jobs:
--output "${{ github.workspace }}/paper-evidence/ib-exercise.json"
- name: Reconcile IB in a fresh process
id: ib-restart
if: ${{ always() }}
if: >-
${{
always() &&
(inputs.extended-provider == 'ib' || inputs.extended-provider == 'all')
}}
continue-on-error: true
env:
IB_ACCOUNT: ${{ secrets.IB_ACCOUNT }}
Expand All @@ -150,6 +163,7 @@ jobs:
--output "${{ github.workspace }}/paper-evidence/ib-restart.json"
- name: Qualify the supported OKX feed against the public service
id: okx-external
if: ${{ inputs.extended-provider == 'okx' || inputs.extended-provider == 'all' }}
continue-on-error: true
run: >-
cd "${{ runner.temp }}/paper-run" &&
Expand All @@ -162,12 +176,31 @@ jobs:
id: provider-soaks
if: >-
${{
inputs.extended-provider != 'none' &&
steps.alpaca-exercise.outcome == 'success' &&
steps.alpaca-restart.outcome == 'success' &&
steps.ib-exercise.outcome == 'success' &&
steps.ib-restart.outcome == 'success' &&
steps.okx-external.outcome == 'success'
always() &&
(
(
inputs.extended-provider == 'alpaca' &&
steps.alpaca-exercise.outcome == 'success' &&
steps.alpaca-restart.outcome == 'success'
) ||
(
inputs.extended-provider == 'ib' &&
steps.ib-exercise.outcome == 'success' &&
steps.ib-restart.outcome == 'success'
) ||
(
inputs.extended-provider == 'okx' &&
steps.okx-external.outcome == 'success'
) ||
(
inputs.extended-provider == 'all' &&
steps.alpaca-exercise.outcome == 'success' &&
steps.alpaca-restart.outcome == 'success' &&
steps.ib-exercise.outcome == 'success' &&
steps.ib-restart.outcome == 'success' &&
steps.okx-external.outcome == 'success'
)
)
}}
continue-on-error: true
env:
Expand Down
44 changes: 44 additions & 0 deletions .github/workflows/provider-health.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Monthly provider health

on:
schedule:
- cron: "0 14 15 * *"
workflow_dispatch:

permissions: {}

env:
UV_NO_SOURCES: "1"
UV_VERSION: "0.10.9"

jobs:
provider-health:
name: ${{ matrix.provider }} health
runs-on: [self-hosted, linux, x64, ib-paper]
timeout-minutes: 20
environment: paper
permissions:
contents: read
strategy:
fail-fast: false
matrix:
include:
- provider: alpaca
test_path: tests/integration/test_alpaca_integration.py
- provider: ib
test_path: tests/integration/test_ib_integration.py
- provider: okx
test_path: tests/integration/test_okx_integration.py
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
with:
version: ${{ env.UV_VERSION }}
- run: uv python install 3.12
- name: Install the default-branch health environment
run: uv sync --python 3.12 --all-extras --dev --locked
- name: Check current provider availability and behavior
env:
ALPACA_API_KEY: ${{ secrets.ALPACA_API_KEY }}
ALPACA_SECRET_KEY: ${{ secrets.ALPACA_SECRET_KEY }}
run: uv run pytest --run-external "${{ matrix.test_path }}" -q
7 changes: 7 additions & 0 deletions .github/workflows/release-recovery.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,12 @@ jobs:
with:
version: "0.10.9"
- run: uv python install 3.12
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
github-token: ${{ github.token }}
run-id: ${{ inputs.qualification-run-id }}
name: dist-${{ inputs.candidate-sha }}
path: dist/
- name: Create a clean evidence runtime
id: paper-runtime
run: |
Expand All @@ -51,6 +57,7 @@ jobs:
scripts/qualification/check_paper_evidence.py
--repository "${{ github.repository }}"
--commit "${{ inputs.candidate-sha }}"
--artifacts-dir dist
--github-output "$GITHUB_OUTPUT"

verify:
Expand Down
7 changes: 6 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ jobs:
candidate-version: ${{ inputs.version }}

paper-evidence:
name: Require exact exercises and reusable provider evidence
name: Bind exact artifacts to reusable provider evidence
runs-on: ubuntu-latest
needs: [ecosystem-qualification, qualification]
outputs:
Expand All @@ -83,6 +83,10 @@ jobs:
with:
version: "0.10.9"
- run: uv python install 3.12
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: dist-${{ inputs.candidate-sha }}
path: dist/
- name: Create a clean paper evidence runtime
id: paper-runtime
run: |
Expand All @@ -97,6 +101,7 @@ jobs:
scripts/qualification/check_paper_evidence.py
--repository "${{ github.repository }}"
--commit "${{ inputs.candidate-sha }}"
--artifacts-dir dist
--github-output "$GITHUB_OUTPUT"

documentation:
Expand Down
4 changes: 2 additions & 2 deletions docs/claim-evidence.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ external paper account.
| Portability | Lifecycle version 1 callbacks and canonical intents are portable across the qualified backtest and live engines | `tests/contracts/test_causal_strategy_parity.py`; minimum, locked, and maximum dependency profiles |
| Safety | Canonical orders fail atomically; live safety may reject or reduce an intent without redefining it | `tests/unit/test_order_contract.py`, `tests/unit/test_safe_broker.py`, `tests/unit/test_secure_persistence.py` |
| Feeds | OKX and bar aggregation emit validated UTC events and halt on overload or continuity loss | installed-wheel external OKX qualification; feed contract, queue, continuity, recovery, and stress tests |
| Brokers | IB and Alpaca implement the documented snapshots, state transitions, canonical order input, capabilities, reconciliation, and six-hour paper-session contract | broker contract and adapter suites; `scripts/qualification/qualify_paper.py`; fresh installed-wheel lifecycle and soak evidence |
| Brokers | IB and Alpaca implement the documented snapshots, state transitions, canonical order input, capabilities, reconciliation, and six-hour paper-session contract | broker contract and adapter suites; exact-candidate artifact qualification; complete lifecycle and soak evidence with a matching provider contract |
| Performance | The credential-free Linux reference workload sustains 100 events per second for one virtual hour with less than 25 MiB post-warmup RSS growth, no-op dispatch p99 below 10 ms, shutdown below 5 seconds, bounded queues, and exact event and intent checksums | `scripts/qualification/qualify_performance.py`; `tests/stress`; candidate performance report |
| Platform | The wheel and source distribution install with Python 3.12, 3.13, and 3.14 on Linux, macOS, and Windows; Python 3.15 prerelease installation and the non-hardware-dependent suite pass on all three operating systems before final support | stable installed-artifact profiles; ecosystem qualification matrix |
| Maturity | IB and Alpaca brokers plus the OKX feed are stable-supported; Alpaca, IB, DataBento, and generic CCXT feeds require experimental opt-in | public claim scan, experimental-feed tests, retained six-hour OKX, IB, and Alpaca evidence tied to the candidate wheel hash |
| Maturity | IB and Alpaca brokers plus the OKX feed are stable-supported; Alpaca, IB, DataBento, and generic CCXT feeds require experimental opt-in | public claim scan, experimental-feed tests, exact candidate artifacts, and retained six-hour OKX, IB, and Alpaca evidence with matching provider contracts |

The complete credential-free command is documented in [Candidate Qualification](qualification.md).
Paper evidence is separate because untrusted pull requests never receive account credentials.
72 changes: 42 additions & 30 deletions docs/qualification.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,11 @@ on Linux; they do not state venue latency or capacity.
## Qualify Paper Accounts And External Feeds Separately

Credentialed IB and Alpaca checks run only through the manually dispatched `Paper Qualification`
workflow in the protected `paper` environment. Supply the exact candidate commit and the successful
qualification run that retained `dist-CANDIDATE_SHA`. The workflow downloads that artifact instead
of rebuilding it, installs its wheel outside the checkout, and rejects a run whose commit or status
does not match.
workflow in the protected `paper` environment. Run it when a provider contract changes. Supply the
exact candidate commit, the successful qualification run that retained `dist-CANDIDATE_SHA`, and
the changed provider. Use `all` only for initial qualification or when shared changes invalidate all
three providers. The workflow downloads the candidate artifact instead of rebuilding it, installs
its wheel outside the checkout, and rejects a run whose commit or status does not match.

Each provider must verify its official paper endpoint and paper account before submission. The
workflow then checks positions, all pending orders and an asset-filtered view, cash, account value,
Expand All @@ -49,9 +50,9 @@ acknowledgement, reconnect reconciliation, replacement, cancellation, cleanup, a
from a fresh process. IB accepts only the standard paper ports and an account identified by IB as a
paper account. Alpaca accepts only the SDK's official paper endpoint in sandbox mode.

The order-lifecycle checks run for every candidate. Extended Alpaca, IB, and OKX soaks run
independently. Set `extended-provider` to the provider whose contract changed, to `all` for initial
qualification, or to `none` when all three current extended results remain applicable.
The selected provider runs its short lifecycle or external-feed check and its six-hour soak. Other
providers do not need to be online. A release with unchanged provider contracts does not dispatch
this workflow; its release gate selects retained matching provider evidence automatically.

Each selected broker remains connected to its paper account for at least six continuous hours.
Five-minute snapshots must preserve exact adapter-to-provider positions and pending orders, valid
Expand All @@ -61,34 +62,45 @@ disconnect within five seconds. Any unexpected disconnect, reconciliation change
identity, or retained error fails the run.

Every extended report records a provider-contract identity. The identity hashes the provider
adapter, qualification logic reached by that soak, the resolved provider dependency, and the Linux
CPython 3.12 runtime contract. OKX also hashes the shared event and queue contracts reached by its
feed. A changed identity invalidates only that provider's result. Documentation, CI metadata, and
source code outside the extended provider path do not invalidate it.
adapter, the qualification logic reached by the check, the resolved provider dependency, the
supported Python runtime, and the shared behavior that the provider reaches. IB and Alpaca include
canonical orders, safety policy, reconciliation, persistence, protocols, and state migration. OKX
includes event, queue, persistence, and protocol behavior. A changed identity invalidates only that
provider's result. Documentation, metadata, workflows, generic build tools, and source code outside
the provider's reachable contract do not invalidate it. Missing inputs or an unsupported identity
schema fail closed.

The workflow stores redacted evidence as a 90-day Actions artifact and as a durable prerelease
asset named `provider-evidence-RUN_ID/provider-evidence.zip`. The release gate validates the exact
candidate's short exercises, then selects a matching extended result for each provider from current
or earlier runs. Evidence has no time-based expiry. Missing, malformed, or mismatched evidence fails
closed. The Aug 10 run at `98c414e9d858427c31e1680faccdc8dca498bf6b` is the only legacy result
accepted without an embedded identity; the verifier recomputes its narrower reviewed identity.

The same workflow qualifies `OKXFundingFeed` against the public OKX service for every candidate. It compares adapter
events with provider-native observations, observes consecutive complete candles across restart,
rejects stale input, forces fail-closed overload, and requires shutdown within five seconds. The
selected OKX extended run then continues for at least six hours with five-minute event, queue, and
RSS snapshots and one retained-state restart. Every complete minute must remain
contiguous, final state must match a native OKX observation, RSS growth must remain below 25 MiB,
and errors, rejections, or overflows fail the run. The retained feed bundle also proves that Alpaca,
IB, DataBento, and generic CCXT feed construction requires `experimental=True` and records each
adapter's missing guarantees. The release gate requires the paper and feed bundles to identify the
same commit and wheel.
asset named `provider-evidence-RUN_ID/provider-evidence.zip`. The release gate takes wheel and
source-distribution hashes from the exact candidate artifacts. It then selects complete lifecycle,
restart, and soak evidence independently for each matching provider contract. Provider evidence may
come from different current or earlier runs. Evidence has no time-based expiry. Missing, malformed,
mismatched, or unclassifiable evidence fails closed before publication. The Aug 10 run at
`98c414e9d858427c31e1680faccdc8dca498bf6b` is the only legacy result accepted without an embedded
identity; the verifier checks its original identity and compares the current contract inputs across
both revisions.

When the OKX contract changes, the same workflow compares `OKXFundingFeed` events with
provider-native observations, observes consecutive complete candles across restart, rejects stale
input, forces fail-closed overload, and requires shutdown within five seconds. The selected OKX run
then continues for at least six hours with five-minute event, queue, and RSS snapshots and one
retained-state restart. Every complete minute must remain contiguous, final state must match a
native OKX observation, RSS growth must remain below 25 MiB, and errors, rejections, or overflows
fail the run. The retained feed bundle also proves that Alpaca, IB, DataBento, and generic CCXT feed
construction requires `experimental=True` and records each adapter's missing guarantees.

The `Monthly provider health` workflow runs the external IB, Alpaca, and OKX integration suites on
the fifteenth day of each month. It detects provider or SDK drift independently of a release. A
failed monthly run is visible for investigation, but it does not revoke matching retained evidence
or block a candidate whose relevant provider contract is unchanged. The health workflow cannot
publish packages or create GitHub releases.

IB Gateway or TWS authentication is an operator prerequisite. Once the paper session is
authenticated, the self-hosted runner can execute the short check or selected soak without further
operator input. IB does not support fully unattended login, so a changed IB contract may still wait
for authentication. Do not expose broker credentials to pull-request code or point IB qualification
at a live port.
operator input. IB does not support fully unattended login. A changed IB contract waits during IB
maintenance or until authentication is available; there is no manual release bypass. An unchanged
IB contract reuses matching retained evidence. Do not expose broker credentials to pull-request
code or point IB qualification at a live port.

## Retain Candidate Identity

Expand Down
Loading
Loading