Skip to content

Make Live workflows and book links verifiable #106

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
stefan-jansen merged 2 commits into from
Sep 24, 2026
Merged

Make Live workflows and book links verifiable #106

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8524412
docs: make Live workflows and book links verifiable
Sep 24, 2026
588b2d5
docs: state feed maturity plainly in Book Guide
Sep 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .github/workflows/docs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,15 @@ jobs:
with:
fetch-depth: 0
ref: ${{ env.CANDIDATE_SHA }}
- name: Check out pinned public book tree
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: stefan-jansen/machine-learning-for-trading
ref: d2edec54b1c7a6a9d7a97d8129eb05db4491e1eb
path: companion
filter: blob:none
sparse-checkout: README.md
persist-credentials: false
- uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
with:
version: "0.10.9"
Expand All @@ -38,6 +47,10 @@ jobs:
ML4T_DOCS_VERSION: ${{ env.CANDIDATE_VERSION }}
ML4T_DOCS_COMMIT: ${{ env.CANDIDATE_SHA }}
run: uv run mkdocs build --strict
- name: Check pinned book files
run: uv run python scripts/qualification/check_book_links.py --companion-repo companion
- name: Check documentation links
run: uv run python scripts/qualification/check_documentation_links.py
- name: Verify rendered documentation identity
run: >-
uv run python scripts/qualification/verify_documentation_identity.py
Expand Down
169 changes: 83 additions & 86 deletions docs/book-guide/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,88 +1,85 @@
# Book Guide

This guide maps `ml4t-live` to **Machine Learning for Trading, Third Edition** so you can move between
the library docs and the book materials without guessing which notebook or chapter matters.

## How To Use This Guide

- Start with the chapter map if you are reading the book.
- Start with the API map if you are coming from the library and want the matching notebook.
- Treat the listed code paths as the canonical references in the `ml4t/code` repository.

## Chapter To Feature Map

| Book material | What it teaches | `ml4t-live` connection |
| --- | --- | --- |
| Chapter 16 strategy simulation | event-driven strategies and parity-friendly design | the strategy surface you carry into `LiveEngine` |
| Chapter 18 costs | execution frictions and turnover budgets | the live costs you compare against post-deployment |
| Chapter 19 risk management | kill switches, drawdowns, limits | `LiveRiskConfig`, `SafeBroker`, staged rollout |
| Chapter 25 live trading systems | brokers, feeds, operational parity, deployment | the core `ml4t-live` library surface |
| Chapter 26 MLOps governance | shadow mode, challenger rollout, circuit breakers | operational procedures around `ml4t-live` |

## Notebook And Script Map

| Book path | Why it matters here |
| --- | --- |
| `code/16_strategy_simulation/06_framework_parity.py` | shows why keeping one strategy interface matters before live deployment |
| `code/25_live_trading/01_unified_framework_demo.py` | compares lifecycle traces, canonical intents, and signals across the two engines |
| `code/25_live_trading/03_ib_paper_trading_demo.py` | Interactive Brokers connectivity path |
| `code/25_live_trading/04_alpaca_paper_trading_demo.py` | Alpaca paper-trading path |
| `code/25_live_trading/05_alpaca_crypto_live_demo.py` | Alpaca crypto workflow |
| `code/25_live_trading/08_pipeline_verification.py` | parity checks between research and live workflows |
| `code/25_live_trading/09_crypto_funding_deployment_loop.py` | OKX funding-rate deployment loop |
| `code/25_live_trading/10_safety_risk_demo.py` | `SafeBroker` limits, shadow mode, and kill-switch behavior |
| `code/26_mlops_governance/03_safe_model_rollout.py` | shadow-mode and staged-promotion procedures around live deployment |
| `code/26_mlops_governance/04_circuit_breakers.py` | broader operational safety concepts that complement `SafeBroker` |

## Case Study Link: Crypto Perpetuals Funding

The clearest live-trading case-study bridge in the current book materials is the crypto perpetuals
workflow:

| Case-study path | Library relevance |
| --- | --- |
| `code/case_studies/crypto_perps_funding/03_financial_features.py` | feature definitions that must stay consistent in live inference |
| `code/case_studies/crypto_perps_funding/14_backtest.py` | the validated backtest side of the strategy |
| `code/case_studies/crypto_perps_funding/17_risk_management.py` | portfolio and risk assumptions before deployment |
| `code/25_live_trading/okx_funding_rate_demo.py` | the live-style deployment bridge using exchange funding data |

## From Book Concepts To Library APIs

| Book concept | Library API |
| --- | --- |
| portable decision logic | `LiveEngine` plus a lifecycle-v1 `Strategy` subclass using supported broker operations |
| sync strategy calling async infrastructure | `ThreadSafeBrokerWrapper` |
| explicit deployment risk policy | `LiveRiskConfig` |
| pre-trade enforcement and kill switch | `SafeBroker` |
| paper-like live validation without routing orders | `execution_mode="shadow"` with `VirtualPortfolio` |
| broker-specific execution path | `IBBroker` or `AlpacaBroker` |
| stable-supported live data source | `OKXFundingFeed` |
| experimental opt-in data source | `AlpacaDataFeed`, `IBDataFeed`, `DataBentoFeed`, `CryptoFeed` |

## What The Book Often Shows Manually

The notebooks are pedagogical and frequently expose mechanics directly. The library turns those same
ideas into reusable interfaces:

- notebook orchestration becomes `LiveEngine`
- ad hoc risk checks become `SafeBroker`
- replay/live feed adapters become `DataFeedProtocol` implementations
- deployment-stage bookkeeping becomes `RiskState` and `VirtualPortfolio`

## Best Reading Path

If you are learning the stack end to end, the most efficient route is:

1. `code/16_strategy_simulation/06_framework_parity.py`
2. [Backtest to Live](../user-guide/backtest-to-live.md)
3. `code/25_live_trading/01_unified_framework_demo.py`
4. the broker page that matches your venue
5. `code/25_live_trading/10_safety_risk_demo.py`
6. [Risk Controls](../user-guide/risk.md)

## Related Docs

- [Home](../index.md)
- [Quickstart](../getting-started/quickstart.md)
- [Backtest to Live](../user-guide/backtest-to-live.md)
- [API Reference](../api/index.md)
The public companion files for *Machine Learning for Trading, Third Edition* show the methods and
operating decisions behind Live workflows. These links point to commit
[`d2edec54b1c7a6a9d7a97d8129eb05db4491e1eb`](https://github.com/stefan-jansen/machine-learning-for-trading/tree/d2edec54b1c7a6a9d7a97d8129eb05db4491e1eb).
Each file below was checked in that Git tree. "Calls Live" means its code imports `ml4t.live`;
"manual" means it teaches a method without that import; "related" means another workflow informs
Live decisions. A matching topic alone does not mean a notebook runs this library.

The Alpaca and IB notebooks use experimental feeds. Their broker adapters have a separate stable
paper contract.

## Run and control a Live strategy

- [Unified framework demo][unified] calls Live and Backtest to compare lifecycle traces and
portable strategy decisions. Follow [Backtest to Live](../user-guide/backtest-to-live.md).
- [IB paper trading demo][ib-paper] calls Live with an IB paper broker and experimental IB feed. It
requires TWS or IB Gateway. Follow [Brokers](../user-guide/brokers.md).
- [Alpaca paper trading demo][alpaca-paper] calls Live with Alpaca paper execution and an
experimental Alpaca feed. It requires credentials. Follow [Brokers](../user-guide/brokers.md).
- [Alpaca crypto live demo][alpaca-crypto] calls Live for a crypto venue workflow. Review its live
order prerequisites before attempting it. Follow [Brokers](../user-guide/brokers.md).
- [Pipeline verification][pipeline] calls Live to compare research and deployment inputs and
decisions. Follow [Backtest to Live](../user-guide/backtest-to-live.md).
- [Safety and risk demo][safety] calls Live to exercise `SafeBroker` and shadow risk behavior.
Follow [Risk Controls](../user-guide/risk.md).
- [Runtime safety showcase][runtime-safety] calls Live to show stale-data rejection, persistent
kill switch, reconciliation, and engine health. Follow the [Operator Guide](../user-guide/operator-guide.md).
- [Crypto funding deployment loop][funding-loop] calls `AlpacaBroker` only in its
operator-authorized paper path. It uses OKX public data for a separate execution rehearsal.
Follow [Data Feeds](../user-guide/feeds.md).

The crypto funding loop trains on Binance-derived perpetual data, observes OKX funding and bars,
and maps a subset to Alpaca USD spot crypto for paper execution. The venues and instruments differ;
its paper orders do not reproduce a perpetual-futures backtest. The notebook needs its case-study
artifacts and outbound data access, and paper submission needs credentials and explicit operator
opt-in. The [synthetic shadow quickstart](../getting-started/quickstart.md) is the local first run.

## Learn the methods before deployment

- [Framework parity][parity] is a related Backtest workflow that compares strategy behavior before
a Live port. It does not import Live. Follow [Backtest to Live](../user-guide/backtest-to-live.md).
- [Safe model rollout][rollout] teaches staged promotion and shadow evaluation manually. It does
not import Live. Follow the [Operator Guide](../user-guide/operator-guide.md).
- [Circuit breakers][circuit] teaches operational safety manually. It does not implement
`SafeBroker`. Follow [Risk Controls](../user-guide/risk.md).
- [Crypto financial features][features] is a related feature workflow using Engineer. It does not
import Live. Follow [Backtest to Live](../user-guide/backtest-to-live.md).
- [Crypto backtest][crypto-backtest] simulates funding-aligned decisions manually. It does not
import Live. Follow [Backtest to Live](../user-guide/backtest-to-live.md).
- [Crypto risk management][crypto-risk] studies sizing and risk manually. It does not import Live.
Follow [Risk Controls](../user-guide/risk.md).

The case-study files use research data, model artifacts, and their own simulation assumptions.
`LiveEngine` needs a feed, venue capability check, broker account state, and explicit `LiveRiskConfig`.
Do not treat a research result as a qualified paper or live run.

## Find the public interface

- Run lifecycle callbacks with `LiveEngine` and `ThreadSafeBrokerWrapper`:
[Backtest to Live](../user-guide/backtest-to-live.md).
- Limit and audit orders with `LiveRiskConfig`, `SafeBroker`, `RiskState`, and `VirtualPortfolio`:
[Risk Controls](../user-guide/risk.md).
- Connect `IBBroker` or `AlpacaBroker`: [Brokers](../user-guide/brokers.md).
- Choose `OKXFundingFeed`, `BarAggregator`, or an experimental provider feed:
[Data Feeds](../user-guide/feeds.md).
- Inspect or restart with `LiveEngine.runtime_status` and `SafeBroker.reconciliation_report`:
[Operator Guide](../user-guide/operator-guide.md).

See the [API Reference](../api/index.md) for exact signatures and options.

[unified]: https://github.com/stefan-jansen/machine-learning-for-trading/blob/d2edec54b1c7a6a9d7a97d8129eb05db4491e1eb/25_live_trading/01_unified_framework_demo.ipynb
[ib-paper]: https://github.com/stefan-jansen/machine-learning-for-trading/blob/d2edec54b1c7a6a9d7a97d8129eb05db4491e1eb/25_live_trading/03_ib_paper_trading_demo.ipynb
[alpaca-paper]: https://github.com/stefan-jansen/machine-learning-for-trading/blob/d2edec54b1c7a6a9d7a97d8129eb05db4491e1eb/25_live_trading/04_alpaca_paper_trading_demo.ipynb
[alpaca-crypto]: https://github.com/stefan-jansen/machine-learning-for-trading/blob/d2edec54b1c7a6a9d7a97d8129eb05db4491e1eb/25_live_trading/05_alpaca_crypto_live_demo.ipynb
[pipeline]: https://github.com/stefan-jansen/machine-learning-for-trading/blob/d2edec54b1c7a6a9d7a97d8129eb05db4491e1eb/25_live_trading/08_pipeline_verification.ipynb
[safety]: https://github.com/stefan-jansen/machine-learning-for-trading/blob/d2edec54b1c7a6a9d7a97d8129eb05db4491e1eb/25_live_trading/10_safety_risk_demo.ipynb
[runtime-safety]: https://github.com/stefan-jansen/machine-learning-for-trading/blob/d2edec54b1c7a6a9d7a97d8129eb05db4491e1eb/25_live_trading/13_runtime_safety_showcase.ipynb
[funding-loop]: https://github.com/stefan-jansen/machine-learning-for-trading/blob/d2edec54b1c7a6a9d7a97d8129eb05db4491e1eb/25_live_trading/09_crypto_funding_deployment_loop.ipynb
[parity]: https://github.com/stefan-jansen/machine-learning-for-trading/blob/d2edec54b1c7a6a9d7a97d8129eb05db4491e1eb/16_strategy_simulation/06_framework_parity.ipynb
[rollout]: https://github.com/stefan-jansen/machine-learning-for-trading/blob/d2edec54b1c7a6a9d7a97d8129eb05db4491e1eb/26_mlops_governance/03_safe_model_rollout.ipynb
[circuit]: https://github.com/stefan-jansen/machine-learning-for-trading/blob/d2edec54b1c7a6a9d7a97d8129eb05db4491e1eb/26_mlops_governance/04_circuit_breakers.ipynb
[features]: https://github.com/stefan-jansen/machine-learning-for-trading/blob/d2edec54b1c7a6a9d7a97d8129eb05db4491e1eb/case_studies/crypto_perps_funding/03_financial_features.ipynb
[crypto-backtest]: https://github.com/stefan-jansen/machine-learning-for-trading/blob/d2edec54b1c7a6a9d7a97d8129eb05db4491e1eb/case_studies/crypto_perps_funding/13_backtest.ipynb
[crypto-risk]: https://github.com/stefan-jansen/machine-learning-for-trading/blob/d2edec54b1c7a6a9d7a97d8129eb05db4491e1eb/case_studies/crypto_perps_funding/15_risk_management.ipynb
151 changes: 47 additions & 104 deletions docs/getting-started/quickstart.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,110 +1,53 @@
# Quickstart
# Quickstart: complete a shadow run

The recommended first run is shadow mode: your strategy executes through the live engine and risk
checks, but no real orders are sent to the broker.
Run a strategy through `LiveEngine` and `SafeBroker` with synthetic bars. The example needs Python
3.12-3.14 and `uv`; it needs no credentials, market-data subscription, or network service after
installation. It never sends an order to a venue.

## First Strategy
## Run the example

```python
import asyncio
Get the maintained example from the public repository, then run it with the published package:

from ml4t.backtest import Strategy
from ml4t.backtest.types import OrderSide
from ml4t.live import AlpacaBroker, AlpacaDataFeed, LiveEngine, LiveRiskConfig, SafeBroker


class BuyOnceStrategy(Strategy):
def on_data(self, timestamp, data, context, broker):
bar = data.get("SPY")
if bar is None:
return

if broker.get_position("SPY") is None:
broker.submit_order("SPY", 10, side=OrderSide.BUY)


async def main():
broker = AlpacaBroker(api_key="...", secret_key="...", paper=True)
feed = AlpacaDataFeed(
api_key="...",
secret_key="...",
symbols=["SPY"],
data_type="bars",
experimental=True,
)
safe_broker = SafeBroker(
broker,
LiveRiskConfig(
execution_mode="shadow",
max_position_value=25_000,
max_order_value=5_000,
),
)

engine = LiveEngine(BuyOnceStrategy(), safe_broker, feed)
await engine.connect()

try:
await engine.run()
finally:
await engine.stop()


asyncio.run(main())
```

## Why This Works

- Your strategy stays synchronous, just like in `ml4t-backtest`
- `LiveEngine` runs broker/feed I/O asynchronously
- `LiveEngine` runs every lifecycle callback on one worker thread, so synchronous broker calls do
not re-enter or block the async event loop
- `SafeBroker` enforces limits before any live order can be placed

## First-Run Checklist

Before you move past this example, confirm that:

1. the strategy receives bars from the feed you expect
2. orders appear in shadow mode instead of hitting the broker
3. positions and cash update through the virtual portfolio
4. you can stop and restart the engine cleanly

## Deployment Progression

1. Shadow mode: `execution_mode="shadow"`
2. Paper trading: `execution_mode="paper"` with paper broker credentials
3. Small live size: `execution_mode="live"` with conservative limits and low notional exposure
4. Gradual scale-up only after observing stable behavior

## Common Variations

### Experimental Interactive Brokers Feed

`IBDataFeed` needs a connected IB session object and explicit experimental opt-in:

```python
broker = IBBroker(port=7497)
await broker.connect()

feed = IBDataFeed(broker.ib, symbols=["SPY", "QQQ"], experimental=True)
```

### Aggregate Ticks Into Bars

```python
raw_feed = IBDataFeed(broker.ib, symbols=["SPY"], experimental=True)
feed = BarAggregator(raw_feed, bar_size_minutes=1, flush_timeout_seconds=2.0)
```bash
git clone https://github.com/ml4t/live.git
cd live
uv run --no-project --with ml4t-live python examples/shadow_mode_demo.py
```

## Next Steps

- [Installation](installation.md)
- [Backtest to Live](../user-guide/backtest-to-live.md)
- [Risk Controls](../user-guide/risk.md)
- [CLI](../user-guide/cli.md)
- [Examples](../user-guide/examples.md)
- [Operator Guide](../user-guide/operator-guide.md)
- [Brokers](../user-guide/brokers.md)
- [Data Feeds](../user-guide/feeds.md)
- [Book Guide](../book-guide/index.md)
The script defines a synthetic feed that emits one `DEMO` bar per second for 65 seconds. A moving
average strategy submits a buy or sell when its signal changes. `SafeBroker` records virtual fills
and positions under `execution_mode="shadow"`; the example broker raises if it receives a real
order. The input and full program are in
[`examples/shadow_mode_demo.py`](https://github.com/ml4t/live/blob/main/examples/shadow_mode_demo.py).

The run prints `Starting shadow mode demo with a synthetic feed.`, timestamped prices, five-second
position heartbeats, and `Finished shadow mode demo. final_positions=...`. The final position may
be flat or long because the synthetic signal changes during the run. A completed final line means
the engine stopped cleanly. The printed positions show what the virtual portfolio held; no venue
state changed.

To check the installed API without waiting for the full run, the [README quick start](https://github.com/ml4t/live/blob/main/README.md)
processes a single virtual fill and asserts its resulting position and cash balance. For the next
operator task, the [reconciliation example](../user-guide/examples.md) shows a restart mismatch
without a broker connection.

## Understand the boundary

This first run proves that the strategy callback, engine lifecycle, safety wrapper, and virtual
portfolio work with synthetic input. It does not qualify a broker, external feed, or restart on an
active account. A real paper session needs broker credentials or TWS/IB Gateway, the applicable
market-data access, a compatible feed, explicit `execution_mode="paper"`, and clean startup
reconciliation. Alpaca and IB feeds require `experimental=True` and do not have stable reconnect
continuity guarantees.

The public OKX funding feed needs outbound HTTPS but no credentials. To try it with the CLI, see
[Run a bounded shadow session](../user-guide/cli.md#shadow). A quiet feed or a run with no trade
intent is not evidence that a provider workflow is ready for paper orders.

## Continue

- [Backtest to Live](../user-guide/backtest-to-live.md) explains what stays portable.
- [Risk Controls](../user-guide/risk.md) covers limits, persistence, and the kill switch.
- [Brokers](../user-guide/brokers.md) and [Data Feeds](../user-guide/feeds.md) state provider requirements.
- [API Reference](../api/index.md) gives the released signatures.
- [Book Guide](../book-guide/index.md) connects the tasks to checked companion files.
Loading
Loading