From 8b76b0394401b083b9b63aefca7109d8ab27550a Mon Sep 17 00:00:00 2001 From: Stefan Jansen Date: Thu, 24 Sep 2026 10:52:48 -0400 Subject: [PATCH] docs: define six-library guide quality standard --- .gitignore | 4 -- docs/standards/documentation.md | 78 ++++++++++++++++++++ reviews/2026-09-24-library-documentation.md | 79 +++++++++++++++++++++ 3 files changed, 157 insertions(+), 4 deletions(-) create mode 100644 reviews/2026-09-24-library-documentation.md diff --git a/.gitignore b/.gitignore index 1813518..4512063 100644 --- a/.gitignore +++ b/.gitignore @@ -49,7 +49,3 @@ /status/current.json /status/current.md /status/monitor.json - -# Pre-existing local review notes. Migrate deliberately before tracking. -/docs/2026-03-26-docs-prompts-coordination-note.md -/docs/2026-03-26-docs-suite-audit.md diff --git a/docs/standards/documentation.md b/docs/standards/documentation.md index 7f3e5e2..2fb2d62 100644 --- a/docs/standards/documentation.md +++ b/docs/standards/documentation.md @@ -66,6 +66,84 @@ guides solve named tasks, reference pages describe the public interface, and exp semantics and design decisions. A page should have one primary purpose. The README remains a concise entry point rather than a duplicate documentation site. +## Reader and content standard for the six user-facing libraries + +Data, Engineer, Models, Diagnostic, Backtest, and Live must give readers the same level of guidance +for their respective responsibilities. Equal quality does not require equal page counts. Each site +must let a new reader answer these questions without reading source code: + +1. What problem does this package solve, and which adjacent package owns a different step? +2. What can I run first with the released package and without private data or credentials? +3. How do I perform each principal supported task with my own input? +4. Which defaults, input contracts, timing rules, failure modes, and limits affect the result? +5. Where are the exact public signatures and supported options? +6. Which public book notebook or case study explains the method or shows a fuller application? + +The home page routes to installation, a completed first workflow, task guides, API reference, and +book examples. The first workflow includes its input, complete command or program, expected result, +and an explanation of that result. A task guide states its preconditions, the steps, how to verify +the result, relevant decisions or pitfalls, and links to the API reference. Explanation pages cover +semantics that a short recipe would conceal, such as temporal alignment, statistical assumptions, +execution order, persistence, and safety boundaries. Reference pages cover the released public +interface uniformly; a guide need not repeat every parameter. + +Each library maintains a capability-to-documentation inventory during review. For every principal +supported workflow, it identifies the task guide, public API reference, and runnable example. An +optional provider, broker, accelerator, or paid data source can use a separate example, but its +requirements and an offline path must be explicit. The inventory also marks experimental features, +unsupported operations, and migration routes. Do not describe an experimental feature as a normal +default or imply that adjacent libraries are required when they are optional. + +Examples must import the published package API, use representative data with stated assumptions, +and show output or an observable assertion. Run the primary quickstart and representative workflows +from a built wheel in a clean environment. Compare documented output where deterministic. Check +optional-service examples with the narrowest safe verification available, and label any part that +cannot run without credentials or hardware. A strict MkDocs build alone does not verify example +behavior or external links. + +## Book and case-study links + +The six sites use *Machine Learning for Trading, Third Edition* as a learning resource, not a +purchase prompt. Each Book Guide maps a verified companion file to the concept it teaches, the +relevant library API, and the guide that explains that API. Major task guides link directly to a +matching notebook or case study when one exists. State whether that file calls the library, builds +the method manually for teaching, or illustrates a related workflow. When no relevant public file +exists, omit the link rather than inventing a match. + +Book links must be clickable GitHub file URLs at a fixed companion commit. Check each path against +that commit's Git tree and update the map when the book changes. A site should use one documented +companion revision for its book links, so readers do not silently cross incompatible notebook +versions. Explain material differences between book inputs and library examples, including data +availability, package versions, training cost, execution settings, or required services. Link from +the Book Guide back to the relevant library guide. Link from the book to library documentation when +the book repository's own work is next updated; library documentation must be useful on its own. + +## Maintained Quantopian libraries and successor guidance + +Zipline, Alphalens, and Pyfolio remain useful reference points. Where a task overlaps with the ML4T +stack, give readers a factual migration map: Zipline simulation to Backtest, Alphalens factor +analysis to Diagnostic and, where relevant, Engineer, and Pyfolio performance analysis to +Diagnostic. Name the specific supported task and API on each side, required input conversion, and +material differences. Acknowledge useful behavior that the newer package does not provide. Do not +assert general superiority, complete compatibility, or an end-of-life date without evidence. The +older repositories' maintainers own any future notice in those repositories; the six library sites +can provide the destination guidance now. + +## Discovery and review evidence + +Use descriptive page titles, one clear heading per page, a factual page summary, stable canonical +URLs, and navigation labels that match readers' tasks. Explain terms people search for in the page +that answers the corresponding task. Avoid repeated keywords, generic comparison claims, or +duplicating the book's teaching text. Keep diagrams, tables, and screenshots legible, with text +alternatives where the visual carries information. + +A documentation change is ready for review when its pull request records the capability inventory, +the companion commit used for book links, the examples actually executed, link-check results, +`uv run mkdocs build --strict`, and any externally dependent examples that could not be run. Review +the rendered navigation and representative pages at desktop and narrow widths. After release, +verify the deployed route and release identity using the checks below. Record exceptions as scoped +findings with an owner; a passing build does not close a content or accuracy gap. + ## Build and deployment - MkDocs is the documentation generator. diff --git a/reviews/2026-09-24-library-documentation.md b/reviews/2026-09-24-library-documentation.md new file mode 100644 index 0000000..6716b64 --- /dev/null +++ b/reviews/2026-09-24-library-documentation.md @@ -0,0 +1,79 @@ +# Six-library documentation review, 2026-09-24 + +## Finding + +All six local documentation sites build strictly, but they do not yet give readers the same quality +of route from a task to a tested example, a public API, and a relevant book notebook. Backtest is the +strongest reference for executable guidance and checked book links. The other sites have useful +content and broadly similar navigation, but most book references are plain paths, some paths have +drifted, and example verification differs by repository. + +This review covers Data, Engineer, Models, Diagnostic, Backtest, and Live. Specs remains part of the +ecosystem's release standard but has no development sidecar and is outside this six-site campaign. + +## Evidence and limits + +- Source: local release checkouts on 2026-09-24. Engineer, Models, Diagnostic, Backtest, and Live + were clean at `main`, matching their configured upstream tracking branches. Data was clean at + `docs/market-alt-sources`; that branch was three commits ahead of and one commit behind the local + `origin/main` ref. Data's page inventory is therefore branch-specific. +- Book path comparison: the local public companion checkout + `stefan-jansen/machine-learning-for-trading` at commit + [`d2edec5`](https://github.com/stefan-jansen/machine-learning-for-trading/tree/d2edec54b1c7a6a9d7a97d8129eb05db4491e1eb). + The check matched literal `.py` and `.ipynb` paths in each Book Guide, allowing the obsolete + `code/` prefix to be removed. Template paths such as `` were counted as unresolved. This + is a path check, not a semantic validation of every book mapping. +- Build: `uv run mkdocs build --strict` passed in all six checkouts during this review. +- Backtest checks: `validation/check_book_links.py` passed for 22 companion paths at commit + `366e1d51ace2d851776499a68da3d6e3c2641b02`; `validation/check_documentation_links.py` + checked 2,005 rendered content links and 29 external destinations; and + `validation/check_documentation_examples.py` ran its 23 selected examples successfully. +- This review did not run each library's full tests, execute every code block, check all external + destinations in the other five sites, or compare deployed pages with these checkouts. + +| Library | Markdown pages | User-guide pages | Tutorials | Book Guide | Strict build | Main finding | +|---|---:|---:|---:|---|---|---| +| Data | 60 | 6 | 6 | Yes | Pass | Broad provider coverage and offline quickstart; book references are plain text and several paths have changed. | +| Engineer | 14 | 8 | 0 | Yes | Pass | Substantial feature, label, and bar guides; none of 17 concrete book file paths resolves in the current companion tree; three case-study paths are templates. | +| Models | 19 | 9 | 0 | Yes | Pass | Strong model-family and contract explanations; six named book files resolve, but the guide provides no direct file links and its first workflow is costly. | +| Diagnostic | 22 | 9 | 0 | Yes | Pass | Good method pages and synthetic quickstart; seven of 26 named book file paths checked here are stale. | +| Backtest | 28 | 13 | 9 | Yes | Pass | Complete tutorial sequence, checked output, and pinned book links; use as the initial quality reference, then review coverage against the full released API. | +| Live | 16 | 8 | 0 | Yes | Pass | Practical broker, feed, risk, and operator guides; three of 14 named book file paths checked here are stale, and the first run requires credentials even in shadow mode. | + +Page counts include material outside primary navigation. They measure inventory, not quality. + +## Cross-library issues + +1. **Book navigation.** Backtest links directly to a fixed book revision and verifies the files. + The other five Book Guides predominantly list paths without clickable links. Engineer's chapter + names and numbering reflect an older book tree: for example, the current companion file is + `03_market_microstructure/14_itch_bar_sampling.py`, while its guide names + `03_market_microstructure/code/08_itch_bar_sampling.py`. A reader cannot rely on those paths. +2. **Example evidence.** Every site builds, but a strict build does not execute Python examples. + Backtest has an installed-wheel runner with expected output for selected workflows. The other + sites need evidence for their own principal examples, scaled to optional services and hardware. +3. **First use.** Data, Backtest, and Diagnostic provide local or synthetic first runs. Engineer's + quickstart uses synthetic data but lacks a recorded expected result. Models opens with a sizeable + IPCA workflow, then shows longer neural training. Live's recommended shadow-mode code still + instantiates credentialed Alpaca adapters. Each site needs a visible success criterion and an + offline path where the package can support one. +4. **Content depth.** The sites share Home, Getting Started, User Guide, API Reference, and Book + Guide in navigation. Backtest adds a linked tutorial sequence. Diagnostic adds statistical + method explanations. Data adds provider catalogues. Equal quality should be judged by task + completion, public API accuracy, and tested examples rather than identical sections or lengths. +5. **Maintained predecessors.** The six sites rarely explain how a reader using Zipline, Alphalens, + or Pyfolio should evaluate a relevant successor workflow. A factual task-by-task comparison is + needed before adding notices to the older repositories. Zipline, Alphalens, and Pyfolio should + not be described as discontinued or fully replaceable without evidence. + +## Review sequence + +1. Backtest checks its current guide against the complete released public surface and documents + any remaining gaps. Keep its example and link checks as the reference pattern. +2. Engineer repairs the book map first, followed by Diagnostic, Live, Data, and Models. Each checks + notebook meaning, not only file existence, and links from principal task pages. +3. Each library records a capability-to-guide-to-reference-to-example inventory, then fills gaps + in task order. Review rendered pages and example output before publishing. +4. The ecosystem rechecks all six sites against the shared standard and records any exceptions. + The older repository maintainers can then add scoped successor notices with verified migration + examples.