From 7b86b36763e99172acfcd8ac35e2217865ef8821 Mon Sep 17 00:00:00 2001 From: Denis Drobyshev Date: Sun, 23 Aug 2026 12:57:56 +0300 Subject: [PATCH] Give stadion documentation Every other library in the organisation publishes a documentation site. stadion did not, so its homepage pointed at PyPI and the README carried everything. Five pages, all built from the README rather than newly invented: what the project is and what a run looks like, the six tasks with their headroom table, writing an agent, the protocol, and the self-check. Two things the README did not say plainly enough to leave implicit: A score slightly above 1 is sampling noise - the optimum is optimal in expectation, and a finite run scatters around its mean. Only a score clearly above 1, with an interval that does not reach back down, is a bug report. Left unqualified, the first reader to see 1.02 would file the wrong issue. `report.degenerate` is worth handling rather than ignoring, so the agent page says so and links the protocol section that explains why the scale sometimes has no denominator. Also corrected: the Status section said "Not yet on PyPI" four lines under an install command that reads `pip install stadion-rl`. It has been on PyPI since the 0.1.0 release. Built with `--strict`, which fails on a broken internal link or a page outside the navigation. The theme is configured `font: false` and the built HTML makes no third-party request - verified against the output, not assumed. --- .github/workflows/docs.yml | 48 ++++++++++++++++++++++++ .gitignore | 3 ++ README.md | 4 +- docs/agents.md | 77 ++++++++++++++++++++++++++++++++++++++ docs/assets/theme.css | 23 ++++++++++++ docs/index.md | 75 +++++++++++++++++++++++++++++++++++++ docs/protocol.md | 47 +++++++++++++++++++++++ docs/requirements.txt | 3 ++ docs/tasks.md | 65 ++++++++++++++++++++++++++++++++ docs/verify.md | 31 +++++++++++++++ mkdocs.yml | 71 +++++++++++++++++++++++++++++++++++ 11 files changed, 446 insertions(+), 1 deletion(-) create mode 100644 .github/workflows/docs.yml create mode 100644 docs/agents.md create mode 100644 docs/assets/theme.css create mode 100644 docs/index.md create mode 100644 docs/protocol.md create mode 100644 docs/requirements.txt create mode 100644 docs/tasks.md create mode 100644 docs/verify.md create mode 100644 mkdocs.yml diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml new file mode 100644 index 0000000..9cc37dd --- /dev/null +++ b/.github/workflows/docs.yml @@ -0,0 +1,48 @@ +name: Publish documentation + +on: + push: + branches: [main] + workflow_dispatch: + +permissions: + contents: read + pages: write + id-token: write + +concurrency: + group: pages + cancel-in-progress: false + +jobs: + build: + name: Build the site + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + - uses: actions/setup-python@v7 + with: + python-version: "3.12" + cache: pip + + - run: pip install -r docs/requirements.txt + + # --strict fails on a broken internal link or a page missing from the + # navigation, which is the breakage nobody notices until a reader hits it. + - run: mkdocs build --strict + + - uses: actions/configure-pages@v6 + - uses: actions/upload-pages-artifact@v5 + with: + path: site + + deploy: + name: Deploy to Pages + needs: build + runs-on: ubuntu-latest + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + steps: + - id: deployment + uses: actions/deploy-pages@v5 diff --git a/.gitignore b/.gitignore index a6ff2e5..ef7763d 100644 --- a/.gitignore +++ b/.gitignore @@ -13,3 +13,6 @@ htmlcov/ runs/ *.jsonl !tests/**/*.jsonl + +# The documentation build output; the site is published from CI, not committed. +site/ diff --git a/README.md b/README.md index 4762b17..e887724 100644 --- a/README.md +++ b/README.md @@ -193,7 +193,9 @@ its own policy is the failure this is built to catch, and it runs in CI. ## Status -v0.1 — six tasks, exact optima, the scoring protocol. Not yet on PyPI. +v0.1 — six tasks, exact optima, the scoring protocol. +[On PyPI](https://pypi.org/project/stadion-rl/) as `stadion-rl`; +[documentation](https://drobyshevdev.github.io/stadion/). That is every applied environment in `decisionrl`, each with its optimum computed rather than estimated. Anything added next has to clear the same bar, diff --git a/docs/agents.md b/docs/agents.md new file mode 100644 index 0000000..91ae5a8 --- /dev/null +++ b/docs/agents.md @@ -0,0 +1,77 @@ +# Writing an agent + +An agent implements one method: + +```python +import stadion + + +class MyAgent(stadion.Agent): + name = "my-agent" + + def act(self, view: stadion.View) -> int: + # view.text and view.choices — what a language model reads + # view.obs and view.env — what a numeric policy reads + return view.choices[0].value +``` + +## Two surfaces, one task + +Both surfaces are always present, so a reinforcement learning policy and a +language model are scored on the same task without either being translated +through the other's interface. + +`view.text` and `view.choices` +: The brief as prose, and the numbered menu of actions available this step. + This is what a language model reads. + +`view.obs` and `view.env` +: The numeric observation and the environment handle. This is what a policy + trained on arrays reads. + +An agent picks whichever it wants. Nothing about the scoring depends on the +choice, which is the point: a comparison between an LLM and a trained policy on +this task is a comparison of decisions rather than of adapters. + +## Language models + +`stadion.llm.LLMAgent` takes any `prompt -> reply` callable: + +```python +from stadion.llm import LLMAgent + +agent = LLMAgent(lambda prompt: my_model(prompt)) +``` + +No client library and no provider — a callable is the whole contract, so the +harness never needs a key, and swapping a model is swapping a function. + +## Running it + +```python +task = stadion.get("pricing") +report = stadion.evaluate(task, agent, instances=30, episodes=20) + +report.score # normalised: 0 = classical method, 1 = optimum +report.vs_baseline.ci # bootstrap interval on the paired difference +report.degenerate # True when the classical method is already optimal +print(report.summary()) +``` + +`report.degenerate` is worth handling rather than ignoring. On some instance +families the textbook rule is already indistinguishable from the optimum, so the +normalised score has no denominator — see [the protocol](protocol.md#sometimes-the-ceiling-is-a-draw). + +## Reading the score + +`0` is the tuned classical method and `1` is the exact optimum. + +A score slightly above 1 is sampling noise: the optimum is optimal in +expectation, and any finite run scatters around its mean. A score *clearly* above +1, with an interval that does not reach back down to it, is a bug report rather +than a triumph — nothing beats the optimum, so it means the agent is playing a +different game from the one the dynamic program solved: a different action menu, +a different horizon, or an instance it was not given. + +A negative score means the agent lost to the textbook rule. That is a normal +result and the reason the scale is drawn this way. diff --git a/docs/assets/theme.css b/docs/assets/theme.css new file mode 100644 index 0000000..43ed444 --- /dev/null +++ b/docs/assets/theme.css @@ -0,0 +1,23 @@ +/* The theme is configured with `font: false`, which stops Material pulling + Roboto from Google Fonts on every page load. That request buys the reader + nothing and costs them a third party, so the typography is a system stack + resolved locally instead. */ + +:root { + --md-text-font: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, + "Helvetica Neue", Arial, "Noto Sans", sans-serif; + --md-code-font: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, + "Liberation Mono", monospace; +} + +/* Every table on this site is a table of measurements. Figures line up on the + decimal, and the numeric columns are right-aligned so orders of magnitude are + comparable down the column rather than only readable across the row. */ +.md-typeset table:not([class]) { + font-variant-numeric: tabular-nums; +} + +.md-typeset table:not([class]) td:nth-child(n + 2), +.md-typeset table:not([class]) th:nth-child(n + 2) { + white-space: nowrap; +} diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..5a8edc0 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,75 @@ +# stadion + +**A proving ground for operational decisions.** An agent is scored against two +references it cannot argue with: the classical operations-research method for the +problem, and the exact optimum. The result is a normalised score with a +confidence interval — and *"indistinguishable from the classical method"* is a +first-class outcome, not a rounding error. + +```bash +pip install stadion-rl +``` + +The import package is `stadion`; the distribution carries the `-rl` suffix +because PyPI's `stadion` belongs to an unrelated causal-modelling package. + +## Why another gym + +The environments an agent can be trained and measured on today are mostly code, +browsers and SaaS workflows. They share a problem: **nobody knows what the right +answer was.** A score of 61% on such a benchmark tells you an agent beat other +agents, not whether it did well. + +στάδιον is both the racecourse and a unit of length. These tasks are picked so +that they can be both: each is a decision a business makes thousands of times a +day, each has a textbook method a practitioner would reach for, and each is small +enough that the true optimum can be *computed* by backward induction rather than +approximated. So every run puts three numbers on one scale. + +## What a run looks like + +`examples/scarcity.py` prices higher when stock is scarce relative to the time +left, lower when it is piling up. The right shape, the rule most people write +first, and obviously better than a fixed price: + +``` +pricing / scarcity + instances 40 + agent 22.503 + classical 24.490 + optimum 26.006 + score -1.310 (0 = classical, 1 = optimum; + one point = 1.516, 6.2% of the classical result) + agent - classical: -1.987 [-3.038, -1.027] (-8.1%) -> worse + agent - optimum: -3.503 [-4.403, -2.670] (-13.5%) -> worse + optimum - classical: +1.516 [+1.265, +1.761] (+6.2%) -> better +``` + +It loses to a plain fixed price by 8.1%, and the interval does not touch zero. On +a leaderboard against other adaptive agents it might have looked fine. + +That is the whole argument for the project: the scale has a top, so a result can +be wrong rather than merely mid-table. + +## Getting started + +```python +import stadion + +task = stadion.get("pricing") +report = stadion.evaluate(task, my_agent, instances=30, episodes=20) +print(report.summary()) +``` + +From the shell: + +```bash +stadion brief pricing --seed 3 +stadion run inventory --agent optimum --instances 30 +``` + +Next: [the tasks](tasks.md) and how much room each one actually has, or +[writing an agent](agents.md). + +`Python 3.10+` · [PyPI](https://pypi.org/project/stadion-rl/) · +[repository](https://github.com/DrobyshevDev/stadion) · MIT diff --git a/docs/protocol.md b/docs/protocol.md new file mode 100644 index 0000000..b6a37e6 --- /dev/null +++ b/docs/protocol.md @@ -0,0 +1,47 @@ +# The protocol + +What the agent is given, what it is not, and how the interval is built. + +## Instances are generated, not stored + +`task.instance(seed)` draws the demand level, the cost structure and the horizon +from a documented distribution. The numbers an agent is asked about did not exist +before the run, so they cannot have been memorised from a public dataset. + +## The agent is told everything the baseline is told + +The brief states the instance's full parameters, because the classical rule is +built from those same parameters. Withholding them would not make the comparison +harder — it would make it dishonest. + +## What the agent does not get is the tuning budget + +Where the classical rule has a free parameter, it is fitted by search over +practice episodes on seeds that never appear in the evaluation set. The agent +reads the brief once and plays. + +A draw against a tuned classical rule is therefore a real result, not a +consolation. + +## Everything is paired + +Agent, baseline and optimum see the same instances and the same episode seeds, +and the interval is a bootstrap over instances. + +One caveat is worth stating plainly: NumPy's Poisson sampler consumes a variable +amount of the random stream, so once two policies diverge their demand paths +diverge too. Pairing removes between-instance variance, which is the large term, +but not within-episode noise — which is why each instance is averaged over +several episodes before the arms are compared. + +## Sometimes the ceiling is a draw + +On some instance families the textbook rule is already indistinguishable from the +optimum. There the normalised score has no denominator, and the report says so +rather than dividing by a small number and reporting a dramatic figure. + +```python +report.degenerate # True when the classical method is already optimal +``` + +A benchmark that hides this is selling a race that cannot be won. diff --git a/docs/requirements.txt b/docs/requirements.txt new file mode 100644 index 0000000..80ab9f5 --- /dev/null +++ b/docs/requirements.txt @@ -0,0 +1,3 @@ +# Building the documentation only. Kept out of the package's own extras so +# installing stadion never pulls a site generator. +mkdocs-material>=9.7.7 diff --git a/docs/tasks.md b/docs/tasks.md new file mode 100644 index 0000000..0f8c795 --- /dev/null +++ b/docs/tasks.md @@ -0,0 +1,65 @@ +# The tasks + +Six decisions, each with a textbook method a practitioner would reach for and an +optimum small enough to compute rather than approximate. + +| Task | The decision | Classical method | Optimum from | +|---|---|---|---| +| `inventory` | how much stock to order each day against Poisson demand | analytic base-stock (newsvendor critical fractile) | backward induction over on-hand stock | +| `pricing` | what price to post each period for a perishable stock with a deadline | the strongest fixed price, tuned by search | backward induction over remaining stock | +| `queueing` | admit or reject each arriving job into a finite buffer | the strongest fixed value threshold, tuned by search | backward induction with the job value integrated in closed form | +| `energy` | when to charge and discharge a battery against a daily price cycle | the strongest fixed price threshold, tuned by search | backward induction over the charge lattice | +| `supply-chain` | how much to order at two echelons, a period before it can help | per-echelon base-stock, tuned by search | backward induction over the collapsed two-dimensional state | +| `joint-pricing` | what to charge and how much to restock, decided together | the strongest static price-and-target pair, tuned jointly | backward induction over on-hand stock | + +Environments and the classical policies come from +[decisionrl](https://github.com/DrobyshevDev/decisionrl) unmodified, so the +opponent is the same code that library ships and tests, not a re-implementation +written to lose. + +The last two have a continuous action space, which every player here meets as the +same numbered menu — nine power settings for the battery, twenty-five order pairs +for the chain. The classical rule's real-valued action is snapped to that menu, +and its free parameter is tuned *through* the snap, so it is optimised for the +game it actually plays rather than for a continuous relaxation of it. + +## How much room is actually in each one + +Measured over 40 instances × 20 episodes. Reproduce with: + +```bash +stadion run --agent classical --instances 40 --episodes 20 +``` + +| Task | Classical | Optimum | Headroom | 95% interval | +|---|---:|---:|---:|---| +| `inventory` | 204.141 | 204.890 | **+0.4%** | [+0.561, +1.031] | +| `joint-pricing` | 110.099 | 114.812 | +4.3% | [+3.794, +5.656] | +| `pricing` | 24.490 | 26.006 | +6.2% | [+1.265, +1.761] | +| `queueing` | 21.911 | 25.611 | +16.9% | [+3.418, +3.988] | +| `supply-chain` | −37.532 | −31.027 | +17.3% | [+5.505, +7.493] | +| `energy` | 16.778 | 21.234 | **+26.6%** | [+4.221, +4.690] | + +The spread is the point. A price threshold with no forecast leaves a quarter of +the battery's value unclaimed, because it cannot decide to arrive at the evening +peak full. At the other end the newsvendor formula is within half a percent of +the exact optimum — there is almost nothing to win on `inventory`, and an agent +that reports a large improvement there has a bug, not a policy. + +!!! note "What a wide benchmark would hide" + + A benchmark whose tasks all have generous headroom has quietly selected for + problems where the classical answer is bad. Publishing the spread is what + makes that visible. + +## joint-pricing, and the claim it corrected + +`joint-pricing` is the case that changed our mind about something. Its +environment is built around a coupling — the right price depends on how much +stock is on the shelf, so no fixed price can be right — and that is true. Priced +out, letting the price answer to the stock is worth 4.3%. Real, measurable, and a +good deal smaller than "no static rule is right" suggests. + +The number is sensitive to how fine the price menu is — 2.2% over six prices, +3.2% over eight, 3.6% over twelve — which is why the menu is set where that has +mostly stopped moving rather than where the headline looks best. diff --git a/docs/verify.md b/docs/verify.md new file mode 100644 index 0000000..43e95d6 --- /dev/null +++ b/docs/verify.md @@ -0,0 +1,31 @@ +# Checking the harness + +Every score here is measured *against* the optimum. That makes the optimum the +one number nothing in an ordinary run would catch being wrong: a dynamic program +with a mistaken recurrence moves the whole scale, and every task still reports a +plausible-looking result. + +One command exists for that: + +```bash +stadion verify +``` + +It computes each dynamic program's analytic value and, separately, simulates the +policy that same program emits. Two independent routes to one number; they have +to agree within Monte Carlo error. + +A dynamic program that quietly disagrees with its own policy is the failure this +is built to catch, and it runs in CI on every push. + +## Why this is the check that matters + +The rest of the suite tests behaviour that a wrong answer would visibly break — +an environment that rejects an illegal action, a report that carries an interval. +The optimum is different: it is the definition of the top of the scale, so +nothing above it exists to compare against. + +Two derivations that have to meet is the only way to check a number with no +external reference. It is the same reason a published benchmark figure here is +reproduced by a command in the repository rather than quoted from a run someone +did once. diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 0000000..b114f86 --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,71 @@ +site_name: stadion +site_description: >- + A proving ground for operational decisions. An agent is scored against the + tuned classical method and the exact optimum, with a confidence interval. +site_url: https://drobyshevdev.github.io/stadion/ +repo_url: https://github.com/DrobyshevDev/stadion +repo_name: DrobyshevDev/stadion +edit_uri: edit/main/docs/ +copyright: MIT. + +theme: + name: material + language: en + # The theme otherwise pulls Roboto from Google Fonts on every page load. The + # typography here is a system stack set in theme.css, so that request buys the + # reader nothing and costs them a third party. + font: false + features: + - navigation.tracking + - navigation.sections + - navigation.top + - content.code.copy + - content.action.edit + - search.suggest + - search.highlight + - toc.follow + # One scheme, dark, as everywhere else in the organisation. A toggle means + # designing the whole thing twice, and the second design is the one nobody + # checks. + palette: + scheme: slate + primary: black + accent: indigo + icon: + repo: fontawesome/brands/github + +plugins: + - search + +markdown_extensions: + - admonition + - attr_list + - def_list + - footnotes + - md_in_html + - tables + - toc: + permalink: true + toc_depth: 3 + - pymdownx.details + - pymdownx.superfences + - pymdownx.highlight: + anchor_linenums: true + - pymdownx.inlinehilite + - pymdownx.arithmatex: + generic: true + +extra_css: + - assets/theme.css + +nav: + - stadion: index.md + - The tasks: tasks.md + - Writing an agent: agents.md + - The protocol: protocol.md + - Checking the harness: verify.md + +extra: + social: + - icon: fontawesome/brands/github + link: https://github.com/DrobyshevDev/stadion