An agentic pipeline that authors a course from a topic and an audience, as a LangGraph state machine with a deterministic validator and Langfuse tracing.
This is the third implementation of the same pipeline:
| Runtime | Sequencing lives in | |
|---|---|---|
| stay-learning | VS Code Copilot .github/ |
an agent's instructions |
| stay-learning-mastra | Mastra workflows + a CLI | three workflow graphs and two ledgers |
| this repo | LangGraph + a CLI | one graph |
What does not change is the part that matters: courses are files. courses/<slug>/
is a tree of YAML and Markdown that git can diff, a person can edit, and the viewer in
web/ can render without this pipeline present. Nothing about a finished course depends
on LangGraph.
- Statistical Validation
- Setup
- Reaching a model
- Running the pipeline
- State and staleness
- Observability
- Validation
- Licence
Python is the native language of the deterministic layer. The original validator was already 1666 lines of Python; the Mastra port re-wrote it in TypeScript and gained nothing. Python reopens the door to statistical validation that neither predecessor can express — punkt sentence segmentation, lemmatised term matching, readability banding, corpus-frequency jargon detection. That layer is designed and not built: see statistical-layer-design.md. This release ships strict parity with the original validator first, because a measurable baseline is what makes the next layer's effect legible.
Requires uv. Python 3.13 is pinned in .python-version;
3.14 is deliberately excluded until every dependency publishes wheels for it.
uv sync
uv run pytestThe read-only commands need no API key at all, and neither does a run a gate stops:
uv run course list # every slug under courses/
uv run course validate <slug> # every mechanical defect
uv run course status <slug> # what is stale, and which input moved
uv run course models # which model is behind each role
uv run course settings # every knob and its current value
uv run course graph # the compiled topologyTwo paths, and the difference is one variable. By default every role resolves to
Anthropic and wants ANTHROPIC_API_KEY. With STAY_MODEL_URL set, roles resolve to an
OpenAI-compatible client pointed there instead and no provider key is needed:
node ../stay-learning-mastra/tools/claude-proxy/server.ts # in one terminal
export STAY_MODEL_URL=http://127.0.0.1:8787/v1That proxy is the Mastra port's, and it serves /v1/chat/completions from a local
Claude Code session. It runs each child with the agent scaffolding switched off, so
function_calling cannot work through it — there are no tools to call. It does
implement response_format: json_schema, mapped onto the CLI's own --json-schema,
which is why this pipeline asks for json_schema on both paths.
course new --topic "how DNS resolution works" \
--audience "backend developers" \
--experience beginner \
--time "90 minutes total, 30 minute sessions"
# → suspends at the design gate
course resume how-dns-resolution-works --approved true
course plan how-dns-resolution-works # → suspends at the plan gate
course resume how-dns-resolution-works --approved true
course write how-dns-resolution-works # the wave
course repair how-dns-resolution-worksnew needs four facts that cannot be inferred — topic, audience, experience, time.
Leaving one out suspends at intake and asks, rather than guessing: an invented
constraint becomes a real one for every stage after it. --topic is the exception,
because it names the thread and a course has to be called something before it can ask
you anything.
course build runs the whole thing end to end with the gates auto-answered — the
capability neither predecessor has. v1 needed a live chat session; v2 needed three
processes with a person between them.
Two points stop and wait for a person, placed where the cost changes. Everything before gate 1 is four model calls; everything after it is one per module and then three per lesson. Gate 2 sits in front of the writing for the same reason: a section in the wrong place costs one plan to fix before it and a whole lesson to fix after.
A gate is not there to check the rules — those are checked mechanically and the gate shows you the count. It is there for the one question a validator cannot answer: whether this is the course you wanted.
Approvals are written to .state/gates.json signed against the hash of every file
they covered, so an approval stops counting in three cases: never answered, answered
no, or answered about files that have since changed. The third is the one worth having.
Once every plan exists, no lesson needs anything from another, so the whole course is
written at once. What makes that safe is not optimism about file locking: no two nodes
in the wave write the same file, and the shared ones — course.yaml, glossary.yaml,
.state/ — are not written by wave nodes at all. They are written once, afterwards, by
the step that gathers the results.
Planning stays out of the wave because the terminology budget is being allocated rather than checked. A module is planned in one call, so a term cannot be introduced twice by construction.
Only leaf artefacts are repaired — prose, exercises, quizzes, the capstone. A defect in anything approved at a gate is reported and never auto-fixed, because everything downstream was written against what it said.
It stops on any of: nothing left, nothing an agent can fix, a round that failed to reduce the count, or the two-round cap. The third matters most — a model asked twice for the same correction usually returns the same text, and without that check the loop spends its whole budget confirming a stalemate.
Staleness is derived, not declared. v1 stored current | stale | missing in
course.yaml and required every agent to keep it honest; a stored flag can disagree
with the disk in either direction and a lost flip is invisible. Here each artefact
records the hashes of its direct inputs, so course status answers which input moved:
how-dns-resolution-works [draft]
edited curriculum.yaml (changed outside this pipeline)
stale outcomes.yaml <- curriculum.yaml
gates the design gate was approved on 2026-08-13, but 1 file it covered
changed afterwards (curriculum.yaml).
Only direct inputs are recorded, so a regeneration that reproduces its input stops the
cascade there — correcting a typo in curriculum.yaml no longer invalidates every
lesson when the outcomes come back byte-identical.
./scripts/langfuse.sh up # generates .env, waits for /api/public/health
./scripts/langfuse.sh status
./scripts/langfuse.sh down # never -v; only `reset` destroys volumes, and it asksSelf-hosted Langfuse v4, vendored and bootstrapped headlessly — the keys are generated
into .env before first boot, so nobody visits a signup form. One session per course,
so a course's whole history is one thread rather than a dozen unrelated traces.
Validator defect counts are pushed as scores on every run, which is the point of having it. An LLM scorer gives an opinion that costs money and moves between runs; the validator gives a count that costs nothing and is reproducible to the defect. That turns Langfuse into a regression dashboard for the authoring method: whether tomorrow's run of the same course produces fewer defects than today's.
Tracing degrades to no-ops when Langfuse is not running, and the pipeline contains no
if tracing: branch anywhere — callbacks() returns [], which LangGraph accepts, so
the degradation path is the same path and cannot rot untested.
Reading scores back is GET /api/public/v3/scores. The v1 and v2 paths still exist on a
v4 deployment and answer with a deprecation notice rather than the data, which is easy
to mistake for "nothing was ingested":
set -a; . ./.env; set +a
curl -s -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" \
'http://localhost:3000/api/public/v3/scores?limit=60'uv run course validate runs every mechanical rule over a course and exits non-zero if
any fails. It is a strict parity port of the original validate.py: same codes, same
messages, same regexes, same WARN/FAIL semantics, same exit codes.
Parity is asserted, not asserted-to. tests/parity/ vendors the original validator and
the fixture course it was developed against, then compares defect multisets and pass
counts run for run. One divergence is sanctioned and named in stay/constants.py
(GLOSSARY_SORT_CASEFOLD): the original sorted glossary terms by code point, so every
capitalised term sorted ahead of every lowercase one and correctly ordered files failed.
The Mastra port fixed that; so does this one.
See the upstream repositories.