paces

Turn instructional media into structured, interactive learning material. Put it through its paces.

Take a video of someone teaching something — a dance routine, a kata, a recipe — plus, optionally, notes and a steering prompt. paces segments it into named steps, builds a structured step document (an AST for step-by-step instruction), and renders that into learning material: a practice page with counts and deep links today, other guides later.

pip install paces

Quick example

from paces import segment, to_document, render_html

seg = segment(
    "https://youtu.be/q_TUyxUhoEw",
    steps=[
        ("Mise en place", 2),
        ("Pas pieds pointe et ronde", 6),
        ("Soleil avec les bras", 4),
        ("Déhanchés", 8),
    ],
    grid={"unit": "eight", "subdivisions": 8, "tempoBpm": "129.2", "origin": "51.2"},
)
doc = to_document(
    seg,
    doc_id="que-calor",
    title="Chorégraphie Que Calor",
    source="https://youtu.be/q_TUyxUhoEw",
)
open("page.html", "w").write(render_html(doc))

The page lists every step with its counts, links each one back into the video (both the at-tempo run-through and the slow breakdown, when both are known), and — because the document carries a metric grid — includes a count-along transport that paces you through the routine at the measured tempo.

Same thing from the shell:

paces segment VIDEO_URL --steps steps.json --grid grid.json --output seg.json
paces to-document seg.json --source VIDEO_URL --title "My routine" --output document.json
paces suggest-excerpts document.json --output document.json   # mark each block's loop window
paces derive document.json --media routine.mp4   # real loop clips + gifs + posters
paces render document.json --output page.html

derive (pip install paces[media]) cuts a loopable mp4, a palette-quality gif and a poster for every excerpt-bearing span, writes them to media/ next to the document, and the practice page embeds them as looping clips. Crop recipes persist in a hand-overridable document.recipes.json sidecar; the subject_locator= seam (default: no crop) is where pose-based auto-crop plugs in. Design record: docs/adr/0005-media-derivation.md.

Auto-crop to the people in frame with pip install paces[pose], then paces derive doc.json --media routine.mp4 --subject-locator paces.pose:rtmlib_pose (or subject_locator=paces.pose.rtmlib_pose from Python). It probes each excerpt window at ~5 fps and reports every person it sees; the crop policy stays in the core, so two people in frame get one box around both. The extra itself is rtmlib (Apache-2.0, pure Python) and onnxruntime (MIT), with model weights downloaded on first use. Detection is YOLOX (Apache-2.0); what is barred from every extra here — rather than quarantined into one — is the ultralytics distribution, which is AGPL-3.0.

What it pulls in is a different question, and worth stating plainly: rtmlib requires opencv, and opencv's bundled FFmpeg is GPL-3.0-or-later on macOS wheels of the versions measured (4.12.0.88 / 4.13.0.92) (built --enable-gpl with libx264/libx265) though LGPL-2.1-or-later on manylinux and Windows — a per-version fact, not a per-platform one: the 5.0.0.93 macOS x86_64 wheel ships no FFmpeg at all. This is measured from the shipped binaries — the wheels' own LICENSE-3RD-PARTY.txt never mentions x264. paces[media] already brings such a wheel, so [pose] adds a second copy rather than a higher tier. Worse: rtmlib's own metadata requires both opencv-python and opencv-contrib-python, unpinned, so a plain pip install paces[media,pose] ends up with two distributions owning one cv2 — harmless until either is uninstalled, at which point the survivor's cv2 can be left with files missing (issue #20). rtmlib only calls plain cv2 APIs (VideoCapture, dnn.readNetFromONNX, drawing helpers — nothing contrib-only), so [media]'s opencv-contrib-python already covers it; to keep the closure single-provider, install in two steps instead of one:

pip install "paces[media]"                    # opencv-contrib-python, the fleet's one cv2
pip install --no-deps "rtmlib>=0.0.16,<0.1"   # skip rtmlib's own opencv-* re-declaration
pip install onnxruntime tqdm                  # rtmlib's other real deps (numpy already arrives via opencv)

paces.pose.check_pose_requirements() reports what you have, and names the repair command if both providers are already present.

How it thinks

Analysis and rendering are separate phases with a serialisable document between them — like a parser emitting an AST and a backend interpreting it. Renderers depend on the document, never on the analyser.

Segmentation is a seam, not a stage. segment(media, segmenter=...) — segmenters are registered capabilities, the default follows from what is present, and "the user typed the boundaries" is a first-class segmenter, not a fallback. A segmenter that cannot name steps returns honest unnamed boundaries (flags: ['naming-abstained']) rather than inventing names.

The document keeps what the learner actually counts. A dance step lasts "4 eights", not "14.86 seconds" — seconds are derived from the metric grid (tempo + origin), never stored. A step can have several source spans (the run-through and the breakdown are the same step seen twice). Uncertainty is content (OpenQuestion), and human edits are protected from regeneration (Lock).

The evidence layer

The document is the contract; the machine evidence behind it — the speech/music split, the metric grid, the beats, the transcript, the step candidates, the crop recipes, and the lineage between them — lives in a lacing store, and the document is a projection of it. The document is derivable from the store; the store is not derivable from the document.

from lacing import MemoryStore                 # pip install 'paces[lacing]'
from paces import segment, from_store, to_store
from paces.model import dumps_document

grid = {"unit": "eight", "subdivisions": 8, "tempoBpm": "129.2", "origin": "51.2"}
asset_sha256 = "a" * 64                        # lacing.hash_file(video) in real use
seg = segment(None, steps=[("Mise en place", 2), ("Déhanchés", 8)], grid=grid)

store = MemoryStore()                          # or SqliteStore('project.annot')
guide = dict(doc_id="que-calor", title="Que Calor", domain="dance",
             source="https://youtu.be/q_TUyxUhoEw")

write = to_store(seg, store=store, asset_id=asset_sha256, **guide)

dumps_document(from_store(store, asset_id=asset_sha256)) == dumps_document(write.document)
# True — the projection is exact

to_store(seg, store=store, asset_id=asset_sha256, **guide).written
# 0 — a re-run of the same analysis writes nothing

The store is injected, never constructed for you: MemoryStore() in tests, a SqliteStore for a project sidecar, any dol store that conforms. Annotation ids are derived (uuid5) from the evidence they stand for, so a re-derivation is the same annotation — which is what keeps was_derived_from lineage resolving and lets write.document's Origin.annotationId be committed. Rows whose value digest did not change are left completely alone, so freshness does not fire on a no-op re-run.

to_store is a re-derivation of the guide, not a merge into it: a step that no longer exists is dropped rather than left to be resurrected by the next projection. Pruning only ever touches this asset, this doc_id, and the guide's own tiers. Evidence about the asset — the speech/music split, the beats, the transcript — is shared by every guide over that asset and is never pruned, so one guide can't delete what another's lineage points at; a re-measure adds a row under a new content-derived key and the old one stays. prune=False opts out of the rest.

Analysis a Segmentation cannot carry rides along as keywords: passes= (the speech/music split), beats=, transcript=, cues=, recipes=. What stays on the document and never flows back into the store: locks, questions, artifacts, and span excerpt windows — those are human decisions, not measurements. Full table and rationale in docs/07-annotation-model.md §6.

import paces does not import lacing: the core stays pydantic-only, and paces.to_store resolves the extra on first use.

The pieces

you want	reach for
cut media into steps	segment(media, steps=..., grid=...) → Segmentation
explicit/human boundaries	segment(media, boundaries=[...], steps=[names])
use the video's own chapters	segment(media, metadata=<yt-dlp info.json>)
measure the grid from the media	segment(local_media, steps=[(name, counts), ...]) — no grid needed; tempo + structure measured, origin estimated and flagged (pip install paces[audio])
protect edits from regeneration	apply_edits(doc, patches, by="user:you") + merge_regenerated(committed, fresh)
the committed artifact	to_document(seg, ...) → StepDocument
real clips/gifs/posters for the page	derive_document(doc, media=..., doc_path=...) / paces derive (pip install paces[media])
auto-crop those clips to the people in frame	derive(..., subject_locator=paces.pose.rtmlib_pose) / --subject-locator paces.pose:rtmlib_pose (pip install paces[pose])
persist the analysis behind a document	to_store(seg, store=..., asset_id=...) / from_store(store, asset_id=...) (pip install paces[lacing])
a practice page	render_html(doc)
wall-clock times from counts	resolve(doc)
sanity checks	validate_document(doc)
what segmenters exist	capabilities() / paces list-segmenters
add a segmenter	register(Capability(name=..., gives="segmentation", target="mymod:fn", needs={...})) — a new file, nothing edited

Status

Young and moving. The document schema is validated by round-tripping a real proof of concept (an interactive dance-practice page) through it — see tests/test_roundtrip_poc.py. Media derivation (auto-cropped looping clips) and the evidence layer have landed; intrinsic segmenters (scene/beat/speech detection) are designed (see docs/) and arrive next.