Skip to content

Build provenance: landing footer, about-this-build page, build_info.json (WP7) - #26

Merged
thorwhalen merged 5 commits into
masterfrom
v2/provenance
Sep 15, 2026
Merged

thorwhalen merged 5 commits into
masterfrom
v2/provenance

Conversation

@thorwhalen

Copy link
Copy Markdown
Member

Every site built by epythet now says which code, version and tools produced it, so a reader can tell whether the docs match the repository and the installed package, and a maintainer can see whether the latest push has been published. WP7 of the v2 roadmap in #16. Closes #7.

What a site gets

  • Landing page: one small gray line at the end of the README content, theme independent: built 2026-09-15 08:20 UTC from 3bee721+dirty (master) · epythet 0.2.5 · about this build. The commit links to GitHub when the remote is known. Nothing else on the landing page, nothing in the navigation.
  • about-this-build.html: an orphan page (reachable from that link only) with the full diagnosis: package name/version and which file it came from; full commit, branch, tags at HEAD, working-tree state, remote; GitHub Actions context with a link to the run; epythet/Sphinx/docutils/Python versions; the configuration as resolved (theme, accent, api_generator, ignore, agent_outputs, aggregates, ai_artifacts); documented-module and documented-object counts; the latest PyPI release and whether the documented version is the same, behind or ahead; the shell lines that reproduce the build. A warning admonition states plainly when the docs and the package may be misaligned (dirty tree, version behind or ahead of PyPI, CI checkout differing from HEAD, no git at all).
  • build_info.json at the site root: the same data with stable keys and schema_version: 1. Listed in llms.txt under a ## Build heading and quoted at the top of the <pkg>.md aggregate.

The seam

[tool.epythet] provenance = true | false | "minimal" (default true; "minimal" keeps the footer line and the JSON, no page), and provenance_template (a project-relative file with the same contract as ai_artifacts_template, the seam WP8's user-level snippets will feed). EPYTHET_PYPI_CHECK=0 skips the network lookup (the test suite sets it), SOURCE_DATE_EPOCH fixes the build time. epythet build-info DIR prints the record without building.

Provenance never fails a build: no git, no git binary, no commits, no remote, no network, an unknown theme, garbage in the handover variable, all degrade to null fields plus a warnings entry, and the build prints one line. Remote URLs are stripped of any user:token@ part before anything is written. The dirty check excludes the docs dir, because the build itself rewrites a committed docsrc/.

How it is wired

epythet.build.build() collects the record once before Sphinx runs (for the html target), writes the marker-guarded docsrc/about-this-build.md, and hands the record to the Sphinx process through EPYTHET_BUILD_INFO. epythet.sphinx_conf collects it itself when the variable is absent, so a direct sphinx-build still gets the footer and the JSON. The extension appends the footer to the rendered body of the root document on html-page-context (every theme renders body, so no per-theme template), fills the counts on the about page from the Sphinx environment, and writes the JSON at build-finished for html builders only. After the build, build() references the JSON from llms.txt and <pkg>.md.

Not done, deliberately: a footer on every page. None of the registry themes exposes a common footer block, so that would be per-theme template work; the landing page carries the line.

Fleet impact, and one change to the action

The action installs epythet>=0.2,<0.3, so every fleet site gets the line, the page and the JSON on its next push.

The adversarial review caught a trap that would have inverted the feature for the whole fleet: the wads CI publishes to PyPI, then pushes a version-bump commit marked [skip ci], and the Pages job ran afterwards on the event SHA, the commit before the bump. Every site would have reported "documented version behind PyPI" forever (today's live sites already show it: the epythet site says 0.2.5 while PyPI has 0.2.6). The action therefore gained a step, right after checkout, that fast-forwards to the tip of the default branch before building, guarded by ref_name == default_branch and never failing the job. Sites are now built from the bump commit: the released version, with its tag. The record's CI note only fires when the event SHA is not in the built commit's history.

Review also led to: every repository value escaped on the about page (a branch name is user input); nothing local in the published record (credentials and users stripped from remotes, path-shaped remotes dropped, git errors scrubbed of paths, reproduce lines name the clone by its remote); stale page/JSON pruned from the output directory when provenance is turned off; the PyPI lookup bounded by a wall-clock timeout and skipped for non-html builders; hermetic, clock-independent tests. The smoke build in tests/test_build.py now runs inside a dirty git repo with a tag and a remote and asserts the footer, the page, the JSON, llms.txt and the aggregate; test_provenance.py covers the record without a build (clean/dirty/tagged/detached repo, CI env, no git, credential stripping, schema key set, PyPI degradation, config coercion). CLI usage goldens were updated deliberately for the new build-info command.

Refs #16.

…son (WP7)

Every html build now says which code, version and tools produced it:

- epythet/provenance.py collects the record once per build (git commit,
  branch, tags, dirty flag with the docs dir excluded, sanitised remote,
  GitHub commit URL; GitHub Actions context; epythet/Sphinx/docutils/Python
  versions; resolved config; latest PyPI release and relation; alignment
  notes; reproduce lines). Never fails the build: missing git or network
  degrade to null fields and one warning.
- The extension appends a one-line footer to the landing page (theme
  independent), fills documented-module/object counts on the about page
  and writes build_info.json (schema_version 1) at build-finished.
- build.py writes the orphan about-this-build.md before Sphinx runs,
  hands the record over via EPYTHET_BUILD_INFO, and afterwards references
  the JSON from llms.txt and the top of <pkg>.md.
- [tool.epythet] provenance = true | false | "minimal" (footer + JSON, no
  page); EPYTHET_PYPI_CHECK=0 skips the lookup. `epythet build-info DIR`
  prints the record; CLI goldens updated deliberately.
- Tests: git fixture (clean/dirty/tagged/detached), CI env, no git, schema
  key set, PyPI degradation, config coercion; the smoke build now runs in
  a dirty git repo and asserts footer, page, JSON, llms.txt and aggregate.
- README paragraph, epythet-setup skill, CLAUDE.md map.

Refs #16. Closes #7.
- Publish action fast-forwards to the default branch tip before building, so
  fleet sites are built from the wads version-bump commit (PyPI version and
  tag match); the CI note fires only when the event SHA is not in HEAD's
  history.
- About page escapes every repository value (ref names, tags, remotes) as
  inline HTML; the footer already did.
- provenance = false / "minimal" remove epythet's own about page source and
  prune stale outputs (page, twin, JSON) left in _build/html.
- epythet_provenance config value declares its types (no Sphinx warning,
  no DR034 finding in `epythet validate` for "minimal").
- SOURCE_DATE_EPOCH honoured; the test suite pins it, so rebuilds are
  byte-identical and the stability test no longer depends on the clock.
- Test git fixtures isolated from the developer's global config.
- git output decoded as UTF-8 with replacement (Windows locales).
- Nothing local in the published record: credentials and users stripped
  from remotes, path-shaped remotes dropped, git errors scrubbed of paths,
  reproduce lines name the clone by its remote.
- github/gitlab copy targets excluded from the dirty check.
- Malformed EPYTHET_BUILD_INFO ignored; page hooks never raise.
- Existing generated docsrc/.gitignore refreshed (about-this-build.md).
- PyPI lookup bounded by a daemon-thread timeout; skipped on the in-Sphinx
  fallback path (non-html builders, validate's render pass).
- provenance_template config key (same contract as ai_artifacts_template),
  the WP8 seam; generated-file helpers shared with scaffold.

Refs #16.
- Table cells and code spans on the about page are Markdown-inert: "|",
  backticks, "*", "_" and "[" in repository values become entities; the
  smoke build asserts on the rendered row for a tag named "t|pipe".
- A hand-written about page (no marker) survives provenance = false: the
  outdir is pruned only when no source file exists; github/gitlab copy
  targets are pruned the same way after the copy.
- Custom templates: front matter stays first (marker goes after it),
  orphan: true is guaranteed, any rendering error is a ConfigError.
- docsrc/.gitignore: an earlier generated version is replaced, an edited
  generated one gets the missing entries appended, hand-written kept.
- Action step reads ref names through env (no script injection) and
  fetches tags with the branch.
- dirhtml builds carry the page and JSON, with a builder-correct link.
- Windows drive-letter remotes are paths, not hosts; trailing slash dropped.
- Test suite pins GIT_CEILING_DIRECTORIES to its basetemp.
- An interrupted PyPI worker thread is an error, not a "checked" result.

Refs #16.
… Windows ref-name limits

GITHUB_ACTIONS is unset for the session so the runner's SHA and ref never
reach a record; the pipe-tag fixture and the ref-name escaping test are
skipped on Windows, whose git refuses those characters in ref names.
@thorwhalen
thorwhalen merged commit 5c7b20d into master Sep 15, 2026
12 checks passed
@thorwhalen
thorwhalen deleted the v2/provenance branch September 15, 2026 09:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Include version and docs generation date.

1 participant