Skip to content

Stale-asset republication: default templates to cache-type execution, or prune html inside restore-jupyter-cache #160

Description

@mmcky

The bug class the templates would carry into the migration

Restoring a full _build and building over it republishes files that have been deleted from source. Sphinx copies html_static_path assets into _build/html/_static but never removes ones that no longer exist in source, so a dataset or asset deleted on Tuesday keeps being served by every preview and publish that restores Monday's cache — until a clean cache rebuild happens to intervene. This is not hypothetical: it made deletion verification unreliable across the Track C dataset migration, and QuantEcon/workspace-lectures#41 is currently rolling a consumer-side jb clean . --html step across 9 lecture repos as the fix for the dawidd6-artifact world (validated live in QuantEcon/lecture-python-advanced.myst#374 — a restored cache that physically contained three deleted files produced a preview that 404'd all three while serving controls at 200).

The migration surface here inherits the same bug: templates/ci.yml pins cache-type: 'build' (full _build, html included), and neither restore-jupyter-cache, build-lectures, nor the docs prune or mention stale assets. A repo cutting over by template today lands back in the pre-fix state, and the per-repo workflow prune steps from the rollout would be removed at exactly that moment.

The structural fix is already half-built

restore-jupyter-cache supports cache-type: 'execution' (.jupyter_cache only). Run that way, every consumer rebuilds _build/html from its own checkout: the stale-asset class cannot exist, no prune is needed anywhere, and the guarantee is immune to which vintage of cache gets matched (relevant because prefix-matching means an old entry can be restored silently). The restored html contributes nothing to build speed anyway — a fresh checkout's mtimes force Sphinx to rewrite every page — so the real cost of execution-only mode is redoing the pdflatex/jupyter render passes, not notebook execution.

Proposal

  1. Flip the site-publishing templates (ci.yml, publish.yml) to cache-type: 'execution' as the default, with a short note in the template explaining why (deletions must take effect at the next build, not the next weekly cache).
  2. For any repo that measurably needs build mode (render passes too slow), add a prune-html input to restore-jupyter-cache — default true — that runs the equivalent of jb clean . --html after restore, so the safeguard lives in one composite action rather than N workflow files. (jb clean needs no _config.yml at the restore path; verified with jupyter-book 1.0.4.post1 that it removes exactly <path>/html — in _build terms: html only, .jupyter_cache/latex/jupyter untouched.)
  3. Note in the migration guide that repos carrying the interim workflow-level prune step from the rollout should drop it at cutover — the action supersedes it.

Filed from the discussion on QuantEcon/lecture-python-programming#616 (repo 2 of the rollout), where the interim prune is landing now; the rollout continues as the bridge until cutover.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions