fix(ci): retry a dependency install only when the transport failed

[nilsonfh]

What

bd syntgrid-ahx. CI jobs keep dying in the dependency-install step on a cut package fetch, and a red check that no change caused trains readers to rerun rather than read.

tools/pip_install_retry.py retries an install only when the failure is a transport error, and the five install steps in ci.yml and pages.yml now route through it.

The measurement

Six failures on 2026-09-16, none of them a failing test, every one green on a rerun of the same commit:

Where	Jobs that died
PR #43	Documentation build
PR #54, first attempt	test + Documentation build
main @ 39df523	Governed project contracts
PR #56, first head	test
PR #57, run 35151372093	test + Documentation build

The byte counts identify the cause: identical across the jobs of one run, different between runs — PR #57 both jobs IncompleteRead(214027 bytes read, 3458 more expected), PR #54 both 230547/273, main 9580/278. One upstream response is cut and whichever jobs are installing at that moment die together, so it is the fetch, not any job.

Why pip's own retries do not cover it: every site already runs with pip's defaults and setup-python's cache: pip. Those cover a connection that fails to open; a body cut mid-download surfaces from the install phase as OSError. The unit that has to be retried is the install.

What it deliberately does not do

It does not retry a resolution error, a missing extra, or a build failure — those are this repository's bugs and must fail on the first attempt, in seconds. TRANSIENT_MARKERS is a narrow, stated list of transport failures, and a test asserts no resolver wording can enter it.

Upgrading pip itself is left as a plain pip install --upgrade pip, so the retry cannot mask a broken pip.

Why a tool rather than a shell loop in the YAML

Five call sites would be five copies of the same loop, and inline workflow logic cannot be tested — the same reason tools/ci_main_status.mjs exists. tests/test_pip_install_retry.py drives every branch against a real child process, not a mock, since what the tool classifies is a subprocess's combined output.

Verification

• 8 tests: the classifier, a cut fetch retried into success, a cut fetch on every attempt still failing with the last exit code, and the branch that matters most — a genuine failure not retried and not slept on.
• Wiring gated both ways: no workflow install may bypass the tool, and the tool must actually appear in both workflows. Without the second, CI could silently stop calling it while the tool kept passing its own tests.
• Real pip, not only the fake: python tools/pip_install_retry.py --attempts 2 -- --dry-run --no-deps --quiet wheel exits 0, and an invocation with nothing to install is rejected by argparse.
• The workflow-parsing gates still pass (test_mypy_ratchet, test_ci_main_status), both YAML files parse, and the docs/hygiene gates pass with tools/README.md updated to thirteen scripts plus its index row.
• Full suite: 2215 passed, 76 skipped (the operator-verified heavy studies, whose outputs are gitignored).
• black, isort, flake8 clean.

Not in this PR

The deeper option — baking the dependency set into the self-hosted runner image — is not taken: that image rebuilds only when its Dockerfile changes, so the baked environment would go stale on every pyproject.toml edit. Its own header states the principle it follows ("the runtime a workload needs belongs in the image that runs it") for the OS-level runtime, which is a different question from Python packages that change with the tree.

🤖 Generated with Claude Code