From 4fecea5f5270c11c72f3767c4ebded9a32e5fd1e Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 13 Sep 2026 04:49:36 +0000 Subject: [PATCH] Harden CI: the pip bootstrap step installs from a hash-pinned set The jobs that already install everything else with --require-hashes still opened with a bare `python -m pip install --upgrade pip`. That resolve was unpinned, so the job fetched whatever pip PyPI served that minute and then used THAT pip to verify every hash below it. A hash-checked install is only worth as much as the installer doing the checking, so this was the one remaining gap in those jobs' supply chain. Nine call sites across five workflows now install from one of two new hash-pinned sets: .github/requirements/pip-bootstrap.txt pip .github/requirements/pip-build-bootstrap.txt pip + build (+ packaging, pyproject_hooks) Deliberately two files. Seven of the ten bootstrap sites in the repo do not build anything, and three of those bootstrap a throwaway venv that the E2E suites then use to prove a freshly installed wheel boots on its own. Adding `build` and its closure to those venvs would widen the dependency surface the test is measuring, which is exactly the bug an isolated-venv test exists to catch. Every job's installed set is identical to what it was before. `--upgrade` is gone because naming an exact version does the same job: pip 26.2.1 is installed over whatever actions/setup-python shipped, which is what the upgrade was reaching for. Both files join the /.github/requirements Dependabot entry, so the pins move rather than freeze. Scope: only the five PR-triggered workflows, so this PR's own CI exercises every one of the nine hunks. The remaining bootstrap sites are deliberately left for a separate change -- see the PR body. No-PRD: CI-only change under .github/; no product behaviour is touched. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011qRmtoZTzuS7CxzZ9GnsU9 --- .github/requirements/pip-bootstrap.txt | 45 +++++++++++++++ .github/requirements/pip-build-bootstrap.txt | 60 ++++++++++++++++++++ .github/workflows/api-latency-smoke.yml | 4 +- .github/workflows/ci.yml | 6 +- .github/workflows/cross-repo-handoff.yml | 2 +- .github/workflows/oss-golden-path.yml | 4 +- .github/workflows/overhead-bench.yml | 2 +- 7 files changed, 114 insertions(+), 9 deletions(-) create mode 100644 .github/requirements/pip-bootstrap.txt create mode 100644 .github/requirements/pip-build-bootstrap.txt diff --git a/.github/requirements/pip-bootstrap.txt b/.github/requirements/pip-bootstrap.txt new file mode 100644 index 0000000000..a66f0c281e --- /dev/null +++ b/.github/requirements/pip-bootstrap.txt @@ -0,0 +1,45 @@ +# Hash-pinned install set for the pip bootstrap step. +# +# Direct requirements: pip +# +# This is the `python -m pip install --upgrade pip` that opens a job, before +# anything else is installed. It was the last unpinned resolve left in the +# jobs that already install everything else with --require-hashes: the job +# would fetch whatever pip PyPI served that minute, and then use THAT pip to +# verify every hash below it. Pinning the verifier is the point -- a +# hash-checked install is only worth as much as the installer doing the +# checking. +# +# `--upgrade` is not needed and not used: naming an exact version installs +# exactly that version over whatever actions/setup-python shipped, which is +# what the upgrade was reaching for anyway. +# +# Installed with `pip install --require-hashes`, which refuses to install +# anything not listed here with a matching hash -- so the job gets the exact +# artifacts this file names, or it fails. pip has no dependencies of its own, +# so the whole closure is the one name. +# +# Each version lists the sha256 of EVERY distribution PyPI publishes for it +# (wheels for each platform, plus the sdist). pip accepts a download that +# matches any one of them, so the pin does not quietly depend on the runner +# resolving to the same wheel this file was generated against. That is what +# lets the same file serve the ubuntu/macos/windows matrix in +# overhead-bench.yml. +# +# pip 26.x requires Python >= 3.10. Every job installing this file runs on +# 3.11 (all ten call sites pin `python-version: "3.11"`). A job on 3.9 must +# not use this file without pinning a pip that supports it. +# +# Updating: Dependabot owns the routine bumps (see the /.github/requirements +# pip entry in .github/dependabot.yml) and rewrites the hashes with them. A +# pin with no updater is the frozen end of the same problem an unpinned +# install is at the other end of. +# +# Regenerating by hand, on Linux / CPython 3.11 to match the job: +# pip install --dry-run --ignore-installed --report r.json pip +# then, for each resolved name==version, take every sha256 under `urls` in +# https://pypi.org/pypi///json. + +pip==26.2.1 \ + --hash=sha256:71138adf1f4ca900cdb7d289c21b7494329f2332b6d85f0e1c42108c0384ed3e \ + --hash=sha256:f6ad667e89a1fe78046c8f13232b247200f5258d7828f3f7883d660878e0813f diff --git a/.github/requirements/pip-build-bootstrap.txt b/.github/requirements/pip-build-bootstrap.txt new file mode 100644 index 0000000000..f615af9145 --- /dev/null +++ b/.github/requirements/pip-build-bootstrap.txt @@ -0,0 +1,60 @@ +# Hash-pinned install set for the pip + build bootstrap step. +# +# Direct requirements: pip build +# +# The three jobs that build the wheel from source open with +# `python -m pip install --upgrade pip build`. That is the same bootstrap +# pinned in pip-bootstrap.txt, plus PyPA `build`, which is the tool that +# produces the artifact those jobs then test. An unpinned resolve there means +# the wheel under test was assembled by whatever build backend PyPI served +# that minute. +# +# Deliberately a SECOND file rather than folding `build` into +# pip-bootstrap.txt. Seven of the ten bootstrap call sites do not build +# anything -- three of them bootstrap a throwaway venv that the E2E suites +# then use to prove a freshly installed wheel boots on its own. Adding +# `build` and its closure to those venvs would widen the dependency surface +# the test is measuring, which is exactly the bug an isolated-venv test +# exists to catch. Keeping the two sets apart keeps every job's installed set +# identical to what it was before this pin. +# +# Installed with `pip install --require-hashes`, which refuses to install +# anything not listed here with a matching hash. Every transitive dependency +# is present because --require-hashes demands the whole closure; `packaging` +# and `pyproject_hooks` are build's, not padding. +# +# `colorama` is deliberately absent: build declares it only under +# `os_name == "nt"`, and all three call sites run on ubuntu-latest. A Windows +# job adopting this file needs colorama added, or pip will refuse the install +# rather than silently skip it. +# +# Each version lists the sha256 of EVERY distribution PyPI publishes for it +# (wheels for each platform, plus the sdist). pip accepts a download that +# matches any one of them, so the pin does not quietly depend on the runner +# resolving to the same wheel this file was generated against. +# +# pip 26.x and build 1.6.x require Python >= 3.10. All three call sites pin +# `python-version: "3.11"`. +# +# Updating: Dependabot owns the routine bumps (see the /.github/requirements +# pip entry in .github/dependabot.yml) and rewrites the hashes with them. A +# pin with no updater is the frozen end of the same problem an unpinned +# install is at the other end of. +# +# Regenerating by hand, on Linux / CPython 3.11 to match the job: +# pip install --dry-run --ignore-installed --report r.json pip build +# then, for each resolved name==version, take every sha256 under `urls` in +# https://pypi.org/pypi///json. + +build==1.6.1 \ + --hash=sha256:ecd351a4be9d35a9eaaba244a7687143c9c7d4aea6ac964e7e7ddab20cbcf4e7 \ + --hash=sha256:51cc11666391ab6f092070437ac747002ff46f3e4113a3622177ee6b488bfc53 +packaging==26.3 \ + --hash=sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c \ + --hash=sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79 +pip==26.2.1 \ + --hash=sha256:71138adf1f4ca900cdb7d289c21b7494329f2332b6d85f0e1c42108c0384ed3e \ + --hash=sha256:f6ad667e89a1fe78046c8f13232b247200f5258d7828f3f7883d660878e0813f +pyproject_hooks==1.2.0 \ + --hash=sha256:9e5c6bfa8dcc30091c74b0cf803c81fdd29d94f01992a7707bc97babb1141913 \ + --hash=sha256:1e859bd5c40fae9448642dd871adf459e5e2084186e8d2c2a79a824c970da1f8 diff --git a/.github/workflows/api-latency-smoke.yml b/.github/workflows/api-latency-smoke.yml index eecb34c72b..a238c945a5 100644 --- a/.github/workflows/api-latency-smoke.yml +++ b/.github/workflows/api-latency-smoke.yml @@ -48,7 +48,7 @@ jobs: - name: Build wheel if: matrix.source == 'wheel' run: | - python -m pip install --upgrade pip build + python -m pip install --require-hashes -r .github/requirements/pip-build-bootstrap.txt python -m build --wheel - name: Install into isolated venv @@ -63,7 +63,7 @@ jobs: SOURCE: ${{ matrix.source }} run: | python -m venv /tmp/vsmoke - /tmp/vsmoke/bin/pip install --upgrade pip + /tmp/vsmoke/bin/pip install --require-hashes -r .github/requirements/pip-bootstrap.txt if [ "$SOURCE" = "pypi" ]; then /tmp/vsmoke/bin/pip install --no-cache-dir clawmetry flask waitress cryptography duckdb requests else diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index bd7bb370f9..225758368b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1371,12 +1371,12 @@ jobs: python-version: "3.11" - name: Build wheel run: | - python -m pip install --upgrade pip build + python -m pip install --require-hashes -r .github/requirements/pip-build-bootstrap.txt python -m build --wheel - name: Install wheel in isolated venv run: | python -m venv /tmp/vwheel - /tmp/vwheel/bin/pip install --upgrade pip + /tmp/vwheel/bin/pip install --require-hashes -r .github/requirements/pip-bootstrap.txt /tmp/vwheel/bin/pip install dist/clawmetry-*.whl flask waitress cryptography pytest requests - name: Verify runtime assets shipped in wheel run: | @@ -1466,7 +1466,7 @@ jobs: cache: pip - name: Install ClawMetry (editable) + eval deps run: | - python -m pip install --upgrade pip + python -m pip install --require-hashes -r .github/requirements/pip-bootstrap.txt pip install -e . pyyaml httpx - name: Check kill-switch id: gate diff --git a/.github/workflows/cross-repo-handoff.yml b/.github/workflows/cross-repo-handoff.yml index edce28ee6b..7d844df6af 100644 --- a/.github/workflows/cross-repo-handoff.yml +++ b/.github/workflows/cross-repo-handoff.yml @@ -102,7 +102,7 @@ jobs: # tests/test_workflow_yaml_valid.py resolve and check the file. working-directory: oss run: | - python -m pip install --upgrade pip + python -m pip install --require-hashes -r .github/requirements/pip-bootstrap.txt pip install --require-hashes \ -r .github/requirements/cross-repo-handoff.txt diff --git a/.github/workflows/oss-golden-path.yml b/.github/workflows/oss-golden-path.yml index b3bcf89a9e..58d196ec69 100644 --- a/.github/workflows/oss-golden-path.yml +++ b/.github/workflows/oss-golden-path.yml @@ -66,13 +66,13 @@ jobs: # the full E2E flow. - name: Build wheel run: | - python -m pip install --upgrade pip build + python -m pip install --require-hashes -r .github/requirements/pip-build-bootstrap.txt python -m build --wheel - name: Install wheel in isolated venv run: | python -m venv /tmp/oss-golden - /tmp/oss-golden/bin/pip install --upgrade pip + /tmp/oss-golden/bin/pip install --require-hashes -r .github/requirements/pip-bootstrap.txt /tmp/oss-golden/bin/pip install \ dist/clawmetry-*.whl \ flask waitress cryptography duckdb requests \ diff --git a/.github/workflows/overhead-bench.yml b/.github/workflows/overhead-bench.yml index 9932a8f1ef..f1e20df4d4 100644 --- a/.github/workflows/overhead-bench.yml +++ b/.github/workflows/overhead-bench.yml @@ -48,7 +48,7 @@ jobs: - name: Install run: | - python -m pip install --upgrade pip + python -m pip install --require-hashes -r .github/requirements/pip-bootstrap.txt python -m pip install -e . requests # --quick keeps a shared runner under a couple of minutes. The full run