Skip to content

Harden CI: the gate jobs install from a hash-pinned set - #5907

Merged
vivekchand merged 2 commits into
mainfrom
harden/pin-gate-job-installs
Sep 12, 2026
Merged

Harden CI: the gate jobs install from a hash-pinned set#5907
vivekchand merged 2 commits into
mainfrom
harden/pin-gate-job-installs

Conversation

@vivekchand

Copy link
Copy Markdown
Owner

Product record: No-PRD: CI-only change; every path is under .github/ or tests/ (exempt per FLYWHEEL.md / scripts/check_product_record.py, which reports "no product-visible code changed").

Risk: Contained to two CI jobs, neither on the release path. If a hash or a resolved version is wrong the affected job fails loudly at the install step — it cannot half-install or silently pick something else, which is the property --require-hashes is bought for. Undone by reverting this commit; the previous loose lines return verbatim. Nothing in flight collides: #5889, #5897, #5899, #5902 and #5903 introduce the same directory but different files, and none of them touches product-record-gate.yml, cross-repo-handoff.yml or tests/test_workflow_yaml_valid.py.


Summary

  • Two workflows installed their Python deps from a loose pip install a b c line, so every run resolved whatever the index served that minute with no integrity check on any artifact. Both now install with pip install --require-hashes, which refuses anything not listed with a matching hash: product-record-gate and cross-repo-handoff.
  • .github/requirements/gate-harness.txt (10 packages) and .github/requirements/cross-repo-handoff.txt (21 packages) carry the full resolved closure and, for each, the sha256 of every distribution PyPI publishes for that version — so the pin does not quietly depend on the runner resolving to the same wheel the file was generated against. Same shape and generation recipe as the files in Harden CI: the security scanners install from a hash-pinned set #5889 and Harden CI: the dashboard-boot jobs install from a hash-pinned set #5897.
  • A gate that decides whether a PR may merge should not itself be installing an unverified, moving set. That is the reason product-record-gate is in this batch rather than a later one.

This batch also has to keep a guard honest

Unlike the dashboard-boot jobs in #5897, both of these jobs run pytest, so they are covered by test_pytest_jobs_install_what_conftest_needs — the guard that stops a pytest job omitting something tests/conftest.py imports at module scope (which makes pytest exit 4 before collecting a single test).

That guard skipped any job whose install text contained -r. Moving these jobs' package names out of the run: line and into a requirements file would therefore have silently retired the guard for both jobs — pinning the installs would have cost the coverage that catches a genuinely different failure. So the guard now:

  • reads a -r <file> this repo owns and checks its contents, instead of skipping on sight of -r;
  • strips comments first, so a file cannot satisfy the guard by mentioning a package in its prose header without actually pinning it;
  • still cannot read a file resolved at run time from outside this repo (cross-repo-handoff's separate "Install cloud deps" step does -r cloud/requirements.txt), but now only skips when something is actually missing — if the readable files already supply everything, it asserts.

Net effect on the suite: 585 passed / 371 skipped → 587 passed / 369 skipped. Two jobs that previously skipped this guard are now really asserted against.

Scope

cross-repo-handoff's "Install cloud deps" step (psycopg2-binary, plus -r cloud/requirements.txt || true) is deliberately left alone: it installs a requirements file checked out from another repository at run time, which this repo does not own and cannot pin from here, and it is already best-effort by design. Both files are generated on Linux / CPython 3.11 to match the two jobs' runners and setup-python version, and each header says so rather than being silently reused for a 3.9 job.

No workflow's permissions: block is touched.

Merge-order note

The updater for this directory is the /.github/requirements pip entry added by #5889, whose group pattern is * and so covers both files too. If this PR lands first, these pins do not move until #5889 merges. Both headers state this plainly rather than claiming an updater that is not there yet.

Test plan

  • Both sets install clean under pip install --require-hashes in a fresh CPython 3.11 venv — exit 0, no hash mismatch (10 and 21 packages)
  • Every direct import resolves in those venvs (pytest, requests; plus flask, cryptography, waitress)
  • The gate's real work reproduced: tests/test_product_record_gate.py — 30 passed, run from the hash-pinned venv
  • All 21 distinct pinned versions checked against OSV — 0 carry advisories
  • tests/test_workflow_yaml_valid.py — 587 passed, 369 skipped (was 585 / 371)
  • Both changed jobs confirmed asserted, not skipped, by the conftest guard
  • Negative test: removing the requests pin from gate-harness.txt makes the guard fail with the right message, proving it reads the file rather than passing vacuously
  • All 40 workflow files parse: python3 -c "import yaml,glob; [yaml.safe_load(open(f)) for f in glob.glob('.github/workflows/*.yml')]"
  • scripts/check_product_record.py — "product-record gate OK - no product-visible code changed"

🤖 Generated with Claude Code

https://claude.ai/code/session_01DMs539CX1THWkjPXRSWYFg


Generated by Claude Code

product-record-gate and cross-repo-handoff each installed their Python
deps from a loose `pip install a b c` line, so every run resolved
whatever the index served that minute with no integrity check on any
artifact. Both now install with `pip install --require-hashes` from a
file under .github/requirements/, which refuses anything not listed
with a matching hash.

Each file carries the full resolved closure and, for every package, the
sha256 of every distribution PyPI publishes for that version, so the pin
does not depend on the runner resolving to the same wheel the file was
generated against.

Both of these jobs run pytest, which the dashboard-boot batch's jobs did
not, so this batch also has to keep tests/test_workflow_yaml_valid.py's
conftest guard honest: it skipped any job whose install line contained
`-r`, meaning pinning a job's installs would have silently retired the
guard for that job. It now reads a requirements file this repo owns and
checks its contents (comments stripped, so a prose header cannot satisfy
it by merely naming a package). A file the repo does not own is still
unreadable, but only forces a skip when something is actually missing.
That converts two previous skips into real assertions: 585 passed / 371
skipped becomes 587 passed / 369 skipped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DMs539CX1THWkjPXRSWYFg
@8090-software-factory

Copy link
Copy Markdown

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

The C4 job checks out three repositories side by side (oss/, landing/,
cloud/), so the OSS tree is not at the workspace root. The previous
`pip install flask pytest ...` line had no path dependency and did not
care; `-r .github/requirements/cross-repo-handoff.txt` does, and the job
failed with "Could not open requirements file".

Pinning the step to `working-directory: oss` keeps the requirements
reference repo-relative, which is also what lets
tests/test_workflow_yaml_valid.py resolve the file and keep asserting
this job against conftest's imports.

product-record-gate checks out at the workspace root and was unaffected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DMs539CX1THWkjPXRSWYFg
@8090-software-factory

Copy link
Copy Markdown

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

@vivekchand
vivekchand merged commit 4c192fe into main Sep 12, 2026
39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants