Skip to content

Harden CI: stop the screenshot job persisting the job credential - #5338

Merged
vivekchand merged 4 commits into
mainfrom
harden/no-persisted-credentials-pr-screenshots
Aug 30, 2026
Merged

Harden CI: stop the screenshot job persisting the job credential#5338
vivekchand merged 4 commits into
mainfrom
harden/no-persisted-credentials-pr-screenshots

Conversation

@vivekchand

Copy link
Copy Markdown
Owner

Product record: No-PRD: CI-only change confined to .github/, a PRD-exempt path.

Risk: Low. One workflow, two actions/checkout steps, nine added lines of which seven are a comment. Nothing in the shipped package changes. Undone by reverting one commit.

Summary

Continues the persist-credentials: false sweep (#5312, #5321, #5327 merged; #5330, #5331 open) with the one remaining workflow whose checkouts are provably not the credential's consumer.

actions/checkout leaves a credential in .git/config by default. pr-screenshots.yml is the job where that matters most and was least necessary:

  • It has no consumer. The publish step never touches either working copy. It creates a fresh temp directory, git inits a new repo there, and pushes to an explicit https://x-access-token:${GH_TOKEN}@github.com/${GITHUB_REPOSITORY}.git remote. The credential the checkouts persist is simply never read.
  • It sits in a job that runs PR head code. This workflow checks out the pull request head, installs its requirements, and boots two dashboards from it to render screenshots. A credential left in .git/config is readable for the remainder of that job — including from the base checkout, which exists only to render "before" images.

Both checkouts are read-only source trees, so persist-credentials: false removes the credential without removing anything that uses it.

Why this workflow and not the other four remaining ones

The sweep is deliberately not finished in this PR. Four checkouts still persist their credential, and each is a different question rather than more of the same:

Workflow Why it is not in this PR
auto-quarantine.yml Pushes with a bare git push -u origin "$BRANCH" — it is the consumer. Needs the push rewritten to an explicit remote first, which is a behaviour change, not a hardening one.
release-on-merge.yml Same pattern, twice (git push origin "refs/tags/$NEW_TAG", git push origin "$BRANCH"), on the release pipeline. Same rewrite, higher blast radius.
i18n-autotranslate.yml, i18n-docs-autotranslate.yml Push via peter-evans/create-pull-request, which authenticates from its own token input. That very likely tolerates persist-credentials: false, but the action's docs do not state it and this repo has no workflow pairing the two, so there is no evidence here either way. Not worth a silently-broken translation bot to find out in passing.

The second auto-deploy-cloud.yml checkout stays as-is by design — #5331 documents that its credential is the one the force-push uses.

Test plan

  • yaml.safe_load over all 34 workflow files → parse
  • yaml.safe_load over all composite action files → parse
  • Semantic re-read of the parsed YAML: both visual-diff checkout steps report persist-credentials: False with path and ref intact
  • python3 scripts/check_action_refs.py → OK, 17 distinct references, pinning unaffected
  • pytest tests/test_workflow_yaml_valid.py tests/test_ci_workflow_invocations_are_real.py189 passed, 2 skipped (unchanged from baseline)
  • Read the publish step end to end to confirm the push path uses mktemp -d + git init + an explicit token remote, not either checkout

Note for reviewers

The one thing worth a second opinion is the claim that nothing reads the persisted credential. The publish step is the only git push in the file and it runs entirely inside $(mktemp -d); the screenshot steps read from head/ and base/ as plain source trees. If there is a path I missed that expects an authenticated remote in either checkout, that is the thing to catch here.


Generated by Claude Code

Both checkouts in pr-screenshots.yml are read-only source trees: they are
rendered and screenshotted, never pushed from. The publish step does not use
either working copy -- it git-inits a fresh temp repo and pushes to an
explicit https://x-access-token:${GH_TOKEN}@... remote.

So the credential actions/checkout leaves in .git/config has no consumer here.
It only sits readable for the rest of a job that builds and executes code from
the pull request head, including on the base checkout.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wb9DAnFEFQS6NvPsqqz7Kq
@8090-software-factory

Copy link
Copy Markdown

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

@github-actions

github-actions Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Visual diff

Comparing f08398b0c28d (head) against the PR base branch.

13 of 66 comparison(s) flagged (>1% pixel diff).

View Before After Diff
desktop overview ⚠️ before after diff · 100.00%
desktop flow ⚠️ before after diff · 2.93%
desktop brain before after diff · 0.01%
desktop usage before after diff · 0.00%
desktop crons before after diff · 0.32%
desktop memory ⚠️ before after diff · 3.31%
desktop security before after diff · 0.06%
desktop subagents ⚠️ before after diff · 2.11%
desktop transcripts before after diff · 0.01%
desktop logs ⚠️ before after diff · 5.23%
desktop skills before after diff · 0.26%
desktop models before after diff · 0.01%
desktop approvals before after diff · 0.18%
desktop alerts before after diff · 0.10%
desktop notifications before after diff · 0.32%
desktop limits before after diff · 0.01%
desktop clusters before after diff · 0.00%
desktop history before after diff · 0.00%
desktop channels before after diff · 0.01%
desktop dives before after diff · 0.01%
desktop harness before after diff · 0.09%
desktop inventory before after diff · 0.01%
desktop nemoclaw before after diff · 0.01%
desktop policy before after diff · 0.26%
desktop selfevolve before after diff · 0.27%
desktop swimlane before after diff · 0.01%
desktop tool-catalog before after diff · 0.00%
desktop tracing before after diff · 0.01%
desktop turn-anatomy before after diff · 0.01%
desktop version-impact before after diff · 0.00%
desktop context-economics before after diff · 0.01%
desktop agents before after diff · 0.00%
desktop evals before after diff · 0.00%
mobile overview ⚠️ before after diff · 100.00%
mobile flow before after diff · 0.00%
mobile brain before after diff · 0.01%
mobile usage ⚠️ before after diff · 100.00%
mobile crons before after diff · 0.01%
mobile memory before after diff · 0.02%
mobile security before after diff · 0.63%
mobile subagents ⚠️ before after diff · 100.00%
mobile transcripts before after diff · 0.01%
mobile logs before after diff · 0.02%
mobile skills ⚠️ before after diff · 100.00%
mobile models before after diff · 0.01%
mobile approvals before after diff · 0.01%
mobile alerts before after diff · 0.00%
mobile notifications ⚠️ before after diff · 100.00%
mobile limits before after diff · 0.01%
mobile clusters before after diff · 0.01%
mobile history ⚠️ before after diff · 1.06%
mobile channels ⚠️ before after diff · 100.00%
mobile dives before after diff · 0.01%
mobile harness before after diff · 0.37%
mobile inventory before after diff · 0.00%
mobile nemoclaw before after diff · 0.01%
mobile policy before after diff · 0.02%
mobile selfevolve ⚠️ before after diff · 100.00%
mobile swimlane before after diff · 0.02%
mobile tool-catalog before after diff · 0.01%
mobile tracing before after diff · 0.01%
mobile turn-anatomy before after diff · 0.00%
mobile version-impact before after diff · 0.00%
mobile context-economics before after diff · 0.01%
mobile agents before after diff · 0.01%
mobile evals before after diff · 0.01%

Folder: f08398b0c28d. Full PNGs also attached as a workflow artefact.

Generated by visual-diff bot. Pixel diffs >1% flagged; eyeball the table before merging. This check is non-blocking — fail = bot bug, not a code problem.

@8090-software-factory

Copy link
Copy Markdown

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

Copy link
Copy Markdown
Owner Author

✨ auto-fixed: merged latest main into branch (branch was BEHIND)


Generated by Claude Code

github-actions Bot pushed a commit that referenced this pull request Aug 29, 2026

Copy link
Copy Markdown
Owner Author

Ignore the visual-diff table on this PR — it is measuring its own noise

Flagging this because the bot's comment is edited in place, so a reviewer only ever sees the latest table, and "38 of 66 flagged" on a PR that cannot change a pixel looks alarming without the context.

This PR's diff is 9 added lines in .github/workflows/pr-screenshots.yml and nothing else. In every run, the bot renders before from the PR base and after from the head, and the head is the base plus that one workflow-file commit. The two trees it screenshots contain identical application code. Every non-zero number in that table is therefore noise by construction — there is no code path by which a persist-credentials key on a checkout step changes a rendered dashboard.

The two runs make that concrete. Same diff, and views swing in both directions between "identical" and "completely different":

View Run 1 (55f1c18e) Run 2 (afcaa437)
mobile skills 100.00% 0.01%
mobile memory 100.00% 0.02%
mobile notifications 100.00% 0.02%
mobile security 100.00% 0.75%
desktop overview 100.00% 3.83%
mobile turn-anatomy 0.01% 100.00%
mobile tool-catalog 0.01% 100.00%
mobile swimlane 0.01% 100.00%
mobile policy 0.02% 100.00%
mobile dives 0.02% 100.00%

A view cannot go from 0.01% to 100.00% because of a YAML key that was already present in the first run. The totals moved 43/66 → 38/66 for the same reason.

Consistent with the bot's own footer: it is non-blocking, and both runs concluded success. Nothing to fix here, and I am not proposing a change to the bot in this PR — worth a separate look at whether the screenshots are captured before the dashboards finish loading, since a comparator that reports 100% on identical code is not giving reviewers a usable signal.

One thing the runs did confirm positively: the publish step pushed to pr-screenshots-pr-5338 successfully on both heads with persist-credentials: false set on both checkouts. That is the PR's central claim — that the persisted credential has no consumer because the push builds its own repo in $(mktemp -d) and uses an explicit token remote — demonstrated end to end rather than only argued.


Generated by Claude Code

Copy link
Copy Markdown
Owner Author

E2E Gate (required) failed on afcaa437 — runner starvation, not this PR

What failed

The gate is a watcher: it polls the 12 required checks and fails if they do not all complete within MAX_WAIT=1800. It ran 18:15:48 → 18:46:05 and timed out with 8 of 12 still pending. No test failed.

FAIL: timed out after 1800s. Still pending:
  - MOAT Keystone: 0/1          - MOAT Verifier: 0/1
  - E2E Browser Tests: 0/1      - Entitlement API tests: 0/1
  - API Tests (3 OS): 0/3       - pip install matrix: 0/4
  - Wheel install & assets: 0/1 - Store invariants: 0/1

0/N leg(s) complete is queued, not failed. Every check that did report passed: OSS golden path, Cross-repo handoff, Syntax & Lint, and Drift Bot.

Why it is not this PR's

  • The diff cannot cause it. 9 added lines in .github/workflows/pr-screenshots.yml, 7 of them a comment. None of the pending checks live in that workflow — they are all ci.yml. pr-screenshots.yml is not even in the required set.
  • The pool is starved, not slow. Those jobs were created at 18:12 and, as of 18:46, still had not been assigned a runner — 34 minutes queued with zero legs started. The gate's whole 1800s elapsed while its dependencies sat waiting for capacity.
  • Everything else on this head is green — 30+ checks including CodeQL ×3, the full install.sh matrix, and visual-diff.
  • Known signature. Harden CI: stop the gate, nightly and audit jobs persisting the job credential #5330 hit this exact failure earlier today (same gate, same timeout, same 0/N pending list) and passed on re-run with no code change.

On porting the existing fix

#5332 is the standing fix and I have read it, but I am not porting it here, for two specific reasons:

  1. It bundles more than the timeout. Alongside MAX_WAIT 1800→3600 it sets skip_if_unreported=True on the Drift Bot spec and rewrites test_a_missing_drift_bot_does_not_pass_the_gate into test_a_missing_drift_bot_skips_not_blocks — a deliberate weakening of when Drift Bot blocks. Carrying a gate-weakening change inside a PR whose entire purpose is tightening CI would be the wrong thing to put in front of a reviewer, and it is not what this failure needs.
  2. Porting only the timeout half would collide. It touches the same three lines of e2e-gate.yml and scripts/e2e_gate.py that fix(e2e-gate): skip Drift Bot when unreported, raise MAX_WAIT to 3600s #5332 edits, guaranteeing a conflict for whichever of the two merges second.

So the minimal correct action here is a re-run, not a code change.

Next step

I have not spent my re-run on this PR. Re-running right now would start a fresh 1800s timer against the same starved pool and would likely time out again for the same reason, so I will re-run the gate once the dependency jobs have actually drained, at which point it should observe them green and pass. Keeping this PR watched until it is green or merged.

Worth fixing at the repo level

This is the second PR today blocked by the same thing, and the cause is structural: MAX_WAIT is a fixed budget measured against a variable queue, so on a busy day the gate fails PRs for the pool's behaviour rather than their own. Merging #5332 (or just its MAX_WAIT/timeout-minutes half, if the Drift Bot change wants separating) would stop this recurring across every PR. Flagging rather than acting on it — that is a repo-wide call and a file this PR deliberately does not touch.


Generated by Claude Code

@8090-software-factory

Copy link
Copy Markdown

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

Copy link
Copy Markdown
Owner Author

✨ auto-fixed: merged main (14572bc) into branch to resolve BEHIND state


Generated by Claude Code

@8090-software-factory

Copy link
Copy Markdown

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

Copy link
Copy Markdown
Owner Author

✨ auto-fixed: merged main into branch (was BEHIND at 14572bc)


Generated by Claude Code

github-actions Bot pushed a commit that referenced this pull request Aug 30, 2026
@vivekchand
vivekchand merged commit 4cef048 into main Aug 30, 2026
42 of 57 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant