Skip to content

fix(ci): the secret scan never read history, and every doc said it did - #18

Open
hseshadr wants to merge 3 commits into
mainfrom
fix/secret-scan-sweeps-full-history
Open

fix(ci): the secret scan never read history, and every doc said it did#18
hseshadr wants to merge 3 commits into
mainfrom
fix/secret-scan-sweeps-full-history

Conversation

@hseshadr

@hseshadr hseshadr commented Aug 6, 2026

Copy link
Copy Markdown
Owner

The claim this touches

"gitleaks over the FULL git history" — secret-scan.yml, line 1
"a credential committed five commits ago is exactly as leaked as one committed at HEAD" — line 6
"gitleaks scans the full git history for secrets" — README, ×4

All false as written. No consumer's pre-existing history had ever been scanned by CI.

Why

gitleaks-action derives its scan range from the event, not from fetch-depth
(gitleaks-action@e0c47f4, src/gitleaks.js:103-115, src/index.js:176):

Event --log-opts Scanned
push, pull_request --no-merges --first-parent BASE^..HEAD only that event's commits
schedule, workflow_dispatch (none) every commit

fetch-depth: 0 makes BASE^ resolvable. It does not widen the scan.

secret-scan.yml is workflow_call-only and every caller in examples/ but one ran on push/PR.

Red run 1 — the property, measured on this repo through this very workflow

Event Run Commits scanned Verdict
push to main 31051347230 0 ✅ No leaks detected
pull_request 30978634362 1 ✅ No leaks detected
schedule 30793713570 41 ✅ No leaks detected

A push whose base is already an ancestor of head scans zero commits and still reports success.

Red run 2 — a real credential, planted where a PR cannot see it

Synthetic repo, 14 commits, a fake AWS key at commit 3, PR branch adds 2 clean commits. Local gitleaks 8.30.1, reproducing the exact command the action builds:

A) push/PR path:  gitleaks detect --log-opts="--no-merges --first-parent $BASE^..$HEAD"
   INF 3 commits scanned.
   INF no leaks found
   exit 0                                        <-- GREEN. The planted key is invisible.

B) schedule path: gitleaks detect
   INF 14 commits scanned.
   WRN leaks found: 1
   exit 1                                        <-- RED. Caught.

The fake key never left a temp dir and was never committed or pushed anywhere.

Red run 3 — the guards can fail (mutation, local, offline)

Mutation Result
Delete the scheduled sweep from examples/edge-reco/security-audit.yml FAIL: examples/edge-reco calls secret-scan.yml but never from a scheduled workflow — its history is never swept, only each push's own commits
Drop name: Secret scan from examples/privacy-core/ci.yml FAIL: examples/privacy-core/ci.yml job 'gitleaks' calls secret-scan.yml with no name: — it reports as 'gitleaks / gitleaks', not 'Secret scan / gitleaks'
Delete the refusal branch from the coverage step FAIL: secret-scan ACCEPTS full-history on a push event — a partial scan reported as a sweep
FAIL: secret-scan ACCEPTS full-history on a pull_request event — a partial scan reported as a sweep

The fix: the caller owns the schedule

A workflow_call workflow cannot carry its own schedule:, so this cannot be fixed inside the brick. Three alternatives were considered and rejected:

  • ci runs a cron that sweeps the other repos — needs a cross-repo PAT and a fan-out; adds a long-lived secret to solve a docs problem.
  • Override GITHUB_EVENT_NAME so the action always takes its full-history branch — adapter code that corrects the library's semantics. That is the tell for a wrong abstraction, not a fix.
  • Run gitleaks directly for the sweep — a second supply-chain pin and a parallel config path that will drift from the action's.

Chosen: every consumer already has security-audit.yml on a weekly cron. Each examples/*/security-audit.yml now calls secret-scan.yml from it with full-history: true, so all five converging callers get a genuine weekly full sweep. examples/*/ci.yml keeps its caller for the per-PR incremental scan. The action is used exactly as designed; the enforcement lives on our side, where it is testable.

full-history does not widen the scan — nothing can, from inside a reusable workflow. It makes the expectation machine-checked: on any event that scans a partial range, the job fails instead of reporting a clean partial scan as clean. Default false, so this release cannot redden an existing push/PR caller.

Every run now prints what it covered. "No leaks detected" over 0 commits and over 41 commits are the same three words.

Second defect: unnamed caller jobs

Five of the six callers in examples/ — and the README's canonical copy-paste snippet — omitted name: Secret scan, so a copy-paste yields the check context gitleaks / gitleaks, not Secret scan / gitleaks. That silently orphans an adopter's required status check. All named now, and the guard refuses an unnamed one.

README: 16 false claims, 13 fixed, 3 deleted

Audited every claim against the tree, the tags, the live consumer repos, and upstream action sources.

# Claim Reality Action
1 Current release ci-v3.0.0 / 2a575cd ci-v3.2.1 / 605e51c, 3 releases later FIX
2 Copy-paste caller pins 2a575cd # ci-v3.0.0 every ref in examples/ is 605e51c # ci-v3.2.1 FIX
3 Canonical caller omits name: Secret scan yields the wrong check context FIX
4 "six call-sites across four repos" 7 across 5 (aml-filter adopted secret-scan) FIX
5 "five of six carry a stale # ci-v2.0.3 label" none do FIX
6 secret-scan "has no consumer runs" aml-filter run 31051313153 SUCCESS FIX
7 drift count = 29 / 30 / 30 in three places 29 FIX
8 allowlist entry "points to an open PR" merged 2026-08-02; entry deleted FIX
9 "live gap at ci-v3.0.0: --allow-unlocked" closed at ci-v3.2.1 DELETE
10 pin table lists # v7, # v6, # v3 floating majors — the defect ae644d7 fixed FIX
11 "gitleaks over the full git history" ×4 event's commit range FIX
12 "ci has no branch protection, no secret scanning" 3 required contexts; both enabled DELETE
13 owner action "cut the release after ci-v3.0.0" done ×3 DELETE
14 "these repos are private, callers 404" all 8 are public FIX
15 publish verification "six attempts, ~a minute" 14 attempts, 600s FIX
16 "all 40 first-party refs pin ci-v3.0.0" 45, all ci-v3.2.1 FIX

New section What the secret scan actually covers, and a check-context table for adopters. python-gate.yml's context is marked UNVERIFIED — no consumer calls it yet, so gate / gate is derived from the rule and has never been emitted by a run. Documenting an unobserved context as fact is how the gitleaks mismatch reached five repos.

Evidence

Gate Result
tests/security-policy.sh ✅ passed (+2 new checks)
tests/lineage-guard-cases.sh ✅ passed
tests/example-fidelity-cases.sh ✅ passed
tests/lint-examples.sh ✅ no findings; 182 resolved, 0 MISSING, 0 UNVERIFIABLE across 7 consumer repos
shellcheck -x ✅ clean
actionlint ✅ clean
uvx zizmor@1.26.1 . (online audits, GH_TOKEN set) ✅ No findings
Guards shown failing ✅ 3 mutations, above
Remote CI ⚠️ UNMEASURED — GitHub Actions is in a major outage (incident open since ~16:25Z 2026-08-06). Any run here is the platform, not this change.

Still open — owner's call

ci-v3 points at 72521e7, a Dependabot merge 21 commits behind ci-v3.2.1. Nothing pins it (every ref is a full SHA), so it misleads readers rather than breaking builds. Now documented in Owner actions with the fix:

git tag -f ci-v3 ci-v3.2.1^{} && git push -f origin ci-v3

Not merging — that stays yours. Note enforce_admins=false with 1 required review, so this needs --admin.

Downstream

The nine consumer repos still need the scheduled caller added to their real security-audit.yml. This PR ships the templates and the guard that refuses a tree without them; it does not touch consumer repos.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Fx1Abdt3o9dLsKsd2xDcgS

hseshadr and others added 2 commits August 6, 2026 11:55
@hseshadr @claude
fix(ci): the secret scan never read history, and said it did
83c0306 
`secret-scan.yml` opened with "gitleaks over the FULL git history" and
"a credential committed five commits ago is exactly as leaked as one
committed at HEAD". Neither was true of what it ran.

gitleaks-action derives its scan range from the EVENT, not from
fetch-depth (gitleaks-action@e0c47f4, src/gitleaks.js:103-115,
src/index.js:176): on push and pull_request it appends
`--log-opts=--no-merges --first-parent BASE^..HEAD`; only on schedule
and workflow_dispatch does it omit --log-opts and read every commit.
`fetch-depth: 0` makes BASE^ resolvable. It does not widen the scan.

Measured on this repository, through this very workflow:

  run 31051347230  push to main   0 commits scanned  reported success
  run 30978634362  pull_request   1 commit  scanned  reported success
  run 30793713570  schedule      41 commits scanned  reported success

The workflow is workflow_call-only, and every caller in examples/ but
one was push/PR. So no consumer's pre-existing history had ever been
scanned by CI, and a push whose base is already an ancestor of head
scanned zero commits and still reported success.

A reusable workflow cannot carry its own `schedule`, so the caller owns
it. Every consumer already has security-audit.yml on a weekly cron;
each now calls secret-scan.yml from it with `full-history: true`.

`full-history` does not widen the scan — nothing can, from inside a
reusable workflow. It makes the expectation machine-checked: on any
event that scans a partial range the job fails instead of reporting a
clean partial scan as clean. It defaults to false so this release
cannot redden an existing push/PR caller.

Also names every secret-scan caller job. An unnamed one reports as
"gitleaks / gitleaks" instead of the documented "Secret scan /
gitleaks", silently orphaning an adopter's required status check.

Guards, in tests/security-policy.sh:
  - validate_secret_scan_history_sweep: a repo that calls
    secret-scan.yml but never from a scheduled workflow is refused, as
    is an unnamed caller job. Carries a vacuity floor.
  - validate_secret_scan_coverage_cases: executes the workflow's real
    coverage script under both event families rather than grepping for
    its error string.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fx1Abdt3o9dLsKsd2xDcgS
@hseshadr @claude
docs(ci): the README was false in 16 places, and the scan claim was o…
cf07634 
…ne of them

Audited every factual claim in README.md against the tree, the tags, the
live consumer repos and the upstream action sources. 16 were false.
Fixed 13, deleted 3.

The two that could actively harm an adopter:

  - The canonical copy-paste caller omitted `name: Secret scan`, so a
    reader got the check context `gitleaks / gitleaks` instead of the
    documented `Secret scan / gitleaks` — silently orphaning a required
    status check. Five of the six callers in examples/ omitted it too.
  - Four places claimed the secret scan covers "the full git history".
    It covers the calling event's commit range. Corrected, with the
    measured evidence, in a new section.

The rest were drift: the README was never updated past ci-v3.0.0 while
three releases and one consumer adoption landed.

  - Current release and every 2a575cd pin -> 605e51c / ci-v3.2.1.
  - Adoption: 7 call-sites across 5 repos, six on the publish path.
    Verified by grepping every consumer's default branch, not asserted.
  - Drift count said 29 in one place, 30 in two others. All three now
    say 29, matching the allowlist.
  - Third-party pin table listed floating majors (`# v7`, `# v6`, `# v3`)
    as the release comment. That is the defect commit ae644d7 fixed; a
    `# v6` on a SHA goes wrong the moment upstream moves the tag. Now
    the exact versions, including the two actions pinned at different
    versions in .github/ and examples/.
  - Publish verification: 14 attempts / 600s, not 6 attempts / 60s.
  - "These repos are private, so callers 404" — all eight are public.
  - "`ci` has no branch protection and no repository secret scanning" —
    three required contexts, force-push off, secret scanning and push
    protection both on.
  - secret-scan.yml is no longer in the "no consumer runs" list.

DELETED rather than fixed, because they cannot be made true:
  - The `--allow-unlocked` "live gap" callout. Closed at ci-v3.2.1;
    python-gate now nests the ci-v3.2.1 composite, which implements it.
  - Two completed owner actions (branch protection, cut the release).

One owner action is genuinely open and is now stated with its fix: the
`ci-v3` pointer is 21 commits behind ci-v3.2.1.

Also adds a check-context table for adopters. python-gate.yml's context
is marked UNVERIFIED — no consumer calls it yet, so `gate / gate` is
derived from the rule, never observed. Documenting an unobserved context
as fact is how the gitleaks mismatch reached five repos.

tests/consumer-drift-allowlist.txt: aml-filter/ci.yml/secret-scan
deleted. aml-filter#93 merged 2026-08-02 and the consumer now calls the
brick (run 31051313153, SUCCESS on main). 30 -> 29.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fx1Abdt3o9dLsKsd2xDcgS
@cursor

cursor Bot commented Aug 6, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@hseshadr @claude
fix(ci): a caller that under-grants does not go red, it goes ABSENT
319afbc 
PR #18's CI run ended in `startup_failure`:

  requesting 'pull-requests: read', but is only allowed 'pull-requests: none'

`secret-scan.yml` declares `pull-requests: read` — gitleaks-action lists a PR's
commits through the API. `ci.yml`'s `secret-scan-sweep` job granted only
`contents: read`. A job-level `permissions:` block REPLACES the top-level one
rather than adding to it, so restating `contents: read` silently dropped
`pull-requests` to `none`. The sibling `secret-scan` job already had it right.

The important half is what that failure reported: NOTHING.

A `startup_failure` emits zero check runs. Run 31127046921 has `jobs: 0`, and the
check-runs API for its head SHA lists only the checks from other workflows.
`Security policy` and `Secret scan (own brick) / gitleaks` were not red on PR #18,
they were missing — and branch protection reads a missing required check as
"pending", never "failed". A gate that CANNOT run is indistinguishable from one
that has not run yet. Same shape as the bug this PR exists to fix: a secret scan
that scanned 0 commits and reported success.

`if:` does not save you either. Permissions are validated before any condition is
evaluated, so the run died on `pull_request` events where this job would never
have started.

The examples ship the bug too. Five `examples/*/security-audit.yml` call
secret-scan.yml from a workflow whose only grant is a top-level `contents: read`,
and at the pinned SHA (605e51c, ci-v3.2.1) that workflow already declared
`pull-requests: read` — so every adopter who copied one got a scheduled sweep that
could never start, and a weekly run that reported nothing rather than failing.

New guard, static by necessity — there is no run to inspect, because the failure
IS the absence of a run:

  tests/lib/scan-caller-permissions.rb
  tests/security-policy.sh::validate_caller_permission_sufficiency
  tests/security-policy.sh::validate_caller_permission_cases

It parses each caller job's effective grant (job-level block, else workflow-level)
and compares it scope-by-scope (none < read < write) against the callee's declared
`permissions:`, over `.github/workflows/` and `examples/` alike. 15 both-polarity
fixtures pin the property: the job-level-replacement trap, a granted `read` against
a required `write`, `read-all`/`write-all` shorthands, a grant hidden behind a YAML
alias, a third-party callee it must not blame, and a caller declaring no
permissions anywhere (refused — the answer lives in a repo setting this tree cannot
read). It also asserts it resolved at least 20 caller->callee pairs (25 today), so
ref resolution that quietly broke cannot look like a clean tree.

Evidence:
  red    (guard, unfixed tree)          exit 1, 6 findings: ci.yml + 5 examples
  green  (guard, fixed tree)            exit 0, 25 pairs resolved
  red    (re-narrow ci.yml sweep)       exit 1, names secret-scan-sweep
  red    (re-narrow examples/edge-proc) exit 1, names its gitleaks job
  green  (restored)                     exit 0
  actionlint 0 · shellcheck 0 · lineage-guard-cases 0 · example-fidelity-cases 0
  lint-examples 0 (zizmor: no findings; fidelity: 182 resolved, 0 missing)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fx1Abdt3o9dLsKsd2xDcgS
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@hseshadr