fix(ci): the secret scan never read history, and every doc said it did

.github/workflows/ci.yml

@@ -24,15 +24,49 @@ jobs:
   # portfolio and this repo — the one that publishes it — was the only repo not running
   # it. A local `./` ref is used deliberately: it always names THIS commit, so the
   # workflow is proven against the version being changed rather than a released SHA.
+  #
+  # TWO callers, because one job cannot answer both questions. gitleaks-action
+  # derives its scan range from the EVENT: on push/pull_request it scans only the
+  # commits that event introduced; on schedule/workflow_dispatch it scans every
+  # commit. Measured here — run 31051347230 (push to main) scanned 0 commits and
+  # still reported success; run 30793713570 (schedule) scanned 41.
   secret-scan:
-    name: Secret scan (own brick)
+    name: Secret scan (own brick) # required status check — do not rename
+    # Skipped on the sweep events so the full-history job below is not duplicated.
+    if: github.event_name != 'schedule' && github.event_name != 'workflow_dispatch'
     # The reusable workflow needs pull-requests: read to list a PR's commits via the API;
     # a caller job may raise a scope the read-only top level does not grant.
     permissions:
       contents: read
       pull-requests: read
     uses: ./.github/workflows/secret-scan.yml
 
+  # The half that actually reads history. `full-history: true` is not a switch that
+  # widens the scan — nothing can widen it from inside a reusable workflow — it is an
+  # assertion: if this job ever runs on an event that scans a partial range, it fails
+  # instead of reporting a partial scan as clean.
+  secret-scan-sweep:
+    name: Full-history secret sweep (own brick)
+    if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
+    # Must match the callee, exactly as the job above does. A job-level block
+    # REPLACES the top-level one, it does not add to it, so granting only
+    # `contents: read` here left `pull-requests` at `none` while secret-scan.yml
+    # declares it needs `read`. GitHub refused the whole run: `startup_failure`,
+    # ZERO check runs — `Security policy` and `Secret scan (own brick) / gitleaks`
+    # were absent from the PR rather than red, and branch protection reads absent
+    # as "pending", never "failed". `if:` does not save you: permissions are
+    # checked before any condition is evaluated, so the run died on pull_request
+    # events where this job would never have started.
+    # tests/security-policy.sh::validate_caller_permission_sufficiency compares
+    # every caller against its callee statically, because a run that cannot start
+    # reports nothing to learn from.
+    permissions:
+      contents: read
+      pull-requests: read
+    uses: ./.github/workflows/secret-scan.yml
+    with:
+      full-history: true
+
   security-policy:
     name: Security policy
     runs-on: ubuntu-latest

.github/workflows/secret-scan.yml

@@ -1,15 +1,43 @@
-# Reusable secret scan — gitleaks over the FULL git history (house standard §4).
+# Reusable secret scan — gitleaks, over the range the CALLING EVENT defines.
 # The single most-duplicated job in the portfolio: five of the six repos run an
 # identical gitleaks job today (almamesh keeps a bespoke one — its gitleaks job
 # carries an extra key-custody tree-guard step; see the README limits section).
 #
-# fetch-depth: 0 is mandatory — a credential committed five commits ago is exactly
-# as leaked as one committed at HEAD. pull-requests:read is required because
-# gitleaks-action lists a PR's commits via the API on pull_request events (else it
-# 403s "Resource not accessible by integration" before scanning). GITLEAKS_LICENSE
-# is NOT needed — these are personal-account repos, not an organization. A repo's
-# own .gitleaks.toml allowlist (aml-filter, almamesh have one) is picked up
-# automatically from the checkout; no input needed.
+# READ THIS BEFORE ASSUMING YOUR HISTORY IS SWEPT.
+# gitleaks-action decides what to scan from the EVENT, not from fetch-depth
+# (gitleaks-action@e0c47f4, src/gitleaks.js:103-115 and src/index.js:176):
+#
+#   push, pull_request           adds --log-opts=--no-merges --first-parent BASE^..HEAD
+#                                -> ONLY the commits that event introduced
+#   schedule, workflow_dispatch  no --log-opts at all
+#                                -> EVERY commit in the repository
+#
+# Measured on this repository, through this very workflow:
+#
+#   event           run           commits scanned   verdict
+#   schedule        30793713570   41                No leaks detected
+#   pull_request    30978634362    1                No leaks detected
+#   push (to main)  31051347230    0                No leaks detected
+#
+# A push whose base is already an ancestor of head scans ZERO commits and still
+# reports success. That is the failure this workflow now refuses to hide: a green
+# scan is not evidence unless you know what it looked at.
+#
+# fetch-depth: 0 is still mandatory, but it only makes BASE^ RESOLVABLE — it does
+# not widen the range. Full history is swept when, and only when, a caller
+# triggers this workflow on `schedule` or `workflow_dispatch`. A reusable workflow
+# cannot carry its own `schedule`, so THE CALLER OWNS THE SCHEDULE: put a second
+# caller job in the repo's scheduled workflow (every consumer already has
+# `security-audit.yml` on a weekly cron) and pass `full-history: true`.
+# tests/security-policy.sh refuses an examples/ tree where a repo calls this
+# workflow but never from a scheduled one.
+#
+# pull-requests:read is required because gitleaks-action lists a PR's commits via
+# the API on pull_request events (else it 403s "Resource not accessible by
+# integration" before scanning). GITLEAKS_LICENSE is NOT needed — these are
+# personal-account repos, not an organization. A repo's own .gitleaks.toml
+# allowlist (aml-filter, almamesh have one) is picked up automatically from the
+# checkout; no input needed.
 #
 # This is a single first-party action, so there is nothing to factor into a
 # composite — the whole job is the shared unit.
@@ -20,10 +48,21 @@
 # and keep the trailing version comment so Dependabot can bump it. Ready-to-copy
 # callers live in examples/.
 #
-# Caller (one job):
+# Caller — TWO jobs, in two files, because they answer two different questions:
+#
+#   # ci.yml             (on: push, pull_request)  — did THIS change add a secret?
 #   jobs:
 #     gitleaks:
-#       uses: hseshadr/ci/.github/workflows/secret-scan.yml@<40-char-sha> # ci-v2.0.1
+#       name: Secret scan        # -> check context "Secret scan / gitleaks"
+#       uses: hseshadr/ci/.github/workflows/secret-scan.yml@<40-char-sha> # ci-v3.2.1
+#
+#   # security-audit.yml (on: schedule)            — is anything in HISTORY a secret?
+#   jobs:
+#     gitleaks:
+#       name: Secret scan
+#       uses: hseshadr/ci/.github/workflows/secret-scan.yml@<40-char-sha> # ci-v3.2.1
+#       with:
+#         full-history: true
 name: Secret scan (reusable)
 
 on:
@@ -32,6 +71,15 @@ on:
       runs-on:
         type: string
         default: "ubuntu-latest"
+      # Declares that this caller expects the whole repository swept. It does not
+      # widen the scan — nothing can, from inside a reusable workflow — it makes
+      # the expectation MACHINE-CHECKED: on any event that scans a partial range,
+      # the job fails loudly instead of reporting a clean partial scan as clean.
+      # Defaults to false so adopting this release cannot redden an existing
+      # push/PR caller; the scheduled caller opts in.
+      full-history:
+        type: boolean
+        default: false
 
 permissions:
   contents: read
@@ -44,7 +92,31 @@ jobs:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
         with:
           persist-credentials: false
-          fetch-depth: 0 # full history — gitleaks scans every commit, not just HEAD
+          # Makes BASE^ resolvable for the push/PR range, and is what lets the
+          # scheduled sweep reach every commit. It does NOT by itself widen the
+          # range — see the event table at the top of this file.
+          fetch-depth: 0
       - uses: gitleaks/gitleaks-action@e0c47f4f8be36e29cdc102c57e68cb5cbf0e8d1e # v3.0.0
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      # Say what was actually scanned. "No leaks detected" over 0 commits and over
+      # 41 commits are the same three words; only this line tells them apart.
+      # Runs unconditionally so the number is in every log, not just failing ones.
+      - name: Report what this scan actually covered
+        env:
+          SCAN_FULL_HISTORY: ${{ inputs.full-history }}
+        run: |
+          commits="$(git rev-list --count HEAD 2>/dev/null || echo '?')"
+          case "${GITHUB_EVENT_NAME:-unknown}" in
+            schedule | workflow_dispatch)
+              echo "gitleaks swept FULL HISTORY: all ${commits} commits (no --log-opts on a ${GITHUB_EVENT_NAME} event)."
+              ;;
+            *)
+              echo "gitleaks scanned ONLY the ${GITHUB_EVENT_NAME:-unknown} event's commit range, out of ${commits} commits in history."
+              echo "History is swept by a schedule/workflow_dispatch caller, not by this one."
+              if [ "${SCAN_FULL_HISTORY}" = "true" ]; then
+                echo "::error::full-history: true was requested, but a ${GITHUB_EVENT_NAME:-unknown} event scans only its own commit range. Call this workflow from a workflow triggered by 'schedule' or 'workflow_dispatch'."
+                exit 1
+              fi
+              ;;
+          esac

CHANGELOG.md

@@ -6,6 +6,94 @@ All notable changes to the shared CI/CD templates. Each release is cut as an imm
 listed below. `tests/security-policy.sh` rejects a moving `@ci-vN` ref, first-party
 included.
 
+## Unreleased
+
+**A brick changed shape**: `secret-scan.yml` gains one optional input, `full-history`
+(boolean, default `false`). Re-pinning without setting it is a drop-in — the default is
+the existing behaviour, so no current caller changes verdict.
+
+**Action required if you copied `examples/<repo>/security-audit.yml`**: its `gitleaks` job
+now carries its own `permissions:` block. Without it that job cannot start — see below.
+
+- **A caller that under-granted did not go red, it went ABSENT.** `secret-scan.yml`
+  declares `pull-requests: read` (gitleaks-action lists a PR's commits through the API).
+  Five `examples/*/security-audit.yml` called it from a workflow whose only grant was a
+  top-level `contents: read`, and so did `ci.yml`'s own `secret-scan-sweep` job — whose
+  job-level `permissions: {contents: read}` *replaced* the top level rather than adding to
+  it, dropping `pull-requests` to `none`. GitHub refuses such a run before any job starts:
+  `requesting 'pull-requests: read', but is only allowed 'pull-requests: none'`. The
+  conclusion is `startup_failure` and it emits **zero check runs** — measured on run
+  [31127046921](https://github.com/hseshadr/ci/actions/runs/31127046921), which reported
+  `jobs: 0` while the check-runs API for its head SHA listed only the checks from other
+  workflows. `Security policy` and `Secret scan (own brick) / gitleaks` were not red, they
+  were missing, and **branch protection reads a missing required check as "pending", never
+  "failed"** — the same shape as the bug this release exists to fix, where a secret scan
+  that scanned 0 commits reported success. An `if:` guard does not help: permissions are
+  checked before any condition is evaluated, so `ci.yml` died on `pull_request` events
+  where the offending job would never have run at all.
+- **New guard: `tests/lib/scan-caller-permissions.rb`**, driven by
+  `validate_caller_permission_sufficiency` in `tests/security-policy.sh`. It parses every
+  caller job's effective grant (job-level block, else workflow-level) and compares it
+  scope-by-scope against the callee's declared `permissions:`, across
+  `.github/workflows/` **and** `examples/`. It is static by necessity — there is no run to
+  inspect, because the failure *is* the absence of a run. 15 both-polarity fixtures pin the
+  property (`validate_caller_permission_cases`), including the job-level-replacement trap,
+  a granted `read` against a required `write`, `read-all`/`write-all` shorthands, a grant
+  supplied through a YAML alias, and a caller that declares no permissions anywhere. The
+  scanner also reports how many caller→callee pairs it resolved (25 today) against a floor
+  of 20, so ref resolution that quietly broke cannot masquerade as a clean tree.
+
+- **The secret scan never read history, and said it did.** `secret-scan.yml` opened with
+  "gitleaks over the FULL git history" and "a credential committed five commits ago is
+  exactly as leaked as one committed at HEAD". Neither described what it ran.
+  `gitleaks-action` derives its scan range from the **event**, not from `fetch-depth`
+  (`gitleaks-action@e0c47f4`, `src/gitleaks.js:103-115`, `src/index.js:176`): on `push`
+  and `pull_request` it appends `--log-opts=--no-merges --first-parent BASE^..HEAD`, and
+  only on `schedule`/`workflow_dispatch` does it omit `--log-opts` and read every commit.
+  `fetch-depth: 0` makes `BASE^` resolvable; it does not widen the scan. Measured on this
+  repository through this very workflow: run
+  [31051347230](https://github.com/hseshadr/ci/actions/runs/31051347230) (push to `main`)
+  scanned **0 commits** and reported success; run
+  [30978634362](https://github.com/hseshadr/ci/actions/runs/30978634362) (pull_request)
+  scanned **1**; run
+  [30793713570](https://github.com/hseshadr/ci/actions/runs/30793713570) (schedule)
+  scanned **41**. The workflow is `workflow_call`-only and every caller in `examples/` but
+  one ran on push/PR, so no consumer's pre-existing history had ever been scanned by CI.
+- **The caller owns the schedule.** A `workflow_call` workflow cannot carry its own
+  `schedule:`, so the fix is not inside the brick. Every consumer already has
+  `security-audit.yml` on a weekly cron; each `examples/*/security-audit.yml` now calls
+  `secret-scan.yml` from it with `full-history: true`.
+- **`full-history` asserts, it does not widen.** Nothing can widen the range from inside a
+  reusable workflow. The input makes the expectation machine-checked: on any event that
+  scans a partial range the job fails instead of reporting a clean partial scan as clean.
+  Every run also now prints what it actually covered, because "No leaks detected" over 0
+  commits and over 41 commits are the same three words.
+- **Every secret-scan caller job is named.** An unnamed one reports as `gitleaks /
+  gitleaks` instead of the documented `Secret scan / gitleaks`, silently orphaning an
+  adopter's required status check. Five of the six callers shipped in `examples/` omitted
+  the `name:`, as did the README's canonical copy-paste snippet.
+- **Two guards, both shown failing.** `validate_secret_scan_history_sweep` refuses an
+  `examples/` tree where a repo calls `secret-scan.yml` but never from a scheduled
+  workflow, and refuses an unnamed caller job; it carries a vacuity floor.
+  `validate_secret_scan_coverage_cases` executes the workflow's real coverage script under
+  both event families instead of grepping for its error string.
+- **README: 16 false claims fixed or deleted.** The file had never been updated past
+  `ci-v3.0.0` while three releases and one consumer adoption landed. Corrected: the current
+  release and every `2a575cd` pin (now `605e51c` / `ci-v3.2.1`), the adoption count (7
+  call-sites across 5 repos, not 6 across 4), the drift count (29, not three different
+  numbers), the third-party pin table (exact versions — a `# v6` comment on a SHA is the
+  defect commit `ae644d7` fixed), the publish-verification bound (14 attempts / 600s, not
+  6 / 60s), the "these repos are private" setup section (all eight are public), and the
+  repository-settings gap (branch protection and secret scanning are both on). Deleted:
+  the `--allow-unlocked` "live gap" callout, closed at `ci-v3.2.1`, and two completed
+  owner actions. The one genuinely open owner action is now stated: `ci-v3` still points
+  at `72521e7`, 21 commits behind `ci-v3.2.1`.
+- **`aml-filter/ci.yml/secret-scan` deleted from the drift allowlist.** aml-filter#93
+  merged on 2026-08-02 and the consumer now calls the brick
+  ([run 31051313153](https://github.com/hseshadr/aml-filter/actions/runs/31051313153),
+  `Secret scan / gitleaks` SUCCESS on `main`). First entry ever removed by an actual
+  convergence rather than by a bug fix. 30 -> 29.
+
 ## ci-v3.2.1 — 2026-08-04
 
 Commit `605e51cbc86f452b56edcf1c9660921da797cbfe`.