From f49a6ffddf0f1550e3cbdfed3168ed614d35af65 Mon Sep 17 00:00:00 2001 From: Juniper Bevensee Date: Wed, 17 Jun 2026 14:16:07 +1200 Subject: [PATCH 1/2] sanitization: close the gate gaps (pytest CI, intl phone, semantic enforcement) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The commons gate had the same gaps found in the Matilde package (which was built from this template): - CI bug: the self-test step ran `python -m pytest` but only `anthropic` was installed → "No module named pytest" failed the gate on every PR, so it never actually enforced. Install pytest. - CI now passes `--require-semantic`: the LLM layer is a hard requirement, not advisory — a content-bearing change with no ANTHROPIC_API_KEY fails loudly instead of silently passing deterministic-only. (Repos must set the secret; onboarding already lists this step. Documented in CONTRIBUTING.) - New deterministic `intl-phone` detector (E.164 +<7–15 digits>): the us-phone pattern only caught North-American formatting, missing international contacts. - Semantic config: added a flag example for messaging/user identifiers (Signal /Telegram UUID, contact phone, allowlisted user ID) — these belong in deployment config/secrets, never in the package. The semantic layer is the right home (it distinguishes a user UUID from a dataset version UUID; a fail-closed regex can't). TDD: new test_detects_international_phone (red→green); 14/14 pass. Smoke: gate hard-fails on +442079460958. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/sanitization.yml | 11 ++++++++--- CONTRIBUTING.md | 8 +++++--- sanitize.config.json | 1 + scripts/check_sanitization.py | 3 +++ tests/test_check_sanitization.py | 8 ++++++++ 5 files changed, 25 insertions(+), 6 deletions(-) diff --git a/.github/workflows/sanitization.yml b/.github/workflows/sanitization.yml index c10cdc8..337e41b 100644 --- a/.github/workflows/sanitization.yml +++ b/.github/workflows/sanitization.yml @@ -24,16 +24,21 @@ jobs: with: python-version: "3.12" - - name: Install semantic-layer dep - run: pip install anthropic + - name: Install deps (semantic layer + self-tests) + run: pip install anthropic pytest - name: Scanner self-tests (deterministic, no key needed) run: python -m pytest tests/test_check_sanitization.py -q - name: Sanitization gate (diff) + # --require-semantic makes the LLM layer a HARD requirement: if a changed + # content-bearing file is present and ANTHROPIC_API_KEY is missing, the gate + # fails (exit 2) instead of silently passing on deterministic-only. This + # repo (and every package built from it) MUST set the ANTHROPIC_API_KEY + # repo secret, or sanitization PRs will fail with a clear "key missing" error. env: ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} run: | BASE="origin/${{ github.base_ref }}" git diff --name-only "$BASE...HEAD" \ - | xargs -r python scripts/check_sanitization.py + | xargs -r python scripts/check_sanitization.py --require-semantic diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a015f24..c805a65 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -60,9 +60,11 @@ flag is **not** an automatic rejection — it routes the PR to a human maintaine makes the call. If your PR is flagged and you believe it's clean, say so in the PR description; a maintainer reviews the flagged content and decides. -The semantic layer runs when `ANTHROPIC_API_KEY` is set in CI secrets. Without the -key it skips with a warning (and the `--require-semantic` flag turns that skip into a -hard failure, for maintainers who need the full gate in CI). +The semantic layer needs `ANTHROPIC_API_KEY`. CI runs with `--require-semantic`, so +this key **must** be set as a repo Actions secret — without it, any PR that touches a +content-bearing file fails with a clear "key missing" error rather than silently +passing on the deterministic layer alone. (Locally the key is optional: the layer +skips with a warning so offline runs still work.) To tune what counts as a particular for this domain, edit `sanitize.config.json` — that is the single configuration surface for the gate. See the inline comments in that diff --git a/sanitize.config.json b/sanitize.config.json index 5ff9790..010ee5d 100644 --- a/sanitize.config.json +++ b/sanitize.config.json @@ -19,6 +19,7 @@ "personal names of subjects, clients, targets, or end-users tied to a specific engagement", "specific organization/company names tied to one active engagement (not generic/public reference)", "document IDs, file names, dataset names, account handles, or URLs specific to one engagement", + "a messaging or account identifier tied to a real person — a Signal/Telegram UUID, a contact phone, an allowlisted user ID (these belong in deployment config/secrets, never in the package)", "a codename or label that identifies one particular engagement (e.g. used as a skill name or heading)", "dates, locations, monetary figures, or case details that only make sense for one engagement" ], diff --git a/scripts/check_sanitization.py b/scripts/check_sanitization.py index 55213fc..ab33baa 100644 --- a/scripts/check_sanitization.py +++ b/scripts/check_sanitization.py @@ -59,6 +59,9 @@ def load_config(root="."): ("private-key-block", re.compile(r"-----BEGIN (?:RSA |EC |OPENSSH |DSA |PGP )?PRIVATE KEY-----")), ("email", re.compile(r"\b[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}\b")), ("us-phone", re.compile(r"(? Date: Wed, 17 Jun 2026 14:26:53 +1200 Subject: [PATCH 2/2] sanitization: semantic enforcement is opt-in (post-key), not red-by-default MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Shipping a scaffold with red-by-default CI (every clone starts failing until an admin wires a secret) is bad ergonomics. The semantic layer still runs whenever ANTHROPIC_API_KEY is present; --require-semantic is documented as the flip-the- switch hardening to enable once the key is wired (workflow comment + CONTRIBUTING). Surfaced finding: the template repo has no ANTHROPIC_API_KEY secret, so its semantic layer has never run in CI — needs wiring to enable enforcement. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/sanitization.yml | 14 ++++++++------ CONTRIBUTING.md | 13 ++++++++----- 2 files changed, 16 insertions(+), 11 deletions(-) diff --git a/.github/workflows/sanitization.yml b/.github/workflows/sanitization.yml index 337e41b..434057f 100644 --- a/.github/workflows/sanitization.yml +++ b/.github/workflows/sanitization.yml @@ -31,14 +31,16 @@ jobs: run: python -m pytest tests/test_check_sanitization.py -q - name: Sanitization gate (diff) - # --require-semantic makes the LLM layer a HARD requirement: if a changed - # content-bearing file is present and ANTHROPIC_API_KEY is missing, the gate - # fails (exit 2) instead of silently passing on deterministic-only. This - # repo (and every package built from it) MUST set the ANTHROPIC_API_KEY - # repo secret, or sanitization PRs will fail with a clear "key missing" error. + # The semantic (LLM) layer runs automatically when ANTHROPIC_API_KEY is set + # as a repo Actions secret — strongly recommended, since it's the layer that + # catches engagement particulars the deterministic regexes can't. Without the + # key it skips with a loud warning (deterministic layer still hard-fails on + # secrets/PII). To make the semantic layer MANDATORY once your key is wired, + # append `--require-semantic` below: the gate then fails closed if the key is + # ever missing instead of silently passing deterministic-only. env: ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} run: | BASE="origin/${{ github.base_ref }}" git diff --name-only "$BASE...HEAD" \ - | xargs -r python scripts/check_sanitization.py --require-semantic + | xargs -r python scripts/check_sanitization.py diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c805a65..c76c183 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -60,11 +60,14 @@ flag is **not** an automatic rejection — it routes the PR to a human maintaine makes the call. If your PR is flagged and you believe it's clean, say so in the PR description; a maintainer reviews the flagged content and decides. -The semantic layer needs `ANTHROPIC_API_KEY`. CI runs with `--require-semantic`, so -this key **must** be set as a repo Actions secret — without it, any PR that touches a -content-bearing file fails with a clear "key missing" error rather than silently -passing on the deterministic layer alone. (Locally the key is optional: the layer -skips with a warning so offline runs still work.) +The semantic layer needs `ANTHROPIC_API_KEY`. Set it as a repo Actions secret (this is +on the onboarding checklist) so the layer runs in CI — it's the part that catches +engagement particulars the regexes can't. Without the key it skips with a loud warning +(the deterministic layer still hard-fails on secrets/PII), so local/offline runs work. +**Once your key is wired, make the semantic layer mandatory** by appending +`--require-semantic` to the gate step in `.github/workflows/sanitization.yml`: the gate +then fails closed if the key is ever missing, instead of silently passing +deterministic-only. To tune what counts as a particular for this domain, edit `sanitize.config.json` — that is the single configuration surface for the gate. See the inline comments in that