Detection par la forme : retrait de la deny-list en clair

.gitattributes

@@ -0,0 +1,6 @@
+# scripts/check_no_client_leak.py is copied byte-for-byte from the canonical source
+# (Statyx/Fab-Analyze-Data-Agent). Its SHA-256 is the proof it was not edited, so it must
+# keep LF endings on every platform — core.autocrlf=true would otherwise rewrite it to CRLF
+# on checkout and break the hash for anyone verifying it on Windows.
+scripts/check_no_client_leak.py text eol=lf
+scripts/check_client_denylist.py text eol=lf

.github/workflows/no-client-leak.yml

@@ -13,36 +13,24 @@ jobs:
     steps:
       - uses: actions/checkout@v4
 
-      # This repo is public. Any real customer name or hardcoded Entra/Fabric identifier
-      # is a leak, so the build fails loudly instead of quietly shipping it.
-      # The names below are written with a single letter parenthesised (ERE groups match the
-      # same string) so that this guard file is not itself a hit for its own scan.
-      - name: Scan for client names
-        run: |
-          PATTERN='p(u)blicis|v(i)vatech|o(r)ange|a(i)r.?liquide|t(e)chnip|f(o)rvia|v(e)olia|a(r)kema|c(a)pgemini|e(i)ffage|o(p)mobility'
-          if grep -rniE "$PATTERN" \
-               --exclude-dir=.git \
-               --exclude-dir=node_modules \
-               --exclude-dir=__pycache__ \
-               . ; then
-            echo "::error::A known client name appears in the repository. Anonymise it before merging."
-            exit 1
-          fi
-          echo "OK - no client name found."
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
 
-      - name: Scan for hardcoded clientId / tenantId GUIDs
-        run: |
-          PATTERN='(client_?id|tenant_?id)["'"'"']?[[:space:]]*[:=][[:space:]]*["'"'"'][0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}'
-          if grep -rniE "$PATTERN" \
-               --exclude-dir=.git \
-               --exclude-dir=node_modules \
-               --exclude-dir=__pycache__ \
-               --exclude-dir=workflows \
-               . ; then
-            echo "::error::A clientId/tenantId GUID is hardcoded. Inject it from the environment instead."
-            exit 1
-          fi
-          echo "OK - no hardcoded clientId/tenantId."
+      # This repo is public. Detection is by SHAPE, never by name: a list of customer
+      # names committed here would itself be the leak it is meant to prevent, and
+      # obfuscating that list only fools full-text search, not a reader.
+      # scripts/check_no_client_leak.py is copied byte-for-byte from the canonical
+      # source (Statyx/Fab-Analyze-Data-Agent@6e4a65e7) — do not edit it locally.
+      - name: Structural scan (GUIDs, Fabric endpoints, personal paths)
+        run: python scripts/check_no_client_leak.py
+
+      # Name-based detection is preserved, but the names live in a secret, never in
+      # the source tree. Secret missing (forks, external PRs) -> warning, still green.
+      - name: Client deny-list scan (from secret, optional)
+        env:
+          CLIENT_DENYLIST: ${{ secrets.CLIENT_DENYLIST }}
+        run: python scripts/check_client_denylist.py
 
   tests:
     name: pytest

.gitignore

@@ -28,6 +28,11 @@ _*.json
 Thumbs.db
 *.swp
 
+# Local client deny-list read by scripts/check_client_denylist.py. It holds real customer
+# names, so it must never be committed to this public repo — in CI the same list comes from
+# the CLIENT_DENYLIST secret instead.
+.clientdeny
+
 # Environment-specific — contain tenant / workspace / item IDs. Use the *.example files as templates.
 src/config.yaml
 src/state.json

README.md

@@ -7,7 +7,7 @@
 ![Microsoft Fabric](https://img.shields.io/badge/Microsoft-Fabric-0078D4?logo=microsoft&logoColor=white)
 ![Fabric IQ](https://img.shields.io/badge/Fabric_IQ-Ontology_+_Graph-6242C9)
 ![Python](https://img.shields.io/badge/Python-3.12-3776AB?logo=python&logoColor=white)
-![Tests](https://img.shields.io/badge/tests-39_passing-2E9E44)
+![Tests](https://img.shields.io/badge/tests-50_passing-2E9E44)
 ![Portal](https://img.shields.io/badge/Portal-FastAPI-009688?logo=fastapi&logoColor=white)
 ![Deploy](https://img.shields.io/badge/deploy-idempotent-896610)
 
@@ -77,7 +77,7 @@ flowchart LR
 copy src\config.example.yaml src\config.yaml
 
 # 2. ✅ Mandatory test gate — never deploy on red
-python -m pytest tests/ -v --tb=short          # 39/39 offline
+python -m pytest tests/ -v --tb=short          # 50/50 offline
 
 # 3. Deploy everything (idempotent — generates data, then ends with a warm-up)
 python src\deploy_all.py
@@ -194,6 +194,7 @@ node build_deck.js          # → presentation/Architecture_Vision.pptx
 - ✅ **Test gate is mandatory** — `python -m pytest tests/ -v` before any `deploy_*.py`. On red: **stop, fix the code**.
 - ♻️ **Idempotent deploys** — every step reads/writes `src/state.json`; re-running is safe.
 - 🔐 **Secrets never committed** — `src/config.yaml`, `src/state.json`, `.vscode/mcp.json` are gitignored (use the `*.example` templates). No tenant / workspace / app-registration ID is hardcoded anywhere; CI enforces this.
+- 🕵️ **Leak detection is by shape, not by name** — `scripts/check_no_client_leak.py` (copied byte-for-byte from the canonical source, do not edit it) flags real GUIDs, real Fabric SQL endpoints and personal paths in Git-tracked files. Customer names are matched separately by `scripts/check_client_denylist.py`, whose list comes from the `CLIENT_DENYLIST` repository secret (one name per line) or a gitignored local `.clientdeny`. **The names are never committed to this public repo** — that list would be the leak. No secret configured: the rule warns and passes.
 - 🧪 **Synthetic data only** — every organisation, venue and sponsor in this repo is fictional.
 - ⚡ **F2+ capacity** — resume the capacity before a deploy/demo (Ontology / Graph / agents need it).
 - 🧠 **Reports are Legacy PBIX** — see the report gotchas below.

scripts/check_client_denylist.py

@@ -0,0 +1,178 @@
+#!/usr/bin/env python3
+"""Fail if a name from the private client deny-list appears in Git-tracked files.
+
+This repo is public, so the deny-list itself must never be committed here. The
+list is injected at run time and stays out of the source tree:
+
+    1. ``CLIENT_DENYLIST`` -- one entry per line (GitHub Actions secret).
+    2. ``.clientdeny`` at the repo root -- gitignored, for local runs.
+
+Neither present? The rule is skipped with a warning and the build stays green.
+An absent secret must never break a fork, a PR from outside the org, or a
+contributor's laptop.
+
+Only files returned by ``git ls-files`` are scanned, for the same reason as the
+structural checker: ``__pycache__/*.pyc`` and ``node_modules`` embed absolute
+paths and produce false positives.
+
+Two deliberate properties:
+
+* **Nothing is echoed.** A finding prints the file, the line number and a hash
+  prefix of the entry -- never the entry itself, never the source line. CI logs
+  of a public repo are public: a chatty scanner would re-publish the very name
+  it is guarding.
+* **Colour contexts are ignored.** Some client names double as ordinary colour
+  words. A hit on a line that is plainly about colour -- a shields.io badge, a
+  hex token, ``rgb()``, a CSS declaration -- is dropped. The rule is generic; no
+  name and no colour is hardcoded here.
+
+Usage:
+    python scripts/check_client_denylist.py
+
+Exit code 0 = clean or not configured, 1 = a deny-listed name was found.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import os
+import re
+import subprocess
+import sys
+from pathlib import Path
+
+REPO_ROOT = Path(__file__).resolve().parent.parent
+LOCAL_DENYLIST = REPO_ROOT / ".clientdeny"
+
+BINARY_SUFFIXES = {
+    ".png", ".jpg", ".jpeg", ".gif", ".ico", ".pdf", ".pptx", ".xlsx",
+    ".docx", ".zip", ".gz", ".woff", ".woff2", ".ttf", ".eot",
+}
+
+# A line that is plainly about colour. Generic on purpose: it keys on the
+# colour syntax around the hit, not on which word was matched.
+COLOUR_CONTEXT_RE = re.compile(
+    r"""
+      img\.shields\.io                      # badge URLs: .../badge/x-y-<colour>
+    | \#[0-9A-Fa-f]{3,8}\b                  # hex token
+    | \brgba?\s*\(                          # rgb() / rgba()
+    | \bhsla?\s*\(                          # hsl() / hsla()
+    | (?:^|[;{,\s])(?:color|background|background-color|fill|stroke|border
+        |border-color|outline|box-shadow|--[\w-]+)\s*:   # CSS declaration
+    | "(?:color|colour|fill|stroke|background)"\s*:      # JSON theme key
+    """,
+    re.IGNORECASE | re.VERBOSE,
+)
+
+# The obfuscation this whole change exists to remove: a single character wrapped
+# in parentheses. `n(a)me` still matches as an ERE and reads fine to a human, but
+# no full-text search for the name finds it. Parentheses are stripped before
+# matching so the trick buys nothing.
+SINGLE_CHAR_PAREN_RE = re.compile(r"\(([A-Za-z0-9])\)")
+
+
+def load_denylist() -> tuple[list[str], str]:
+    """Return (entries, source-label). Empty list means "not configured"."""
+    raw = os.environ.get("CLIENT_DENYLIST", "")
+    source = "the CLIENT_DENYLIST environment variable"
+    if not raw.strip() and LOCAL_DENYLIST.is_file():
+        raw = LOCAL_DENYLIST.read_text(encoding="utf-8", errors="ignore")
+        source = str(LOCAL_DENYLIST.name)
+
+    entries = []
+    for line in raw.splitlines():
+        entry = line.strip()
+        if entry and not entry.startswith("#"):
+            entries.append(entry)
+    return entries, source
+
+
+def compile_entry(entry: str) -> re.Pattern[str]:
+    """Word-bounded, case-insensitive, separator-tolerant literal match.
+
+    A two-word entry also catches the hyphenated, underscored and glued
+    spellings -- the separator is where obfuscation is cheapest.
+    """
+    parts = [re.escape(part) for part in entry.split()]
+    body = r"[\s._\-]*".join(parts)
+    return re.compile(rf"\b{body}\b", re.IGNORECASE)
+
+
+def fingerprint(entry: str) -> str:
+    """Stable, non-reversible label so a maintainer can tell entries apart."""
+    digest = hashlib.sha256(entry.strip().lower().encode("utf-8")).hexdigest()
+    return digest[:8]
+
+
+def tracked_files() -> list[str]:
+    out = subprocess.run(
+        ["git", "ls-files", "-z"],
+        cwd=REPO_ROOT,
+        capture_output=True,
+        text=True,
+        check=True,
+    ).stdout
+    return [p for p in out.split("\0") if p]
+
+
+def scan_file(rel_path: str, patterns: list[tuple[int, str, re.Pattern[str]]]) -> list[str]:
+    path = REPO_ROOT / rel_path
+    if path.suffix.lower() in BINARY_SUFFIXES or not path.is_file():
+        return []
+    try:
+        text = path.read_text(encoding="utf-8", errors="ignore")
+    except OSError:
+        return []
+
+    findings = []
+    for lineno, line in enumerate(text.splitlines(), start=1):
+        colour_line = COLOUR_CONTEXT_RE.search(line) is not None
+        if colour_line:
+            continue  # the word is being used as a colour, not as a name
+        variants = {line, SINGLE_CHAR_PAREN_RE.sub(r"\1", line)}
+        for index, mark, pattern in patterns:
+            if any(pattern.search(variant) for variant in variants):
+                findings.append(f"{rel_path}:{lineno}: deny-list entry #{index} [{mark}]")
+    return findings
+
+
+def main() -> int:
+    entries, source = load_denylist()
+    if not entries:
+        print(
+            "::warning::Client deny-list not configured "
+            "(set the CLIENT_DENYLIST secret, or a gitignored .clientdeny file) "
+            "- name-based scanning skipped. Structural checks still ran."
+        )
+        return 0
+
+    patterns = [
+        (index, fingerprint(entry), compile_entry(entry))
+        for index, entry in enumerate(entries, start=1)
+    ]
+
+    findings: list[str] = []
+    for rel_path in tracked_files():
+        findings.extend(scan_file(rel_path, patterns))
+
+    if findings:
+        # Entries are reported by index + hash prefix only: printing the name
+        # here would leak it into public CI logs.
+        print(
+            f"Deny-listed client name(s) found in Git-tracked files "
+            f"({len(entries)} entries loaded from {source}):\n"
+        )
+        for finding in findings:
+            print(f"  {finding}")
+        print(
+            "\nThe matched text is deliberately not shown. Open the file at the "
+            "line above and anonymise it before merging."
+        )
+        return 1
+
+    print(f"No deny-listed client name found ({len(entries)} entries checked).")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())