Add SecretPatternFilter — block real credential formats, not just keywords

AGENTS.md

@@ -109,21 +109,36 @@ and keyword argument is checked independently.
 
 ## Rules you must follow
 
-1. **Do not claim ModelFuzz detects secrets.** The bundled `SensitiveDataFilter` matches the
-   literal strings `secret`, `password`, and `api_key`. It does **not** recognise credential
-   formats — a real `sk-...` or `AKIA...` key passes straight through it. It is a demo default,
-   not a credential scanner. If the user needs secret detection, tell them to write a policy for
-   their own threat model.
+1. **Do not claim `SensitiveDataFilter` detects secrets.** It matches the literal strings
+   `secret`, `password`, and `api_key`. It does **not** recognise credential formats — a real
+   `sk-...` or `AKIA...` key passes straight through it. It is a demo default, not a credential
+   scanner. For credentials, reach for `SecretPatternFilter` (rule 4).
 2. **Prefer `@shield_tool(engine=my_engine)`.** The bare `@shield_tool` applies only the keyword
    default above, which is rarely what a real application wants.
 3. **`URLAllowList` is the strong bundled rule** — default-deny on hosts, rejects non-`http(s)`
    schemes, catches userinfo tricks like `http://api.internal.com@evil.com`, and walks nested
    containers so a URL hidden in a dict or list payload is still checked. Lead with it.
-4. **Only `str`, `bytes`, `list`, `tuple`, `set`, and `dict` keys and values are inspected.** A value
+4. **`SecretPatternFilter` is the credential rule.** It matches the *shape* of known credentials
+   — Anthropic, OpenAI, Stripe, AWS, GitHub, Google, Slack keys, JWTs, PEM private-key headers —
+   rather than keywords, and reports the format without ever quoting the matched text. Suggest it
+   whenever a tool argument could carry a live key. Two honest caveats to pass on: it covers
+   **listed formats only** (use `extra_patterns={"internal token": r"..."}` for bespoke ones, or
+   `patterns=` to replace the table), and it matches **shape, not validity**, so a placeholder or
+   expired key in the right shape is blocked like a live one.
+
+   ```python
+   from modelfuzz import PolicyEngine, SecretPatternFilter, URLAllowList
+
+   engine = PolicyEngine([
+       URLAllowList(allowed_domains=["api.mycompany.com"]),
+       SecretPatternFilter(),
+   ])
+   ```
+5. **Only `str`, `bytes`, `list`, `tuple`, `set`, and `dict` keys and values are inspected.** A value
    in a custom object is not inspected and will pass. Do not assume full coverage.
-5. **Policies see one argument at a time.** A rule cannot express "amount > 1000 only when
+6. **Policies see one argument at a time.** A rule cannot express "amount > 1000 only when
    account is external", because it never sees the whole call.
-6. **Catch `ModelFuzzBlockError` in the agent loop.** Feed the block reason back to the model as
+7. **Catch `ModelFuzzBlockError` in the agent loop.** Feed the block reason back to the model as
    a tool error so it can recover, rather than letting it crash the run.
 
 ## Red-teaming a target

CHANGELOG.md

@@ -4,6 +4,10 @@ All notable changes to this project are documented here.
 
 ## [Unreleased]
 
+- feat: add `SecretPatternFilter`, a bundled policy that blocks tool-call arguments carrying a recognisable credential — Anthropic, OpenAI, Stripe, AWS, GitHub, Google and Slack key formats, JWTs, and PEM private-key headers. Where `SensitiveDataFilter` matches the *word* "password", this matches the *shape* of a real key, closing the gap where a live `sk-…` or `AKIA…` passed straight through the bundled default. Opt-in: the bare `@shield_tool` default is unchanged. Extend with `extra_patterns=` or replace the table with `patterns=`. Fixes #70
+- fix: the block reason for a matched credential names the format only and never quotes the matched text — blocks are logged at `WARNING`, and a reason carrying the key would leak the very thing the rule exists to contain
+- docs: add a "Blocking real credentials" README section, record `SecretPatternFilter`'s limits (listed formats only; matches shape, not validity) in Limitations, and update `AGENTS.md` so assistants stop reporting that ModelFuzz cannot detect credentials
+
 - feat: `scan` gains `--attacker-model`, `--attacker-endpoint`, and `--attacker-api-key`, decoupling payload mutation from the target model (defaults to the target for backwards compatibility). Previously the attacker call always used the target model, so an aligned target refused to author an injection against itself and every lineage died at generation 1 — the adaptive fuzzer never adapted against exactly the models worth testing. Fixes #35
 - docs: qualify the README's "may still fall to a later mutation" claim — that only holds when `--attacker-model` points at a model willing to author an injection; with the default (attacker == target), an aligned target's refusal to attack itself is the more common outcome
 

README.md

@@ -79,7 +79,37 @@ except ModelFuzzBlockError as e:
 
 Blocks are also logged at `WARNING` on the `modelfuzz` logger with structured fields (`modelfuzz_tool`, `modelfuzz_rule`, `modelfuzz_reason`) for your audit trail. Nothing is ever written to stdout.
 
-> **Using the bare `@shield_tool`?** It applies a default `SensitiveDataFilter` that matches the literal strings `secret`, `password`, and `api_key` — a demo default, not a credential scanner. See [Limitations](#limitations).
+> **Using the bare `@shield_tool`?** It applies a default `SensitiveDataFilter` that matches the literal strings `secret`, `password`, and `api_key` — a demo default, not a credential scanner. For real credential formats, add [`SecretPatternFilter`](#blocking-real-credentials) to your engine. See [Limitations](#limitations).
+
+### Blocking real credentials
+
+`SensitiveDataFilter` matches the *word* "password". `SecretPatternFilter` matches the *shape* of a real credential, so an agent talked into pasting a live key into a tool argument is stopped before the call runs:
+
+```python
+from modelfuzz import PolicyEngine, SecretPatternFilter, URLAllowList, shield_tool
+
+engine = PolicyEngine([
+    URLAllowList(allowed_domains=["api.mycompany.com"]),
+    SecretPatternFilter(),
+])
+
+@shield_tool(engine=engine)
+def http_post(url: str, body: str) -> str:
+    return f"POST {url}"
+
+http_post("https://api.mycompany.com/v1", "AKIAIOSFODNN7EXAMPLE")
+# ModelFuzzBlockError: String contains a possible AWS access key ID
+```
+
+It recognises Anthropic, OpenAI, Stripe, AWS, GitHub, Google, and Slack key formats, JWTs, and PEM private-key headers. The block reason names the *format* and never quotes the matched text — blocks are logged, and a reason carrying the credential would leak the very thing the rule exists to contain.
+
+Cover your own token formats without giving up the bundled ones:
+
+```python
+SecretPatternFilter(extra_patterns={"internal token": r"INT-[0-9]{8}"})
+```
+
+Pass `patterns=` instead of `extra_patterns=` to replace the bundled table entirely. It is a format matcher, not a validity check or an entropy scanner — see [Limitations](#limitations).
 
 ## When to use ModelFuzz
 
@@ -95,7 +125,7 @@ Blocks are also logged at `WARNING` on the `modelfuzz` logger with structured fi
 
 - **If** your application only generates or classifies text and calls no tools, **then** ModelFuzz adds nothing — there is no tool call to intercept.
 - **If** you need prompt filtering, input sanitisation, or content moderation, **then** ModelFuzz is the wrong layer. It never inspects prompts or model output, only tool-call arguments.
-- **If** you expect the bundled default to detect credentials, **then** see [Limitations](#limitations) first — `SensitiveDataFilter` matches three literal keywords and will not catch a real `sk-…` or `AKIA…` key. Write a policy for your own threat model.
+- **If** you expect the bundled *default* to detect credentials, **then** see [Limitations](#limitations) first — `SensitiveDataFilter` matches three literal keywords and will not catch a real `sk-…` or `AKIA…` key. Add [`SecretPatternFilter`](#blocking-real-credentials) for that, and write policies for anything specific to your own threat model.
 
 Building on this with an AI coding assistant? See [AGENTS.md](AGENTS.md).
 
@@ -173,7 +203,8 @@ Output:
 
 ModelFuzz is pre-1.0 and provides the interception point, the policy protocol, and an adaptive fuzzer. Know these before relying on it:
 
-- **The default filter is a keyword tripwire, not a secret scanner.** `SensitiveDataFilter` matches the literal strings `secret`, `password`, and `api_key`. It does not recognise credential formats, so a real `sk-…` or `AKIA…` key passes straight through — while ordinary prose containing "password" is blocked. Treat it as a demo default and write policies for your own threat model.
+- **The default filter is a keyword tripwire, not a secret scanner.** `SensitiveDataFilter` matches the literal strings `secret`, `password`, and `api_key`. It does not recognise credential formats, so a real `sk-…` or `AKIA…` key passes straight through — while ordinary prose containing "password" is blocked. Treat it as a demo default; add `SecretPatternFilter` for credential formats, and write policies for your own threat model.
+- **`SecretPatternFilter` matches known formats, not secrets in general.** It recognises the credential shapes listed in [Blocking real credentials](#blocking-real-credentials) and nothing else: a bespoke internal token, a bare high-entropy string, or a provider not in the table passes untouched. It also matches *shape, not validity* — a revoked key, a docs placeholder, or a test fixture in the right shape is blocked exactly like a live credential. Use `extra_patterns=` for your own formats.
 - **Unrecognised argument types are not inspected, and pass.** Only `str`, `bytes`, `list`, `tuple`, `set`, and `dict` keys and values are walked. A secret carried in a custom object is *not* checked and the call proceeds — the default is to allow what it cannot read.
 - **Policies see one argument at a time.** A rule cannot express "amount > 1000 only when account is external", because it never sees the whole call.
 - **It does not inspect prompts or model output** — only tool-call arguments. It is not a content filter.

src/modelfuzz/__init__.py

@@ -5,11 +5,19 @@
 from modelfuzz.decorator import shield_tool
 from modelfuzz.engine import PolicyEngine, PolicyResult
 from modelfuzz.exceptions import ModelFuzzBlockError
-from modelfuzz.rules import SensitiveDataFilter, URLAllowList, Violation
+from modelfuzz.rules import (
+    DEFAULT_SECRET_PATTERNS,
+    SecretPatternFilter,
+    SensitiveDataFilter,
+    URLAllowList,
+    Violation,
+)
 
 __all__ = [
     "shield_tool",
     "ModelFuzzBlockError",
+    "DEFAULT_SECRET_PATTERNS",
+    "SecretPatternFilter",
     "SensitiveDataFilter",
     "URLAllowList",
     "Violation",

src/modelfuzz/rules.py

@@ -1,5 +1,7 @@
 """Security rules for ModelFuzz."""
 
+import re
+from collections.abc import Iterator
 from dataclasses import dataclass
 from urllib.parse import urlparse
 
@@ -12,6 +14,41 @@ class Violation:
     reason: str
 
 
+def _iter_strings(data: object, seen: set[int]) -> Iterator[str]:
+    """Yield every string reachable from a tool-call argument.
+
+    Walks ``dict`` keys as well as values, decodes ``bytes``/``bytearray`` as
+    UTF-8, and recurses into ``list``/``tuple``/``set``/``frozenset``. ``seen``
+    carries container ``id()`` values so a self-referential argument -- which a
+    hand-built call can contain even though a JSON-derived one cannot -- ends
+    the walk instead of recursing forever.
+
+    Anything else (an int, a ``None``, a custom object) yields nothing: rules
+    built on this walk cannot read those carriers, and the documented default is
+    to allow what they cannot read.
+    """
+    if isinstance(data, str):
+        yield data
+
+    elif isinstance(data, (bytes, bytearray)):
+        yield data.decode("utf-8", errors="ignore")
+
+    elif isinstance(data, dict):
+        if id(data) in seen:
+            return
+        seen.add(id(data))
+        for key, value in data.items():
+            yield from _iter_strings(key, seen)
+            yield from _iter_strings(value, seen)
+
+    elif isinstance(data, (list, tuple, set, frozenset)):
+        if id(data) in seen:
+            return
+        seen.add(id(data))
+        for item in data:
+            yield from _iter_strings(item, seen)
+
+
 DEFAULT_URL_SCHEMES = frozenset({"http", "https"})
 
 
@@ -122,7 +159,13 @@ def _block(reason: str) -> Violation:
 
 
 class SensitiveDataFilter:
-    """A policy that blocks strings containing sensitive keywords."""
+    """A policy that blocks strings containing sensitive keywords.
+
+    This is a keyword tripwire, not a credential scanner: it matches the literal
+    strings it is given, so it flags ordinary prose containing "password" while
+    a real ``sk-...`` or ``AKIA...`` key passes straight through. For credential
+    formats use :class:`SecretPatternFilter`.
+    """
 
     def __init__(self, sensitive_keywords: list[str] | None = None) -> None:
         self.sensitive_keywords = (
@@ -142,42 +185,111 @@ def __call__(self, data: object) -> Violation | None:
         Returns:
             A Violation object if sensitive data is found, otherwise None.
         """
-        return self._check_recursive(data, set())
-
-    def _check_recursive(self, data: object, seen: set[int]) -> Violation | None:
-        if isinstance(data, str):
-            lower_data = data.lower()
+        for text in _iter_strings(data, set()):
+            lower_text = text.lower()
             for keyword in self.sensitive_keywords:
-                if keyword in lower_data:
+                if keyword in lower_text:
                     return Violation(
                         rule_name="SensitiveDataFilter",
                         reason=f"String contains sensitive keyword: '{keyword}'",
                     )
-        elif isinstance(data, dict):
-            if id(data) in seen:
-                return None
-            seen.add(id(data))
+        return None
 
-            for key, value in data.items():
-                violation = self._check_recursive(key, seen)
-                if violation:
-                    return violation
 
-                violation = self._check_recursive(value, seen)
-                if violation:
-                    return violation
+# Credential formats that are recognisable on sight. Each entry is a
+# (label, pattern) pair; the label names the credential in the block reason.
+#
+# Order matters: the first match wins, so a more specific format is listed
+# before a broader one that would also match it -- an Anthropic key
+# ("sk-ant-...") is also a match for the generic OpenAI "sk-..." shape, and
+# should be reported as the former.
+#
+# These are format matchers, not proof of validity: a revoked key and a live one
+# look identical, and a random string in the same shape trips the same wire.
+DEFAULT_SECRET_PATTERNS: tuple[tuple[str, str], ...] = (
+    # The leading \b matters: without it "sk-" matches inside ordinary hyphenated
+    # words -- "task-oriented-…", "risk-management-…" -- and long ones would trip
+    # the wire as OpenAI keys.
+    ("Anthropic API key", r"\bsk-ant-[A-Za-z0-9_-]{16,}"),
+    ("OpenAI API key", r"\bsk-(?:proj-)?[A-Za-z0-9_-]{20,}"),
+    ("Stripe secret key", r"\b[sr]k_(?:live|test)_[A-Za-z0-9]{16,}"),
+    ("AWS access key ID", r"\b(?:AKIA|ASIA|ABIA|ACCA)[0-9A-Z]{16}\b"),
+    ("GitHub fine-grained token", r"\bgithub_pat_[A-Za-z0-9_]{22,}"),
+    ("GitHub token", r"\bgh[pousr]_[A-Za-z0-9]{36,}"),
+    ("Google API key", r"\bAIza[0-9A-Za-z_-]{35}\b"),
+    ("Slack token", r"\bxox[abprs]-[A-Za-z0-9-]{10,}"),
+    ("JSON Web Token", r"\beyJ[A-Za-z0-9_-]{8,}\.eyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]+"),
+    ("private key block", r"-----BEGIN (?:[A-Z0-9 ]+ )?PRIVATE KEY-----"),
+)
+
+
+class SecretPatternFilter:
+    """A policy that blocks arguments carrying a recognisable credential.
+
+    Where :class:`SensitiveDataFilter` matches the *word* "password", this
+    matches the *shape* of a real credential -- ``sk-ant-...``, ``AKIA...``,
+    ``ghp_...``, a JWT, a PEM private-key header -- so an agent that has been
+    talked into pasting a live key into a tool argument is stopped before the
+    call runs.
+
+    Containers are walked exactly as :class:`SensitiveDataFilter` walks them:
+    ``dict`` keys and values, ``bytes`` decoded as UTF-8, and nested
+    ``list``/``tuple``/``set``/``frozenset``, with a guard against
+    self-referential arguments.
+
+    The block reason never quotes the matched text. A violation is logged at
+    ``WARNING`` with the reason attached, and a rule that echoed the credential
+    into the audit trail would leak the very thing it exists to contain -- so it
+    names the format and where it was found, and nothing else.
+
+    Known limits, in the same spirit as the rest of the bundled rules:
+
+    - It recognises *listed formats only*. A bespoke internal token, a bare
+      high-entropy string, or a provider not in the table passes untouched.
+      Pass ``extra_patterns`` for formats specific to your own systems.
+    - It matches shape, not validity. An expired key, a documentation
+      placeholder, or a test fixture in the right shape is blocked the same as a
+      live credential.
+    - Only values reachable through the walk above are inspected. A credential
+      held in a custom object is not seen, and the call proceeds.
+    """
 
-        elif isinstance(data, (bytes, bytearray)):
-            return self._check_recursive(data.decode("utf-8", errors="ignore"), seen)
+    def __init__(
+        self,
+        patterns: dict[str, str] | None = None,
+        extra_patterns: dict[str, str] | None = None,
+    ) -> None:
+        """Build the filter.
 
-        elif isinstance(data, (list, tuple, set, frozenset)):
-            if id(data) in seen:
-                return None
-            seen.add(id(data))
+        Args:
+            patterns: Replaces the bundled table entirely. Use this to scan for
+                only your own formats.
+            extra_patterns: Added to the bundled table, checked after it. Use
+                this -- the common case -- to cover an internal token format
+                without giving up coverage of the well-known providers.
+        """
+        table = DEFAULT_SECRET_PATTERNS if patterns is None else tuple(patterns.items())
+        if extra_patterns:
+            table = (*table, *extra_patterns.items())
+        self.patterns: tuple[tuple[str, re.Pattern[str]], ...] = tuple(
+            (label, re.compile(pattern)) for label, pattern in table
+        )
 
-            for item in data:
-                violation = self._check_recursive(item, seen)
-                if violation:
-                    return violation
+    def __call__(self, data: object) -> Violation | None:
+        """Check every string reachable from a value for a known credential format.
+
+        Args:
+            data: The value to check. Containers are walked recursively.
 
+        Returns:
+            A Violation naming the credential format if one is found, otherwise
+            None. The matched text is deliberately not included.
+        """
+        for text in _iter_strings(data, set()):
+            for label, pattern in self.patterns:
+                if pattern.search(text):
+                    return Violation(
+                        rule_name="SecretPatternFilter",
+                        reason=f"String contains a possible {label}",
+                    )
         return None