-
Notifications
You must be signed in to change notification settings - Fork 75
Secrets Leak Guardrail skill #228
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Tim (Timziito)
wants to merge
5
commits into
microsoft:main
Choose a base branch
from
Timziito:add-secrets-leak-guardrail
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
a972155
Add Secrets Leak Guardrail skill
Timziito e0f9603
Merge branch 'main' into add-secrets-leak-guardrail
Timziito 389c644
Fix script usage docstring to include scripts/ prefix
Timziito 96bd1bc
Potential fix for pull request finding
Timziito 7f56b54
Address Copilot review feedback
Timziito File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| # Secrets Leak Guardrail | ||
|
|
||
| Before code, logs, or an exported file leave the conversation, this skill | ||
| scans for the credential shapes that show up by accident: an AWS key pasted | ||
| into a debugging session, a GitHub token left in a `.env` dump, a connection | ||
| string with a live password in it. | ||
|
|
||
| ## How it works | ||
|
|
||
| `scripts/scan_secrets.py` does two passes: known credential formats by regex | ||
| (AWS, GitHub, GitLab, Slack, Google, JWTs, PEM keys, bearer tokens, | ||
| `password=` connection strings), plus a Shannon-entropy check on any | ||
| `SECRET`/`TOKEN`/`PASSWORD`-named assignment that doesn't match a known | ||
| format, catching the vendor-specific key formats the regex list doesn't know | ||
| about yet. Every finding is reported, never silently fixed. The agent asks | ||
| before stripping anything, since a docs example and a live credential can look | ||
| identical to a regex. | ||
|
|
||
| ## Usage | ||
|
|
||
| ```bash | ||
| python scripts/scan_secrets.py path/to/file-or-folder | ||
| python scripts/scan_secrets.py path/to/file --json | ||
| echo "some pasted text" | python scripts/scan_secrets.py - | ||
| ``` | ||
|
|
||
| No dependencies beyond the Python standard library. | ||
|
|
||
| ## Limits | ||
|
|
||
| This is a pattern scanner, not a secrets-management tool. It won't catch a | ||
| credential format it doesn't know, and the entropy check is a heuristic that | ||
| can both miss short low-entropy passwords and flag legitimate random IDs. It | ||
| buys a last-look-before-sharing check, not a guarantee. | ||
|
|
||
| --- | ||
|
|
||
| Skill by Tim Karlsson (╯°□°)╯︵ ┻━┻ Works 60% of the time, every time. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| --- | ||
| name: secrets-leak-guardrail | ||
| description: >- | ||
| Use this skill before sharing, sending, committing, or displaying any code, | ||
| config, log output, or exported file the agent generated or was asked to | ||
| paste, to catch API keys, tokens, passwords, and private keys before they | ||
| leave the conversation. | ||
| --- | ||
|
|
||
| ## Instructions | ||
|
|
||
| 1. This applies whenever the agent is about to: write code containing real | ||
| configuration, paste log or console output, export a file, post to an | ||
| external destination (chat, email, a ticket, a repository), or answer a | ||
| question by quoting real environment variables or connection strings. | ||
|
|
||
| 2. Run the bundled scanner when a Python environment is available: | ||
|
|
||
| ```bash | ||
| python scripts/scan_secrets.py <file-or-directory> | ||
| ``` | ||
|
|
||
| It also reads stdin (`... | python scripts/scan_secrets.py -`) for pasted | ||
| text that isn't in a file yet. Without Python available, read the content | ||
| directly and check it against the patterns in the **What counts as a | ||
| secret** section below. | ||
|
|
||
| 3. The scanner flags two kinds of finding: | ||
| - **High confidence**: matched a known credential format (AWS keys, | ||
| GitHub/GitLab/Slack tokens, PEM private keys, JWTs, bearer tokens, | ||
| `password=`-style connection strings). | ||
| - **Medium confidence**: a `SECRET`/`TOKEN`/`PASSWORD`-named variable | ||
| assigned a high-entropy value that didn't match a known format. Judge | ||
| these yourself; entropy is a heuristic, not proof. | ||
|
|
||
| 4. For every finding, before sharing the content: stop and tell the user | ||
| exactly what was found and where (file/line, redacted so the actual value | ||
| isn't repeated back). Ask whether it's a real, live credential to strip, or | ||
| a test/example value that's fine to leave. Don't assume either way. | ||
|
|
||
| 5. If the user confirms it's real, redact or remove it and explain what | ||
| replaced it (an environment-variable reference, a placeholder, removal | ||
| entirely) rather than silently deleting the line. | ||
|
|
||
| 6. Never treat a finding as resolved just because it was mentioned once. | ||
| Re-scan after edits before the content actually goes out. | ||
|
|
||
| ## What counts as a secret | ||
|
|
||
| Cloud provider keys (AWS `AKIA…`/`ASIA…`, Google `AIza…`), platform tokens | ||
| (GitHub `ghp_…`, GitLab `glpat-…`, Slack `xox…`), API keys matching common | ||
| vendor formats, PEM-format private keys, JWTs, bearer tokens, and | ||
| `password=`/`pwd=` values inside connection strings. Anything else that reads | ||
| as a live, working credential even if its format isn't on this list. The | ||
| scanner's list is a floor, not a ceiling. | ||
|
|
||
| ## Guardrails | ||
|
|
||
| - Never repeat a found secret back in full, even to describe it. Always | ||
| redact the middle of the value. | ||
| - Never silently strip or silently allow a finding through. Every finding gets | ||
| surfaced to the user before the content ships, no exceptions. | ||
| - Don't flag obvious documentation placeholders (`your_api_key_here`, | ||
| `sk-EXAMPLE...`, `changeme`) as if they were real, but if genuinely | ||
| uncertain whether something is a placeholder, ask rather than assume either | ||
| way. | ||
| - This is a leak check, not a secrets-management setup. Don't turn a "found a | ||
| key in this file" moment into an unsolicited lecture on secret rotation | ||
| unless asked. | ||
|
|
||
| ## Tone | ||
|
|
||
| Direct and calm: what was found, where, what it looks like, what to do next. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| { | ||
| "name": "Secrets Leak Guardrail", | ||
| "description": "Scan code, config, logs, or exported files for API keys, tokens, and passwords before they're shared. Flags every finding to the user instead of redacting silently.", | ||
| "platforms": ["Cowork", "Copilot Studio", "Scout"], | ||
| "tags": ["security", "guardrail", "secrets", "credentials", "scripts", "governance"], | ||
| "author": "Tim Karlsson", | ||
| "authorUrl": "https://github.com/Timziito", | ||
| "version": "1.0.0", | ||
| "createdAt": "2026-07-26", | ||
| "updatedAt": "2026-07-26" | ||
| } |
158 changes: 158 additions & 0 deletions
158
submissions/secrets-leak-guardrail/scripts/scan_secrets.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,158 @@ | ||
| #!/usr/bin/env python3 | ||
| """Scan text or files for likely leaked secrets before they're shared. | ||
|
|
||
| Deterministic only: known credential formats by regex, plus a Shannon-entropy | ||
| check for `KEY = <random-looking string>` assignments that don't match a named | ||
| pattern. Judging whether a match is a *real* secret versus a fictional | ||
| documentation example is left to the agent. This script only finds | ||
| candidates. | ||
|
|
||
| Usage: | ||
| python scripts/scan_secrets.py <file-or-directory> [--json] | ||
| echo "some text" | python scripts/scan_secrets.py - | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import argparse | ||
| import json | ||
| import math | ||
| import os | ||
| import re | ||
| import sys | ||
|
|
||
| # Known formats: (name, regex). Ordered roughly by how load-bearing a false | ||
| # positive would be to explain. | ||
| PATTERNS: list[tuple[str, re.Pattern]] = [ | ||
| ("AWS Access Key ID", re.compile(r"\b(AKIA|ASIA)[0-9A-Z]{16}\b")), | ||
| ("AWS Secret Access Key (heuristic)", re.compile(r"(?i)aws_secret_access_key\s*[:=]\s*['\"]?[A-Za-z0-9/+=]{40}['\"]?")), | ||
| ("GitHub token", re.compile(r"\bgh[pousr]_[A-Za-z0-9]{36,255}\b")), | ||
| ("GitLab token", re.compile(r"\bglpat-[A-Za-z0-9\-_]{20}\b")), | ||
| ("Slack token", re.compile(r"\bxox[baprs]-[A-Za-z0-9-]{10,72}\b")), | ||
| ("OpenAI/Anthropic-style API key", re.compile(r"\b(sk|rk)-[A-Za-z0-9]{20,64}\b")), | ||
| ("Google API key", re.compile(r"\bAIza[0-9A-Za-z\-_]{35}\b")), | ||
| ("Azure Storage connection string", re.compile(r"AccountKey=[A-Za-z0-9+/=]{20,}")), | ||
| ("Generic connection-string password", re.compile(r"(?i)(password|pwd)\s*=\s*[^;'\"\s]{6,}")), | ||
| ("JWT", re.compile(r"\beyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\b")), | ||
| ("PEM private key", re.compile(r"-----BEGIN (?:RSA |EC |OPENSSH |DSA )?PRIVATE KEY-----")), | ||
| ("Bearer token", re.compile(r"(?i)authorization:\s*bearer\s+[A-Za-z0-9\-._~+/]{20,}")), | ||
| ] | ||
|
|
||
| # KEY = value assignments where the value's own shape suggests a secret, even | ||
| # with no format above matching. Ceiling: entropy is a heuristic, not proof. | ||
| # Flag as "possible", never as a confirmed hit. | ||
| ASSIGNMENT = re.compile( | ||
| r"(?im)^\s*([A-Za-z_][A-Za-z0-9_]*(?:SECRET|TOKEN|API_?KEY|PASSWORD|PWD|CREDENTIAL)[A-Za-z0-9_]*)\s*[:=]\s*['\"]?([^\s'\"]{12,})['\"]?\s*$" | ||
| ) | ||
|
|
||
| ENTROPY_THRESHOLD = 3.5 # bits/char; typical English prose sits well below this | ||
|
|
||
|
|
||
| def shannon_entropy(value: str) -> float: | ||
| if not value: | ||
| return 0.0 | ||
| counts: dict[str, int] = {} | ||
| for ch in value: | ||
| counts[ch] = counts.get(ch, 0) + 1 | ||
| length = len(value) | ||
| return -sum((n / length) * math.log2(n / length) for n in counts.values()) | ||
|
|
||
|
|
||
| def looks_like_placeholder(value: str) -> bool: | ||
| lowered = value.lower() | ||
| return any(word in lowered for word in ("example", "your_", "changeme", "placeholder", "xxxx", "<", "{{")) | ||
|
|
||
|
|
||
| def scan_text(text: str, source: str) -> list[dict]: | ||
| findings: list[dict] = [] | ||
| for line_number, line in enumerate(text.splitlines(), start=1): | ||
| for name, pattern in PATTERNS: | ||
| for match in pattern.finditer(line): | ||
| if looks_like_placeholder(match.group(0)): | ||
| continue | ||
| findings.append({ | ||
| "source": source, | ||
| "line": line_number, | ||
| "kind": name, | ||
| "confidence": "high", | ||
| "excerpt": _redact_middle(match.group(0)), | ||
| }) | ||
|
|
||
| for match in ASSIGNMENT.finditer(line): | ||
| key, value = match.group(1), match.group(2) | ||
| if looks_like_placeholder(value): | ||
| continue | ||
| entropy = shannon_entropy(value) | ||
| if entropy >= ENTROPY_THRESHOLD: | ||
| findings.append({ | ||
| "source": source, | ||
| "line": line_number, | ||
| "kind": f"High-entropy value assigned to {key}", | ||
| "confidence": "medium", | ||
| "excerpt": _redact_middle(f"{key}={value}"), | ||
| }) | ||
| return findings | ||
|
|
||
|
|
||
| def _redact_middle(value: str) -> str: | ||
| # Short values (a short password, a connection-string fragment) get no | ||
| # characters revealed at all, revealing even 2+2 chars of something | ||
| # this short exposes too large a fraction of the actual secret. | ||
| if len(value) <= 12: | ||
| return "…redacted…" | ||
| return value[:6] + "…redacted…" + value[-4:] | ||
|
|
||
|
|
||
| def iter_files(path: str): | ||
| if os.path.isfile(path): | ||
| yield path | ||
| return | ||
| for root, _dirs, files in os.walk(path): | ||
| if os.sep + ".git" in root + os.sep: | ||
| continue | ||
| for name in files: | ||
| yield os.path.join(root, name) | ||
|
|
||
|
|
||
| def run(path: str) -> list[dict]: | ||
| findings: list[dict] = [] | ||
| for file_path in iter_files(path): | ||
| try: | ||
| with open(file_path, encoding="utf-8", errors="ignore") as handle: | ||
| text = handle.read() | ||
| except (OSError, UnicodeDecodeError): | ||
| continue | ||
| findings.extend(scan_text(text, file_path)) | ||
| return findings | ||
|
|
||
|
|
||
| def main() -> int: | ||
| parser = argparse.ArgumentParser(description=__doc__) | ||
| parser.add_argument("path", help="file, directory, or '-' for stdin") | ||
| parser.add_argument("--json", action="store_true") | ||
| args = parser.parse_args() | ||
|
|
||
| if args.path == "-": | ||
| findings = scan_text(sys.stdin.read(), "<stdin>") | ||
| elif os.path.exists(args.path): | ||
| findings = run(args.path) | ||
| else: | ||
| raise SystemExit(f"No such file or directory: {args.path}") | ||
|
|
||
| if args.json: | ||
| print(json.dumps(findings, indent=2)) | ||
| return 0 | ||
|
|
||
| if not findings: | ||
| print("No known secret patterns or high-entropy assignments found.") | ||
| return 0 | ||
|
|
||
| print(f"{len(findings)} possible secret(s) found:\n") | ||
| for item in findings: | ||
| print(f"[{item['confidence']}] {item['kind']}") | ||
| print(f" {item['source']}:{item['line']} {item['excerpt']}\n") | ||
| return 0 | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| sys.exit(main()) | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The redaction still leaks part of real secrets. For longer values it prints the first 6 and last 4 characters. For a skill whose job is preventing leakage, echoing 10 characters of a live token into output that might get copied is problematic. The clean fix is the one Copilot suggested, replace it with a non-reversible fingerprint (length + short hash) so nothing from the real value is echoed.