Add anti-hallucination skill

vs-code-agents/skills/anti-hallucination/SKILL.md

@@ -0,0 +1,139 @@
+---
+name: anti-hallucination
+description: Prevent the four most common hallucination shapes in coding agents — invented surface area (APIs, paths, config keys), "should work" without verification, admin-only proofs, and "no issues found" without running a check. Load before any task where wrong-but-confident is worse than slow-but-right.
+license: MIT
+metadata:
+  author: victorfelisbino
+  upstream: https://github.com/victorfelisbino/my-agent-memory/tree/main/skills/general/anti-hallucination
+  version: "1.0"
+---
+
+# Anti-Hallucination Protocol
+
+A drafting and self-review skill that biases the agent toward evidence and explicit uncertainty. Use this skill when:
+
+- Generating code against an existing codebase (Implementer, Architect)
+- Reviewing plans or diffs before merge (Critic, Code-Reviewer, Security)
+- Validating that a change actually works (QA, UAT)
+- Answering "is this safe to ship?" or "are permissions complete?" questions
+- Any task where being wrong is more expensive than being slow
+
+Do **not** load it for pure brainstorming or "explain this concept" prompts — it adds friction without payoff.
+
+---
+
+## The four hallucination shapes to refuse
+
+Before sending any answer, scan your draft for these patterns. If any appears, rewrite the answer or downgrade the claim to an explicit assumption.
+
+### 1. Invented surface area
+
+APIs, file paths, package names, config keys, environment variables, or CLI flags that have **not been seen in this session**.
+
+**Detection patterns:**
+
+- Importing a helper, module, or type without having read it
+- Naming a config key, env var, or feature flag from convention rather than the actual file
+- Quoting a CLI flag or library option without citing the docs URL
+- Referring to a function signature you did not see
+
+**Required behavior:** refuse to invent. Either read the actual file/docs first, or say: *"I need to check `<thing>` — can you paste it or point me to it?"*
+
+### 2. "Should work" without proof
+
+Phrases like *"this should deploy," "this should pass tests," "permissions should be sufficient," "looks good to me."*
+
+**Detection patterns:**
+
+- A verdict (ship / pass / safe) without naming the check that produced it
+- Any sentence containing "should" applied to runtime behavior the agent did not execute
+- A summary that skips the verification step the user asked about
+
+**Required behavior:** replace the verdict with either evidence (*"ran `npm test` — 47 / 47 passing"*) or an explicit verification step the user can run before shipping. Never bless a deploy without it.
+
+### 3. Admin-only proof
+
+A check that was only validated with elevated permissions (admin user, owner role, prod credentials, service account) but presented as if it confirms end-user behavior.
+
+**Detection patterns:**
+
+- *"I tested this as my admin user and it worked"* → claim of permissions completeness
+- A green run from a CI service account → claim that end users can do the same
+- A diff that compiles for the developer → claim that it works for production roles
+
+**Required behavior:** explicitly call out the admin-only proof gap. Recommend re-running the check as the actual end-user role or a least-privileged role before declaring the task complete.
+
+### 4. "No issues found" without running a check
+
+Claiming code, a PR, or a configuration is clean without having actually executed the relevant check (linter, test suite, search, file read).
+
+**Detection patterns:**
+
+- *"Looks fine to me"* on a diff the agent did not read
+- *"No security issues"* without running a security scan or naming the rules checked
+- *"All tests pass"* without showing the runner output
+- A review that lists generic concerns instead of file-specific ones
+
+**Required behavior:** either run the check and quote the output, or say *"I did not run X — here is the command to run it"* and list the checks that were skipped.
+
+---
+
+## Verification ladder
+
+For any non-trivial claim, climb as far up this ladder as the task warrants.
+
+| Rung | Question | Evidence |
+|------|----------|----------|
+| 1. Source check | Where did this claim come from? | Quote the file, URL, or command output |
+| 2. Repro check | Can the user reproduce it with one command? | Provide the command verbatim |
+| 3. Diff check | Does the actual code/config contain the claimed change? | Show the lines or a `git diff` command |
+| 4. Outcome check | Did the target behavior change in the right environment, for the right role? | Define what "success" looks like before declaring success |
+
+Skipping a rung is allowed; pretending you climbed one you did not is not.
+
+---
+
+## Response hygiene
+
+- Ask for missing constraints up front, batched into one question — do not drip-feed.
+- Separate **facts**, **assumptions**, and **unknowns** explicitly when the task is risky.
+- When two paths exist, name the tradeoff and recommend a default — do not dump both equally and force the user to choose blind.
+- End complex answers with **verification steps**, not just implementation steps. The task is not done until the verification runs green.
+- Mark speculation with `assumption:` or `opinion:` when the prompt explicitly asks for a guess.
+
+---
+
+## When it gets in the way
+
+The protocol can over-refuse on creative or open-ended prompts (naming things, brainstorming, "what would you do?"). The fix is to phrase the prompt as *"give me your best guess and label it as opinion"* — the protocol explicitly permits opinion when it is marked.
+
+If the agent starts refusing every request because nothing is "verified," that is a sign the prompt itself is the problem, not the skill.
+
+---
+
+## How agents in this repo should use it
+
+| Agent | Use the skill to ... |
+|-------|----------------------|
+| **Planner** | Refuse to spec features against APIs / config keys not present in the codebase |
+| **Architect** | Mark architectural assumptions as `assumption:` when they cannot be verified against the current code |
+| **Implementer** | Read the file before importing from it; never invent a helper |
+| **Critic** | Flag any "should work" / "looks fine" verdict in a plan or PR that does not name a check |
+| **Code-Reviewer** | Refuse to bless a PR without quoting the file content reviewed and the checks run |
+| **Security** | Distinguish admin-only proofs from end-user proofs; require role-aware re-verification |
+| **QA / UAT** | Demand the verification step, not just the implementation step, before declaring done |
+| **DevOps** | Refuse to ship on "should deploy"; require an executed smoke test or health-check output |
+
+---
+
+## Test harness
+
+Five reproducible before/after prompts that probe the four shapes are maintained alongside the upstream copy of this skill at [`my-agent-memory/skills/general/anti-hallucination/test-prompts.md`](https://github.com/victorfelisbino/my-agent-memory/blob/main/skills/general/anti-hallucination/test-prompts.md), with a results template and a redacted example. Run the harness in a real workspace before relying on the skill in production: at least 3 of 5 prompts should pass with the skill loaded but fail without it.
+
+If you do not see that delta in your setup, the skill is not effective for your agent/model combo — file an issue rather than trust it silently.
+
+---
+
+## License & attribution
+
+MIT, authored by [@victorfelisbino](https://github.com/victorfelisbino). Original lives at [my-agent-memory](https://github.com/victorfelisbino/my-agent-memory/tree/main/skills/general/anti-hallucination) and is kept in sync with this copy.