Skip to content

docs: Phase D D8 agent failure modes -- catalog of 10 known agent failure patterns with detection signals and mitigations - #892

Merged
cschanhniem merged 1 commit into
mainfrom
feat/d8-agent-failure-modes
Jul 10, 2026
Merged

cschanhniem merged 1 commit into
mainfrom
feat/d8-agent-failure-modes

Conversation

@cschanhniem

Copy link
Copy Markdown
Collaborator

Implements D8 from NEXT_100_PR_MAP: docs/AGENT_FAILURE_MODES.md.

What

  • 10 documented failure modes (FM-01 through FM-10): claim escalation, scope creep, benchmark theater, safety weakening, certificate confusion, calibration self-service, hidden dependencies, novelty over-attribution, unsafe parallelism, stop-condition bypass
  • Each entry: what it is, why it happens, detection signal, mitigation
  • Cross-references check_claims.py, pr_claim_checker.py, cheap_enemy_comparison.py, similarity_neighbor_report.py, PROOF_LADDER.md
  • Docs-only PR; no code or test changes

Why

An agent that does not know it can fail is more dangerous than a human who knows they can be wrong. FM-04 (safety weakening by omission) and FM-10 (stop condition ignored) are highest-risk and have been observed in agent loops.

Disconfirming pass

  • All 10 failure modes are observable in principle, not hypothetical
  • No safety/release policy changed; no threshold or calibration policy touched

…lure patterns with detection signals and mitigations (#892)
@cschanhniem
cschanhniem merged commit eac6e50 into main Jul 10, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant