Skip to content

[codex] add failure diagnosis contract - #27

Merged
divo12 merged 2 commits into
freshfrom
codex/failure-mining-contract
Sep 2, 2026
Merged

divo12 merged 2 commits into
freshfrom
codex/failure-mining-contract

Conversation

@divo12

@divo12 divo12 commented Aug 28, 2026

Copy link
Copy Markdown
Owner

Summary

  • add an immutable FailureDiagnosis tied to an authoritative failed OutcomeEvaluation
  • define five stable failure types and supported/inconclusive evidence invariants
  • bound identifiers, diagnosis text, and cited Langfuse observations
  • export the provider-agnostic contract from the public package

Non-goals

  • no failure miner or judge
  • no Langfuse failure writer
  • no report generation, pattern clustering, or taxonomy registry

Local verification

  • uv sync --frozen --extra dev --extra plugin
  • uv run ruff check src tests plugins/openflywheel/scripts/mcp_server.py
  • uv run mypy src tests plugins/openflywheel/scripts/mcp_server.py
  • uv run pytest --cov=ofw --cov-report=term-missing --cov-fail-under=90 -q: 150 passed, 2 skipped, 95.30% coverage
  • radon: new contract maximum A (5), repository average A (2.36)
  • trace-query-planner and outcome-recorder skill validation
  • plugin validation

All verification listed above is local evidence, not CI evidence.


Summary by cubic

Adds a provider-agnostic FailureDiagnosis contract for failure mining, tied to an authoritative failed OutcomeEvaluation, and exports it from the ofw package.

Contract

  • Defines five stable failure types and supported/inconclusive evidence invariants.
  • Bounds identifiers, diagnosis text, and cited Langfuse observations.
  • Raises a typed FailureDiagnosisError with a stable error code on invalid input.
  • Leaves failure mining, judging, and report generation out of scope.

Written for commit ad68880. Summary will update on new commits.

Review in cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 5 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread src/ofw/evaluation/failure.py
Comment thread src/ofw/evaluation/failure.py Outdated
Comment thread tests/test_failure_diagnosis.py
Comment thread src/ofw/evaluation/failure.py
@divo12
divo12 merged commit c1bedb6 into fresh Sep 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant