[codex] unify trace queries and outcome recording

[divo12]

Summary

• add an immutable, benchmark-agnostic OutcomeEvaluation contract with typed task, verifier, trace, verdict, normalized score, UTC evaluation time, and bounded evidence
• store authoritative outcomes as idempotent categorical ofw.outcome scores on exact Langfuse traces
• rename openflywheel-trace-query to one unified openflywheel plugin
• preserve four read-only trace tools and add one scoped, non-destructive, idempotent record_outcome write tool
• add separate trace-query-planner and outcome-recorder skills so read and write workflows retain distinct behavioral boundaries

Unified plugin surface

Read-only tools:

• list_traces
• get_trace_schema
• query_spans
• get_span_context

Outcome write tool:

• record_outcome(trace_id, task_id, verifier_id, evaluated_at, verdict, evidence, score?)

Invalid outcome contracts fail before the Langfuse project or write client is opened. The tool accepts no credentials and returns the deterministic score_id receipt.

Langfuse representation

• score name: ofw.outcome
• value: pass | fail | abstain | error
• metadata: schema version, task ID, verifier ID, normalized score, and evidence references
• deterministic score ID plus verifier timestamp makes exact retries idempotent

Live verification

Stored through both the library and unified MCP function, then read back by ID:

• trace: c858913df08130acc3878db368dee053
• task: task-iam-12
• verifier result: 25/25 checks passed, reward 1.0
• Langfuse score ID: da648ceb-f2f6-54a1-80f9-2eb000f86616
• matching scores after repeated submissions: exactly 1

Verification

• Ruff passed
• strict mypy passed across source, tests, and unified MCP server
• 116 passed, 2 skipped; 96.11% project coverage
• opt-in live Langfuse write/read test passed
• both skills and the renamed plugin validate
• all new/changed production functions have cyclomatic complexity 5 or lower
• provider failures are mapped to a sanitized typed MCP error; contract-validation errors remain precise

Breaking rename

• plugin ID/path: openflywheel-trace-query → openflywheel
• optional dependency extra: trace-query → plugin

Non-goals

• failure mining or classification
• dataset promotion
• metric aggregation
• local outcome or trace persistence

[cubic-dev-ai]

All reported issues were addressed across 8 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 17 files (changes from recent commits).

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>