feat: QED Math Environment with MCP tools and LLM-judge rubric

[rycerzes]

Summary

Adds the QED Math Environment, a mathematical proof generation and evaluation env ported from QED-Nano. It deeply integrates both MCP tools (the sole agent API) and LLM-judge rubrics (structured 0–7 grading with normalized rewards), making it a reference implementation for the MCP + Rubrics features.

Supersedes #446 (closed). This PR is that branch rebased cleanly onto current main, plus a pass that aligns the per-rollout reward signal with QED-Nano's actual defaults (see Fidelity alignment below).

What's included

Environment (qed_math_env)

• QEDMathEnvironment — extends MCPEnvironment; manages problem lifecycle, dataset loading, and proof submission
• MathProofRubric — extends openenv.core.rubrics.base.Rubric; LLM-judge grading via OpenAI-compatible endpoints with <score>N</score> parsing, retry logic, and optional score thresholding
• QEDMathEnv client — extends MCPToolClient with typed ProblemObservation / ProofSubmissionObservation models
• 3 MCP tools: get_problem, submit_proof, get_grading_guidelines

Key features

• Flexible dataset loading: local JSONL/JSON, Hugging Face Hub, or built-in bootstrap problems
• Answer-mode verification: process-pool math_verify \boxed{} checking (no LLM call needed)
• Reward shaping: discount factor (γ^tokens), length penalty (buffer zone), optional score thresholding (collapses 1–5 → 1)
• Reasoning stripping: configurable delimiters (e.g. </think>) removed before grading
• Multi-step problems: configurable max attempts with per-attempt feedback
• Verifier metrics: QED-Nano-compatible metrics (verifier/rollouts/*, verifier/failures/*, latency, token counts) in observation metadata

Fidelity alignment with QED-Nano

This branch reconciles the port against the upstream CMU-AIRe/QED-Nano source (rollouts.py, conf/base.yaml, evaluator prompts):

• Evaluator prompts: ship v0/v1 alongside v2; default v1 (full 0–7 range, the QED-Nano default) instead of v2 (which constrains scores to {0,1,6,7}).
• Reasoning stripping now defaults to ["</think>"], so only the post-think answer is graded — matching upstream.
• Grader sampling params (temperature=1.0, optional max_output_tokens) are forwarded, and real provider token usage is captured (heuristic only as a fallback).
• Proof vs. answer routing is auto-detected from grading-schema presence (upstream's if "schema" in problem) when a row sets no explicit type.
• Answer-mode rewards are selectable via answer_reward_preset (pure_success default, base adds wrong/no_answer/unparsable penalties), keyed on verifier status.
• is_correct success threshold aligned to upstream score == 7.
• Stale refs fixed: source link meta-pytorch → CMU-AIRe; build refs → huggingface/OpenEnv.

Out of scope (by design): QED-Nano's Reasoning-Cache (RC) training loop — the iterative summarize→refine cycle, separate summarization model, and stream batching — is training orchestration, not an environment concern (and would conflict with the "controls only for orchestration" + dual-API invariants). The env already provides what RC needs: grading against original_problem and reasoning stripping.

Testing

tests/envs/test_qed_math_environment.py — 142 unit tests pass (5 server-integration tests marked @pytest.mark.integration):

Area	Coverage
MCP tools	ListToolsAction / CallToolAction for all 3 tools
Rubric grading	mocked grade(), score→reward normalization, retry/failure paths
Answer verification	correct/wrong/missing \boxed{} via math_verify service
Reward shaping	discount factor, length penalty, score thresholding
Reasoning stripping	</think> handling + fallback
Config defaults	v1 default, v0/v1 prompt loading, </think> default, grader sampling
Mode routing	proof/answer auto-detection from schema presence
Answer rewards	pure_success / base presets
Success threshold	score == 7 semantics
Token usage	real-usage capture with heuristic fallback
Dataset loading	local JSONL/JSON, problem-ID selection, \boxed{} wrapping
Multi-step / original_problem	attempt tracking + RC-stream grading target
Verifier metrics	GradingResult + submission payload

PYTHONPATH=src:envs uv run pytest tests/envs/test_qed_math_environment.py -v

Lint (ruff format / ruff check src tests), usort, and scripts/sync_env_docs.py --check all pass.

---

CC: @burtenshaw