Add thirdeye-review skill, rename use-thirdeye-evals → thirdeye-evals

[duncankmckinnon]

Summary

• Add a new bundled skill, thirdeye-review, for auditing other agents' traces and turning findings into concrete invocation, skill, or convention changes. Five-step workflow: scope cohort → identify tasks → audit tool/execution patterns → quantify token cost & errors → tie back to invocation surface and recommend.
• Rename use-thirdeye-evals → thirdeye-evals so the eval skill follows the same thirdeye-* naming as thirdeye-filter and thirdeye-review. (use-thirdeye stays as-is — it's the consumer-style "use the CLI" skill.)
• Update README.md and pyproject.toml package-data accordingly so the new files ship in the wheel.

What's in thirdeye-review

SKILL.md plus five focused references:

• population-scoping.md — agent-type / cwd / time-window cohort selection, sizing guidance.
• task-identification.md — labelling sessions from the first user turn, baselines per task shape.
• pattern-audit.md — tool-mix counts, waste signatures with thresholds (re-reads, grep_via_bash, unbounded Reads, retry loops, MCP dumps), execution-order patterns.
• token-and-error-analysis.md — spike isolation, tool-error clustering, recovery-shape reading.
• invocation-tieback-and-recommendations.md — pattern→surface map (prompt, skill description, conventions, allowlist, retry flags) with worked recommendation examples.

Anti-patterns spelled out in the top-level skill: no reviewing a single session, no counting without comparing, no "be more careful" recommendations, no ignoring the invocation, no skipping the tagging step.

Test plan

• python -m pytest tests/test_skill_install.py -q → 14 passed
• python -m thirdeye skill list → shows thirdeye-evals, thirdeye-filter, thirdeye-review, use-thirdeye
• On a checkout of this branch: pip install -e . then thirdeye skill install --only thirdeye-review /tmp/skills/ lands a symlink to the new skill.
• Spot-check thirdeye skill install --only thirdeye-evals still works under the renamed name.

🤖 Generated with Claude Code