thirdeye agent command

[duncankmckinnon]

Context

thirdeye (PyPI: thrdi) is a Python CLI that hooks into agentic coding tools (Claude Code,
Codex, Gemini CLI, Cursor) and records their events into a per-session event store at
~/.thirdeye/traces/<platform>/<session_id>/.

This plan adds a new thirdeye agent "<task>" command — a one-shot agent harness that
injects thirdeye's shipped skills as context, then invokes a coding agent (Claude Code by
default) to work through the task. The primary use case is session introspection: reviewing
today's sessions, auditing token usage, or identifying inefficiencies. With --fix, the
agent's tool access is broadened to allow file edits so it can apply recommendations.

The command reuses the existing eval/agents/ adapter infrastructure (AgentAdapter,
ClaudeAdapter, CodexAdapter, GeminiAdapter, get_adapter()) without modifying it.
All new code lives in a new src/thirdeye/agent/ package.

After the agent run completes, any new sessions the agent spawned on its platform are
automatically tagged thirdeye-agent (using TagStore.add(0, "thirdeye-agent", source="auto")).
This lets users filter agent-initiated sessions with thirdeye list --tag thirdeye-agent and
exclude them from general session analysis. Tagging is best-effort — failures are swallowed
so they never block the command.

The four deliverables, spread across three waves:

src/thirdeye/agent/
  __init__.py       # empty package marker
  prompt.py         # skill loading + prompt composition
  harness.py        # wraps AgentAdapter with mode-specific arg overrides
  exec.py           # streaming subprocess execution
src/thirdeye/commands/agent.py   # click CLI command
src/thirdeye/cli.py              # one-line addition to wire in the command

Key existing modules to understand

src/thirdeye/eval/agents/base.py — AgentAdapter ABC, AgentConfig dataclass,
ConfigAdapter concrete implementation. AgentAdapter.build_command(prompt, cwd) returns
[command, *resolved_args] with {prompt} substituted.

src/thirdeye/eval/agents/__init__.py — get_adapter(name, thirdeye_home) returns
an adapter instance; list_agent_names(thirdeye_home) returns sorted list of available
names (builtins + user YAML overrides).

src/thirdeye/eval/agents/claude.py — ClaudeAdapter uses
claude -p {prompt} --output-format json --allowedTools "Bash(thirdeye *) ...". The
harness overrides these args for agent mode (text output, mode-appropriate tools).

src/thirdeye/web/agentic.py — reference for the importlib.resources pattern for
loading shipped skills and stripping YAML frontmatter. Study _read_skill_body() and
_FRONTMATTER_RE.

src/thirdeye/config.py — Config.load() reads THIRDEYE_HOME env var or defaults
to ~/.thirdeye.

Shipped skills (all under src/thirdeye/skills/<name>/SKILL.md):

• use-thirdeye — how to use the thirdeye CLI to retrieve and inspect sessions
• thirdeye-review — five-step workflow for auditing agent sessions for inefficiencies
• thirdeye-evals — how to run and interpret session evaluations
• thirdeye-filter — NL-to-filter skill (used by the web UI; not typically needed here)

Tasks

• Prompt builder
• Agent harness
• Streaming exec
• Agent command

🤖 Reviewed by workbench. Report: .workbench/agent-command/wrap-up/agent-command/review.md