[codex] Add optimization intelligence reports#129
Conversation
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (35)
📝 WalkthroughWalkthroughThis PR adds optimization intelligence analytics across all interfaces of tokenleak. Three new report builders—routing simulation, agent waste detection, and agent behavior comparison—are integrated into the CLI, TUI, MCP, and renderers with unified data models, caching, and navigation support. ChangesOptimization Intelligence Feature
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Poem
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
| since: z.string().optional().describe('Start date in YYYY-MM-DD format'), | ||
| until: z.string().optional().describe('End date in YYYY-MM-DD format (default: today)'), | ||
| provider: z.string().optional().describe('Filter to a specific provider by name'), | ||
| strategy: z.string().optional().describe('Routing strategy name, default conservative'), |
There was a problem hiding this comment.
The MCP routing tool exposes strategy, including manual by description, but it has no rules input and handleGetRoutingSimulation always calls buildRoutingSimulationReport without rules. strategy: "manual" therefore returns an empty/no-op simulation for every request. Please add a routing-rule schema and pass it through, or reject/manual-hide that strategy so callers do not get a misleading successful report.
| import { handleGetAgentBehaviorDiff } from './get-agent-behavior-diff.js'; | ||
|
|
||
| const behaviorSelectorSchema = z.object({ | ||
| label: z.string(), |
There was a problem hiding this comment.
label is required here, but the planned MCP shape for get_agent_behavior_diff passes selectors like { dimension: "provider", provider: "claude-code" } without labels. That makes the documented/expected agent call fail schema validation before it reaches the handler. Either make label optional and derive a fallback from the selector, or update all public examples/contracts to require it.
| ensureRoutingSimulationReport(state); | ||
| }); | ||
| } | ||
| return createSimulatorPanel(state.cachedRoutingSimulationReport); |
There was a problem hiding this comment.
The new optimization views update simulatorScrollOffset / wasteScrollOffset / behaviorScrollOffset, but the offsets are not passed into these panels and the panels themselves hard-slice to the first 8 rows. As a result j/k changes state but cannot reveal candidates/signals beyond the first page, despite the status bar advertising scrolling. Please pass the offset into the panels and slice from it, or remove the scroll handling for these views until row navigation is implemented.
| case '--repo': | ||
| cliArgs['repo'] = next(); | ||
| break; | ||
| case '--strategy': |
There was a problem hiding this comment.
simulate-routing does not accept the --rule flag from the implementation plan (tokenleak simulate-routing --rule premium-short-output:claude-opus->claude-sonnet). I reproduced this locally and it fails at parse time with Unknown optimization flag "--rule", so users cannot exercise manual routing from the CLI even though the core builder supports rules. Please add parsing/validation for rule strings and pass them into buildRoutingSimulationReport, or remove the manual/rule surface from the advertised command until it is wired.
1 participant
Summary
Builds Tokenleak's Optimization Intelligence surface across core, CLI, MCP, terminal dashboards, OpenTUI, and live dashboard.
Validation
PATH="/Users/yansh/.bun/bin:$PATH" /Users/yansh/.bun/bin/bun run checkPATH="/Users/yansh/.bun/bin:$PATH" /Users/yansh/.bun/bin/bun run testNote: Turborepo prints a lockfile parse warning from this external worktree path, but all tasks pass.
Summary by CodeRabbit
New Features
Documentation