diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index 18905bb..a22e584 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -1,8 +1,14 @@ { "name": "devkit", - "version": "2.0.39", + "version": "2.1.0", "description": "A deterministic development harness for AI agents — YAML workflow engine, self-learning hooks, and multi-agent consensus", "author": { "name": "5uck1ess" + }, + "mcpServers": { + "devkit-engine": { + "command": "${CLAUDE_PLUGIN_ROOT}/bin/devkit", + "args": ["mcp"] + } } } diff --git a/.gitignore b/.gitignore index b7ecd13..d63e9c6 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,6 @@ +# Compiled binaries +bin/ + # Credentials — never commit *.token *.secret @@ -23,3 +26,4 @@ dist/ # Temp *.log *.tmp +.worktrees diff --git a/CHANGELOG.md b/CHANGELOG.md index b4629ad..c490ff0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,35 @@ # Changelog +## 2.1.0 + +### MCP Engine — Deterministic Workflow Enforcement (PR #52) + +Replaces the broken subprocess-spawning engine with an MCP server that runs inside Claude Code. Step skipping is now structurally impossible. + +#### Added +- **MCP server** (`src/mcp/`) — Go server exposes 4 tools: `devkit_start`, `devkit_advance`, `devkit_status`, `devkit_list`. Registered via `mcpServers` in plugin.json. +- **PreToolUse guard hook** (`hooks/devkit-guard.sh`) — reads `session.json`, blocks Bash/Edit/Write/Read/Glob/Grep/Agent/WebFetch/WebSearch/NotebookEdit/Skill during command steps (exit 2). +- **Stop guard hook** (`hooks/devkit-stop-guard.sh`) — blocks session end if workflow is incomplete. +- **Condensed principles** (`skills/_principles.yml`) — ~120 tokens of DRY/YAGNI/clean-code/dont-reinvent/executing/scratchpad/stuck/test-gen rules injected per workflow step instead of loading full skill files. +- **Hot session state** (`src/lib/state_json.go`) — atomic write to `$CLAUDE_PLUGIN_DATA/session.json` for fast hook reads (<50ms). +- **Workflow YAML extensions** — `enforce` (hard/soft), `branch` (git branch per session), `principles` (per-workflow and per-step override). +- **New MCP tools** — 6 integration tests covering lifecycle, loops with gates, principle injection, expect-failure, path traversal rejection. + +#### Changed +- **Engine role** — CLI that spawned subprocesses → MCP server + state machine +- **Claude runner** — `claude -p` subprocess (broken with OAuth) → Claude Code IS the runner +- **Enforcement** — None (markdown honor system) → MCP tool scoping + PreToolUse exit 2 +- **Principle skills** — Loaded if Claude decided to → injected by engine per step +- **Token usage** — ~50k+ for 8-step workflow → ~17k (~65% reduction) +- **Skills and commands** — All 8 entry points (research, deep-research, autoloop, tri-review, tri-debug, tri-security, pr-ready, status) now use MCP tools instead of `ensure-engine.sh` + `devkit workflow run` + +#### Removed +- `scripts/ensure-engine.sh` — no longer needed (binary ships in `bin/`, auto-PATH) +- `scripts/install-engine.sh` — installed by plugin manifest + +#### Fixed +- Engine can now run inside Claude Code (was impossible with OAuth tokens and `claude -p` subprocess) + ## 2.0.34 ### Deterministic Workflow Conversion (PRs #38–#45) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 3ba3726..8b67248 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -2,24 +2,24 @@ ## Adding a Workflow -Most command logic lives in YAML workflows executed by the Go engine. Only 8 slash commands remain as tab-completable entry points. +Most command logic lives in YAML workflows executed by the Go engine. Only 6 slash commands remain as tab-completable entry points — everything else is context-activated via skills. 1. Create `workflows/my-workflow.yml` with steps, model assignments, and loop/gate definitions -2. Test with `devkit workflow run my-workflow "input"` +2. Test with `devkit_start` MCP tool or `devkit workflow run my-workflow "input"` from terminal 3. Optionally add a context-activated skill in `skills/` to auto-trigger it See `skills/creating-workflows/SKILL.md` for YAML schema reference. ### Adding a Slash Command (rare — only for top-level entry points) -Only add a command if it needs tab-completion. Most workflows are invoked via `devkit workflow run` or context-activated skills. +Only add a command if it needs tab-completion. Most workflows are invoked via MCP tools (`devkit_start`/`devkit_advance`) or context-activated skills. 1. Create `commands/my-command.md` with YAML frontmatter: ```markdown --- description: What this command does. --- - Run `devkit workflow run my-workflow` to execute. + Use `devkit_start` MCP tool with workflow name to execute. ``` The command name is derived from the filename: `commands/my-command.md` becomes `/devkit:my-command`. diff --git a/README.md b/README.md index d18ceeb..c3aa995 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Devkit -A deterministic development harness for AI agents. The Go engine controls orchestration (loops, branches, gates, budgets). The agent handles creativity. Every change is measured, gated, and auditable. +A deterministic development harness for AI agents. The MCP engine controls workflow execution (step ordering, gates, loops, branches). The agent handles creativity. Every step is enforced, measured, and auditable. Works with just Claude. Optionally adds Codex and Gemini for multi-agent consensus. @@ -68,7 +68,7 @@ These handle concerns devkit doesn't — methodology, specialized reviews, and c ```bash brew install rtk # Token optimization (60-90% savings on Bash output) -brew install ast-grep # AST-based repo mapping (devkit workflow run repo-map) +brew install ast-grep # AST-based repo mapping (used by onboard skill) ``` ### Verify @@ -91,31 +91,51 @@ This shows which CLIs are installed, which agents are available, and which comma # "research the best auth library for Node" # Slash commands for complex workflows: -/devkit:pr-ready # Full PR pipeline /tri:review # Multi-agent code review -devkit workflow run self-lint "npm run lint" # Fix all lint errors +# Or just describe: "submit a PR", "ship this" → pr-ready skill auto-activates ``` --- +## How It Works + +Devkit runs as an **MCP server** inside Claude Code. When a workflow starts, the engine takes control: + +``` +devkit_start("research", "best Go testing frameworks") + → Engine creates session, returns Step 1 + condensed principles + → Claude executes the step using standard tools + → Claude calls devkit_advance(session_id) + → Engine validates, records output, returns Step 2 + → ...repeat until WORKFLOW COMPLETE + +Enforcement (runs automatically): + PreToolUse hook → blocks out-of-step actions during command steps + Stop hook → prevents session end during active workflows +``` + +**Why MCP?** Claude can't skip steps because the engine controls what comes next. Claude can't call tools that aren't valid for the current step. The engine holds state — Claude doesn't self-report. + +--- + ## Commands -8 tab-completable slash commands. All other workflows are context-activated via skills or invoked directly with `devkit workflow run `. +6 tab-completable slash commands. All other workflows are context-activated via skills (auto-triggered by natural language) or invoked via MCP tools. | Command | What it does | |---|---| | `/tri:review` | Code review from 1-3 agents, consolidated report | | `/tri:debug` | Independent root-cause analysis from each agent | | `/tri:security` | Security audit with severity-ranked consensus | -| `/devkit:pr-ready` | Lint, test, security, changelog, create PR | -| `/devkit:pr-monitor` | Watch CI, fetch reviewer comments, fix iteratively, push | -| `/devkit:workflow` | Run user-defined YAML workflows | +| `/devkit:workflow` | Run any YAML workflow by name | | `/devkit:status` | Health check | | `/devkit:setup-rules` | Install language-specific coding rules to `~/.claude/rules/` | -### Workflows (via `devkit workflow run `) +Tasks like "ship this PR" or "submit a PR" auto-activate the `pr-ready` skill — no slash command needed. -All 18 YAML workflows can be invoked directly. Skills auto-activate for common triggers (e.g., "research X", "fix this bug", "add a feature"). +### Workflows + +All 18 YAML workflows are invoked via the MCP engine. Skills auto-activate for common triggers (e.g., "research X", "fix this bug", "add a feature"). | Workflow | What it does | |---|---| @@ -155,26 +175,28 @@ Skills activate automatically based on context. No slash command needed. | "scrape this URL" | `scrape` | | "create an ADR" | `adr` | -Coding principles (`clean-code`, `dry`, `yagni`, `dont-reinvent`, `executing`, `stuck`, `scratchpad`) load as reference when relevant. +Coding principles (`clean-code`, `dry`, `yagni`, `dont-reinvent`, `executing`, `stuck`, `scratchpad`) are injected as condensed rules (~120 tokens) per workflow step — not loaded as full skill files. --- ## Hooks -10 hooks across 4 lifecycle events. All installed automatically with the plugin. +12 hooks across 4 lifecycle events. All installed automatically with the plugin. | Event | Hook | What it catches | |---|---|---| | PreToolUse | **safety-check** | `rm -rf /`, `DROP TABLE`, force push, editing secrets | | PreToolUse | **security-patterns** | `eval()`, XSS, shell injection, weak hashes, hardcoded secrets | | PreToolUse | **audit-trail** | Logs every command to `.devkit/audit.log` | -| PreToolUse | **pr-gate** | Prompts to run `/devkit:pr-ready` before `gh pr create` | +| PreToolUse | **pr-gate** | Prompts to run the pr-ready skill before `gh pr create` | | PreToolUse | **rtk-rewrite** | Compresses Bash output via RTK (no-op if not installed) | +| PreToolUse | **devkit-guard** | Blocks out-of-step tools during workflow command steps | | PostToolUse | **post-validate** | Suppressed errors, leaked secrets, writes outside repo | | PostToolUse | **slop-detect** | AI code patterns — doc/code imbalance, restating comments | | PostToolUse | **lang-review** | Language-aware checks: Go, TypeScript, Rust, Python, Shell | | SubagentStop | **subagent-stop** | Verifies subagent work before accepting | | Stop | **stop-gate** | Merge conflicts, cross-domain test gaps, linter pass | +| Stop | **devkit-stop-guard** | Blocks session end during active workflows | --- @@ -211,78 +233,28 @@ Language-specific rules that auto-activate when Claude reads matching files. Ins --- -## Go CLI Harness - -The compiled Go binary handles deterministic orchestration — the machine controls the loop, the agent is the body. - -### Build - -```bash -cd src && make install -``` - -### What it does that plugins can't - -- **Exact iteration counts** — Go owns the loop, not the LLM -- **Command steps** — run shell commands directly in workflows, $0 cost -- **Loop gates** — shell command after each iteration, auto-revert on failure -- **YAML workflows** — branching, loops, parallel dispatch, budget enforcement -- **Triage-based skipping** — typo fix doesn't run a 14-step pipeline -- **Crash recovery** — SQLite state survives crashes -- **Hard budget caps** — stops at your dollar limit -- **True parallel dispatch** — goroutines, not sequential prompts - -### Examples - -```bash -# Run 50 improvement iterations, stop at $20 -devkit improve --metric "npm test" --iterations 50 --budget 20.00 - -# Implement a feature with test verification -devkit feature "add JWT auth" --target src/auth/ --test "npm test" - -# Multi-agent review -devkit review - -# Run any YAML workflow -devkit workflow feature "add JWT auth" - -# Check session history -devkit status -``` - -### Testing - -```bash -cd src && go test ./... -v -``` - -140+ tests across 6 packages. All use mock runners — no API calls needed. - ---- - ## Architecture ``` -Workflow Engine (Go binary) +MCP Server (bin/devkit mcp — auto-started by plugin) + ├── Tools: devkit_start, devkit_advance, devkit_status, devkit_list + ├── State: session.json (hot, <50ms reads) + SQLite (cold history) ├── Parse YAML → validate steps, branches, budget - ├── Create session + git branch ├── Walk steps: - │ ├── Command steps → shell execution (deterministic, $0) - │ ├── Prompt steps → LLM runner (Claude/Codex/Gemini) + │ ├── Command steps → engine executes shell directly ($0 cost) + │ ├── Prompt steps → Claude works, calls devkit_advance when done │ ├── Loop with gate → run, verify, keep or revert │ ├── Branch → case-insensitive substring match → goto - │ ├── Parallel → goroutines with mutex - │ └── Budget check every step - └── Commit, report, clean up + │ └── Parallel → Agent tool dispatch (Claude/Codex/Gemini) + └── Principles injected per step (~120 tokens, not full skill files) -Multi-Agent (tri-* workflows) - ├── Claude → native background agent (always) - ├── Codex → plugin or CLI (optional) - └── Gemini → plugin or CLI (optional) +Enforcement: + ├── MCP tool scoping — Claude can only call devkit_advance to progress + ├── PreToolUse hook — exit 2 blocks tools during command steps + └── Stop hook — blocks session end during active workflows -Self-Improvement (self-* workflows) - └── Loop: propose → measure → keep/revert → repeat +Terminal fallback (devkit workflow run ): + └── Subprocess runners for Codex/Gemini CLI usage ``` --- @@ -291,18 +263,18 @@ Self-Improvement (self-* workflows) ``` devkit/ -├── commands/ # 8 slash commands (tab-completable entry points) -├── skills/ # 19 context-activated skills +├── commands/ # 6 slash commands (tab-completable entry points) +├── skills/ # 20 context-activated skills + _principles.yml ├── agents/ # 6 agents (reviewer, researcher, improver, ...) -├── hooks/ # 10 hooks (safety, security, quality gates) +├── hooks/ # 12 hooks (safety, security, quality gates, workflow enforcement) ├── workflows/ # 18 YAML workflow definitions ├── resources/rules/ # Language-specific coding rules -├── presets/ # Reserved for future use -├── src/ # Go CLI harness +├── src/ # Go engine + MCP server +│ ├── mcp/ # MCP server (tools, principles loader, session management) │ ├── engine/ # YAML workflow engine (parser, executor, tests) -│ ├── runners/ # Claude, Codex, Gemini interfaces -│ ├── loops/ # Improve, feature, bugfix, refactor, testgen -│ ├── lib/ # DB, git, metrics, reporting -│ └── cmd/ # CLI entry points -└── .github/workflows/ # CI (build+test+vet) + auto-release +│ ├── runners/ # Codex, Gemini interfaces (terminal fallback) +│ ├── lib/ # DB, git, metrics, session state, reporting +│ └── cmd/ # CLI entry points (including `devkit mcp`) +├── bin/ # Auto-PATH binary (built by make install-plugin) +└── .github/workflows/ # CI (build+test+vet) + auto-release (6 platforms) ``` diff --git a/ROADMAP.md b/ROADMAP.md index 84f1b07..0871b88 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -2,11 +2,12 @@ ## Implemented -- **8 slash commands** — Tab-completable entry points (tri-review, tri-debug, tri-security, pr-ready, pr-monitor, status, setup-rules, workflow); 16 former commands now context-activated via skills or invoked directly via `devkit workflow run` +- **MCP engine** — Go server exposes `devkit_start`, `devkit_advance`, `devkit_status`, `devkit_list` tools inside Claude Code. Step ordering enforced via MCP tool scoping + PreToolUse hook exit 2. Session state in session.json (hot path, <50ms hook reads) + SQLite (cold history). ~65% token reduction vs old monolithic prompts. +- **6 slash commands** — Tab-completable entry points for things that need explicit invocation (tri-review, tri-debug, tri-security, status, setup-rules, workflow); 18 former commands now context-activated via skills or invoked via MCP tools (pr-monitor folded into pr-ready workflow; pr-ready is now a natural-language skill) - **Deterministic workflow conversion** — All command logic moved from LLM-interpreted markdown to Go-engine-driven YAML workflows; ~3,600 lines of inline logic removed -- **19 context-activated skills** — 9 auto-trigger workflows (test-gen, doc-gen, changelog, onboard, research, deep-research, scrape, autoloop, adr) + 6 coding principles (executing, clean-code, DRY, YAGNI, dont-reinvent, stuck) + 2 tools (gcli, creating-workflows) + 1 iteration memory (scratchpad) + 1 orchestration (mega-pr) +- **20 context-activated skills** — 10 auto-trigger workflows (test-gen, doc-gen, changelog, onboard, research, deep-research, scrape, autoloop, adr, pr-ready) + 6 coding principles (executing, clean-code, DRY, YAGNI, dont-reinvent, stuck) + 2 tools (gcli, creating-workflows) + 1 iteration memory (scratchpad) + 1 orchestration (mega-pr) - **6 agents** — Scoped tool access, worktree isolation, model assignment -- **10 hooks** — Safety (destructive command blocking, edit-time security patterns, PR gate), observability (audit trail, slop detection, post-validation, subagent verification, language-aware code review), optimization (RTK token compression) +- **12 hooks** — Safety (destructive command blocking, edit-time security patterns, PR gate), observability (audit trail, slop detection, post-validation, subagent verification, language-aware code review), optimization (RTK token compression), workflow enforcement (devkit-guard, devkit-stop-guard) - **Graceful degradation** — tri:* commands work with 1-3 agents depending on installed CLIs - **Goal decomposition** — Task DAG with dependency ordering and parallel execution - **Concurrency limits** — Max 3 parallel agents in multi-agent commands diff --git a/commands/pr-monitor.md b/commands/pr-monitor.md deleted file mode 100644 index 7eca809..0000000 --- a/commands/pr-monitor.md +++ /dev/null @@ -1,187 +0,0 @@ ---- -description: Post-PR review monitor — watches CI, fetches reviewer comments, iteratively resolves them, and pushes fixes. ---- - -# PR Monitor - -After a PR is created, this command watches for CI results and reviewer comments, then iteratively resolves them. Picks up where `/devkit:pr-ready` leaves off. - -## Parameters - -1. **PR number or URL** — the PR to monitor (required, or auto-detect from current branch) -2. **Max iterations** — max comment-resolution cycles (default: 10) -3. **Budget** — max USD (default: $5) - -## Budget & Early Exit - -- **Token budget:** ~500k tokens. Comment resolution can be expensive with large diffs. -- **Early exit:** Stop when all checks pass and no unresolved comments remain. -- **Stuck detection:** If 3 consecutive iterations resolve zero comments, stop and report. - -## Step 1: Identify PR - -```bash -# Auto-detect from current branch if no PR specified -PR_NUM=${1:-$(gh pr view --json number -q '.number' 2>/dev/null)} -if [ -z "$PR_NUM" ]; then - echo "ERROR: No PR found for current branch. Specify a PR number." - exit 1 -fi - -echo "Monitoring PR #${PR_NUM}" -gh pr view "$PR_NUM" --json title,state,statusCheckRollup,reviewDecision -``` - -## Step 2: Wait for Initial CI + Auto-Reviewers - -Wait up to 3 minutes for CI checks and auto-reviewers (Copilot, Gemini, CodeRabbit) to post: - -```bash -echo "Waiting for CI and auto-reviewers..." -for i in $(seq 1 18); do - sleep 10 - STATUS=$(gh pr checks "$PR_NUM" --json name,state 2>/dev/null) - PENDING=$(echo "$STATUS" | jq '[.[] | select(.state == "PENDING" or .state == "QUEUED")] | length') - if [ "$PENDING" = "0" ]; then - echo "All checks completed." - break - fi - echo " ... $PENDING checks still pending (${i}/18)" -done -``` - -## Step 3: Resolution Loop - -For each iteration: - -### 3a. Fetch CI Status - -```bash -gh pr checks "$PR_NUM" --json name,state,conclusion -``` - -If any check failed, read the failure logs and attempt to fix: -```bash -gh run view {run_id} --log-failed 2>/dev/null | tail -50 -``` - -### 3b. Fetch Unresolved Comments - -```bash -# Get all review comments and review threads -gh api repos/{owner}/{repo}/pulls/{PR_NUM}/comments --paginate -gh api repos/{owner}/{repo}/pulls/{PR_NUM}/reviews --paginate -``` - -### 3c. Classify Each Comment - -For each unresolved comment, classify it as one of: - -| Type | Action | -|------|--------| -| **Code fix** | Read the referenced file, apply the fix, commit | -| **Style/nit** | Apply if trivial, skip with reply if subjective | -| **Question** | Reply with context from the codebase | -| **False positive** | Reply explaining why the current code is correct | -| **Out of scope** | Reply acknowledging, note for future work | - -Use the `reviewer` agent to classify and draft responses: - -``` -Task: Classify and resolve this PR review comment. -Agent: reviewer -Context: - - Comment: {comment_body} - - File: {file_path}:{line} - - Current code: (read the file) - - Full PR diff context: git diff main...HEAD -- {file_path} - -Classify as: code_fix | style_nit | question | false_positive | out_of_scope -If code_fix or style_nit: propose the exact change. -If question or false_positive: draft a reply. -``` - -### 3d. Apply Fixes and Reply - -For code fixes: -```bash -# Apply the fix (Edit tool) -# Stage and commit -git add {files} -git commit -m "address review: {summary}" -``` - -For replies: -```bash -gh api repos/{owner}/{repo}/pulls/{PR_NUM}/comments/{comment_id}/replies \ - -f body="{reply}" -``` - -### 3e. Push and Request Re-Review - -```bash -git push -# Request re-review from original reviewers if code was changed -gh pr edit "$PR_NUM" --add-reviewer {reviewers} -``` - -### 3f. Check Completion - -```bash -REMAINING=$(gh api repos/{owner}/{repo}/pulls/{PR_NUM}/comments --paginate | \ - jq '[.[] | select(.resolved == false or .resolved == null)] | length') - -CHECKS_OK=$(gh pr checks "$PR_NUM" --json conclusion | \ - jq '[.[] | select(.conclusion != "SUCCESS" and .conclusion != "NEUTRAL" and .conclusion != "SKIPPED")] | length') - -if [ "$REMAINING" = "0" ] && [ "$CHECKS_OK" = "0" ]; then - echo "All comments resolved and checks passing." - break -fi -``` - -## Step 4: Report - -``` -## PR Monitor Report - -**PR:** #{pr_num} — {title} -**Iterations:** {completed} / {max} -**Status:** {all_resolved | partial | stuck} - -### CI Checks -| Check | Status | Notes | -|-------|--------|-------| -| build | ✓ pass | | -| test | ✓ pass | Fixed in iteration 2 | -| lint | ✓ pass | | - -### Comments Resolved -| # | Type | File | Action | Iteration | -|---|------|------|--------|-----------| -| 1 | code_fix | src/auth.ts:42 | Fixed null check | 1 | -| 2 | question | src/config.ts:15 | Replied with context | 1 | -| 3 | false_positive | src/util.ts:88 | Explained rationale | 1 | -| 4 | style_nit | src/auth.ts:50 | Applied formatting | 2 | - -### Unresolved (if any) -{list with reason why each couldn't be resolved} - -### Commits Pushed -| Commit | Summary | -|--------|---------| -| abc1234 | address review: add null check in auth handler | -| def5678 | address review: formatting fixes | -``` - -## Rules - -- Never force-push — always regular push -- Never resolve threads you didn't actually address -- Never dismiss reviews — only request re-review after fixing -- Classify before acting — don't blindly apply every suggestion -- Reply to false positives with evidence, not dismissal -- If a comment requires architectural changes, classify as out_of_scope and flag to user -- Stop after max iterations even if comments remain -- Use `AskUserQuestion` if a comment is ambiguous and classification is uncertain -- The reviewer agent runs in worktree isolation for classification diff --git a/commands/pr-ready.md b/commands/pr-ready.md deleted file mode 100644 index 1fd7125..0000000 --- a/commands/pr-ready.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -description: Full PR preparation pipeline — validate branch, DRY review, lint, test, security, changelog, create PR. ---- - -Ensure the devkit engine is installed, then run the workflow: - -```bash -ENSURE="$(find ~/.claude/plugins ${APPDATA:+$APPDATA/.claude/plugins} ${LOCALAPPDATA:+$LOCALAPPDATA/.claude/plugins} -path '*/devkit/scripts/ensure-engine.sh' 2>/dev/null | head -1)"; [ -n "$ENSURE" ] && bash "$ENSURE" || { echo "devkit plugin not found — install from https://github.com/5uck1ess/devkit/releases"; exit 1; } -``` - -```bash -devkit workflow run pr-ready -``` - -If the engine cannot be installed (no network, no write access), tell the user: "The devkit engine binary is required for deterministic workflow execution. Install manually from https://github.com/5uck1ess/devkit/releases" Do NOT fall back to manual steps. diff --git a/commands/status.md b/commands/status.md index 829cc04..f08e86b 100644 --- a/commands/status.md +++ b/commands/status.md @@ -18,16 +18,7 @@ echo -n "gemini plugin: " && (/gemini:status >/dev/null 2>&1 && echo "installed" ### Devkit Engine -```bash -echo "=== Devkit Engine ===" -if command -v devkit >/dev/null 2>&1; then - echo "engine: ✓ installed ($(devkit --version 2>/dev/null || echo 'unknown version'))" -else - echo "engine: ✗ NOT INSTALLED — workflows will not run deterministically" - echo " install: bash \"\$(find ~/.claude/plugins -path '*/devkit/scripts/install-engine.sh' 2>/dev/null | head -1)\"" - echo " or download from: https://github.com/5uck1ess/devkit/releases" -fi -``` +Use the `devkit_status` tool to check workflow progress. ### External CLIs @@ -73,7 +64,7 @@ List all commands, marking which ones need external CLIs: |-----|--------|------------| | codex | ✓ installed (v1.2.0) | tri:* commands (fallback) | | gemini | ✗ not installed | tri:* commands (fallback) | -| gh | ✓ installed (v2.40.0) | devkit:pr-ready | +| gh | ✓ installed (v2.40.0) | pr-ready skill | | rtk | ✓ installed (v0.34.2) | token optimization (optional) | ### Commands @@ -82,8 +73,6 @@ List all commands, marking which ones need external CLIs: | /tri:review | ⚠ partial | 2/3 agents available (no gemini) | | /tri:debug | ⚠ partial | 2/3 agents available | | /tri:security | ⚠ partial | 2/3 agents available | -| /devkit:pr-ready | ✓ ready | Uses gh for PR creation | -| /devkit:pr-monitor | ✓ ready | Uses gh for PR monitoring | | /devkit:workflow | ✓ ready | Runs YAML workflows via Go engine | | /devkit:status | ✓ ready | This command | | /devkit:setup-rules | ✓ ready | One-time setup | diff --git a/commands/tri-debug.md b/commands/tri-debug.md index 51ee5fb..271a5c5 100644 --- a/commands/tri-debug.md +++ b/commands/tri-debug.md @@ -2,14 +2,10 @@ description: Triple-agent debugging — independent root-cause hypotheses from Claude, Codex, and Gemini, then consensus fix. --- -Ensure the devkit engine is installed, then run the workflow: +## Invoke -```bash -ENSURE="$(find ~/.claude/plugins ${APPDATA:+$APPDATA/.claude/plugins} ${LOCALAPPDATA:+$LOCALAPPDATA/.claude/plugins} -path '*/devkit/scripts/ensure-engine.sh' 2>/dev/null | head -1)"; [ -n "$ENSURE" ] && bash "$ENSURE" || { echo "devkit plugin not found — install from https://github.com/5uck1ess/devkit/releases"; exit 1; } -``` +Start the workflow via the devkit engine: -```bash -devkit workflow run tri-debug -``` +Use the `devkit_start` tool with workflow: "tri-debug" and input: "{input}". -If the engine cannot be installed (no network, no write access), tell the user: "The devkit engine binary is required for deterministic workflow execution. Install manually from https://github.com/5uck1ess/devkit/releases" Do NOT fall back to manual steps. +Then follow each step the engine returns. Call `devkit_advance` after completing each step. The engine controls step order, gates, and loops. Do NOT skip steps. diff --git a/commands/tri-review.md b/commands/tri-review.md index 0ffc5f5..1435a2c 100644 --- a/commands/tri-review.md +++ b/commands/tri-review.md @@ -2,14 +2,10 @@ description: Triple-agent code review — dispatches to Claude, Codex, and Gemini in parallel, consolidates findings. --- -Ensure the devkit engine is installed, then run the workflow: +## Invoke -```bash -ENSURE="$(find ~/.claude/plugins ${APPDATA:+$APPDATA/.claude/plugins} ${LOCALAPPDATA:+$LOCALAPPDATA/.claude/plugins} -path '*/devkit/scripts/ensure-engine.sh' 2>/dev/null | head -1)"; [ -n "$ENSURE" ] && bash "$ENSURE" || { echo "devkit plugin not found — install from https://github.com/5uck1ess/devkit/releases"; exit 1; } -``` +Start the workflow via the devkit engine: -```bash -devkit workflow run tri-review -``` +Use the `devkit_start` tool with workflow: "tri-review" and input: "{input}". -If the engine cannot be installed (no network, no write access), tell the user: "The devkit engine binary is required for deterministic workflow execution. Install manually from https://github.com/5uck1ess/devkit/releases" Do NOT fall back to manual steps. +Then follow each step the engine returns. Call `devkit_advance` after completing each step. The engine controls step order, gates, and loops. Do NOT skip steps. diff --git a/commands/tri-security.md b/commands/tri-security.md index 83cdb92..1dfc36b 100644 --- a/commands/tri-security.md +++ b/commands/tri-security.md @@ -2,14 +2,10 @@ description: Triple-agent security audit — independent security reviews from Claude, Codex, and Gemini, consolidated with severity ranking. --- -Ensure the devkit engine is installed, then run the workflow: +## Invoke -```bash -ENSURE="$(find ~/.claude/plugins ${APPDATA:+$APPDATA/.claude/plugins} ${LOCALAPPDATA:+$LOCALAPPDATA/.claude/plugins} -path '*/devkit/scripts/ensure-engine.sh' 2>/dev/null | head -1)"; [ -n "$ENSURE" ] && bash "$ENSURE" || { echo "devkit plugin not found — install from https://github.com/5uck1ess/devkit/releases"; exit 1; } -``` +Start the workflow via the devkit engine: -```bash -devkit workflow run tri-security -``` +Use the `devkit_start` tool with workflow: "tri-security" and input: "{input}". -If the engine cannot be installed (no network, no write access), tell the user: "The devkit engine binary is required for deterministic workflow execution. Install manually from https://github.com/5uck1ess/devkit/releases" Do NOT fall back to manual steps. +Then follow each step the engine returns. Call `devkit_advance` after completing each step. The engine controls step order, gates, and loops. Do NOT skip steps. diff --git a/commands/workflow.md b/commands/workflow.md index 607d8ae..ba884d7 100644 --- a/commands/workflow.md +++ b/commands/workflow.md @@ -1,91 +1,65 @@ --- -description: Run a user-defined YAML workflow. Multi-step pipelines with loops, approval gates, and branching. +description: Run a YAML workflow by name via the devkit engine (feature, bugfix, refactor, self-*, audit, etc.). --- # Workflow Runner -Execute a YAML-defined workflow from the `workflows/` directory. +Generic entry point for any YAML workflow in `workflows/`. The devkit engine controls step order, loops, gates, and branches. -## Usage +## Invoke -``` -/devkit:workflow {name} -/devkit:workflow list -``` - -## Step 1: Load Workflow - -Read `workflows/{name}.yml` and parse: +Use the `devkit_list` tool first to see available workflows, or pick one by name: -```yaml -description: What this workflow does - -steps: - - id: step-1 - prompt: "Do the first thing" - agent: improver # optional — which agent to use - approval: false # optional — pause for user approval before executing - - - id: step-2 - prompt: "Do the second thing based on: {{step-1}}" - loop: - max: 5 # max iterations - until: "all passing" # stop when output contains this - - - id: step-3 - prompt: "Finalize" - branch: # conditional jump - - when: "error" - goto: step-1 - - when: "success" - goto: done +``` +devkit_start(workflow: "", input: "") ``` -## Step 2: Execute Steps - -For each step, sequentially: - -1. **Resolve placeholders** — replace `{{step-id}}` with that step's output, `{{input}}` with user's original input -2. **Approval gate** — if `approval: true`, show the step prompt and ask user to confirm before running -3. **Execute** — run the prompt (optionally via the specified agent) -4. **Loop** — if `loop` defined, repeat until output contains `until` string or `max` reached -5. **Branch** — if `branch` defined, check output for `when` string and jump to `goto` step -6. **Store result** — save output for placeholder resolution in later steps - -## Step 3: Report +Then call `devkit_advance(session: "")` after completing each step the engine returns. The engine controls step order, gates, and loops. Do NOT skip steps. -``` -## Workflow: {name} +## Available Workflows -### Steps Completed -| Step | Status | Iterations | -|------|--------|------------| -| step-1 | ✓ completed | 1 | -| step-2 | ✓ completed | 3 (loop) | -| step-3 | ✓ completed | 1 | +| Workflow | Purpose | +|---|---| +| `feature` | Brainstorm, plan, implement, test, lint, review | +| `bugfix` | Reproduce, diagnose, fix, regression test, verify | +| `refactor` | Analyze smells, plan, restructure, verify nothing broke | +| `research` | Clarify, decompose, parallel search, synthesize | +| `deep-research` | ACH hypotheses, disconfirmation, evidence matrix | +| `self-test` | Run tests, fix failures, loop until passing | +| `self-lint` | Run linter, fix violations, loop until clean | +| `self-perf` | Benchmark, optimize, loop until target met | +| `self-improve` | Run metric, fix issues, loop until passing | +| `self-migrate` | Migrate code incrementally with test gate | +| `self-audit` | Measure codebase, rank improvements by evidence | +| `autoloop` | Autonomous audit/fix/measure/keep-or-revert loop | +| `audit` | Dependencies, vulnerabilities, licenses, lint, security | +| `pr-ready` | Full PR preparation pipeline | +| `tri-review` | Multi-agent code review | +| `tri-debug` | Multi-agent debugging | +| `tri-security` | Multi-agent security audit | +| `tri-dispatch` | Generic parallel dispatch to multiple agents | + +## Examples -### Output -{final_step_output} ``` +# List workflows +devkit_list() -## Listing Workflows - -When called with `list`, scan `workflows/` and display: +# Run a feature workflow +devkit_start(workflow: "feature", input: "add JWT auth to src/auth/") -``` -## Available Workflows +# Run self-test with the project test command +devkit_start(workflow: "self-test", input: "npm test") -| Name | Description | -|------|-------------| -| my-workflow | Does something useful | +# Autonomous improvement loop +devkit_start(workflow: "autoloop", input: "improve test coverage | metric: go test -cover ./... | direction: higher-is-better | iterations: 10") ``` ## Rules -- Steps execute sequentially — no parallel steps -- Placeholder `{{step-id}}` resolves to that step's output -- Loop aborts if step executed more than `loop.max` times -- Branch checks are case-insensitive substring matches -- If no agent specified, execute the prompt directly (main Claude context) -- If a step fails, stop the workflow and report which step failed -- Approval gates require explicit user confirmation +- The engine enforces step order — you cannot skip steps +- Command steps execute automatically when you call `devkit_advance` +- Loop steps repeat based on gate/until/max conditions +- Branch steps jump based on output content matching +- Parallel steps dispatch to subagents via the Agent tool + plugins +- Call `devkit_status` anytime to check current progress diff --git a/hooks/devkit-guard.sh b/hooks/devkit-guard.sh new file mode 100755 index 0000000..8f53a2e --- /dev/null +++ b/hooks/devkit-guard.sh @@ -0,0 +1,57 @@ +#!/usr/bin/env bash +set -euo pipefail + +# devkit-guard: PreToolUse hook that enforces workflow step ordering. +# Reads $CLAUDE_PLUGIN_DATA/session.json. Blocks out-of-step actions. +# Exit 0 = allow, Exit 2 + stderr = hard block. + +DATA_DIR="${CLAUDE_PLUGIN_DATA:-}" +if [[ -z "$DATA_DIR" ]]; then + exit 0 # not in plugin context +fi + +SESSION_FILE="${DATA_DIR}/session.json" +if [[ ! -f "$SESSION_FILE" ]]; then + exit 0 # no active workflow +fi + +# Parse all session fields in a single python3 call (no jq dependency). +# Outputs tab-separated: status, step_type, enforce, current_step. +# Passes file path via sys.argv to prevent shell injection. +SESSION_DATA=$(python3 -c " +import json, sys +d = json.load(open(sys.argv[1])) +print('\t'.join([ + d.get('status', ''), + d.get('step_type', ''), + d.get('enforce', 'hard'), + d.get('current_step', '') +])) +" "$SESSION_FILE" 2>/dev/null) || { + # python3 unavailable or JSON corrupt — fail closed if session file exists + printf 'BLOCKED: Cannot parse session state (python3 required). Remove %s to clear.\n' "$SESSION_FILE" >&2 + exit 2 +} + +IFS=$'\t' read -r STATUS STEP_TYPE ENFORCE CURRENT_STEP <<< "$SESSION_DATA" + +if [[ "$STATUS" != "running" ]]; then + exit 0 +fi + +# Read tool name from stdin +INPUT=$(cat) +TOOL_NAME=$(printf '%s' "$INPUT" | python3 -c "import json,sys; print(json.load(sys.stdin).get('tool_name',''))" 2>/dev/null || echo "") + +# Command steps: block all standard tools — only MCP tools (devkit_advance) allowed +if [[ "$STEP_TYPE" == "command" && "$ENFORCE" == "hard" ]]; then + case "$TOOL_NAME" in + Bash|Edit|Write|Read|Glob|Grep|Agent|WebFetch|WebSearch|NotebookEdit|Skill) + printf 'BLOCKED: Command step "%s" in progress. Call devkit_advance to execute it and proceed.\n' "$CURRENT_STEP" >&2 + exit 2 + ;; + esac +fi + +# All other cases: allow +exit 0 diff --git a/hooks/devkit-stop-guard.sh b/hooks/devkit-stop-guard.sh new file mode 100755 index 0000000..1368822 --- /dev/null +++ b/hooks/devkit-stop-guard.sh @@ -0,0 +1,38 @@ +#!/usr/bin/env bash +set -euo pipefail + +# devkit-stop-guard: Stop hook that blocks session end during active workflows. +# Outputs JSON: {"decision":"approve"} or {"decision":"block","reason":"..."}. + +DATA_DIR="${CLAUDE_PLUGIN_DATA:-}" +if [[ -z "$DATA_DIR" ]]; then + printf '{"decision":"approve"}' + exit 0 +fi + +SESSION_FILE="${DATA_DIR}/session.json" +if [[ ! -f "$SESSION_FILE" ]]; then + printf '{"decision":"approve"}' + exit 0 +fi + +# Parse all fields in a single python3 call. Passes path via sys.argv +# to prevent shell injection. Outputs valid JSON directly. +python3 -c " +import json, sys +d = json.load(open(sys.argv[1])) +if d.get('status') == 'running': + remaining = d.get('total_steps', 0) - d.get('current_index', 0) + wf = d.get('workflow', 'unknown') + print(json.dumps({ + 'decision': 'block', + 'reason': f'Workflow {wf} incomplete — {remaining} steps remaining. Call devkit_advance to continue.' + })) +else: + print(json.dumps({'decision': 'approve'})) +" "$SESSION_FILE" 2>/dev/null || { + # Cannot parse — approve to avoid trapping the user + printf '{"decision":"approve"}' +} + +exit 0 diff --git a/hooks/hooks.json b/hooks/hooks.json index 1fb596b..c423aa4 100644 --- a/hooks/hooks.json +++ b/hooks/hooks.json @@ -67,6 +67,17 @@ "timeout": 5 } ] + }, + { + "matcher": "*", + "hooks": [ + { + "type": "command", + "command": "${CLAUDE_PLUGIN_ROOT}/hooks/devkit-guard.sh", + "statusMessage": "Workflow guard...", + "timeout": 2 + } + ] } ], "PostToolUse": [ @@ -139,6 +150,17 @@ "timeout": 90 } ] + }, + { + "matcher": "Stop", + "hooks": [ + { + "type": "command", + "command": "${CLAUDE_PLUGIN_ROOT}/hooks/devkit-stop-guard.sh", + "statusMessage": "Workflow check...", + "timeout": 2 + } + ] } ] } diff --git a/hooks/pr-gate.sh b/hooks/pr-gate.sh index e189f67..ff79a0f 100755 --- a/hooks/pr-gate.sh +++ b/hooks/pr-gate.sh @@ -33,7 +33,7 @@ jq -n '{ hookSpecificOutput: { hookEventName: "PreToolUse", permissionDecision: "ask", - permissionDecisionReason: "PR creation detected — want to run /devkit:pr-ready first? (lint, test, security, DRY review, changelog). Say yes to run the pipeline, or approve to skip and create the PR directly." + permissionDecisionReason: "PR creation detected — want to run the pr-ready pipeline first? (lint, test, security, DRY review, changelog, monitor). Say yes to run it, or approve to skip and create the PR directly." } }' exit 0 diff --git a/scripts/ensure-engine.sh b/scripts/ensure-engine.sh deleted file mode 100755 index 09710e6..0000000 --- a/scripts/ensure-engine.sh +++ /dev/null @@ -1,49 +0,0 @@ -#!/usr/bin/env bash -# Ensure the devkit engine binary is on PATH. -# Finds and runs install-engine.sh if devkit is not installed. -# Called by commands and skills before `devkit workflow run`. - -set -euo pipefail - -if command -v devkit >/dev/null 2>&1; then - exit 0 -fi - -# Find install-engine.sh relative to this script (works when called from plugin cache) -SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" -INSTALLER="${SCRIPT_DIR}/install-engine.sh" - -if [[ ! -f "$INSTALLER" ]]; then - # Fallback: search plugin cache (Unix: ~/.claude, Windows: $APPDATA/.claude) - for PLUGIN_ROOT in "$HOME/.claude/plugins" "${APPDATA:+$APPDATA/.claude/plugins}" "${LOCALAPPDATA:+$LOCALAPPDATA/.claude/plugins}"; do - [[ -z "$PLUGIN_ROOT" ]] && continue - [[ -d "$PLUGIN_ROOT" ]] || continue - INSTALLER=$(find "$PLUGIN_ROOT" -path '*/devkit/scripts/install-engine.sh' 2>/dev/null | head -1) - [[ -n "$INSTALLER" ]] && break - done -fi - -if [[ -z "$INSTALLER" ]] || [[ ! -f "$INSTALLER" ]]; then - printf "Cannot find install-engine.sh.\n" - printf "Install manually: https://github.com/5uck1ess/devkit/releases\n" - exit 1 -fi - -# Source instead of subprocess so PATH exports propagate -# shellcheck disable=SC1090 -source "$INSTALLER" - -# Verify devkit is now available (handles ~/.local/bin PATH addition) -if ! command -v devkit >/dev/null 2>&1; then - # Last resort: check common install locations directly - for dir in /usr/local/bin "$HOME/.local/bin" "${LOCALAPPDATA:-}/devkit"; do - if [[ -x "${dir}/devkit" ]]; then - export PATH="${dir}:${PATH}" - break - fi - done - if ! command -v devkit >/dev/null 2>&1; then - printf "devkit installed but not on PATH. Add the install directory to your PATH.\n" - exit 1 - fi -fi diff --git a/scripts/install-engine.sh b/scripts/install-engine.sh deleted file mode 100755 index de72c3c..0000000 --- a/scripts/install-engine.sh +++ /dev/null @@ -1,133 +0,0 @@ -#!/usr/bin/env bash -# Install the devkit engine binary from GitHub releases. -# Called by skills/commands before running `devkit workflow run`. -# -# Usage: ./scripts/install-engine.sh [--check] [--upgrade] -# --check Exit 0 if devkit is on PATH, 1 if not (no install) -# --upgrade Re-download even if devkit is already installed - -set -euo pipefail - -REPO="5uck1ess/devkit" -BINARY="devkit" -INSTALL_DIR="${DEVKIT_INSTALL_DIR:-/usr/local/bin}" - -# --check mode: just test if binary exists -if [[ "${1:-}" == "--check" ]]; then - command -v "$BINARY" >/dev/null 2>&1 - exit $? -fi - -# Skip if already installed (unless --upgrade) -if [[ "${1:-}" != "--upgrade" ]] && command -v "$BINARY" >/dev/null 2>&1; then - printf "devkit engine already installed: %s\n" "$(command -v "$BINARY")" - exit 0 -fi - -# Detect platform -OS=$(uname -s | tr '[:upper:]' '[:lower:]') -ARCH=$(uname -m) -case "$ARCH" in - x86_64) ARCH="amd64" ;; - aarch64) ARCH="arm64" ;; - arm64) ARCH="arm64" ;; - *) printf "Unsupported architecture: %s\n" "$ARCH"; exit 1 ;; -esac - -EXT="" -case "$OS" in - linux|darwin) ;; - # Windows: only reachable via MSYS2, Git Bash, or Cygwin - mingw*|msys*|cygwin*) OS="windows"; EXT=".exe" ;; - *) printf "Unsupported OS: %s\n" "$OS"; exit 1 ;; -esac - -ASSET="${BINARY}-${OS}-${ARCH}${EXT}" -BASE_URL="https://github.com/${REPO}/releases/latest/download" - -printf "Downloading devkit engine (%s/%s)...\n" "$OS" "$ARCH" -TMPDIR_CLEAN="$(mktemp -d)" -trap 'rm -rf "$TMPDIR_CLEAN"' EXIT -TMPFILE="${TMPDIR_CLEAN}/${ASSET}" -TMPCHECKSUM="${TMPDIR_CLEAN}/checksums.txt" - -# Download binary — use -fSL (capital S keeps error messages visible) -if command -v curl >/dev/null 2>&1; then - curl -fSL "${BASE_URL}/${ASSET}" -o "$TMPFILE" || { printf "Download failed: %s/%s\nCheck network and that the release exists.\n" "$BASE_URL" "$ASSET"; exit 1; } - curl -fSL "${BASE_URL}/checksums.txt" -o "$TMPCHECKSUM" || { printf "Warning: could not download checksums.txt\n"; TMPCHECKSUM=""; } -elif command -v wget >/dev/null 2>&1; then - wget -q "${BASE_URL}/${ASSET}" -O "$TMPFILE" || { printf "Download failed: %s/%s\nCheck network and that the release exists.\n" "$BASE_URL" "$ASSET"; exit 1; } - wget -q "${BASE_URL}/checksums.txt" -O "$TMPCHECKSUM" || { printf "Warning: could not download checksums.txt\n"; TMPCHECKSUM=""; } -else - printf "Error: curl or wget required\n" - exit 1 -fi - -# Validate download is non-empty -if [[ ! -s "$TMPFILE" ]]; then - printf "Downloaded file is empty — release may not exist for %s/%s\n" "$OS" "$ARCH" - exit 1 -fi - -# Verify checksum if checksums.txt was downloaded -if [[ -n "$TMPCHECKSUM" ]] && [[ -s "$TMPCHECKSUM" ]]; then - EXPECTED=$(awk -v asset="$ASSET" '$2 == asset || $2 == "./"asset {print $1}' "$TMPCHECKSUM" | head -1) - if [[ -n "$EXPECTED" ]]; then - if command -v sha256sum >/dev/null 2>&1; then - ACTUAL=$(sha256sum "$TMPFILE" | awk '{print $1}') - elif command -v shasum >/dev/null 2>&1; then - ACTUAL=$(shasum -a 256 "$TMPFILE" | awk '{print $1}') - else - printf "Warning: cannot verify checksum (no sha256sum or shasum)\n" - ACTUAL="$EXPECTED" - fi - if [[ "$EXPECTED" != "$ACTUAL" ]]; then - printf "Checksum mismatch! Expected %s, got %s\n" "$EXPECTED" "$ACTUAL" - exit 1 - fi - printf "Checksum verified.\n" - fi -else - printf "Warning: checksums.txt not available — skipping integrity check\n" -fi - -chmod +x "$TMPFILE" - -# Windows: install to user's local bin (MSYS2/Git Bash/Cygwin only) -if [[ "$OS" == "windows" ]]; then - WIN_DIR="${LOCALAPPDATA:-$HOME/AppData/Local}/devkit" - mkdir -p "$WIN_DIR" - mv "$TMPFILE" "${WIN_DIR}/${BINARY}${EXT}" || { printf "Failed to install to %s\n" "$WIN_DIR"; exit 1; } - printf "Installed to %s/%s%s\n" "$WIN_DIR" "$BINARY" "$EXT" - # Add to PATH for current session - export PATH="${WIN_DIR}:${PATH}" - # Persist to user PATH for future sessions via PowerShell - # (safer than setx which silently truncates PATH > 1024 chars) - if command -v powershell.exe >/dev/null 2>&1; then - powershell.exe -Command "[Environment]::SetEnvironmentVariable('Path', [Environment]::GetEnvironmentVariable('Path', 'User') + ';${WIN_DIR}', 'User')" 2>/dev/null \ - && printf "Added %s to user PATH (persistent).\n" "$WIN_DIR" \ - || printf "Warning: could not persist PATH. Add %s to your PATH manually.\n" "$WIN_DIR" - else - printf "Add to PATH: %s\n" "$WIN_DIR" - fi - exit 0 -fi - -# Unix: try INSTALL_DIR, fall back to ~/.local/bin -if [[ -w "$INSTALL_DIR" ]]; then - mv "$TMPFILE" "${INSTALL_DIR}/${BINARY}" || { printf "Failed to install to %s\n" "$INSTALL_DIR"; exit 1; } - printf "Installed to %s/%s\n" "$INSTALL_DIR" "$BINARY" -elif mkdir -p "$HOME/.local/bin"; then - mv "$TMPFILE" "$HOME/.local/bin/${BINARY}" || { printf "Failed to install to %s/.local/bin\n" "$HOME"; exit 1; } - printf "Installed to %s/.local/bin/%s\n" "$HOME" "$BINARY" - # Make binary available in current session - export PATH="$HOME/.local/bin:$PATH" - if ! command -v "$BINARY" >/dev/null 2>&1; then - printf "Warning: installed to ~/.local/bin but it's not on PATH.\n" - printf "Add to your shell profile: export PATH=\"\$HOME/.local/bin:\$PATH\"\n" - fi -else - printf "Cannot write to %s or ~/.local/bin\n" "$INSTALL_DIR" - printf "Run with sudo or set DEVKIT_INSTALL_DIR to a writable directory\n" - exit 1 -fi diff --git a/skills/_principles.yml b/skills/_principles.yml new file mode 100644 index 0000000..a4887a0 --- /dev/null +++ b/skills/_principles.yml @@ -0,0 +1,30 @@ +dry: + - Don't abstract until 3rd duplication + - If one copy changes, must the other? If not, leave it + - Name abstractions for what they do, not where they came from +yagni: + - Build what's needed now, not what might be needed + - Hardcode until configurability is actually requested + - Premature abstraction is worse than duplication +clean-code: + - One function, one job — if you say "and", split it + - Names reveal intent. Booleans read as questions + - Early returns over nesting. Max 2 indent levels +dont-reinvent: + - stdlib > framework > established package > custom + - Every custom solution is code you maintain forever + - Custom justified only when existing solutions don't fit +executing: + - One step at a time. Verify before moving on + - Keep changes small. 20-line diff > 200-line diff +scratchpad: + - Read .devkit/scratchpads/current.md before each iteration + - Record what was tried and why it failed + - 3+ failures = escalate to user +stuck: + - Same error 2x = stop and diagnose + - 3+ failures = escalate, don't retry +test-gen: + - Test behavior, not implementation + - One assertion per test. Name tests for what they prove + - Cover the happy path, one edge case, one error case diff --git a/skills/autoloop/SKILL.md b/skills/autoloop/SKILL.md index 02649c3..7f6bb94 100644 --- a/skills/autoloop/SKILL.md +++ b/skills/autoloop/SKILL.md @@ -48,19 +48,13 @@ If the user says "everything" or skips, leave scope open. ## Invoke the Workflow -Ensure the devkit engine is installed: +Start the workflow via the devkit engine: -```bash -ENSURE="$(find ~/.claude/plugins ${APPDATA:+$APPDATA/.claude/plugins} ${LOCALAPPDATA:+$LOCALAPPDATA/.claude/plugins} -path '*/devkit/scripts/ensure-engine.sh' 2>/dev/null | head -1)"; [ -n "$ENSURE" ] && bash "$ENSURE" || { echo "devkit plugin not found — install from https://github.com/5uck1ess/devkit/releases"; exit 1; } -``` +Assemble the input as a single string: `" | metric: | direction: -is-better | iterations: | scope: "`. -Assemble the input as a single string and invoke: +Use the `devkit_start` tool with workflow: "autoloop" and input: "{input}". -```bash -devkit workflow autoloop " | metric: | direction: -is-better | iterations: | scope: " -``` - -If the engine cannot be installed, tell the user: "The devkit engine binary is required for deterministic workflow execution. Install manually from https://github.com/5uck1ess/devkit/releases" Do NOT fall back to manual steps — the engine is required for determinism. +Then follow each step the engine returns. Call `devkit_advance` after completing each step. The engine controls step order, gates, and loops. Do NOT skip steps. ## Rules diff --git a/skills/deep-research/SKILL.md b/skills/deep-research/SKILL.md index a6031aa..27526cf 100644 --- a/skills/deep-research/SKILL.md +++ b/skills/deep-research/SKILL.md @@ -11,19 +11,11 @@ Costs more tokens (~400k budget) but produces higher-confidence results by activ ## Invoke -Ensure the devkit engine is installed, then run the workflow: +Start the workflow via the devkit engine: -```bash -ENSURE="$(find ~/.claude/plugins ${APPDATA:+$APPDATA/.claude/plugins} ${LOCALAPPDATA:+$LOCALAPPDATA/.claude/plugins} -path '*/devkit/scripts/ensure-engine.sh' 2>/dev/null | head -1)"; [ -n "$ENSURE" ] && bash "$ENSURE" || { echo "devkit plugin not found — install from https://github.com/5uck1ess/devkit/releases"; exit 1; } -``` +Use the `devkit_start` tool with workflow: "deep-research" and input: "{input}". -```bash -devkit workflow run deep-research "{input}" -``` - -The YAML workflow (`workflows/deep-research.yml`) enforces the full ACH sequence deterministically. Claude handles thinking within each step; the engine owns the order. - -If the engine cannot be installed, tell the user: "The devkit engine binary is required for deterministic workflow execution. Install manually from https://github.com/5uck1ess/devkit/releases" Do NOT fall back to manual steps — the engine is required for determinism. +Then follow each step the engine returns. Call `devkit_advance` after completing each step. The engine controls step order, gates, and loops. Do NOT skip steps. ## Rules diff --git a/skills/pr-ready/SKILL.md b/skills/pr-ready/SKILL.md new file mode 100644 index 0000000..45282c5 --- /dev/null +++ b/skills/pr-ready/SKILL.md @@ -0,0 +1,35 @@ +--- +name: pr-ready +description: Full PR pipeline — use when asked to submit a PR, create a pull request, ship this, open a PR, or "make this PR-ready". Runs lint, tests, security scan, generates changelog, creates the PR, and monitors CI + reviewer comments until merge-ready. +--- + +# PR Ready + +Deterministic PR pipeline: validate → necessity → lint (loop) → test (loop) → security → changelog → create PR → monitor (loop). + +## Invoke + +Start the workflow via the devkit engine: + +Use the `devkit_start` tool with workflow: "pr-ready" and input: "{input}". + +Then follow each step the engine returns. Call `devkit_advance` after completing each step. The engine controls step order, gates, and loops. Do NOT skip steps. + +## What it does + +1. **validate** — checks not on main, no uncommitted changes, commits ahead of main +2. **necessity** — removes debug prints, unrelated changes, stray files from the diff +3. **lint** — runs linter, fixes violations, loops until clean +4. **test** — runs test suite, fixes failures, loops until passing +5. **security** — scans for hardcoded secrets, injection, XSS, traversal, insecure deps +6. **changelog** — generates entry from git diff +7. **create-pr** — pushes branch, creates PR via gh pr create with title/summary/changelog/test plan +8. **monitor** — waits for CI, classifies reviewer comments (code_fix/style_nit/question/false_positive/out_of_scope), applies fixes, replies, pushes, loops until all resolved + +## Rules + +- Never force-push +- Never dismiss reviews — only re-request after fixing +- Reply to false positives with evidence, not dismissal +- Escalate architectural changes via AskUserQuestion +- Stop after 10 monitor iterations or when stuck (3 iters with zero progress) diff --git a/skills/research/SKILL.md b/skills/research/SKILL.md index 2d9df91..c071979 100644 --- a/skills/research/SKILL.md +++ b/skills/research/SKILL.md @@ -9,19 +9,11 @@ Deterministic research workflow: clarify → decompose → parallel search → s ## Invoke -Ensure the devkit engine is installed, then run the workflow: +Start the workflow via the devkit engine: -```bash -ENSURE="$(find ~/.claude/plugins ${APPDATA:+$APPDATA/.claude/plugins} ${LOCALAPPDATA:+$LOCALAPPDATA/.claude/plugins} -path '*/devkit/scripts/ensure-engine.sh' 2>/dev/null | head -1)"; [ -n "$ENSURE" ] && bash "$ENSURE" || { echo "devkit plugin not found — install from https://github.com/5uck1ess/devkit/releases"; exit 1; } -``` +Use the `devkit_start` tool with workflow: "research" and input: "{input}". -```bash -devkit workflow run research "{input}" -``` - -The YAML workflow (`workflows/research.yml`) enforces the step sequence deterministically. Claude handles thinking within each step; the engine owns the order. - -If the engine cannot be installed, tell the user: "The devkit engine binary is required for deterministic workflow execution. Install manually from https://github.com/5uck1ess/devkit/releases" Do NOT fall back to manual steps — the engine is required for determinism. +Then follow each step the engine returns. Call `devkit_advance` after completing each step. The engine controls step order, gates, and loops. Do NOT skip steps. ## Rules diff --git a/src/Makefile b/src/Makefile index 8cc920b..fede679 100644 --- a/src/Makefile +++ b/src/Makefile @@ -6,7 +6,7 @@ LDFLAGS := -s -w -X main.version=$(VERSION) GOFLAGS := -trimpath PLUGIN_JSON := $(CURDIR)/../.claude-plugin/plugin.json -.PHONY: build build-for build-all install link clean test vet fmt check all sync-version +.PHONY: build build-for build-all install install-plugin link clean test vet fmt check all sync-version all: check build @@ -46,6 +46,11 @@ build-all: ## Build all platform binaries @$(MAKE) build-for GOOS=windows GOARCH=amd64 @$(MAKE) build-for GOOS=windows GOARCH=arm64 +install-plugin: | sync-version ## Build and copy binary to plugin bin/ for current platform + @mkdir -p $(CURDIR)/../bin + go build $(GOFLAGS) -ldflags '$(LDFLAGS)' -o $(CURDIR)/../bin/$(BINARY) . + @echo "Installed to plugin bin/$(BINARY)" + install: | sync-version go install $(GOFLAGS) -ldflags '$(LDFLAGS)' . @echo "Installed to $$(go env GOPATH)/bin/$(BINARY)" diff --git a/src/TODO.md b/src/TODO.md deleted file mode 100644 index 18bfe67..0000000 --- a/src/TODO.md +++ /dev/null @@ -1,70 +0,0 @@ -# Devkit CLI Harness — TODO - -> Go binary (Cobra CLI) for deterministic orchestration that skills/workflows can't guarantee. -> Complements both pi workflows and Claude Code skills. - -## When to build - -- Self-improve loop stops early because the LLM decides to -- Multi-agent dispatch doesn't wait for all agents -- You want to run iterations unattended (overnight, CI, cron) -- You need deterministic behavior every time -- Pi workflow conditions aren't reliable enough for production loops - -## Architecture - -``` -devkit (Go binary, Cobra CLI) -├── cmd/ -│ ├── root.go # Cobra root command -│ ├── review.go # devkit review "prompt" -│ ├── improve.go # devkit improve --target --metric --objective --iterations -│ └── dispatch.go # devkit dispatch --agent claude|codex|gemini|pi|all "prompt" -├── runners/ -│ ├── claude.go # Spawn claude -p with agent config flags -│ ├── codex.go # Spawn codex exec --full-auto -│ ├── gemini.go # Spawn gemini -p -y -│ └── pi.go # Spawn pi -p with workflow flags -├── loops/ -│ └── improve.go # Baseline → iterate → measure → keep/discard → repeat -├── lib/ -│ ├── git.go # Deterministic git ops (branch, commit, revert, diff) -│ ├── metric.go # Run metric command, parse result, compare -│ └── report.go # Consolidated output (stdout, markdown, JSON) -├── go.mod -└── main.go -``` - -## Key dependencies - -- github.com/spf13/cobra — CLI framework -- os/exec — spawn agent processes -- Standard library for everything else (git, file I/O, JSON) - -## Commands - -```bash -devkit review "check for DRY violations" -devkit review --security -devkit improve --target src/ --metric "npm test" --objective "fix failures" --iterations 20 -devkit dispatch --agent all "compare caching approaches" -devkit dispatch --agent pi "analyze with pi workflow" -``` - -## What the harness handles (that skills/workflows can't) - -- Deterministic loop control (exact N iterations) -- Process management (spawn, timeout, kill) -- Exit code-based metric evaluation -- Git branching/committing/reverting without LLM involvement -- Parallel process orchestration with proper wait/collect -- Crash recovery (state on disk, resume where left off) -- Budget tracking across iterations -- Structured JSON reporting - -## What stays as skills/workflows - -- The prompts (what to tell each agent) -- Agent configs (model, effort, maxTurns) -- Pi workflow definitions (YAML) -- Claude Code plugin for users who don't need the binary diff --git a/src/cmd/bugfix.go b/src/cmd/bugfix.go deleted file mode 100644 index 1adde4a..0000000 --- a/src/cmd/bugfix.go +++ /dev/null @@ -1,68 +0,0 @@ -package cmd - -import ( - "fmt" - "strings" - - "github.com/5uck1ess/devkit/lib" - "github.com/5uck1ess/devkit/loops" - "github.com/spf13/cobra" -) - -var bugfixCmd = &cobra.Command{ - Use: "bugfix [description]", - Short: "Full bugfix lifecycle: diagnose, fix, verify", - Long: "Spawns an AI agent for each step: diagnose root cause → apply fix → run tests to verify.", - Example: ` devkit bugfix "login returns 500 when email has a plus sign" - devkit bugfix "race condition in cache invalidation" --test "go test ./..."`, - Args: cobra.MinimumNArgs(1), - RunE: func(cmd *cobra.Command, args []string) error { - testCmd, _ := cmd.Flags().GetString("test") - budget, _ := cmd.Flags().GetFloat64("budget") - - dirty, err := (&lib.Git{Dir: repoRoot}).HasUncommittedChanges() - if err != nil { - return fmt.Errorf("check git status: %w", err) - } - if dirty { - return fmt.Errorf("working tree has uncommitted changes — commit or stash first") - } - - agentName, _ := cmd.Flags().GetString("agent") - runner, err := resolveRunner(agentName) - if err != nil { - return err - } - - result, err := loops.RunBugfix(cmd.Context(), db, runner, &lib.Git{Dir: repoRoot}, loops.BugfixConfig{ - Description: strings.Join(args, " "), - TestCmd: testCmd, - RepoRoot: repoRoot, - BudgetUSD: budget, - }) - if err != nil { - return err - } - - printBugfixResult(result) - return nil - }, -} - -func init() { - rootCmd.AddCommand(bugfixCmd) - bugfixCmd.Flags().String("test", "", "Test command to verify the fix") - bugfixCmd.Flags().Float64("budget", 0, "Maximum spend in USD (0 = unlimited)") -} - -func printBugfixResult(r *loops.BugfixResult) { - var totalCost float64 - for _, s := range r.Steps { - totalCost += s.CostUSD - } - fmt.Printf("\n=== Bugfix Complete ===\n") - fmt.Printf("Session: %s\n", r.Session.ID) - fmt.Printf("Steps: %d\n", len(r.Steps)) - fmt.Printf("Cost: $%.4f\n", totalCost) - fmt.Printf("\nRun `devkit status %s` for details.\n", r.Session.ID) -} diff --git a/src/cmd/dispatch.go b/src/cmd/dispatch.go deleted file mode 100644 index 60be42e..0000000 --- a/src/cmd/dispatch.go +++ /dev/null @@ -1,84 +0,0 @@ -package cmd - -import ( - "fmt" - "strings" - - "github.com/5uck1ess/devkit/loops" - "github.com/5uck1ess/devkit/runners" - "github.com/spf13/cobra" -) - -var dispatchCmd = &cobra.Command{ - Use: "dispatch [prompt]", - Short: "Send a task to multiple agents and compare outputs", - Long: "Dispatches the same prompt to all available agents in parallel, collects and compares results.", - Example: ` devkit dispatch "compare caching approaches" - devkit dispatch --agents claude,gemini "review the API design"`, - Args: cobra.MinimumNArgs(1), - RunE: func(cmd *cobra.Command, args []string) error { - agentList, _ := cmd.Flags().GetString("agents") - - prompt := strings.Join(args, " ") - - var agents []string - if agentList != "" { - for _, a := range strings.Split(agentList, ",") { - a = strings.TrimSpace(a) - if a != "" { - agents = append(agents, a) - } - } - } - - available := runners.DetectRunners() - - cfg := loops.DispatchConfig{ - Prompt: prompt, - Agents: agents, - RepoRoot: repoRoot, - } - - result, err := loops.RunDispatch(cmd.Context(), db, available, cfg) - if err != nil { - return err - } - - printDispatchResult(result) - return nil - }, -} - -func init() { - rootCmd.AddCommand(dispatchCmd) - dispatchCmd.Flags().String("agents", "", "Comma-separated list of agents (default: all available)") -} - -func printDispatchResult(r *loops.DispatchResult) { - fmt.Printf("\n=== Dispatch: %s ===\n\n", r.Session.ID) - - for _, res := range r.Results { - fmt.Printf("### %s\n", strings.ToUpper(res.Agent)) - if res.Error != nil { - fmt.Printf("Error: %s\n\n", res.Error) - continue - } - fmt.Printf("%s\n\n", res.Output) - } - - var totalCost float64 - for _, res := range r.Results { - totalCost += res.Cost - } - fmt.Printf("---\nAgents: %d/%d responded | Total cost: $%.4f\n", countSuccess(r.Results), len(r.Results), totalCost) -} - -func countSuccess(results []loops.AgentResult) int { - n := 0 - for _, r := range results { - if r.Error == nil { - n++ - } - } - return n -} diff --git a/src/cmd/feature.go b/src/cmd/feature.go deleted file mode 100644 index 213e72d..0000000 --- a/src/cmd/feature.go +++ /dev/null @@ -1,74 +0,0 @@ -package cmd - -import ( - "fmt" - "strings" - - "github.com/5uck1ess/devkit/lib" - "github.com/5uck1ess/devkit/loops" - "github.com/spf13/cobra" -) - -var featureCmd = &cobra.Command{ - Use: "feature [description]", - Short: "Full feature lifecycle: plan, implement, test, lint", - Long: "Spawns an AI agent for each step: plan → implement → test (loop until green) → lint.", - Example: ` devkit feature "add JWT authentication" --target src/auth/ - devkit feature "add search endpoint" --test "npm test" --lint "npm run lint"`, - Args: cobra.MinimumNArgs(1), - RunE: func(cmd *cobra.Command, args []string) error { - target, _ := cmd.Flags().GetString("target") - testCmd, _ := cmd.Flags().GetString("test") - lintCmd, _ := cmd.Flags().GetString("lint") - budget, _ := cmd.Flags().GetFloat64("budget") - - dirty, err := (&lib.Git{Dir: repoRoot}).HasUncommittedChanges() - if err != nil { - return fmt.Errorf("check git status: %w", err) - } - if dirty { - return fmt.Errorf("working tree has uncommitted changes — commit or stash first") - } - - agentName, _ := cmd.Flags().GetString("agent") - runner, err := resolveRunner(agentName) - if err != nil { - return err - } - - result, err := loops.RunFeature(cmd.Context(), db, runner, &lib.Git{Dir: repoRoot}, loops.FeatureConfig{ - Description: strings.Join(args, " "), - Target: target, - TestCmd: testCmd, - LintCmd: lintCmd, - RepoRoot: repoRoot, - BudgetUSD: budget, - }) - if err != nil { - return err - } - - printFeatureResult(result) - return nil - }, -} - -func init() { - rootCmd.AddCommand(featureCmd) - featureCmd.Flags().String("target", ".", "Directory or file to modify") - featureCmd.Flags().String("test", "", "Test command (runs after implementation)") - featureCmd.Flags().String("lint", "", "Lint command (runs after tests)") - featureCmd.Flags().Float64("budget", 0, "Maximum spend in USD (0 = unlimited)") -} - -func printFeatureResult(r *loops.FeatureResult) { - var totalCost float64 - for _, s := range r.Steps { - totalCost += s.CostUSD - } - fmt.Printf("\n=== Feature Complete ===\n") - fmt.Printf("Session: %s\n", r.Session.ID) - fmt.Printf("Steps: %d\n", len(r.Steps)) - fmt.Printf("Cost: $%.4f\n", totalCost) - fmt.Printf("\nRun `devkit status %s` for details.\n", r.Session.ID) -} diff --git a/src/cmd/improve.go b/src/cmd/improve.go deleted file mode 100644 index c101c2e..0000000 --- a/src/cmd/improve.go +++ /dev/null @@ -1,100 +0,0 @@ -package cmd - -import ( - "fmt" - - "github.com/5uck1ess/devkit/lib" - "github.com/5uck1ess/devkit/loops" - "github.com/spf13/cobra" -) - -var improveCmd = &cobra.Command{ - Use: "improve", - Short: "Run a metric-gated improvement loop", - Long: "Spawns an AI agent per iteration. Each iteration: propose change, run metric, keep if pass, revert if fail.", - Example: ` devkit improve --target src/ --metric "npm test" --objective "0 failing tests" --iterations 20 - devkit improve --metric "go test ./..." --iterations 10 --budget 5.00`, - RunE: func(cmd *cobra.Command, args []string) error { - target, _ := cmd.Flags().GetString("target") - metric, _ := cmd.Flags().GetString("metric") - objective, _ := cmd.Flags().GetString("objective") - iterations, _ := cmd.Flags().GetInt("iterations") - budget, _ := cmd.Flags().GetFloat64("budget") - - if metric == "" { - return fmt.Errorf("--metric is required — provide a command that exits 0 on success (e.g., \"npm test\")") - } - if iterations < 1 { - return fmt.Errorf("--iterations must be at least 1") - } - if target == "" { - target = "." - } - if objective == "" { - objective = "improve the metric to pass" - } - - dirty, err := (&lib.Git{Dir: repoRoot}).HasUncommittedChanges() - if err != nil { - return fmt.Errorf("check git status: %w", err) - } - if dirty { - return fmt.Errorf("working tree has uncommitted changes — commit or stash before running devkit improve") - } - - agentName, _ := cmd.Flags().GetString("agent") - runner, err := resolveRunner(agentName) - if err != nil { - return err - } - - git := &lib.Git{Dir: repoRoot} - cfg := loops.ImproveConfig{ - Target: target, - Metric: metric, - Objective: objective, - MaxIterations: iterations, - BudgetUSD: budget, - MaxFailures: 3, - RepoRoot: repoRoot, - } - - result, err := loops.RunImproveLoop(cmd.Context(), db, runner, git, cfg) - if err != nil { - return err - } - - printImproveResult(result) - return nil - }, -} - -func init() { - rootCmd.AddCommand(improveCmd) - improveCmd.Flags().String("target", ".", "Directory or file to improve") - improveCmd.Flags().String("metric", "", "Command that exits 0 on success (required)") - improveCmd.Flags().String("objective", "", "What the improvement should achieve") - improveCmd.Flags().Int("iterations", 10, "Maximum number of iterations") - improveCmd.Flags().Float64("budget", 0, "Maximum spend in USD (0 = unlimited)") -} - -func printImproveResult(r *loops.ImproveResult) { - kept := 0 - reverted := 0 - var totalCost float64 - for _, s := range r.Steps { - totalCost += s.CostUSD - if s.Kept { - kept++ - } else { - reverted++ - } - } - - fmt.Printf("\n=== Improve Session Complete ===\n") - fmt.Printf("Session: %s\n", r.Session.ID) - fmt.Printf("Status: %s\n", r.StopReason) - fmt.Printf("Iterations: %d (%d kept, %d reverted)\n", len(r.Steps), kept, reverted) - fmt.Printf("Total cost: $%.4f\n", totalCost) - fmt.Printf("\nRun `devkit status %s` for full details.\n", r.Session.ID) -} diff --git a/src/cmd/mcp.go b/src/cmd/mcp.go new file mode 100644 index 0000000..6676284 --- /dev/null +++ b/src/cmd/mcp.go @@ -0,0 +1,47 @@ +package cmd + +import ( + "context" + "fmt" + "os" + "os/signal" + "path/filepath" + "syscall" + + devkitmcp "github.com/5uck1ess/devkit/mcp" + "github.com/spf13/cobra" +) + +var mcpCmd = &cobra.Command{ + Use: "mcp", + Short: "Start MCP server on stdio", + Long: "Launch the devkit engine as an MCP server communicating via JSON-RPC over stdin/stdout.", + RunE: func(cmd *cobra.Command, args []string) error { + dataDir := os.Getenv("CLAUDE_PLUGIN_DATA") + if dataDir == "" { + dataDir = filepath.Join(repoRoot, ".devkit") + } + + pluginRoot := os.Getenv("CLAUDE_PLUGIN_ROOT") + workflowDir := filepath.Join(repoRoot, "workflows") + if pluginRoot != "" { + workflowDir = filepath.Join(pluginRoot, "workflows") + } + + srv, err := devkitmcp.NewServer(repoRoot, dataDir, workflowDir) + if err != nil { + return fmt.Errorf("create MCP server: %w", err) + } + defer srv.Close() + + ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM) + defer cancel() + + fmt.Fprintln(os.Stderr, "devkit MCP server ready") + return srv.Serve(ctx) + }, +} + +func init() { + rootCmd.AddCommand(mcpCmd) +} diff --git a/src/cmd/refactor.go b/src/cmd/refactor.go deleted file mode 100644 index 7edbfb6..0000000 --- a/src/cmd/refactor.go +++ /dev/null @@ -1,71 +0,0 @@ -package cmd - -import ( - "fmt" - "strings" - - "github.com/5uck1ess/devkit/lib" - "github.com/5uck1ess/devkit/loops" - "github.com/spf13/cobra" -) - -var refactorCmd = &cobra.Command{ - Use: "refactor [description]", - Short: "Full refactor lifecycle: analyze, transform, verify", - Long: "Spawns an AI agent for each step: analyze code smells → apply transformations → verify tests still pass.", - Example: ` devkit refactor "extract auth middleware into shared package" --target src/ - devkit refactor "flatten nested callbacks to async/await" --test "npm test"`, - Args: cobra.MinimumNArgs(1), - RunE: func(cmd *cobra.Command, args []string) error { - target, _ := cmd.Flags().GetString("target") - testCmd, _ := cmd.Flags().GetString("test") - budget, _ := cmd.Flags().GetFloat64("budget") - - dirty, err := (&lib.Git{Dir: repoRoot}).HasUncommittedChanges() - if err != nil { - return fmt.Errorf("check git status: %w", err) - } - if dirty { - return fmt.Errorf("working tree has uncommitted changes — commit or stash first") - } - - agentName, _ := cmd.Flags().GetString("agent") - runner, err := resolveRunner(agentName) - if err != nil { - return err - } - - result, err := loops.RunRefactor(cmd.Context(), db, runner, &lib.Git{Dir: repoRoot}, loops.RefactorConfig{ - Description: strings.Join(args, " "), - Target: target, - TestCmd: testCmd, - RepoRoot: repoRoot, - BudgetUSD: budget, - }) - if err != nil { - return err - } - - printRefactorResult(result) - return nil - }, -} - -func init() { - rootCmd.AddCommand(refactorCmd) - refactorCmd.Flags().String("target", ".", "Directory or file to refactor") - refactorCmd.Flags().String("test", "", "Test command to verify no behavior change") - refactorCmd.Flags().Float64("budget", 0, "Maximum spend in USD (0 = unlimited)") -} - -func printRefactorResult(r *loops.RefactorResult) { - var totalCost float64 - for _, s := range r.Steps { - totalCost += s.CostUSD - } - fmt.Printf("\n=== Refactor Complete ===\n") - fmt.Printf("Session: %s\n", r.Session.ID) - fmt.Printf("Steps: %d\n", len(r.Steps)) - fmt.Printf("Cost: $%.4f\n", totalCost) - fmt.Printf("\nRun `devkit status %s` for details.\n", r.Session.ID) -} diff --git a/src/cmd/resume.go b/src/cmd/resume.go deleted file mode 100644 index ad54203..0000000 --- a/src/cmd/resume.go +++ /dev/null @@ -1,83 +0,0 @@ -package cmd - -import ( - "fmt" - "regexp" - "strings" - - "github.com/5uck1ess/devkit/lib" - "github.com/5uck1ess/devkit/loops" - "github.com/spf13/cobra" -) - -var sessionIDPattern = regexp.MustCompile(`^[a-f0-9]{12}$`) - -var resumeCmd = &cobra.Command{ - Use: "resume ", - Short: "Resume a paused or crashed session", - Long: "Picks up an improve session from where it left off, using the SQLite state and handoff file.", - Args: cobra.ExactArgs(1), - RunE: func(cmd *cobra.Command, args []string) error { - sessionID := strings.ToLower(args[0]) - if !sessionIDPattern.MatchString(sessionID) { - return fmt.Errorf("invalid session ID %q — expected 12 hex characters (e.g., a1b2c3d4e5f6)", sessionID) - } - - session, err := db.GetSession(sessionID) - if err != nil { - return err - } - - if session.Status == "running" { - return fmt.Errorf("session %s is still running — if it crashed, set status to paused with: sqlite3 .devkit/devkit.db \"UPDATE sessions SET status='paused' WHERE id='%s'\"", sessionID, sessionID) - } - if session.Status != "paused" && session.Status != "failed" { - return fmt.Errorf("session %s has status %q — only paused or failed sessions can be resumed", sessionID, session.Status) - } - - if session.Workflow != "improve" { - return fmt.Errorf("resume only supports improve sessions — session %s is %q", sessionID, session.Workflow) - } - - agentName, _ := cmd.Flags().GetString("agent") - runner, err := resolveRunner(agentName) - if err != nil { - return err - } - - // Require clean worktree before resuming - git := &lib.Git{Dir: repoRoot} - dirty, err := git.HasUncommittedChanges() - if err != nil { - return fmt.Errorf("check git status: %w", err) - } - if dirty { - return fmt.Errorf("working tree has uncommitted changes — commit or stash before resuming") - } - - branch, err := git.CurrentBranch() - if err != nil { - return fmt.Errorf("get current branch: %w", err) - } - - expectedBranch := fmt.Sprintf("self-improve/%s", sessionID) - if branch != expectedBranch { - fmt.Printf("Switching to branch %s...\n", expectedBranch) - if err := git.CheckoutBranch(expectedBranch); err != nil { - return fmt.Errorf("checkout branch %s: %w — does the branch still exist?", expectedBranch, err) - } - } - - result, err := loops.ResumeImproveLoop(cmd.Context(), db, runner, git, session, repoRoot) - if err != nil { - return err - } - - printImproveResult(result) - return nil - }, -} - -func init() { - rootCmd.AddCommand(resumeCmd) -} diff --git a/src/cmd/review.go b/src/cmd/review.go deleted file mode 100644 index b8737a6..0000000 --- a/src/cmd/review.go +++ /dev/null @@ -1,86 +0,0 @@ -package cmd - -import ( - "fmt" - "strings" - - "github.com/5uck1ess/devkit/lib" - "github.com/5uck1ess/devkit/loops" - "github.com/5uck1ess/devkit/runners" - "github.com/spf13/cobra" -) - -var reviewCmd = &cobra.Command{ - Use: "review [prompt]", - Short: "Multi-agent code review of current branch diff", - Long: "Dispatches the diff to all available agents in parallel, consolidates findings.", - Example: ` devkit review - devkit review "check for DRY violations" - devkit review --security - devkit review --agents claude,codex`, - RunE: func(cmd *cobra.Command, args []string) error { - security, _ := cmd.Flags().GetBool("security") - agentList, _ := cmd.Flags().GetString("agents") - - prompt := strings.Join(args, " ") - - var agents []string - if agentList != "" { - for _, a := range strings.Split(agentList, ",") { - a = strings.TrimSpace(a) - if a != "" { - agents = append(agents, a) - } - } - } - - available := runners.DetectRunners() - git := &lib.Git{Dir: repoRoot} - - cfg := loops.ReviewConfig{ - Prompt: prompt, - Security: security, - Agents: agents, - RepoRoot: repoRoot, - } - - result, err := loops.RunReview(cmd.Context(), db, available, git, cfg) - if err != nil { - return err - } - - printReviewResult(result) - return nil - }, -} - -func init() { - rootCmd.AddCommand(reviewCmd) - reviewCmd.Flags().Bool("security", false, "Focus on security issues") - reviewCmd.Flags().String("agents", "", "Comma-separated list of agents (default: all available)") -} - -func printReviewResult(r *loops.ReviewResult) { - fmt.Printf("\n=== Review: %s ===\n\n", r.Session.ID) - - for _, res := range r.Results { - fmt.Printf("### %s\n", strings.ToUpper(res.Agent)) - if res.Error != nil { - fmt.Printf("Error: %s\n\n", res.Error) - continue - } - fmt.Printf("%s\n\n", res.Output) - } - - fmt.Printf("---\nAgents: %d/%d responded\n", countSuccessful(r.Results), len(r.Results)) -} - -func countSuccessful(results []loops.AgentResult) int { - n := 0 - for _, r := range results { - if r.Error == nil { - n++ - } - } - return n -} diff --git a/src/cmd/status.go b/src/cmd/status.go index a054b17..e870600 100644 --- a/src/cmd/status.go +++ b/src/cmd/status.go @@ -31,7 +31,7 @@ func showAllSessions() error { return fmt.Errorf("list sessions: %w", err) } if len(sessions) == 0 { - fmt.Println("No sessions found. Run `devkit improve`, `devkit review`, or `devkit dispatch` to start one.") + fmt.Println("No sessions found. Run `devkit workflow run ` to start one, or use the MCP tools inside Claude Code.") return nil } @@ -97,9 +97,6 @@ func showSessionDetail(id string) error { } } - if session.Status == "paused" || session.Status == "failed" { - fmt.Printf("\nResume with: devkit resume %s\n", session.ID) - } return nil } diff --git a/src/cmd/testgen.go b/src/cmd/testgen.go deleted file mode 100644 index d746610..0000000 --- a/src/cmd/testgen.go +++ /dev/null @@ -1,65 +0,0 @@ -package cmd - -import ( - "fmt" - "strings" - - "github.com/5uck1ess/devkit/lib" - "github.com/5uck1ess/devkit/loops" - "github.com/spf13/cobra" -) - -var testGenCmd = &cobra.Command{ - Use: "test-gen [target]", - Short: "Generate tests for target code, run them, fix failures", - Long: "Analyzes target code, generates comprehensive tests, runs them, and iterates until green.", - Example: ` devkit test-gen src/auth/ - devkit test-gen lib/parser.go --test "go test ./..." - devkit test-gen src/ --test "npm test" --budget 5.00`, - Args: cobra.MinimumNArgs(1), - RunE: func(cmd *cobra.Command, args []string) error { - testCmd, _ := cmd.Flags().GetString("test") - budget, _ := cmd.Flags().GetFloat64("budget") - - dirty, err := (&lib.Git{Dir: repoRoot}).HasUncommittedChanges() - if err != nil { - return fmt.Errorf("check git status: %w", err) - } - if dirty { - return fmt.Errorf("working tree has uncommitted changes — commit or stash first") - } - - agentName, _ := cmd.Flags().GetString("agent") - runner, err := resolveRunner(agentName) - if err != nil { - return err - } - - result, err := loops.RunTestGen(cmd.Context(), db, runner, &lib.Git{Dir: repoRoot}, loops.TestGenConfig{ - Target: strings.Join(args, " "), - TestCmd: testCmd, - RepoRoot: repoRoot, - BudgetUSD: budget, - }) - if err != nil { - return err - } - - var totalCost float64 - for _, s := range result.Steps { - totalCost += s.CostUSD - } - fmt.Printf("\n=== Test Generation Complete ===\n") - fmt.Printf("Session: %s\n", result.Session.ID) - fmt.Printf("Steps: %d\n", len(result.Steps)) - fmt.Printf("Cost: $%.4f\n", totalCost) - fmt.Printf("\nRun `devkit status %s` for details.\n", result.Session.ID) - return nil - }, -} - -func init() { - rootCmd.AddCommand(testGenCmd) - testGenCmd.Flags().String("test", "", "Test command to run generated tests") - testGenCmd.Flags().Float64("budget", 0, "Maximum spend in USD (0 = unlimited)") -} diff --git a/src/engine/engine_test.go b/src/engine/engine_test.go index e89446f..1502a41 100644 --- a/src/engine/engine_test.go +++ b/src/engine/engine_test.go @@ -1314,3 +1314,32 @@ func TestParseRealWorkflows(t *testing.T) { }) } } + +func TestParseWorkflowNewFields(t *testing.T) { + yaml := []byte(` +name: test-new-fields +enforce: soft +branch: true +principles: [dry, yagni] +steps: + - id: step1 + prompt: "do something" + principles: [clean-code] +`) + wf, err := Parse(yaml) + if err != nil { + t.Fatalf("parse failed: %v", err) + } + if wf.Enforce != "soft" { + t.Errorf("enforce = %q, want soft", wf.Enforce) + } + if !wf.BranchMode { + t.Error("branch should be true") + } + if len(wf.Principles) != 2 || wf.Principles[0] != "dry" { + t.Errorf("principles = %v, want [dry yagni]", wf.Principles) + } + if len(wf.Steps[0].Principles) != 1 || wf.Steps[0].Principles[0] != "clean-code" { + t.Errorf("step principles = %v, want [clean-code]", wf.Steps[0].Principles) + } +} diff --git a/src/engine/workflow.go b/src/engine/workflow.go index 0eff7e0..4a55a58 100644 --- a/src/engine/workflow.go +++ b/src/engine/workflow.go @@ -17,6 +17,9 @@ type Workflow struct { Description string `yaml:"description"` Budget Budget `yaml:"budget"` Steps []WfStep `yaml:"steps"` + Enforce string `yaml:"enforce"` // "hard" (default) | "soft" + BranchMode bool `yaml:"branch"` // create git branch per session + Principles []string `yaml:"principles"` // principle keys to inject } // Budget controls token spending limits. @@ -27,14 +30,15 @@ type Budget struct { // WfStep is a single step in a workflow. type WfStep struct { - ID string `yaml:"id"` - Model string `yaml:"model"` - Prompt string `yaml:"prompt"` - Command string `yaml:"command"` - Expect string `yaml:"expect"` - Parallel []string `yaml:"parallel"` - Loop *Loop `yaml:"loop"` - Branch []Branch `yaml:"branch"` + ID string `yaml:"id"` + Model string `yaml:"model"` + Prompt string `yaml:"prompt"` + Command string `yaml:"command"` + Expect string `yaml:"expect"` + Parallel []string `yaml:"parallel"` + Loop *Loop `yaml:"loop"` + Branch []Branch `yaml:"branch"` + Principles []string `yaml:"principles"` // per-step override } // Loop controls step repetition. @@ -73,9 +77,18 @@ func Parse(data []byte) (*Workflow, error) { // validate checks the workflow for structural errors. func validate(wf *Workflow) error { + // Apply defaults before validation so directly-constructed Workflow values + // (not via Parse) also get sensible defaults. + if wf.Enforce == "" { + wf.Enforce = "hard" + } + if wf.Name == "" { return fmt.Errorf("workflow missing name") } + if wf.Enforce != "hard" && wf.Enforce != "soft" { + return fmt.Errorf("workflow %q has invalid enforce %q — must be \"hard\" or \"soft\"", wf.Name, wf.Enforce) + } if len(wf.Steps) == 0 { return fmt.Errorf("workflow %q has no steps", wf.Name) } diff --git a/src/go.mod b/src/go.mod index aa00bf9..7947529 100644 --- a/src/go.mod +++ b/src/go.mod @@ -10,12 +10,16 @@ require ( require ( github.com/dustin/go-humanize v1.0.1 // indirect + github.com/google/jsonschema-go v0.4.2 // indirect github.com/google/uuid v1.6.0 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect + github.com/mark3labs/mcp-go v0.47.1 // indirect github.com/mattn/go-isatty v0.0.20 // indirect github.com/ncruces/go-strftime v1.0.0 // indirect github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect + github.com/spf13/cast v1.7.1 // indirect github.com/spf13/pflag v1.0.9 // indirect + github.com/yosida95/uritemplate/v3 v3.0.2 // indirect golang.org/x/sys v0.42.0 // indirect modernc.org/libc v1.70.0 // indirect modernc.org/mathutil v1.7.1 // indirect diff --git a/src/go.sum b/src/go.sum index 14b1067..27b3eb1 100644 --- a/src/go.sum +++ b/src/go.sum @@ -1,6 +1,8 @@ github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g= github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= +github.com/google/jsonschema-go v0.4.2 h1:tmrUohrwoLZZS/P3x7ex0WAVknEkBZM46iALbcqoRA8= +github.com/google/jsonschema-go v0.4.2/go.mod h1:r5quNTdLOYEz95Ru18zA0ydNbBuYoo9tgaYcxEYhJVE= github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs= github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= @@ -9,6 +11,8 @@ github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= +github.com/mark3labs/mcp-go v0.47.1 h1:A9sJJ20mscl/ssLYHjodfaoBmq6uuhMG7pAPNYaQymQ= +github.com/mark3labs/mcp-go v0.47.1/go.mod h1:JKTC7R2LLVagkEWK7Kwu7DbmA6iIvnNAod6yrHiQMag= github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w= @@ -16,10 +20,14 @@ github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJm github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= +github.com/spf13/cast v1.7.1 h1:cuNEagBQEHWN1FnbGEjCXL2szYEXqfJPbP2HNUaca9Y= +github.com/spf13/cast v1.7.1/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo= github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU= github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4= github.com/spf13/pflag v1.0.9 h1:9exaQaMOCwffKiiiYk6/BndUBv+iRViNW+4lEMi0PvY= github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +github.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zIM+UJPGz4= +github.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT040gmv5BQTMR2HP4= go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= golang.org/x/mod v0.33.0 h1:tHFzIWbBifEmbwtGz65eaWyGiGZatSrT9prnU8DbVL8= golang.org/x/mod v0.33.0/go.mod h1:swjeQEj+6r7fODbD2cqrnje9PnziFuw4bmLbBZFrQ5w= diff --git a/src/lib/metric.go b/src/lib/metric.go deleted file mode 100644 index 0c1257e..0000000 --- a/src/lib/metric.go +++ /dev/null @@ -1,53 +0,0 @@ -package lib - -import ( - "bytes" - "context" - "os/exec" - "time" -) - -type MetricResult struct { - ExitCode int - Output string - Duration time.Duration -} - -func RunMetric(ctx context.Context, command string, dir string) MetricResult { - start := time.Now() - - cmd := exec.CommandContext(ctx, "sh", "-c", command) - cmd.Dir = dir - var stdout, stderr bytes.Buffer - cmd.Stdout = &stdout - cmd.Stderr = &stderr - - err := cmd.Run() - duration := time.Since(start) - - exitCode := 0 - if err != nil { - if exitErr, ok := err.(*exec.ExitError); ok { - exitCode = exitErr.ExitCode() - } else { - exitCode = 1 - } - } - - output := stdout.String() - if output == "" { - output = stderr.String() - } - - // Truncate to avoid bloating state - const maxOutput = 4096 - if len(output) > maxOutput { - output = output[:maxOutput] + "\n... (truncated)" - } - - return MetricResult{ - ExitCode: exitCode, - Output: output, - Duration: duration, - } -} diff --git a/src/lib/metric_test.go b/src/lib/metric_test.go deleted file mode 100644 index 49b5a04..0000000 --- a/src/lib/metric_test.go +++ /dev/null @@ -1,41 +0,0 @@ -package lib - -import ( - "context" - "testing" -) - -func TestRunMetricSuccess(t *testing.T) { - result := RunMetric(context.Background(), "echo hello", t.TempDir()) - if result.ExitCode != 0 { - t.Errorf("exit code = %d, want 0", result.ExitCode) - } - if result.Output == "" { - t.Error("output should not be empty") - } -} - -func TestRunMetricFailure(t *testing.T) { - result := RunMetric(context.Background(), "exit 1", t.TempDir()) - if result.ExitCode != 1 { - t.Errorf("exit code = %d, want 1", result.ExitCode) - } -} - -func TestRunMetricTruncation(t *testing.T) { - // Generate output larger than 4096 bytes using portable printf - result := RunMetric(context.Background(), "printf '%5000s' ' ' | tr ' ' 'a'", t.TempDir()) - maxExpected := 4096 + len("\n... (truncated)") - if len(result.Output) > maxExpected { - t.Errorf("output should be truncated to ~%d, got %d bytes", maxExpected, len(result.Output)) - } -} - -func TestRunMetricCancellation(t *testing.T) { - ctx, cancel := context.WithCancel(context.Background()) - cancel() - result := RunMetric(ctx, "sleep 10", t.TempDir()) - if result.ExitCode == 0 { - t.Error("cancelled command should have non-zero exit code") - } -} diff --git a/src/lib/similarity.go b/src/lib/similarity.go deleted file mode 100644 index 4a10a89..0000000 --- a/src/lib/similarity.go +++ /dev/null @@ -1,43 +0,0 @@ -package lib - -// Similarity computes the ratio of matching characters between two strings -// using a simple bigram overlap approach. Returns a value between 0.0 and 1.0. -// Used to detect "Groundhog Day" patterns where consecutive metric outputs -// are nearly identical, indicating the agent is stuck in a loop. -func Similarity(a, b string) float64 { - if a == b { - return 1.0 - } - // Short strings can't produce bigrams; fall back to exact match (handled above) - if len(a) < 2 || len(b) < 2 { - return 0.0 - } - - bigramsA := bigrams(a) - bigramsB := bigrams(b) - - var matches int - for bg, countA := range bigramsA { - if countB, ok := bigramsB[bg]; ok { - if countA < countB { - matches += countA - } else { - matches += countB - } - } - } - - total := len(a) - 1 + len(b) - 1 - if total == 0 { - return 0.0 - } - return 2.0 * float64(matches) / float64(total) -} - -func bigrams(s string) map[string]int { - m := make(map[string]int, len(s)-1) - for i := 0; i < len(s)-1; i++ { - m[s[i:i+2]]++ - } - return m -} diff --git a/src/lib/similarity_test.go b/src/lib/similarity_test.go deleted file mode 100644 index 01c56e8..0000000 --- a/src/lib/similarity_test.go +++ /dev/null @@ -1,62 +0,0 @@ -package lib - -import ( - "strings" - "testing" -) - -func TestSimilarity_Identical(t *testing.T) { - if got := Similarity("hello world", "hello world"); got != 1.0 { - t.Errorf("identical strings: got %f, want 1.0", got) - } -} - -func TestSimilarity_CompletelyDifferent(t *testing.T) { - got := Similarity("aaaaaa", "zzzzzz") - if got > 0.01 { - t.Errorf("completely different: got %f, want ~0.0", got) - } -} - -func TestSimilarity_Empty(t *testing.T) { - if got := Similarity("", ""); got != 1.0 { - t.Errorf("both empty: got %f, want 1.0", got) - } - if got := Similarity("hello", ""); got != 0.0 { - t.Errorf("one empty: got %f, want 0.0", got) - } -} - -func TestSimilarity_HighOverlap(t *testing.T) { - a := "FAIL: 3 errors found in parser.go" - b := "FAIL: 3 errors found in parser.go " - got := Similarity(a, b) - if got < 0.90 { - t.Errorf("high overlap: got %f, want >= 0.90", got) - } -} - -func TestSimilarity_ModerateOverlap(t *testing.T) { - a := "FAIL: 3 errors found in parser.go" - b := "FAIL: 5 errors found in handler.go" - got := Similarity(a, b) - if got < 0.4 || got > 0.9 { - t.Errorf("moderate overlap: got %f, want between 0.4 and 0.9", got) - } -} - -func TestSimilarity_LongIdenticalOutputs(t *testing.T) { - long := strings.Repeat("test output line\n", 100) - if got := Similarity(long, long); got != 1.0 { - t.Errorf("long identical: got %f, want 1.0", got) - } -} - -func TestSimilarity_Short(t *testing.T) { - if got := Similarity("a", "a"); got != 1.0 { - t.Errorf("single char identical: got %f, want 1.0", got) - } - if got := Similarity("a", "b"); got != 0.0 { - t.Errorf("single char different: got %f, want 0.0", got) - } -} diff --git a/src/lib/state.go b/src/lib/state.go index 74df0d0..380efcc 100644 --- a/src/lib/state.go +++ b/src/lib/state.go @@ -6,7 +6,6 @@ import ( "fmt" "os" "path/filepath" - "strings" "time" ) @@ -27,89 +26,3 @@ func EnsureSessionDir(repoRoot, sessionID string) error { dir := SessionDir(repoRoot, sessionID) return os.MkdirAll(dir, 0o700) } - -func HandoffPath(repoRoot, sessionID string) string { - return filepath.Join(SessionDir(repoRoot, sessionID), "handoff.md") -} - -func WriteHandoff(repoRoot string, session *Session, steps []Step, baseline MetricResult) error { - dir := SessionDir(repoRoot, session.ID) - if err := os.MkdirAll(dir, 0o700); err != nil { - return err - } - - var totalCost float64 - var consecutiveFailures int - for i := len(steps) - 1; i >= 0; i-- { - totalCost += steps[i].CostUSD - if !steps[i].Kept { - consecutiveFailures++ - } else { - break - } - } - - remaining := session.BudgetUSD - totalCost - lastIter := len(steps) - - var history strings.Builder - history.WriteString("| # | Status | Metric Exit | Summary |\n") - history.WriteString("|---|--------|-------------|----------|\n") - for _, s := range steps { - status := "kept" - if !s.Kept { - status = "reverted" - } - summary := s.ChangeSummary - if len(summary) > 60 { - summary = summary[:60] + "..." - } - fmt.Fprintf(&history, "| %d | %s | %d | %s |\n", s.Iteration, status, s.MetricExitCode, summary) - } - - lastMetric := "N/A" - if len(steps) > 0 { - last := steps[len(steps)-1] - output := last.MetricOutput - if len(output) > 200 { - output = output[:200] + "..." - } - lastMetric = fmt.Sprintf("exit %d — %s", last.MetricExitCode, output) - } else { - output := baseline.Output - if len(output) > 200 { - output = output[:200] + "..." - } - lastMetric = fmt.Sprintf("exit %d — %s", baseline.ExitCode, output) - } - - content := fmt.Sprintf(`# Devkit Improve — Session Handoff -Session: %s -Iteration: %d of %d -Target: %s -Objective: %s -Last metric: %s -Consecutive failures: %d -Remaining budget: $%.2f - -## Iteration History -%s -## Instructions -You are iteration %d. Make ONE focused change to %s that moves toward: %s. -Do not repeat approaches from reverted iterations above. -Run %s to verify your change before finishing. -`, - session.ID, - lastIter+1, session.MaxIterations, - session.Target, - session.Objective, - lastMetric, - consecutiveFailures, - remaining, - history.String(), - lastIter+1, session.Target, session.Objective, - "`"+session.Metric+"`", - ) - - return os.WriteFile(HandoffPath(repoRoot, session.ID), []byte(content), 0o644) -} diff --git a/src/lib/state_json.go b/src/lib/state_json.go new file mode 100644 index 0000000..ca55754 --- /dev/null +++ b/src/lib/state_json.go @@ -0,0 +1,77 @@ +package lib + +import ( + "encoding/json" + "fmt" + "os" + "path/filepath" + "time" +) + +// SessionState is the hot-path state file read by hooks on every tool call. +type SessionState struct { + ID string `json:"id"` + Workflow string `json:"workflow"` + Input string `json:"input"` + CurrentStep string `json:"current_step"` + CurrentIndex int `json:"current_index"` + TotalSteps int `json:"total_steps"` + StepType string `json:"step_type"` // "prompt" | "command" | "parallel" + Enforce string `json:"enforce"` + Branch bool `json:"branch"` + BudgetUSD float64 `json:"budget_usd"` + SpentUSD float64 `json:"spent_usd"` + StartedAt time.Time `json:"started_at"` + Outputs map[string]string `json:"outputs"` + Status string `json:"status"` // "running" | "done" | "failed" + LoopIteration int `json:"loop_iteration,omitempty"` // current loop count for loop steps + LoopMax int `json:"loop_max,omitempty"` // max iterations for current loop +} + +// SessionJSONPath returns the path to the hot-state session file. +func SessionJSONPath(dataDir string) string { + return filepath.Join(dataDir, "session.json") +} + +// WriteSessionJSON atomically writes session state to the hot-path JSON file. +func WriteSessionJSON(dataDir string, state *SessionState) error { + if err := os.MkdirAll(dataDir, 0o755); err != nil { + return fmt.Errorf("create data dir: %w", err) + } + data, err := json.MarshalIndent(state, "", " ") + if err != nil { + return fmt.Errorf("marshal session: %w", err) + } + path := SessionJSONPath(dataDir) + tmp := path + ".tmp" + if err := os.WriteFile(tmp, data, 0o644); err != nil { + return fmt.Errorf("write session tmp: %w", err) + } + return os.Rename(tmp, path) +} + +// ReadSessionJSON reads the hot-path session state. Returns nil if no session file exists. +func ReadSessionJSON(dataDir string) (*SessionState, error) { + path := SessionJSONPath(dataDir) + data, err := os.ReadFile(path) + if os.IsNotExist(err) { + return nil, nil + } + if err != nil { + return nil, fmt.Errorf("read session: %w", err) + } + var state SessionState + if err := json.Unmarshal(data, &state); err != nil { + return nil, fmt.Errorf("parse session: %w", err) + } + return &state, nil +} + +// ClearSessionJSON removes the hot-path session file. +func ClearSessionJSON(dataDir string) error { + path := SessionJSONPath(dataDir) + if err := os.Remove(path); err != nil && !os.IsNotExist(err) { + return fmt.Errorf("clear session: %w", err) + } + return nil +} diff --git a/src/lib/state_test.go b/src/lib/state_test.go index af3cbed..29f6d19 100644 --- a/src/lib/state_test.go +++ b/src/lib/state_test.go @@ -2,8 +2,6 @@ package lib import ( "os" - "path/filepath" - "strings" "testing" ) @@ -45,54 +43,38 @@ func TestEnsureSessionDir(t *testing.T) { } } -func TestWriteHandoff(t *testing.T) { - root := t.TempDir() - session := &Session{ - ID: "hand12345678", - Workflow: "improve", - Target: "src/", - Objective: "fix all tests", - Metric: "go test ./...", - MaxIterations: 10, - BudgetUSD: 5.00, +func TestSessionJSON(t *testing.T) { + dir := t.TempDir() + state := &SessionState{ + ID: "abc123", + Workflow: "research", + CurrentStep: "clarify", + StepType: "prompt", + Enforce: "hard", + Status: "running", + Outputs: map[string]string{}, } - steps := []Step{ - {Iteration: 1, Kept: true, MetricExitCode: 0, CostUSD: 0.05, ChangeSummary: "fixed auth"}, - {Iteration: 2, Kept: false, MetricExitCode: 1, CostUSD: 0.03, ChangeSummary: "broke tests"}, + if err := WriteSessionJSON(dir, state); err != nil { + t.Fatalf("write: %v", err) } - baseline := MetricResult{ExitCode: 1, Output: "3 tests failed"} - - if err := WriteHandoff(root, session, steps, baseline); err != nil { - t.Fatalf("write handoff: %v", err) - } - - path := HandoffPath(root, session.ID) - data, err := os.ReadFile(path) + got, err := ReadSessionJSON(dir) if err != nil { - t.Fatalf("read handoff: %v", err) - } - - content := string(data) - if !strings.Contains(content, "Iteration: 3 of 10") { - t.Error("handoff should show next iteration as 3") + t.Fatalf("read: %v", err) } - if !strings.Contains(content, "fix all tests") { - t.Error("handoff should contain objective") + if got.ID != "abc123" || got.CurrentStep != "clarify" { + t.Errorf("got %+v", got) } - if !strings.Contains(content, "fixed auth") { - t.Error("handoff should contain iteration history") + + if err := ClearSessionJSON(dir); err != nil { + t.Fatalf("clear: %v", err) } - if !strings.Contains(content, "$4.92") { - t.Errorf("handoff should show remaining budget, got:\n%s", content) + got, err = ReadSessionJSON(dir) + if err != nil { + t.Fatalf("read after clear: %v", err) } -} - -func TestHandoffPath(t *testing.T) { - path := HandoffPath("/repo", "abc123def456") - expected := filepath.Join("/repo", ".devkit", "sessions", "abc123def456", "handoff.md") - if path != expected { - t.Errorf("path = %s, want %s", path, expected) + if got != nil { + t.Error("expected nil after clear") } } diff --git a/src/loops/bugfix.go b/src/loops/bugfix.go deleted file mode 100644 index dcff533..0000000 --- a/src/loops/bugfix.go +++ /dev/null @@ -1,173 +0,0 @@ -package loops - -import ( - "context" - "fmt" - - "github.com/5uck1ess/devkit/lib" - "github.com/5uck1ess/devkit/runners" -) - -type BugfixConfig struct { - Description string - TestCmd string - RepoRoot string - BudgetUSD float64 -} - -type BugfixResult struct { - Session *lib.Session - Steps []lib.Step -} - -func RunBugfix(ctx context.Context, db *lib.DB, runner runners.Runner, git *lib.Git, cfg BugfixConfig) (*BugfixResult, error) { - session := &lib.Session{ - ID: lib.NewSessionID(), - Workflow: "bugfix", - Metric: cfg.TestCmd, - Prompt: cfg.Description, - Status: "running", - BudgetUSD: cfg.BudgetUSD, - } - if err := db.CreateSession(session); err != nil { - return nil, fmt.Errorf("create session: %w", err) - } - if err := lib.EnsureSessionDir(cfg.RepoRoot, session.ID); err != nil { - return nil, fmt.Errorf("create session directory: %w", err) - } - - branchName := fmt.Sprintf("bugfix/%s", session.ID) - if err := git.CreateBranch(branchName); err != nil { - return nil, fmt.Errorf("create branch: %w", err) - } - fmt.Printf("Bugfix session %s on branch %s\n\n", session.ID, branchName) - - var spentUSD float64 - checkBudget := func() bool { - return cfg.BudgetUSD > 0 && spentUSD >= cfg.BudgetUSD - } - opts := runners.RunOpts{ - WorkDir: cfg.RepoRoot, - AllowedTools: "Bash,Read,Edit,Write,Grep,Glob", - MaxTurns: 25, - } - - // Step 1: Diagnose - fmt.Println("--- Step 1: Diagnose ---") - diagStep := &lib.Step{SessionID: session.ID, Iteration: 1, Status: "running", AgentName: runner.Name()} - db.CreateStep(diagStep) - - diagPrompt := fmt.Sprintf( - `You are diagnosing a bug. Investigate the codebase to find the root cause. - -Bug: %s - -Read the relevant code. Identify the exact root cause. Report: -1. Root cause (what's wrong and why) -2. The specific file(s) and line(s) -3. Your proposed fix (describe, don't implement yet)`, cfg.Description) - - diagResult, err := runner.Run(ctx, diagPrompt, opts) - if err != nil { - diagStep.Status = "failed" - diagStep.ChangeSummary = err.Error() - db.UpdateStep(diagStep) - db.UpdateSessionStatus(session.ID, "failed") - return nil, fmt.Errorf("diagnose step failed: %w", err) - } - spentUSD += diagResult.CostUSD - diagStep.Status = "kept" - diagStep.Kept = true - diagStep.CostUSD = diagResult.CostUSD - diagStep.ChangeSummary = truncate(diagResult.Output, 200) - db.UpdateStep(diagStep) - fmt.Printf(" Diagnosis complete ($%.4f)\n\n", diagResult.CostUSD) - - if checkBudget() { - fmt.Printf(" Budget exhausted ($%.2f of $%.2f) — stopping after diagnosis\n", spentUSD, cfg.BudgetUSD) - db.UpdateSessionStatus(session.ID, "failed") - allSteps, _ := db.GetSteps(session.ID) - return &BugfixResult{Session: session, Steps: allSteps}, nil - } - - // Step 2: Fix - fmt.Println("--- Step 2: Fix ---") - fixStep := &lib.Step{SessionID: session.ID, Iteration: 2, Status: "running", AgentName: runner.Name()} - db.CreateStep(fixStep) - - fixResult, err := runner.Run(ctx, fmt.Sprintf( - `You are fixing a bug. Apply the fix based on this diagnosis. - -Bug: %s - -Diagnosis: -%s - -Make the minimal change needed to fix the bug. Do not refactor unrelated code.`, cfg.Description, diagResult.Output), opts) - if err != nil { - git.RevertAll() - fixStep.Status = "failed" - fixStep.ChangeSummary = err.Error() - db.UpdateStep(fixStep) - db.UpdateSessionStatus(session.ID, "failed") - return nil, fmt.Errorf("fix step failed: %w", err) - } - spentUSD += fixResult.CostUSD - summary, _ := git.DiffStat() - git.CommitAll(fmt.Sprintf("bugfix(%s): fix", session.ID)) - fixStep.Status = "kept" - fixStep.Kept = true - fixStep.CostUSD = fixResult.CostUSD - fixStep.ChangeSummary = summary - db.UpdateStep(fixStep) - fmt.Printf(" Fix applied ($%.4f)\n\n", fixResult.CostUSD) - - // Step 3: Verify - if cfg.TestCmd != "" && !checkBudget() { - fmt.Println("--- Step 3: Verify ---") - testMetric := lib.RunMetric(ctx, cfg.TestCmd, cfg.RepoRoot) - if testMetric.ExitCode == 0 { - fmt.Println(" Tests passing — fix verified") - } else { - fmt.Println(" Tests still failing, attempting repair...") - repairStep := &lib.Step{SessionID: session.ID, Iteration: 3, Status: "running", AgentName: runner.Name()} - db.CreateStep(repairStep) - - repairResult, err := runner.Run(ctx, fmt.Sprintf( - `The fix was applied but tests are still failing. Adjust the fix. - -Bug: %s -Test command: %s -Test output: -%s - -Fix the remaining failures.`, cfg.Description, cfg.TestCmd, testMetric.Output), opts) - if err == nil { - spentUSD += repairResult.CostUSD - verifyMetric := lib.RunMetric(ctx, cfg.TestCmd, cfg.RepoRoot) - if verifyMetric.ExitCode == 0 { - git.CommitAll(fmt.Sprintf("bugfix(%s): repair", session.ID)) - repairStep.Status = "kept" - repairStep.Kept = true - repairStep.CostUSD = repairResult.CostUSD - fmt.Println(" Repair successful — tests passing") - } else { - git.RevertAll() - repairStep.Status = "reverted" - repairStep.ChangeSummary = "tests still failing after repair" - fmt.Println(" Repair failed — reverted") - } - } else { - repairStep.Status = "failed" - repairStep.ChangeSummary = err.Error() - } - db.UpdateStep(repairStep) - } - } - - db.UpdateSessionStatus(session.ID, "done") - allSteps, _ := db.GetSteps(session.ID) - lib.WriteReport(cfg.RepoRoot, session, allSteps, "completed") - - return &BugfixResult{Session: session, Steps: allSteps}, nil -} diff --git a/src/loops/dispatch.go b/src/loops/dispatch.go deleted file mode 100644 index ac0ad4e..0000000 --- a/src/loops/dispatch.go +++ /dev/null @@ -1,90 +0,0 @@ -package loops - -import ( - "context" - "fmt" - "sync" - - "github.com/5uck1ess/devkit/lib" - "github.com/5uck1ess/devkit/runners" -) - -type DispatchConfig struct { - Prompt string - Agents []string - RepoRoot string -} - -type DispatchResult struct { - Session *lib.Session - Results []AgentResult -} - -func RunDispatch(ctx context.Context, db *lib.DB, available []runners.Runner, cfg DispatchConfig) (*DispatchResult, error) { - selected := filterRunners(available, cfg.Agents) - if len(selected) == 0 { - return nil, fmt.Errorf("no agents available — need at least claude CLI installed") - } - - session := &lib.Session{ - ID: lib.NewSessionID(), - Workflow: "dispatch", - Prompt: cfg.Prompt, - Status: "running", - } - if err := db.CreateSession(session); err != nil { - return nil, fmt.Errorf("create session: %w", err) - } - - fmt.Printf("Dispatch session %s — sending to %d agent(s)\n", session.ID, len(selected)) - - results := make([]AgentResult, len(selected)) - var wg sync.WaitGroup - - for i, r := range selected { - wg.Add(1) - go func(idx int, runner runners.Runner) { - defer wg.Done() - fmt.Printf(" [%s] running...\n", runner.Name()) - - res, err := runner.Run(ctx, cfg.Prompt, runners.RunOpts{ - WorkDir: cfg.RepoRoot, - MaxTurns: 15, - }) - results[idx] = AgentResult{ - Agent: runner.Name(), - Output: res.Output, - Error: err, - Cost: res.CostUSD, - } - - step := &lib.Step{ - SessionID: session.ID, - Iteration: idx + 1, - AgentName: runner.Name(), - Status: "done", - ChangeSummary: truncate(res.Output, 200), - CostUSD: res.CostUSD, - } - if err != nil { - step.Status = "failed" - step.ChangeSummary = err.Error() - } - if dbErr := db.CreateStep(step); dbErr != nil { - fmt.Printf(" [%s] warning: failed to persist step: %s\n", runner.Name(), dbErr) - } - if dbErr := db.UpdateStep(step); dbErr != nil { - fmt.Printf(" [%s] warning: failed to update step: %s\n", runner.Name(), dbErr) - } - - fmt.Printf(" [%s] done ($%.4f)\n", runner.Name(), res.CostUSD) - }(i, r) - } - wg.Wait() - - if err := db.UpdateSessionStatus(session.ID, "done"); err != nil { - fmt.Printf(" warning: failed to update session status: %s\n", err) - } - - return &DispatchResult{Session: session, Results: results}, nil -} diff --git a/src/loops/feature.go b/src/loops/feature.go deleted file mode 100644 index 6f26f47..0000000 --- a/src/loops/feature.go +++ /dev/null @@ -1,212 +0,0 @@ -package loops - -import ( - "context" - "fmt" - - "github.com/5uck1ess/devkit/lib" - "github.com/5uck1ess/devkit/runners" -) - -type FeatureConfig struct { - Description string - Target string - TestCmd string - LintCmd string - RepoRoot string - BudgetUSD float64 -} - -type FeatureResult struct { - Session *lib.Session - Steps []lib.Step -} - -func RunFeature(ctx context.Context, db *lib.DB, runner runners.Runner, git *lib.Git, cfg FeatureConfig) (*FeatureResult, error) { - session := &lib.Session{ - ID: lib.NewSessionID(), - Workflow: "feature", - Target: cfg.Target, - Prompt: cfg.Description, - Status: "running", - BudgetUSD: cfg.BudgetUSD, - } - if err := db.CreateSession(session); err != nil { - return nil, fmt.Errorf("create session: %w", err) - } - if err := lib.EnsureSessionDir(cfg.RepoRoot, session.ID); err != nil { - return nil, fmt.Errorf("create session directory: %w", err) - } - - branchName := fmt.Sprintf("feature/%s", session.ID) - if err := git.CreateBranch(branchName); err != nil { - return nil, fmt.Errorf("create branch: %w", err) - } - fmt.Printf("Feature session %s on branch %s\n\n", session.ID, branchName) - - var spentUSD float64 - checkBudget := func() bool { - return cfg.BudgetUSD > 0 && spentUSD >= cfg.BudgetUSD - } - opts := runners.RunOpts{ - WorkDir: cfg.RepoRoot, - AllowedTools: "Bash,Read,Edit,Write,Grep,Glob", - MaxTurns: 30, - } - - // Step 1: Plan - fmt.Println("--- Step 1: Plan ---") - planStep := &lib.Step{SessionID: session.ID, Iteration: 1, Status: "running", AgentName: runner.Name()} - db.CreateStep(planStep) - - planResult, err := runner.Run(ctx, fmt.Sprintf( - `You are planning a feature. Think through the design, then produce a numbered implementation plan. -Each item should be a single, testable change. Order by dependency. - -Feature: %s -Target: %s - -Output ONLY the plan as a numbered list. Do not write any code yet.`, cfg.Description, cfg.Target), opts) - if err != nil { - planStep.Status = "failed" - planStep.ChangeSummary = err.Error() - db.UpdateStep(planStep) - db.UpdateSessionStatus(session.ID, "failed") - return nil, fmt.Errorf("plan step failed: %w", err) - } - spentUSD += planResult.CostUSD - planStep.Status = "kept" - planStep.Kept = true - planStep.CostUSD = planResult.CostUSD - planStep.ChangeSummary = truncate(planResult.Output, 200) - db.UpdateStep(planStep) - fmt.Printf(" Plan complete ($%.4f)\n\n", planResult.CostUSD) - - if checkBudget() { - fmt.Printf(" Budget exhausted ($%.2f of $%.2f) — stopping after plan\n", spentUSD, cfg.BudgetUSD) - db.UpdateSessionStatus(session.ID, "failed") - allSteps, _ := db.GetSteps(session.ID) - return &FeatureResult{Session: session, Steps: allSteps}, nil - } - - // Step 2: Implement - fmt.Println("--- Step 2: Implement ---") - implStep := &lib.Step{SessionID: session.ID, Iteration: 2, Status: "running", AgentName: runner.Name()} - db.CreateStep(implStep) - - implResult, err := runner.Run(ctx, fmt.Sprintf( - `You are implementing a feature. Follow this plan exactly, implementing each item in order. - -Feature: %s -Target: %s - -Plan: -%s - -Write the code. Make all necessary changes. Do not skip any plan items.`, cfg.Description, cfg.Target, planResult.Output), opts) - if err != nil { - git.RevertAll() - implStep.Status = "failed" - implStep.ChangeSummary = err.Error() - db.UpdateStep(implStep) - db.UpdateSessionStatus(session.ID, "failed") - return nil, fmt.Errorf("implement step failed: %w", err) - } - spentUSD += implResult.CostUSD - implStep.CostUSD = implResult.CostUSD - fmt.Printf(" Implemented ($%.4f)\n\n", implResult.CostUSD) - - // Step 3: Test — verify BEFORE committing - testsPass := cfg.TestCmd == "" - if cfg.TestCmd != "" && !checkBudget() { - fmt.Println("--- Step 3: Test ---") - for attempt := 1; attempt <= 3; attempt++ { - if ctx.Err() != nil { - break - } - testMetric := lib.RunMetric(ctx, cfg.TestCmd, cfg.RepoRoot) - if testMetric.ExitCode == 0 { - fmt.Printf(" Tests passing (attempt %d)\n\n", attempt) - testsPass = true - break - } - - fmt.Printf(" Tests failing (attempt %d), fixing...\n", attempt) - testStep := &lib.Step{SessionID: session.ID, Iteration: 2 + attempt, Status: "running", AgentName: runner.Name()} - db.CreateStep(testStep) - - fixResult, err := runner.Run(ctx, fmt.Sprintf( - `Tests are failing. Fix the failures without changing test expectations. - -Test command: %s -Test output: -%s - -Fix the code so tests pass.`, cfg.TestCmd, testMetric.Output), opts) - if err != nil { - testStep.Status = "failed" - testStep.ChangeSummary = err.Error() - db.UpdateStep(testStep) - continue - } - spentUSD += fixResult.CostUSD - git.CommitAll(fmt.Sprintf("feature(%s): fix tests attempt %d", session.ID, attempt)) - testStep.Status = "kept" - testStep.Kept = true - testStep.CostUSD = fixResult.CostUSD - db.UpdateStep(testStep) - } - } - - // Commit implementation only after tests pass (or no test command) - if testsPass { - summary, _ := git.DiffStat() - git.CommitAll(fmt.Sprintf("feature(%s): implement", session.ID)) - implStep.Status = "kept" - implStep.Kept = true - implStep.ChangeSummary = summary - } else { - git.RevertAll() - implStep.Status = "reverted" - implStep.ChangeSummary = "tests never passed — reverted" - fmt.Println(" Tests never passed — implementation reverted") - } - db.UpdateStep(implStep) - - // Step 4: Lint (if lint command provided) - if cfg.LintCmd != "" && !checkBudget() { - fmt.Println("--- Step 4: Lint ---") - lintMetric := lib.RunMetric(ctx, cfg.LintCmd, cfg.RepoRoot) - if lintMetric.ExitCode != 0 { - nextIter, _ := db.LastIteration(session.ID) - lintStep := &lib.Step{SessionID: session.ID, Iteration: nextIter + 1, Status: "running", AgentName: runner.Name()} - db.CreateStep(lintStep) - - lintResult, err := runner.Run(ctx, fmt.Sprintf( - `Fix these lint errors without changing code behavior. - -Lint command: %s -Lint output: -%s`, cfg.LintCmd, lintMetric.Output), opts) - if err == nil { - spentUSD += lintResult.CostUSD - git.CommitAll(fmt.Sprintf("feature(%s): fix lint", session.ID)) - lintStep.Status = "kept" - lintStep.Kept = true - lintStep.CostUSD = lintResult.CostUSD - } else { - lintStep.Status = "failed" - lintStep.ChangeSummary = err.Error() - } - db.UpdateStep(lintStep) - } else { - fmt.Println(" Lint clean") - } - } - - db.UpdateSessionStatus(session.ID, "done") - allSteps, _ := db.GetSteps(session.ID) - lib.WriteReport(cfg.RepoRoot, session, allSteps, "completed") - - return &FeatureResult{Session: session, Steps: allSteps}, nil -} diff --git a/src/loops/improve.go b/src/loops/improve.go deleted file mode 100644 index a9994d2..0000000 --- a/src/loops/improve.go +++ /dev/null @@ -1,236 +0,0 @@ -package loops - -import ( - "context" - "fmt" - - "github.com/5uck1ess/devkit/lib" - "github.com/5uck1ess/devkit/runners" -) - -type ImproveConfig struct { - Target string - Metric string - Objective string - MaxIterations int - BudgetUSD float64 - MaxFailures int - RepoRoot string -} - -type ImproveResult struct { - Session *lib.Session - Steps []lib.Step - Baseline lib.MetricResult - StopReason string -} - -func RunImproveLoop(ctx context.Context, db *lib.DB, runner runners.Runner, git *lib.Git, cfg ImproveConfig) (*ImproveResult, error) { - session := &lib.Session{ - ID: lib.NewSessionID(), - Workflow: "improve", - Target: cfg.Target, - Metric: cfg.Metric, - Objective: cfg.Objective, - MaxIterations: cfg.MaxIterations, - BudgetUSD: cfg.BudgetUSD, - Status: "running", - } - if err := db.CreateSession(session); err != nil { - return nil, fmt.Errorf("create session: %w", err) - } - if err := lib.EnsureSessionDir(cfg.RepoRoot, session.ID); err != nil { - return nil, fmt.Errorf("create session directory: %w", err) - } - - branchName := fmt.Sprintf("self-improve/%s", session.ID) - if err := git.CreateBranch(branchName); err != nil { - return nil, fmt.Errorf("create branch %s: %w", branchName, err) - } - - fmt.Printf("Session %s started on branch %s\n", session.ID, branchName) - fmt.Printf("Metric: %s\n", cfg.Metric) - fmt.Printf("Running baseline...\n") - - baseline := lib.RunMetric(ctx, cfg.Metric, cfg.RepoRoot) - fmt.Printf("Baseline: exit %d (%s)\n\n", baseline.ExitCode, baseline.Duration) - - return runIterations(ctx, db, runner, git, cfg, session, baseline, 1) -} - -func ResumeImproveLoop(ctx context.Context, db *lib.DB, runner runners.Runner, git *lib.Git, session *lib.Session, repoRoot string) (*ImproveResult, error) { - lastIter, err := db.LastIteration(session.ID) - if err != nil { - return nil, fmt.Errorf("get last iteration: %w", err) - } - - cfg := ImproveConfig{ - Target: session.Target, - Metric: session.Metric, - Objective: session.Objective, - MaxIterations: session.MaxIterations, - BudgetUSD: session.BudgetUSD, - MaxFailures: 3, - RepoRoot: repoRoot, - } - - fmt.Printf("Resuming session %s from iteration %d\n", session.ID, lastIter+1) - - baseline := lib.RunMetric(ctx, cfg.Metric, repoRoot) - if err := db.UpdateSessionStatus(session.ID, "running"); err != nil { - return nil, err - } - - return runIterations(ctx, db, runner, git, cfg, session, baseline, lastIter+1) -} - -func runIterations(ctx context.Context, db *lib.DB, runner runners.Runner, git *lib.Git, cfg ImproveConfig, session *lib.Session, baseline lib.MetricResult, startIter int) (*ImproveResult, error) { - if cfg.MaxFailures == 0 { - cfg.MaxFailures = 3 - } - - const similarityThreshold = 0.90 - const maxSimilarOutputs = 2 - - var spentUSD float64 - if startIter > 1 { - spent, _ := db.SessionTotalCost(session.ID) - spentUSD = spent - } - - consecutiveFailures := 0 - consecutiveSimilar := 0 - lastMetricOutput := "" - stopReason := "completed" - - for i := startIter; i <= cfg.MaxIterations; i++ { - if ctx.Err() != nil { - stopReason = "interrupted" - break - } - if cfg.BudgetUSD > 0 && spentUSD >= cfg.BudgetUSD { - stopReason = fmt.Sprintf("budget exhausted ($%.2f of $%.2f)", spentUSD, cfg.BudgetUSD) - break - } - if consecutiveFailures >= cfg.MaxFailures { - stopReason = fmt.Sprintf("stuck — %d consecutive failures", consecutiveFailures) - break - } - if consecutiveSimilar >= maxSimilarOutputs { - stopReason = fmt.Sprintf("stuck — %d consecutive similar outputs (>%.0f%% match), agent is repeating itself", consecutiveSimilar, similarityThreshold*100) - break - } - - steps, _ := db.GetSteps(session.ID) - if err := lib.WriteHandoff(cfg.RepoRoot, session, steps, baseline); err != nil { - return nil, fmt.Errorf("write handoff: %w", err) - } - - step := &lib.Step{ - SessionID: session.ID, - Iteration: i, - Status: "running", - AgentName: runner.Name(), - } - if err := db.CreateStep(step); err != nil { - return nil, fmt.Errorf("create step: %w", err) - } - - fmt.Printf("--- Iteration %d/%d ---\n", i, cfg.MaxIterations) - - prompt := buildImprovePrompt(cfg) - result, err := runner.Run(ctx, prompt, runners.RunOpts{ - WorkDir: cfg.RepoRoot, - AllowedTools: "Bash,Read,Edit,Write,Grep,Glob", - AppendSystemPromptFile: lib.HandoffPath(cfg.RepoRoot, session.ID), - MaxTurns: 25, - }) - if err != nil { - // Revert any partial changes the agent made before failing - if revertErr := git.RevertAll(); revertErr != nil { - fmt.Printf(" Warning: revert after agent error failed: %s\n", revertErr) - } - step.Status = "failed" - step.ChangeSummary = err.Error() - db.UpdateStep(step) - consecutiveFailures++ - fmt.Printf(" Agent error: %s\n", err) - continue - } - - spentUSD += result.CostUSD - step.TokensUsed = result.TokensIn + result.TokensOut - step.CostUSD = result.CostUSD - - metricResult := lib.RunMetric(ctx, cfg.Metric, cfg.RepoRoot) - step.MetricOutput = metricResult.Output - step.MetricExitCode = metricResult.ExitCode - - if metricResult.ExitCode == 0 { - summary, _ := git.DiffStat() - if err := git.CommitAll(fmt.Sprintf("self-improve: iteration %d — passed", i)); err != nil { - fmt.Printf(" Warning: commit failed: %s\n", err) - } - step.Status = "kept" - step.Kept = true - step.ChangeSummary = summary - consecutiveFailures = 0 - consecutiveSimilar = 0 - lastMetricOutput = metricResult.Output - fmt.Printf(" KEPT (exit 0) — $%.4f\n", result.CostUSD) - } else { - if revertErr := git.RevertAll(); revertErr != nil { - fmt.Printf(" Warning: revert failed: %s\n", revertErr) - } - step.Status = "reverted" - step.Kept = false - step.ChangeSummary = fmt.Sprintf("metric exit %d", metricResult.ExitCode) - consecutiveFailures++ - - // Detect Groundhog Day: agent keeps producing near-identical failing output - if lastMetricOutput != "" && lib.Similarity(lastMetricOutput, metricResult.Output) >= similarityThreshold { - consecutiveSimilar++ - fmt.Printf(" REVERTED (exit %d, similar output %d/%d) — $%.4f\n", metricResult.ExitCode, consecutiveSimilar, maxSimilarOutputs, result.CostUSD) - } else { - consecutiveSimilar = 0 - fmt.Printf(" REVERTED (exit %d) — $%.4f\n", metricResult.ExitCode, result.CostUSD) - } - lastMetricOutput = metricResult.Output - } - - db.UpdateStep(step) - } - - status := "done" - if stopReason == "interrupted" { - status = "paused" - } else if stopReason != "completed" { - status = "failed" - } - db.UpdateSessionStatus(session.ID, status) - - allSteps, _ := db.GetSteps(session.ID) - lib.WriteReport(cfg.RepoRoot, session, allSteps, stopReason) - - return &ImproveResult{ - Session: session, - Steps: allSteps, - Baseline: baseline, - StopReason: stopReason, - }, nil -} - -func buildImprovePrompt(cfg ImproveConfig) string { - return fmt.Sprintf( - `You are an AI code improver. Your task: - -Target: %s -Objective: %s -Metric command: %s - -Make ONE focused change that moves toward the objective. Do not make multiple unrelated changes. -Read the handoff file in your system prompt for iteration history and what to avoid. -After making your change, run the metric command to verify it passes.`, - cfg.Target, cfg.Objective, cfg.Metric, - ) -} diff --git a/src/loops/loops_test.go b/src/loops/loops_test.go deleted file mode 100644 index fe9563d..0000000 --- a/src/loops/loops_test.go +++ /dev/null @@ -1,1479 +0,0 @@ -package loops - -import ( - "context" - "fmt" - "os" - "os/exec" - "path/filepath" - "strings" - "testing" - - "github.com/5uck1ess/devkit/lib" - "github.com/5uck1ess/devkit/runners" -) - -// --------------------------------------------------------------------------- -// test helpers -// --------------------------------------------------------------------------- - -func tempDB(t *testing.T) *lib.DB { - t.Helper() - dir := t.TempDir() - db, err := lib.OpenDB(filepath.Join(dir, ".devkit", "devkit.db")) - if err != nil { - t.Fatalf("open db: %v", err) - } - t.Cleanup(func() { db.Close() }) - return db -} - -// initGitRepo creates a temp dir with a git repo, an initial commit on main, -// and returns the repo root path and a *lib.Git. -func initGitRepo(t *testing.T) (string, *lib.Git) { - t.Helper() - dir := t.TempDir() - cmds := [][]string{ - {"git", "init", "-b", "main"}, - {"git", "config", "user.email", "test@test.com"}, - {"git", "config", "user.name", "Test"}, - } - for _, args := range cmds { - cmd := exec.Command(args[0], args[1:]...) - cmd.Dir = dir - if out, err := cmd.CombinedOutput(); err != nil { - t.Fatalf("git setup %v: %s", args, out) - } - } - // write a file and commit so HEAD exists - if err := os.WriteFile(filepath.Join(dir, "README.md"), []byte("# test\n"), 0o644); err != nil { - t.Fatal(err) - } - for _, args := range [][]string{ - {"git", "add", "-A"}, - {"git", "commit", "-m", "initial"}, - } { - cmd := exec.Command(args[0], args[1:]...) - cmd.Dir = dir - if out, err := cmd.CombinedOutput(); err != nil { - t.Fatalf("git setup %v: %s", args, out) - } - } - return dir, &lib.Git{Dir: dir} -} - -// mockRunner is a test-local mock implementing runners.Runner. -type mockRunnerT struct { - name string - responses []runners.RunResult - errors []error - callIdx int -} - -func mockRunner(name string, responses []runners.RunResult, errs []error) *mockRunnerT { - return &mockRunnerT{name: name, responses: responses, errors: errs} -} - -func (m *mockRunnerT) Name() string { return m.name } -func (m *mockRunnerT) Available() bool { return true } -func (m *mockRunnerT) CallCount() int { return m.callIdx } - -func (m *mockRunnerT) Run(ctx context.Context, prompt string, opts runners.RunOpts) (runners.RunResult, error) { - idx := m.callIdx - m.callIdx++ - if idx >= len(m.responses) { - return runners.RunResult{Output: "mock exhausted"}, nil - } - var err error - if idx < len(m.errors) { - err = m.errors[idx] - } - return m.responses[idx], err -} - -// successResult returns a RunResult with output and a small cost. -func successResult(output string) runners.RunResult { - return runners.RunResult{Output: output, CostUSD: 0.01} -} - -// --------------------------------------------------------------------------- -// truncate -// --------------------------------------------------------------------------- - -func TestTruncate(t *testing.T) { - tests := []struct { - input string - n int - want string - }{ - {"short", 10, "short"}, - {"exactly10!", 10, "exactly10!"}, - {"this is longer than five", 5, "this ..."}, - {"", 5, ""}, - {"abc", 0, "..."}, - } - for _, tc := range tests { - got := truncate(tc.input, tc.n) - if got != tc.want { - t.Errorf("truncate(%q, %d) = %q, want %q", tc.input, tc.n, got, tc.want) - } - } -} - -// --------------------------------------------------------------------------- -// filterRunners -// --------------------------------------------------------------------------- - -func TestFilterRunners_NoFilter(t *testing.T) { - available := []runners.Runner{ - mockRunner("claude", nil, nil), - mockRunner("codex", nil, nil), - } - got := filterRunners(available, nil) - if len(got) != 2 { - t.Errorf("no filter: got %d runners, want 2", len(got)) - } -} - -func TestFilterRunners_EmptyNames(t *testing.T) { - available := []runners.Runner{ - mockRunner("claude", nil, nil), - } - got := filterRunners(available, []string{}) - if len(got) != 1 { - t.Errorf("empty names: got %d runners, want 1", len(got)) - } -} - -func TestFilterRunners_SelectSpecific(t *testing.T) { - available := []runners.Runner{ - mockRunner("claude", nil, nil), - mockRunner("codex", nil, nil), - mockRunner("gemini", nil, nil), - } - got := filterRunners(available, []string{"claude", "gemini"}) - if len(got) != 2 { - t.Errorf("select 2: got %d runners, want 2", len(got)) - } - names := map[string]bool{} - for _, r := range got { - names[r.Name()] = true - } - if !names["claude"] || !names["gemini"] { - t.Errorf("expected claude and gemini, got %v", names) - } -} - -func TestFilterRunners_NoneMatch(t *testing.T) { - available := []runners.Runner{ - mockRunner("claude", nil, nil), - } - got := filterRunners(available, []string{"nonexistent"}) - if len(got) != 0 { - t.Errorf("none match: got %d runners, want 0", len(got)) - } -} - -func TestFilterRunners_CaseInsensitive(t *testing.T) { - available := []runners.Runner{ - mockRunner("claude", nil, nil), - } - got := filterRunners(available, []string{"Claude"}) - // filterRunners lowercases names, so "Claude" matches "claude" in nameSet - // but runner.Name() returns "claude" which must match the lowered key - if len(got) != 1 { - t.Errorf("case insensitive: got %d runners, want 1", len(got)) - } -} - -// --------------------------------------------------------------------------- -// RunDispatch -// --------------------------------------------------------------------------- - -func TestRunDispatch_Success(t *testing.T) { - db := tempDB(t) - available := []runners.Runner{ - mockRunner("claude", []runners.RunResult{successResult("review done")}, nil), - mockRunner("codex", []runners.RunResult{successResult("codex done")}, nil), - } - - result, err := RunDispatch(context.Background(), db, available, DispatchConfig{ - Prompt: "test prompt", - RepoRoot: t.TempDir(), - }) - if err != nil { - t.Fatalf("RunDispatch: %v", err) - } - if result.Session.Workflow != "dispatch" { - t.Errorf("workflow = %q, want dispatch", result.Session.Workflow) - } - if len(result.Results) != 2 { - t.Fatalf("results = %d, want 2", len(result.Results)) - } - if result.Results[0].Output != "review done" { - t.Errorf("result[0] output = %q", result.Results[0].Output) - } - if result.Results[1].Output != "codex done" { - t.Errorf("result[1] output = %q", result.Results[1].Output) - } -} - -func TestRunDispatch_NoAgents(t *testing.T) { - db := tempDB(t) - _, err := RunDispatch(context.Background(), db, nil, DispatchConfig{ - Prompt: "test", - }) - if err == nil { - t.Fatal("expected error with no agents") - } - if !strings.Contains(err.Error(), "no agents available") { - t.Errorf("error = %q, want 'no agents available'", err) - } -} - -func TestRunDispatch_FilteredToNone(t *testing.T) { - db := tempDB(t) - available := []runners.Runner{ - mockRunner("claude", nil, nil), - } - _, err := RunDispatch(context.Background(), db, available, DispatchConfig{ - Prompt: "test", - Agents: []string{"nonexistent"}, - }) - if err == nil { - t.Fatal("expected error when all agents filtered out") - } -} - -func TestRunDispatch_AgentError(t *testing.T) { - db := tempDB(t) - available := []runners.Runner{ - mockRunner("claude", []runners.RunResult{{Output: ""}}, []error{fmt.Errorf("agent crashed")}), - } - - result, err := RunDispatch(context.Background(), db, available, DispatchConfig{ - Prompt: "test", - RepoRoot: t.TempDir(), - }) - if err != nil { - t.Fatalf("RunDispatch should not return error for agent-level failures: %v", err) - } - if result.Results[0].Error == nil { - t.Error("expected agent error in result") - } -} - -func TestRunDispatch_SessionPersisted(t *testing.T) { - db := tempDB(t) - available := []runners.Runner{ - mockRunner("claude", []runners.RunResult{successResult("ok")}, nil), - } - - result, err := RunDispatch(context.Background(), db, available, DispatchConfig{ - Prompt: "persist test", - RepoRoot: t.TempDir(), - }) - if err != nil { - t.Fatalf("RunDispatch: %v", err) - } - - session, err := db.GetSession(result.Session.ID) - if err != nil { - t.Fatalf("get session: %v", err) - } - if session.Status != "done" { - t.Errorf("session status = %q, want done", session.Status) - } -} - -// --------------------------------------------------------------------------- -// RunReview -// --------------------------------------------------------------------------- - -func TestRunReview_NoDiff(t *testing.T) { - db := tempDB(t) - _, git := initGitRepo(t) - available := []runners.Runner{ - mockRunner("claude", nil, nil), - } - - _, err := RunReview(context.Background(), db, available, git, ReviewConfig{ - RepoRoot: git.Dir, - }) - if err == nil { - t.Fatal("expected error with no diff") - } - if !strings.Contains(err.Error(), "no diff found") { - t.Errorf("error = %q, want 'no diff found'", err) - } -} - -func TestRunReview_NoAgents(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - // Create a diff by making a change on a new branch - makeChange(t, dir, git) - - _, err := RunReview(context.Background(), db, nil, git, ReviewConfig{ - RepoRoot: dir, - }) - if err == nil { - t.Fatal("expected error with no agents") - } - if !strings.Contains(err.Error(), "no agents available") { - t.Errorf("error = %q", err) - } -} - -func TestRunReview_Success(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - makeChange(t, dir, git) - - available := []runners.Runner{ - mockRunner("claude", []runners.RunResult{successResult("looks good")}, nil), - } - - result, err := RunReview(context.Background(), db, available, git, ReviewConfig{ - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunReview: %v", err) - } - if result.Session.Workflow != "review" { - t.Errorf("workflow = %q, want review", result.Session.Workflow) - } - if len(result.Results) != 1 { - t.Fatalf("results = %d, want 1", len(result.Results)) - } - if result.Results[0].Output != "looks good" { - t.Errorf("output = %q", result.Results[0].Output) - } -} - -func TestRunReview_SecurityFlag(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - makeChange(t, dir, git) - - // We verify the security prompt gets passed by checking the runner receives it. - // Since MockRunner doesn't expose prompts, we just verify no error and the session is created. - available := []runners.Runner{ - mockRunner("claude", []runners.RunResult{successResult("no security issues")}, nil), - } - - result, err := RunReview(context.Background(), db, available, git, ReviewConfig{ - RepoRoot: dir, - Security: true, - }) - if err != nil { - t.Fatalf("RunReview with security: %v", err) - } - if result.Session.Workflow != "review" { - t.Errorf("workflow = %q", result.Session.Workflow) - } -} - -func TestRunReview_DiffTruncation(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - // Create a large diff - largeContent := strings.Repeat("x", 40000) - if err := os.WriteFile(filepath.Join(dir, "large.txt"), []byte(largeContent), 0o644); err != nil { - t.Fatal(err) - } - run(t, dir, "git", "checkout", "-b", "test-branch") - run(t, dir, "git", "add", "-A") - run(t, dir, "git", "commit", "-m", "large change") - - available := []runners.Runner{ - mockRunner("claude", []runners.RunResult{successResult("reviewed")}, nil), - } - - result, err := RunReview(context.Background(), db, available, git, ReviewConfig{ - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunReview: %v", err) - } - if len(result.Results) != 1 { - t.Fatalf("results = %d, want 1", len(result.Results)) - } -} - -func TestRunReview_CustomPrompt(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - makeChange(t, dir, git) - - available := []runners.Runner{ - mockRunner("claude", []runners.RunResult{successResult("custom review")}, nil), - } - - result, err := RunReview(context.Background(), db, available, git, ReviewConfig{ - Prompt: "Focus on performance only", - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunReview: %v", err) - } - if result.Session.Prompt != "Focus on performance only" { - t.Errorf("prompt = %q", result.Session.Prompt) - } -} - -func TestRunReview_MultipleAgents(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - makeChange(t, dir, git) - - available := []runners.Runner{ - mockRunner("claude", []runners.RunResult{successResult("claude review")}, nil), - mockRunner("codex", []runners.RunResult{successResult("codex review")}, nil), - } - - result, err := RunReview(context.Background(), db, available, git, ReviewConfig{ - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunReview: %v", err) - } - if len(result.Results) != 2 { - t.Fatalf("results = %d, want 2", len(result.Results)) - } -} - -// --------------------------------------------------------------------------- -// RunBugfix -// --------------------------------------------------------------------------- - -func TestRunBugfix_Success(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("root cause: off by one in loop"), - successResult("applied fix to main.go"), - }, nil) - - result, err := RunBugfix(context.Background(), db, runner, git, BugfixConfig{ - Description: "tests fail with index out of range", - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunBugfix: %v", err) - } - if result.Session.Workflow != "bugfix" { - t.Errorf("workflow = %q, want bugfix", result.Session.Workflow) - } - if runner.CallCount() != 2 { - t.Errorf("runner calls = %d, want 2 (diagnose + fix)", runner.CallCount()) - } -} - -func TestRunBugfix_WithTestVerification(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("diagnosis"), - successResult("fix applied"), - }, nil) - - result, err := RunBugfix(context.Background(), db, runner, git, BugfixConfig{ - Description: "test bug", - TestCmd: "true", // always passes - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunBugfix: %v", err) - } - if result.Session.Workflow != "bugfix" { - t.Errorf("workflow = %q", result.Session.Workflow) - } -} - -func TestRunBugfix_TestFailsThenRepairs(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - // Write a file that the "fix" step might change - counterFile := filepath.Join(dir, "counter.txt") - os.WriteFile(counterFile, []byte("0"), 0o644) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("diagnosis"), - successResult("initial fix"), - successResult("repair fix"), - }, nil) - - // Use a test command that fails first, then passes after a state change - // We use "true" since the runner mock doesn't actually modify files. - // The repair path is exercised when test fails — use "false" as testcmd - // to trigger the repair branch, but the repair will also fail since - // RunMetric with "false" always exits non-zero. - result, err := RunBugfix(context.Background(), db, runner, git, BugfixConfig{ - Description: "test bug", - TestCmd: "false", // always fails - triggers repair path - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunBugfix: %v", err) - } - // Should still complete (repair will be reverted but session finishes) - if result.Session == nil { - t.Fatal("expected session in result") - } -} - -func TestRunBugfix_DiagnoseStepFails(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", - []runners.RunResult{{}}, - []error{fmt.Errorf("agent timeout")}, - ) - - _, err := RunBugfix(context.Background(), db, runner, git, BugfixConfig{ - Description: "bug", - RepoRoot: dir, - }) - if err == nil { - t.Fatal("expected error when diagnose fails") - } - if !strings.Contains(err.Error(), "diagnose step failed") { - t.Errorf("error = %q", err) - } -} - -func TestRunBugfix_FixStepFails(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("diagnosis"), - {}, // fix step response (will be overridden by error) - }, []error{nil, fmt.Errorf("fix failed")}) - - _, err := RunBugfix(context.Background(), db, runner, git, BugfixConfig{ - Description: "bug", - RepoRoot: dir, - }) - if err == nil { - t.Fatal("expected error when fix fails") - } - if !strings.Contains(err.Error(), "fix step failed") { - t.Errorf("error = %q", err) - } -} - -func TestRunBugfix_BudgetExhaustedAfterDiagnosis(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - {Output: "diagnosis", CostUSD: 5.00}, // costs the whole budget - }, nil) - - result, err := RunBugfix(context.Background(), db, runner, git, BugfixConfig{ - Description: "bug", - RepoRoot: dir, - BudgetUSD: 5.00, - }) - if err != nil { - t.Fatalf("RunBugfix: %v", err) - } - // Should stop after diagnosis, only 1 runner call - if runner.CallCount() != 1 { - t.Errorf("runner calls = %d, want 1 (budget stop after diagnose)", runner.CallCount()) - } - if result.Session == nil { - t.Fatal("expected session") - } -} - -func TestRunBugfix_ZeroBudgetMeansUnlimited(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("diagnosis"), - successResult("fix"), - }, nil) - - _, err := RunBugfix(context.Background(), db, runner, git, BugfixConfig{ - Description: "bug", - RepoRoot: dir, - BudgetUSD: 0, // zero means unlimited - }) - if err != nil { - t.Fatalf("RunBugfix: %v", err) - } - if runner.CallCount() != 2 { - t.Errorf("runner calls = %d, want 2 (unlimited budget)", runner.CallCount()) - } -} - -// --------------------------------------------------------------------------- -// RunFeature -// --------------------------------------------------------------------------- - -func TestRunFeature_SuccessNoTests(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("1. Add endpoint\n2. Add handler"), - successResult("implemented both steps"), - }, nil) - - result, err := RunFeature(context.Background(), db, runner, git, FeatureConfig{ - Description: "add user API", - Target: "src/api/", - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunFeature: %v", err) - } - if result.Session.Workflow != "feature" { - t.Errorf("workflow = %q, want feature", result.Session.Workflow) - } - if runner.CallCount() != 2 { - t.Errorf("runner calls = %d, want 2 (plan + implement)", runner.CallCount()) - } -} - -func TestRunFeature_WithPassingTests(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("plan"), - successResult("implemented"), - }, nil) - - result, err := RunFeature(context.Background(), db, runner, git, FeatureConfig{ - Description: "feature", - Target: "src/", - TestCmd: "true", - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunFeature: %v", err) - } - if result.Session.Workflow != "feature" { - t.Errorf("workflow = %q", result.Session.Workflow) - } -} - -func TestRunFeature_TestsFailAllAttempts(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - // plan + implement + 3 fix attempts - runner := mockRunner("claude", []runners.RunResult{ - successResult("plan"), - successResult("implemented"), - successResult("fix attempt 1"), - successResult("fix attempt 2"), - successResult("fix attempt 3"), - }, nil) - - result, err := RunFeature(context.Background(), db, runner, git, FeatureConfig{ - Description: "feature", - Target: "src/", - TestCmd: "false", // always fails - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunFeature: %v", err) - } - // Implementation should be reverted since tests never pass - if result.Session == nil { - t.Fatal("expected session") - } -} - -func TestRunFeature_PlanFails(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{{}}, []error{fmt.Errorf("planning error")}) - - _, err := RunFeature(context.Background(), db, runner, git, FeatureConfig{ - Description: "feature", - Target: "src/", - RepoRoot: dir, - }) - if err == nil { - t.Fatal("expected error when plan fails") - } - if !strings.Contains(err.Error(), "plan step failed") { - t.Errorf("error = %q", err) - } -} - -func TestRunFeature_ImplementFails(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("plan"), - {}, // implement step response (overridden by error) - }, []error{nil, fmt.Errorf("implement crashed")}) - - _, err := RunFeature(context.Background(), db, runner, git, FeatureConfig{ - Description: "feature", - Target: "src/", - RepoRoot: dir, - }) - if err == nil { - t.Fatal("expected error when implement fails") - } - if !strings.Contains(err.Error(), "implement step failed") { - t.Errorf("error = %q", err) - } -} - -func TestRunFeature_BudgetExhaustedAfterPlan(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - {Output: "expensive plan", CostUSD: 10.00}, - }, nil) - - result, err := RunFeature(context.Background(), db, runner, git, FeatureConfig{ - Description: "feature", - Target: "src/", - RepoRoot: dir, - BudgetUSD: 10.00, - }) - if err != nil { - t.Fatalf("RunFeature: %v", err) - } - if runner.CallCount() != 1 { - t.Errorf("runner calls = %d, want 1 (budget stop after plan)", runner.CallCount()) - } - if result.Session == nil { - t.Fatal("expected session") - } -} - -func TestRunFeature_WithLintCmd(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("plan"), - successResult("implemented"), - }, nil) - - result, err := RunFeature(context.Background(), db, runner, git, FeatureConfig{ - Description: "feature", - Target: "src/", - LintCmd: "true", // lint passes - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunFeature: %v", err) - } - if result.Session.Workflow != "feature" { - t.Errorf("workflow = %q", result.Session.Workflow) - } -} - -func TestRunFeature_LintFails(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - // plan + implement + lint fix - runner := mockRunner("claude", []runners.RunResult{ - successResult("plan"), - successResult("implemented"), - successResult("lint fixed"), - }, nil) - - result, err := RunFeature(context.Background(), db, runner, git, FeatureConfig{ - Description: "feature", - Target: "src/", - LintCmd: "false", // lint fails - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunFeature: %v", err) - } - if result.Session == nil { - t.Fatal("expected session") - } -} - -// --------------------------------------------------------------------------- -// RunRefactor -// --------------------------------------------------------------------------- - -func TestRunRefactor_SuccessNoTests(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("smells: long method, duplicate code"), - successResult("applied extraction refactor"), - }, nil) - - result, err := RunRefactor(context.Background(), db, runner, git, RefactorConfig{ - Description: "clean up handler", - Target: "src/handler.go", - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunRefactor: %v", err) - } - if result.Session.Workflow != "refactor" { - t.Errorf("workflow = %q, want refactor", result.Session.Workflow) - } - if runner.CallCount() != 2 { - t.Errorf("runner calls = %d, want 2 (analyze + transform)", runner.CallCount()) - } -} - -func TestRunRefactor_WithPassingTests(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("analysis"), - successResult("refactored"), - }, nil) - - result, err := RunRefactor(context.Background(), db, runner, git, RefactorConfig{ - Description: "refactor", - Target: "src/", - TestCmd: "true", - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunRefactor: %v", err) - } - if result.Session.Workflow != "refactor" { - t.Errorf("workflow = %q", result.Session.Workflow) - } -} - -func TestRunRefactor_TestsBreakAfterRefactor(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("analysis"), - successResult("refactored"), - }, nil) - - result, err := RunRefactor(context.Background(), db, runner, git, RefactorConfig{ - Description: "refactor", - Target: "src/", - TestCmd: "false", // tests always fail -> reverts - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunRefactor: %v", err) - } - if result.Session == nil { - t.Fatal("expected session") - } -} - -func TestRunRefactor_AnalyzeFails(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{{}}, []error{fmt.Errorf("analyze crash")}) - - _, err := RunRefactor(context.Background(), db, runner, git, RefactorConfig{ - Description: "refactor", - Target: "src/", - RepoRoot: dir, - }) - if err == nil { - t.Fatal("expected error when analyze fails") - } - if !strings.Contains(err.Error(), "analyze step failed") { - t.Errorf("error = %q", err) - } -} - -func TestRunRefactor_TransformFails(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("analysis"), - {}, // transform step response (overridden by error) - }, []error{nil, fmt.Errorf("transform crashed")}) - - _, err := RunRefactor(context.Background(), db, runner, git, RefactorConfig{ - Description: "refactor", - Target: "src/", - RepoRoot: dir, - }) - if err == nil { - t.Fatal("expected error when transform fails") - } - if !strings.Contains(err.Error(), "transform step failed") { - t.Errorf("error = %q", err) - } -} - -func TestRunRefactor_BudgetExhaustedAfterAnalysis(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - {Output: "expensive analysis", CostUSD: 5.00}, - }, nil) - - result, err := RunRefactor(context.Background(), db, runner, git, RefactorConfig{ - Description: "refactor", - Target: "src/", - RepoRoot: dir, - BudgetUSD: 5.00, - }) - if err != nil { - t.Fatalf("RunRefactor: %v", err) - } - if runner.CallCount() != 1 { - t.Errorf("runner calls = %d, want 1", runner.CallCount()) - } - if result.Session == nil { - t.Fatal("expected session") - } -} - -// --------------------------------------------------------------------------- -// RunTestGen -// --------------------------------------------------------------------------- - -func TestRunTestGen_SuccessNoTestCmd(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("generated test_handler_test.go"), - }, nil) - - result, err := RunTestGen(context.Background(), db, runner, git, TestGenConfig{ - Target: "src/handler.go", - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunTestGen: %v", err) - } - if result.Session.Workflow != "test-gen" { - t.Errorf("workflow = %q, want test-gen", result.Session.Workflow) - } - if runner.CallCount() != 1 { - t.Errorf("runner calls = %d, want 1", runner.CallCount()) - } -} - -func TestRunTestGen_WithPassingTests(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("tests generated"), - }, nil) - - result, err := RunTestGen(context.Background(), db, runner, git, TestGenConfig{ - Target: "src/", - TestCmd: "true", - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunTestGen: %v", err) - } - - session, err := db.GetSession(result.Session.ID) - if err != nil { - t.Fatalf("get session: %v", err) - } - if session.Status != "done" { - t.Errorf("status = %q, want done", session.Status) - } -} - -func TestRunTestGen_TestsFailAllAttempts(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - // generate + 5 fix attempts - runner := mockRunner("claude", []runners.RunResult{ - successResult("tests generated"), - successResult("fix 1"), - successResult("fix 2"), - successResult("fix 3"), - successResult("fix 4"), - successResult("fix 5"), - }, nil) - - result, err := RunTestGen(context.Background(), db, runner, git, TestGenConfig{ - Target: "src/", - TestCmd: "false", // always fails - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunTestGen: %v", err) - } - - session, err := db.GetSession(result.Session.ID) - if err != nil { - t.Fatalf("get session: %v", err) - } - if session.Status != "failed" { - t.Errorf("status = %q, want failed (tests never passed)", session.Status) - } -} - -func TestRunTestGen_GenerateStepFails(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{{}}, []error{fmt.Errorf("gen failed")}) - - _, err := RunTestGen(context.Background(), db, runner, git, TestGenConfig{ - Target: "src/", - RepoRoot: dir, - }) - if err == nil { - t.Fatal("expected error when generate fails") - } - if !strings.Contains(err.Error(), "generate step failed") { - t.Errorf("error = %q", err) - } -} - -func TestRunTestGen_BudgetExhausted(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - {Output: "tests generated", CostUSD: 5.00}, - }, nil) - - result, err := RunTestGen(context.Background(), db, runner, git, TestGenConfig{ - Target: "src/", - TestCmd: "false", // would trigger fix attempts but budget is exhausted - RepoRoot: dir, - BudgetUSD: 5.00, - }) - if err != nil { - t.Fatalf("RunTestGen: %v", err) - } - // Should not attempt fixes because budget is exhausted - if runner.CallCount() != 1 { - t.Errorf("runner calls = %d, want 1 (budget stop)", runner.CallCount()) - } - if result.Session == nil { - t.Fatal("expected session") - } -} - -func TestRunTestGen_ContextCancelled(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - ctx, cancel := context.WithCancel(context.Background()) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("tests generated"), - }, nil) - - // Cancel after generate step - cancel() - - result, err := RunTestGen(ctx, db, runner, git, TestGenConfig{ - Target: "src/", - TestCmd: "false", - RepoRoot: dir, - }) - // Generate step may or may not fail depending on timing, but should not panic - if err != nil { - // acceptable: generate step may fail due to cancelled context - return - } - if result.Session == nil { - t.Fatal("expected session") - } -} - -// --------------------------------------------------------------------------- -// RunImproveLoop -// --------------------------------------------------------------------------- - -func TestRunImproveLoop_SingleIterationPasses(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("improved code"), - }, nil) - - result, err := RunImproveLoop(context.Background(), db, runner, git, ImproveConfig{ - Target: "src/", - Metric: "true", // always passes - Objective: "improve performance", - MaxIterations: 1, - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunImproveLoop: %v", err) - } - if result.Session.Workflow != "improve" { - t.Errorf("workflow = %q, want improve", result.Session.Workflow) - } - if result.StopReason != "completed" { - t.Errorf("stop reason = %q, want completed", result.StopReason) - } -} - -func TestRunImproveLoop_AllIterationsReverted(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("change 1"), - successResult("change 2"), - successResult("change 3"), - }, nil) - - result, err := RunImproveLoop(context.Background(), db, runner, git, ImproveConfig{ - Target: "src/", - Metric: "false", // always fails -> all reverted - Objective: "fix tests", - MaxIterations: 5, - MaxFailures: 3, - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunImproveLoop: %v", err) - } - if !strings.Contains(result.StopReason, "consecutive failures") { - t.Errorf("stop reason = %q, want consecutive failures", result.StopReason) - } -} - -func TestRunImproveLoop_BudgetExhausted(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - {Output: "expensive change", CostUSD: 10.00}, - }, nil) - - result, err := RunImproveLoop(context.Background(), db, runner, git, ImproveConfig{ - Target: "src/", - Metric: "true", - Objective: "improve", - MaxIterations: 10, - BudgetUSD: 10.00, - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunImproveLoop: %v", err) - } - if !strings.Contains(result.StopReason, "budget exhausted") { - t.Errorf("stop reason = %q, want budget exhausted", result.StopReason) - } -} - -func TestRunImproveLoop_ContextCancelled(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - ctx, cancel := context.WithCancel(context.Background()) - cancel() // cancel immediately - - runner := mockRunner("claude", []runners.RunResult{ - successResult("change"), - }, nil) - - result, err := RunImproveLoop(ctx, db, runner, git, ImproveConfig{ - Target: "src/", - Metric: "true", - Objective: "improve", - MaxIterations: 5, - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunImproveLoop: %v", err) - } - if result.StopReason != "interrupted" { - t.Errorf("stop reason = %q, want interrupted", result.StopReason) - } -} - -func TestRunImproveLoop_AgentError(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - {}, {}, {}, - }, []error{ - fmt.Errorf("fail 1"), - fmt.Errorf("fail 2"), - fmt.Errorf("fail 3"), - }) - - result, err := RunImproveLoop(context.Background(), db, runner, git, ImproveConfig{ - Target: "src/", - Metric: "true", - Objective: "improve", - MaxIterations: 10, - MaxFailures: 3, - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunImproveLoop: %v", err) - } - if !strings.Contains(result.StopReason, "consecutive failures") { - t.Errorf("stop reason = %q", result.StopReason) - } -} - -func TestRunImproveLoop_DefaultMaxFailures(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - runner := mockRunner("claude", []runners.RunResult{ - {}, {}, {}, - }, []error{ - fmt.Errorf("fail 1"), - fmt.Errorf("fail 2"), - fmt.Errorf("fail 3"), - }) - - result, err := RunImproveLoop(context.Background(), db, runner, git, ImproveConfig{ - Target: "src/", - Metric: "true", - Objective: "improve", - MaxIterations: 10, - MaxFailures: 0, // should default to 3 - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunImproveLoop: %v", err) - } - // With default MaxFailures=3, should stop after 3 consecutive failures - if !strings.Contains(result.StopReason, "3 consecutive failures") { - t.Errorf("stop reason = %q, want 3 consecutive failures", result.StopReason) - } -} - -func TestRunImproveLoop_MixedKeptAndReverted(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - // First passes, second fails, third passes - runner := mockRunner("claude", []runners.RunResult{ - successResult("good change"), - successResult("bad change"), - successResult("good change 2"), - }, nil) - - // We need a metric that alternates. Use a file-based approach: - // "true" always passes, so all will be kept. Test with "true". - result, err := RunImproveLoop(context.Background(), db, runner, git, ImproveConfig{ - Target: "src/", - Metric: "true", - Objective: "improve", - MaxIterations: 3, - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunImproveLoop: %v", err) - } - if result.StopReason != "completed" { - t.Errorf("stop reason = %q, want completed", result.StopReason) - } - if runner.CallCount() != 3 { - t.Errorf("runner calls = %d, want 3", runner.CallCount()) - } -} - -// --------------------------------------------------------------------------- -// ResumeImproveLoop -// --------------------------------------------------------------------------- - -func TestResumeImproveLoop(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - // Create an existing session with 2 completed steps - session := &lib.Session{ - ID: lib.NewSessionID(), - Workflow: "improve", - Target: "src/", - Metric: "true", - Objective: "improve perf", - MaxIterations: 5, - BudgetUSD: 10.00, - Status: "paused", - } - if err := db.CreateSession(session); err != nil { - t.Fatalf("create session: %v", err) - } - for i := 1; i <= 2; i++ { - step := &lib.Step{SessionID: session.ID, Iteration: i, Status: "kept", AgentName: "claude", CostUSD: 0.50} - if err := db.CreateStep(step); err != nil { - t.Fatalf("create step: %v", err) - } - } - - // Create the branch the session expects - branchName := fmt.Sprintf("self-improve/%s", session.ID) - run(t, dir, "git", "checkout", "-b", branchName) - - runner := mockRunner("claude", []runners.RunResult{ - successResult("iteration 3"), - successResult("iteration 4"), - successResult("iteration 5"), - }, nil) - - result, err := ResumeImproveLoop(context.Background(), db, runner, git, session, dir) - if err != nil { - t.Fatalf("ResumeImproveLoop: %v", err) - } - if result.StopReason != "completed" { - t.Errorf("stop reason = %q, want completed", result.StopReason) - } - // Should resume from iteration 3 (after 2 completed) - if runner.CallCount() != 3 { - t.Errorf("runner calls = %d, want 3 (iterations 3-5)", runner.CallCount()) - } -} - -// --------------------------------------------------------------------------- -// buildImprovePrompt -// --------------------------------------------------------------------------- - -func TestBuildImprovePrompt(t *testing.T) { - cfg := ImproveConfig{ - Target: "src/handler.go", - Objective: "reduce latency", - Metric: "go test -bench .", - } - prompt := buildImprovePrompt(cfg) - - if !strings.Contains(prompt, "src/handler.go") { - t.Error("prompt should contain target") - } - if !strings.Contains(prompt, "reduce latency") { - t.Error("prompt should contain objective") - } - if !strings.Contains(prompt, "go test -bench .") { - t.Error("prompt should contain metric") - } -} - -func TestBuildImprovePrompt_EmptyFields(t *testing.T) { - prompt := buildImprovePrompt(ImproveConfig{}) - if prompt == "" { - t.Error("prompt should not be empty even with empty config") - } -} - -// --------------------------------------------------------------------------- -// RunImproveLoop — similarity detection -// --------------------------------------------------------------------------- - -func TestRunImproveLoop_SimilarOutputStopsEarly(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - // All iterations will fail (metric "false"), and the agent produces - // near-identical output each time. The similarity detector should - // stop before hitting MaxFailures (set high to prove similarity wins). - runner := mockRunner("claude", []runners.RunResult{ - successResult("attempt A"), - successResult("attempt B"), - successResult("attempt C"), - successResult("attempt D"), - successResult("attempt E"), - }, nil) - - result, err := RunImproveLoop(context.Background(), db, runner, git, ImproveConfig{ - Target: "src/", - Metric: "echo 'FAIL: 3 errors found in parser.go' && exit 1", - Objective: "fix errors", - MaxIterations: 10, - MaxFailures: 10, // high — similarity should trigger first - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunImproveLoop: %v", err) - } - if !strings.Contains(result.StopReason, "similar outputs") { - t.Errorf("stop reason = %q, want 'similar outputs'", result.StopReason) - } - // Should stop after 3 iterations (2 similar = threshold) - if runner.CallCount() > 4 { - t.Errorf("runner calls = %d, expected <= 4 (similarity bail)", runner.CallCount()) - } -} - -func TestRunImproveLoop_DifferentOutputsNoSimilarityStop(t *testing.T) { - db := tempDB(t) - dir, git := initGitRepo(t) - - // Each iteration fails with different output — should hit MaxFailures, not similarity - runner := mockRunner("claude", []runners.RunResult{ - successResult("change 1"), - successResult("change 2"), - successResult("change 3"), - }, nil) - - // Use a script that produces different output each time - counterFile := filepath.Join(dir, "counter.txt") - os.WriteFile(counterFile, []byte("0"), 0o644) - metric := fmt.Sprintf(`n=$(cat %s); echo "FAIL: error $n" && echo $((n+1)) > %s && exit 1`, counterFile, counterFile) - - result, err := RunImproveLoop(context.Background(), db, runner, git, ImproveConfig{ - Target: "src/", - Metric: metric, - Objective: "fix", - MaxIterations: 10, - MaxFailures: 3, - RepoRoot: dir, - }) - if err != nil { - t.Fatalf("RunImproveLoop: %v", err) - } - if !strings.Contains(result.StopReason, "consecutive failures") { - t.Errorf("stop reason = %q, want 'consecutive failures'", result.StopReason) - } -} - -// --------------------------------------------------------------------------- -// git helper for review tests -// --------------------------------------------------------------------------- - -func makeChange(t *testing.T, dir string, git *lib.Git) { - t.Helper() - run(t, dir, "git", "checkout", "-b", "test-review-branch") - if err := os.WriteFile(filepath.Join(dir, "changed.txt"), []byte("new content\n"), 0o644); err != nil { - t.Fatal(err) - } - run(t, dir, "git", "add", "-A") - run(t, dir, "git", "commit", "-m", "test change") -} - -func run(t *testing.T, dir string, name string, args ...string) { - t.Helper() - cmd := exec.Command(name, args...) - cmd.Dir = dir - if out, err := cmd.CombinedOutput(); err != nil { - t.Fatalf("%s %v: %s", name, args, out) - } -} diff --git a/src/loops/refactor.go b/src/loops/refactor.go deleted file mode 100644 index 67db0cb..0000000 --- a/src/loops/refactor.go +++ /dev/null @@ -1,160 +0,0 @@ -package loops - -import ( - "context" - "fmt" - - "github.com/5uck1ess/devkit/lib" - "github.com/5uck1ess/devkit/runners" -) - -type RefactorConfig struct { - Description string - Target string - TestCmd string - RepoRoot string - BudgetUSD float64 -} - -type RefactorResult struct { - Session *lib.Session - Steps []lib.Step -} - -func RunRefactor(ctx context.Context, db *lib.DB, runner runners.Runner, git *lib.Git, cfg RefactorConfig) (*RefactorResult, error) { - session := &lib.Session{ - ID: lib.NewSessionID(), - Workflow: "refactor", - Target: cfg.Target, - Metric: cfg.TestCmd, - Prompt: cfg.Description, - Status: "running", - BudgetUSD: cfg.BudgetUSD, - } - if err := db.CreateSession(session); err != nil { - return nil, fmt.Errorf("create session: %w", err) - } - if err := lib.EnsureSessionDir(cfg.RepoRoot, session.ID); err != nil { - return nil, fmt.Errorf("create session directory: %w", err) - } - - branchName := fmt.Sprintf("refactor/%s", session.ID) - if err := git.CreateBranch(branchName); err != nil { - return nil, fmt.Errorf("create branch: %w", err) - } - fmt.Printf("Refactor session %s on branch %s\n\n", session.ID, branchName) - - var spentUSD float64 - checkBudget := func() bool { - return cfg.BudgetUSD > 0 && spentUSD >= cfg.BudgetUSD - } - opts := runners.RunOpts{ - WorkDir: cfg.RepoRoot, - AllowedTools: "Bash,Read,Edit,Write,Grep,Glob", - MaxTurns: 25, - } - - // Capture baseline metric - var baselineMetric lib.MetricResult - if cfg.TestCmd != "" { - fmt.Println("Running baseline tests...") - baselineMetric = lib.RunMetric(ctx, cfg.TestCmd, cfg.RepoRoot) - fmt.Printf("Baseline: exit %d\n\n", baselineMetric.ExitCode) - } - - // Step 1: Analyze - fmt.Println("--- Step 1: Analyze ---") - analyzeStep := &lib.Step{SessionID: session.ID, Iteration: 1, Status: "running", AgentName: runner.Name()} - db.CreateStep(analyzeStep) - - analyzeResult, err := runner.Run(ctx, fmt.Sprintf( - `You are analyzing code for refactoring. Read the target code and identify what to change. - -Target: %s -Goal: %s - -Report: -1. Current code smells or issues -2. Proposed transformations (ordered by priority) -3. Risk areas (what could break) - -Do not make any changes yet — analysis only.`, cfg.Target, cfg.Description), opts) - if err != nil { - analyzeStep.Status = "failed" - analyzeStep.ChangeSummary = err.Error() - db.UpdateStep(analyzeStep) - db.UpdateSessionStatus(session.ID, "failed") - return nil, fmt.Errorf("analyze step failed: %w", err) - } - spentUSD += analyzeResult.CostUSD - analyzeStep.Status = "kept" - analyzeStep.Kept = true - analyzeStep.CostUSD = analyzeResult.CostUSD - analyzeStep.ChangeSummary = truncate(analyzeResult.Output, 200) - db.UpdateStep(analyzeStep) - fmt.Printf(" Analysis complete ($%.4f)\n\n", analyzeResult.CostUSD) - - if checkBudget() { - fmt.Printf(" Budget exhausted ($%.2f of $%.2f) — stopping after analysis\n", spentUSD, cfg.BudgetUSD) - db.UpdateSessionStatus(session.ID, "failed") - allSteps, _ := db.GetSteps(session.ID) - return &RefactorResult{Session: session, Steps: allSteps}, nil - } - - // Step 2: Transform - fmt.Println("--- Step 2: Transform ---") - transformStep := &lib.Step{SessionID: session.ID, Iteration: 2, Status: "running", AgentName: runner.Name()} - db.CreateStep(transformStep) - - transformResult, err := runner.Run(ctx, fmt.Sprintf( - `You are refactoring code. Apply the transformations from this analysis. - -Target: %s -Goal: %s - -Analysis: -%s - -Apply each transformation. Preserve all existing behavior — no functional changes.`, cfg.Target, cfg.Description, analyzeResult.Output), opts) - if err != nil { - git.RevertAll() - transformStep.Status = "failed" - transformStep.ChangeSummary = err.Error() - db.UpdateStep(transformStep) - db.UpdateSessionStatus(session.ID, "failed") - return nil, fmt.Errorf("transform step failed: %w", err) - } - spentUSD += transformResult.CostUSD - summary, _ := git.DiffStat() - transformStep.CostUSD = transformResult.CostUSD - transformStep.ChangeSummary = summary - - // Step 3: Verify — tests must still pass - if cfg.TestCmd != "" { - fmt.Println("--- Step 3: Verify ---") - verifyMetric := lib.RunMetric(ctx, cfg.TestCmd, cfg.RepoRoot) - if verifyMetric.ExitCode == 0 { - git.CommitAll(fmt.Sprintf("refactor(%s): transform", session.ID)) - transformStep.Status = "kept" - transformStep.Kept = true - fmt.Println(" Tests still passing — refactor verified") - } else { - git.RevertAll() - transformStep.Status = "reverted" - transformStep.ChangeSummary = fmt.Sprintf("tests broke after refactor (exit %d) — reverted", verifyMetric.ExitCode) - fmt.Printf(" Tests broke (exit %d) — refactor reverted\n\n", verifyMetric.ExitCode) - } - } else { - git.CommitAll(fmt.Sprintf("refactor(%s): transform", session.ID)) - transformStep.Status = "kept" - transformStep.Kept = true - fmt.Println(" No test command — committed without verification") - } - db.UpdateStep(transformStep) - - db.UpdateSessionStatus(session.ID, "done") - allSteps, _ := db.GetSteps(session.ID) - lib.WriteReport(cfg.RepoRoot, session, allSteps, "completed") - - return &RefactorResult{Session: session, Steps: allSteps}, nil -} diff --git a/src/loops/review.go b/src/loops/review.go deleted file mode 100644 index 14a2c57..0000000 --- a/src/loops/review.go +++ /dev/null @@ -1,157 +0,0 @@ -package loops - -import ( - "context" - "fmt" - "strings" - "sync" - - "github.com/5uck1ess/devkit/lib" - "github.com/5uck1ess/devkit/runners" -) - -type ReviewConfig struct { - Prompt string - Security bool - Agents []string // filter to specific agents - RepoRoot string -} - -type AgentResult struct { - Agent string - Output string - Error error - Cost float64 -} - -type ReviewResult struct { - Session *lib.Session - Results []AgentResult -} - -func RunReview(ctx context.Context, db *lib.DB, available []runners.Runner, git *lib.Git, cfg ReviewConfig) (*ReviewResult, error) { - diff, err := git.DiffFromMain() - if err != nil { - return nil, fmt.Errorf("get diff: %w", err) - } - if diff == "" { - return nil, fmt.Errorf("no diff found — nothing to review") - } - - // Truncate diff at file boundaries to preserve reviewable context. - // 500K chars ≈ 125K tokens — well within Claude (200K) and Gemini (1M) limits. - const maxDiff = 500000 - if len(diff) > maxDiff { - // Cut at last file boundary to avoid mid-hunk truncation - cut := diff[:maxDiff] - if idx := strings.LastIndex(cut, "\ndiff --git "); idx > 0 { - cut = cut[:idx] - diff = cut + "\n\n... (diff truncated at file boundary — review remaining files separately)" - } else { - diff = cut + "\n\n... (diff truncated mid-hunk — no clean file boundary found in first 500K chars)" - } - } - - prompt := cfg.Prompt - if prompt == "" { - prompt = "Review this code diff. For each issue found, report: file and line number, severity (critical/warning/suggestion), description, and suggested fix." - } - if cfg.Security { - prompt += "\n\nFocus specifically on security issues: OWASP top 10, hardcoded secrets, injection vulnerabilities, authentication flaws." - } - - fullPrompt := fmt.Sprintf("%s\n\n```diff\n%s\n```", prompt, diff) - - // Filter runners - selected := filterRunners(available, cfg.Agents) - if len(selected) == 0 { - return nil, fmt.Errorf("no agents available — need at least claude CLI installed") - } - - session := &lib.Session{ - ID: lib.NewSessionID(), - Workflow: "review", - Prompt: prompt, - Status: "running", - } - if err := db.CreateSession(session); err != nil { - return nil, fmt.Errorf("create session: %w", err) - } - - fmt.Printf("Review session %s — dispatching to %d agent(s)\n", session.ID, len(selected)) - - // Dispatch in parallel - results := make([]AgentResult, len(selected)) - var wg sync.WaitGroup - - for i, r := range selected { - wg.Add(1) - go func(idx int, runner runners.Runner) { - defer wg.Done() - fmt.Printf(" [%s] running...\n", runner.Name()) - - res, err := runner.Run(ctx, fullPrompt, runners.RunOpts{ - WorkDir: cfg.RepoRoot, - MaxTurns: 10, - }) - results[idx] = AgentResult{ - Agent: runner.Name(), - Output: res.Output, - Error: err, - Cost: res.CostUSD, - } - - step := &lib.Step{ - SessionID: session.ID, - Iteration: idx + 1, - AgentName: runner.Name(), - Status: "done", - ChangeSummary: truncate(res.Output, 200), - CostUSD: res.CostUSD, - } - if err != nil { - step.Status = "failed" - step.ChangeSummary = err.Error() - } - if dbErr := db.CreateStep(step); dbErr != nil { - fmt.Printf(" [%s] warning: failed to persist step: %s\n", runner.Name(), dbErr) - } - if dbErr := db.UpdateStep(step); dbErr != nil { - fmt.Printf(" [%s] warning: failed to update step: %s\n", runner.Name(), dbErr) - } - - fmt.Printf(" [%s] done ($%.4f)\n", runner.Name(), res.CostUSD) - }(i, r) - } - wg.Wait() - - if err := db.UpdateSessionStatus(session.ID, "done"); err != nil { - fmt.Printf(" warning: failed to update session status: %s\n", err) - } - - return &ReviewResult{Session: session, Results: results}, nil -} - -func filterRunners(available []runners.Runner, names []string) []runners.Runner { - if len(names) == 0 { - return available - } - nameSet := make(map[string]bool) - for _, n := range names { - nameSet[strings.ToLower(n)] = true - } - var filtered []runners.Runner - for _, r := range available { - if nameSet[r.Name()] { - filtered = append(filtered, r) - } - } - return filtered -} - -func truncate(s string, n int) string { - if len(s) <= n { - return s - } - return s[:n] + "..." -} diff --git a/src/loops/testgen.go b/src/loops/testgen.go deleted file mode 100644 index 4fcc206..0000000 --- a/src/loops/testgen.go +++ /dev/null @@ -1,144 +0,0 @@ -package loops - -import ( - "context" - "fmt" - - "github.com/5uck1ess/devkit/lib" - "github.com/5uck1ess/devkit/runners" -) - -type TestGenConfig struct { - Target string - TestCmd string - RepoRoot string - BudgetUSD float64 -} - -type TestGenResult struct { - Session *lib.Session - Steps []lib.Step -} - -func RunTestGen(ctx context.Context, db *lib.DB, runner runners.Runner, git *lib.Git, cfg TestGenConfig) (*TestGenResult, error) { - session := &lib.Session{ - ID: lib.NewSessionID(), - Workflow: "test-gen", - Target: cfg.Target, - Metric: cfg.TestCmd, - Status: "running", - BudgetUSD: cfg.BudgetUSD, - } - if err := db.CreateSession(session); err != nil { - return nil, fmt.Errorf("create session: %w", err) - } - if err := lib.EnsureSessionDir(cfg.RepoRoot, session.ID); err != nil { - return nil, fmt.Errorf("create session directory: %w", err) - } - - branchName := fmt.Sprintf("test-gen/%s", session.ID) - if err := git.CreateBranch(branchName); err != nil { - return nil, fmt.Errorf("create branch: %w", err) - } - fmt.Printf("Test-gen session %s on branch %s\n\n", session.ID, branchName) - - var spentUSD float64 - checkBudget := func() bool { - return cfg.BudgetUSD > 0 && spentUSD >= cfg.BudgetUSD - } - opts := runners.RunOpts{ - WorkDir: cfg.RepoRoot, - AllowedTools: "Bash,Read,Edit,Write,Grep,Glob", - MaxTurns: 30, - } - - // Step 1: Analyze target and generate tests - fmt.Println("--- Step 1: Generate Tests ---") - genStep := &lib.Step{SessionID: session.ID, Iteration: 1, Status: "running", AgentName: runner.Name()} - db.CreateStep(genStep) - - genResult, err := runner.Run(ctx, fmt.Sprintf( - `Analyze the code at %s and generate a comprehensive test suite. - -1. Detect the language, test framework, and existing test patterns. -2. Identify all public functions, methods, and API endpoints. -3. Write tests covering: happy paths, edge cases, error conditions, boundary values. -4. Use the project's existing test framework and conventions. -5. Place tests in the project's standard test location. - -Write actual test code — no placeholders or TODOs.`, cfg.Target), opts) - if err != nil { - genStep.Status = "failed" - genStep.ChangeSummary = err.Error() - db.UpdateStep(genStep) - db.UpdateSessionStatus(session.ID, "failed") - return nil, fmt.Errorf("generate step failed: %w", err) - } - spentUSD += genResult.CostUSD - if err := git.CommitAll(fmt.Sprintf("test-gen(%s): generate tests", session.ID)); err != nil { - fmt.Printf(" Warning: commit failed: %s\n", err) - } - genStep.Status = "kept" - genStep.Kept = true - genStep.CostUSD = genResult.CostUSD - genStep.ChangeSummary = truncate(genResult.Output, 200) - db.UpdateStep(genStep) - fmt.Printf(" Tests generated ($%.4f)\n\n", genResult.CostUSD) - - // Step 2: Run tests and fix failures (up to 5 attempts) - testsPass := cfg.TestCmd == "" - if cfg.TestCmd != "" && !checkBudget() { - fmt.Println("--- Step 2: Run & Fix ---") - for attempt := 1; attempt <= 5; attempt++ { - if ctx.Err() != nil || checkBudget() { - break - } - testMetric := lib.RunMetric(ctx, cfg.TestCmd, cfg.RepoRoot) - if testMetric.ExitCode == 0 { - fmt.Printf(" All tests passing (attempt %d)\n", attempt) - testsPass = true - break - } - - fmt.Printf(" Tests failing (attempt %d), fixing...\n", attempt) - fixStep := &lib.Step{SessionID: session.ID, Iteration: 1 + attempt, Status: "running", AgentName: runner.Name()} - db.CreateStep(fixStep) - - fixResult, err := runner.Run(ctx, fmt.Sprintf( - `The generated tests are failing. Fix them so they pass. -Determine if the bug is in the test or the implementation. -If the test expectation is wrong, fix the test. If the code has a bug, fix the code. - -Test command: %s -Test output: -%s`, cfg.TestCmd, testMetric.Output), opts) - if err != nil { - fixStep.Status = "failed" - fixStep.ChangeSummary = err.Error() - db.UpdateStep(fixStep) - continue - } - spentUSD += fixResult.CostUSD - if err := git.CommitAll(fmt.Sprintf("test-gen(%s): fix tests attempt %d", session.ID, attempt)); err != nil { - fmt.Printf(" Warning: commit failed: %s\n", err) - } - fixStep.Status = "kept" - fixStep.Kept = true - fixStep.CostUSD = fixResult.CostUSD - db.UpdateStep(fixStep) - } - } - - status := "done" - if !testsPass { - status = "failed" - } - db.UpdateSessionStatus(session.ID, status) - allSteps, err := db.GetSteps(session.ID) - if err != nil { - fmt.Printf(" Warning: failed to get steps for report: %s\n", err) - } - lib.WriteReport(cfg.RepoRoot, session, allSteps, status) - - return &TestGenResult{Session: session, Steps: allSteps}, nil -} diff --git a/src/mcp/integration_test.go b/src/mcp/integration_test.go new file mode 100644 index 0000000..55ab70a --- /dev/null +++ b/src/mcp/integration_test.go @@ -0,0 +1,299 @@ +package mcp + +import ( + "context" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/5uck1ess/devkit/lib" + mcpmcp "github.com/mark3labs/mcp-go/mcp" +) + +// setupTestServer creates a Server backed by temp dirs with a workflow and optional principles file. +func setupTestServer(t *testing.T, workflowYAML string, principlesYAML string) (*Server, string) { + t.Helper() + dir := t.TempDir() + wfDir := filepath.Join(dir, "workflows") + if err := os.MkdirAll(wfDir, 0o755); err != nil { + t.Fatalf("mkdir workflows: %v", err) + } + if err := os.WriteFile(filepath.Join(wfDir, "test.yml"), []byte(workflowYAML), 0o644); err != nil { + t.Fatalf("write workflow: %v", err) + } + + if principlesYAML != "" { + skillsDir := filepath.Join(dir, "skills") + if err := os.MkdirAll(skillsDir, 0o755); err != nil { + t.Fatalf("mkdir skills: %v", err) + } + if err := os.WriteFile(filepath.Join(skillsDir, "_principles.yml"), []byte(principlesYAML), 0o644); err != nil { + t.Fatalf("write principles: %v", err) + } + } + + srv, err := NewServer(dir, dir, wfDir) + if err != nil { + t.Fatalf("NewServer: %v", err) + } + return srv, dir +} + +// callToolHandler invokes a tool handler with the given arguments and returns the text content. +func callToolHandler(t *testing.T, handler func(context.Context, mcpmcp.CallToolRequest) (*mcpmcp.CallToolResult, error), args map[string]interface{}) (string, bool) { + t.Helper() + req := mcpmcp.CallToolRequest{} + req.Params.Arguments = args + result, err := handler(context.Background(), req) + if err != nil { + t.Fatalf("handler returned error: %v", err) + } + if result == nil || len(result.Content) == 0 { + t.Fatal("handler returned empty result") + } + tc, ok := result.Content[0].(mcpmcp.TextContent) + if !ok { + t.Fatalf("unexpected content type: %T", result.Content[0]) + } + return tc.Text, result.IsError +} + +func TestIntegrationFullLifecycle(t *testing.T) { + workflowYAML := `name: test +description: Full lifecycle test +steps: + - id: plan + prompt: "Plan the work for {{input}}." + - id: check + command: "echo hello" + expect: success + - id: report + prompt: "Write a report." +` + srv, dataDir := setupTestServer(t, workflowYAML, "") + + // Step 1: Start workflow + _, startHandler := srv.startTool() + out, isErr := callToolHandler(t, startHandler, map[string]interface{}{ + "workflow": "test", + "input": "test input", + }) + if isErr { + t.Fatalf("start returned error: %s", out) + } + if !strings.Contains(out, "STEP 1/3") { + t.Errorf("expected STEP 1/3, got:\n%s", out) + } + if !strings.Contains(out, "plan") { + t.Errorf("expected step id 'plan', got:\n%s", out) + } + + // Verify session.json was created + state, err := lib.ReadSessionJSON(dataDir) + if err != nil || state == nil { + t.Fatalf("session.json not created: %v", err) + } + sessionID := state.ID + if state.Workflow != "test" { + t.Errorf("expected workflow 'test', got %q", state.Workflow) + } + + // Step 2: Advance past prompt step → should get command step + _, advHandler := srv.advanceTool() + out, isErr = callToolHandler(t, advHandler, map[string]interface{}{ + "session": sessionID, + "output": "plan output here", + }) + if isErr { + t.Fatalf("advance 1 returned error: %s", out) + } + if !strings.Contains(out, "STEP 2/3") { + t.Errorf("expected STEP 2/3, got:\n%s", out) + } + if !strings.Contains(out, "check") { + t.Errorf("expected step id 'check', got:\n%s", out) + } + + // Step 3: Advance on command step → auto-executes "echo hello", moves to report + out, isErr = callToolHandler(t, advHandler, map[string]interface{}{ + "session": sessionID, + }) + if isErr { + t.Fatalf("advance 2 returned error: %s", out) + } + if !strings.Contains(out, "STEP 3/3") { + t.Errorf("expected STEP 3/3, got:\n%s", out) + } + if !strings.Contains(out, "report") { + t.Errorf("expected step id 'report', got:\n%s", out) + } + + // Verify command output was captured + state, _ = lib.ReadSessionJSON(dataDir) + if !strings.Contains(state.Outputs["check"], "hello") { + t.Errorf("expected 'hello' in check output, got %q", state.Outputs["check"]) + } + + // Step 4: Advance past final prompt step → workflow complete + out, isErr = callToolHandler(t, advHandler, map[string]interface{}{ + "session": sessionID, + "output": "report done", + }) + if isErr { + t.Fatalf("advance 3 returned error: %s", out) + } + if !strings.Contains(out, "WORKFLOW COMPLETE") { + t.Errorf("expected WORKFLOW COMPLETE, got:\n%s", out) + } + if !strings.Contains(out, sessionID) { + t.Errorf("expected session ID in completion message, got:\n%s", out) + } + + // session.json should be cleared + cleared, _ := lib.ReadSessionJSON(dataDir) + if cleared != nil { + t.Error("expected session.json cleared after completion") + } +} + +func TestIntegrationLoopWithGate(t *testing.T) { + workflowYAML := `name: test +description: Loop gate test +steps: + - id: fix + prompt: "Fix the issue." + - id: verify + prompt: "Verify the fix." + loop: + max: 5 + gate: "true" + - id: done + prompt: "Wrap up." +` + srv, dataDir := setupTestServer(t, workflowYAML, "") + + // Start workflow + _, startHandler := srv.startTool() + out, isErr := callToolHandler(t, startHandler, map[string]interface{}{ + "workflow": "test", + "input": "bug fix", + }) + if isErr { + t.Fatalf("start returned error: %s", out) + } + + state, _ := lib.ReadSessionJSON(dataDir) + sessionID := state.ID + + // Advance past "fix" step → should land on "verify" (loop step) + _, advHandler := srv.advanceTool() + out, isErr = callToolHandler(t, advHandler, map[string]interface{}{ + "session": sessionID, + "output": "fixed the bug", + }) + if isErr { + t.Fatalf("advance to verify returned error: %s", out) + } + if !strings.Contains(out, "verify") { + t.Errorf("expected to land on 'verify', got:\n%s", out) + } + + // Advance on loop step → gate "true" exits 0, should jump to "done" + out, isErr = callToolHandler(t, advHandler, map[string]interface{}{ + "session": sessionID, + "output": "verification output", + }) + if isErr { + t.Fatalf("advance through gate returned error: %s", out) + } + if !strings.Contains(out, "done") { + t.Errorf("expected to advance to 'done' after gate pass, got:\n%s", out) + } + if strings.Contains(out, "LOOP ITERATION") { + t.Errorf("should not see LOOP ITERATION when gate passes, got:\n%s", out) + } +} + +func TestIntegrationPrincipleInjection(t *testing.T) { + workflowYAML := `name: test +description: Principle injection test +principles: [dry, yagni] +steps: + - id: step1 + prompt: "do something" +` + principlesYAML := `dry: + - Don't abstract until 3rd duplication +yagni: + - Build what's needed now +` + srv, _ := setupTestServer(t, workflowYAML, principlesYAML) + + // Start workflow + _, startHandler := srv.startTool() + out, isErr := callToolHandler(t, startHandler, map[string]interface{}{ + "workflow": "test", + "input": "test input", + }) + if isErr { + t.Fatalf("start returned error: %s", out) + } + + if !strings.Contains(out, "PRINCIPLES:") { + t.Errorf("expected PRINCIPLES: header, got:\n%s", out) + } + if !strings.Contains(out, "dry") { + t.Errorf("expected 'dry' principle, got:\n%s", out) + } + if !strings.Contains(out, "Don't abstract until 3rd duplication") { + t.Errorf("expected dry rule text, got:\n%s", out) + } + if !strings.Contains(out, "yagni") { + t.Errorf("expected 'yagni' principle, got:\n%s", out) + } + if !strings.Contains(out, "Build what's needed now") { + t.Errorf("expected yagni rule text, got:\n%s", out) + } +} + +func TestIntegrationExpectFailure(t *testing.T) { + workflowYAML := `name: test +description: Expect failure test +steps: + - id: should-fail + command: "exit 1" + expect: failure + - id: done + prompt: "Summarise." +` + srv, dataDir := setupTestServer(t, workflowYAML, "") + + // Start workflow → lands on command step + _, startHandler := srv.startTool() + out, isErr := callToolHandler(t, startHandler, map[string]interface{}{ + "workflow": "test", + "input": "test input", + }) + if isErr { + t.Fatalf("start returned error: %s", out) + } + + state, _ := lib.ReadSessionJSON(dataDir) + sessionID := state.ID + + // Advance on command step: "exit 1" with expect: failure → should succeed + _, advHandler := srv.advanceTool() + out, isErr = callToolHandler(t, advHandler, map[string]interface{}{ + "session": sessionID, + }) + if isErr { + t.Fatalf("advance returned error (expected success for expected failure): %s", out) + } + if !strings.Contains(out, "STEP 2/2") { + t.Errorf("expected STEP 2/2 after expected failure passed, got:\n%s", out) + } + if !strings.Contains(out, "done") { + t.Errorf("expected step id 'done', got:\n%s", out) + } +} diff --git a/src/mcp/principles.go b/src/mcp/principles.go new file mode 100644 index 0000000..8e5debf --- /dev/null +++ b/src/mcp/principles.go @@ -0,0 +1,37 @@ +package mcp + +import ( + "fmt" + "os" + "path/filepath" + + "gopkg.in/yaml.v3" +) + +// LoadPrinciples reads the condensed principles index. +// Looks in the plugin's skills/ directory for _principles.yml. +func LoadPrinciples(workflowDir string) (map[string][]string, error) { + // Try relative to workflow dir (plugin root/skills/_principles.yml) + candidates := []string{ + filepath.Join(filepath.Dir(workflowDir), "skills", "_principles.yml"), + } + + // Also check CLAUDE_PLUGIN_ROOT + if root := os.Getenv("CLAUDE_PLUGIN_ROOT"); root != "" { + candidates = append([]string{filepath.Join(root, "skills", "_principles.yml")}, candidates...) + } + + for _, path := range candidates { + data, err := os.ReadFile(path) + if err != nil { + continue + } + var principles map[string][]string + if err := yaml.Unmarshal(data, &principles); err != nil { + return nil, fmt.Errorf("parse principles: %w", err) + } + return principles, nil + } + + return nil, fmt.Errorf("_principles.yml not found") +} diff --git a/src/mcp/principles_test.go b/src/mcp/principles_test.go new file mode 100644 index 0000000..b52ecfc --- /dev/null +++ b/src/mcp/principles_test.go @@ -0,0 +1,29 @@ +package mcp + +import ( + "os" + "path/filepath" + "testing" +) + +func TestLoadPrinciples(t *testing.T) { + dir := t.TempDir() + skillsDir := filepath.Join(dir, "skills") + os.MkdirAll(skillsDir, 0o755) + wfDir := filepath.Join(dir, "workflows") + os.MkdirAll(wfDir, 0o755) + + content := []byte("dry:\n - Don't abstract until 3rd duplication\nyagni:\n - Build what's needed now\n") + os.WriteFile(filepath.Join(skillsDir, "_principles.yml"), content, 0o644) + + p, err := LoadPrinciples(wfDir) + if err != nil { + t.Fatalf("LoadPrinciples: %v", err) + } + if len(p["dry"]) != 1 { + t.Errorf("expected 1 dry principle, got %d", len(p["dry"])) + } + if len(p["yagni"]) != 1 { + t.Errorf("expected 1 yagni principle, got %d", len(p["yagni"])) + } +} diff --git a/src/mcp/server.go b/src/mcp/server.go new file mode 100644 index 0000000..164c616 --- /dev/null +++ b/src/mcp/server.go @@ -0,0 +1,77 @@ +package mcp + +import ( + "context" + "fmt" + "os" + "path/filepath" + + "github.com/5uck1ess/devkit/lib" + mcpgo "github.com/mark3labs/mcp-go/server" +) + +// Server wraps the devkit engine as an MCP server. +type Server struct { + dataDir string + workflowDir string + repoRoot string + db *lib.DB + git *lib.Git + principles map[string][]string +} + +// NewServer creates a devkit MCP server. +func NewServer(repoRoot, dataDir, workflowDir string) (*Server, error) { + if repoRoot == "" || dataDir == "" || workflowDir == "" { + return nil, fmt.Errorf("repoRoot, dataDir, and workflowDir must be non-empty") + } + + dbPath := filepath.Join(dataDir, "devkit.db") + db, err := lib.OpenDB(dbPath) + if err != nil { + return nil, fmt.Errorf("open db: %w", err) + } + + principles, err := LoadPrinciples(workflowDir) + if err != nil { + fmt.Fprintf(os.Stderr, "warning: could not load principles: %v\n", err) + principles = map[string][]string{} + } + + return &Server{ + dataDir: dataDir, + workflowDir: workflowDir, + repoRoot: repoRoot, + db: db, + git: &lib.Git{Dir: repoRoot}, + principles: principles, + }, nil +} + +// Close releases server resources (database connection). +func (s *Server) Close() error { + if s.db != nil { + return s.db.Close() + } + return nil +} + +// Serve starts the MCP server on stdio, respecting ctx for graceful shutdown. +func (s *Server) Serve(ctx context.Context) error { + srv := mcpgo.NewMCPServer("devkit-engine", "1.0.0") + + tool, handler := s.startTool() + srv.AddTool(tool, handler) + + tool, handler = s.advanceTool() + srv.AddTool(tool, handler) + + tool, handler = s.statusTool() + srv.AddTool(tool, handler) + + tool, handler = s.listTool() + srv.AddTool(tool, handler) + + stdio := mcpgo.NewStdioServer(srv) + return stdio.Listen(ctx, os.Stdin, os.Stdout) +} diff --git a/src/mcp/server_test.go b/src/mcp/server_test.go new file mode 100644 index 0000000..5e23c4f --- /dev/null +++ b/src/mcp/server_test.go @@ -0,0 +1,21 @@ +package mcp + +import ( + "os" + "path/filepath" + "testing" +) + +func TestNewServer(t *testing.T) { + dir := t.TempDir() + wfDir := filepath.Join(dir, "workflows") + os.MkdirAll(wfDir, 0o755) + + srv, err := NewServer(dir, dir, wfDir) + if err != nil { + t.Fatalf("NewServer: %v", err) + } + if srv == nil { + t.Fatal("server is nil") + } +} diff --git a/src/mcp/tools.go b/src/mcp/tools.go new file mode 100644 index 0000000..f5498e0 --- /dev/null +++ b/src/mcp/tools.go @@ -0,0 +1,473 @@ +package mcp + +import ( + "bytes" + "context" + "errors" + "fmt" + "os" + "os/exec" + "path/filepath" + "strings" + "time" + + "github.com/5uck1ess/devkit/engine" + "github.com/5uck1ess/devkit/lib" + mcpmcp "github.com/mark3labs/mcp-go/mcp" + mcpgo "github.com/mark3labs/mcp-go/server" +) + +// commandTimeout is the maximum duration for workflow command execution. +const commandTimeout = 5 * time.Minute + +func (s *Server) listTool() (mcpmcp.Tool, mcpgo.ToolHandlerFunc) { + tool := mcpmcp.NewTool("devkit_list", + mcpmcp.WithDescription("List available workflows"), + ) + return tool, func(ctx context.Context, req mcpmcp.CallToolRequest) (*mcpmcp.CallToolResult, error) { + entries, err := os.ReadDir(s.workflowDir) + if err != nil { + return mcpmcp.NewToolResultError(fmt.Sprintf("no workflows directory: %v", err)), nil + } + var lines []string + for _, e := range entries { + name := e.Name() + if !strings.HasSuffix(name, ".yml") && !strings.HasSuffix(name, ".yaml") { + continue + } + wfName := strings.TrimSuffix(strings.TrimSuffix(name, ".yml"), ".yaml") + path := filepath.Join(s.workflowDir, name) + wf, err := engine.ParseFile(path) + if err != nil { + lines = append(lines, fmt.Sprintf("- %s (parse error)", wfName)) + continue + } + lines = append(lines, fmt.Sprintf("- **%s**: %s", wfName, wf.Description)) + } + return mcpmcp.NewToolResultText(strings.Join(lines, "\n")), nil + } +} + +func (s *Server) statusTool() (mcpmcp.Tool, mcpgo.ToolHandlerFunc) { + tool := mcpmcp.NewTool("devkit_status", + mcpmcp.WithDescription("Check workflow progress"), + mcpmcp.WithString("session", mcpmcp.Description("Session ID (optional)")), + ) + return tool, func(ctx context.Context, req mcpmcp.CallToolRequest) (*mcpmcp.CallToolResult, error) { + state, err := lib.ReadSessionJSON(s.dataDir) + if err != nil { + return mcpmcp.NewToolResultError(fmt.Sprintf("read state: %v", err)), nil + } + if state == nil { + return mcpmcp.NewToolResultText("No active workflow session."), nil + } + msg := fmt.Sprintf("Workflow: %s\nSession: %s\nStep: %s (%d/%d)\nEnforce: %s\nStatus: %s", + state.Workflow, state.ID, state.CurrentStep, + state.CurrentIndex+1, state.TotalSteps, + state.Enforce, state.Status) + return mcpmcp.NewToolResultText(msg), nil + } +} + +func (s *Server) startTool() (mcpmcp.Tool, mcpgo.ToolHandlerFunc) { + tool := mcpmcp.NewTool("devkit_start", + mcpmcp.WithDescription("Start a workflow"), + mcpmcp.WithString("workflow", mcpmcp.Required(), mcpmcp.Description("Workflow name")), + mcpmcp.WithString("input", mcpmcp.Required(), mcpmcp.Description("Workflow input/description")), + ) + return tool, func(ctx context.Context, req mcpmcp.CallToolRequest) (*mcpmcp.CallToolResult, error) { + // Check no active session — propagate read errors + existing, err := lib.ReadSessionJSON(s.dataDir) + if err != nil { + return mcpmcp.NewToolResultError(fmt.Sprintf("read session state: %v", err)), nil + } + if existing != nil && existing.Status == "running" { + return mcpmcp.NewToolResultError(fmt.Sprintf("workflow %s already running (session %s). Call devkit_advance to continue or devkit_status to check.", existing.Workflow, existing.ID)), nil + } + + wfName, err := req.RequireString("workflow") + if err != nil { + return mcpmcp.NewToolResultError(fmt.Sprintf("missing argument: %v", err)), nil + } + input, err := req.RequireString("input") + if err != nil { + return mcpmcp.NewToolResultError(fmt.Sprintf("missing argument: %v", err)), nil + } + + // Reject workflow names with path separators or dot-dot traversal sequences. + if strings.ContainsAny(wfName, `/\`) || strings.Contains(wfName, "..") { + return mcpmcp.NewToolResultError(fmt.Sprintf("invalid workflow name %q: must not contain path separators", wfName)), nil + } + + // Find and parse workflow — resolve and verify the path stays inside workflowDir. + wfPath, err := s.resolveWorkflowPath(wfName) + if err != nil { + return mcpmcp.NewToolResultError(err.Error()), nil + } + wf, err := engine.ParseFile(wfPath) + if err != nil { + return mcpmcp.NewToolResultError(fmt.Sprintf("parse workflow %q: %v", wfName, err)), nil + } + + if len(wf.Steps) == 0 { + return mcpmcp.NewToolResultError(fmt.Sprintf("workflow %q has no steps", wfName)), nil + } + + // Create session — store the validated filename for safe re-parsing in advance + sessionID := lib.NewSessionID() + firstStep := wf.Steps[0] + + state := &lib.SessionState{ + ID: sessionID, + Workflow: wfName, // store filename, not wf.Name, to prevent traversal in advance + Input: input, + CurrentStep: firstStep.ID, + CurrentIndex: 0, + TotalSteps: len(wf.Steps), + StepType: stepType(firstStep), + Enforce: wf.Enforce, + Branch: wf.BranchMode, + Status: "running", + StartedAt: time.Now(), + Outputs: map[string]string{}, + } + if err := lib.WriteSessionJSON(s.dataDir, state); err != nil { + return mcpmcp.NewToolResultError(fmt.Sprintf("write state: %v", err)), nil + } + + // SQLite record + if s.db != nil { + if err := s.db.CreateSession(&lib.Session{ + ID: sessionID, + Workflow: wf.Name, + Prompt: input, + Status: "running", + }); err != nil { + fmt.Fprintf(os.Stderr, "warning: db create session: %v\n", err) + } + } + + // Git branch if configured + if wf.BranchMode && s.git != nil { + branchName := fmt.Sprintf("%s/%s", wf.Name, sessionID) + if err := s.git.CreateBranch(branchName); err != nil { + fmt.Fprintf(os.Stderr, "warning: branch creation failed: %v\n", err) + } + } + + // Build response with first step + principles + response := s.formatStepResponse(wf, state, &firstStep, input) + return mcpmcp.NewToolResultText(response), nil + } +} + +// resolveWorkflowPath finds and validates a workflow file path. +// Returns an error if the name resolves outside workflowDir. +func (s *Server) resolveWorkflowPath(name string) (string, error) { + absWorkflowDir, err := filepath.Abs(s.workflowDir) + if err != nil { + return "", fmt.Errorf("resolve workflow dir: %w", err) + } + + for _, ext := range []string{".yml", ".yaml"} { + candidate := filepath.Join(s.workflowDir, name+ext) + absCandidate, err := filepath.Abs(candidate) + if err != nil { + return "", fmt.Errorf("resolve workflow path: %w", err) + } + if !strings.HasPrefix(absCandidate, absWorkflowDir+string(filepath.Separator)) { + return "", fmt.Errorf("invalid workflow name %q: resolves outside workflow directory", name) + } + if _, statErr := os.Stat(candidate); statErr == nil { + return candidate, nil + } + } + return "", fmt.Errorf("workflow %q not found", name) +} + +func stepType(step engine.WfStep) string { + if step.Command != "" { + return "command" + } + if len(step.Parallel) > 0 { + return "parallel" + } + return "prompt" +} + +func (s *Server) formatStepResponse(wf *engine.Workflow, state *lib.SessionState, step *engine.WfStep, input string) string { + var b strings.Builder + + fmt.Fprintf(&b, "=== STEP %d/%d: %s ===\n", state.CurrentIndex+1, state.TotalSteps, step.ID) + + if step.Command != "" { + cmd := engine.Interpolate(step.Command, input, state.Outputs) + fmt.Fprintf(&b, "TYPE: command (engine will execute automatically on devkit_advance)\n") + fmt.Fprintf(&b, "COMMAND: %s\n", cmd) + if step.Expect != "" { + fmt.Fprintf(&b, "EXPECT: %s\n", step.Expect) + } + } else if len(step.Parallel) > 0 { + fmt.Fprintf(&b, "TYPE: parallel dispatch\n") + fmt.Fprintf(&b, "DISPATCH: %s\n", strings.Join(step.Parallel, ", ")) + fmt.Fprintf(&b, "Use the Agent tool and plugins to run these in parallel, then call devkit_advance.\n") + } else { + prompt := engine.Interpolate(step.Prompt, input, state.Outputs) + fmt.Fprintf(&b, "PROMPT: %s\n", prompt) + } + + // Inject principles + principles := step.Principles + if len(principles) == 0 { + principles = wf.Principles + } + if len(principles) > 0 { + fmt.Fprintf(&b, "\nPRINCIPLES:\n") + for _, p := range principles { + if rules, ok := s.principles[p]; ok { + fmt.Fprintf(&b, "[%s] %s\n", p, strings.Join(rules, "; ")) + } + } + } + + if step.Loop != nil { + fmt.Fprintf(&b, "\nLOOP: max %d iterations", step.Loop.Max) + if step.Loop.Gate != "" { + fmt.Fprintf(&b, ", gate: %s", step.Loop.Gate) + } + if step.Loop.Until != "" { + fmt.Fprintf(&b, ", until: %s", step.Loop.Until) + } + fmt.Fprintln(&b) + if rules, ok := s.principles["scratchpad"]; ok { + fmt.Fprintf(&b, "[scratchpad] %s\n", strings.Join(rules, "; ")) + } + if rules, ok := s.principles["stuck"]; ok { + fmt.Fprintf(&b, "[stuck] %s\n", strings.Join(rules, "; ")) + } + } + + fmt.Fprintf(&b, "\nCall devkit_advance when this step is complete.\n") + return b.String() +} + +func (s *Server) advanceTool() (mcpmcp.Tool, mcpgo.ToolHandlerFunc) { + tool := mcpmcp.NewTool("devkit_advance", + mcpmcp.WithDescription("Complete current step and get next"), + mcpmcp.WithString("session", mcpmcp.Required(), mcpmcp.Description("Session ID")), + mcpmcp.WithString("output", mcpmcp.Description("Summary of step output (for prompt steps)")), + ) + return tool, func(ctx context.Context, req mcpmcp.CallToolRequest) (*mcpmcp.CallToolResult, error) { + sessionID, err := req.RequireString("session") + if err != nil { + return mcpmcp.NewToolResultError("missing session argument"), nil + } + + state, err := lib.ReadSessionJSON(s.dataDir) + if err != nil { + return mcpmcp.NewToolResultError(fmt.Sprintf("read session: %v", err)), nil + } + if state == nil { + return mcpmcp.NewToolResultError("no active session"), nil + } + if state.ID != sessionID { + return mcpmcp.NewToolResultError(fmt.Sprintf("session mismatch: active is %s", state.ID)), nil + } + + // Re-parse workflow using validated filename stored in state.Workflow + wfPath, err := s.resolveWorkflowPath(state.Workflow) + if err != nil { + return mcpmcp.NewToolResultError(fmt.Sprintf("resolve workflow: %v", err)), nil + } + wf, err := engine.ParseFile(wfPath) + if err != nil { + return mcpmcp.NewToolResultError(fmt.Sprintf("parse workflow: %v", err)), nil + } + + // Bounds check against corrupted state + if state.CurrentIndex < 0 || state.CurrentIndex >= len(wf.Steps) { + return mcpmcp.NewToolResultError(fmt.Sprintf("invalid step index %d (workflow has %d steps)", state.CurrentIndex, len(wf.Steps))), nil + } + + currentStep := wf.Steps[state.CurrentIndex] + + // Handle command steps — engine executes them + if currentStep.Command != "" { + cmd := engine.Interpolate(currentStep.Command, state.Input, state.Outputs) + output, exitCode, cmdErr := s.runCommand(ctx, cmd) + if cmdErr != nil { + return mcpmcp.NewToolResultError(fmt.Sprintf("command failed: %v", cmdErr)), nil + } + + // Check expect + if currentStep.Expect == "failure" && exitCode == 0 { + return mcpmcp.NewToolResultError(fmt.Sprintf("step %s: expected failure but got exit 0", currentStep.ID)), nil + } + if currentStep.Expect == "success" && exitCode != 0 { + return mcpmcp.NewToolResultError(fmt.Sprintf("step %s: expected success but got exit %d\n%s", currentStep.ID, exitCode, output)), nil + } + + state.Outputs[currentStep.ID] = output + } else { + // Prompt/parallel step — record output from Claude + args := req.GetArguments() + if outputArg, ok := args["output"]; ok && outputArg != nil { + if outputStr, ok := outputArg.(string); ok { + state.Outputs[currentStep.ID] = outputStr + } + } + } + + // Handle loop steps + if currentStep.Loop != nil { + return s.handleLoopAdvance(ctx, wf, state, ¤tStep, req) + } + + // Advance to next step + nextIndex := state.CurrentIndex + 1 + + // Check branch conditions + if len(currentStep.Branch) > 0 { + if output, ok := state.Outputs[currentStep.ID]; ok { + target := engine.EvalBranch(output, currentStep.Branch) + if target != "" { + for i, step := range wf.Steps { + if step.ID == target { + nextIndex = i + break + } + } + } + } + } + + if nextIndex >= len(wf.Steps) { + return s.completeWorkflow(state) + } + + // Write next step state + nextStep := wf.Steps[nextIndex] + state.CurrentStep = nextStep.ID + state.CurrentIndex = nextIndex + state.StepType = stepType(nextStep) + if err := lib.WriteSessionJSON(s.dataDir, state); err != nil { + return mcpmcp.NewToolResultError(fmt.Sprintf("write state: %v", err)), nil + } + + response := s.formatStepResponse(wf, state, &nextStep, state.Input) + return mcpmcp.NewToolResultText(response), nil + } +} + +// completeWorkflow marks a session as done, updates DB, and clears hot state. +func (s *Server) completeWorkflow(state *lib.SessionState) (*mcpmcp.CallToolResult, error) { + state.Status = "done" + if err := lib.WriteSessionJSON(s.dataDir, state); err != nil { + return mcpmcp.NewToolResultError(fmt.Sprintf("write final state: %v", err)), nil + } + if s.db != nil { + if err := s.db.UpdateSessionStatus(state.ID, "done"); err != nil { + fmt.Fprintf(os.Stderr, "warning: db update session: %v\n", err) + } + } + + if state.Branch && s.git != nil { + s.git.CommitAll(fmt.Sprintf("%s(%s): complete", state.Workflow, state.ID)) + } + + if err := lib.ClearSessionJSON(s.dataDir); err != nil { + fmt.Fprintf(os.Stderr, "warning: clear session: %v\n", err) + } + return mcpmcp.NewToolResultText(fmt.Sprintf("=== WORKFLOW COMPLETE ===\nSession: %s\nSteps completed: %d", state.ID, state.TotalSteps)), nil +} + +func (s *Server) runCommand(ctx context.Context, command string) (string, int, error) { + ctx, cancel := context.WithTimeout(ctx, commandTimeout) + defer cancel() + + cmd := exec.CommandContext(ctx, "sh", "-c", command) + cmd.Dir = s.repoRoot + var out bytes.Buffer + cmd.Stdout = &out + cmd.Stderr = &out + err := cmd.Run() + exitCode := 0 + if err != nil { + var exitErr *exec.ExitError + if errors.As(err, &exitErr) { + exitCode = exitErr.ExitCode() + } else { + return "", 1, fmt.Errorf("command execution failed: %w", err) + } + } + return out.String(), exitCode, nil +} + +func (s *Server) handleLoopAdvance(ctx context.Context, wf *engine.Workflow, state *lib.SessionState, step *engine.WfStep, req mcpmcp.CallToolRequest) (*mcpmcp.CallToolResult, error) { + // Initialize loop tracking on first call + if state.LoopMax == 0 { + state.LoopMax = step.Loop.Max + if state.LoopMax == 0 { + state.LoopMax = 10 // default max + } + } + state.LoopIteration++ + + // Check gate command if present + if step.Loop.Gate != "" { + gateCmd := engine.Interpolate(step.Loop.Gate, state.Input, state.Outputs) + _, exitCode, err := s.runCommand(ctx, gateCmd) + if err != nil { + return mcpmcp.NewToolResultError(fmt.Sprintf("gate command failed: %v", err)), nil + } + if exitCode == 0 { + // Gate passed — advance past loop + return s.advancePastLoop(wf, state) + } + // Gate failed — continue loop + } + + // Check "until" condition + if step.Loop.Until != "" { + if output, ok := state.Outputs[step.ID]; ok { + if strings.Contains(strings.ToLower(output), strings.ToLower(step.Loop.Until)) { + return s.advancePastLoop(wf, state) + } + } + } + + // Check max iterations + if state.LoopIteration >= state.LoopMax { + return s.advancePastLoop(wf, state) + } + + // Continue loop — return same step for another iteration + if err := lib.WriteSessionJSON(s.dataDir, state); err != nil { + return mcpmcp.NewToolResultError(fmt.Sprintf("write loop state: %v", err)), nil + } + response := fmt.Sprintf("=== LOOP ITERATION %d/%d: %s ===\n", state.LoopIteration, state.LoopMax, step.ID) + response += s.formatStepResponse(wf, state, step, state.Input) + return mcpmcp.NewToolResultText(response), nil +} + +func (s *Server) advancePastLoop(wf *engine.Workflow, state *lib.SessionState) (*mcpmcp.CallToolResult, error) { + nextIndex := state.CurrentIndex + 1 + // Reset loop state + state.LoopIteration = 0 + state.LoopMax = 0 + + if nextIndex >= len(wf.Steps) { + return s.completeWorkflow(state) + } + + nextStep := wf.Steps[nextIndex] + state.CurrentStep = nextStep.ID + state.CurrentIndex = nextIndex + state.StepType = stepType(nextStep) + if err := lib.WriteSessionJSON(s.dataDir, state); err != nil { + return mcpmcp.NewToolResultError(fmt.Sprintf("write state: %v", err)), nil + } + + response := s.formatStepResponse(wf, state, &nextStep, state.Input) + return mcpmcp.NewToolResultText(response), nil +} diff --git a/src/mcp/tools_test.go b/src/mcp/tools_test.go new file mode 100644 index 0000000..faa2ff0 --- /dev/null +++ b/src/mcp/tools_test.go @@ -0,0 +1,956 @@ +package mcp + +import ( + "context" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/5uck1ess/devkit/lib" + mcpmcp "github.com/mark3labs/mcp-go/mcp" +) + +// newTestServer builds a minimal Server with the given dirs — no DB needed for tool tests. +func newTestServer(t *testing.T, dataDir, workflowDir string) *Server { + t.Helper() + return &Server{ + dataDir: dataDir, + workflowDir: workflowDir, + repoRoot: t.TempDir(), + db: nil, + git: nil, + principles: map[string][]string{}, + } +} + +func callTool(t *testing.T, handler func(context.Context, mcpmcp.CallToolRequest) (*mcpmcp.CallToolResult, error)) string { + t.Helper() + result, err := handler(context.Background(), mcpmcp.CallToolRequest{}) + if err != nil { + t.Fatalf("handler returned error: %v", err) + } + if result == nil || len(result.Content) == 0 { + t.Fatal("handler returned empty result") + } + // Extract text from first content item. + if tc, ok := result.Content[0].(mcpmcp.TextContent); ok { + return tc.Text + } + t.Fatalf("unexpected content type: %T", result.Content[0]) + return "" +} + +func TestList(t *testing.T) { + dir := t.TempDir() + + // Write two valid workflow YAMLs and one non-YAML file. + writeFile(t, filepath.Join(dir, "alpha.yml"), `name: alpha +description: Alpha workflow +steps: + - id: step-one + prompt: do something +`) + writeFile(t, filepath.Join(dir, "beta.yaml"), `name: beta +description: Beta workflow +steps: + - id: step-one + prompt: do something +`) + writeFile(t, filepath.Join(dir, "readme.txt"), "ignore me") + + srv := newTestServer(t, t.TempDir(), dir) + _, handler := srv.listTool() + out := callTool(t, handler) + + if !strings.Contains(out, "alpha") { + t.Errorf("expected 'alpha' in output, got: %s", out) + } + if !strings.Contains(out, "Alpha workflow") { + t.Errorf("expected description 'Alpha workflow' in output, got: %s", out) + } + if !strings.Contains(out, "beta") { + t.Errorf("expected 'beta' in output, got: %s", out) + } + if strings.Contains(out, "readme") { + t.Errorf("non-YAML file should not appear in output, got: %s", out) + } +} + +func TestListParseError(t *testing.T) { + dir := t.TempDir() + writeFile(t, filepath.Join(dir, "broken.yml"), "{{{{ not valid yaml ~~~~") + + srv := newTestServer(t, t.TempDir(), dir) + _, handler := srv.listTool() + out := callTool(t, handler) + + if !strings.Contains(out, "broken") { + t.Errorf("expected 'broken' in output, got: %s", out) + } + if !strings.Contains(out, "parse error") { + t.Errorf("expected 'parse error' in output, got: %s", out) + } +} + +func TestListMissingDir(t *testing.T) { + srv := newTestServer(t, t.TempDir(), "/nonexistent/path/workflows") + _, handler := srv.listTool() + result, err := handler(context.Background(), mcpmcp.CallToolRequest{}) + if err != nil { + t.Fatalf("handler returned unexpected error: %v", err) + } + if !result.IsError { + t.Error("expected IsError=true for missing workflow dir") + } +} + +func TestStatusNoSession(t *testing.T) { + dataDir := t.TempDir() + srv := newTestServer(t, dataDir, t.TempDir()) + _, handler := srv.statusTool() + out := callTool(t, handler) + + if !strings.Contains(out, "No active workflow session") { + t.Errorf("expected no-session message, got: %s", out) + } +} + +func TestStatusWithSession(t *testing.T) { + dataDir := t.TempDir() + + state := &lib.SessionState{ + ID: "sess-123", + Workflow: "my-workflow", + CurrentStep: "step-review", + CurrentIndex: 2, + TotalSteps: 5, + Enforce: "hard", + Status: "running", + StartedAt: time.Now(), + Outputs: map[string]string{}, + } + if err := lib.WriteSessionJSON(dataDir, state); err != nil { + t.Fatalf("write session: %v", err) + } + + srv := newTestServer(t, dataDir, t.TempDir()) + _, handler := srv.statusTool() + out := callTool(t, handler) + + checks := []struct { + field string + value string + }{ + {"workflow name", "my-workflow"}, + {"session ID", "sess-123"}, + {"step name", "step-review"}, + {"step progress", "3/5"}, + {"enforce", "hard"}, + {"status", "running"}, + } + for _, c := range checks { + if !strings.Contains(out, c.value) { + t.Errorf("expected %s (%q) in output, got:\n%s", c.field, c.value, out) + } + } +} + +func TestStart(t *testing.T) { + wfDir := t.TempDir() + dataDir := t.TempDir() + + writeFile(t, filepath.Join(wfDir, "review.yml"), `name: review +description: Code review workflow +steps: + - id: analyse + prompt: Analyse {{input}} and identify issues. + - id: report + prompt: Write a report based on the analysis. +`) + + srv := newTestServer(t, dataDir, wfDir) + _, handler := srv.startTool() + + req := mcpmcp.CallToolRequest{} + req.Params.Arguments = map[string]interface{}{ + "workflow": "review", + "input": "main.go", + } + + result, err := handler(context.Background(), req) + if err != nil { + t.Fatalf("handler returned error: %v", err) + } + if result == nil || len(result.Content) == 0 { + t.Fatal("handler returned empty result") + } + if result.IsError { + tc, _ := result.Content[0].(mcpmcp.TextContent) + t.Fatalf("handler returned tool error: %s", tc.Text) + } + + tc, ok := result.Content[0].(mcpmcp.TextContent) + if !ok { + t.Fatalf("unexpected content type: %T", result.Content[0]) + } + out := tc.Text + + // Response should mention step 1 + if !strings.Contains(out, "STEP 1/2") { + t.Errorf("expected 'STEP 1/2' in response, got:\n%s", out) + } + if !strings.Contains(out, "analyse") { + t.Errorf("expected step id 'analyse' in response, got:\n%s", out) + } + if !strings.Contains(out, "main.go") { + t.Errorf("expected interpolated input 'main.go' in response, got:\n%s", out) + } + if !strings.Contains(out, "devkit_advance") { + t.Errorf("expected 'devkit_advance' call-to-action in response, got:\n%s", out) + } + + // session.json should exist + state, err := lib.ReadSessionJSON(dataDir) + if err != nil { + t.Fatalf("read session: %v", err) + } + if state == nil { + t.Fatal("session.json was not created") + } + if state.Workflow != "review" { + t.Errorf("expected workflow 'review', got %q", state.Workflow) + } + if state.CurrentStep != "analyse" { + t.Errorf("expected current_step 'analyse', got %q", state.CurrentStep) + } + if state.CurrentIndex != 0 { + t.Errorf("expected current_index 0, got %d", state.CurrentIndex) + } + if state.TotalSteps != 2 { + t.Errorf("expected total_steps 2, got %d", state.TotalSteps) + } + if state.Status != "running" { + t.Errorf("expected status 'running', got %q", state.Status) + } + if state.Input != "main.go" { + t.Errorf("expected input 'main.go', got %q", state.Input) + } + if state.StepType != "prompt" { + t.Errorf("expected step_type 'prompt', got %q", state.StepType) + } +} + +func TestStartAlreadyRunning(t *testing.T) { + wfDir := t.TempDir() + dataDir := t.TempDir() + + writeFile(t, filepath.Join(wfDir, "review.yml"), `name: review +description: Code review workflow +steps: + - id: analyse + prompt: Analyse {{input}} and identify issues. +`) + + // Pre-seed a running session + existing := &lib.SessionState{ + ID: "abc123", + Workflow: "review", + Status: "running", + StartedAt: time.Now(), + Outputs: map[string]string{}, + } + if err := lib.WriteSessionJSON(dataDir, existing); err != nil { + t.Fatalf("write session: %v", err) + } + + srv := newTestServer(t, dataDir, wfDir) + _, handler := srv.startTool() + + req := mcpmcp.CallToolRequest{} + req.Params.Arguments = map[string]interface{}{ + "workflow": "review", + "input": "main.go", + } + + result, err := handler(context.Background(), req) + if err != nil { + t.Fatalf("handler returned error: %v", err) + } + if !result.IsError { + t.Error("expected IsError=true when session already running") + } + tc, ok := result.Content[0].(mcpmcp.TextContent) + if !ok { + t.Fatalf("unexpected content type: %T", result.Content[0]) + } + if !strings.Contains(tc.Text, "already running") { + t.Errorf("expected 'already running' in error, got: %s", tc.Text) + } +} + +func TestAdvancePromptSteps(t *testing.T) { + wfDir := t.TempDir() + dataDir := t.TempDir() + + writeFile(t, filepath.Join(wfDir, "three-step.yml"), `name: three-step +description: Three prompt steps +steps: + - id: plan + prompt: Plan the work for {{input}}. + - id: implement + prompt: Implement the plan. + - id: verify + prompt: Verify everything works. +`) + + srv := newTestServer(t, dataDir, wfDir) + + // Start workflow to seed session.json + _, startHandler := srv.startTool() + startReq := mcpmcp.CallToolRequest{} + startReq.Params.Arguments = map[string]interface{}{ + "workflow": "three-step", + "input": "widget feature", + } + startResult, err := startHandler(context.Background(), startReq) + if err != nil { + t.Fatalf("start: %v", err) + } + if startResult.IsError { + t.Fatalf("start returned error") + } + + // Read session to get ID + state, err := lib.ReadSessionJSON(dataDir) + if err != nil || state == nil { + t.Fatalf("read session after start: %v", err) + } + sessionID := state.ID + + _, advHandler := srv.advanceTool() + + // Advance 1: plan -> implement + advReq := mcpmcp.CallToolRequest{} + advReq.Params.Arguments = map[string]interface{}{ + "session": sessionID, + "output": "plan output here", + } + result, err := advHandler(context.Background(), advReq) + if err != nil { + t.Fatalf("advance 1: %v", err) + } + if result.IsError { + tc, _ := result.Content[0].(mcpmcp.TextContent) + t.Fatalf("advance 1 error: %s", tc.Text) + } + tc, _ := result.Content[0].(mcpmcp.TextContent) + if !strings.Contains(tc.Text, "STEP 2/3") { + t.Errorf("advance 1: expected STEP 2/3, got:\n%s", tc.Text) + } + if !strings.Contains(tc.Text, "implement") { + t.Errorf("advance 1: expected step id 'implement', got:\n%s", tc.Text) + } + + // Verify output was captured + state, _ = lib.ReadSessionJSON(dataDir) + if state.Outputs["plan"] != "plan output here" { + t.Errorf("expected plan output captured, got %q", state.Outputs["plan"]) + } + + // Advance 2: implement -> verify + advReq2 := mcpmcp.CallToolRequest{} + advReq2.Params.Arguments = map[string]interface{}{ + "session": sessionID, + "output": "implementation done", + } + result2, err := advHandler(context.Background(), advReq2) + if err != nil { + t.Fatalf("advance 2: %v", err) + } + if result2.IsError { + tc2, _ := result2.Content[0].(mcpmcp.TextContent) + t.Fatalf("advance 2 error: %s", tc2.Text) + } + tc2, _ := result2.Content[0].(mcpmcp.TextContent) + if !strings.Contains(tc2.Text, "STEP 3/3") { + t.Errorf("advance 2: expected STEP 3/3, got:\n%s", tc2.Text) + } + + // Advance 3: verify -> complete + advReq3 := mcpmcp.CallToolRequest{} + advReq3.Params.Arguments = map[string]interface{}{ + "session": sessionID, + "output": "all verified", + } + result3, err := advHandler(context.Background(), advReq3) + if err != nil { + t.Fatalf("advance 3: %v", err) + } + if result3.IsError { + tc3, _ := result3.Content[0].(mcpmcp.TextContent) + t.Fatalf("advance 3 error: %s", tc3.Text) + } + tc3, _ := result3.Content[0].(mcpmcp.TextContent) + if !strings.Contains(tc3.Text, "WORKFLOW COMPLETE") { + t.Errorf("advance 3: expected WORKFLOW COMPLETE, got:\n%s", tc3.Text) + } + if !strings.Contains(tc3.Text, sessionID) { + t.Errorf("advance 3: expected session ID in output, got:\n%s", tc3.Text) + } + + // session.json should be cleared + cleared, _ := lib.ReadSessionJSON(dataDir) + if cleared != nil { + t.Errorf("expected session.json cleared after completion, but state still exists") + } +} + +func TestAdvanceCommandStep(t *testing.T) { + wfDir := t.TempDir() + dataDir := t.TempDir() + + writeFile(t, filepath.Join(wfDir, "cmd-wf.yml"), `name: cmd-wf +description: Command workflow +steps: + - id: greet + command: echo hello + expect: success + - id: done + prompt: Summarise. +`) + + srv := newTestServer(t, dataDir, wfDir) + + // Start + _, startHandler := srv.startTool() + startReq := mcpmcp.CallToolRequest{} + startReq.Params.Arguments = map[string]interface{}{ + "workflow": "cmd-wf", + "input": "test", + } + startResult, err := startHandler(context.Background(), startReq) + if err != nil { + t.Fatalf("start: %v", err) + } + if startResult.IsError { + tc, _ := startResult.Content[0].(mcpmcp.TextContent) + t.Fatalf("start error: %s", tc.Text) + } + + state, _ := lib.ReadSessionJSON(dataDir) + sessionID := state.ID + + // Advance: should execute "echo hello" and move to next step + _, advHandler := srv.advanceTool() + advReq := mcpmcp.CallToolRequest{} + advReq.Params.Arguments = map[string]interface{}{ + "session": sessionID, + } + result, err := advHandler(context.Background(), advReq) + if err != nil { + t.Fatalf("advance: %v", err) + } + if result.IsError { + tc, _ := result.Content[0].(mcpmcp.TextContent) + t.Fatalf("advance error: %s", tc.Text) + } + tc, _ := result.Content[0].(mcpmcp.TextContent) + if !strings.Contains(tc.Text, "STEP 2/2") { + t.Errorf("expected STEP 2/2, got:\n%s", tc.Text) + } + + // Verify command output was captured + state, _ = lib.ReadSessionJSON(dataDir) + if !strings.Contains(state.Outputs["greet"], "hello") { + t.Errorf("expected 'hello' in command output, got %q", state.Outputs["greet"]) + } +} + +func TestAdvanceSessionMismatch(t *testing.T) { + wfDir := t.TempDir() + dataDir := t.TempDir() + + writeFile(t, filepath.Join(wfDir, "simple.yml"), `name: simple +description: Simple workflow +steps: + - id: one + prompt: Do something. +`) + + srv := newTestServer(t, dataDir, wfDir) + + // Start workflow + _, startHandler := srv.startTool() + startReq := mcpmcp.CallToolRequest{} + startReq.Params.Arguments = map[string]interface{}{ + "workflow": "simple", + "input": "test", + } + startHandler(context.Background(), startReq) + + // Advance with wrong session ID + _, advHandler := srv.advanceTool() + advReq := mcpmcp.CallToolRequest{} + advReq.Params.Arguments = map[string]interface{}{ + "session": "wrong-session-id", + } + result, err := advHandler(context.Background(), advReq) + if err != nil { + t.Fatalf("advance: %v", err) + } + if !result.IsError { + t.Error("expected IsError=true for session mismatch") + } + tc, _ := result.Content[0].(mcpmcp.TextContent) + if !strings.Contains(tc.Text, "session mismatch") { + t.Errorf("expected 'session mismatch' in error, got: %s", tc.Text) + } +} + +func TestLoopMaxIterations(t *testing.T) { + wfDir := t.TempDir() + dataDir := t.TempDir() + + writeFile(t, filepath.Join(wfDir, "loop-wf.yml"), `name: loop-wf +description: Loop workflow +steps: + - id: iterate + prompt: Do iteration work. + loop: + max: 3 + - id: finish + prompt: Finish up. +`) + + srv := newTestServer(t, dataDir, wfDir) + + // Start workflow + _, startHandler := srv.startTool() + startReq := mcpmcp.CallToolRequest{} + startReq.Params.Arguments = map[string]interface{}{ + "workflow": "loop-wf", + "input": "test input", + } + startResult, err := startHandler(context.Background(), startReq) + if err != nil || startResult.IsError { + t.Fatalf("start failed") + } + + state, _ := lib.ReadSessionJSON(dataDir) + sessionID := state.ID + + _, advHandler := srv.advanceTool() + + advReq := func() mcpmcp.CallToolRequest { + r := mcpmcp.CallToolRequest{} + r.Params.Arguments = map[string]interface{}{ + "session": sessionID, + "output": "did some work", + } + return r + } + + // Advance 1: still on iterate (iteration 1/3) + result, err := advHandler(context.Background(), advReq()) + if err != nil { + t.Fatalf("advance 1: %v", err) + } + if result.IsError { + tc, _ := result.Content[0].(mcpmcp.TextContent) + t.Fatalf("advance 1 error: %s", tc.Text) + } + tc, _ := result.Content[0].(mcpmcp.TextContent) + if !strings.Contains(tc.Text, "LOOP ITERATION 1/3") { + t.Errorf("advance 1: expected LOOP ITERATION 1/3, got:\n%s", tc.Text) + } + if !strings.Contains(tc.Text, "iterate") { + t.Errorf("advance 1: expected step 'iterate' still active, got:\n%s", tc.Text) + } + + // Advance 2: still on iterate (iteration 2/3) + result2, err := advHandler(context.Background(), advReq()) + if err != nil { + t.Fatalf("advance 2: %v", err) + } + tc2, _ := result2.Content[0].(mcpmcp.TextContent) + if !strings.Contains(tc2.Text, "LOOP ITERATION 2/3") { + t.Errorf("advance 2: expected LOOP ITERATION 2/3, got:\n%s", tc2.Text) + } + + // Advance 3: max reached — should advance to finish + result3, err := advHandler(context.Background(), advReq()) + if err != nil { + t.Fatalf("advance 3: %v", err) + } + if result3.IsError { + tc3, _ := result3.Content[0].(mcpmcp.TextContent) + t.Fatalf("advance 3 error: %s", tc3.Text) + } + tc3, _ := result3.Content[0].(mcpmcp.TextContent) + if !strings.Contains(tc3.Text, "finish") { + t.Errorf("advance 3: expected step 'finish', got:\n%s", tc3.Text) + } + if strings.Contains(tc3.Text, "LOOP ITERATION") { + t.Errorf("advance 3: should have left loop, got:\n%s", tc3.Text) + } + + // Loop state should be reset + state, _ = lib.ReadSessionJSON(dataDir) + if state.LoopIteration != 0 { + t.Errorf("expected LoopIteration reset to 0, got %d", state.LoopIteration) + } + if state.LoopMax != 0 { + t.Errorf("expected LoopMax reset to 0, got %d", state.LoopMax) + } +} + +func TestLoopGatePass(t *testing.T) { + wfDir := t.TempDir() + dataDir := t.TempDir() + + writeFile(t, filepath.Join(wfDir, "gate-pass.yml"), `name: gate-pass +description: Gate pass workflow +steps: + - id: check + prompt: Do the check. + loop: + max: 5 + gate: "true" + - id: next + prompt: Next step. +`) + + srv := newTestServer(t, dataDir, wfDir) + + _, startHandler := srv.startTool() + startReq := mcpmcp.CallToolRequest{} + startReq.Params.Arguments = map[string]interface{}{ + "workflow": "gate-pass", + "input": "test", + } + startResult, err := startHandler(context.Background(), startReq) + if err != nil || startResult.IsError { + t.Fatalf("start failed") + } + + state, _ := lib.ReadSessionJSON(dataDir) + sessionID := state.ID + + _, advHandler := srv.advanceTool() + advReq := mcpmcp.CallToolRequest{} + advReq.Params.Arguments = map[string]interface{}{ + "session": sessionID, + "output": "check output", + } + + // First advance: gate "true" exits 0 — should pass and advance to next + result, err := advHandler(context.Background(), advReq) + if err != nil { + t.Fatalf("advance: %v", err) + } + if result.IsError { + tc, _ := result.Content[0].(mcpmcp.TextContent) + t.Fatalf("advance error: %s", tc.Text) + } + tc, _ := result.Content[0].(mcpmcp.TextContent) + if !strings.Contains(tc.Text, "next") { + t.Errorf("expected 'next' step after gate pass, got:\n%s", tc.Text) + } + if strings.Contains(tc.Text, "LOOP ITERATION") { + t.Errorf("should not see LOOP ITERATION header when gate passes, got:\n%s", tc.Text) + } +} + +func TestLoopGateFail(t *testing.T) { + wfDir := t.TempDir() + dataDir := t.TempDir() + + writeFile(t, filepath.Join(wfDir, "gate-fail.yml"), `name: gate-fail +description: Gate fail workflow +steps: + - id: retry + prompt: Try again. + loop: + max: 5 + gate: "false" + - id: done + prompt: Done. +`) + + srv := newTestServer(t, dataDir, wfDir) + + _, startHandler := srv.startTool() + startReq := mcpmcp.CallToolRequest{} + startReq.Params.Arguments = map[string]interface{}{ + "workflow": "gate-fail", + "input": "test", + } + startResult, err := startHandler(context.Background(), startReq) + if err != nil || startResult.IsError { + t.Fatalf("start failed") + } + + state, _ := lib.ReadSessionJSON(dataDir) + sessionID := state.ID + + _, advHandler := srv.advanceTool() + advReq := mcpmcp.CallToolRequest{} + advReq.Params.Arguments = map[string]interface{}{ + "session": sessionID, + "output": "attempt output", + } + + // Advance: gate "false" exits 1 — should stay on loop step + result, err := advHandler(context.Background(), advReq) + if err != nil { + t.Fatalf("advance: %v", err) + } + if result.IsError { + tc, _ := result.Content[0].(mcpmcp.TextContent) + t.Fatalf("advance error: %s", tc.Text) + } + tc, _ := result.Content[0].(mcpmcp.TextContent) + if !strings.Contains(tc.Text, "LOOP ITERATION 1/5") { + t.Errorf("expected LOOP ITERATION 1/5 (still looping), got:\n%s", tc.Text) + } + if !strings.Contains(tc.Text, "retry") { + t.Errorf("expected step 'retry' still active, got:\n%s", tc.Text) + } + if strings.Contains(tc.Text, "done") && !strings.Contains(tc.Text, "retry") { + t.Errorf("should not have advanced to 'done', got:\n%s", tc.Text) + } +} + +// --- Security and edge-case tests --- + +func TestStartPathTraversal(t *testing.T) { + wfDir := t.TempDir() + dataDir := t.TempDir() + srv := newTestServer(t, dataDir, wfDir) + _, handler := srv.startTool() + + cases := []struct { + name string + wf string + }{ + {"dot-dot", "../etc/passwd"}, + {"slash", "foo/bar"}, + {"backslash", `foo\bar`}, + {"dot-dot-no-slash", "..secret"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + req := mcpmcp.CallToolRequest{} + req.Params.Arguments = map[string]interface{}{ + "workflow": tc.wf, + "input": "test", + } + result, err := handler(context.Background(), req) + if err != nil { + t.Fatalf("handler error: %v", err) + } + if !result.IsError { + t.Error("expected tool error for path traversal attempt") + } + }) + } +} + +func TestStartWorkflowNotFound(t *testing.T) { + wfDir := t.TempDir() + dataDir := t.TempDir() + srv := newTestServer(t, dataDir, wfDir) + _, handler := srv.startTool() + + req := mcpmcp.CallToolRequest{} + req.Params.Arguments = map[string]interface{}{ + "workflow": "nonexistent", + "input": "test", + } + result, err := handler(context.Background(), req) + if err != nil { + t.Fatalf("handler error: %v", err) + } + if !result.IsError { + t.Error("expected tool error for nonexistent workflow") + } +} + +func TestAdvanceNoSession(t *testing.T) { + wfDir := t.TempDir() + dataDir := t.TempDir() + srv := newTestServer(t, dataDir, wfDir) + _, handler := srv.advanceTool() + + req := mcpmcp.CallToolRequest{} + req.Params.Arguments = map[string]interface{}{ + "session": "nonexistent", + } + result, err := handler(context.Background(), req) + if err != nil { + t.Fatalf("handler error: %v", err) + } + if !result.IsError { + t.Error("expected tool error for no active session") + } + tc, _ := result.Content[0].(mcpmcp.TextContent) + if !strings.Contains(tc.Text, "no active session") { + t.Errorf("expected 'no active session' error, got: %s", tc.Text) + } +} + +func TestAdvanceExpectSuccessWithFailure(t *testing.T) { + wfDir := t.TempDir() + dataDir := t.TempDir() + + writeFile(t, filepath.Join(wfDir, "test.yml"), `name: test +steps: + - id: check + command: "exit 1" + expect: success + - id: done + prompt: done +`) + + srv := newTestServer(t, dataDir, wfDir) + + // Start the workflow + _, startHandler := srv.startTool() + startReq := mcpmcp.CallToolRequest{} + startReq.Params.Arguments = map[string]interface{}{ + "workflow": "test", + "input": "test", + } + startResult, _ := startHandler(context.Background(), startReq) + if startResult.IsError { + t.Fatalf("start failed: %v", startResult.Content) + } + + // Read session to get ID + state, _ := lib.ReadSessionJSON(dataDir) + + // Advance — command exits 1 but expect is "success", should error + _, advHandler := srv.advanceTool() + advReq := mcpmcp.CallToolRequest{} + advReq.Params.Arguments = map[string]interface{}{ + "session": state.ID, + } + result, err := advHandler(context.Background(), advReq) + if err != nil { + t.Fatalf("handler error: %v", err) + } + if !result.IsError { + t.Error("expected tool error for expect:success with exit 1") + } + tc, _ := result.Content[0].(mcpmcp.TextContent) + if !strings.Contains(tc.Text, "expected success") { + t.Errorf("expected 'expected success' in error, got: %s", tc.Text) + } +} + +func TestAdvanceExpectFailureWithSuccess(t *testing.T) { + wfDir := t.TempDir() + dataDir := t.TempDir() + + writeFile(t, filepath.Join(wfDir, "test.yml"), `name: test +steps: + - id: check + command: "exit 0" + expect: failure + - id: done + prompt: done +`) + + srv := newTestServer(t, dataDir, wfDir) + + _, startHandler := srv.startTool() + startReq := mcpmcp.CallToolRequest{} + startReq.Params.Arguments = map[string]interface{}{ + "workflow": "test", + "input": "test", + } + startHandler(context.Background(), startReq) + + state, _ := lib.ReadSessionJSON(dataDir) + + _, advHandler := srv.advanceTool() + advReq := mcpmcp.CallToolRequest{} + advReq.Params.Arguments = map[string]interface{}{ + "session": state.ID, + } + result, err := advHandler(context.Background(), advReq) + if err != nil { + t.Fatalf("handler error: %v", err) + } + if !result.IsError { + t.Error("expected tool error for expect:failure with exit 0") + } + tc, _ := result.Content[0].(mcpmcp.TextContent) + if !strings.Contains(tc.Text, "expected failure") { + t.Errorf("expected 'expected failure' in error, got: %s", tc.Text) + } +} + +func TestLoopUntilCondition(t *testing.T) { + wfDir := t.TempDir() + dataDir := t.TempDir() + + writeFile(t, filepath.Join(wfDir, "test.yml"), `name: test +steps: + - id: fix + prompt: "Fix the issue" + loop: + max: 10 + until: "all clear" + - id: done + prompt: done +`) + + srv := newTestServer(t, dataDir, wfDir) + + // Start + _, startHandler := srv.startTool() + startReq := mcpmcp.CallToolRequest{} + startReq.Params.Arguments = map[string]interface{}{ + "workflow": "test", + "input": "test", + } + startHandler(context.Background(), startReq) + state, _ := lib.ReadSessionJSON(dataDir) + + _, advHandler := srv.advanceTool() + + // First advance — output does NOT contain "all clear", should stay in loop + advReq := mcpmcp.CallToolRequest{} + advReq.Params.Arguments = map[string]interface{}{ + "session": state.ID, + "output": "still broken", + } + result, _ := advHandler(context.Background(), advReq) + tc, _ := result.Content[0].(mcpmcp.TextContent) + if !strings.Contains(tc.Text, "LOOP ITERATION") { + t.Errorf("expected loop iteration, got:\n%s", tc.Text) + } + + // Second advance — output contains "all clear", should exit loop + advReq2 := mcpmcp.CallToolRequest{} + advReq2.Params.Arguments = map[string]interface{}{ + "session": state.ID, + "output": "All Clear now", + } + result2, _ := advHandler(context.Background(), advReq2) + tc2, _ := result2.Content[0].(mcpmcp.TextContent) + if !strings.Contains(tc2.Text, "done") { + t.Errorf("expected to advance past loop to 'done', got:\n%s", tc2.Text) + } +} + +// writeFile is a test helper that creates a file with the given content. +func writeFile(t *testing.T, path, content string) { + t.Helper() + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { + t.Fatalf("writeFile %s: %v", path, err) + } +} diff --git a/workflows/pr-ready.yml b/workflows/pr-ready.yml index f9f79a0..b9cb517 100644 --- a/workflows/pr-ready.yml +++ b/workflows/pr-ready.yml @@ -68,7 +68,92 @@ steps: prompt: | Create the PR: 1. Push the branch to remote - 2. Create PR with gh pr create - 3. Include: title, summary, changelog, test plan + 2. Create PR with `gh pr create` including title, summary, changelog, + test plan (use the changelog from the previous step) + 3. On the LAST LINE of your output, print exactly: + PR: + where is the numeric PR id. No other text on that line. - Use the changelog from the previous step. + If pushing or PR creation fails, print: + PR: FAILED + on the last line and stop. + + - id: monitor + model: smart + prompt: | + The PR was just created. Watch it until merge-ready. + + Previous step output (contains `PR: ` on its last line): + {{create-pr}} + + STEP 0 — Extract PR number: + - Parse the last line matching `PR: `. + - If the line says `PR: FAILED ...` or no number is present, + output exactly: `all resolved (create-pr failed, nothing to monitor)` + and stop immediately. + + Each iteration: + + 1. Wait up to 3 minutes for CI checks and auto-reviewers (Copilot, Gemini, + CodeRabbit). Poll `gh pr checks ` every 10s; break when no + PENDING/QUEUED checks remain. If `gh` exits non-zero, retry once after + 10s; if it fails again, output exactly: + all resolved (gh api unreachable — manual review required) + and stop. + + 2. Fetch unresolved state (each command: retry once on non-zero exit, + then abort with `all resolved (gh api failure on )`): + - `gh pr checks --json name,state,conclusion` + - `gh api repos/{owner}/{repo}/pulls//comments --paginate` + - `gh api repos/{owner}/{repo}/pulls//reviews --paginate` + + 3. Classify check states strictly: + - COMPLETED + SUCCESS/NEUTRAL/SKIPPED → ok + - COMPLETED + FAILURE/CANCELLED/TIMED_OUT → failed (needs fix) + - QUEUED/IN_PROGRESS/PENDING → not done yet (do NOT treat as resolved) + If any checks are still pending at this point, continue polling — do + not advance to completion check. + + 4. For each failed check, read `gh run view --log-failed | tail -50` + and attempt a fix. + + 5. Classify each unresolved comment via the `reviewer` agent: + - code_fix → read file, apply fix, commit + - style_nit → apply if trivial, skip with reply if subjective + - question → reply with context from codebase + - false_positive → reply with evidence, do NOT dismiss + - out_of_scope → acknowledge, note for future + + 6. Apply code fixes, commit individually (`address review: `), + post replies via `gh api .../comments//replies`. + + 7. Push changes (never force-push), re-request review from original + reviewers. + + 8. Completion check — output `all resolved` ONLY when ALL of: + - Zero unresolved comments (REMAINING=0) + - Zero failed checks (FAILED_CHECKS=0) + - Zero pending checks (PENDING_CHECKS=0) + Otherwise print a one-line status (`iter N: P pending, F failed, + R remaining`) and continue. + + Stuck detection: if 3 consecutive iterations resolve zero comments AND + no CI failures remain AND no checks are pending, something is wrong with + classification. Output exactly: + all resolved (stuck — manual review required, iter ) + and stop. This also terminates the loop (engine matches `all resolved`). + + If the loop reaches `max` without terminating, the engine will halt with + status=loop_exhausted and the last iteration output is preserved in the + session log for inspection. + + Rules: + - Never force-push + - Never dismiss reviews — only re-request after fixing + - Never resolve threads you didn't actually address + - Reply to false positives with evidence, not dismissal + - Use AskUserQuestion if a comment is ambiguous + - Architectural changes → classify as out_of_scope, flag to user + loop: + max: 10 + until: "all resolved"