From 4fce8d0b28e1ac52df588b60c5cde49a8cf5197c Mon Sep 17 00:00:00 2001 From: Tym Rabchuk Date: Thu, 9 Apr 2026 19:42:36 -0400 Subject: [PATCH 1/3] Convert remaining 12 commands to thin wrappers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PR 4 of deterministic workflow conversion. Converts all remaining command files to thin wrappers with invoke + fallback pattern: Shipping: pr-ready, audit (no YAML yet — fallback only) Tri-agent: tri-review, tri-debug, tri-security, tri-dispatch, tri-test-gen Utility: autoloop, refactor, self-audit, decompose, repo-map 12 files, -1,767 lines of inline logic removed. All commands now follow consistent pattern: invoke → fallback → rules. --- commands/audit.md | 185 +++------------------------- commands/autoloop.md | 112 +++-------------- commands/decompose.md | 106 ++-------------- commands/pr-ready.md | 185 +++------------------------- commands/refactor.md | 78 +++--------- commands/repo-map.md | 256 +++------------------------------------ commands/self-audit.md | 197 +++--------------------------- commands/tri-debug.md | 167 +++---------------------- commands/tri-dispatch.md | 122 ++----------------- commands/tri-review.md | 200 +++--------------------------- commands/tri-security.md | 165 ++----------------------- commands/tri-test-gen.md | 179 +++------------------------ 12 files changed, 185 insertions(+), 1767 deletions(-) diff --git a/commands/audit.md b/commands/audit.md index 44c3977..d9b1574 100644 --- a/commands/audit.md +++ b/commands/audit.md @@ -1,182 +1,31 @@ --- -description: Unified project health audit — dependencies, vulnerabilities, outdated packages, licenses, lint, and security in one report. +description: Unified project health audit — dependencies, vulnerabilities, outdated packages, licenses, lint, and security. --- # Project Audit -Full-project health check in a single pass. Consolidates dependency vulnerabilities, outdated packages, license compliance, code quality, and security findings into one scored report. +Deterministic project health check: detect ecosystem → audit dependencies → check licenses → lint → security scan → report. -## Parameters - -1. **Target** — directory to audit (default: project root) -2. **Budget** — max USD (default: $1) - -## Budget - -- **Token budget:** ~100k tokens. Mostly tool output from audit commands. -- Prefer `haiku` for parsing/summarizing tool output, `sonnet` for the final report. - -## Step 1: Detect Ecosystem - -Detect which package managers and tools are present: - -```bash -echo "=== Ecosystem Detection ===" -[ -f package.json ] && echo "node: package.json found" -[ -f package-lock.json ] && echo "node: package-lock.json found" -[ -f yarn.lock ] && echo "node: yarn.lock found" -[ -f pnpm-lock.yaml ] && echo "node: pnpm-lock.yaml found" -[ -f requirements.txt ] && echo "python: requirements.txt found" -[ -f pyproject.toml ] && echo "python: pyproject.toml found" -[ -f Pipfile.lock ] && echo "python: Pipfile.lock found" -[ -f go.mod ] && echo "go: go.mod found" -[ -f Cargo.toml ] && echo "rust: Cargo.toml found" -[ -f Gemfile.lock ] && echo "ruby: Gemfile.lock found" -``` - -## Step 2: Dependency Vulnerabilities - -Run the appropriate audit tool for the detected ecosystem: - -| Ecosystem | Command | -|-----------|---------| -| Node (npm) | `npm audit --json 2>/dev/null` | -| Node (yarn) | `yarn audit --json 2>/dev/null` | -| Node (pnpm) | `pnpm audit --json 2>/dev/null` | -| Python | `pip-audit --format json 2>/dev/null` or `safety check --json 2>/dev/null` | -| Go | `govulncheck ./... 2>/dev/null` | -| Rust | `cargo audit --json 2>/dev/null` | -| Ruby | `bundle audit check 2>/dev/null` | - -If the audit tool isn't installed, note it as "skipped — {tool} not installed" and continue. - -Parse the JSON output and extract: -- Total vulnerabilities by severity (critical, high, moderate, low) -- Top 5 most severe findings with package name + advisory - -## Step 3: Outdated Packages - -```bash -# Node -npm outdated --json 2>/dev/null || true - -# Python -pip list --outdated --format json 2>/dev/null || true - -# Go -go list -m -u all 2>/dev/null | grep '\[' || true - -# Rust -cargo outdated --format json 2>/dev/null || true -``` - -Categorize as: -- **Major** — breaking version behind (e.g., v3 → v5) -- **Minor** — feature versions behind -- **Patch** — only patch versions behind - -## Step 4: License Compliance - -Check for potentially problematic licenses in dependencies: - -```bash -# Node -npx license-checker --json --production 2>/dev/null | head -200 || true - -# Python -pip-licenses --format json 2>/dev/null | head -200 || true -``` - -Flag: -- **Copyleft** licenses (GPL, AGPL, LGPL) in non-copyleft projects -- **Unknown** or missing licenses -- License conflicts with the project's own license (read LICENSE or package.json license field) - -If license tools aren't installed, skip with a note. - -## Step 5: Code Quality Summary - -Run whatever linter/type checker is configured: - -```bash -# Detect and run -[ -f .eslintrc* ] || [ -f eslint.config.* ] && npx eslint . --format json 2>/dev/null | head -100 -[ -f tsconfig.json ] && npx tsc --noEmit 2>&1 | tail -5 -[ -f pyproject.toml ] && (ruff check . --output-format json 2>/dev/null || flake8 . --format json 2>/dev/null) | head -100 -[ -f .golangci.yml ] && golangci-lint run --out-format json 2>/dev/null | head -100 -``` - -Count total errors and warnings. Don't fix anything — just report. - -## Step 6: Security Patterns - -Spawn the `security-auditor` agent for a quick scan: +## Invoke ``` -Task: Quick security scan of {target}. Check for: - - Hardcoded secrets, API keys, tokens - - SQL injection patterns - - XSS vulnerabilities - - Insecure dependencies usage - - Exposed debug/admin endpoints -Agent: security-auditor -Report: list of findings with severity and file:line +devkit workflow run audit ``` -## Step 7: Generate Report - -Produce a scored report: - -``` -## Project Audit Report - -**Project:** {name} -**Date:** {date} -**Overall Score:** {score}/100 - -### Scoring -| Category | Score | Weight | Details | -|----------|-------|--------|---------| -| Vulnerabilities | {0-100} | 30% | {critical}C {high}H {moderate}M {low}L | -| Dependencies | {0-100} | 20% | {major_outdated} major, {minor_outdated} minor behind | -| Licenses | {0-100} | 15% | {issues} issues found | -| Code Quality | {0-100} | 20% | {errors} errors, {warnings} warnings | -| Security | {0-100} | 15% | {findings} findings | - -### Critical Findings (fix immediately) -{list critical/high severity items across all categories} - -### Warnings (fix soon) -{list moderate items} - -### Informational -{list low-severity and suggestions} - -### Skipped Checks -{list any tools that weren't available and how to install them} - -### Recommended Actions -1. {highest priority fix} -2. {second priority} -3. {third priority} -``` - -## Scoring Guide - -- **Vulnerabilities:** Start at 100. -25 per critical, -10 per high, -3 per moderate, -1 per low. -- **Dependencies:** Start at 100. -5 per major-version-behind package, -1 per minor. -- **Licenses:** Start at 100. -20 per copyleft in non-copyleft project, -10 per unknown license. -- **Code Quality:** Start at 100. -2 per error, -0.5 per warning. Floor at 0. -- **Security:** Start at 100. -25 per critical, -10 per high, -3 per moderate. +If `devkit workflow` is not available, follow this manually: -Overall = weighted average, rounded to nearest integer. +1. **Detect ecosystem** — Check for go.mod, package.json, requirements.txt/pyproject.toml, Cargo.toml +2. **Dependency vulnerabilities** — Run ecosystem audit tools (`npm audit`, `go vuln`, `pip-audit`, `cargo audit`) +3. **Outdated packages** — Check for outdated dependencies, report major/minor/patch updates available +4. **License compliance** — Scan dependency licenses, flag non-permissive or unknown licenses +5. **Code quality** — Run linters for detected ecosystems, count errors and warnings +6. **Security patterns** — Grep for common vulnerabilities: hardcoded secrets, SQL injection, command injection, path traversal +7. **Generate report** — Health score (A-F) with breakdown by category, actionable recommendations ## Rules -- Never fix anything — audit only, report only -- Always continue if a tool is missing — skip and note it -- Use `--json` output formats where available for reliable parsing -- Cap command output with `head` to avoid context bloat -- Run ecosystem-specific checks only for detected ecosystems -- The security-auditor agent runs in worktree isolation -- Report should be actionable — every finding needs a "what to do" line +- Auto-detect ecosystem — don't ask the user what language +- Run actual audit tools — don't guess at vulnerabilities +- Score overall health: A (excellent) through F (critical issues) +- Prioritize findings by severity +- Token budget: ~100k tokens diff --git a/commands/autoloop.md b/commands/autoloop.md index 3f148b6..33f4951 100644 --- a/commands/autoloop.md +++ b/commands/autoloop.md @@ -1,108 +1,32 @@ --- -description: Autonomous improvement loop inspired by karpathy/autoresearch — audit, fix, measure, keep or revert, repeat. +description: Autonomous improvement loop — audit, fix, measure, keep or revert, repeat. --- # Autoloop -Autonomous codebase improvement loop. Each cycle: audit → pick hypothesis → fix → measure → keep or revert → repeat. +Autonomous iterative improvement: gather inputs → baseline → audit → fix → measure → compare → keep/revert → repeat. -Inspired by [karpathy/autoresearch](https://github.com/karpathy/autoresearch): one metric, keep or discard, loop. +## Invoke -## Step 0: Harness Detection - -```bash -if command -v devkit >/dev/null 2>&1; then - echo "Go harness detected — delegating to devkit workflow autoloop." - devkit workflow autoloop "{input}" - exit 0 -fi ``` - -## Step 1: Gather Inputs - -If the input doesn't contain a metric command, use `AskUserQuestion` to collect: - -1. **Objective** — what to improve -2. **Metric command** — how to measure (auto-detect from stack if not provided) -3. **Direction** — higher-is-better or lower-is-better -4. **Iterations** — how many cycles (default 10) -5. **Scope** — file/package constraints (optional) -6. **Guard** — optional safety-net command that must always pass (e.g., `npm test`, `tsc --noEmit`). Changes that improve the metric but break the guard are treated as regressions and reverted. Guard commands must be side-effect free (no generated files, snapshots, or caches that persist between runs). - -## Step 2: Baseline - -Run the metric command. Record the starting number and direction. - -If a guard command is set, run it now. If the guard fails at baseline, stop and tell the user — the invariant must hold before the loop can start. - -## Step 3: Audit - -Analyze the codebase for the single highest-impact change. Read the scratchpad to avoid repeating failed approaches. Output one hypothesis with target files. - -## Step 4: Fix - -Make the recommended change. Minimal, focused, no unrelated refactoring. - -## Step 5: Measure - -Run the EXACT same metric command. Record the new number. - -## Step 6: Compare - -Compare baseline vs measurement using the direction: -- higher-is-better: new > old → IMPROVED -- lower-is-better: new < old → IMPROVED -- Equal or failed → REGRESSED - -## Step 6.5: Guard Check (if guard command is set) - -Guard is only evaluated when the metric improved. Regressions are already reverted in Step 7, so running the guard on them would be wasted work. If the metric improved, run the guard with a timeout: - -```bash -timeout 120 {guard_command} 2>&1 +devkit workflow run autoloop "{metric_command}" ``` -- Guard passes (exit 0) → proceed to Step 7 as IMPROVED -- Guard fails or times out → treat as REGRESSED regardless of metric improvement. Log: "Metric improved but guard failed — reverting to protect invariant." - -## Step 7: Keep or Revert - -- **IMPROVED** → stage only modified files + `git commit`, update scratchpad, update baseline to new number, loop back to Step 3 -- **REGRESSED** → `git checkout -- ` (no `git clean -fd` — protect untracked work), update scratchpad with failure reason, increment per-file failure counter (see Escalation below), loop back to Step 3 - -### Escalation: Repeated Failures - -Track consecutive failures by primary modified file. After 3 failed attempts where the same file is the main edit target: - -1. **Log** what was tried and why each approach failed -2. **Skip** — move the hypothesis to a "blocked" list in the scratchpad -3. **Pivot** — choose a completely different hypothesis targeting different files -4. **Report** — include blocked items in the final report with context for manual investigation - -Never loop on the same failing approach. Each attempt must use a materially different strategy. - -## Step 8: Report - -After all iterations or budget exhausted: -- Starting vs final metric -- List of kept changes with impact -- List of reverted attempts with failure reason -- Net improvement -- Recommendation for next steps - -## Budget +If `devkit workflow` is not available, follow this manually: -- **Token budget:** ~500k tokens. Each cycle costs ~30-50k tokens. -- **Iteration limit:** User-specified (default 10). -- Budget or iteration limit, whichever hits first, stops the loop. +1. **Gather inputs** — Identify metric command, objective, guard command (optional), and iteration count +2. **Baseline** — Run metric command, capture starting state +3. **Audit** — Analyze codebase for improvement opportunities +4. **Fix** — Make one targeted improvement +5. **Measure** — Re-run metric command +6. **Compare** — Did the metric improve? Run guard command if set. +7. **Keep or revert** — Keep if improved and guard passes; revert otherwise. If 3+ consecutive failures, escalate. +8. **Report** — Summary of iterations, improvements kept, and final state ## Rules -- Every change must be measured — no skipping the metric step -- Never keep a regression — always revert -- One hypothesis at a time — no bundling -- Use the scratchpad to prevent repeating failures -- The metric command must be identical in baseline and measure steps -- Update the baseline number after each kept change (so the next cycle compares against the new state, not the original) -- Guard failures override metric improvements — never accept a change that breaks the guard -- 3 failures on the same file → skip and pivot, don't grind +- One change per iteration — don't bundle +- Always measure before and after +- Revert on regression or guard failure +- Stop after 3 consecutive failures and report +- Guard command (if set) must pass to keep a change diff --git a/commands/decompose.md b/commands/decompose.md index 89e53be..2e7f25e 100644 --- a/commands/decompose.md +++ b/commands/decompose.md @@ -4,105 +4,25 @@ description: Decompose a high-level goal into a task DAG — break down, assign # Goal Decomposition -Break a high-level goal into a dependency-ordered task graph, assign tasks to available agents, and execute. +Break a high-level goal into an executable task graph: clarify → decompose → resolve order → execute → report. -## Step 1: Clarify the Goal - -Use `AskUserQuestion` to clarify if the goal is vague: -- What is the desired end state? -- What files or systems are involved? -- Are there constraints (language, framework, time)? - -## Step 2: Decompose into Tasks - -Break the goal into discrete tasks. For each task: - -``` -| # | Task | Agent | Depends On | Est. Effort | -|---|------|-------|------------|-------------| -| 1 | Research existing patterns | researcher | — | Low | -| 2 | Design the approach | — (orchestrator) | 1 | Low | -| 3 | Implement core logic | improver | 2 | High | -| 4 | Write tests | test-writer | 3 | Medium | -| 5 | Security review | security-auditor | 3 | Medium | -| 6 | Final review | reviewer | 4, 5 | Low | -``` - -### Rules for Decomposition - -- Each task should be completable by a single agent in one pass -- Tasks with no dependencies can run in parallel -- Tasks must declare all dependencies explicitly -- Prefer small, focused tasks over large monolithic ones -- Include verification tasks (tests, review) — don't skip them - -## Step 3: Resolve Execution Order - -**[PARALLEL]** Topologically sort the DAG. Tasks at the same depth with no mutual dependencies run concurrently. - -``` -Depth 0: [Task 1] — no dependencies -Depth 1: [Task 2] — depends on 1 -Depth 2: [Task 3] — depends on 2 -Depth 3: [Task 4, Task 5] — both depend on 3, run in parallel -Depth 4: [Task 6] — depends on 4 and 5 -``` - -**Concurrency limit:** Max 3 parallel agents to avoid API rate limits. - -## Step 4: Execute - -For each depth level: - -1. Dispatch all tasks at this depth **[PARALLEL]** (up to concurrency limit) -2. Wait for all to complete -3. If any task fails: - - **Cascade skip** all tasks that depend on the failed task - - Continue executing independent tasks at deeper levels - - Report the failure and skipped tasks -4. Inject upstream task outputs as context into downstream task prompts - -### Agent Dispatch +## Invoke ``` -Task: {task description} -Agent: {assigned agent} -Context: - - Goal: {original goal} - - Upstream results: {outputs from dependency tasks} - - Constraints: {any user-specified constraints} +devkit workflow run decompose "{goal_description}" ``` -## Step 5: Report +If `devkit workflow` is not available, follow this manually: -``` -## Decomposition Report: {goal} - -### Task Graph -{ASCII or table representation of the DAG} - -### Execution Summary -| # | Task | Agent | Status | Duration | -|---|------|-------|--------|----------| -| 1 | Research | researcher | Done | — | -| 2 | Design | orchestrator | Done | — | -| 3 | Implement | improver | Done | — | -| 4 | Tests | test-writer | Done | — | -| 5 | Security | security-auditor | Done | — | -| 6 | Review | reviewer | Done | — | - -### Results -{consolidated output from all tasks} - -### Failed / Skipped -{any tasks that failed or were cascade-skipped} -``` +1. **Clarify the goal** — Restate precisely. Ask user for clarification if ambiguous. +2. **Decompose into tasks** — Break into 3-10 concrete, testable tasks. Each must have clear done criteria. Identify parallelizable groups. +3. **Resolve execution order** — Build dependency graph. Tasks with no dependencies can run in parallel. +4. **Execute** — Run tasks in dependency order, dispatching independent tasks to agents in parallel where possible. +5. **Report** — Summary of completed tasks, any failures, and remaining work. ## Rules -- Clarify before decomposing — don't guess at vague goals -- Every task needs a clear definition of done -- Cascade failures — if a task fails, skip all dependents -- Max 3 concurrent agents (rate limit protection) -- Inject upstream context into downstream tasks -- Report which tasks ran, which were skipped, and why +- Each task must be independently testable +- No circular dependencies +- Prefer parallel execution where dependencies allow +- Stop and report if a blocking task fails diff --git a/commands/pr-ready.md b/commands/pr-ready.md index 1d14d91..613ddfc 100644 --- a/commands/pr-ready.md +++ b/commands/pr-ready.md @@ -1,181 +1,32 @@ --- -description: Full PR preparation pipeline — necessity check, DRY review, lint, test, security, changelog, and create PR. +description: Full PR preparation pipeline — validate branch, DRY review, lint, test, security, changelog, create PR. --- # PR Ready -Multi-step pipeline to prepare a branch for PR review. +Deterministic PR preparation: validate → necessity check → DRY review → lint → test → security → changelog → create PR. -## Pipeline +## Invoke -### Step 1: Validate Branch - -```bash -BRANCH=$(git branch --show-current) -if [ "$BRANCH" = "main" ] || [ "$BRANCH" = "master" ]; then - echo "ERROR: Cannot PR from main/master. Create a feature branch first." - exit 1 -fi - -DIFF_STAT=$(git diff main...HEAD --stat) -if [ -z "$DIFF_STAT" ]; then - echo "ERROR: No changes vs main." - exit 1 -fi -``` - -### Step 2: Necessity Check - -Before anything else, evaluate whether this branch justifies a PR. - -Review the full diff and commit history: -```bash -git log main...HEAD --oneline -git diff main...HEAD --stat -git diff main...HEAD -``` - -Answer these questions: -1. **Does this solve a real problem?** — Is there a bug fix, feature request, or measurable improvement? Or is this churn (renaming for style, reshuffling without functional change)? -2. **Is the scope right?** — Does the branch do one coherent thing, or is it a grab-bag of unrelated changes that should be separate PRs? -3. **Does this duplicate existing capability?** — Is there already a command, function, or tool in the codebase that does the same thing? Search for overlapping functionality. -4. **Is the approach justified?** — Could the same result be achieved with a simpler change (config tweak, one-line fix, existing tool)? - -Report a verdict: -- **Necessary** — clear justification, proceed with pipeline -- **Questionable** — concerns exist, list them and ask the user before proceeding -- **Unnecessary** — this branch adds no real value, explain why and recommend against PR - -If the verdict is "Unnecessary", stop the pipeline and explain. If "Questionable", use `AskUserQuestion` to confirm before continuing. - -### Step 3: DRY Review - -**[PARALLEL with Step 4-5]** Spawn the `reviewer` agent: - -``` -Task: DRY and code quality review of this branch's changes. -Agent: reviewer -Input: git diff main...HEAD - -Review the diff for: - -1. **Duplication within the diff** — Are there repeated blocks of text, logic, or structure - in the new/changed code that should be extracted or consolidated? - -2. **Duplication against existing code** — Do the changes duplicate functionality that - already exists elsewhere in the codebase? Search for similar patterns, function names, - and logic in the repo. - -3. **Abstraction quality** — If the changes introduce shared code or utilities, is the - abstraction well-named, properly parameterized, and not over-engineered? - -4. **Rule of Three** — Flag duplication only when there are 3+ instances. Two similar - blocks are fine. Don't suggest premature abstractions. - -Apply the DRY principle correctly: DRY is about knowledge, not code. Two functions with -identical code but representing different concerns are NOT a violation. Only flag -duplication where a change in one copy would require changing the other. - -Report: -- DRY violations found (with file paths and line numbers) -- Suggestions for extraction or consolidation -- Cases where duplication is acceptable and why -``` - -### Step 4: Lint (if linter detected) - -Auto-detect and run linter: -```bash -# Check for common linters: eslint, prettier, ruff, golangci-lint, clippy, etc. -# Run if found. Report issues but don't block. -``` - -If fixable errors found, offer to auto-fix. - -### Step 5: Test - -Auto-detect and run tests: -```bash -# npm test, pytest, go test ./..., cargo test, etc. -``` - -Report results. Warn if tests fail but don't block. - -### Step 6: Security Quick-Check - -Spawn the `security-auditor` agent on the diff: - -``` -Task: Quick security review of this diff — focus on OWASP top 10, hardcoded secrets, SQL injection, XSS. -Agent: security-auditor -Input: git diff main...HEAD -``` - -Report findings with severity levels. - -### Step 7: Generate Changelog - -Analyze commits on the branch: -```bash -git log main...HEAD --oneline -``` - -Categorize changes: features, fixes, refactors, docs, tests. - -### Step 8: Create PR - -```bash -gh pr create --title "{title}" --body "{body}" ``` - -Body includes: -- Summary of changes (from commit analysis) -- Necessity justification (from Step 2) -- DRY review findings (from Step 3) -- Test results -- Security findings (if any) -- Changelog - -## Output - +devkit workflow run pr-ready ``` -## PR Ready Report - -### Pre-flight -- [x] Branch: feature/add-auth -- [x] Necessity: Justified — adds JWT auth required by PROJ-142 -- [x] DRY: No violations found -- [x] Lint: 0 errors -- [x] Tests: 14/14 passing -- [⚠] Security: 1 warning (low severity) -### PR Created -{pr_url} +If `devkit workflow` is not available, follow this manually: -### Changelog -**Features** -- Added JWT authentication middleware - -**Fixes** -- Fixed token expiry validation -``` - -## Presets - -``` -/devkit:pr-ready -/devkit:pr-ready --skip-security -/devkit:pr-ready --skip-necessity -/devkit:pr-ready --draft -``` +1. **Validate branch** — Confirm not on main/master, check for uncommitted changes, verify remote tracking +2. **Necessity check** — Review each changed file: is it needed? Remove unnecessary additions, debug artifacts, or unrelated changes. +3. **DRY review** — Check for duplicated logic across changed files. Extract shared code if 3+ repetitions. +4. **Lint** — Run project linter on changed files. Fix violations. +5. **Test** — Run full test suite. Fix failures. +6. **Security quick-check** — Scan for: hardcoded secrets, SQL injection, XSS, command injection, path traversal, insecure dependencies +7. **Generate changelog** — Create changelog entry from git diff summarizing what changed and why +8. **Create PR** — Push branch, create PR with title + summary + test plan ## Rules -- Never force-push or modify commit history -- Necessity check gates the pipeline — unnecessary branches don't get PRs -- DRY review uses the Rule of Three — don't flag premature abstractions -- Security findings are advisory, not blocking -- If tests fail, warn but still offer to create PR as draft -- Auto-detect all tooling — don't assume any specific stack -- Use `gh` CLI for PR creation -- Steps 3, 4, and 5 (DRY, lint, test) run in parallel when possible +- Every step must pass before proceeding to the next +- Remove unnecessary changes before reviewing code quality +- Fix lint and test failures — don't skip them +- Security check is not optional +- Changelog should explain WHY, not just WHAT diff --git a/commands/refactor.md b/commands/refactor.md index b3d2515..7776b39 100644 --- a/commands/refactor.md +++ b/commands/refactor.md @@ -2,76 +2,28 @@ description: Full lifecycle refactor — analyze code smells, plan transformations, restructure, verify nothing broke. --- -# Refactor Workflow +# Refactor -Complete refactor lifecycle: analyze → plan → restructure → verify → compare. +Deterministic refactor lifecycle: analyze → plan → refactor → test → compare. -## Budget & Early Exit - -- **Token budget:** ~400k tokens. Refactors can touch many files. -- **Early exit:** Stop the refactor loop when all planned steps are complete and tests pass. -- **Stuck detection:** If 3 consecutive refactor steps break tests, stop and report. See the `stuck` skill. - -## Step 1: Analyze +## Invoke ``` -Analyze the code to refactor: {user's input} - -Identify: -- Code smells (duplication, long functions, deep nesting, god objects) -- Complexity hotspots -- Coupling issues -- Naming problems -- What the code SHOULD look like after refactoring - -Don't change anything yet — just assess. +devkit workflow run refactor "{target_and_objective}" ``` -## Step 2: Plan - -``` -Based on the analysis, create a step-by-step refactoring plan. +If `devkit workflow` is not available, follow this manually: -Each step should be a single, safe transformation that preserves behavior. -Order matters — do renames before extractions, extractions before moves. -Include "run tests" checkpoints between risky steps. -``` - -## Step 3: Refactor - -``` -Execute the next incomplete step from the plan. -Make the change, verify it compiles, and confirm behavior is preserved. -``` - -Loop until all steps are complete. Max 15 iterations. - -## Step 4: Run Tests - -Run the full test suite to verify the refactoring didn't break anything. If tests fail, fix them — update tests to match the new structure, or fix regressions. Loop up to 6 times until all pass. - -## Step 5: Before/After Comparison - -``` -## What Changed -List each transformation that was made. - -## Improvements -What's better now (readability, complexity, coupling). - -## Metrics -Lines added/removed, functions extracted, files changed. - -## Risk Areas -Anything that should be watched closely after this refactor. -``` +1. **Analyze** — Identify code smells, duplication, complexity hotspots in the target +2. **Plan** — Create ordered list of transformations; each should be a single testable change +3. **Refactor** — Execute transformations one at a time; run tests after each (loop max 10) +4. **Run tests** — Full test suite must pass +5. **Before/after comparison** — Show what changed: complexity metrics, line counts, readability improvements ## Rules -- Analyze before changing — understand the full picture first -- Behavior-preserving transformations only — refactoring must not change what the code does -- One transformation per step — don't combine rename + extract + move -- Order matters — renames before extractions, extractions before moves -- Test between risky steps — verify nothing broke at each checkpoint -- All tests must pass before reporting done -- Don't add features during a refactor — separate concerns +- Tests must pass after every transformation — no "fix later" +- One transformation at a time +- Preserve behavior — refactoring changes structure, not functionality +- If tests fail, revert the transformation +- Token budget: ~300k tokens diff --git a/commands/repo-map.md b/commands/repo-map.md index 5e54211..52bd4db 100644 --- a/commands/repo-map.md +++ b/commands/repo-map.md @@ -4,254 +4,26 @@ description: Build an AST-based symbol index of the repository — exports, func # Repo Map -Builds a structural map of the codebase using AST parsing. The map is cached and used by other commands and agents for faster, more accurate navigation. +Build and cache an AST-based symbol index for fast codebase navigation. -## Parameters +## Invoke -1. **Target** — directory to map (default: project root) -2. **Format** — output format: `summary` (default), `full`, `json` - -## Prerequisites - -Requires [ast-grep](https://ast-grep.github.io/) (`sg`) for AST parsing. Falls back to regex-based extraction if not installed. - -```bash -# Check for ast-grep -if command -v sg >/dev/null 2>&1; then - echo "ast-grep: $(sg --version)" - MODE="ast" -else - echo "ast-grep not installed — using regex fallback" - echo "Install: brew install ast-grep OR npm i -g @ast-grep/cli" - MODE="regex" -fi -``` - -## Step 1: Detect Languages - -```bash -echo "=== Language Detection ===" -[ -n "$(find . -name '*.ts' -o -name '*.tsx' | head -1)" ] && echo "typescript" -[ -n "$(find . -name '*.js' -o -name '*.jsx' | head -1)" ] && echo "javascript" -[ -n "$(find . -name '*.py' | head -1)" ] && echo "python" -[ -n "$(find . -name '*.go' | head -1)" ] && echo "go" -[ -n "$(find . -name '*.rs' | head -1)" ] && echo "rust" -[ -n "$(find . -name '*.java' | head -1)" ] && echo "java" -[ -n "$(find . -name '*.rb' | head -1)" ] && echo "ruby" -``` - -## Step 2: Extract Symbols (AST mode) - -For each detected language, use `sg` to extract structural elements: - -### TypeScript / JavaScript - -```bash -# Exported functions -sg --pattern 'export function $NAME($$$PARAMS): $RET { $$$ }' --lang ts -j -sg --pattern 'export const $NAME = ($$$PARAMS) => $$$' --lang ts -j -sg --pattern 'export default function $NAME($$$) { $$$ }' --lang ts -j - -# Exported classes -sg --pattern 'export class $NAME $$${ $$$ }' --lang ts -j - -# Exported interfaces/types -sg --pattern 'export interface $NAME { $$$ }' --lang ts -j -sg --pattern 'export type $NAME = $$$' --lang ts -j - -# Imports (to build dependency graph) -sg --pattern 'import { $$$ } from "$SOURCE"' --lang ts -j -sg --pattern 'import $NAME from "$SOURCE"' --lang ts -j -``` - -### Python - -```bash -# Functions -sg --pattern 'def $NAME($$$):' --lang python -j - -# Classes -sg --pattern 'class $NAME($$$):' --lang python -j -sg --pattern 'class $NAME:' --lang python -j - -# Imports -sg --pattern 'from $MODULE import $$$' --lang python -j -sg --pattern 'import $MODULE' --lang python -j -``` - -### Go - -```bash -# Exported functions (capitalized) -sg --pattern 'func $NAME($$$) $$${ $$$ }' --lang go -j - -# Structs -sg --pattern 'type $NAME struct { $$$ }' --lang go -j - -# Interfaces -sg --pattern 'type $NAME interface { $$$ }' --lang go -j - -# Imports -sg --pattern 'import "$PKG"' --lang go -j -``` - -### Rust - -```bash -# Public functions -sg --pattern 'pub fn $NAME($$$) -> $$$ { $$$ }' --lang rust -j -sg --pattern 'pub fn $NAME($$$) { $$$ }' --lang rust -j - -# Structs -sg --pattern 'pub struct $NAME { $$$ }' --lang rust -j - -# Traits -sg --pattern 'pub trait $NAME { $$$ }' --lang rust -j - -# Impls -sg --pattern 'impl $NAME { $$$ }' --lang rust -j ``` - -## Step 3: Extract Symbols (Regex fallback) - -If `sg` is not installed, use grep-based extraction: - -```bash -# Functions -grep -rnE '^\s*(export\s+)?(async\s+)?function\s+\w+' --include='*.ts' --include='*.js' | head -200 -grep -rnE '^\s*def\s+\w+' --include='*.py' | head -200 -grep -rnE '^func\s+\w+' --include='*.go' | head -200 - -# Classes -grep -rnE '^\s*(export\s+)?class\s+\w+' --include='*.ts' --include='*.js' | head -200 -grep -rnE '^class\s+\w+' --include='*.py' | head -200 - -# Types/Interfaces -grep -rnE '^\s*(export\s+)?(interface|type)\s+\w+' --include='*.ts' | head -200 -grep -rnE '^type\s+\w+\s+(struct|interface)' --include='*.go' | head -200 -``` - -## Step 4: Build Dependency Graph - -From the import data, build a file-level dependency graph: - -``` -src/auth/handler.ts - → imports from: src/auth/middleware.ts, src/db/users.ts, src/config.ts - ← imported by: src/routes/api.ts - -src/db/users.ts - → imports from: src/db/connection.ts, src/types.ts - ← imported by: src/auth/handler.ts, src/admin/users.ts -``` - -Identify: -- **Entry points** — files with no importers (likely main/index files) -- **Hubs** — files imported by 5+ other files (high-impact change targets) -- **Orphans** — files that are never imported (possibly dead code) - -## Step 5: Cache the Map - -Write the map to `.devkit/repo-map.json`: - -```json -{ - "generated": "2026-04-04T12:00:00Z", - "commit": "abc1234", - "languages": ["typescript", "python"], - "mode": "ast", - "symbols": { - "src/auth/handler.ts": { - "exports": [ - { "name": "handleLogin", "type": "function", "line": 15 }, - { "name": "handleLogout", "type": "function", "line": 42 }, - { "name": "AuthConfig", "type": "interface", "line": 5 } - ], - "imports": ["src/auth/middleware", "src/db/users", "src/config"] - } - }, - "graph": { - "entry_points": ["src/index.ts", "src/cli.ts"], - "hubs": ["src/config.ts", "src/types.ts", "src/db/connection.ts"], - "orphans": ["src/legacy/old-parser.ts"] - }, - "stats": { - "files": 47, - "functions": 128, - "classes": 12, - "interfaces": 23, - "total_exports": 163 - } -} +devkit workflow run repo-map ``` -The cache is invalidated when the current commit differs from the stored commit. - -## Step 6: Generate Report - -### Summary format (default) - -``` -## Repo Map - -**Languages:** TypeScript, Python -**Mode:** AST (ast-grep) -**Files:** 47 | **Functions:** 128 | **Classes:** 12 | **Interfaces:** 23 - -### Entry Points -- src/index.ts (main application entry) -- src/cli.ts (CLI entry) - -### Hubs (high-impact files) -- src/config.ts — imported by 12 files -- src/types.ts — imported by 9 files -- src/db/connection.ts — imported by 7 files - -### Orphans (possibly dead code) -- src/legacy/old-parser.ts — never imported - -### Top-Level Structure -src/ - auth/ — 4 files, 8 exports (handleLogin, handleLogout, ...) - db/ — 3 files, 6 exports (getUser, createUser, ...) - routes/ — 5 files, 10 exports (apiRouter, authRouter, ...) - utils/ — 2 files, 4 exports (parseDate, formatCurrency, ...) - -Cached to .devkit/repo-map.json -``` - -## Usage by Other Commands - -Other devkit commands and agents can reference the cached map: - -```bash -# Check if map exists and is current -if [ -f .devkit/repo-map.json ]; then - MAP_COMMIT=$(jq -r '.commit' .devkit/repo-map.json) - CURRENT=$(git rev-parse --short HEAD) - if [ "$MAP_COMMIT" = "$CURRENT" ]; then - echo "Using cached repo map" - cat .devkit/repo-map.json - else - echo "Map is stale — rebuilding" - # Trigger rebuild - fi -fi -``` +If `devkit workflow` is not available, follow this manually: -Commands that benefit from the map: -- **tri-review** — knows which files are hubs and need more careful review -- **refactor** — sees the dependency graph to understand blast radius -- **decompose** — uses the graph to identify natural task boundaries -- **onboard** — uses entry points and hubs to guide the tour -- **audit** — identifies orphaned/dead code +1. **Detect languages** — Scan for language markers (go.mod, package.json, pyproject.toml, Cargo.toml, etc.) +2. **Extract symbols (AST)** — Use language-specific AST tools to extract exports, functions, classes, types, interfaces. Fall back to regex if AST tools unavailable. +3. **Build dependency graph** — Map imports/requires between files to build a dependency graph +4. **Cache the map** — Write to `.devkit/repo-map.json` with current commit hash for staleness detection +5. **Generate report** — Summary: entry points, hub files (most imported), orphan files (never imported), symbol counts ## Rules -- Always respect .gitignore — don't index node_modules, dist, etc. -- Cache to `.devkit/repo-map.json` (already gitignored) -- Invalidate cache on commit change -- Prefer AST mode — fall back to regex only when sg is not installed -- Cap extraction at 200 results per query to avoid context bloat -- The map is read-only — it never modifies code -- Skip binary files and files larger than 100KB +- Prefer AST extraction over regex — more accurate +- Cache results — don't rebuild on every invocation +- Include commit hash for staleness detection +- Identify entry points, hubs, and orphans +- Other commands (tri-review, refactor, decompose) can reference the cached map diff --git a/commands/self-audit.md b/commands/self-audit.md index 7082295..b49eea3 100644 --- a/commands/self-audit.md +++ b/commands/self-audit.md @@ -1,191 +1,34 @@ --- -description: Automated self-audit — measure the codebase, rank improvement hypotheses by evidence, present actionable plan. Inspired by karpathy/autoresearch. +description: Automated self-audit — measure the codebase, rank improvement hypotheses by evidence, present actionable plan. --- # Self-Audit -Systematic codebase audit inspired by [karpathy/autoresearch](https://github.com/karpathy/autoresearch): one file, one metric, keep or discard, loop. Applied to codebase quality instead of ML training. +Evidence-based codebase health assessment: detect stack → measure → analyze → rank and present. -## Philosophy - -autoresearch gives an AI agent a training setup and lets it experiment autonomously — modify code, measure the metric, keep or revert, repeat ~100 times overnight. We apply the same pattern to codebase quality: measure everything with a single pass, form ranked hypotheses, then test one at a time with a clear keep/discard metric. - -This command does NOT fix anything. It measures, analyzes, and presents a ranked hypothesis list. You decide which to test. One at a time. - -## Step 1: Detect Stack - -```bash -# Detect what's in this repo -HAS_GO=$([ -f go.mod ] || find . -maxdepth 2 -name go.mod -print -quit | grep -q . && echo yes || echo no) -HAS_TS=$([ -f tsconfig.json ] && echo yes || echo no) -HAS_JS=$([ -f package.json ] && echo yes || echo no) -HAS_PYTHON=$([ -f pyproject.toml ] || [ -f requirements.txt ] || [ -f setup.py ] && echo yes || echo no) -HAS_RUST=$([ -f Cargo.toml ] && echo yes || echo no) -HAS_TESTS=$([ -n "$(find . -name '*_test.go' -o -name '*.test.ts' -o -name '*.test.js' -o -name 'test_*.py' -o -name '*_test.rs' 2>/dev/null | head -1)" ] && echo yes || echo no) -HAS_CI=$([ -d .github/workflows ] || [ -f .gitlab-ci.yml ] || [ -f Jenkinsfile ] && echo yes || echo no) -HAS_DOCKER=$([ -f Dockerfile ] || [ -f docker-compose.yml ] && echo yes || echo no) -``` - -Report what was detected. Skip measurements for stacks not present. - -## Step 2: Measure (the data collection phase) - -Run ALL applicable measurements. Do not skip any. Collect raw numbers. - -**[PARALLEL]** Run these measurement groups concurrently: - -### Code Quality - -```bash -# Go -go vet ./... 2>&1 | wc -l # vet issues -go test -cover ./... 2>&1 # coverage per package -gofmt -l . 2>&1 | wc -l # formatting issues - -# TypeScript/JavaScript -npx tsc --noEmit 2>&1 | grep -c 'error TS' # type errors -npx eslint . --format compact 2>&1 | wc -l # lint issues - -# Python -ruff check . 2>&1 | wc -l # lint issues -mypy . 2>&1 | grep -c 'error:' # type errors - -# Rust -cargo clippy 2>&1 | grep -c 'error\[' # clippy errors -``` - -### Test Coverage - -```bash -# Get per-package/per-file coverage — the NUMBERS matter -# Go: go test -cover ./... -# TS: npx jest --coverage --coverageReporters text-summary -# Python: pytest --cov --cov-report term-missing -# Rust: cargo tarpaulin --out stdout -``` - -Record: total coverage %, lowest-coverage packages, untested files. - -### Security - -```bash -# Dependency vulnerabilities -# Go: govulncheck ./... -# Node: npm audit -# Python: pip-audit or safety check -# Rust: cargo audit - -# Hardcoded secrets — use the patterns from references/stub-patterns.md -# "Hardcoded Values Where Dynamic Expected" section (excludes test files, includes .tsx) -``` - -### Stale Code - -```bash -# Dead files (not imported/referenced) -# Unused dependencies -# go mod tidy -diff (shows removable deps) -# npm prune --dry-run -# TODO/FIXME/HACK count — matches stub-patterns.md canonical list -grep -rn 'TODO\|FIXME\|HACK\|XXX\|PLACEHOLDER' \ - --include='*.go' --include='*.ts' --include='*.tsx' --include='*.js' --include='*.jsx' \ - --include='*.py' --include='*.rs' --include='*.rb' . 2>/dev/null | wc -l -``` - -For deeper stub detection (empty implementations, placeholder text, hardcoded values, skipped tests), run the patterns from `references/stub-patterns.md`. Run unconditionally — a repo can have zero TODOs but still contain empty catch blocks, placeholder UI copy, or hardcoded localhost URLs. - -### Documentation - -```bash -# README exists and is recent? -git log -1 --format='%ai' -- README.md 2>/dev/null -# API docs? -# Changelog? -# Are counts/claims in docs accurate? (run validate-counts if available) -``` - -### Git Health - -```bash -# Recent commit frequency -git log --oneline --since='30 days ago' | wc -l -# Stale branches -git branch -r --merged main | grep -v main | wc -l -# Large files in history -git rev-list --objects --all | git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' | awk '/^blob/ {print $3, $4}' | sort -rn | head -5 -``` - -## Step 3: Analyze (turn measurements into hypotheses) +## Invoke ``` -For each measurement category, form hypotheses: - -IF coverage < 50% in a package THEN "Improve test coverage in {package} — currently {N}%" -IF vet/lint issues > 0 THEN "Fix {N} vet/lint issues" -IF vulnerability count > 0 THEN "Patch {N} known vulnerabilities" -IF TODO count > 20 THEN "Triage {N} TODOs — many may be stale" -IF stale branches > 5 THEN "Clean up {N} merged branches" -IF no CI THEN "Add CI pipeline — no automated checks exist" -IF hardcoded secrets found THEN "Move {N} hardcoded secrets to environment variables" -IF lowest coverage package < 30% THEN "Package {name} has {N}% coverage — highest risk area" - -For each hypothesis, estimate: -- IMPACT: How much does this improve the codebase? (high/medium/low) -- EFFORT: How hard is this to fix? (high/medium/low) -- EVIDENCE: What measurement supports this? (cite the number) -``` - -## Step 4: Rank and Present - -``` -## Self-Audit: {repo-name} - -### Stack Detected -{languages, frameworks, CI, Docker, etc.} - -### Raw Measurements -| Category | Metric | Value | -|----------|--------|-------| -| Coverage | Overall | X% | -| Coverage | Lowest package | {name} at X% | -| Lint | Issues | N | -| Security | Vulnerabilities | N | -| Stale | TODOs | N | -| Stale | Merged branches | N | -| Git | 30-day commits | N | - -### Ranked Hypotheses (by impact/effort ratio) - -| # | Hypothesis | Impact | Effort | Evidence | -|---|-----------|--------|--------|----------| -| 1 | {highest impact/effort ratio} | high | low | {measurement} | -| 2 | ... | ... | ... | ... | -| N | ... | ... | ... | ... | - -### Recommended Next Steps -1. Test hypothesis #1 first — highest impact for lowest effort -2. Use `/devkit:self-improve` or `/devkit:self-test` to execute -3. Measure again after each fix to verify improvement -4. One change at a time — don't bundle - -### What NOT to Do -- Don't fix anything from this report directly — use the suggested devkit command instead -- Don't fix everything at once — one hypothesis at a time -- Don't start with low-impact items just because they're easy -- Don't add features — this is about quality, not functionality +devkit workflow run self-audit ``` -## Budget +If `devkit workflow` is not available, follow this manually: -- **Token budget:** ~200k tokens. Measurement is cheap; the value is in the analysis. -- **Early exit:** If the codebase is clean (no lint issues, >80% coverage, no vulns), say so and skip the hypothesis phase. +1. **Detect stack** — Auto-detect languages, frameworks, package managers, test runners, linters +2. **Measure** — Collect data across 6 dimensions: + - Code quality (lint errors, warnings, complexity) + - Test coverage (percentage, untested critical paths) + - Security (vulnerability scan, secrets detection) + - Stale code (dead code, unused exports, orphan files) + - Documentation (README freshness, API docs, changelog) + - Git health (branch age, large files, merge conflicts) +3. **Analyze** — Turn measurements into ranked improvement hypotheses with predicted impact and effort +4. **Report** — Present top 5-10 improvements sorted by impact/effort ratio, with specific file references ## Rules -- Measure EVERYTHING before forming opinions — no guessing -- Cite numbers — every hypothesis must reference a specific measurement -- Rank by impact/effort ratio — not by what's easiest or most interesting -- One at a time — never suggest bundling fixes -- Don't fix anything — this command only measures and analyzes -- Be honest — if the codebase is clean, say so. Don't manufacture problems. -- Suggest the right devkit command for each hypothesis (self-improve, self-test, self-lint, tri-security, etc.) +- Measure first, hypothesize second — no guessing at problems +- Run actual tools — don't estimate metrics +- Rank by evidence strength, not gut feel +- Every recommendation must cite specific measurements +- Token budget: ~200k tokens diff --git a/commands/tri-debug.md b/commands/tri-debug.md index 7640bea..4162e12 100644 --- a/commands/tri-debug.md +++ b/commands/tri-debug.md @@ -1,166 +1,29 @@ --- -description: Multi-agent debugging — send a bug report to available agents (Claude + Codex + Gemini) via plugin or CLI, get independent root-cause hypotheses, and a consensus fix. +description: Triple-agent debugging — independent root-cause hypotheses from Claude, Codex, and Gemini, then consensus fix. --- # Triple-Agent Debug -Send a bug description to all available agents in parallel, get independent root-cause analyses, and consolidate into a recommended fix. +Dispatch a bug report to 2-3 AI agents in parallel, get independent root-cause hypotheses, and consolidate a fix. -## Step 0: Harness Detection +## Invoke -```bash -if command -v devkit >/dev/null 2>&1; then - echo "Go harness detected — delegating to devkit dispatch for full output capture." - devkit dispatch {prompt with bug context} - exit 0 -fi ``` - -If the `devkit` binary is in PATH, delegate entirely to it. Only fall through to plugin-based steps if the harness is not installed. - -## Step 1: Gather Context - -Collect from the user: -- Bug description or error message -- Stack trace (if available) -- Steps to reproduce (if known) -- Relevant files - -If a stack trace is provided, extract the relevant source files: -```bash -# Parse file paths from stack trace and read them -``` - -## Step 2: Detect Available Agents - -Check for plugins first (preferred), then fall back to CLI: - -```bash -# Plugin detection (preferred — structured job management) -HAS_CODEX_PLUGIN=$(/codex:status >/dev/null 2>&1 && echo "yes" || echo "no") -HAS_GEMINI_PLUGIN=$(/gemini:status >/dev/null 2>&1 && echo "yes" || echo "no") - -# CLI fallback detection -HAS_CODEX_CLI=$(command -v codex && echo "yes" || echo "no") -HAS_GEMINI_CLI=$(command -v gemini && echo "yes" || echo "no") +devkit workflow run tri-debug "{bug_description}" ``` -Run with whatever is available. 1 agent minimum (Claude), up to 3. Prefer plugin over CLI. - -## Step 3: Build the Prompt - -``` -Debug this issue. Provide: -1. Root cause — what is actually wrong and why -2. Evidence — specific lines of code that cause the bug -3. Fix — exact code changes to resolve it -4. Verification — how to confirm the fix works - -Bug: {description} -Stack trace: {stack_trace} -Relevant code: {source_files} -``` - -## Concurrency & Budget - -- **Concurrency limit:** Max 3 parallel agents. -- **Token budget:** ~300k tokens across all agents. -- **Rate limiting:** If API throttles, stagger agent launches. - -## Step 4: Dispatch (Hybrid, Graceful Degradation) - -**[PARALLEL]** Launch all available agents concurrently: - -**CRITICAL:** All context (bug description, stack trace, relevant code) MUST be passed inline in each agent's prompt. Worktree-isolated agents cannot see the latest commits or local state. - -### Claude — always runs (native background agent) - -``` -Task: Debug this issue. -Agent: researcher -Input: {prompt} - -{context — bug description, stack trace, relevant code inlined here} -``` - - - -### Codex — if available - -``` -/codex:rescue --effort high --background "{prompt} {context}" -``` - -Retrieve result with `/codex:result` when done. Omit `--model` to use the account default. - -### Gemini — if available - -**Plugin (preferred):** - -``` -/gemini:rescue --background "{prompt} {context}" -``` - -Retrieve result with `/gemini:result` when done. Omit `--model` to use the account default. - -**CLI fallback (only if plugin not installed):** - -```bash -if [ "$HAS_GEMINI_CLI" = "yes" ]; then - gemini -p "{prompt} {context}" -y \ - --output-format text > /tmp/tri-debug-gemini.txt 2>&1 & - GEMINI_PID=$! -fi - -wait -``` - -## Step 5: Consolidate - -``` -## Debug Report: {summary} - -### Agents Used: {count}/3 -{list which agents ran} - -### Consensus Root Cause (agreed by {n}+ agents) -- ... - -### Root Cause Analysis by Agent -- **Claude:** ... -- **Codex:** ... (if available) -- **Gemini:** ... (if available) - -### Recommended Fix -{merged fix based on consensus} - -### Verification Steps -1. ... -``` - -## Investigation Techniques - -When building the debug prompt or analyzing results, apply the most appropriate technique: - -| Technique | When to Use | How | -|-----------|-------------|-----| -| **Binary search** | Bug exists somewhere in a range of changes/code | Bisect the search space — disable half, test, narrow | -| **Differential debugging** | "It worked before" | Compare working vs broken state — `git diff`, env diff, config diff | -| **Minimal reproduction** | Complex bug with many variables | Strip away everything unrelated until the simplest trigger remains | -| **Trace execution** | Control flow is unclear | Add logging or step through — follow the actual path, not the assumed one | -| **Working backwards** | You know the symptom but not the cause | Start at the error, trace data/control flow backwards to the source | -| **5 Whys** | Surface fix isn't enough | Ask "why?" at each layer: symptom → immediate cause → deeper cause → root cause → systemic issue | - -## Domain-Specific Debugging Checklists +If `devkit workflow` is not available, follow this manually: -When the bug domain is identifiable (API, database, auth, async, performance), read the relevant section from `references/debug-checklists.md` (relative to this `commands/` directory) and include it in each agent's prompt. Only include the matching domain — don't load the full file. +1. **Gather context** — Collect error messages, stack traces, reproduction steps, and relevant code +2. **Detect agents** — Check for Codex and Gemini availability. Claude always runs. +3. **Build prompt** — Include: error output, relevant source code, recent changes (`git log -5`), and reproduction steps +4. **Dispatch in parallel** — Launch all available agents concurrently with the full context +5. **Consolidate** — Consensus diagnosis (2+ agents agree) → high confidence. Unique hypotheses → worth investigating. ## Rules -- Claude always runs — Codex and Gemini are optional -- If only Claude is available, still provide the full report format (just one perspective) -- Report which agents participated -- If agents disagree on root cause, present all hypotheses ranked by evidence -- When agents agree on symptoms but disagree on root cause, apply the 5 Whys to the shared symptoms to converge on a deeper cause. If they disagree on symptoms too, prioritize verifying symptoms through additional logs or traces before root-cause analysis -- Include the relevant domain checklist in each agent's prompt when the bug category is identifiable -- Clean up temp files after +- Claude always runs as native background agent +- Codex and Gemini are optional — skip gracefully if not installed +- Each agent works independently — don't share findings between agents +- Report consensus vs unique diagnoses +- Include specific file:line references and suggested fixes diff --git a/commands/tri-dispatch.md b/commands/tri-dispatch.md index 0616120..d6a117f 100644 --- a/commands/tri-dispatch.md +++ b/commands/tri-dispatch.md @@ -1,123 +1,27 @@ --- -description: Dispatch a task to all three agents (Claude, Codex, Gemini) in parallel and compare results. Claude uses native background agent, others via plugin or CLI. +description: Dispatch a task to all available agents (Claude, Codex, Gemini) in parallel and compare results. --- # Triple-Agent Dispatch -Send the same task to Claude, Codex, and Gemini in parallel. Compare outputs. +Send an arbitrary task to 2-3 AI agents in parallel and compare their results. -## Step 0: Harness Detection +## Invoke -```bash -if command -v devkit >/dev/null 2>&1; then - echo "Go harness detected — delegating to devkit dispatch for full output capture." - devkit dispatch {prompt} - exit 0 -fi ``` - -If the `devkit` binary is in PATH, delegate entirely to it. Only fall through to plugin-based steps if the harness is not installed. - -## When to use - -- Comparing approaches to a problem -- Getting multiple implementation ideas -- Validating a solution across models - -## Detect Available Agents - -Check for plugins first (preferred), then fall back to CLI: - -```bash -# Plugin detection (preferred — structured job management) -HAS_CODEX_PLUGIN=$(/codex:status >/dev/null 2>&1 && echo "yes" || echo "no") -HAS_GEMINI_PLUGIN=$(/gemini:status >/dev/null 2>&1 && echo "yes" || echo "no") - -# CLI fallback detection -HAS_CODEX_CLI=$(command -v codex && echo "yes" || echo "no") -HAS_GEMINI_CLI=$(command -v gemini && echo "yes" || echo "no") -``` - -Run with whatever is available. Claude always runs. Prefer plugin over CLI. - -## Concurrency & Budget - -- **Concurrency limit:** Max 3 parallel agents. -- **Token budget:** ~300k tokens across all agents. -- **Rate limiting:** If API throttles, stagger agent launches with brief delays. - -## Execution (Hybrid, Graceful Degradation) - -**[PARALLEL]** Launch all available agents concurrently: - -**CRITICAL:** Pass the full prompt and any relevant context inline to each agent. Worktree-isolated agents cannot see the latest commits or local state. - -### Claude — always runs (native background agent) - -Spawn the `researcher` agent as a background task with the full prompt inline: - -``` -Task: {user's task — full prompt inlined here} -Agent: researcher -``` - - - -### Codex — if available - -``` -/codex:rescue --effort high --background "$PROMPT" -``` - -Retrieve result with `/codex:result` when done. Omit `--model` to use the account default. - -### Gemini — if available - -**Plugin (preferred):** - -``` -/gemini:rescue --background "$PROMPT" -``` - -Retrieve result with `/gemini:result` when done. Omit `--model` to use the account default. - -**CLI fallback (only if plugin not installed):** - -```bash -if [ "$HAS_GEMINI_CLI" = "yes" ]; then - gemini -p "$PROMPT" -y \ - --output-format text > /tmp/tri-dispatch-gemini.txt 2>&1 & -fi - -wait -``` - -## Output - +devkit workflow run tri-dispatch "{task_description}" ``` -## Triple Dispatch: {summary} - -### Claude (researcher agent) -{agent result} -### Codex -{output} +If `devkit workflow` is not available, follow this manually: -### Gemini -{output} - -### Analysis -- Where they agree: ... -- Where they differ: ... -- Recommended approach: ... -``` +1. **Detect agents** — Check for Codex and Gemini availability. Claude always runs. +2. **Dispatch in parallel** — Launch all available agents with the same prompt. Claude uses native background agent; others use plugin or CLI. +3. **Collect results** — Wait for all agents to complete. +4. **Compare** — Present results side by side. Highlight consensus and divergence. ## Rules -- Claude always runs as native background agent for token efficiency -- Codex and Gemini are optional — run if installed, skip gracefully if not -- Report which agents participated (e.g., "2/3 agents" or "Claude only") -- If only Claude is available, still provide the full report format -- If one fails, report the others -- Clean up temp files after -- For file-modifying tasks, use Codex sandbox `full` instead of `read-only` +- Claude always runs as native background agent +- Codex and Gemini are optional — skip gracefully +- Same prompt goes to all agents — no agent-specific modifications +- Report which agents participated diff --git a/commands/tri-review.md b/commands/tri-review.md index b34c9a9..baefd79 100644 --- a/commands/tri-review.md +++ b/commands/tri-review.md @@ -1,199 +1,29 @@ --- -description: Triple-agent PR/code review. Claude runs as native background agent (token-efficient), Codex and Gemini via plugin or CLI. Consolidates findings. +description: Triple-agent code review — dispatches to Claude, Codex, and Gemini in parallel, consolidates findings. --- # Triple-Agent Review -Run the same code review across three AI agents in parallel and consolidate results. +Dispatch the same code review to 2-3 AI agents in parallel and consolidate results by consensus. -## Step 0: Harness Detection +## Invoke -```bash -if command -v devkit >/dev/null 2>&1; then - echo "Go harness detected — delegating to devkit review for full output capture." - devkit review {prompt or default} - # The harness handles parallel dispatch, full stdout capture (no truncation), - # SQLite session tracking, and consolidated output. Skip all steps below. - exit 0 -fi ``` - -If the `devkit` binary is in PATH, delegate entirely to it. The harness avoids output truncation, captures full agent responses, and tracks sessions in SQLite. Only fall through to the plugin-based steps below if the harness is not installed. - -## Step 1: Gather Context - -```bash -# Write directly to file — avoids shell variable limits and content mangling -git diff main...HEAD > /tmp/tri-review-diff.txt 2>/dev/null -if [ ! -s /tmp/tri-review-diff.txt ]; then git diff HEAD~1..HEAD > /tmp/tri-review-diff.txt 2>/dev/null; fi -if [ ! -s /tmp/tri-review-diff.txt ]; then git diff --cached > /tmp/tri-review-diff.txt 2>/dev/null; fi - -DIFF_LINES=$(wc -l < /tmp/tri-review-diff.txt) -if [ "$DIFF_LINES" -gt 5000 ]; then - echo "WARNING: Diff is $DIFF_LINES lines. Consider narrowing to specific files." -fi -``` - -If all empty, ask the user what to review. - -**CRITICAL:** The diff MUST be passed inline in each agent's prompt — do NOT rely on agents fetching the diff themselves. Worktree-isolated agents cannot see the latest commits. - -## Step 2: Build the Prompt - -Use the user's custom prompt if provided. Otherwise default to: - -``` -Review this code diff. For each issue found, report: -- File and line number -- Severity (critical / warning / suggestion) -- Description of the issue -- Suggested fix - -Focus on: bugs, security issues, DRY violations, unnecessary complexity, missing edge cases. -``` - -## Step 3: Detect Available Agents - -Check for plugins first (preferred), then fall back to CLI: - -```bash -# Plugin detection (preferred — structured job management) -HAS_CODEX_PLUGIN=$(/codex:status >/dev/null 2>&1 && echo "yes" || echo "no") -HAS_GEMINI_PLUGIN=$(/gemini:status >/dev/null 2>&1 && echo "yes" || echo "no") - -# CLI fallback detection -HAS_CODEX_CLI=$(command -v codex && echo "yes" || echo "no") -HAS_GEMINI_CLI=$(command -v gemini && echo "yes" || echo "no") -``` - -Run with whatever is available. Claude always runs. Codex and Gemini are optional. Prefer plugin over CLI. - -## Concurrency & Budget - -- **Concurrency limit:** Max 3 parallel agents. All dispatches below run concurrently. -- **Token budget:** ~300k tokens across all agents. -- **Rate limiting:** If you hit API rate limits, wait and retry. Don't launch all agents simultaneously if the API is throttling. - -## Step 4: Dispatch in Parallel (Hybrid, Graceful Degradation) - -**[PARALLEL]** Launch all available agents concurrently: - -### Claude — always runs (native background agent, token-efficient) - -Spawn the `reviewer` agent as a background task. **Pass the full diff inline in the prompt** — the agent runs in a worktree and cannot see recent commits. - -``` -Task: Review this code diff. -Agent: reviewer -Input: {prompt} - -```diff -{diff} -``` -``` - - - -**IMPORTANT instruction to include in the Claude agent prompt:** - -> The diff above is the ONLY source of truth. Do NOT read files from the worktree to verify whether changes were applied — the worktree is based on main, not the PR branch, so files will appear unchanged. Review the diff as provided. If you need to check for stale references or orphan files, grep the worktree but understand that the diff's changes are NOT reflected there. - -### Codex — if available - -**Plugin (preferred):** - -``` -/codex:rescue --effort high --background \ - "{prompt} $(cat /tmp/tri-review-diff.txt)" -``` - -Retrieve result with `/codex:result` when done. Omit `--model` to use the account default. - -**CLI fallback (only if plugin not installed):** - -```bash -if [ "$HAS_CODEX_CLI" = "yes" ]; then - cat /tmp/tri-review-diff.txt | codex exec --full-auto "{prompt}" \ - > /tmp/tri-review-codex.txt 2>&1 & - CODEX_PID=$! -fi -``` - -### Gemini — if available - -**Plugin (preferred):** - -``` -/gemini:rescue --background \ - "{prompt} $(cat /tmp/tri-review-diff.txt)" -``` - -Retrieve result with `/gemini:result` when done. Omit `--model` to use the account default. - -**CLI fallback (only if plugin not installed):** - -```bash -if [ "$HAS_GEMINI_CLI" = "yes" ]; then - cat /tmp/tri-review-diff.txt | gemini -p "{prompt}" \ - -y --output-format text \ - > /tmp/tri-review-gemini.txt 2>&1 & - GEMINI_PID=$! -fi - -wait -``` - -Note: Gemini CLI defaults to the best available model. Don't hardcode a model name — it may not be available on all accounts. - -### Post-dispatch validation - -After `wait`, check each CLI output file. If empty, report the failure instead of silently dropping the agent: - -```bash -for agent_file in /tmp/tri-review-codex.txt /tmp/tri-review-gemini.txt; do - if [ -f "$agent_file" ] && [ ! -s "$agent_file" ]; then - agent=$(basename "$agent_file" | sed 's/tri-review-//;s/\.txt//') - echo "WARNING: $agent produced empty output — check CLI installation and authentication" - fi -done -``` - -If an agent's output contains only stderr (error messages, not review content), note it in the consolidation as a failure rather than omitting it silently. - -## Autonomy Flags - -| Agent | Method | Flags | -|---|---|---| -| Claude | Native background agent | `isolation: worktree`, `background: true` | -| Codex | Plugin (preferred) / CLI fallback | `/codex:rescue --background` or `codex exec --full-auto` | -| Gemini | Plugin (preferred) / CLI fallback | `/gemini:rescue --background` or `-y` | - -## Step 5: Consolidate - -``` -## Triple-Agent Review: {branch_name} - -### Consensus (flagged by 2+ agents — high confidence) -- ... - -### Unique Findings (one agent only — worth investigating) -- Claude: ... -- Codex: ... -- Gemini: ... +devkit workflow run tri-review "{prompt or default}" ``` -## Presets +If `devkit workflow` is not available, follow this manually: -- `/tri:review` — full default review -- `/tri:review check for DRY violations` — custom prompt -- `/tri:review security focus` — security-oriented +1. **Gather context** — Capture diff via `git diff main...HEAD` (fall back to `HEAD~1..HEAD` or `--cached`). Write to temp file. Warn if >5000 lines. +2. **Build prompt** — Use custom prompt if provided, otherwise default: bugs, security, DRY violations, unnecessary complexity, missing edge cases. +3. **Detect agents** — Check for Codex plugin/CLI and Gemini plugin/CLI. Claude always runs. +4. **Dispatch in parallel** — Launch all available agents concurrently. Pass diff inline in each prompt (worktree agents can't see latest commits). Claude uses native background agent; Codex/Gemini use plugin (preferred) or CLI fallback. +5. **Consolidate** — Consensus findings (2+ agents) ranked higher. Unique findings listed per agent. ## Rules -- Claude always runs as native background agent (not `claude -p`) for token efficiency -- Codex and Gemini are optional — run if installed, skip gracefully if not -- Report which agents participated and how many perspectives were gathered -- If only Claude is available, still provide the full report format -- If one agent fails, report the others -- For large diffs (>5000 lines), warn and suggest specific files -- Clean up temp files after +- Claude always runs as native background agent (not `claude -p`) +- Codex and Gemini are optional — skip gracefully if not installed +- Diff MUST be passed inline — agents can't fetch it themselves +- Report which agents participated +- Sort by severity, then consensus count diff --git a/commands/tri-security.md b/commands/tri-security.md index 41929a3..148d6ba 100644 --- a/commands/tri-security.md +++ b/commands/tri-security.md @@ -1,172 +1,29 @@ --- -description: Multi-agent security audit — independent security reviews from available agents, consolidated with severity-ranked findings. +description: Triple-agent security audit — independent security reviews from Claude, Codex, and Gemini, consolidated with severity ranking. --- # Triple-Agent Security Audit -Independent security reviews from all available agents, consolidated into a severity-ranked report. +Dispatch a security audit to 2-3 AI agents in parallel and consolidate with severity-ranked findings. -## Step 0: Harness Detection +## Invoke -```bash -if command -v devkit >/dev/null 2>&1; then - echo "Go harness detected — delegating to devkit review --security for full output capture." - devkit review --security {prompt or default} - exit 0 -fi ``` - -If the `devkit` binary is in PATH, delegate entirely to it. Only fall through to plugin-based steps if the harness is not installed. - -## Step 1: Gather Scope - -Determine what to audit: -- If args specify files/dirs, use those -- If on a branch with changes, audit the diff: `git diff main...HEAD` -- Otherwise, audit the full project - -```bash -# Write directly to file — avoids shell variable limits -git diff main...HEAD > /tmp/tri-security-diff.txt 2>/dev/null -if [ ! -s /tmp/tri-security-diff.txt ]; then git diff HEAD~1..HEAD > /tmp/tri-security-diff.txt 2>/dev/null; fi -if [ ! -s /tmp/tri-security-diff.txt ]; then git diff --cached > /tmp/tri-security-diff.txt 2>/dev/null; fi -``` - -**CRITICAL:** All code/diff MUST be passed inline in each agent's prompt. Worktree-isolated agents cannot see the latest commits. - -## Step 2: Detect Available Agents - -Check for plugins first (preferred), then fall back to CLI: - -```bash -# Plugin detection (preferred — structured job management) -HAS_CODEX_PLUGIN=$(/codex:status >/dev/null 2>&1 && echo "yes" || echo "no") -HAS_GEMINI_PLUGIN=$(/gemini:status >/dev/null 2>&1 && echo "yes" || echo "no") - -# CLI fallback detection -HAS_CODEX_CLI=$(command -v codex && echo "yes" || echo "no") -HAS_GEMINI_CLI=$(command -v gemini && echo "yes" || echo "no") -``` - -Prefer plugin over CLI. - -## Step 3: Build the Prompt - -``` -Perform a security audit of this code. Check for: - -1. **Injection** — SQL injection, command injection, XSS, template injection -2. **Authentication/Authorization** — broken auth, missing access controls, privilege escalation -3. **Secrets** — hardcoded credentials, API keys, tokens in source -4. **Data Exposure** — sensitive data in logs, responses, or error messages -5. **Dependencies** — known vulnerable packages -6. **Cryptography** — weak algorithms, improper key handling -7. **Configuration** — debug mode in prod, permissive CORS, missing security headers -8. **Input Validation** — missing or insufficient validation at boundaries - -For each finding, report: -- Severity: CRITICAL / HIGH / MEDIUM / LOW -- Location: file and line number -- Description: what the vulnerability is -- Impact: what an attacker could do -- Fix: specific code change to remediate - -Code: {diff_or_source} -``` - -## Concurrency & Budget - -- **Concurrency limit:** Max 3 parallel agents. -- **Token budget:** ~300k tokens across all agents. -- **Rate limiting:** If API throttles, stagger agent launches. - -## Step 4: Dispatch (Hybrid, Graceful Degradation) - -**[PARALLEL]** Launch all available agents concurrently: - -### Claude — always runs - -Pass the code/diff inline — the agent runs in a worktree and cannot see recent commits. - +devkit workflow run tri-security "{scope or default}" ``` -Task: Security audit of this code. -Agent: security-auditor -Input: {prompt} -```diff -{diff} -``` -``` - - - -### Codex — if available - -``` -/codex:rescue --effort high --background "{prompt} $(cat /tmp/tri-security-diff.txt)" -``` - -Retrieve result with `/codex:result` when done. Omit `--model` to use the account default. - -### Gemini — if available - -**Plugin (preferred):** - -``` -/gemini:rescue --background "{prompt} $(cat /tmp/tri-security-diff.txt)" -``` +If `devkit workflow` is not available, follow this manually: -Retrieve result with `/gemini:result` when done. Omit `--model` to use the account default. - -**CLI fallback (only if plugin not installed):** - -```bash -if [ "$HAS_GEMINI_CLI" = "yes" ]; then - cat /tmp/tri-security-diff.txt | gemini -p "{prompt}" -y \ - --output-format text > /tmp/tri-security-gemini.txt 2>&1 & -fi - -wait -``` - -## Step 5: Consolidate - -``` -## Security Audit Report - -### Agents Used: {count}/3 -### Scope: {files_or_diff_range} - -### Critical / High Findings (consensus — flagged by 2+ agents) -| # | Severity | Location | Finding | Agents | -|---|----------|----------|---------|--------| -| 1 | CRITICAL | src/api.ts:42 | SQL injection in query builder | Claude, Codex, Gemini | -| 2 | HIGH | src/auth.ts:15 | Missing rate limiting on login | Claude, Gemini | - -### Medium / Low Findings -| # | Severity | Location | Finding | Agent | -|---|----------|----------|---------|-------| -| 3 | MEDIUM | config.ts:8 | Debug mode flag checked via env var | Claude | -| 4 | LOW | utils.ts:22 | Overly permissive regex | Codex | - -### Unique Findings (single agent — worth investigating) -- **Claude:** ... -- **Codex:** ... -- **Gemini:** ... - -### Summary -- Critical: {n} -- High: {n} -- Medium: {n} -- Low: {n} -- **Consensus findings (high confidence):** {n} -``` +1. **Gather scope** — Determine audit scope: full repo, specific directory, or changed files only. Capture relevant code. +2. **Detect agents** — Check for Codex and Gemini availability. Claude always runs. +3. **Build prompt** — Include OWASP top 10 categories, language-specific patterns, auth/authz checks, input validation, secrets detection, dependency vulnerabilities +4. **Dispatch in parallel** — Launch all available agents concurrently with full scope context +5. **Consolidate** — Rank by severity (critical/high/medium/low), then by consensus count (2+ agents = high confidence) ## Rules - Claude always runs — others are optional -- Consensus findings (2+ agents) ranked higher than single-agent findings +- Consensus findings ranked higher than single-agent findings - Sort by severity, then by consensus count - Include specific file:line references - Provide actionable fix for each finding -- Clean up temp files after diff --git a/commands/tri-test-gen.md b/commands/tri-test-gen.md index f5d925f..591271a 100644 --- a/commands/tri-test-gen.md +++ b/commands/tri-test-gen.md @@ -1,177 +1,30 @@ --- -description: Multi-agent test generation — each available agent generates tests independently, then merge for maximum coverage. +description: Triple-agent test generation — each agent generates tests independently, then merge for maximum coverage. --- # Triple-Agent Test Generation -Generate tests from all available agents in parallel, then merge the best tests into a comprehensive suite. +Dispatch test generation to 2-3 AI agents in parallel, then merge and deduplicate for maximum coverage. -## Step 0: Harness Detection +## Invoke -```bash -if command -v devkit >/dev/null 2>&1; then - echo "Go harness detected — delegating to devkit test-gen for full output capture." - devkit test-gen {target} --test {test_command} - exit 0 -fi ``` - -If the `devkit` binary is in PATH, delegate entirely to it. Only fall through to plugin-based steps if the harness is not installed. - -## Step 1: Analyze Target - -Read the target files and detect: -- Language, test framework, existing test patterns -- Public API surface and code paths to test - -## Step 2: Detect Available Agents - -Check for plugins first (preferred), then fall back to CLI: - -```bash -# Plugin detection (preferred — structured job management) -HAS_CODEX_PLUGIN=$(/codex:status >/dev/null 2>&1 && echo "yes" || echo "no") -HAS_GEMINI_PLUGIN=$(/gemini:status >/dev/null 2>&1 && echo "yes" || echo "no") - -# CLI fallback detection -HAS_CODEX_CLI=$(command -v codex && echo "yes" || echo "no") -HAS_GEMINI_CLI=$(command -v gemini && echo "yes" || echo "no") -``` - -Prefer plugin over CLI. - -## Step 2.5: Scenario Expansion (orchestrator selects applicable techniques — only selected items are injected into sub-agent prompts) - -When analyzing the target in Step 1, identify which scenario expansion techniques are highest-yield for each public function/method based on its behavior and risk profile. Use this to guide prompt construction — don't apply every technique to every function. - -| Technique | When high-yield | Example | -|-----------|-----------------|---------| -| Missing data | Functions with required parameters | "What if the required field is null/undefined?" | -| Boundary | Functions with numeric/collection inputs | "0, -1, MAX_INT, empty array, single element" | -| What-if | Functions with branching logic | "What if the input is empty string instead of valid?" | -| Ordering | Functions called in sequences or pipelines | "What if step 2 happens before step 1?" | -| Interruption | I/O or network-dependent functions | "What if the network drops mid-request?" | -| Stale data | Functions using caches or shared state | "What if the cached value changed between read and use?" | - -Prioritize unless existing test patterns in the repo establish a different convention: missing/null data > boundary conditions > what-if > ordering > interruption > stale data. - -Interruption and stale data scenarios often require integration-level test infrastructure (mocking I/O, time manipulation). Skip these when generating pure unit tests. - -Include only the applicable techniques in each agent's prompt — not the full table. - -## Step 3: Build the Prompt - -``` -Generate a comprehensive test suite for the following code. -- Cover happy paths, edge cases, error conditions, and boundary values. -- Use {framework} conventions. -- Match existing test patterns in the repo. -- Write tests that actually run — no placeholder assertions. -{scenario_guidance — applicable techniques from Step 2.5, tailored to this target} - -Target: {target_files} -Code: {source_code} -``` - -## Concurrency & Budget - -- **Concurrency limit:** Max 3 parallel agents. -- **Token budget:** ~400k tokens across all agents. Test generation can be verbose. -- **Rate limiting:** If API throttles, stagger agent launches. - -## Step 4: Dispatch (Hybrid, Graceful Degradation) - -**[PARALLEL]** Launch all available agents concurrently: - -**CRITICAL:** All source code MUST be passed inline in each agent's prompt. Worktree-isolated agents cannot see the latest commits. - -### Claude — always runs - -Pass the source code inline — the agent runs in a worktree and cannot see recent changes. - -``` -Task: Generate tests for {target}. -Agent: test-writer -Input: {prompt} - -{source_code — inlined here by the orchestrator} -``` - - - -### Codex — if available - -``` -/codex:rescue --effort high --background "{prompt} {source_code}" +devkit workflow run tri-test-gen "{target_file_or_directory}" ``` -Retrieve result with `/codex:result` when done. Omit `--model` to use the account default. - -### Gemini — if available - -**Plugin (preferred):** +If `devkit workflow` is not available, follow this manually: -``` -/gemini:rescue --background "{prompt} {source_code}" -``` - -Retrieve result with `/gemini:result` when done. Omit `--model` to use the account default. - -**CLI fallback (only if plugin not installed):** - -```bash -if [ "$HAS_GEMINI_CLI" = "yes" ]; then - gemini -p "{prompt} {source_code}" -y \ - --output-format text > /tmp/tri-test-gemini.txt 2>&1 & -fi - -wait -``` - -## Step 5: Merge & Deduplicate - -Analyze test suites from all agents: -1. Identify unique test cases across all suites -2. Remove duplicates (same assertion, different wording) -3. Keep the best implementation of each test (clearest, most thorough) -4. Combine into one unified test file - -## Step 6: Run & Fix - -```bash -{test_command} 2>&1 -``` - -If tests fail, fix them (up to 3 attempts). - -## Step 7: Report - -``` -## Triple Test Generation: {target} - -### Agents Used: {count}/3 - -### Test Contributions -| Agent | Tests Generated | Unique Tests Kept | -|-------|----------------|-------------------| -| Claude | 12 | 8 | -| Codex | 10 | 4 | -| Gemini | 8 | 3 | - -### Final Suite -- **Total tests:** 15 -- **All passing:** ✓ -- **Coverage:** {coverage}% - -### Files Created -- {test_file_1} -- {test_file_2} -``` +1. **Analyze target** — Read source code, identify public API, detect test framework and conventions +2. **Scenario expansion** — Generate test scenarios using: boundary values, equivalence partitioning, error injection, state transitions, concurrency (if applicable) +3. **Detect agents** — Check for Codex and Gemini availability. Claude always runs. +4. **Dispatch in parallel** — Each agent generates tests independently for the target +5. **Merge & deduplicate** — Combine test files, remove duplicates, resolve naming conflicts +6. **Run & fix** — Execute merged tests, fix any failures +7. **Report** — Coverage summary, test count per agent, merged result ## Rules -- Claude always runs — others are optional -- Merge the best from each, don't just concatenate -- Final suite must pass before reporting success -- Match existing project test conventions -- Clean up temp files after +- Claude always runs — others optional +- Each agent generates independently — no shared context between agents +- Merge by deduplicating equivalent test cases +- All merged tests must pass before reporting From b6902695cb9aa5738c2123d1d3ddc4310c9e8b92 Mon Sep 17 00:00:00 2001 From: Tym Rabchuk Date: Thu, 9 Apr 2026 19:46:24 -0400 Subject: [PATCH 2/3] Address review findings: create missing workflows, fix references MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Create audit.yml and pr-ready.yml (spec-defined workflows) - Remove invoke sections from tri-test-gen, decompose, repo-map (no YAML workflows — these stay as standalone commands) - Fix autoloop guard-check timing in fallback - Add source-of-truth guardrail to tri-review fallback - Update README/ROADMAP: 16 → 18 workflows --- README.md | 2 +- ROADMAP.md | 4 +-- commands/autoloop.md | 2 +- commands/decompose.md | 8 +---- commands/repo-map.md | 8 +---- commands/tri-review.md | 2 +- commands/tri-test-gen.md | 8 +---- workflows/audit.yml | 49 ++++++++++++++++++++++++++ workflows/pr-ready.yml | 74 ++++++++++++++++++++++++++++++++++++++++ 9 files changed, 131 insertions(+), 26 deletions(-) create mode 100644 workflows/audit.yml create mode 100644 workflows/pr-ready.yml diff --git a/README.md b/README.md index d3804fa..4fb6ee9 100644 --- a/README.md +++ b/README.md @@ -310,7 +310,7 @@ devkit/ ├── skills/ # 18 context-activated skills ├── agents/ # 6 agents (reviewer, researcher, improver, ...) ├── hooks/ # 10 hooks (safety, security, quality gates) -├── workflows/ # 16 YAML workflow definitions +├── workflows/ # 18 YAML workflow definitions ├── resources/rules/ # Language-specific coding rules ├── presets/ # Reserved for future use ├── src/ # Go CLI harness diff --git a/ROADMAP.md b/ROADMAP.md index 83a42ea..03c93a0 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -12,7 +12,7 @@ - **Early-exit conditions** — Self-improvement loops stop when goal is met, not just at max iterations - **Token budget guidance** — Per-command budget recommendations with model downgrade patterns - **RTK token optimization** — Optional PreToolUse hook compresses Bash output via RTK (60-90% savings) -- **16 YAML workflows** — Portable workflow definitions (feature, bugfix, refactor, research, deep-research, autoloop, self-*, tri-*) +- **18 YAML workflows** — Portable workflow definitions (feature, bugfix, refactor, research, deep-research, autoloop, self-*, tri-*) - **Separate marketplace** — Multi-plugin marketplace at `5uck1ess/marketplace` - **Companion ecosystem** — Evaluated official marketplace, documented holistic setup with 7 complementary plugins - **Hypothesis-driven perf** — Evidence gathering, ranked hypotheses, one-at-a-time testing replaces blind benchmark loops @@ -34,6 +34,6 @@ Items below were on the roadmap but determined to be unnecessary — either alre | Stop hook redesign | Still fires every turn, but exits early with `approve` when no files are changed — near-instant on clean trees, so the performance concern is moot. Revisit only if it causes measurable latency. | | Cost event hooks | Budget enforcement already exists in the Go engine via `overBudget()` + `addCost()` callbacks with hard limits | | Execution registry | Step tracking already handled by SQLite via `lib.DB` with status, cost, and timing per step | -| Preset library | The 16 YAML workflows and 18 skills already serve this purpose | +| Preset library | The 18 YAML workflows and 18 skills already serve this purpose | | Framework-specific review checklists | `lang-review.sh` covers language-level patterns; framework-specific rules are better added per-project via hookify | | Conditional hook firing | Hooks already self-filter internally (extension checks, changed-file checks); a generic condition system adds complexity for no current need | diff --git a/commands/autoloop.md b/commands/autoloop.md index 33f4951..2e08529 100644 --- a/commands/autoloop.md +++ b/commands/autoloop.md @@ -19,7 +19,7 @@ If `devkit workflow` is not available, follow this manually: 3. **Audit** — Analyze codebase for improvement opportunities 4. **Fix** — Make one targeted improvement 5. **Measure** — Re-run metric command -6. **Compare** — Did the metric improve? Run guard command if set. +6. **Compare + guard** — Did the metric improve? If yes AND a guard command is set, run it. If guard fails, treat as regression and revert. 7. **Keep or revert** — Keep if improved and guard passes; revert otherwise. If 3+ consecutive failures, escalate. 8. **Report** — Summary of iterations, improvements kept, and final state diff --git a/commands/decompose.md b/commands/decompose.md index 2e7f25e..03e1db5 100644 --- a/commands/decompose.md +++ b/commands/decompose.md @@ -6,13 +6,7 @@ description: Decompose a high-level goal into a task DAG — break down, assign Break a high-level goal into an executable task graph: clarify → decompose → resolve order → execute → report. -## Invoke - -``` -devkit workflow run decompose "{goal_description}" -``` - -If `devkit workflow` is not available, follow this manually: +## Steps 1. **Clarify the goal** — Restate precisely. Ask user for clarification if ambiguous. 2. **Decompose into tasks** — Break into 3-10 concrete, testable tasks. Each must have clear done criteria. Identify parallelizable groups. diff --git a/commands/repo-map.md b/commands/repo-map.md index 52bd4db..3efc6d2 100644 --- a/commands/repo-map.md +++ b/commands/repo-map.md @@ -6,13 +6,7 @@ description: Build an AST-based symbol index of the repository — exports, func Build and cache an AST-based symbol index for fast codebase navigation. -## Invoke - -``` -devkit workflow run repo-map -``` - -If `devkit workflow` is not available, follow this manually: +## Steps 1. **Detect languages** — Scan for language markers (go.mod, package.json, pyproject.toml, Cargo.toml, etc.) 2. **Extract symbols (AST)** — Use language-specific AST tools to extract exports, functions, classes, types, interfaces. Fall back to regex if AST tools unavailable. diff --git a/commands/tri-review.md b/commands/tri-review.md index baefd79..8f7475b 100644 --- a/commands/tri-review.md +++ b/commands/tri-review.md @@ -17,7 +17,7 @@ If `devkit workflow` is not available, follow this manually: 1. **Gather context** — Capture diff via `git diff main...HEAD` (fall back to `HEAD~1..HEAD` or `--cached`). Write to temp file. Warn if >5000 lines. 2. **Build prompt** — Use custom prompt if provided, otherwise default: bugs, security, DRY violations, unnecessary complexity, missing edge cases. 3. **Detect agents** — Check for Codex plugin/CLI and Gemini plugin/CLI. Claude always runs. -4. **Dispatch in parallel** — Launch all available agents concurrently. Pass diff inline in each prompt (worktree agents can't see latest commits). Claude uses native background agent; Codex/Gemini use plugin (preferred) or CLI fallback. +4. **Dispatch in parallel** — Launch all available agents concurrently. Pass diff inline in each prompt — worktree-isolated agents can't see latest commits, so the provided diff is the ONLY source of truth. Do NOT instruct agents to read files from the worktree to verify changes. Claude uses native background agent; Codex/Gemini use plugin (preferred) or CLI fallback. 5. **Consolidate** — Consensus findings (2+ agents) ranked higher. Unique findings listed per agent. ## Rules diff --git a/commands/tri-test-gen.md b/commands/tri-test-gen.md index 591271a..da5e0b2 100644 --- a/commands/tri-test-gen.md +++ b/commands/tri-test-gen.md @@ -6,13 +6,7 @@ description: Triple-agent test generation — each agent generates tests indepen Dispatch test generation to 2-3 AI agents in parallel, then merge and deduplicate for maximum coverage. -## Invoke - -``` -devkit workflow run tri-test-gen "{target_file_or_directory}" -``` - -If `devkit workflow` is not available, follow this manually: +## Steps 1. **Analyze target** — Read source code, identify public API, detect test framework and conventions 2. **Scenario expansion** — Generate test scenarios using: boundary values, equivalence partitioning, error injection, state transitions, concurrency (if applicable) diff --git a/workflows/audit.yml b/workflows/audit.yml new file mode 100644 index 0000000..e314dd6 --- /dev/null +++ b/workflows/audit.yml @@ -0,0 +1,49 @@ +name: Project Audit +description: Unified project health audit — detect ecosystem, audit deps, lint, security, report + +steps: + - id: detect + command: | + echo "go:$(test -f go.mod && echo yes || echo no)" + echo "node:$(test -f package.json && echo yes || echo no)" + echo "python:$(test -f requirements.txt -o -f pyproject.toml && echo yes || echo no)" + echo "rust:$(test -f Cargo.toml && echo yes || echo no)" + + - id: deps + model: smart + prompt: | + Detected ecosystems: {{detect}} + + Run dependency audit commands for each detected ecosystem: + - Node: npm audit --json + - Go: govulncheck ./... + - Python: pip-audit + - Rust: cargo audit + + Report vulnerabilities, outdated packages, and license issues. + + - id: lint + model: smart + prompt: | + Detected ecosystems: {{detect}} + + Run linters for each detected ecosystem: + - Node: npx eslint . or npx tsc --noEmit + - Go: golangci-lint run + - Python: ruff check . + - Rust: cargo clippy + + Report errors and warnings. + + - id: report + model: fast + prompt: | + Compile audit report. + + Ecosystem detection: {{detect}} + Dependencies: {{deps}} + Lint: {{lint}} + + Score overall health (A-F) with breakdown by category. + Prioritize findings by severity. + Provide actionable recommendations. diff --git a/workflows/pr-ready.yml b/workflows/pr-ready.yml new file mode 100644 index 0000000..f9f79a0 --- /dev/null +++ b/workflows/pr-ready.yml @@ -0,0 +1,74 @@ +name: PR Ready +description: Full PR preparation pipeline — lint, test, security, changelog, create PR + +steps: + - id: validate + model: fast + prompt: | + Validate the current branch is ready for PR preparation: + - Not on main/master + - No uncommitted changes + - Has commits ahead of main + + Run git commands to check. If any validation fails, report and stop. + + - id: necessity + model: smart + prompt: | + Review the diff (git diff main...HEAD) for unnecessary changes: + - Debug artifacts (console.log, print statements) + - Unrelated file changes + - Generated files that shouldn't be committed + + Remove anything that doesn't belong. Report what was cleaned. + + - id: lint + model: smart + prompt: | + Run the project's linter on changed files. + Fix any violations. + + If clean, say DONE. + loop: + max: 5 + until: DONE + + - id: test + model: smart + prompt: | + Run the full test suite. + Fix any failures. + + If all pass, say DONE. + loop: + max: 5 + until: DONE + + - id: security + model: smart + prompt: | + Review changed files for security issues: + - Hardcoded secrets or API keys + - SQL injection patterns + - XSS vulnerabilities + - Command injection + - Path traversal + - Insecure dependencies + + Report findings with severity and file:line references. + + - id: changelog + model: fast + prompt: | + Generate a changelog entry from git diff main...HEAD. + Summarize what changed and why. + + - id: create-pr + model: smart + prompt: | + Create the PR: + 1. Push the branch to remote + 2. Create PR with gh pr create + 3. Include: title, summary, changelog, test plan + + Use the changelog from the previous step. From 7751346cf0759a87c31a4a8e6cb7fddbaf920d29 Mon Sep 17 00:00:00 2001 From: Tym Rabchuk Date: Thu, 9 Apr 2026 19:47:12 -0400 Subject: [PATCH 3/3] Fix refactor loop/budget mismatch, document tri-* architecture divergence MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - refactor.md: loop max 10→15, budget 300k→400k (match YAML) - tri-review/debug/security/dispatch: add note explaining YAML uses model tiers while fallback uses external agents (intentional — richer path when engine unavailable) --- commands/refactor.md | 4 ++-- commands/tri-debug.md | 2 ++ commands/tri-dispatch.md | 2 ++ commands/tri-review.md | 2 ++ commands/tri-security.md | 2 ++ 5 files changed, 10 insertions(+), 2 deletions(-) diff --git a/commands/refactor.md b/commands/refactor.md index 7776b39..6154ebc 100644 --- a/commands/refactor.md +++ b/commands/refactor.md @@ -16,7 +16,7 @@ If `devkit workflow` is not available, follow this manually: 1. **Analyze** — Identify code smells, duplication, complexity hotspots in the target 2. **Plan** — Create ordered list of transformations; each should be a single testable change -3. **Refactor** — Execute transformations one at a time; run tests after each (loop max 10) +3. **Refactor** — Execute transformations one at a time; run tests after each (loop max 15) 4. **Run tests** — Full test suite must pass 5. **Before/after comparison** — Show what changed: complexity metrics, line counts, readability improvements @@ -26,4 +26,4 @@ If `devkit workflow` is not available, follow this manually: - One transformation at a time - Preserve behavior — refactoring changes structure, not functionality - If tests fail, revert the transformation -- Token budget: ~300k tokens +- Token budget: ~400k tokens diff --git a/commands/tri-debug.md b/commands/tri-debug.md index 4162e12..db7ad52 100644 --- a/commands/tri-debug.md +++ b/commands/tri-debug.md @@ -12,6 +12,8 @@ Dispatch a bug report to 2-3 AI agents in parallel, get independent root-cause h devkit workflow run tri-debug "{bug_description}" ``` +The YAML workflow uses model tiers (smart/general/fast) for parallelism. The fallback below uses external agents (Claude/Codex/Gemini) — the richer path when the engine is unavailable. + If `devkit workflow` is not available, follow this manually: 1. **Gather context** — Collect error messages, stack traces, reproduction steps, and relevant code diff --git a/commands/tri-dispatch.md b/commands/tri-dispatch.md index d6a117f..700b616 100644 --- a/commands/tri-dispatch.md +++ b/commands/tri-dispatch.md @@ -12,6 +12,8 @@ Send an arbitrary task to 2-3 AI agents in parallel and compare their results. devkit workflow run tri-dispatch "{task_description}" ``` +The YAML workflow uses model tiers (smart/general/fast) for parallelism. The fallback below uses external agents (Claude/Codex/Gemini) — the richer path when the engine is unavailable. + If `devkit workflow` is not available, follow this manually: 1. **Detect agents** — Check for Codex and Gemini availability. Claude always runs. diff --git a/commands/tri-review.md b/commands/tri-review.md index 8f7475b..c5b1517 100644 --- a/commands/tri-review.md +++ b/commands/tri-review.md @@ -12,6 +12,8 @@ Dispatch the same code review to 2-3 AI agents in parallel and consolidate resul devkit workflow run tri-review "{prompt or default}" ``` +The YAML workflow uses model tiers (smart/general/fast) for parallelism. The fallback below uses external agents (Claude/Codex/Gemini) — the richer path when the engine is unavailable. + If `devkit workflow` is not available, follow this manually: 1. **Gather context** — Capture diff via `git diff main...HEAD` (fall back to `HEAD~1..HEAD` or `--cached`). Write to temp file. Warn if >5000 lines. diff --git a/commands/tri-security.md b/commands/tri-security.md index 148d6ba..19ee340 100644 --- a/commands/tri-security.md +++ b/commands/tri-security.md @@ -12,6 +12,8 @@ Dispatch a security audit to 2-3 AI agents in parallel and consolidate with seve devkit workflow run tri-security "{scope or default}" ``` +The YAML workflow dispatches by security domain (injection/auth/config) across model tiers. The fallback below uses external agents (Claude/Codex/Gemini) — the richer path when the engine is unavailable. + If `devkit workflow` is not available, follow this manually: 1. **Gather scope** — Determine audit scope: full repo, specific directory, or changed files only. Capture relevant code.