From 91fca2973b3811dff1cfdf00360c9f102e62782f Mon Sep 17 00:00:00 2001 From: Tym Rabchuk Date: Sun, 5 Apr 2026 19:21:16 -0400 Subject: [PATCH 1/4] =?UTF-8?q?Add=20/self:audit=20=E2=80=94=20Karpathy-st?= =?UTF-8?q?yle=20measure-first=20codebase=20audit?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Systematic audit that measures everything before forming opinions: coverage per package, lint issues, security vulns, stale code, git health. Ranks hypotheses by impact/effort ratio with evidence citations. Suggests which devkit command to use for each hypothesis. Does NOT fix anything — only measures and analyzes. You decide what to test, one at a time. --- README.md | 5 +- ROADMAP.md | 6 +- commands/self-audit.md | 187 +++++++++++++++++++++++++++++++++++++++ workflows/self-audit.yml | 103 +++++++++++++++++++++ 4 files changed, 296 insertions(+), 5 deletions(-) create mode 100644 commands/self-audit.md create mode 100644 workflows/self-audit.yml diff --git a/README.md b/README.md index 6ca4c68..06fd906 100644 --- a/README.md +++ b/README.md @@ -133,6 +133,7 @@ Automated propose → measure → keep/discard → repeat cycles. | `/self:lint` | Iteratively fix lint/type errors until zero remain | | `/self:perf` | Hypothesis-driven performance investigation — evidence, hypotheses, one-at-a-time testing | | `/self:migrate` | Incremental migration (JS→TS, class→hooks, etc.) with test gate | +| `/self:audit` | Karpathy-style codebase audit — measure everything, rank hypotheses by evidence, one-at-a-time | ### Multi-Agent Commands (Claude + optional Codex/Gemini) @@ -311,7 +312,7 @@ devkit/ │ └── plugin.json # Plugin metadata (name, version, author) ├── ROADMAP.md # Implemented features and future plans ├── PREFERENCES.md # Agent behavior guidelines -├── commands/ # 21 slash commands (tab-completable) +├── commands/ # 22 slash commands (tab-completable) │ ├── tri-*.md # Multi-agent dispatch (5) │ ├── self-*.md # Self-improvement loops (5) │ ├── pr-ready.md # PR preparation pipeline @@ -360,7 +361,7 @@ devkit/ │ ├── lang-review.sh # Language-aware code quality (Go/TS/Rust/Python/Shell) │ ├── subagent-stop.sh # Subagent work verification │ └── stop-gate.sh # Consolidated quality gate (cross-domain + vet/lint) -├── workflows/ # 13 YAML workflow definitions +├── workflows/ # 14 YAML workflow definitions ├── presets/ # Reserved for future use ├── .github/workflows/ # CI/CD │ ├── ci.yml # Build + test + vet on push/PR diff --git a/ROADMAP.md b/ROADMAP.md index 8102b02..201e79f 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -2,7 +2,7 @@ ## Implemented -- **21 slash commands** — Lifecycle workflows, self-improvement loops, multi-agent dispatch, project health audit, post-PR monitoring, AST repo mapping +- **22 slash commands** — Lifecycle workflows, self-improvement loops, multi-agent dispatch, project health audit, post-PR monitoring, AST repo mapping, Karpathy-style self-audit - **16 context-activated skills** — 7 auto-trigger workflows (test-gen, doc-gen, changelog, onboard, research, deep-research, scrape) + 6 coding principles (executing, clean-code, DRY, YAGNI, dont-reinvent, stuck) + 2 tools (gcli, creating-workflows) + 1 iteration memory (scratchpad) - **6 agents** — Scoped tool access, worktree isolation, model assignment - **10 hooks** — Safety (destructive command blocking, edit-time security patterns, PR gate), observability (audit trail, slop detection, post-validation, subagent verification, language-aware code review), optimization (RTK token compression) @@ -12,7 +12,7 @@ - **Early-exit conditions** — Self-improvement loops stop when goal is met, not just at max iterations - **Token budget guidance** — Per-command budget recommendations with model downgrade patterns - **RTK token optimization** — Optional PreToolUse hook compresses Bash output via RTK (60-90% savings) -- **13 YAML workflows** — Portable workflow definitions (feature, bugfix, refactor, research, deep-research, self-*, tri-*) +- **14 YAML workflows** — Portable workflow definitions (feature, bugfix, refactor, research, deep-research, self-*, tri-*) - **Separate marketplace** — Multi-plugin marketplace at `5uck1ess/marketplace` - **Companion ecosystem** — Evaluated official marketplace, documented holistic setup with 7 complementary plugins - **Hypothesis-driven perf** — Evidence gathering, ranked hypotheses, one-at-a-time testing replaces blind benchmark loops @@ -34,6 +34,6 @@ Items below were on the roadmap but determined to be unnecessary — either alre | Stop hook redesign | Still fires every turn, but exits early with `approve` when no files are changed — near-instant on clean trees, so the performance concern is moot. Revisit only if it causes measurable latency. | | Cost event hooks | Budget enforcement already exists in the Go engine via `overBudget()` + `addCost()` callbacks with hard limits | | Execution registry | Step tracking already handled by SQLite via `lib.DB` with status, cost, and timing per step | -| Preset library | The 13 YAML workflows and 16 skills already serve this purpose | +| Preset library | The 14 YAML workflows and 16 skills already serve this purpose | | Framework-specific review checklists | `lang-review.sh` covers language-level patterns; framework-specific rules are better added per-project via hookify | | Conditional hook firing | Hooks already self-filter internally (extension checks, changed-file checks); a generic condition system adds complexity for no current need | diff --git a/commands/self-audit.md b/commands/self-audit.md new file mode 100644 index 0000000..cd8554d --- /dev/null +++ b/commands/self-audit.md @@ -0,0 +1,187 @@ +--- +description: Karpathy-style self-audit — measure the codebase, rank improvement hypotheses by evidence, present actionable plan. Don't guess, measure. +--- + +# Self-Audit + +Systematic codebase audit using Karpathy's research methodology: measure first, form hypotheses from evidence, rank by impact. No guessing, no speculative improvements. + +## Philosophy + +> "Most of the value comes from understanding the system deeply. People jump to building stuff without knowing what's actually broken." — Karpathy approach + +This command does NOT fix anything. It measures, analyzes, and presents a ranked hypothesis list. You decide which to test. One at a time. + +## Step 1: Detect Stack + +```bash +# Detect what's in this repo +HAS_GO=$([ -f go.mod ] || [ -f */go.mod ] && echo yes || echo no) +HAS_TS=$([ -f tsconfig.json ] || [ -f package.json ] && echo yes || echo no) +HAS_PYTHON=$([ -f pyproject.toml ] || [ -f requirements.txt ] || [ -f setup.py ] && echo yes || echo no) +HAS_RUST=$([ -f Cargo.toml ] && echo yes || echo no) +HAS_TESTS=$(find . -name '*_test.go' -o -name '*.test.ts' -o -name '*.test.js' -o -name 'test_*.py' -o -name '*_test.rs' 2>/dev/null | head -1) +HAS_CI=$([ -d .github/workflows ] || [ -f .gitlab-ci.yml ] || [ -f Jenkinsfile ] && echo yes || echo no) +HAS_DOCKER=$([ -f Dockerfile ] || [ -f docker-compose.yml ] && echo yes || echo no) +``` + +Report what was detected. Skip measurements for stacks not present. + +## Step 2: Measure (the data collection phase) + +Run ALL applicable measurements. Do not skip any. Collect raw numbers. + +**[PARALLEL]** Run these measurement groups concurrently: + +### Code Quality + +```bash +# Go +go vet ./... 2>&1 | wc -l # vet issues +go test -cover ./... 2>&1 # coverage per package +gofmt -l . 2>&1 | wc -l # formatting issues + +# TypeScript/JavaScript +npx tsc --noEmit 2>&1 | grep -c 'error TS' # type errors +npx eslint . --format compact 2>&1 | wc -l # lint issues + +# Python +ruff check . 2>&1 | wc -l # lint issues +mypy . 2>&1 | grep -c 'error:' # type errors + +# Rust +cargo clippy 2>&1 | grep -c 'error\[' # clippy errors +``` + +### Test Coverage + +```bash +# Get per-package/per-file coverage — the NUMBERS matter +# Go: go test -cover ./... +# TS: npx jest --coverage --coverageReporters text-summary +# Python: pytest --cov --cov-report term-missing +# Rust: cargo tarpaulin --out stdout +``` + +Record: total coverage %, lowest-coverage packages, untested files. + +### Security + +```bash +# Dependency vulnerabilities +# Go: govulncheck ./... +# Node: npm audit +# Python: pip-audit or safety check +# Rust: cargo audit + +# Hardcoded secrets +grep -rn 'password\s*=\s*"[^"]*"' --include='*.go' --include='*.ts' --include='*.py' . 2>/dev/null | head -5 +grep -rn 'api_key\s*=\s*"[^"]*"' --include='*.go' --include='*.ts' --include='*.py' . 2>/dev/null | head -5 +``` + +### Stale Code + +```bash +# Dead files (not imported/referenced) +# Unused dependencies +# go mod tidy -diff (shows removable deps) +# npm prune --dry-run +# TODO/FIXME/HACK count +grep -rn 'TODO\|FIXME\|HACK\|XXX' --include='*.go' --include='*.ts' --include='*.py' --include='*.rs' . 2>/dev/null | wc -l +``` + +### Documentation + +```bash +# README exists and is recent? +stat -f '%Sm' README.md 2>/dev/null || stat -c '%y' README.md 2>/dev/null +# API docs? +# Changelog? +# Are counts/claims in docs accurate? (run validate-counts if available) +``` + +### Git Health + +```bash +# Recent commit frequency +git log --oneline --since='30 days ago' | wc -l +# Stale branches +git branch -r --merged main | grep -v main | wc -l +# Large files in history +git rev-list --objects --all | git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' | awk '/^blob/ {print $3, $4}' | sort -rn | head -5 +``` + +## Step 3: Analyze (turn measurements into hypotheses) + +``` +For each measurement category, form hypotheses: + +IF coverage < 50% in a package THEN "Improve test coverage in {package} — currently {N}%" +IF vet/lint issues > 0 THEN "Fix {N} vet/lint issues" +IF vulnerability count > 0 THEN "Patch {N} known vulnerabilities" +IF TODO count > 20 THEN "Triage {N} TODOs — many may be stale" +IF stale branches > 5 THEN "Clean up {N} merged branches" +IF no CI THEN "Add CI pipeline — no automated checks exist" +IF hardcoded secrets found THEN "Move {N} hardcoded secrets to environment variables" +IF lowest coverage package < 30% THEN "Package {name} has {N}% coverage — highest risk area" + +For each hypothesis, estimate: +- IMPACT: How much does this improve the codebase? (high/medium/low) +- EFFORT: How hard is this to fix? (high/medium/low) +- EVIDENCE: What measurement supports this? (cite the number) +``` + +## Step 4: Rank and Present + +``` +## Self-Audit: {repo-name} + +### Stack Detected +{languages, frameworks, CI, Docker, etc.} + +### Raw Measurements +| Category | Metric | Value | +|----------|--------|-------| +| Coverage | Overall | X% | +| Coverage | Lowest package | {name} at X% | +| Lint | Issues | N | +| Security | Vulnerabilities | N | +| Stale | TODOs | N | +| Stale | Merged branches | N | +| Git | 30-day commits | N | + +### Ranked Hypotheses (by impact/effort ratio) + +| # | Hypothesis | Impact | Effort | Evidence | +|---|-----------|--------|--------|----------| +| 1 | {highest impact/effort ratio} | high | low | {measurement} | +| 2 | ... | ... | ... | ... | +| N | ... | ... | ... | ... | + +### Recommended Next Steps +1. Test hypothesis #1 first — highest impact for lowest effort +2. Use `/devkit:self-improve` or `/devkit:self-test` to execute +3. Measure again after each fix to verify improvement +4. One change at a time — don't bundle + +### What NOT to Do +- Don't fix everything at once +- Don't start with low-impact items just because they're easy +- Don't add features — this is about quality, not functionality +- Don't refactor code that has no tests (add tests first) +``` + +## Budget + +- **Token budget:** ~200k tokens. Measurement is cheap; the value is in the analysis. +- **Early exit:** If the codebase is clean (no lint issues, >80% coverage, no vulns), say so and skip the hypothesis phase. + +## Rules + +- Measure EVERYTHING before forming opinions — no guessing +- Cite numbers — every hypothesis must reference a specific measurement +- Rank by impact/effort ratio — not by what's easiest or most interesting +- One at a time — never suggest bundling fixes +- Don't fix anything — this command only measures and analyzes +- Be honest — if the codebase is clean, say so. Don't manufacture problems. +- Suggest the right devkit command for each hypothesis (self-improve, self-test, self-lint, tri-security, etc.) diff --git a/workflows/self-audit.yml b/workflows/self-audit.yml new file mode 100644 index 0000000..ad75f7a --- /dev/null +++ b/workflows/self-audit.yml @@ -0,0 +1,103 @@ +name: Self-Audit +description: Karpathy-style codebase audit — measure everything, rank hypotheses by evidence, present actionable plan + +budget: + limit: 200000 + downgrade: fast + +steps: + - id: detect + model: fast + prompt: | + Detect the tech stack in this repository. + Check for: Go (go.mod), TypeScript/JS (package.json, tsconfig.json), + Python (pyproject.toml, requirements.txt), Rust (Cargo.toml), + CI (.github/workflows), Docker (Dockerfile), tests. + + Report what you found as a structured list. + + - id: measure-quality + model: general + prompt: | + Stack detected: {{detect}} + + Run code quality measurements for each detected language: + - Go: go vet, go test -cover, gofmt -l + - TS/JS: tsc --noEmit, eslint + - Python: ruff check, mypy + - Rust: cargo clippy + + Record raw numbers. Skip languages not in the stack. + + - id: measure-security + model: general + prompt: | + Stack detected: {{detect}} + + Run security measurements: + - Dependency vulnerabilities (govulncheck, npm audit, pip-audit, cargo audit) + - Hardcoded secrets grep + - TODO/FIXME/HACK count + + Record raw numbers. + + - id: measure-git + model: fast + prompt: | + Measure git health: + - 30-day commit count + - Stale merged branches count + - Large files in history (top 5 by size) + - README last modified date + + Record raw numbers. + + - id: dispatch-measurements + parallel: [measure-quality, measure-security, measure-git] + + - id: analyze + model: smart + prompt: | + Stack: {{detect}} + Quality measurements: {{measure-quality}} + Security measurements: {{measure-security}} + Git health: {{measure-git}} + + Form hypotheses from the evidence. For each issue found: + 1. State the hypothesis clearly + 2. Cite the specific measurement that supports it + 3. Rate IMPACT (high/medium/low) and EFFORT (high/medium/low) + 4. Suggest which devkit command to use (self-improve, self-test, self-lint, tri-security) + + Rank by impact/effort ratio. Do NOT suggest fixes — only measure and analyze. + + If the codebase is clean (no lint issues, >80% coverage, no vulns), say so. + Do not manufacture problems. + + - id: synthesize + model: smart + prompt: | + Analysis: {{analyze}} + + Write the final self-audit report: + + ## Self-Audit: {repo} + + ### Stack Detected + {from detect step} + + ### Raw Measurements + {table of all metrics with values} + + ### Ranked Hypotheses + {numbered list, highest impact/effort first, with evidence citations} + + ### Recommended Next Steps + 1. Which hypothesis to test first and why + 2. Which devkit command to run + 3. Reminder: one change at a time, measure after each + + ### What NOT to Do + - Don't fix everything at once + - Don't start with easy low-impact items + - Don't refactor untested code (add tests first) From eb111350b206a0d0e4431ef4cbe366887e49e1cf Mon Sep 17 00:00:00 2001 From: Tym Rabchuk Date: Sun, 5 Apr 2026 19:30:36 -0400 Subject: [PATCH 2/4] Fix review findings: self-* count, quote attribution, contradictions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - README tree: self-*.md (5) → (6) after adding self-audit - Remove fake Karpathy quote marks — reframe as paraphrase - Fix "What NOT to Do" contradiction: was implying fixing is in scope, now says "use the suggested devkit command" instead - Sync workflow "What NOT to Do" with command (4 bullets, same content) - Separate HAS_TS from HAS_JS detection (package.json alone ≠ TypeScript) --- README.md | 2 +- commands/self-audit.md | 9 +++++---- workflows/self-audit.yml | 5 +++-- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 06fd906..133f768 100644 --- a/README.md +++ b/README.md @@ -314,7 +314,7 @@ devkit/ ├── PREFERENCES.md # Agent behavior guidelines ├── commands/ # 22 slash commands (tab-completable) │ ├── tri-*.md # Multi-agent dispatch (5) -│ ├── self-*.md # Self-improvement loops (5) +│ ├── self-*.md # Self-improvement loops (6) │ ├── pr-ready.md # PR preparation pipeline │ ├── pr-monitor.md # Post-PR review monitor │ ├── bugfix.md # Bug fix lifecycle diff --git a/commands/self-audit.md b/commands/self-audit.md index cd8554d..8243656 100644 --- a/commands/self-audit.md +++ b/commands/self-audit.md @@ -8,7 +8,7 @@ Systematic codebase audit using Karpathy's research methodology: measure first, ## Philosophy -> "Most of the value comes from understanding the system deeply. People jump to building stuff without knowing what's actually broken." — Karpathy approach +> Most of the value comes from understanding the system deeply. Don't jump to building stuff without knowing what's actually broken. — inspired by Karpathy's research methodology This command does NOT fix anything. It measures, analyzes, and presents a ranked hypothesis list. You decide which to test. One at a time. @@ -17,7 +17,8 @@ This command does NOT fix anything. It measures, analyzes, and presents a ranked ```bash # Detect what's in this repo HAS_GO=$([ -f go.mod ] || [ -f */go.mod ] && echo yes || echo no) -HAS_TS=$([ -f tsconfig.json ] || [ -f package.json ] && echo yes || echo no) +HAS_TS=$([ -f tsconfig.json ] && echo yes || echo no) +HAS_JS=$([ -f package.json ] && echo yes || echo no) HAS_PYTHON=$([ -f pyproject.toml ] || [ -f requirements.txt ] || [ -f setup.py ] && echo yes || echo no) HAS_RUST=$([ -f Cargo.toml ] && echo yes || echo no) HAS_TESTS=$(find . -name '*_test.go' -o -name '*.test.ts' -o -name '*.test.js' -o -name 'test_*.py' -o -name '*_test.rs' 2>/dev/null | head -1) @@ -165,10 +166,10 @@ For each hypothesis, estimate: 4. One change at a time — don't bundle ### What NOT to Do -- Don't fix everything at once +- Don't fix anything from this report directly — use the suggested devkit command instead +- Don't fix everything at once — one hypothesis at a time - Don't start with low-impact items just because they're easy - Don't add features — this is about quality, not functionality -- Don't refactor code that has no tests (add tests first) ``` ## Budget diff --git a/workflows/self-audit.yml b/workflows/self-audit.yml index ad75f7a..7ba5283 100644 --- a/workflows/self-audit.yml +++ b/workflows/self-audit.yml @@ -98,6 +98,7 @@ steps: 3. Reminder: one change at a time, measure after each ### What NOT to Do - - Don't fix everything at once + - Don't fix anything from this report directly — use the suggested devkit command + - Don't fix everything at once — one hypothesis at a time - Don't start with easy low-impact items - - Don't refactor untested code (add tests first) + - Don't add features — this is about quality, not functionality From 6339a9466b75c4b866ce797702398a1b1ad44e9b Mon Sep 17 00:00:00 2001 From: Tym Rabchuk Date: Sun, 5 Apr 2026 19:34:03 -0400 Subject: [PATCH 3/4] Fix tri-review findings: add {{input}}, fix shell bugs, portability MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Workflow detect step now includes {{input}} so user args aren't discarded - Fix shell glob: [ -f */go.mod ] → find with -maxdepth 2 -print -quit - Fix HAS_TESTS: now returns yes/no like all other variables - Replace platform-specific stat with portable git log --format='%ai' - README: "one-at-a-time" → "report only" (command doesn't fix anything) --- README.md | 2 +- commands/self-audit.md | 6 +++--- workflows/self-audit.yml | 2 ++ 3 files changed, 6 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 133f768..4c73eab 100644 --- a/README.md +++ b/README.md @@ -133,7 +133,7 @@ Automated propose → measure → keep/discard → repeat cycles. | `/self:lint` | Iteratively fix lint/type errors until zero remain | | `/self:perf` | Hypothesis-driven performance investigation — evidence, hypotheses, one-at-a-time testing | | `/self:migrate` | Incremental migration (JS→TS, class→hooks, etc.) with test gate | -| `/self:audit` | Karpathy-style codebase audit — measure everything, rank hypotheses by evidence, one-at-a-time | +| `/self:audit` | Karpathy-style codebase audit — measure everything, rank hypotheses by evidence, report only | ### Multi-Agent Commands (Claude + optional Codex/Gemini) diff --git a/commands/self-audit.md b/commands/self-audit.md index 8243656..2650c90 100644 --- a/commands/self-audit.md +++ b/commands/self-audit.md @@ -16,12 +16,12 @@ This command does NOT fix anything. It measures, analyzes, and presents a ranked ```bash # Detect what's in this repo -HAS_GO=$([ -f go.mod ] || [ -f */go.mod ] && echo yes || echo no) +HAS_GO=$([ -f go.mod ] || find . -maxdepth 2 -name go.mod -print -quit | grep -q . && echo yes || echo no) HAS_TS=$([ -f tsconfig.json ] && echo yes || echo no) HAS_JS=$([ -f package.json ] && echo yes || echo no) HAS_PYTHON=$([ -f pyproject.toml ] || [ -f requirements.txt ] || [ -f setup.py ] && echo yes || echo no) HAS_RUST=$([ -f Cargo.toml ] && echo yes || echo no) -HAS_TESTS=$(find . -name '*_test.go' -o -name '*.test.ts' -o -name '*.test.js' -o -name 'test_*.py' -o -name '*_test.rs' 2>/dev/null | head -1) +HAS_TESTS=$([ -n "$(find . -name '*_test.go' -o -name '*.test.ts' -o -name '*.test.js' -o -name 'test_*.py' -o -name '*_test.rs' 2>/dev/null | head -1)" ] && echo yes || echo no) HAS_CI=$([ -d .github/workflows ] || [ -f .gitlab-ci.yml ] || [ -f Jenkinsfile ] && echo yes || echo no) HAS_DOCKER=$([ -f Dockerfile ] || [ -f docker-compose.yml ] && echo yes || echo no) ``` @@ -95,7 +95,7 @@ grep -rn 'TODO\|FIXME\|HACK\|XXX' --include='*.go' --include='*.ts' --include='* ```bash # README exists and is recent? -stat -f '%Sm' README.md 2>/dev/null || stat -c '%y' README.md 2>/dev/null +git log -1 --format='%ai' -- README.md 2>/dev/null # API docs? # Changelog? # Are counts/claims in docs accurate? (run validate-counts if available) diff --git a/workflows/self-audit.yml b/workflows/self-audit.yml index 7ba5283..d0ce00e 100644 --- a/workflows/self-audit.yml +++ b/workflows/self-audit.yml @@ -10,6 +10,8 @@ steps: model: fast prompt: | Detect the tech stack in this repository. + User context: {{input}} + Check for: Go (go.mod), TypeScript/JS (package.json, tsconfig.json), Python (pyproject.toml, requirements.txt), Rust (Cargo.toml), CI (.github/workflows), Docker (Dockerfile), tests. From 740285abcf22f858abc8d1a423a3422c0846c823 Mon Sep 17 00:00:00 2001 From: Tym Rabchuk Date: Sun, 5 Apr 2026 19:36:14 -0400 Subject: [PATCH 4/4] Replace hallucinated Karpathy quote with actual autoresearch reference The "quote" was fabricated. The actual inspiration is karpathy/autoresearch: one file, one metric, keep or discard, loop ~100 times overnight. We apply the same pattern to codebase quality instead of ML training. Now references the real repo with a link and describes the actual methodology. --- README.md | 2 +- ROADMAP.md | 2 +- commands/self-audit.md | 6 +++--- workflows/self-audit.yml | 2 +- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 4c73eab..553cfce 100644 --- a/README.md +++ b/README.md @@ -133,7 +133,7 @@ Automated propose → measure → keep/discard → repeat cycles. | `/self:lint` | Iteratively fix lint/type errors until zero remain | | `/self:perf` | Hypothesis-driven performance investigation — evidence, hypotheses, one-at-a-time testing | | `/self:migrate` | Incremental migration (JS→TS, class→hooks, etc.) with test gate | -| `/self:audit` | Karpathy-style codebase audit — measure everything, rank hypotheses by evidence, report only | +| `/self:audit` | Codebase audit inspired by karpathy/autoresearch — measure everything, rank hypotheses, report only | ### Multi-Agent Commands (Claude + optional Codex/Gemini) diff --git a/ROADMAP.md b/ROADMAP.md index 201e79f..37ab273 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -2,7 +2,7 @@ ## Implemented -- **22 slash commands** — Lifecycle workflows, self-improvement loops, multi-agent dispatch, project health audit, post-PR monitoring, AST repo mapping, Karpathy-style self-audit +- **22 slash commands** — Lifecycle workflows, self-improvement loops, multi-agent dispatch, project health audit, post-PR monitoring, AST repo mapping, autoresearch-inspired self-audit - **16 context-activated skills** — 7 auto-trigger workflows (test-gen, doc-gen, changelog, onboard, research, deep-research, scrape) + 6 coding principles (executing, clean-code, DRY, YAGNI, dont-reinvent, stuck) + 2 tools (gcli, creating-workflows) + 1 iteration memory (scratchpad) - **6 agents** — Scoped tool access, worktree isolation, model assignment - **10 hooks** — Safety (destructive command blocking, edit-time security patterns, PR gate), observability (audit trail, slop detection, post-validation, subagent verification, language-aware code review), optimization (RTK token compression) diff --git a/commands/self-audit.md b/commands/self-audit.md index 2650c90..9cb447f 100644 --- a/commands/self-audit.md +++ b/commands/self-audit.md @@ -1,14 +1,14 @@ --- -description: Karpathy-style self-audit — measure the codebase, rank improvement hypotheses by evidence, present actionable plan. Don't guess, measure. +description: Automated self-audit — measure the codebase, rank improvement hypotheses by evidence, present actionable plan. Inspired by karpathy/autoresearch. --- # Self-Audit -Systematic codebase audit using Karpathy's research methodology: measure first, form hypotheses from evidence, rank by impact. No guessing, no speculative improvements. +Systematic codebase audit inspired by [karpathy/autoresearch](https://github.com/karpathy/autoresearch): one file, one metric, keep or discard, loop. Applied to codebase quality instead of ML training. ## Philosophy -> Most of the value comes from understanding the system deeply. Don't jump to building stuff without knowing what's actually broken. — inspired by Karpathy's research methodology +autoresearch gives an AI agent a training setup and lets it experiment autonomously — modify code, measure the metric, keep or revert, repeat ~100 times overnight. We apply the same pattern to codebase quality: measure everything with a single pass, form ranked hypotheses, then test one at a time with a clear keep/discard metric. This command does NOT fix anything. It measures, analyzes, and presents a ranked hypothesis list. You decide which to test. One at a time. diff --git a/workflows/self-audit.yml b/workflows/self-audit.yml index d0ce00e..ce0ef0f 100644 --- a/workflows/self-audit.yml +++ b/workflows/self-audit.yml @@ -1,5 +1,5 @@ name: Self-Audit -description: Karpathy-style codebase audit — measure everything, rank hypotheses by evidence, present actionable plan +description: Codebase audit inspired by karpathy/autoresearch — measure everything, rank hypotheses by evidence, present actionable plan budget: limit: 200000