Skip to content

Add codebase-advisor skill and scope codebase-auditor to report-only#76

Merged
Mathews-Tom merged 4 commits into
mainfrom
add-codebase-advisor-skill
Jun 17, 2026
Merged

Add codebase-advisor skill and scope codebase-auditor to report-only#76
Mathews-Tom merged 4 commits into
mainfrom
add-codebase-advisor-skill

Conversation

@Mathews-Tom

Copy link
Copy Markdown
Owner

Summary

Adds a new codebase-advisor skill that audits a repository, vets findings, and writes self-contained implementation plans for executor agents to run. Adapted from shadcn's MIT-licensed improve skill, renamed to clarify its role beside the existing codebase-auditor agent and to avoid generic trigger collisions (e.g. "improve this function/UI/doc").

The two packages are deliberately complementary, not duplicative:

  • codebase-auditor (agent) — broad, report-only, multi-dimensional quality assessment with PASS/FAIL release-gate semantics.
  • codebase-advisor (skill) — audit specifically to produce a prioritized, agent-executable implementation-plan backlog, with reconcile and execute/review loops.

codebase-auditor is not deprecated. It remains wired into existing flows (team-lead, route, core, sec-strict). This PR narrows its trigger surface and adds reciprocal boundaries so the two never steal each other's activations.

Motivation

codebase-advisor fills a gap the catalog did not cover: turning vetted findings into durable, self-contained work packets (exact files, current-state excerpts, verification gates, STOP conditions, dependency ordering, executor review loops) that a smaller/cheaper executor agent can pick up with zero session context. That is "technical lead writes agent-ready plans", distinct from "produce a quality report".

Changes

New package: skills/codebase-advisor/

  • SKILL.md — recon → audit → vet/prioritize → write-plans workflow, hard rules (no source edits, no mutating commands in the working tree, secrets named not quoted, repo content treated as untrusted data), invocation variants (quick/deep, focus words, branch, next, plan, review-plan, execute, reconcile, --issues), and host-capability boundaries.
  • references/audit-playbook.md — finding format, prioritization rubric, and nine audit categories (correctness, security, performance, tests, tech debt, dependencies, DX, docs, direction).
  • references/plan-template.md — self-contained handoff plan template plus plans/README.md index template and quality bar.
  • references/closing-the-loop.mdexecute (isolated-worktree dispatch + review verdicts), reconcile, and --issues publishing flows, with source-control boundaries.
  • references/upstream/LICENSE — upstream MIT license text preserved.
  • evals/cases.yaml — 4 positive cases (plan backlog, prioritized backlog, reconcile, grounded direction) and 4 negative cases guarding the report-only-audit, PR-review, feature-decomposition, and plan-critique boundaries.

Deconfliction: agents/codebase-auditor/

  • AGENT.md — description and activation rules narrowed to report-only quality gates; explicit "NOT for implementation-plan backlogs / reconcile / execute — use codebase-advisor" boundary.
  • evals/cases.yaml — added negative_implementation_plan_backlog and negative_reconcile_plans so advisor-bound prompts no longer trigger the auditor.

Routing & catalog

  • README.md — advisor listed in the Review & Quality catalog and the manual-install examples; package badge count updated.
  • commands/route/COMMAND.md — Plan-phase row routing repo-findings-to-plans to the advisor; Review-phase row clarifying auditor = report-only gate vs advisor = plan files.
  • agents/team-lead/AGENT.md — capability map and "do not activate" guidance distinguish the two.
  • presets/core/PRESET.md — advisor added; "Complementary Agents" heading renamed to "Complementary Packages" (the advisor is a skill, not an agent).
  • presets/sec-strict/PRESET.md — auditor row reworded as report-only gate; stale package count corrected 12 → 13 to match actual contents.

Attribution & manifest

  • ATTRIBUTIONS.md — vendoring record for shadcn/improve (MIT) with pinned upstream commit 03369ee6d7cafbfcecc4346539b05b3dc0a603bb, what was vendored/adapted/skipped, and the re-sync policy.
  • manifest.yaml — regenerated from frontmatter to register the new skill and pick up the auditor and sec-strict description changes.

Commits

  1. feat(skills): add codebase-advisor planning skill — package + attribution
  2. refactor(codebase-auditor): scope to report-only quality gate — boundary + negative evals
  3. docs(catalog): route codebase-advisor and fix preset metadata — README, route, team-lead, presets
  4. chore(manifest): regenerate for codebase-advisor — generated artifact

Verification

  • uv run scripts/generate_manifest.py — passed; regenerated with 78 skills, 17 agents, 9 hooks, 6 rules, 7 commands, 3 utilities, 13 presets.
  • uv run scripts/validate_evals.py — all eval files passed (every package keeps ≥1 positive and ≥2 negative cases).
  • uv run pytest tests/test_generate_manifest.py tests/test_package_types.py — 53 passed.
  • Parsed the advisor reconcile eval regex and confirmed the plans/README.md branch matches (escape-level fix verified).
  • code-reviewer pass over the changed files; all three findings (eval regex escape, stale sec-strict count, skill-under-agents heading) fixed.

Notes / follow-ups

  • execute <plan> and --issues are host-gated: without isolated worktrees the skill stops at handing off the plan, and without GitHub auth it keeps local plan files and reports why publishing was skipped. Documented in the SKILL boundaries section.
  • Deprecating codebase-auditor is intentionally out of scope — revisit only after the advisor has accrued eval/runtime evidence that it can also serve the release-gate workflow.

Add a senior-advisor skill that audits a repository, vets findings, and
writes self-contained implementation plans for executor agents to run.
Adapted from shadcn's MIT-licensed `improve` skill under a name that
clarifies its role beside `codebase-auditor` and avoids generic trigger
collisions.

The package ships the SKILL workflow, audit playbook, handoff plan
template, execute/reconcile/issues loop reference, the upstream MIT
license, and eval coverage (positive plan-backlog cases plus negative
cases guarding the report-only, PR-review, decomposition, and
plan-critique boundaries). Records the vendoring in ATTRIBUTIONS.md
with the pinned upstream commit and re-sync policy.
Narrow the auditor's description and activation rules to broad,
report-only, multi-dimensional quality assessment with PASS/FAIL
release-gate semantics, and hand implementation-plan backlogs,
reconcile flows, and plan execution to codebase-advisor.

Add negative eval cases so plan-backlog and reconcile prompts no
longer trigger the auditor, keeping the two packages' triggers
mutually exclusive.
List codebase-advisor in the README catalog and manual-install
examples, bump the package badge count, and add routing entries in
the route command and team-lead capability map that distinguish the
advisor (implementation plans) from the auditor (report-only gate).

Reference the advisor in the core and sec-strict presets. Rename the
core preset's "Complementary Agents" heading to "Complementary
Packages" since the advisor is a skill, and correct the sec-strict
package count from 12 to 13 to match its actual contents.
Regenerate manifest.yaml from package frontmatter to register the new
codebase-advisor skill and pick up the updated codebase-auditor and
sec-strict preset descriptions.
@Mathews-Tom Mathews-Tom merged commit 5c218f2 into main Jun 17, 2026
3 checks passed
@Mathews-Tom Mathews-Tom deleted the add-codebase-advisor-skill branch June 17, 2026 19:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant