Skip to content

chore(skills): audit skill suite against doctor checks and forge conventions - #44

Merged
mgratzer merged 10 commits into
mainfrom
chore/skill-suite-audit
Jul 31, 2026
Merged

chore(skills): audit skill suite against doctor checks and forge conventions#44
mgratzer merged 10 commits into
mainfrom
chore/skill-suite-audit

Conversation

@mgratzer

@mgratzer mgratzer commented Jul 27, 2026

Copy link
Copy Markdown
Owner

Why

The forge skills had never had a systematic health check, and recent refactors — extracting forge-ship from forge-implement, moving to the _shared layer, the #42 token reduction — left drift behind: cross-skill files stranded in implement's private directories, CONTEXT.md describing removed designs, and a consistency table that predates ship's existence. On top of the audit, this PR applies the current Claude 5 context-engineering guidance (judgment over defensive rules, single-statement guidance, progressive disclosure) to rightsize every skill to the repo's own ~35-instruction budget.

The audit ran in three layers: a mechanical checker (frontmatter validity, link integrity, _shared consumer graph), three fresh-context audit agents (instruction budget/structure, shared-layer integrity, vocabulary/descriptions), and verification of every raw finding before fixing. Each substantive commit then got its own fresh-context review pass; confirmed findings were fixed in follow-up commits.

Changes

Audit fixes

  • Shared layer — moved phase-execution.md, pattern-audit.md, and roles/forge-scout.md into skills/_shared/; dropped the Refs # trailer stated nowhere else
  • CONTEXT.md — realigned Reflection, Composite skill, Issue/AFK-HITL persistence, quality gates; added Vertical phase and Tiny diff; removed two single-consumer terms
  • forge-reflect — removed context: fork (forking the whole skill would strand the interactive triage step); triage renamed and reordered before the report
  • forge-implement — documents --unattended; action-verb step titles; placeholder casing; description disambiguated from ship
  • forge-create-issue / forge-shapeallowed-tools completed to cover their own steps; AFK/HITL mode recorded at creation
  • issue-operations — label discovery/update and epic sub-issue sections absorb inlined provider conditionals; sub-issue linking works on every gh version
  • Docs — consistency table, architecture tree, validation lists, README skill count, instruction-budget contradiction

Review-rule change

  • Tiny-diff inline review is now conditional on authorship: inline only when the reviewing session didn't author the changes. Ship's session always authors the diff, so its review always delegates — restoring the fresh-context guarantee context: fork used to provide, on every runtime.

Rightsizing (Claude 5 context rules)

  • ship executes forge-implement's process by reference instead of paraphrasing it as 1a–1g (the duplicate had already drifted); own body ~50 → ~20 instructions
  • implement references _shared/phase-execution.md, pattern-audit.md, and deep-modules.md instead of inlining copies; defensive flag-lists become judgment calls with an unattended fallback (~52 → ~35)
  • setup-project collapses the three-mode table into one assessment step; interrogation lists compress to intent; the scoring rubric and tier model stay as the skill's core opinion (~55 → ~33)
  • shape / create-issue / reflect — guidelines restating process steps or shared modules deleted
  • coding-guidelines now notes composite budgets count per invocation (ship + implement load together)

Test plan

  • Link integrity: 55 relative links, 0 real breaks (2 known false positives inside a fenced example)
  • _shared consumer graph: no orphans, no cross-skill links into skill-private directories
  • Architecture tree matches find skills -type f
  • Fresh-context review pass per substantive commit; all confirmed findings fixed or explicitly rejected with reasons in session notes

https://claude.ai/code/session_0187yZ3Nnp45s9BVYxnpJ7Ca

mgratzer added 5 commits July 27, 2026 11:02
phase-execution.md, pattern-audit.md, and forge-scout.md were consumed by
forge-ship (and forge-scout by forge-implement) while living in
forge-implement's private directories — violating the self-containment
rule. Also drops the Refs # trailer convention stated nowhere else and
removes stale step/dimension references from pattern-audit.

Claude-Session: https://claude.ai/code/session_0187yZ3Nnp45s9BVYxnpJ7Ca
Reflection described the removed four-reviewer design; Composite skill
claimed pure orchestration; Issue claimed an unpersisted AFK/HITL field
and a priority no skill uses; quality gates omitted coverage. Adds
Vertical phase and Tiny diff (both cross-skill and load-bearing),
removes Smart/Dumb zone and Three-tier context per the file's own
single-consumer rule — they live in architecture.md and setup-project.

Claude-Session: https://claude.ai/code/session_0187yZ3Nnp45s9BVYxnpJ7Ca
- reflect: drop context: fork (forking the whole skill strands the
  interactive triage step; fresh context already comes from the
  (delegate) review step), rename Triage Deferred Items to Triage
  Findings — items only become deferred as the outcome of triage
- implement: document --unattended, fold Sub-Issue Handling into
  Step 1, action-verb step titles, placeholder casing, disambiguate
  from ship in the description
- ship: bash example for push/PR, coverage fallback, Format and
  Coverage rows in the output gates, scout delegation fallback
- create-issue/shape: complete allowed-tools their own steps require;
  create-issue gains trailing-context support and records the
  AFK/HITL mode in the Issue body
- address-pr-feedback: Follow-up renamed to Deferred item (canonical)
- issue-operations: new label discovery/update and epic sub-issue
  sections absorb the provider conditionals skills had inlined;
  sub-issue linking uses task lists with --parent as opt-in (the
  flag is not available on all gh versions)

Claude-Session: https://claude.ai/code/session_0187yZ3Nnp45s9BVYxnpJ7Ca
The Cross-Skill Consistency table's Referenced In column predated
forge-ship's extraction from implement and missed it in ~8 rows; the
architecture tree omitted the three most-consumed _shared files and
listed directories that no longer exist; testing/development check
lists enumerated only 5 of 7 skills; README counted six skills; the
architecture Operating Constraints table said ~150-200 instructions
per skill where coding-guidelines says under ~35.

Claude-Session: https://claude.ai/code/session_0187yZ3Nnp45s9BVYxnpJ7Ca
The review caught contradictions introduced while syncing the
consistency table (trailing-context and vertical-slices rows
over-claimed; README/AGENTS/architecture still scoped trailing
context to structured-input skills), plus pre-existing defects the
renames exposed: reflect reported before triaging, ship's Step 2
delegate lacked its Inputs/Expected blocks, and CONTEXT's Deferred
item definition didn't cover review-feedback items. Also documents
context: fork in the frontmatter conventions and compresses
three-sentence descriptions to two.

Claude-Session: https://claude.ai/code/session_0187yZ3Nnp45s9BVYxnpJ7Ca
Copilot AI review requested due to automatic review settings July 27, 2026 12:34

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Audits and realigns the Forge skill suite against “doctor” checks and Forge conventions, reducing cross-skill drift by centralizing shared references/roles under skills/_shared/ and updating skill/docs vocabulary and workflows accordingly.

Changes:

  • Consolidates cross-skill reference material and roles in skills/_shared/ and updates skill links/usages to match.
  • Normalizes skill interfaces and flows (trailing context, delegation inputs/outputs, triage/report ordering, naming consistency).
  • Refreshes project documentation (architecture, guidelines, development/testing checklists, README, shared vocabulary in CONTEXT.md).

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
skills/forge-ship/SKILL.md Updates ship flow (shared references, delegation metadata, quality gates, PR creation guidance).
skills/forge-shape/SKILL.md Expands allowed tools and clarifies wording around facts/questions and optional delegation.
skills/forge-setup-project/SKILL.md Clarifies $ARGUMENTS usage and makes step titles more action-oriented.
skills/forge-setup-project/references/output-format.md Updates recommendations/next-steps wording and points to shape as an alternative.
skills/forge-reflect/SKILL.md Reorders triage/reporting steps and tightens wording around quality gates + findings.
skills/forge-implement/SKILL.md Aligns inputs, unattended mode, shared scout role usage, and action-verb step titles.
skills/forge-create-issue/SKILL.md Completes allowed-tools, adds trailing-context support, and aligns on shared issue-ops guidance.
skills/forge-address-pr-feedback/SKILL.md Adds AskUserQuestion to allowed tools and renames follow-ups to “Deferred” items.
skills/_shared/roles/forge-scout.md Updates scout framing/terminology to match Issue/plan language.
skills/_shared/plan-folder-spec.md Aligns plan-folder body structure terminology with current Issue conventions.
skills/_shared/phase-execution.md Fixes internal shared-linking and removes the unsupported Refs #... commit trailer guidance.
skills/_shared/pattern-audit.md Generalizes audit timing language (implementation/reflection) to match updated skill structure.
skills/_shared/issue-operations.md Adds label discovery/update + epic/sub-issue guidance and provider fallbacks.
README.md Updates skill count and broadens trailing-context statement to all skills.
docs/testing.md Updates consistency-check grep patterns to include the full skill suite and trailing context for all skills.
docs/development.md Simplifies validation grep examples to reflect trailing context support across all skills.
docs/coding-guidelines.md Updates conventions (including context: fork guidance) and shared-role references.
docs/architecture.md Refreshes tree, operating constraints wording, and delegation/composition design decisions.
CONTEXT.md Realigns shared vocabulary (Reflection/Tiny diff/Vertical phase/etc.) and removes obsolete terms.
AGENTS.md Updates “one invocation convention” principle to apply to every skill.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread skills/forge-ship/SKILL.md Outdated
Comment thread docs/coding-guidelines.md Outdated
Comment thread skills/_shared/issue-operations.md Outdated
Removing context: fork from forge-reflect meant inline tiny-diff
reviews run in the session that may have just written the code —
the fork previously gave even those fresh eyes. Rather than always
delegating (which would erase the lean-review design), the inline
path is now allowed only when the reviewing session did not author
the changes. For forge-ship the session always authors the diff, so
its review now always delegates.

Claude-Session: https://claude.ai/code/session_0187yZ3Nnp45s9BVYxnpJ7Ca
@mgratzer
mgratzer force-pushed the chore/skill-suite-audit branch from b48a048 to 0f26ad7 Compare July 27, 2026 14:19
mgratzer added 4 commits July 27, 2026 17:13
Applies the current context-engineering guidance for Claude 5 models:
prefer judgment over defensive rules, never state the same guidance in
two places, and use progressive disclosure for detail.

- ship: Step 1 now executes forge-implement's process by reference
  instead of paraphrasing it as 1a-1g — the duplication had already
  drifted once; ship's own body drops to ~20 instructions (note: a
  ship invocation still loads implement's ~35, so the budget is
  honest per file, not per invocation — recorded in coding-guidelines)
- implement: references _shared/phase-execution.md, pattern-audit.md,
  and deep-modules.md instead of inlining copies; defensive flag-lists
  become judgment calls with an unattended fallback
- setup-project: three-way mode table collapses into one assessment
  step (file state determines the work); interrogation lists compress
  to intent; scoring rubric and tier model stay as the skill's opinion
- shape/create-issue/reflect: guidelines restating process steps or
  shared modules deleted; the authoritative statement survives in one
  place
- post-review fixes: implement's clarification step is unattended-
  aware so ship --unattended cannot block; pr-workflow.md content spec
  restored; output-format headings match the collapsed flow; the
  lettered-sub-step rule that invited re-inlining removed; ship's
  composition described consistently everywhere

Claude-Session: https://claude.ai/code/session_0187yZ3Nnp45s9BVYxnpJ7Ca
@mgratzer
mgratzer merged commit d5e169e into main Jul 31, 2026
@mgratzer
mgratzer deleted the chore/skill-suite-audit branch July 31, 2026 06:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants