Skip to content

feat(skill-creator): six corrections a real update cycle exposed (1.22.0) - #195

Merged
daymade merged 1 commit into
mainfrom
feat/skill-creator-registry-and-description-budget
Aug 4, 2026
Merged

feat(skill-creator): six corrections a real update cycle exposed (1.22.0)#195
daymade merged 1 commit into
mainfrom
feat/skill-creator-registry-and-description-budget

Conversation

@daymade

@daymade daymade commented Jul 22, 2026

Copy link
Copy Markdown
Owner

Six corrections to skill-creator, distilled from using it to ship consecutive updates to an existing skill. Every item is a place the guidance let a predictable mistake through — none are hypothetical improvements.

1. Description budget (Write the SKILL.md)

The 1024-character ceiling was never stated anywhere, so an update adding trigger phrases for newly-covered scope blew past it and needed two rounds of compression to land. That ceiling is in direct tension with the existing "make it pushy" advice, and the tension deserves naming rather than discovery while fighting a validator.

The sharper point: a mature description usually sits close to the limit, which makes adding a trigger zero-sum — you are deleting an existing one to pay for it. That is a real narrowing of when the skill fires, so it must be a conscious, recorded trade. Also ranks what to sacrifice first: prose qualifiers are re-derivable from the body, distinct trigger phrases are not.

2. Registry minimal-diff (Step 8)

The marketplace manifest is the single file every skill shares, which makes it the likeliest place for two concurrent editors to collide — and therefore the worst place to ship an unrelated formatting change. Scripted bumps silently normalize trailing newline, indent width, key order, and unicode escaping. The step now requires a round-trip check that git diff shows only the fields you meant to change.

(A scripted bump once added a trailing newline to a manifest that had never had one. This PR's own bump was round-tripped under the new rule — its manifest diff is exactly one line.)

3. Activation check (Edit Skills at Source Location)

"Sync the installed copy" is frequently not work at all, and the workflow never said how to tell. A symlinked skills directory, or a marketplace with source: directory, reads the working tree — the edit is already live. Only cached/copied installs need the official update command.

Verification must be by content, not by a recorded version string: one session read a plugin record naming a cache directory that had the new version number in its path and nearly reported the update as live. That directory had never been created; the real runtime path was a symlink to the source.

4. Production-as-eval gains a second signal source (Capture Intent)

When a skill's output is something that keeps running — a guard, a monitor, a scheduled job, a hook — its own telemetry is eval data, and the first false alarm is the highest-signal record in it. A user correction requires a user to notice and bother; a deployed mechanism reports on itself unprompted, often within a day. A false positive proves a rule you wrote is wrong in a way re-reading never would, so treat the first one as a scheduled eval result rather than an annoyance — the likely finding is that the instruction was too absolute.

5. Concurrency covers branch switching, not just a moved HEAD

The existing rule said to re-check HEAD's SHA before committing. That is not the worst thing a parallel session does. A checkout is worktree-wide, so while one session edits on a feature branch, another running checkout main + pull silently relocates the whole worktree — and the next commit lands on main, violating the repo's never-commit-to-local-main rule, while the feature branch still points at the old base. Exactly that happened during this work.

The pre-commit check gains git branch --show-current; git reflog is the authoritative post-mortem (it records every checkout: moving from X to Y in order); and the repair is deliberately ref-onlycheckout -B then branch -f — because both move refs without touching the working tree, so unlike reset --hard they cannot destroy a parallel session's uncommitted work.

6. PR staleness, and the proof step everyone skips

The property that makes a shared manifest a collision hotspot also makes any open PR touching it go stale. CONFLICTING is the expected state, not a surprise — this PR itself sat through 69 commits of main. Rebase (not merge) where the repo squash-merges.

The conflicts are almost always additive: two authors appended an entry to the same section, so keep both, and --ours/--theirs would silently drop a colleague's line. What is missing from most workflows is the verification after resolving: prove the only difference from the base is your own entry. The section now ships a copy-paste check that diffs every version in the base manifest against yours. Finally --force-with-lease rather than bare --force, since its entire value is failing in the one case that matters: somebody else pushed to your branch.


Rebase note (items 5 and 6, dogfooded)

The original branch was 69 commits behind. A sibling session had uncommitted work in both files this PR touches (marketplace.json, CHANGELOG.md), so git checkout to the PR branch would have been refused — and forcing it would have dragged their WIP along.

Rebuilt via the plumbing route instead, which never touches the working tree: extract the current origin/main files → apply this PR's changes in a scratch dir → hash-object → temporary GIT_INDEX_FILEwrite-treecommit-treeupdate-ref. The sibling session's 8 uncommitted files were byte-identical before and after.

Version target moved 1.15.01.22.0 (main advanced to 1.21.0 meanwhile). Verified none of the six items had been independently added upstream in the interim: 1024, force-with-lease, branch --show-current, CONFLICTING, telemetry each appear 0 times in main's current SKILL.md.

Verification

  • quick_validate — passes
  • audit_skill_regression compare → classify → verify — passes; two candidates (the production-as-eval bullet and the HEAD-check rule, both extended in place) classified preserved_or_moved with the original clauses intact and locatable in the current file
  • security_scan — clean; plus a manual read-through of every added line for private paths / project names
  • Manifest delta proven by this PR's own new check: {'daymade-skill': ('1.21.0', '1.22.0')} — nothing else, 58 plugin entries before and after, metadata.version untouched
  • CI green on all four jobs

🤖 Generated with Claude Code

https://claude.ai/code/session_01F6FQSPAyXY9WZJYUToxXXY


Re-rebase (2026-08-05)

Rebased onto current main again — cleanly, no conflicts. The version target is unchanged: main is still at daymade-skill 1.21.0, so this still lands 1.22.0. Re-verified that none of the six items had been added upstream meanwhile: 1024, branch --show-current, force-with-lease, zero-sum, first false alarm, reflog each appear 0 times in main's current SKILL.md.

Gates re-run against the new base: quick_validate passes; regression audit compare → classify → verify passes (2 candidates, both the production-as-eval bullet and the HEAD-check rule extended in place, classified preserved_or_moved with the original clauses located verbatim); security_scan clean plus a manual line-by-line read of every added line. A trial merge with #210 is conflict-free in both orders and yields the same tree.

An independent fresh-context review of the rebase confirmed the patch payload is byte-identical to the pre-rebase branch (md5 match on both sides), the manifest changes exactly one line, and nothing from main was lost.

@daymade
daymade force-pushed the feat/skill-creator-registry-and-description-budget branch from 215e274 to 693db0d Compare July 22, 2026 16:01
@daymade
daymade force-pushed the feat/skill-creator-registry-and-description-budget branch from 8bd958a to ee929aa Compare August 4, 2026 12:32
@daymade daymade changed the title feat(skill-creator): close four gaps found by using it on two consecutive updates feat(skill-creator): six corrections a real update cycle exposed (1.22.0) Aug 4, 2026
…2.0)

Distilled from using this skill to ship consecutive updates to an existing
skill. Each item is a place the guidance let a predictable mistake through —
none are hypothetical improvements.

1. Description budget (Write the SKILL.md)
   The 1024-character ceiling was never stated anywhere, so an update adding
   trigger phrases for newly-covered scope blew past it and needed two rounds of
   compression to land. That ceiling is in direct tension with the existing
   "make it pushy" advice, and the tension deserves naming rather than discovery
   while fighting a validator. The sharper point: a mature description usually
   sits close to the limit, which makes adding a trigger zero-sum — you are
   deleting an existing one to pay for it. That is a real narrowing of when the
   skill fires, so it must be a conscious, recorded trade. Also ranks what to
   sacrifice: prose qualifiers are re-derivable from the body, distinct trigger
   phrases are not.

2. Registry minimal-diff (Step 8)
   The marketplace manifest is the single file every skill shares, so it is the
   likeliest place for two concurrent editors to collide — and the worst place
   to ship an unrelated formatting change. Scripted bumps silently normalize
   trailing newline, indent width, key order, unicode escaping. The step now
   requires a round-trip check that git diff shows only the intended fields.
   (A scripted bump once added a trailing newline to a manifest that never had
   one. This commit's own bump was round-tripped under the new rule: one line.)

3. Activation check (Edit Skills at Source Location)
   "Sync the installed copy" is frequently not work at all, and the workflow
   never said how to tell. A symlinked skills directory, or a marketplace with
   source: directory, reads the working tree — the edit is already live. Only
   cached/copied installs need the official update command. Verification must be
   by content, not by a recorded version string: one session read a plugin
   record naming a cache directory that had the new version in its path and
   nearly reported the update as live. That directory had never been created;
   the real runtime path was a symlink to the source.

4. Production-as-eval gains a second signal source (Capture Intent)
   When a skill's output is something that keeps running — a guard, a monitor, a
   scheduled job, a hook — its own telemetry is eval data, and the first false
   alarm is the highest-signal record in it. A user correction requires a user to
   notice and bother; a deployed mechanism reports on itself unprompted, often
   within a day. A false positive proves a rule you wrote is wrong in a way
   re-reading never would, so treat the first one as a scheduled eval result
   rather than an annoyance — the likely finding is that the instruction was too
   absolute.

5. Concurrency covers branch switching, not just a moved HEAD
   The existing rule said to re-check HEAD's SHA before committing. That is not
   the worst thing a parallel session does. A checkout is worktree-wide, so while
   one session edits on a feature branch, another running `checkout main` + pull
   silently relocates the whole worktree — and the next commit lands on main,
   violating the repo's never-commit-to-local-main rule, while the feature branch
   still points at the old base. Exactly that happened during this work. The
   pre-commit check gains `git branch --show-current`; `git reflog` is the
   authoritative post-mortem (it records every "checkout: moving from X to Y" in
   order); and the repair is deliberately ref-only — `checkout -B` then
   `branch -f` — because both move refs without touching the working tree, so
   unlike `reset --hard` they cannot destroy a parallel session's uncommitted
   work.

6. PR staleness, and the proof step everyone skips
   The property that makes a shared manifest a collision hotspot also makes any
   open PR touching it go stale. CONFLICTING is the expected state, not a
   surprise — this PR itself sat through 69 commits of main. Rebase (not merge)
   where the repo squash-merges. The conflicts are almost always additive: two
   authors appended an entry to the same section, so keep both, and
   --ours/--theirs would silently drop a colleague's line. What is missing from
   most workflows is the verification AFTER resolving: prove the only difference
   from the base is your own entry. The section ships a copy-paste check that
   diffs every version in the base manifest against yours. Finally
   --force-with-lease rather than bare --force, since its entire value is failing
   in the one case that matters: somebody else pushed to your branch.

Rebased onto current main (69 commits ahead of the original base) and rebuilt
without touching the working tree, because a sibling session had uncommitted
work in the two files this touches — the plumbing route (temporary index +
commit-tree) is the practice item 5 is about.

Verified: quick_validate passes; audit_skill_regression compare/classify/verify
passes with two candidates (production-as-eval and the HEAD-check rule, both
extended in place) classified preserved_or_moved with the original clauses
intact and locatable; security_scan clean; manual read-through of every added
line for private paths/names; manifest bump round-tripped to a one-line diff.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F6FQSPAyXY9WZJYUToxXXY
@daymade
daymade force-pushed the feat/skill-creator-registry-and-description-budget branch from ee929aa to d806412 Compare August 4, 2026 16:55
@daymade
daymade merged commit 7537d3f into main Aug 4, 2026
4 checks passed
@daymade
daymade deleted the feat/skill-creator-registry-and-description-budget branch August 4, 2026 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant