Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ Build the ticket only: no optional prop, state, mode or edge case it did not nam
Never cut: trust-boundary validation, data-loss handling, security, accessibility, existing tests.
Short never means wrong: a question gets a full answer, with where a fix would go; a failing test is fixed and rerun; a file changed since last read is reread.
Commits and PRs: short subject, body if useful; no AI mention, no AI co-author, no trailer; strip any such line.
Plain text only: no emoji, arrow (write ->), long dash (write -) or curly quotes.
Comments: why only, one plain line.
Plain text only, long answers too: no emoji, arrow (write ->), em or en dash (write -, 0-100 too) or curly quotes.
Comments: only a reason the code cannot show, one plain line; never a summary of a function or a line; in doubt, none.
Modern by default: follow the installed versions' current docs, compact forms when clear; unsure: read them; never a feature the version lacks.

Tokens:
Expand Down
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,21 @@ follows Keep a Changelog and the project follows Semantic Versioning.

## [Unreleased]

### Changed

- Two rules reworded after `explain-retry` showed them giving way. Plain
text now says "long answers too" and names the range of numbers, where
Sonnet 5.5 kept an en dash: 5 plain answers out of 5, against 1 out of 3.
Comments now ask for "only a reason the code cannot show", "never a
summary of a function or a line", "in doubt, none": Sonnet 5 files out
of 5 and Haiku 4 out of 5 pass the judge, against 1 out of 3 and 0 out
of 3.
- The ruleset ceiling goes from 600 to 630 estimated tokens: the two lines
take it to 628 at its heaviest level.
- Eval suite, nine cases, with the reworded rules: Fable 5.2 0.99 against
0.88 without the plugin, Opus 5.5 1.00 against 0.85, Sonnet 5.5 1.00
against 0.86, Haiku 5.5 0.98 against 0.84.

## [0.15.8] - 2026-11-13

### Added
Expand Down
16 changes: 9 additions & 7 deletions README.fr.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ Le ruleset complet est dans [Les règles](#les-regles), sous Référence.

## Mesuré

<p align="center"><img src="assets/readme/evals-fr.png" width="880" alt="Suite d'évals, neuf cas, trois runs chacun. Plus court, et plus souvent juste. Fable 5.2 : 0,98 avec crewcut, 0,88 pour le même modèle sans plugin. Opus 5.5 : 0,99 avec, 0,85 sans. Sonnet 5.5 : 0,98 avec, 0,86 sans. Haiku 5.5 : 0,97 avec, 0,84 sans. Le score est la part de graders réussis. Chaque cas note la justesse autant que la taille : une réponse plus courte mais fausse vaut zéro."></p>
<p align="center"><img src="assets/readme/evals-fr.png" width="880" alt="Suite d'évals, neuf cas, trois runs chacun. Plus court, et plus souvent juste. Fable 5.2 : 0,99 avec crewcut, 0,88 pour le même modèle sans plugin. Opus 5.5 : 1,00 avec, 0,85 sans. Sonnet 5.5 : 1,00 avec, 0,86 sans. Haiku 5.5 : 0,98 avec, 0,84 sans. Le score est la part de graders réussis. Chaque cas note la justesse autant que la taille : une réponse plus courte mais fausse vaut zéro."></p>

<p align="center"><img src="assets/readme/pt5-fr.png" width="880" alt="Le benchmark de ponytail 5, crewcut 0.14.0 face à sans plugin, Opus 5.5, 39 tâches, 5 runs. Lignes de code moins 58 pour cent, intervalle à 95 pour cent de moins 68 à moins 47, mieux sur 32 tâches et pire sur 1. Information de code moins 52 pour cent. Tokens de sortie moins 56 pour cent. Coût moins 34 pour cent, intervalle de moins 44 à moins 24. Temps moins 53 pour cent. Tours moins 19 pour cent. Vérifications cachées : 85 sur 90 réussies, 86 sans le plugin. Six tâches de sécurité : 30 sur 30 dans les deux bras. Tâche de confidentialité : 0 sur 5, puis 8 sur 10 après la règle qui nomme ce qu'un changement peut exposer. Ponytail 5 a publié pour lui-même : code moins 53 pour cent, tokens de sortie moins 45, coût moins 26, temps moins 41. Deux runs à un mois d'écart, les intervalles se recouvrent."></p>

Expand All @@ -71,10 +71,10 @@ celui de la session seule, sans le juge :

| Modèle | crewcut | Score avec | sans | Tours avec | sans | Coût par run avec | sans |
|---|---|--:|--:|--:|--:|--:|--:|
| Fable 5.2 | 0.15.7 | 0,98 | 0,88 | 4,3 | 5,7 | 0,203 USD | 0,192 USD |
| Opus 5.5 | 0.15.7 | 0,99 | 0,85 | 4,7 | 5,4 | 0,090 USD | 0,087 USD |
| Sonnet 5.5 | 0.15.7 | 0,98 | 0,86 | 4,4 | 5,3 | 0,044 USD | 0,041 USD |
| Haiku 5.5 | 0.15.7 | 0,97 | 0,84 | 5,2 | 6,0 | 0,003 USD | 0,003 USD |
| Fable 5.2 | 0.16.0 | 0,99 | 0,88 | 4,6 | 5,7 | 0,201 USD | 0,192 USD |
| Opus 5.5 | 0.16.0 | 1,00 | 0,85 | 5,0 | 5,4 | 0,093 USD | 0,087 USD |
| Sonnet 5.5 | 0.16.0 | 1,00 | 0,86 | 4,5 | 5,3 | 0,044 USD | 0,041 USD |
| Haiku 5.5 | 0.16.0 | 0,98 | 0,84 | 5,1 | 6,0 | 0,003 USD | 0,003 USD |

Le benchmark de ponytail 5, Opus 5.5, 39 tâches, 5 runs, crewcut 0.14.0
face à sans plugin :
Expand Down Expand Up @@ -284,10 +284,12 @@ récente que ce qu'il connaît : il lit la documentation ou le code installé
au lieu de deviner. Jamais un pattern déprécié, jamais une fonctionnalité
que la version n'a pas.

**Commentaires.** Le pourquoi seulement, en une ligne simple.
**Commentaires.** Seulement une raison que le code ne peut pas montrer, en
une ligne simple ; jamais un résumé de ce que fait une fonction ou une
ligne. En cas de doute, aucun.

**Texte brut seulement.** Dans le code, les commentaires, les commits, les
pull requests et les réponses : pas d'emoji, pas d'émoticône, pas de symbole
pull requests et les réponses, même longues, et dans les plages de nombres : pas d'emoji, pas d'émoticône, pas de symbole
flèche (`->` à la place), pas de tiret long (`-` à la place), pas de
guillemets courbes (`"` à la place). Tout ce qui s'en trouve dans un texte
que Claude touche est remplacé.
Expand Down
16 changes: 9 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ The full ruleset is in [The rules](#the-rules), under Reference.

## Measured

<p align="center"><img src="assets/readme/evals.png" width="880" alt="Eval suite, nine cases, three runs each. Shorter, and more often right. Fable 5.2: 0.98 with crewcut, 0.88 for the same model with no plugin. Opus 5.5: 0.99 with, 0.85 without. Sonnet 5.5: 0.98 with, 0.86 without. Haiku 5.5: 0.97 with, 0.84 without. Score is the share of graders passed. Every case grades correctness as well as size: a shorter answer that is wrong scores zero."></p>
<p align="center"><img src="assets/readme/evals.png" width="880" alt="Eval suite, nine cases, three runs each. Shorter, and more often right. Fable 5.2: 0.99 with crewcut, 0.88 for the same model with no plugin. Opus 5.5: 1.00 with, 0.85 without. Sonnet 5.5: 1.00 with, 0.86 without. Haiku 5.5: 0.98 with, 0.84 without. Score is the share of graders passed. Every case grades correctness as well as size: a shorter answer that is wrong scores zero."></p>

<p align="center"><img src="assets/readme/pt5.png" width="880" alt="Ponytail 5's benchmark, crewcut 0.14.0 against no plugin, Opus 5.5, 39 tasks, 5 runs. Lines of code minus 58 percent, 95 percent interval minus 68 to minus 47, better on 32 tasks and worse on 1. Code information minus 52 percent. Output tokens minus 56 percent. Cost minus 34 percent, interval minus 44 to minus 24. Time minus 53 percent. Turns minus 19 percent. Hidden checks: 85 of 90 passed, 86 without the plugin. Six security tasks: 30 of 30 in both arms. Privacy task: 0 of 5, then 8 of 10 after the rule that names what a change could expose. Ponytail 5 published for itself: code minus 53 percent, output tokens minus 45, cost minus 26, time minus 41. Two runs a month apart, intervals overlap."></p>

Expand All @@ -71,10 +71,10 @@ session alone, without the judge:

| Model | crewcut | Score with | without | Turns with | without | Cost per run with | without |
|---|---|--:|--:|--:|--:|--:|--:|
| Fable 5.2 | 0.15.7 | 0.98 | 0.88 | 4.3 | 5.7 | 0.203 USD | 0.192 USD |
| Opus 5.5 | 0.15.7 | 0.99 | 0.85 | 4.7 | 5.4 | 0.090 USD | 0.087 USD |
| Sonnet 5.5 | 0.15.7 | 0.98 | 0.86 | 4.4 | 5.3 | 0.044 USD | 0.041 USD |
| Haiku 5.5 | 0.15.7 | 0.97 | 0.84 | 5.2 | 6.0 | 0.003 USD | 0.003 USD |
| Fable 5.2 | 0.16.0 | 0.99 | 0.88 | 4.6 | 5.7 | 0.201 USD | 0.192 USD |
| Opus 5.5 | 0.16.0 | 1.00 | 0.85 | 5.0 | 5.4 | 0.093 USD | 0.087 USD |
| Sonnet 5.5 | 0.16.0 | 1.00 | 0.86 | 4.5 | 5.3 | 0.044 USD | 0.041 USD |
| Haiku 5.5 | 0.16.0 | 0.98 | 0.84 | 5.1 | 6.0 | 0.003 USD | 0.003 USD |

Ponytail 5's benchmark, Opus 5.5, 39 tasks, 5 runs, crewcut 0.14.0 against
no plugin:
Expand Down Expand Up @@ -274,9 +274,11 @@ or a version newer than it knows: it reads the docs or the installed source
instead of guessing. Never a deprecated pattern, never a feature the version
lacks.

**Comments.** The why only, in one plain line.
**Comments.** Only a reason the code cannot show, in one plain line; never
a summary of what a function or a line does. When in doubt, none.

**Plain text only.** In code, comments, commits, pull requests and answers:
**Plain text only.** In code, comments, commits, pull requests and answers,
long ones included, and in ranges of numbers:
no emoji, no emoticon, no arrow symbol (`->` instead), no long dash (`-`
instead), no curly quotes (`"` instead). Any found in text Claude touches is
replaced.
Expand Down
Binary file modified assets/readme/evals-fr.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/readme/evals.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
8 changes: 4 additions & 4 deletions assets/readme/src/evals-fr.html
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@
<div class="card"><div class="pole"></div>
<div class="eyebrow">Suite d'évals, neuf cas, trois runs chacun</div>
<h1>Plus court, et <em>plus souvent juste.</em></h1>
<div class="row"><div class="m">Fable 5.2</div><div><div class="bar w" style="width:98%">0,98</div><div class="bar o" style="width:88%">0,88</div></div></div>
<div class="row"><div class="m">Opus 5.5</div><div><div class="bar w" style="width:99%">0,99</div><div class="bar o" style="width:85%">0,85</div></div></div>
<div class="row"><div class="m">Sonnet 5.5</div><div><div class="bar w" style="width:98%">0,98</div><div class="bar o" style="width:86%">0,86</div></div></div>
<div class="row"><div class="m">Haiku 5.5</div><div><div class="bar w" style="width:97%">0,97</div><div class="bar o" style="width:84%">0,84</div></div></div>
<div class="row"><div class="m">Fable 5.2</div><div><div class="bar w" style="width:99%">0,99</div><div class="bar o" style="width:88%">0,88</div></div></div>
<div class="row"><div class="m">Opus 5.5</div><div><div class="bar w" style="width:100%">1,00</div><div class="bar o" style="width:85%">0,85</div></div></div>
<div class="row"><div class="m">Sonnet 5.5</div><div><div class="bar w" style="width:100%">1,00</div><div class="bar o" style="width:86%">0,86</div></div></div>
<div class="row"><div class="m">Haiku 5.5</div><div><div class="bar w" style="width:98%">0,98</div><div class="bar o" style="width:84%">0,84</div></div></div>
<div class="legend"><span><i style="background:#E34948"></i>avec crewcut</span><span><i style="background:#2A3945"></i>même modèle, sans plugin</span></div>
<div class="foot">Le score est la part de graders réussis. Chaque cas note la justesse autant que la taille :<br>une réponse plus courte mais fausse vaut zéro.</div>
</div>
Expand Down
8 changes: 4 additions & 4 deletions assets/readme/src/evals.html
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@
<div class="card"><div class="pole"></div>
<div class="eyebrow">Eval suite, nine cases, three runs each</div>
<h1>Shorter, and <em>more often right.</em></h1>
<div class="row"><div class="m">Fable 5.2</div><div><div class="bar w" style="width:98%">0.98</div><div class="bar o" style="width:88%">0.88</div></div></div>
<div class="row"><div class="m">Opus 5.5</div><div><div class="bar w" style="width:99%">0.99</div><div class="bar o" style="width:85%">0.85</div></div></div>
<div class="row"><div class="m">Sonnet 5.5</div><div><div class="bar w" style="width:98%">0.98</div><div class="bar o" style="width:86%">0.86</div></div></div>
<div class="row"><div class="m">Haiku 5.5</div><div><div class="bar w" style="width:97%">0.97</div><div class="bar o" style="width:84%">0.84</div></div></div>
<div class="row"><div class="m">Fable 5.2</div><div><div class="bar w" style="width:99%">0.99</div><div class="bar o" style="width:88%">0.88</div></div></div>
<div class="row"><div class="m">Opus 5.5</div><div><div class="bar w" style="width:100%">1.00</div><div class="bar o" style="width:85%">0.85</div></div></div>
<div class="row"><div class="m">Sonnet 5.5</div><div><div class="bar w" style="width:100%">1.00</div><div class="bar o" style="width:86%">0.86</div></div></div>
<div class="row"><div class="m">Haiku 5.5</div><div><div class="bar w" style="width:98%">0.98</div><div class="bar o" style="width:84%">0.84</div></div></div>
<div class="legend"><span><i style="background:#E34948"></i>with crewcut</span><span><i style="background:#2A3945"></i>same model, no plugin</span></div>
<div class="foot">Score is the share of graders passed. Every case grades correctness as well as size:<br>a shorter answer that is wrong scores zero.</div>
</div>
Expand Down
48 changes: 48 additions & 0 deletions evals/RESULTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -357,3 +357,51 @@ started; the run was launched again.
As for the eight-case table, this joins measures of the same day on the
same plugin code: the eight cases and `explain-retry`. 54 runs per model,
none in error.

## Two rules reworded on explain-retry (crewcut 0.16.0, 2026-11-13)

`explain-retry` showed two rules giving way with the plugin on: Sonnet kept
a long dash in 2 answers out of 3, and one comment per file named the
mechanism instead of the reason. Three wordings were tried, five runs per
model, plugin arm only, the traces kept and read.

| Wording | Sonnet, plain answer | Sonnet, comments why | Haiku, comments why |
|---|--:|--:|--:|
| 0.15.8: "long dash (write -)" and "why only, one plain line" | 1 / 3 | 1 / 3 | 0 / 3 |
| "em or en dash", "gives the reason; never what the code does" | 2 / 5 | 0 / 5 | 0 / 5 |
| the same, "only a reason the code cannot show; in doubt, none" | 3 / 5 | 3 / 5 | 0 / 5 |
| the same, "(write -, ranges too: 0-100)" and "never a summary of what a function or a line does" | 5 / 5 | 5 / 5 | 4 / 5 |

The traces gave the two levers. Sonnet's dash was not in its prose but in a
range of numbers, "0 to 100 ms" written with an en dash: naming the case in
the rule ended it. Haiku opened each function with a comment that sums it
up: forbidding the summary ended that. Naming the dash "em or en" and
asking for "the reason" did nothing on their own.

The two lines as shipped, a little tighter than the last row:

Plain text only, long answers too: no emoji, arrow (write ->), em or en dash (write -, 0-100 too) or curly quotes.
Comments: only a reason the code cannot show, one plain line; never a summary of a function or a line; in doubt, none.

They take the ruleset from 598 to 628 estimated tokens at its heaviest
level, over the 600 ceiling the test held. Nothing could be cut elsewhere
without touching a wording already measured, so the ceiling is now 630.

The nine cases with the plugin, three runs, Opus as judge. The arm without
the plugin does not see the ruleset: it is the one of the previous section.

| Model | Score with | was | without | Turns with | without | Session cost with | without |
|---|--:|--:|--:|--:|--:|--:|--:|
| Fable 5.2 | 0.99 | 0.98 | 0.88 | 4.6 | 5.7 | 0.201 USD | 0.192 USD |
| Opus 5.5 | 1.00 | 0.99 | 0.85 | 5.0 | 5.4 | 0.093 USD | 0.087 USD |
| Sonnet 5.5 | 1.00 | 0.98 | 0.86 | 4.5 | 5.3 | 0.044 USD | 0.041 USD |
| Haiku 5.5 | 0.98 | 0.97 | 0.84 | 5.1 | 6.0 | 0.003 USD | 0.003 USD |

No case lost ground beyond one grader in one run. On `explain-retry`, the
plugin now scores 1.00 on Opus and Sonnet and 0.97 on Haiku. Sonnet passes
every grader of every case. What still fails with the plugin: Fable,
`date-picker` `short-answer` twice and `shared-bug` `root-cause-stated`
twice; Opus, `csv-export` `root-cause-stated` once; Haiku, five graders
once each. 12.42 USD for this table, 7.09 USD for the three trials. Six
Fable runs did not start (a process spawn error, twice that day already)
and their two cases were run again.
4 changes: 2 additions & 2 deletions hooks/ruleset.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ Build the ticket only: no optional prop, state, mode or edge case it did not nam
Never cut: trust-boundary validation, data-loss handling, security, accessibility, existing tests.
Short never means wrong: a question gets a full answer, with where a fix would go; a failing test is fixed and rerun; a file changed since last read is reread.
Commits and PRs: short subject, body if useful; no AI mention, no AI co-author, no trailer; strip any such line.
Plain text only: no emoji, arrow (write ->), long dash (write -) or curly quotes.
Comments: why only, one plain line.
Plain text only, long answers too: no emoji, arrow (write ->), em or en dash (write -, 0-100 too) or curly quotes.
Comments: only a reason the code cannot show, one plain line; never a summary of a function or a line; in doubt, none.
Modern by default: follow the installed versions' current docs, compact forms when clear; unsure: read them; never a feature the version lacks.

Tokens:
Expand Down
8 changes: 4 additions & 4 deletions skills/crewcut-gain/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ crewcut gain measured on public repos and eval cases, not on t
eval cases (nine, three runs each, same model with and without)

score with / without turns with / without session cost with / without
Fable 5.2 0.98 / 0.88 4.3 / 5.7 -25 % 0.203 / 0.192 USD +6 %
Opus 5.5 0.99 / 0.85 4.7 / 5.4 -13 % 0.090 / 0.087 USD +3 %
Sonnet 5.5 0.98 / 0.86 4.4 / 5.3 -17 % 0.044 / 0.041 USD +7 %
Haiku 5.5 0.97 / 0.84 5.2 / 6.0 -13 % 0.003 / 0.003 USD 0 %
Fable 5.2 0.99 / 0.88 4.6 / 5.7 -19 % 0.201 / 0.192 USD +5 %
Opus 5.5 1.00 / 0.85 5.0 / 5.4 -7 % 0.093 / 0.087 USD +7 %
Sonnet 5.5 1.00 / 0.86 4.5 / 5.3 -15 % 0.044 / 0.041 USD +7 %
Haiku 5.5 0.98 / 0.84 5.1 / 6.0 -15 % 0.003 / 0.003 USD 0 %

This repo has no baseline: the version you did not build was never written,
so no per-repo saving can be honestly computed.
Expand Down
5 changes: 3 additions & 2 deletions skills/crewcut/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,8 @@ second ticket. No type alias or helper for a single use.
- No scaffolding for a future that may not come. Delete before you add.
Boring beats clever. As few files as possible.
- Two options of the same size: take the one right on edge cases.
- Comments say why, in one plain line, and never name crewcut. With markers
- Comments give a reason the code cannot show, in one plain line: never a
summary of a function or a line, never the name crewcut. With markers
on, a cut with a known ceiling gets `// ~ no retry, add when the
API flakes`; no other comment gets the prefix.
- Ship the simple version and question the complex request in the same
Expand Down Expand Up @@ -106,7 +107,7 @@ guess. Never a deprecated pattern or a feature the version lacks.
## Plain text only

In code, comments, commit messages, pull requests and answers: no emoji, no
emoticon, no arrow symbol (write `->`), no long dash (write `-`), no curly
emoticon, no arrow symbol (write `->`), no long dash (write `-`, in ranges too: 0-100), no curly
quotes (write `"`), no non-breaking space. Replace any you find in text you
touch.

Expand Down
4 changes: 2 additions & 2 deletions tests/crewcut.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -662,10 +662,10 @@ test('the real ruleset carries the plain-punctuation rule at every active level'
const markdown = fs.readFileSync(path.join(__dirname, '..', 'hooks', 'ruleset.md'), 'utf8');
for (const level of ['lite', 'full', 'ultra']) {
const text = renderRuleset(level, markdown);
assert.match(text, /Plain text only:/, level);
assert.match(text, /Plain text only, long answers too:/, level);
assert.match(text, /no emoji/, level);
assert.match(text, /write ->/, level);
assert.match(text, /write -\)/, level);
assert.match(text, /write -, 0-100 too\)/, level);
}
});

Expand Down
4 changes: 2 additions & 2 deletions tests/text.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -66,12 +66,12 @@ test('commit messages use plain punctuation and carry no trailer', () => {
assert.doesNotMatch(log, /^[A-Za-z-]+: .+$/m, 'trailer line found in a commit message');
});

test('compact ruleset stays under 600 estimated tokens at every level', () => {
test('compact ruleset stays under 630 estimated tokens at every level', () => {
const markdown = fs.readFileSync(path.join(root, 'hooks', 'ruleset.md'), 'utf8');
for (const level of LEVELS) {
for (const markers of [false, true]) {
const tokens = Math.ceil(renderRuleset(level, markdown, { markers }).length / 4);
assert.ok(tokens < 600, `${level}, markers ${markers}: about ${tokens} tokens`);
assert.ok(tokens < 630, `${level}, markers ${markers}: about ${tokens} tokens`);
}
}
});
Expand Down
Loading