diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1f1b2e8..a4336ee 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,22 @@ follows Keep a Changelog and the project follows Semantic Versioning.
## [Unreleased]
+### Added
+
+- A ninth eval case, `explain-retry`: a long explanation and comments to
+ write, where a model reaches for long dashes, arrows and comment blocks.
+ Without the plugin no model writes a plain answer or one-line comments,
+ 0 run out of 12 each. With it, 10 and 11 out of 12, and the explanation
+ stays complete in every run. Two weak spots: Sonnet 5.5 keeps a long
+ dash in 2 answers out of 3, and one comment per file often names the
+ mechanism instead of the reason.
+
+### Changed
+
+- The eval tables and the eval card count nine cases: Fable 5.2 0.98
+ against 0.88 without the plugin, Opus 5.5 0.99 against 0.85, Sonnet 5.5
+ 0.98 against 0.86, Haiku 5.5 0.97 against 0.84.
+
## [0.15.7] - 2026-11-13
### Changed
diff --git a/README.fr.md b/README.fr.md
index 559e560..a782191 100644
--- a/README.fr.md
+++ b/README.fr.md
@@ -46,7 +46,7 @@ Le ruleset complet est dans [Les règles](#les-regles), sous Référence.
## Mesuré
-

+

@@ -66,15 +66,15 @@ par tâche, limites et reproduction :
Les chiffres en texte
-Suite d'évals, huit cas, trois runs chacun, Opus 5.5 comme juge. Le coût est
+Suite d'évals, neuf cas, trois runs chacun, Opus 5.5 comme juge. Le coût est
celui de la session seule, sans le juge :
| Modèle | crewcut | Score avec | sans | Tours avec | sans | Coût par run avec | sans |
|---|---|--:|--:|--:|--:|--:|--:|
-| Fable 5.2 | 0.15.6 | 0,98 | 0,89 | 4,3 | 6,0 | 0,197 USD | 0,189 USD |
-| Opus 5.5 | 0.15.6 | 0,99 | 0,86 | 4,8 | 5,7 | 0,089 USD | 0,086 USD |
-| Sonnet 5.5 | 0.15.6 | 0,99 | 0,87 | 4,4 | 5,6 | 0,043 USD | 0,040 USD |
-| Haiku 5.5 | 0.15.6 | 0,98 | 0,86 | 5,3 | 6,4 | 0,003 USD | 0,003 USD |
+| Fable 5.2 | 0.15.7 | 0,98 | 0,88 | 4,3 | 5,7 | 0,203 USD | 0,192 USD |
+| Opus 5.5 | 0.15.7 | 0,99 | 0,85 | 4,7 | 5,4 | 0,090 USD | 0,087 USD |
+| Sonnet 5.5 | 0.15.7 | 0,98 | 0,86 | 4,4 | 5,3 | 0,044 USD | 0,041 USD |
+| Haiku 5.5 | 0.15.7 | 0,97 | 0,84 | 5,2 | 6,0 | 0,003 USD | 0,003 USD |
Le benchmark de ponytail 5, Opus 5.5, 39 tâches, 5 runs, crewcut 0.14.0
face à sans plugin :
diff --git a/README.md b/README.md
index b439dd8..7c12856 100644
--- a/README.md
+++ b/README.md
@@ -46,7 +46,7 @@ The full ruleset is in [The rules](#the-rules), under Reference.
## Measured
-
+

@@ -66,15 +66,15 @@ limits and how to reproduce:
The figures as text
-Eval suite, eight cases, three runs each, Opus 5.5 as judge. The cost is the
+Eval suite, nine cases, three runs each, Opus 5.5 as judge. The cost is the
session alone, without the judge:
| Model | crewcut | Score with | without | Turns with | without | Cost per run with | without |
|---|---|--:|--:|--:|--:|--:|--:|
-| Fable 5.2 | 0.15.6 | 0.98 | 0.89 | 4.3 | 6.0 | 0.197 USD | 0.189 USD |
-| Opus 5.5 | 0.15.6 | 0.99 | 0.86 | 4.8 | 5.7 | 0.089 USD | 0.086 USD |
-| Sonnet 5.5 | 0.15.6 | 0.99 | 0.87 | 4.4 | 5.6 | 0.043 USD | 0.040 USD |
-| Haiku 5.5 | 0.15.6 | 0.98 | 0.86 | 5.3 | 6.4 | 0.003 USD | 0.003 USD |
+| Fable 5.2 | 0.15.7 | 0.98 | 0.88 | 4.3 | 5.7 | 0.203 USD | 0.192 USD |
+| Opus 5.5 | 0.15.7 | 0.99 | 0.85 | 4.7 | 5.4 | 0.090 USD | 0.087 USD |
+| Sonnet 5.5 | 0.15.7 | 0.98 | 0.86 | 4.4 | 5.3 | 0.044 USD | 0.041 USD |
+| Haiku 5.5 | 0.15.7 | 0.97 | 0.84 | 5.2 | 6.0 | 0.003 USD | 0.003 USD |
Ponytail 5's benchmark, Opus 5.5, 39 tasks, 5 runs, crewcut 0.14.0 against
no plugin:
diff --git a/assets/readme/evals-fr.png b/assets/readme/evals-fr.png
index b7591d4..a3de490 100644
Binary files a/assets/readme/evals-fr.png and b/assets/readme/evals-fr.png differ
diff --git a/assets/readme/evals.png b/assets/readme/evals.png
index 0ee9a84..a7ffcba 100644
Binary files a/assets/readme/evals.png and b/assets/readme/evals.png differ
diff --git a/assets/readme/src/evals-fr.html b/assets/readme/src/evals-fr.html
index 102822f..5310ecc 100644
--- a/assets/readme/src/evals-fr.html
+++ b/assets/readme/src/evals-fr.html
@@ -9,12 +9,12 @@
.legend i { display: inline-block; width: 12px; height: 12px; border-radius: 3px; margin-right: 8px; }
-
Suite d'évals, huit cas, trois runs chacun
+
Suite d'évals, neuf cas, trois runs chacun
Plus court, et plus souvent juste.
-
-
-
-
+
+
+
+
avec crewcutmême modèle, sans plugin
diff --git a/assets/readme/src/evals.html b/assets/readme/src/evals.html
index cbd7990..dcb548e 100644
--- a/assets/readme/src/evals.html
+++ b/assets/readme/src/evals.html
@@ -9,12 +9,12 @@
.legend i { display: inline-block; width: 12px; height: 12px; border-radius: 3px; margin-right: 8px; }
-
Eval suite, eight cases, three runs each
+
Eval suite, nine cases, three runs each
Shorter, and more often right.
-
-
-
-
+
+
+
+
with crewcutsame model, no plugin
diff --git a/evals/RESULTS.md b/evals/RESULTS.md
index 6a0eb12..512457e 100644
--- a/evals/RESULTS.md
+++ b/evals/RESULTS.md
@@ -304,3 +304,56 @@ boundary and fails if any check disappears; `explain-bug` asks a question
and fails if the explanation is cut short; `vat-country` and `csv-export`
fail if an existing test or an untouched module changes.
+## A ninth case for long text and comments: explain-retry (crewcut 0.15.7, 2026-11-13)
+
+On the older cases the plain text and comment graders passed almost
+everywhere, in both arms: short tasks do not invite long prose or comment
+blocks. This case does. One file, a fetch with retries and no comment; the
+ticket: "A new hire starts on Monday. Explain in your answer how
+fetchWithRetry in src/retry.js works, step by step, and add the comments
+the code needs." Fourteen graders: the code is untouched, the answer covers
+six behaviours (missing two fails), no banned character in the answer or
+the file, no block comment and no two comment lines in a row, every comment
+gives a reason.
+
+| Runs passed, with / without the plugin | Fable | Opus | Sonnet | Haiku |
+|---|--:|--:|--:|--:|
+| Score | 1.00 / 0.79 | 0.94 / 0.70 | 0.88 / 0.73 | 0.88 / 0.70 |
+| full-explanation | 3 / 3 | 3 / 3 | 3 / 3 | 3 / 3 |
+| plain-text-answer | 3 / 0 | 3 / 0 | 1 / 0 | 3 / 0 |
+| comments-one-line | 3 / 0 | 3 / 0 | 3 / 0 | 2 / 0 |
+| comments-say-why | 3 / 2 | 1 / 0 | 1 / 0 | 0 / 0 |
+
+Three runs per arm, 4.46 USD with the judge. The explanation is complete in
+every run of both arms: the plugin's shorter answer loses nothing here.
+
+**Without the plugin, no model writes a plain answer and none keeps its
+comments to one line**: 0 run out of 12 on each. An explanation in steps
+brings long dashes and arrows, and "add the comments" brings blocks.
+
+**With the plugin, two weak spots.** Sonnet still puts a long dash in its
+answer in 2 runs out of 3: the plain text rule holds on short answers and
+gives way on a long one. And `comments-say-why` passes in 5 runs out of 12
+with the plugin. The comments were read: most give a reason ("4xx is the
+caller's fault, so retrying cannot help"), and one per file names the
+mechanism instead ("Exponential backoff, capped at 5s"). The grader asks
+that every comment give a reason, so one such line fails the file. That is
+what "why only" says; the rule is not yet followed to the letter.
+
+A first launch of this run found no `claude` command for any model: the
+Claude Code binary was being rewritten on disk at that minute, at the same
+version, as during the benchmark earlier in the day. No session had
+started; the run was launched again.
+
+## Nine cases (crewcut 0.15.7, 2026-11-13)
+
+| Model | Score with | without | Turns with | without | Session cost with | without |
+|---|--:|--:|--:|--:|--:|--:|
+| Fable 5.2 | 0.98 | 0.88 | 4.3 | 5.7 | 0.203 USD | 0.192 USD |
+| Opus 5.5 | 0.99 | 0.85 | 4.7 | 5.4 | 0.090 USD | 0.087 USD |
+| Sonnet 5.5 | 0.98 | 0.86 | 4.4 | 5.3 | 0.044 USD | 0.041 USD |
+| Haiku 5.5 | 0.97 | 0.84 | 5.2 | 6.0 | 0.003 USD | 0.003 USD |
+
+As for the eight-case table, this joins measures of the same day on the
+same plugin code: the eight cases and `explain-retry`. 54 runs per model,
+none in error.
diff --git a/evals/explain-retry/case.yaml b/evals/explain-retry/case.yaml
new file mode 100644
index 0000000..90e139e
--- /dev/null
+++ b/evals/explain-retry/case.yaml
@@ -0,0 +1,73 @@
+schema_version: "1.0"
+name: explain-retry
+description: "A request for a long explanation and for comments, where a model reaches for arrows, long dashes, emoji and comment blocks. Expected: a full explanation in plain text, few one-line comments that give a reason, the code untouched."
+context:
+ scaffold_script: scaffold.sh
+execution:
+ prompt: A new hire starts on Monday. Explain in your answer how fetchWithRetry in src/retry.js works, step by step, and add the comments the code needs.
+ max_turns: 10
+ timeout_seconds: 300
+ allowed_tools: [Read, Edit, Grep, Glob, Skill]
+runs: 3
+graders:
+ - type: file_exists
+ name: no-new-file
+ path: "**/*"
+ exists: false
+ - type: regex
+ name: keeps-attempts
+ target: { source: file, path: src/retry.js }
+ pattern: 'attempts = 4'
+ match: contains
+ - type: regex
+ name: keeps-status-check
+ target: { source: file, path: src/retry.js }
+ pattern: 'if \(response\.status < 500\) return response;'
+ match: contains
+ - type: regex
+ name: keeps-abort
+ target: { source: file, path: src/retry.js }
+ pattern: 'if \(error\.name === ''AbortError''\) throw error;'
+ match: contains
+ - type: regex
+ name: keeps-backoff
+ target: { source: file, path: src/retry.js }
+ pattern: 'delay = Math\.min\(delay \* 2, 5000\);'
+ match: contains
+ - type: regex
+ name: plain-text-answer
+ pattern: '[\u2013\u2014\u2018\u2019\u201C\u201D\u00A0\u202F\u2190-\u21FF\u2600-\u27BF\uD83C-\uD83E]'
+ match: not_contains
+ - type: regex
+ name: plain-text-in-code
+ target: { source: file, path: src/retry.js }
+ pattern: '[\u2013\u2014\u2018\u2019\u201C\u201D\u00A0\u202F\u2190-\u21FF\u2600-\u27BF\uD83C-\uD83E]'
+ match: not_contains
+ - type: regex
+ name: no-crewcut-in-code
+ target: { source: file, path: src/retry.js }
+ pattern: '[Cc]rewcut'
+ match: not_contains
+ - type: regex
+ name: comments-one-line
+ target: { source: file, path: src/retry.js }
+ pattern: '/\*|//[^\n]*\n\s*//'
+ match: not_contains
+ - type: llm
+ name: comments-say-why
+ focus: { source: file, path: src/retry.js }
+ criteria: >
+ Every comment in the file gives a reason or a constraint the code
+ cannot show. No comment restates what the next line does, and none
+ names a tool or an assistant. A file with no comment fails: the
+ request asked for the comments the code needs.
+ - type: llm
+ name: full-explanation
+ criteria: >
+ The final message explains all of these: it tries up to four times by
+ default; a response with a status below 500, 4xx included, is returned
+ at once and not retried; a 5xx response or a network error is retried;
+ an aborted request is rethrown at once; the wait starts at 200 ms,
+ doubles each time, is capped at 5 seconds and gets a small random
+ extra; after the last attempt the last error is thrown. Missing two or
+ more of these fails.
diff --git a/evals/explain-retry/scaffold.sh b/evals/explain-retry/scaffold.sh
new file mode 100755
index 0000000..0eebb74
--- /dev/null
+++ b/evals/explain-retry/scaffold.sh
@@ -0,0 +1,33 @@
+#!/usr/bin/env bash
+set -eu
+cat > package.json <<'EOF'
+{
+ "name": "demo-http-client",
+ "version": "1.0.0",
+ "private": true,
+ "type": "module",
+ "engines": { "node": ">=22" }
+}
+EOF
+mkdir -p src
+cat > src/retry.js <<'EOF'
+export async function fetchWithRetry(url, options = {}, attempts = 4) {
+ let delay = 200;
+ let lastError;
+ for (let attempt = 1; attempt <= attempts; attempt++) {
+ try {
+ const response = await fetch(url, options);
+ if (response.status < 500) return response;
+ lastError = new Error(`HTTP ${response.status}`);
+ } catch (error) {
+ if (error.name === 'AbortError') throw error;
+ lastError = error;
+ }
+ if (attempt < attempts) {
+ await new Promise((resolve) => setTimeout(resolve, delay + Math.random() * 100));
+ delay = Math.min(delay * 2, 5000);
+ }
+ }
+ throw lastError;
+}
+EOF
diff --git a/skills/crewcut-gain/SKILL.md b/skills/crewcut-gain/SKILL.md
index 2749ee4..71c9a43 100644
--- a/skills/crewcut-gain/SKILL.md
+++ b/skills/crewcut-gain/SKILL.md
@@ -21,13 +21,13 @@ crewcut gain measured on public repos and eval cases, not on t
Next.js boilerplate, Opus 5.5 18 % 45 % 40 % 33 % 21/21
controls on Haiku, LOC: caveman 104 % and 85 %, yagni prompt 112 % and 71 %
- eval cases (eight, three runs each, same model with and without)
+ eval cases (nine, three runs each, same model with and without)
score with / without turns with / without session cost with / without
- Fable 5.2 0.98 / 0.89 4.3 / 6.0 -28 % 0.197 / 0.189 USD +4 %
- Opus 5.5 0.99 / 0.86 4.8 / 5.7 -16 % 0.089 / 0.086 USD +3 %
- Sonnet 5.5 0.99 / 0.87 4.4 / 5.6 -21 % 0.043 / 0.040 USD +8 %
- Haiku 5.5 0.98 / 0.86 5.3 / 6.4 -17 % 0.003 / 0.003 USD 0 %
+ Fable 5.2 0.98 / 0.88 4.3 / 5.7 -25 % 0.203 / 0.192 USD +6 %
+ Opus 5.5 0.99 / 0.85 4.7 / 5.4 -13 % 0.090 / 0.087 USD +3 %
+ Sonnet 5.5 0.98 / 0.86 4.4 / 5.3 -17 % 0.044 / 0.041 USD +7 %
+ Haiku 5.5 0.97 / 0.84 5.2 / 6.0 -13 % 0.003 / 0.003 USD 0 %
This repo has no baseline: the version you did not build was never written,
so no per-repo saving can be honestly computed.