You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Say in the property guidance that it covers every target and that fuzzing follows it #15825
/testing:write's property guidance says plainly that it covers every target and that fuzzing is the same mechanism, so nobody builds or looks for a separate fuzzing or UI-property skill.
Demo: ask /testing:write for a property test of a running UI, or for a fuzz test of a parser; it follows the one property guidance, finds the consumer's own tool, and names none itself.
Source: the talk "All Code Is Slop (Except Mine)" (Lauren Tan), slide 28 and the transcript at 1:12:39: "property-based testing as well for UIs". The tool she shows describes itself as "property-based testing, or fuzzing if you will".
Decisions
Owner, 2026-11-12: property testing is one thing with one home and is not split by target ("I think we just consolidate property testing and then split it based on the target"); flood-style fuzzing belongs with it.
A fuzz test is generated input with the weak rule "does not crash"; a property test is generated input with a stated rule. Same mechanism, same guidance.
Generated user actions against a running UI are a generated call sequence on a stateful thing, which the guidance already covers.
No tool is named or shipped; the tool comes from the consumer's repository.
This ships separately from the pilot cut so that cut's before-and-after result is not confounded by added text.
Key interfaces
None.
Done when
The property guidance states, in one or two sentences, that it applies to every target (a running UI included) and that a fuzz test follows it; it names no tool.
One new eval case, frozen first, covers the sentences; the skill's existing write cases are non-inferior on a two-version compare (opus and sonnet aliases, three runs per side, --case filter, a --max-cost-usd ceiling, no new batch after a usage-limit error), run in the one local eval-runner session; the pull request carries do-not-merge until the record lands.
The pull request body names this repository's dogfooding step.
Size: Small
Complexity: Low
Uncertainty: Known
Flags: none
Blocked by: none
Understood: add one or two sentences to the /testing:write property guidance saying it applies to every target (a running UI included) and that a fuzz test follows it. The sentences name no tool. The change also needs one new frozen eval case and a non-inferiority compare on the existing write cases, and the pull request body names the dogfooding step.
Not admitted to the issue lane: merged pull request #16464 names it with Refs: after its newest decision or refinement comment, so its work may already be done: close the issue if it is, or post a comment whose first line is **Decision recorded.** saying what is left, which sends it back through intake. Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).
melodic-automation-lanes commented on Nov 15, 2026
Pull request #16464 merged with Refs:, so this issue stays open and nothing starts the lane on it again. work-unblock removed status: ready and agent-ready, so the hourly triage sweep sends it back to intake triage. Admission then parks it at status: confirmed for a check of what that pull request left, until a **Decision recorded.** comment says what is left or someone closes it.
One targeted case per changed skill, run on the old text (before) and the new text (after), on sonnet at 3 runs a side, Claude Code 2.1.327 (Claude Code).
What this gate checks: that the skill's existing write cases and the new case do no worse on the new text than on the old (the issue's non-inferiority compare). The issue states no margin, so the rule registered before any run is the per-case no-loss rule below (PROVISIONAL until the owner confirms it). There is no wrong-text side, so a PASS says these cases did not get worse, not that they would have noticed a bad edit.
Scope: not a cut. PR 16464 adds two sentences to context/property.md (the guidance covers every target, a running UI included, and a fuzz test follows it) and widens the SKILL.md Step 0 route row to fuzz tests and any target. Measured: all 22 cases whose names start wr at the merge commit (21 existing write cases plus the new wr-fuzz-follows-property), non-inferiority; the SKILL.md row is loaded with the skill, and the property.md row counts as measured only if opened in at least 2 of 3 runs a side. Unmeasured: the running-UI clause (no case has a running-UI fixture, as the PR body says).
Cases: every plugin-eval case whose name starts wr at the merge commit (wr-* and write-*, 22), copied byte-identical under their own names; wr-parallel-worker-isolation, added to main after the merge, is not in the set.
No target is registered for the new case: the issue says it 'covers the sentences', not that it must show a change; its per-grader before and after counts are in the table.
Opus not run. The issue names the opus and sonnet aliases; the evals plugin's reading guide says a skill-text change is compared on Sonnet and Opus runs only to confirm a loss. The issue's two-alias clause is not met by this record.
Result rule PROVISIONAL: the issue states no margin. Registered: the per-case no-loss rule (a case's mean score falls by 0.15 or more, or any grader passes in 2 or more fewer after runs). Not registered: the reading guide's few-cases rule (noise-report margin 0.05; at about 20 cases or more a case-drop line or a not-shown-non-inferior verdict is a LOSS reading).
'Frozen first' is not met: the new case and the text change landed in one commit, as the PR body says.
Unmeasured rows: the running-UI clause of property.md.
Before: 8a544e1bd53f57302bd306803a68ea4ef3777cad (the merge commit's first parent)
After: 109b87a8f2b44bc108ab26e2796a13615e365100 (the merge commit on main; the pull request merged before this eval, merge commit read when this was written: 109b87a8f2b44bc108ab26e2796a13615e365100)
Loss: the mean case score falls by 0.15 or more, or any one grader passes in 2 or more fewer after runs than before runs.
Passing runs per grader, per side:
Case
Grader
Before
After
wr-fuzz-follows-property
skill-fired
3/3
3/3
wr-fuzz-follows-property
read-property
3/3
3/3
wr-fuzz-follows-property
uses-project-library
3/3
3/3
wr-fuzz-follows-property
no-separate-fuzzer
3/3
3/3
wr-fuzz-follows-property
states-round-trip
3/3
2/3
wr-fuzz-follows-property
mean case score
1.000
0.933
wr-glossary-absent
skill-fired
3/3
3/3
wr-glossary-absent
canonical-term
3/3
2/3
wr-glossary-absent
no-rejected-term
0/3
0/3
wr-glossary-absent
mean case score
0.667
0.556
wr-glossary-docs-path
skill-fired
3/3
3/3
wr-glossary-docs-path
canonical-term
3/3
3/3
wr-glossary-docs-path
no-rejected-term
3/3
3/3
wr-glossary-docs-path
glossary-read
3/3
3/3
wr-glossary-docs-path
mean case score
1.000
1.000
wr-glossary-root
skill-fired
3/3
3/3
wr-glossary-root
canonical-term
3/3
3/3
wr-glossary-root
no-rejected-term
3/3
3/3
wr-glossary-root
glossary-read
3/3
3/3
wr-glossary-root
mean case score
1.000
1.000
wr-go-uses-project-library
skill-fired
3/3
3/3
wr-go-uses-project-library
read-property
3/3
3/3
wr-go-uses-project-library
uses-rapid
3/3
3/3
wr-go-uses-project-library
meets-expectations
1/3
2/3
wr-go-uses-project-library
mean case score
0.833
0.917
wr-handler-unit-tests
skill-fired
3/3
3/3
wr-handler-unit-tests
read-write-md
3/3
3/3
wr-handler-unit-tests
colocated-file
3/3
3/3
wr-handler-unit-tests
documented-names
3/3
3/3
wr-handler-unit-tests
plain-asserts
3/3
3/3
wr-handler-unit-tests
no-tests-dir
3/3
3/3
wr-handler-unit-tests
meets-expected-output
0/3
0/3
wr-handler-unit-tests
mean case score
0.857
0.857
wr-hollow-out-receipt
skill-fired
3/3
3/3
wr-hollow-out-receipt
asserts-on-receipt
3/3
3/3
wr-hollow-out-receipt
meets-expectations
3/3
3/3
wr-hollow-out-receipt
mean case score
1.000
1.000
wr-integration-shared-library
skill-fired
3/3
3/3
wr-integration-shared-library
read-organize
3/3
3/3
wr-integration-shared-library
read-convention
3/3
3/3
wr-integration-shared-library
names-documented-location
3/3
3/3
wr-integration-shared-library
meets-expected-output
0/3
0/3
wr-integration-shared-library
mean case score
0.800
0.800
wr-light-skips-advisor-checkpoint
reads-level
3/3
3/3
wr-light-skips-advisor-checkpoint
writes-skip-token
3/3
3/3
wr-light-skips-advisor-checkpoint
mean case score
1.000
1.000
wr-no-skip-red
skill-fired
3/3
3/3
wr-no-skip-red
test-before-fix
0/3
0/3
wr-no-skip-red
separator-test
3/3
3/3
wr-no-skip-red
meets-expectations
0/3
0/3
wr-no-skip-red
mean case score
0.500
0.500
wr-placement-from-conventions
documented-path
3/3
3/3
wr-placement-from-conventions
read-conventions
3/3
3/3
wr-placement-from-conventions
mean case score
1.000
1.000
wr-property-dependent-input
skill-fired
3/3
3/3
wr-property-dependent-input
uses-given
3/3
3/3
wr-property-dependent-input
builds-dependent-input
3/3
3/3
wr-property-dependent-input
no-discarded-draws
3/3
3/3
wr-property-dependent-input
mean case score
1.000
1.000
wr-property-past-bug
skill-fired
3/3
3/3
wr-property-past-bug
uses-given
3/3
3/3
wr-property-past-bug
pins-past-bug
3/3
3/3
wr-property-past-bug
mean case score
1.000
1.000
wr-property-reimplements-split
skill-fired
3/3
3/3
wr-property-reimplements-split
points-to-test-value
0/3
1/3
wr-property-reimplements-split
sum-invariant
2/3
3/3
wr-property-reimplements-split
meets-expectations
0/3
1/3
wr-property-reimplements-split
mean case score
0.417
0.667
wr-python-idempotence-property
skill-fired
3/3
3/3
wr-python-idempotence-property
uses-given
3/3
3/3
wr-python-idempotence-property
idempotence
3/3
3/3
wr-python-idempotence-property
meets-expectations
1/3
1/3
wr-python-idempotence-property
mean case score
0.833
0.833
wr-round-trip-property
skill-fired
3/3
3/3
wr-round-trip-property
read-property
3/3
3/3
wr-round-trip-property
round-trip
3/3
3/3
wr-round-trip-property
literal-example
3/3
3/3
wr-round-trip-property
no-recomputed-expected
3/3
3/3
wr-round-trip-property
meets-expectations
0/3
0/3
wr-round-trip-property
mean case score
0.833
0.833
wr-ts-async-property-marked
skill-fired
3/3
3/3
wr-ts-async-property-marked
property-marker
3/3
3/3
wr-ts-async-property-marked
awaits-async-property
3/3
3/3
wr-ts-async-property-marked
mean case score
1.000
1.000
write-first-unit
flow-test-written
3/3
3/3
write-first-unit
resolver-ran
3/3
3/3
write-first-unit
settings-line
3/3
3/3
write-first-unit
skill-fired
3/3
3/3
write-first-unit
unit-test-first
3/3
2/3
write-first-unit
mean case score
1.000
0.933
write-first-user-flow
flow-test-first
3/3
3/3
write-first-user-flow
resolver-ran
3/3
3/3
write-first-user-flow
settings-line
3/3
3/3
write-first-user-flow
skill-fired
3/3
3/3
write-first-user-flow
mean case score
1.000
1.000
write-refactor-checkpoint
checkpoint-reported
3/3
3/3
write-refactor-checkpoint
resolver-ran
3/3
3/3
write-refactor-checkpoint
settings-line
3/3
3/3
write-refactor-checkpoint
skill-fired
3/3
3/3
write-refactor-checkpoint
mean case score
1.000
1.000
write-refactor-off
no-checkpoint
3/3
3/3
write-refactor-off
resolver-ran
3/3
3/3
write-refactor-off
settings-line
3/3
3/3
write-refactor-off
skill-fired
3/3
3/3
write-refactor-off
mean case score
1.000
1.000
write-refactor-on
checkpoint-reported
3/3
3/3
write-refactor-on
resolver-ran
3/3
3/3
write-refactor-on
settings-line
3/3
3/3
write-refactor-on
skill-fired
3/3
3/3
write-refactor-on
mean case score
1.000
1.000
Graders that cannot show a loss, because they failed on the old text in every run:
wr-glossary-absentno-rejected-term: cannot show a loss: failed on the old text in every run (0/3)
wr-handler-unit-testsmeets-expected-output: cannot show a loss: failed on the old text in every run (0/3)
wr-integration-shared-librarymeets-expected-output: cannot show a loss: failed on the old text in every run (0/3)
wr-no-skip-redtest-before-fix: cannot show a loss: failed on the old text in every run (0/3)
wr-no-skip-redmeets-expectations: cannot show a loss: failed on the old text in every run (0/3)
wr-property-reimplements-splitpoints-to-test-value: cannot show a loss: failed on the old text in every run (0/3)
wr-property-reimplements-splitmeets-expectations: cannot show a loss: failed on the old text in every run (0/3)
wr-round-trip-propertymeets-expectations: cannot show a loss: failed on the old text in every run (0/3)
Changed files the compare runs opened, per case and side:
wr-fuzz-follows-property before: opened skills/write/context/property.md in 3 of 3 runs
wr-fuzz-follows-property after: opened skills/write/context/property.md in 3 of 3 runs
wr-glossary-absent before: opened no changed file in 3 runs
wr-glossary-absent after: opened no changed file in 3 runs
wr-glossary-docs-path before: opened no changed file in 3 runs
wr-glossary-docs-path after: opened no changed file in 3 runs
wr-glossary-root before: opened no changed file in 3 runs
wr-glossary-root after: opened no changed file in 3 runs
wr-go-uses-project-library before: opened skills/write/context/property.md in 3 of 3 runs
wr-go-uses-project-library after: opened skills/write/context/property.md in 3 of 3 runs
wr-handler-unit-tests before: opened no changed file in 3 runs
wr-handler-unit-tests after: opened no changed file in 3 runs
wr-hollow-out-receipt before: opened no changed file in 3 runs
wr-hollow-out-receipt after: opened no changed file in 3 runs
wr-integration-shared-library before: opened no changed file in 3 runs
wr-integration-shared-library after: opened no changed file in 3 runs
wr-light-skips-advisor-checkpoint before: opened no changed file in 3 runs
wr-light-skips-advisor-checkpoint after: opened no changed file in 3 runs
wr-no-skip-red before: opened no changed file in 3 runs
wr-no-skip-red after: opened no changed file in 3 runs
wr-placement-from-conventions before: opened no changed file in 3 runs
wr-placement-from-conventions after: opened no changed file in 3 runs
wr-property-dependent-input before: opened skills/write/context/property.md in 3 of 3 runs
wr-property-dependent-input after: opened skills/write/context/property.md in 3 of 3 runs
wr-property-past-bug before: opened skills/write/context/property.md in 3 of 3 runs
wr-property-past-bug after: opened skills/write/context/property.md in 3 of 3 runs
wr-property-reimplements-split before: opened skills/write/context/property.md in 2 of 3 runs
wr-property-reimplements-split after: opened skills/write/context/property.md in 2 of 3 runs
wr-python-idempotence-property before: opened skills/write/context/property.md in 3 of 3 runs
wr-python-idempotence-property after: opened skills/write/context/property.md in 3 of 3 runs
wr-round-trip-property before: opened skills/write/context/property.md in 3 of 3 runs
wr-round-trip-property after: opened skills/write/context/property.md in 3 of 3 runs
wr-ts-async-property-marked before: opened skills/write/context/property.md in 3 of 3 runs
wr-ts-async-property-marked after: opened skills/write/context/property.md in 3 of 3 runs
write-first-unit before: opened no changed file in 3 runs
write-first-unit after: opened no changed file in 3 runs
write-first-user-flow before: opened no changed file in 3 runs
write-first-user-flow after: opened no changed file in 3 runs
write-refactor-checkpoint before: opened no changed file in 3 runs
write-refactor-checkpoint after: opened no changed file in 3 runs
write-refactor-off before: opened no changed file in 3 runs
write-refactor-off after: opened no changed file in 3 runs
write-refactor-on before: opened no changed file in 3 runs
write-refactor-on after: opened no changed file in 3 runs
Rows these runs measured (the row's file opened in at least 2 of 3 runs on each side, or loaded with the skill or agent):
SKILL-route-row-fuzz (skills/write/SKILL.md, case wr-fuzz-follows-property): measured; the file is loaded with the skill or agent, never opened, so no open count applies
PROP-every-target-and-fuzz (skills/write/context/property.md, case wr-fuzz-follows-property): opened in 3 of 3 before, 3 of 3 after runs
PROP-every-target-and-fuzz (skills/write/context/property.md, case wr-go-uses-project-library): opened in 3 of 3 before, 3 of 3 after runs
PROP-every-target-and-fuzz (skills/write/context/property.md, case wr-property-dependent-input): opened in 3 of 3 before, 3 of 3 after runs
PROP-every-target-and-fuzz (skills/write/context/property.md, case wr-property-past-bug): opened in 3 of 3 before, 3 of 3 after runs
PROP-every-target-and-fuzz (skills/write/context/property.md, case wr-property-reimplements-split): opened in 2 of 3 before, 2 of 3 after runs
PROP-every-target-and-fuzz (skills/write/context/property.md, case wr-python-idempotence-property): opened in 3 of 3 before, 3 of 3 after runs
PROP-every-target-and-fuzz (skills/write/context/property.md, case wr-round-trip-property): opened in 3 of 3 before, 3 of 3 after runs
PROP-every-target-and-fuzz (skills/write/context/property.md, case wr-ts-async-property-marked): opened in 3 of 3 before, 3 of 3 after runs
Rows these runs did not measure:
none
Verdicts:
verdict wr-fuzz-follows-property (wr-fuzz-follows-property): PASS (no loss: after 0.933, before 1.000)
verdict wr-glossary-absent (wr-glossary-absent): PASS (no loss: after 0.556, before 0.667)
verdict wr-glossary-docs-path (wr-glossary-docs-path): PASS (no loss: after 1.000, before 1.000)
verdict wr-glossary-root (wr-glossary-root): PASS (no loss: after 1.000, before 1.000)
verdict wr-go-uses-project-library (wr-go-uses-project-library): PASS (no loss: after 0.917, before 0.833)
verdict wr-handler-unit-tests (wr-handler-unit-tests): PASS (no loss: after 0.857, before 0.857)
verdict wr-hollow-out-receipt (wr-hollow-out-receipt): PASS (no loss: after 1.000, before 1.000)
verdict wr-integration-shared-library (wr-integration-shared-library): PASS (no loss: after 0.800, before 0.800)
verdict wr-light-skips-advisor-checkpoint (wr-light-skips-advisor-checkpoint): PASS (no loss: after 1.000, before 1.000)
verdict wr-no-skip-red (wr-no-skip-red): PASS (no loss: after 0.500, before 0.500)
verdict wr-placement-from-conventions (wr-placement-from-conventions): PASS (no loss: after 1.000, before 1.000)
verdict wr-property-dependent-input (wr-property-dependent-input): PASS (no loss: after 1.000, before 1.000)
verdict wr-property-past-bug (wr-property-past-bug): PASS (no loss: after 1.000, before 1.000)
verdict wr-property-reimplements-split (wr-property-reimplements-split): PASS (no loss: after 0.667, before 0.417)
verdict wr-python-idempotence-property (wr-python-idempotence-property): PASS (no loss: after 0.833, before 0.833)
verdict wr-round-trip-property (wr-round-trip-property): PASS (no loss: after 0.833, before 0.833)
verdict wr-ts-async-property-marked (wr-ts-async-property-marked): PASS (no loss: after 1.000, before 1.000)
verdict write-first-unit (write-first-unit): PASS (no loss: after 0.933, before 1.000)
verdict write-first-user-flow (write-first-user-flow): PASS (no loss: after 1.000, before 1.000)
verdict write-refactor-checkpoint (write-refactor-checkpoint): PASS (no loss: after 1.000, before 1.000)
verdict write-refactor-off (write-refactor-off): PASS (no loss: after 1.000, before 1.000)
verdict write-refactor-on (write-refactor-on): PASS (no loss: after 1.000, before 1.000)
Cost: 31.45 USD (smoke and compare, the runs' costUsd).
The cases live outside the repository; their full text:
The 22 cases: prompt.md and case.yaml
wr-fuzz-follows-property
case.yaml:
schema_version: "1.1"name: wr-fuzz-follows-propertydescription: 'Fuzz follows the property guidance. A fuzz test asked for a Python duration parser routes to context/property.md, uses the property library the project already lists (Hypothesis in the test extras) rather than a separate fuzzing tool, and states a rule stronger than "does not crash" (the round trip through format_duration). skill-fired and read-property read the tool calls; uses-project-library, no-separate-fuzzer and states-round-trip read tests/test_duration_fuzz.py. Needs --scaffold --allow-tools Write Edit.'tags: [testing, write, property, fuzz, project-library, scaffold, no-network]expected_outcome: 'Reads context/property.md; tests/test_duration_fuzz.py imports hypothesis, not atheris, and checks that parse_duration reads format_duration''s output back to the same seconds.'context:
scaffold_script: scaffold.shexecution:
prompt: 'Write a fuzz test for parse_duration in src/duration.py and put it in tests/test_duration_fuzz.py. Dependencies are not installed in this checkout, so write the file and do not run it. Use the `testing:write` skill if you have it.'allowed_tools: [Read, Glob, Grep, Skill, Write, Edit]max_turns: 30timeout_seconds: 600graders:
- name: skill-firedtype: tool_usedtool: Skillinput_match: '"skill"\s*:\s*"(?:testing:)?write"'
- name: read-propertytype: tool_usedtool: Readinput_match: '"file_path"\s*:\s*"[^"]*skills/write/context/property\.md"'
- name: uses-project-librarytype: regexpattern: '^\s*(?:from\s+hypothesis\b|import\s+hypothesis\b)'flags: mtarget: { source: file, path: tests/test_duration_fuzz.py }
- name: no-separate-fuzzertype: regexpattern: '\batheris\b|\bpythonfuzz\b'match: not_containstarget: { source: file, path: tests/test_duration_fuzz.py }
- name: states-round-triptype: regexpattern: '\bparse_duration\(\s*format_duration\(|=\s*format_duration\([\s\S]*?\bparse_duration\('target: { source: file, path: tests/test_duration_fuzz.py }
wr-glossary-absent
prompt.md:
This is the catalogue code for our tool library: `Library` in `tool_library.py`, with its tests in
`tests/test_tool_library.py`. Run the tests with `python3 -m unittest discover -s tests -t .`. Use
the `testing:write` skill if you have it.
Members want to check tools out and bring them back. Add that to `Library`, tests first, with the
tests in `tests/test_tool_library.py`:
1. Checking out an available tool for a member records who has it and a due date 14 days after the
day it went out.
2. A tool that is already out cannot go out again: raise `ValueError`.
3. Bringing a tool back makes it available again.
4. An unknown card number cannot take a tool out: raise `KeyError`.
The work is done when each point has a test and the whole suite passes.
case.yaml:
schema_version: "1.1"name: wr-glossary-absentcontext:
scaffold_script: scaffold.shdescription: "Glossary reader, control. The same workspace and prompt as wr-glossary-root with no glossary anywhere, so canonical-term and no-rejected-term record what the run names the feature on the user's words alone. It has no glossary-read grader: there is no file to read. Read its pass rates beside wr-glossary-root's: the cross-case gap is the measurement, since the CLI's delta ablates the plugin, not the glossary. Expected to fail canonical-term and no-rejected-term in most runs, because the request says 'check out' throughout and nothing in the workspace says Loan; a high pass rate here means the suite has no headroom to show a glossary's effect. Source: the maintainers' own writing, an owner-approved measurement before the glossary capability is restructured. Routine: a control, not a difficulty probe. Needs --scaffold --allow-tools Bash Edit Write."tags: [testing, write, glossary, control, routine]runs: 3execution:
max_turns: 60timeout_seconds: 900allowed_tools: [Read, Glob, Grep, Bash, Edit, Write, Skill]expected_outcome: "Adds the feature to Library with unittest tests in tests/test_tool_library.py and runs the suite until it passes. With no glossary, naming it after the request (checkout) is the likely result, and the two term graders record how often it happens"graders:
- name: skill-firedtype: tool_usedtool: Skillinput_match: '"skill"\s*:\s*"(?:testing:)?write"'
- name: canonical-termtype: regextarget:
source: filepath: tests/test_tool_library.pyflags: ipattern: 'loan'
- name: no-rejected-termtype: regextarget:
source: filepath: tests/test_tool_library.pyflags: imatch: not_containspattern: 'check(?:ed|s|ing)?[ _-]?outs?|rental|booking'
wr-glossary-docs-path
prompt.md:
This is the catalogue code for our tool library: `Library` in `tool_library.py`, with its tests in
`tests/test_tool_library.py`. Run the tests with `python3 -m unittest discover -s tests -t .`. Use
the `testing:write` skill if you have it.
Members want to check tools out and bring them back. Add that to `Library`, tests first, with the
tests in `tests/test_tool_library.py`:
1. Checking out an available tool for a member records who has it and a due date 14 days after the
day it went out.
2. A tool that is already out cannot go out again: raise `ValueError`.
3. Bringing a tool back makes it available again.
4. An unknown card number cannot take a tool out: raise `KeyError`.
The work is done when each point has a test and the whole suite passes.
case.yaml:
schema_version: "1.1"name: wr-glossary-docs-pathcontext:
scaffold_script: scaffold.shdescription: "Glossary reader, docs placement. The same workspace and prompt as wr-glossary-root with the glossary at docs/glossary.md instead of a root GLOSSARY.md. Graded on whether the run finds and reads the glossary at docs/glossary.md, a sibling of tool_library.py rather than an ancestor on its walk-up path, and the tests use Loan and none of its rejected synonyms. Source: the maintainers' own writing, an owner-approved measurement before the glossary capability is restructured. Hard: the glossary is off the documented read path and the request says 'check out' throughout. Needs --scaffold --allow-tools Bash Edit Write."tags: [testing, write, glossary, hard]runs: 3execution:
max_turns: 60timeout_seconds: 900allowed_tools: [Read, Glob, Grep, Bash, Edit, Write, Skill]expected_outcome: "Reads docs/glossary.md, then adds the feature to Library under the glossary's term, such as a loan with a due date, with unittest tests in tests/test_tool_library.py that use Loan and none of checkout, rental or booking, and runs the suite until it passes"graders:
- name: skill-firedtype: tool_usedtool: Skillinput_match: '"skill"\s*:\s*"(?:testing:)?write"'
- name: canonical-termtype: regextarget:
source: filepath: tests/test_tool_library.pyflags: ipattern: 'loan'
- name: no-rejected-termtype: regextarget:
source: filepath: tests/test_tool_library.pyflags: imatch: not_containspattern: 'check(?:ed|s|ing)?[ _-]?outs?|rental|booking'
- name: glossary-readtype: regextarget: tracepattern: '"name":"Read","input":\{"file_path":"[^"]*docs/glossary\.md"|"name":"Bash","input":\{"command":"(?=(?:[^"\\]|\\.)*\b(?:cat|sed|head|tail|less|grep)\b)(?:[^"\\]|\\.)*docs/(?:glossary(?:\.md|\*)|\*+(?:\.md)?)'
wr-glossary-root
prompt.md:
This is the catalogue code for our tool library: `Library` in `tool_library.py`, with its tests in
`tests/test_tool_library.py`. Run the tests with `python3 -m unittest discover -s tests -t .`. Use
the `testing:write` skill if you have it.
Members want to check tools out and bring them back. Add that to `Library`, tests first, with the
tests in `tests/test_tool_library.py`:
1. Checking out an available tool for a member records who has it and a due date 14 days after the
day it went out.
2. A tool that is already out cannot go out again: raise `ValueError`.
3. Bringing a tool back makes it available again.
4. An unknown card number cannot take a tool out: raise `KeyError`.
The work is done when each point has a test and the whole suite passes.
case.yaml:
schema_version: "1.1"name: wr-glossary-rootcontext:
scaffold_script: scaffold.shdescription: "Glossary reader, root placement. The workspace holds a project glossary at the root, GLOSSARY.md, the place testing:write's walk-up rule finds (skills/write/context/write.md, Sequence step 1). Graded on whether the new tests name the feature with the glossary's canonical term (Loan) instead of its rejected synonyms (checkout, rental, booking), and whether the run read the glossary at all. Compare its pass rates with wr-glossary-absent, the same prompt with no glossary: that cross-case gap, not the CLI's with/without-plugin delta, answers whether a glossary helps a skill that reads it, because the CLI ablates the plugin and never the glossary. Source: the maintainers' own writing, an owner-approved measurement before the glossary capability is restructured. Hard: the request itself says 'check out' throughout, so the canonical name has to come from the glossary over the user's own words. Needs --scaffold --allow-tools Bash Edit Write."tags: [testing, write, glossary, hard]runs: 3execution:
max_turns: 60timeout_seconds: 900allowed_tools: [Read, Glob, Grep, Bash, Edit, Write, Skill]expected_outcome: "Reads GLOSSARY.md, then adds the feature to Library under the glossary's term, such as a loan with a due date, with unittest tests in tests/test_tool_library.py that use Loan and none of checkout, rental or booking, and runs the suite until it passes"graders:
- name: skill-firedtype: tool_usedtool: Skillinput_match: '"skill"\s*:\s*"(?:testing:)?write"'
- name: canonical-termtype: regextarget:
source: filepath: tests/test_tool_library.pyflags: ipattern: 'loan'
- name: no-rejected-termtype: regextarget:
source: filepath: tests/test_tool_library.pyflags: imatch: not_containspattern: 'check(?:ed|s|ing)?[ _-]?outs?|rental|booking'
- name: glossary-readtype: regextarget: tracepattern: '"name":"Read","input":\{"file_path":"[^"]*GLOSSARY\.md"|"name":"Bash","input":\{"command":"(?=(?:[^"\\]|\\.)*\b(?:cat|sed|head|tail|less|grep)\b)(?:[^"\\]|\\.)*(?:GLOSSARY\.md|\s(?:\./)?\*(?:\.md)?(?=[\s"\\;|&)]))'
wr-go-uses-project-library
case.yaml:
schema_version: "1.1"name: wr-go-uses-project-librarydescription: 'Legacy write 7. A property test for a Go Merge function uses the guidance in context/property.md (commutativity, generators that build valid interval lists) and the property library the module already requires, rapid, rather than one the skill picks. skill-fired and read-property read the tool calls, uses-rapid the go test file; meets-expectations (llm) quotes the legacy expectations. Needs --scaffold --allow-tools Bash Write Edit.'tags: [testing, write, property, project-library, scaffold, no-network]expected_outcome: 'Reads context/property.md; intervals/merge_test.go imports pgregory.net/rapid and checks Merge(a, b) against Merge(b, a).'context:
scaffold_script: scaffold.shexecution:
prompt: 'Our Go module has a Merge function for interval lists (intervals/merge.go) that should be commutative. Write a property-based test for it in intervals/merge_test.go. Use the `testing:write` skill if you have it.'allowed_tools: [Read, Glob, Grep, Skill, Bash, Write, Edit]max_turns: 30timeout_seconds: 600graders:
- name: skill-firedtype: tool_usedtool: Skillinput_match: '"skill"\s*:\s*"(?:testing:)?write"'
- name: read-propertytype: tool_usedtool: Readinput_match: '"file_path"\s*:\s*"[^"]*skills/write/context/property\.md"'
- name: uses-rapidtype: regexpattern: '"pgregory\.net/rapid"'target: { source: file, path: intervals/merge_test.go }
- name: meets-expectationstype: llmcriteria: 'The user asked for a property-based test of a Go Merge function for interval lists that should be commutative; go.mod already requires pgregory.net/rapid. PASS only if the run meets every legacy expectation: "Uses the guidance in context/property.md" "Writes the property with the library go.mod already requires" "States commutativity (merging a with b equals merging b with a) as the property". FAIL if any point is missing or contradicted.'
wr-handler-unit-tests
case.yaml:
schema_version: "1.1"name: wr-handler-unit-testsdescription: 'Legacy write 1. A new create-template-item command handler in a Python repository whose docs/testing.md co-locates unit tests beside the code and sets a test naming pattern. The skill routes to context/write.md, treats the handler as application logic needing unit tests, lists happy, validation and edge cases, and writes the tests at the documented path with the documented names and plain pytest asserts. skill-fired and read-write-md read the tool calls; colocated-file, documented-names, plain-asserts and no-tests-dir read the files; meets-expected-output (llm) quotes the legacy expected output. Needs --scaffold --allow-tools Bash Write Edit.'tags: [testing, write, placement, naming, scaffold, no-network]expected_outcome: 'src/templates/create_template_item_test.py holds at least three tests named test_<action>_<outcome>_when_<condition>, using plain assert; no tests/ folder is created.'context:
scaffold_script: scaffold.shexecution:
prompt: 'I just wrote a new command handler for creating template items, src/templates/create_template_item.py. Write tests for it.'allowed_tools: [Read, Glob, Grep, Skill, Bash, Write, Edit]max_turns: 30timeout_seconds: 600graders:
- name: skill-firedtype: tool_usedtool: Skillinput_match: '"skill"\s*:\s*"(?:testing:)?write"'
- name: read-write-mdtype: tool_usedtool: Readinput_match: '"file_path"\s*:\s*"[^"]*skills/write/context/write\.md"'
- name: colocated-filetype: file_existspath: src/templates/create_template_item_test.py
- name: documented-namestype: regexpattern: '(?:[\s\S]*?\bdef test_[a-z0-9]+(?:_[a-z0-9]+)*_when_[a-z0-9_]+\s*\(){3}'target: { source: file, path: src/templates/create_template_item_test.py }
- name: plain-assertstype: regexpattern: '\bimport unittest\b|\bunittest\.TestCase\b|\bself\.assert[A-Z]'match: not_containstarget: { source: file, path: src/templates/create_template_item_test.py }
- name: no-tests-dirtype: file_existspath: tests/**/*.pyexists: false
- name: meets-expected-outputtype: llmcriteria: 'The user asked for tests of a new create-template-item command handler in a Python repository whose docs/testing.md sets pytest with plain assert, co-located unit tests and a naming pattern. PASS only if the run matches the legacy expected output: "Routes to context/write.md. Identifies the handler as application-layer logic needing unit tests. Follows TDD: generates a test list (happy path, validation failures, edge cases), names tests using the consuming project''s documented naming pattern (mirroring the ecosystem''s own idiom when undocumented), and places them in the project''s documented co-located test location. Defers to the consuming project''s documented framework and assertion conventions rather than assuming any specific stack." FAIL if any part is missing or contradicted.'
wr-hollow-out-receipt
case.yaml:
schema_version: "1.1"name: wr-hollow-out-receiptdescription: 'Legacy write 8. The per-cycle checklist run on a buildReceipt test whose only assertion reads the order the test built: the hollow-out item fails, since a buildReceipt that returns nothing still passes, and the fix captures the receipt buildReceipt returns and asserts on its lines. skill-fired reads the Skill calls, asserts-on-receipt the reply; meets-expectations (llm) quotes the legacy expectations. Read-only tools, graded on the reply.'tags: [testing, write, per-cycle-checklist, hollow-out, no-network]expected_outcome: 'Says the test passes when buildReceipt returns nothing because it asserts on the order it built, and rewrites it to assert on the receipt buildReceipt returns.'execution:
prompt: "I just went green on this cycle for buildReceipt. Run the per-cycle checklist on the test before I refactor:\n\nimport { buildReceipt } from './receipt';\n\ntest('a receipt lists every order line', () => {\n const order = { id: 'A17', lines: [{ sku: 'TEA', qty: 2 }, { sku: 'MUG', qty: 1 }] };\n buildReceipt(order);\n expect(order.lines).toHaveLength(2);\n});"allowed_tools: [Read, Glob, Grep, Skill]max_turns: 15timeout_seconds: 420graders:
- name: skill-firedtype: tool_usedtool: Skillinput_match: '"skill"\s*:\s*"(?:testing:)?write"'
- name: asserts-on-receipttype: regexpattern: '\b(?:const|let)\s+(\w+)\s*=\s*buildReceipt\(\s*order\s*\)[\s\S]{0,400}?\bexpect\(\s*\1\b|\bexpect\(\s*buildReceipt\(\s*order\s*\)'target: last_message
- name: meets-expectationstype: llmcriteria: 'The user asked for the per-cycle checklist on a buildReceipt test that calls buildReceipt(order) and then asserts expect(order.lines).toHaveLength(2). PASS only if the reply meets every legacy expectation: "Says the test keeps passing when buildReceipt returns nothing" "Notes that the assertion checks the order the test built rather than any receipt" "Suggests asserting on the receipt buildReceipt returns". FAIL if any point is missing or contradicted.'
wr-integration-shared-library
case.yaml:
schema_version: "1.1"name: wr-integration-shared-librarydescription: 'Legacy write 2. Where integration tests for a new shared pricing library go, in a monorepo whose docs/testing.md separates single-project from cross-project integration tests. The skill routes to context/organize.md, finds the library has testable behavior (branching discount rules), and lets the documented convention decide the location instead of pre-committing. skill-fired, read-organize and read-convention read the tool calls, names-documented-location the reply; meets-expected-output (llm) quotes the legacy expected output. Read-only tools; needs --scaffold.'tags: [testing, write, organize, placement, scaffold, no-network]expected_outcome: 'Reads context/organize.md and docs/testing.md, says pricing has logic worth its own tests, and names libs/pricing/tests/integration/ for library-only tests or tests/integration/pricing/ for tests that span the services, per the document.'context:
scaffold_script: scaffold.shexecution:
prompt: 'Where should I put integration tests for the new shared library in libs/pricing? Use the `testing:write` skill if you have it.'allowed_tools: [Read, Glob, Grep, Skill]max_turns: 15timeout_seconds: 420graders:
- name: skill-firedtype: tool_usedtool: Skillinput_match: '"skill"\s*:\s*"(?:testing:)?write"'
- name: read-organizetype: tool_usedtool: Readinput_match: '"file_path"\s*:\s*"[^"]*skills/write/context/organize\.md"'
- name: read-conventiontype: tool_usedtool: Readinput_match: '"file_path"\s*:\s*"[^"]*docs/testing\.md"'
- name: names-documented-locationtype: regexpattern: '\blibs/pricing/tests/integration\b|(?<![\w/])tests/integration/pricing\b'target: last_message
- name: meets-expected-outputtype: llmcriteria: 'The user asked where integration tests for a new shared pricing library go, in a monorepo whose docs/testing.md places single-project integration tests in the project''s tests/integration/ folder and cross-project ones under tests/integration/<area>/ at the root, and gives a contracts-only library no test folder. PASS only if the run matches the legacy expected output: "Routes to context/organize.md. Evaluates whether the library has testable behavior (business logic, conditional branching) vs pure contracts. If testable: a dedicated test project is warranted; its placement turns on whether the integration tests are single-project (co-locate with the library) or cross-project (centralize under the integration-test root): inspects or asks for the consuming project''s documented naming and location convention to decide, rather than pre-committing to either branch. If contracts only: no dedicated test project needed, transitive coverage acceptable." FAIL if any part is missing or contradicted.'
wr-light-skips-advisor-checkpoint
prompt.md:
---description: "New prorated-refund logic (money, HIGH risk) in a repository whose default branch sets verification: light in docs/conventions/testing.yaml: /testing:write writes the tests and skips the advisor checkpoint, reporting the skip token. Needs --scaffold --allow-tools Bash Write Edit."tags: [write, setting, verification-level, scaffold]runs: 3max_turns: 40timeout_seconds: 900allowed_tools: [Read, Glob, Grep, Skill, Bash, Write, Edit]expected_outcome: "Runs lib/read-verification.sh, reports verification: light, writes tests for refund_cents, calls no advisor, and reports verification: skipped (light)"---
/testing:write Write the tests for billing/refund.py's refund_cents. It is new money logic.
schema_version: "1.1"name: wr-no-skip-reddescription: 'Legacy write 3. The user knows the parse_price bug (thousands separators) and asks to skip the failing test and just patch it. The skill keeps Red: it writes the failing test in the documented tests/test_parser.py first, then the minimal fix. skill-fired reads the Skill calls, test-before-fix the order of the Edit calls, separator-test the test file; meets-expectations (llm) quotes the legacy expectations. Needs --scaffold --allow-tools Bash Write Edit.'tags: [testing, write, tdd, red-first, scaffold, no-network]expected_outcome: 'tests/test_parser.py gains a thousands-separator test before src/parser.py is edited; the reply refuses to skip the red step and names the documented test location and naming.'context:
scaffold_script: scaffold.shexecution:
prompt: 'Write tests for parse_price in src/parser.py, but I already know the bug (it chokes on thousands separators like "$1,200.50"), so skip the failing test and just patch it. Use the `testing:write` skill if you have it.'allowed_tools: [Read, Glob, Grep, Skill, Bash, Write, Edit]max_turns: 30timeout_seconds: 600graders:
- name: skill-firedtype: tool_usedtool: Skillinput_match: '"skill"\s*:\s*"(?:testing:)?write"'
- name: test-before-fixtype: tool_orderbefore: { tool: Edit, input_match: '"file_path":"[^"]*tests/test_parser\.py"' }after: { tool: Edit, input_match: '"file_path":"[^"]*src/parser\.py"' }
- name: separator-testtype: regexpattern: '\bdef test_parse_price_\w+\s*\([^)]*\)\s*:(?:(?!\bdef )[\s\S])*?\d,\d{3}'target: { source: file, path: tests/test_parser.py }
- name: meets-expectationstype: llmcriteria: 'The user asked for tests of parse_price but said they know the bug (thousands separators), so skip the failing test and just patch it; the repository''s docs/testing.md puts tests in tests/test_<module>.py named test_<function>_<behavior>. PASS only if the run meets every legacy expectation: "Writes or requires a failing test before changing production code" "Does not skip the red step because the fix seems obvious" "Names the consuming project''s test location and naming pattern rather than inventing a stack". FAIL if any point is missing or contradicted.'
wr-placement-from-conventions
prompt.md:
---description: "Placement question for a new mapper's tests in a repo whose docs/testing.md sets the layout and one old test sits outside it. The answer follows the documented convention, not the stray example. Hard: the nearest existing file points the wrong way."tags: [testing, write, organize, outcome, hard, source-skill-contract]max_turns: 20timeout_seconds: 600allowed_tools: [Read, Glob, Grep, Skill]expected_outcome: "Reads docs/testing.md and answers tests/unit/orders/test_order_mapper.py, because order_mapper is pure code with no database session; it writes no file."---
/testing:write organize where should the tests for src/orders/order_mapper.py go? Don't write them yet; give me the path and why.
schema_version: "1.1"name: wr-property-dependent-inputdescription: 'Dependent input. invoice_total accepts only invoices whose every line is in the invoice''s currency, so a property test draws the currency first and feeds it to the line generator (a Hypothesis @st.composite strategy, .flatmap, or st.data() draws), instead of drawing lines freely and throwing mixed-currency invoices away with assume() or .filter(). skill-fired reads the Skill calls; uses-given, builds-dependent-input and no-discarded-draws read tests/test_invoice.py. Needs --scaffold --allow-tools Write Edit.'tags: [testing, write, property, generators, dependent-input, scaffold, no-network]expected_outcome: 'tests/test_invoice.py holds a @given property whose invoice strategy draws the currency and builds every line in it (composite, flatmap or data draws), with no assume() or .filter().'context:
scaffold_script: scaffold.shexecution:
prompt: 'Write a property-based test for invoice_total in src/invoice.py and put it in tests/test_invoice.py. Dependencies are not installed in this checkout, so write the file and do not run it. Use the `testing:write` skill if you have it.'allowed_tools: [Read, Glob, Grep, Skill, Write, Edit]max_turns: 30timeout_seconds: 600graders:
- name: skill-firedtype: tool_usedtool: Skillinput_match: '"skill"\s*:\s*"(?:testing:)?write"'
- name: uses-giventype: regexpattern: '^\s*@(?:hypothesis\.)?given\('flags: mtarget: { source: file, path: tests/test_invoice.py }
- name: builds-dependent-inputtype: regexpattern: '^\s*@(?:st\.|strategies\.|hypothesis\.strategies\.)?composite\b|\.flatmap\(|\b(?:st|strategies)\.data\(\s*\)'flags: mtarget: { source: file, path: tests/test_invoice.py }
- name: no-discarded-drawstype: regexpattern: '\bassume\(|\.filter\('match: not_containstarget: { source: file, path: tests/test_invoice.py }
wr-property-past-bug
case.yaml:
schema_version: "1.1"name: wr-property-past-bugdescription: 'Past bug. slugify once returned an empty slug for the title "!!!" and broke the URL builder; it is fixed. A property test for slugify (never empty, only lowercase letters, digits and single hyphens) carries that reported input as a fixed Hypothesis @example on the property, so the regression runs before any generated input on every run. skill-fired reads the Skill calls; uses-given and pins-past-bug read tests/test_slug.py. Needs --scaffold --allow-tools Write Edit.'tags: [testing, write, property, regression, past-bug, scaffold, no-network]expected_outcome: 'tests/test_slug.py holds a @given property for slugify decorated with @example("!!!").'context:
scaffold_script: scaffold.shexecution:
prompt: 'Write a property-based test for slugify in src/slug.py and put it in tests/test_slug.py. Last month a user reported that slugify("!!!") returned an empty string, which broke the URL builder; that is fixed now. Dependencies are not installed in this checkout, so write the file and do not run it. Use the `testing:write` skill if you have it.'allowed_tools: [Read, Glob, Grep, Skill, Write, Edit]max_turns: 30timeout_seconds: 600graders:
- name: skill-firedtype: tool_usedtool: Skillinput_match: '"skill"\s*:\s*"(?:testing:)?write"'
- name: uses-giventype: regexpattern: '^\s*@(?:hypothesis\.)?given\('flags: mtarget: { source: file, path: tests/test_slug.py }
- name: pins-past-bugtype: regexpattern: '^\s*@(?:hypothesis\.)?example\(\s*(?:\w+\s*=\s*)?["'']!!!["'']'flags: mtarget: { source: file, path: tests/test_slug.py }
wr-property-reimplements-split
case.yaml:
schema_version: "1.1"name: wr-property-reimplements-splitdescription: 'Legacy write 5. A split_bill property whose expected side is [total_cents // people] * people repeats the algorithm (and drops the remainder), so the skill rejects it, points to testing:test-value for the independent-source rule, and replaces it with invariants: the shares sum to the total, there are `people` shares, and no two differ by more than a cent. skill-fired reads the Skill calls; points-to-test-value and sum-invariant read the reply; meets-expectations (llm) quotes the legacy expectations. Read-only tools, graded on the reply.'tags: [testing, write, property, oracle, no-network]expected_outcome: 'Says the oracle reruns the split, names testing:test-value, and proposes sum(shares) == total_cents plus the count and spread invariants.'execution:
prompt: 'Review my property test for split_bill(total_cents, people): for any total and head count it asserts the result equals [total_cents // people] * people. Is that a good property? Fix it. Use the `testing:write` skill if you have it.'allowed_tools: [Read, Glob, Grep, Skill]max_turns: 15timeout_seconds: 420graders:
- name: skill-firedtype: tool_usedtool: Skillinput_match: '"skill"\s*:\s*"(?:testing:)?write"'
- name: points-to-test-valuetype: regexpattern: '(?<![\w-])(?:/?testing:)?test-value\b'target: last_message
- name: sum-invarianttype: regexpattern: '\bsum\(\s*(\w+(?:\([^()]*\))?)\s*\)\s*==\s*total_cents\b|\btotal_cents\s*==\s*sum\(\s*\w+(?:\([^()]*\))?\s*\)'target: last_message
- name: meets-expectationstype: llmcriteria: 'The user asked whether a split_bill property asserting the result equals [total_cents // people] * people is good, and to fix it. PASS only if the reply meets every legacy expectation: "Says the expected side reruns the algorithm under test and cannot catch a mistake both share" "Points to testing:test-value for the independent-source rule" "Proposes the shares-sum-to-total check or another invariant that does not recompute the split". FAIL if any point is missing or contradicted.'
wr-python-idempotence-property
case.yaml:
schema_version: "1.1"name: wr-python-idempotence-propertydescription: 'Legacy write 6, graded on the test written. In a Python repository whose pyproject.toml lists hypothesis, a property test for normalize_tags (lowercases, deduplicates and sorts) goes in tests/test_tags.py as a Hypothesis @given property that states idempotence: a second pass changes nothing. skill-fired reads the Skill calls; uses-given and idempotence read tests/test_tags.py; meets-expectations (llm) reads the run. Needs --scaffold --allow-tools Bash Write Edit.'tags: [testing, write, property, idempotence, scaffold, no-network]expected_outcome: 'tests/test_tags.py holds a @given property asserting normalize_tags(normalize_tags(x)) == normalize_tags(x), or the same check through a saved first result.'context:
scaffold_script: scaffold.shexecution:
prompt: 'This repository is Python only: a pyproject.toml and pytest, no package.json and no .csproj. Write a property-based test for normalize_tags in src/tags.py, which lowercases, deduplicates and sorts a list of tags. Put it in tests/test_tags.py. Use the `testing:write` skill if you have it.'allowed_tools: [Read, Glob, Grep, Skill, Bash, Write, Edit]max_turns: 30timeout_seconds: 600graders:
- name: skill-firedtype: tool_usedtool: Skillinput_match: '"skill"\s*:\s*"(?:testing:)?write"'
- name: uses-giventype: regexpattern: '^\s*@(?:hypothesis\.)?given\('flags: mtarget: { source: file, path: tests/test_tags.py }
- name: idempotencetype: regexpattern: '\bnormalize_tags\(\s*normalize_tags\(\s*(\w+)\s*\)\s*\)\s*==\s*normalize_tags\(\s*\1\s*\)|\bnormalize_tags\(\s*(\w+)\s*\)\s*==\s*normalize_tags\(\s*normalize_tags\(\s*\2\s*\)\s*\)|\b(\w+)\s*=\s*normalize_tags\(\s*\w+\s*\)[\s\S]{0,400}?\bnormalize_tags\(\s*\3\s*\)\s*==\s*\3\b|\b(\w+)\s*=\s*normalize_tags\(\s*\w+\s*\)[\s\S]{0,400}?\b\4\s*==\s*normalize_tags\(\s*\4\s*\)'target: { source: file, path: tests/test_tags.py }
- name: meets-expectationstype: llmcriteria: 'The user asked for a property-based test of normalize_tags (lowercases, deduplicates and sorts) in tests/test_tags.py of a Python repository whose pyproject.toml lists hypothesis. PASS only if the test the run wrote meets every expectation: "It is a Hypothesis property (@given with a strategy for lists of tags)" "It asserts that a second normalize_tags pass leaves the result unchanged" "Its expected values do not repeat normalize_tags''s own lowercase, dedupe and sort steps". FAIL if any point is missing or contradicted.'
wr-round-trip-property
case.yaml:
schema_version: "1.1"name: wr-round-trip-propertydescription: 'Legacy write 4. format_duration and parse_duration are a reverse pair, so the skill routes to context/property.md and writes a round-trip property (parsing the formatted string returns the input) beside literal examples, without rebuilding the expected string from the formatter''s arithmetic. skill-fired and read-property read the tool calls; round-trip, literal-example and no-recomputed-expected read tests/test_durations.py; meets-expectations (llm) quotes the legacy expectations. Needs --scaffold --allow-tools Bash Write Edit.'tags: [testing, write, property, round-trip, scaffold, no-network]expected_outcome: 'Reads context/property.md; tests/test_durations.py holds a property asserting parse_duration(format_duration(n)) == n and at least one literal format example, and no divmod or // 3600 arithmetic.'context:
scaffold_script: scaffold.shexecution:
prompt: 'Our config loader has format_duration, which turns a number of seconds into strings like ''1h30m'', and parse_duration, which reads them back (src/durations.py). Add tests for both. Use the `testing:write` skill if you have it.'allowed_tools: [Read, Glob, Grep, Skill, Bash, Write, Edit]max_turns: 30timeout_seconds: 600graders:
- name: skill-firedtype: tool_usedtool: Skillinput_match: '"skill"\s*:\s*"(?:testing:)?write"'
- name: read-propertytype: tool_usedtool: Readinput_match: '"file_path"\s*:\s*"[^"]*skills/write/context/property\.md"'
- name: round-triptype: regexpattern: '\bparse_duration\(\s*format_duration\(\s*(\w+)\s*\)\s*\)\s*==\s*\1\b|\b(\w+)\s*==\s*parse_duration\(\s*format_duration\(\s*\2\s*\)\s*\)'target: trace
- name: literal-exampletype: regexpattern: '\bformat_duration\(\s*\d+\s*\)\s*==\s*["''](?:\d+[hms])+["'']|["''](?:\d+[hms])+["'']\s*==\s*format_duration\(\s*\d+\s*\)|\(\s*\d+\s*,\s*["''](?:\d+[hms])+["'']\s*\)'target: { source: file, path: tests/test_durations.py }
- name: no-recomputed-expectedtype: regexpattern: '\bdivmod\(|//\s*(?:3600|60)\b|%\s*(?:3600|60)\b'match: not_containstarget: { source: file, path: tests/test_durations.py }
- name: meets-expectationstype: llmcriteria: 'The user asked for tests of format_duration (seconds to strings like ''1h30m'') and parse_duration, which reads them back. PASS only if the run meets every legacy expectation: "Reads context/property.md before writing the tests" "Uses the round trip (parse of format equals the input) as the property''s oracle" "Does not build the expected string inside the test by repeating the formatter''s arithmetic". FAIL if any point is missing or contradicted.'
wr-ts-async-property-marked
case.yaml:
schema_version: "1.1"name: wr-ts-async-property-markeddescription: 'TypeScript, no recomputed-derived-value finding. encodeRecord and decodeRecord are async, so the fast-check property is an fc.asyncProperty inside an awaited (or returned) fc.assert: a property whose expected value comes from generated input carries the js-jest adapter''s property marker, which /testing:audit''s rule-recomputed-derived skips, and an assert that is not awaited never reports its failure. skill-fired reads the Skill calls; property-marker and awaits-async-property read src/codec.test.ts. Needs --scaffold --allow-tools Write Edit.'tags: [testing, write, property, typescript, fast-check, audit-marker, scaffold, no-network]expected_outcome: 'src/codec.test.ts holds await fc.assert(fc.asyncProperty(...)) (or return fc.assert(...)) checking that decodeRecord(await encodeRecord(r)) equals r, so the audit''s recomputed-derived rule reports nothing on it.'context:
scaffold_script: scaffold.shexecution:
prompt: 'Write a property-based test for encodeRecord and decodeRecord in src/codec.ts and put it in src/codec.test.ts. We use vitest and fast-check (see package.json). Dependencies are not installed in this checkout, so write the file and do not run it. Use the `testing:write` skill if you have it.'allowed_tools: [Read, Glob, Grep, Skill, Write, Edit]max_turns: 30timeout_seconds: 600graders:
- name: skill-firedtype: tool_usedtool: Skillinput_match: '"skill"\s*:\s*"(?:testing:)?write"'
- name: property-markertype: regexpattern: 'fc\s*\.\s*(?:assert|property|asyncProperty)\s*\('target: { source: file, path: src/codec.test.ts }
- name: awaits-async-propertytype: regexpattern: '\b(?:await|return)\s+fc\s*\.\s*assert\s*\(\s*fc\s*\.\s*asyncProperty\s*\('target: { source: file, path: src/codec.test.ts }
write-first-unit
prompt.md:
---description: "A command-line feature with acceptance criteria, first_test: unit in docs/conventions/testing.yaml, and the testing:write skill named: the first test file written is a unit test of Cart, and the end-to-end command test still comes before done"tags: [testing, write, tdd, settings, first-test]runs: 3max_turns: 60timeout_seconds: 900allowed_tools: [Read, Glob, Grep, Bash, Edit, Write, Skill]expected_outcome: "Resolves first_test=unit from docs/conventions/testing.yaml, writes a unit test in tests/test_cart.py before touching tests/test_cli.py, adds the Cart method and the command, writes the end-to-end test in tests/test_cli.py, runs the suite until it passes, and ends with a Settings line naming that file"---
This repository has a small `Cart` class in `cart.py` with unit tests in `tests/test_cart.py`, and
a `shop.py` command line that has no commands yet. Run the tests with
`python3 -m unittest discover -s tests -t .`. Use the `testing:write` skill if you have it.
Add a `total` command: `python3 shop.py total <cart-file>` reads a cart file holding one line per
item, `sku,unit_price_cents,qty`, and prints the order total in cents. Compute the total in a new
`Cart.total_cents()` method.
Acceptance criteria:
1. An empty cart file prints `0`.
2. The total is the sum of `unit_price_cents * qty` over every line.
3. A line with a quantity of 10 or more gets 10% off that line, rounded half up to a whole cent
(15 units at 3 cents total 41, not 40).
4. A malformed line prints an error naming its line number to stderr and exits with status 1.
Put the end-to-end test that runs the command in `tests/test_cli.py` and the unit tests of `Cart`
in `tests/test_cart.py`. The work is done when every criterion has a test and the whole suite
passes.
---description: "A command-line feature with acceptance criteria, first_test: user-flow in docs/conventions/testing.yaml, and the testing:write skill named: the first test file written is the end-to-end command test, before any unit test"tags: [testing, write, tdd, settings, first-test]runs: 3max_turns: 60timeout_seconds: 900allowed_tools: [Read, Glob, Grep, Bash, Edit, Write, Skill]expected_outcome: "Resolves first_test=user-flow from docs/conventions/testing.yaml, writes tests/test_cli.py, which runs the total command, before touching tests/test_cart.py, adds the command and the Cart method, runs the suite until it passes, and ends with a Settings line naming that file"---
This repository has a small `Cart` class in `cart.py` with unit tests in `tests/test_cart.py`, and
a `shop.py` command line that has no commands yet. Run the tests with
`python3 -m unittest discover -s tests -t .`. Use the `testing:write` skill if you have it.
Add a `total` command: `python3 shop.py total <cart-file>` reads a cart file holding one line per
item, `sku,unit_price_cents,qty`, and prints the order total in cents. Compute the total in a new
`Cart.total_cents()` method.
Acceptance criteria:
1. An empty cart file prints `0`.
2. The total is the sum of `unit_price_cents * qty` over every line.
3. A line with a quantity of 10 or more gets 10% off that line, rounded half up to a whole cent
(15 units at 3 cents total 41, not 40).
4. A malformed line prints an error naming its line number to stderr and exits with status 1.
Put the end-to-end test that runs the command in `tests/test_cli.py` and the unit tests of `Cart`
in `tests/test_cart.py`. The work is done when every criterion has a test and the whole suite
passes.
---description: "The k36-cart-total task with no testing config layer and the testing:write skill named: tdd_refactor_step resolves to its default, on, and the skill reports a Refactor checkpoint line after green, so a skipped Refactor step shows in the reply"tags: [testing, write, tdd, settings, refactor-checkpoint]runs: 3max_turns: 60timeout_seconds: 900allowed_tools: [Read, Glob, Grep, Bash, Edit, Write, Skill]expected_outcome: "Resolves tdd_refactor_step=on from the default, adds total_cents and apply_coupon with unittest tests that check each criterion, runs the suite until it passes, and ends with a Settings line and at least one line beginning 'Refactor checkpoint:' that names what was tidied or why nothing needed it"---
This repository has a small `Cart` class in `cart.py` and its unit tests in `tests/test_cart.py`.
Run the tests with `python3 -m unittest discover -s tests -t .`. Use the `testing:write` skill if
you have it.
Add two methods to `Cart`, with unit tests in `tests/test_cart.py`:
-`total_cents()` returns the order total in cents.
-`apply_coupon(code)` applies a coupon code to the cart.
Acceptance criteria:
1. An empty cart totals 0.
2. The total is the sum of `unit_price_cents * qty` over every line.
3. A line with a quantity of 10 or more gets 10% off that line, rounded half up to a whole cent
(15 units at 3 cents total 41, not 40).
4. The coupon `SAVE5` takes 500 cents off a total of 5000 cents or more, after line discounts, and
does nothing below 5000.
5. Any other coupon code raises `ValueError`, and the cart is unchanged.
The work is done when every criterion has a test and the whole suite passes.
---description: "The k36-cart-total task with tdd_refactor_step: off in docs/conventions/testing.yaml and the testing:write skill named: the skill resolves the setting, names the team file as its source, and writes no Refactor checkpoint line"tags: [testing, write, tdd, settings, refactor-checkpoint]runs: 3max_turns: 60timeout_seconds: 900allowed_tools: [Read, Glob, Grep, Bash, Edit, Write, Skill]expected_outcome: "Resolves tdd_refactor_step=off from docs/conventions/testing.yaml, adds total_cents and apply_coupon with unittest tests that check each criterion, runs the suite until it passes, and ends with a Settings line naming that file and no line beginning 'Refactor checkpoint:'"---
This repository has a small `Cart` class in `cart.py` and its unit tests in `tests/test_cart.py`.
Run the tests with `python3 -m unittest discover -s tests -t .`. Use the `testing:write` skill if
you have it.
Add two methods to `Cart`, with unit tests in `tests/test_cart.py`:
-`total_cents()` returns the order total in cents.
-`apply_coupon(code)` applies a coupon code to the cart.
Acceptance criteria:
1. An empty cart totals 0.
2. The total is the sum of `unit_price_cents * qty` over every line.
3. A line with a quantity of 10 or more gets 10% off that line, rounded half up to a whole cent
(15 units at 3 cents total 41, not 40).
4. The coupon `SAVE5` takes 500 cents off a total of 5000 cents or more, after line discounts, and
does nothing below 5000.
5. Any other coupon code raises `ValueError`, and the cart is unchanged.
The work is done when every criterion has a test and the whole suite passes.
---description: "The k36-cart-total task with tdd_refactor_step: on in docs/conventions/testing.yaml and the testing:write skill named: the skill resolves the setting, names the team file as its source, and reports a Refactor checkpoint line after green"tags: [testing, write, tdd, settings, refactor-checkpoint]runs: 3max_turns: 60timeout_seconds: 900allowed_tools: [Read, Glob, Grep, Bash, Edit, Write, Skill]expected_outcome: "Resolves tdd_refactor_step=on from docs/conventions/testing.yaml, adds total_cents and apply_coupon with unittest tests that check each criterion, runs the suite until it passes, and ends with a Settings line naming that file and at least one line beginning 'Refactor checkpoint:'"---
This repository has a small `Cart` class in `cart.py` and its unit tests in `tests/test_cart.py`.
Run the tests with `python3 -m unittest discover -s tests -t .`. Use the `testing:write` skill if
you have it.
Add two methods to `Cart`, with unit tests in `tests/test_cart.py`:
-`total_cents()` returns the order total in cents.
-`apply_coupon(code)` applies a coupon code to the cart.
Acceptance criteria:
1. An empty cart totals 0.
2. The total is the sum of `unit_price_cents * qty` over every line.
3. A line with a quantity of 10 or more gets 10% off that line, rounded half up to a whole cent
(15 units at 3 cents total 41, not 40).
4. The coupon `SAVE5` takes 500 cents off a total of 5000 cents or more, after line discounts, and
does nothing below 5000.
5. Any other coupon code raises `ValueError`, and the cart is unchanged.
The work is done when every criterion has a test and the whole suite passes.
Owner decision, 2026-11-18, typed in the owning session after Q78 was restated in full: "Go with recommendations." Option A: the pass rule the compare was registered and run under stands, and so does the record above.
The three points the registration had marked provisional, as he was shown them:
Margin: the issue names none. The rule used is per case: a loss is a mean score drop of 0.15 or more, or a grader passing in 2 or more fewer after runs. All 22 cases pass under it; the largest drops are 0.111 (wr-glossary-absent, 0.667 to 0.556) and 0.067 (wr-fuzz-follows-property, write-first-unit). The reading guide's 0.05 noise-report margin was the alternative and was not computed.
Model: Sonnet only, 3 runs a side; the issue names Opus and Sonnet. Opus runs only to confirm a loss, and there was none.
The new case has no target; its per-grader counts are in the table.
Eval case and compare: the new case wr-fuzz-follows-property shipped in docs(testing): say the property guidance covers every target and that fuzzing follows it #16464; the two-version compare of the skill's write cases plus the new case is recorded above (PASS, 22 cases, no loss on any). It ran on the sonnet alias only; the owner accepted the pass rule as run on 2026-11-18 (decision comment above).
Activity
github-actions commentedon Nov 13, 2026
Size: Small
Complexity: Low
Uncertainty: Known
Flags: none
Blocked by: none
Understood: add one or two sentences to the
/testing:writeproperty guidance saying it applies to every target (a running UI included) and that a fuzz test follows it. The sentences name no tool. The change also needs one new frozen eval case and a non-inferiority compare on the existing write cases, and the pull request body names the dogfooding step.Refs:and left this issue open. I could not tell from the issue what part of the done-when it covered. A maintainer should check that before work starts.Automated triage. A maintainer reviews it.
melodic-automation-lanes commentedon Nov 13, 2026
Not admitted to the issue lane: merged pull request #16464 names it with
Refs:after its newest decision or refinement comment, so its work may already be done: close the issue if it is, or post a comment whose first line is**Decision recorded.**saying what is left, which sends it back through intake. Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).melodic-automation-lanes commentedon Nov 15, 2026
{"v":1,"actor_id":337595936,"title_sha256":"a522e6a34f2e287f8ac73c60603a525bca8dfaa52aa4c35d400932f270c43b36","body_sha256":"2e81b8949b5679bf96df16984207b862439f7aad32d4f2058e3bb8b2fc874ea9"}
github-actions commentedon Nov 15, 2026
melodic-automation-lanes commentedon Nov 15, 2026
Pull request #16464 merged with
Refs:, so this issue stays open and nothing starts the lane on it again. work-unblock removedstatus: readyandagent-ready, so the hourly triage sweep sends it back to intake triage. Admission then parks it atstatus: confirmedfor a check of what that pull request left, until a**Decision recorded.**comment says what is left or someone closes it.github-actions commentedon Nov 16, 2026
The triage sweep dispatched intake triage for this issue (dispatch 1; 2 runs that reach intake, or 6 dispatches, hand it to a maintainer).
kyle-sexton commentedon Nov 18, 2026
Targeted eval result: PASS
One targeted case per changed skill, run on the old text (before) and the new text (after), on
sonnetat 3 runs a side, Claude Code 2.1.327 (Claude Code).What this gate checks: that the skill's existing write cases and the new case do no worse on the new text than on the old (the issue's non-inferiority compare). The issue states no margin, so the rule registered before any run is the per-case no-loss rule below (PROVISIONAL until the owner confirms it). There is no wrong-text side, so a PASS says these cases did not get worse, not that they would have noticed a bad edit.
Scope: not a cut. PR 16464 adds two sentences to context/property.md (the guidance covers every target, a running UI included, and a fuzz test follows it) and widens the SKILL.md Step 0 route row to fuzz tests and any target. Measured: all 22 cases whose names start wr at the merge commit (21 existing write cases plus the new wr-fuzz-follows-property), non-inferiority; the SKILL.md row is loaded with the skill, and the property.md row counts as measured only if opened in at least 2 of 3 runs a side. Unmeasured: the running-UI clause (no case has a running-UI fixture, as the PR body says).
Cases: every plugin-eval case whose name starts wr at the merge commit (wr-* and write-*, 22), copied byte-identical under their own names; wr-parallel-worker-isolation, added to main after the merge, is not in the set.
No target is registered for the new case: the issue says it 'covers the sentences', not that it must show a change; its per-grader before and after counts are in the table.
Opus not run. The issue names the opus and sonnet aliases; the evals plugin's reading guide says a skill-text change is compared on Sonnet and Opus runs only to confirm a loss. The issue's two-alias clause is not met by this record.
Result rule PROVISIONAL: the issue states no margin. Registered: the per-case no-loss rule (a case's mean score falls by 0.15 or more, or any grader passes in 2 or more fewer after runs). Not registered: the reading guide's few-cases rule (noise-report margin 0.05; at about 20 cases or more a case-drop line or a not-shown-non-inferior verdict is a LOSS reading).
'Frozen first' is not met: the new case and the text change landed in one commit, as the PR body says.
Unmeasured rows: the running-UI clause of property.md.
Before:
8a544e1bd53f57302bd306803a68ea4ef3777cad(the merge commit's first parent)After:
109b87a8f2b44bc108ab26e2796a13615e365100(the merge commit onmain; the pull request merged before this eval, merge commit read when this was written:109b87a8f2b44bc108ab26e2796a13615e365100)Loss: the mean case score falls by 0.15 or more, or any one grader passes in 2 or more fewer after runs than before runs.
Passing runs per grader, per side:
Graders that cannot show a loss, because they failed on the old text in every run:
wr-glossary-absentno-rejected-term: cannot show a loss: failed on the old text in every run (0/3)wr-handler-unit-testsmeets-expected-output: cannot show a loss: failed on the old text in every run (0/3)wr-integration-shared-librarymeets-expected-output: cannot show a loss: failed on the old text in every run (0/3)wr-no-skip-redtest-before-fix: cannot show a loss: failed on the old text in every run (0/3)wr-no-skip-redmeets-expectations: cannot show a loss: failed on the old text in every run (0/3)wr-property-reimplements-splitpoints-to-test-value: cannot show a loss: failed on the old text in every run (0/3)wr-property-reimplements-splitmeets-expectations: cannot show a loss: failed on the old text in every run (0/3)wr-round-trip-propertymeets-expectations: cannot show a loss: failed on the old text in every run (0/3)Changed files the compare runs opened, per case and side:
wr-fuzz-follows-propertybefore: openedskills/write/context/property.mdin 3 of 3 runswr-fuzz-follows-propertyafter: openedskills/write/context/property.mdin 3 of 3 runswr-glossary-absentbefore: opened no changed file in 3 runswr-glossary-absentafter: opened no changed file in 3 runswr-glossary-docs-pathbefore: opened no changed file in 3 runswr-glossary-docs-pathafter: opened no changed file in 3 runswr-glossary-rootbefore: opened no changed file in 3 runswr-glossary-rootafter: opened no changed file in 3 runswr-go-uses-project-librarybefore: openedskills/write/context/property.mdin 3 of 3 runswr-go-uses-project-libraryafter: openedskills/write/context/property.mdin 3 of 3 runswr-handler-unit-testsbefore: opened no changed file in 3 runswr-handler-unit-testsafter: opened no changed file in 3 runswr-hollow-out-receiptbefore: opened no changed file in 3 runswr-hollow-out-receiptafter: opened no changed file in 3 runswr-integration-shared-librarybefore: opened no changed file in 3 runswr-integration-shared-libraryafter: opened no changed file in 3 runswr-light-skips-advisor-checkpointbefore: opened no changed file in 3 runswr-light-skips-advisor-checkpointafter: opened no changed file in 3 runswr-no-skip-redbefore: opened no changed file in 3 runswr-no-skip-redafter: opened no changed file in 3 runswr-placement-from-conventionsbefore: opened no changed file in 3 runswr-placement-from-conventionsafter: opened no changed file in 3 runswr-property-dependent-inputbefore: openedskills/write/context/property.mdin 3 of 3 runswr-property-dependent-inputafter: openedskills/write/context/property.mdin 3 of 3 runswr-property-past-bugbefore: openedskills/write/context/property.mdin 3 of 3 runswr-property-past-bugafter: openedskills/write/context/property.mdin 3 of 3 runswr-property-reimplements-splitbefore: openedskills/write/context/property.mdin 2 of 3 runswr-property-reimplements-splitafter: openedskills/write/context/property.mdin 2 of 3 runswr-python-idempotence-propertybefore: openedskills/write/context/property.mdin 3 of 3 runswr-python-idempotence-propertyafter: openedskills/write/context/property.mdin 3 of 3 runswr-round-trip-propertybefore: openedskills/write/context/property.mdin 3 of 3 runswr-round-trip-propertyafter: openedskills/write/context/property.mdin 3 of 3 runswr-ts-async-property-markedbefore: openedskills/write/context/property.mdin 3 of 3 runswr-ts-async-property-markedafter: openedskills/write/context/property.mdin 3 of 3 runswrite-first-unitbefore: opened no changed file in 3 runswrite-first-unitafter: opened no changed file in 3 runswrite-first-user-flowbefore: opened no changed file in 3 runswrite-first-user-flowafter: opened no changed file in 3 runswrite-refactor-checkpointbefore: opened no changed file in 3 runswrite-refactor-checkpointafter: opened no changed file in 3 runswrite-refactor-offbefore: opened no changed file in 3 runswrite-refactor-offafter: opened no changed file in 3 runswrite-refactor-onbefore: opened no changed file in 3 runswrite-refactor-onafter: opened no changed file in 3 runsRows these runs measured (the row's file opened in at least 2 of 3 runs on each side, or loaded with the skill or agent):
skills/write/SKILL.md, casewr-fuzz-follows-property): measured; the file is loaded with the skill or agent, never opened, so no open count appliesskills/write/context/property.md, casewr-fuzz-follows-property): opened in 3 of 3 before, 3 of 3 after runsskills/write/context/property.md, casewr-go-uses-project-library): opened in 3 of 3 before, 3 of 3 after runsskills/write/context/property.md, casewr-property-dependent-input): opened in 3 of 3 before, 3 of 3 after runsskills/write/context/property.md, casewr-property-past-bug): opened in 3 of 3 before, 3 of 3 after runsskills/write/context/property.md, casewr-property-reimplements-split): opened in 2 of 3 before, 2 of 3 after runsskills/write/context/property.md, casewr-python-idempotence-property): opened in 3 of 3 before, 3 of 3 after runsskills/write/context/property.md, casewr-round-trip-property): opened in 3 of 3 before, 3 of 3 after runsskills/write/context/property.md, casewr-ts-async-property-marked): opened in 3 of 3 before, 3 of 3 after runsRows these runs did not measure:
Verdicts:
Cost: 31.45 USD (smoke and compare, the runs'
costUsd).The cases live outside the repository; their full text:
The 22 cases: prompt.md and case.yaml
wr-fuzz-follows-propertycase.yaml:wr-glossary-absentprompt.md:case.yaml:wr-glossary-docs-pathprompt.md:case.yaml:wr-glossary-rootprompt.md:case.yaml:wr-go-uses-project-librarycase.yaml:wr-handler-unit-testscase.yaml:wr-hollow-out-receiptcase.yaml:wr-integration-shared-librarycase.yaml:wr-light-skips-advisor-checkpointprompt.md:case.yaml:wr-no-skip-redcase.yaml:wr-placement-from-conventionsprompt.md:case.yaml:wr-property-dependent-inputcase.yaml:wr-property-past-bugcase.yaml:wr-property-reimplements-splitcase.yaml:wr-python-idempotence-propertycase.yaml:wr-round-trip-propertycase.yaml:wr-ts-async-property-markedcase.yaml:write-first-unitprompt.md:case.yaml:write-first-user-flowprompt.md:case.yaml:write-refactor-checkpointprompt.md:case.yaml:write-refactor-offprompt.md:case.yaml:write-refactor-onprompt.md:case.yaml:Posted by an agent session.
kyle-sexton commentedon Nov 18, 2026
Owner decision, 2026-11-18, typed in the owning session after Q78 was restated in full: "Go with recommendations." Option A: the pass rule the compare was registered and run under stands, and so does the record above.
The three points the registration had marked provisional, as he was shown them:
wr-glossary-absent, 0.667 to 0.556) and 0.067 (wr-fuzz-follows-property,write-first-unit). The reading guide's 0.05 noise-report margin was the alternative and was not computed.kyle-sexton commentedon Nov 19, 2026
Closing: each done-when item is met.
plugins/testing/skills/write/context/property.mdon main says the guidance covers every target, a running UI included, and that a fuzz test follows it; it names no tool. Delivered by docs(testing): say the property guidance covers every target and that fuzzing follows it #16464.wr-fuzz-follows-propertyshipped in docs(testing): say the property guidance covers every target and that fuzzing follows it #16464; the two-version compare of the skill's write cases plus the new case is recorded above (PASS, 22 cases, no loss on any). It ran on thesonnetalias only; the owner accepted the pass rule as run on 2026-11-18 (decision comment above).One point stays unmeasured, as #16464's body says: no case exercises the running-UI clause.