From b1fa5fd856baf263fad5e52c729c492c041754d3 Mon Sep 17 00:00:00 2001 From: Black Sun <18490995+BlandineRdl@users.noreply.github.com> Date: Fri, 23 Oct 2026 10:25:21 +0200 Subject: [PATCH 1/6] feat(aidd-dev): require effective tests and disciplined test doubles The test action now defines its quality criteria: assert observable behavior, keep the subject real, place doubles only at seams, justify interaction mocks, reject tautological mock-return assertions, and confirm each test fails under a named break before calling it done. Closes #952 Co-Authored-By: Claude Opus 5.5 --- .../skills/06-test/actions/01-test.md | 20 ++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/plugins/aidd-dev/skills/06-test/actions/01-test.md b/plugins/aidd-dev/skills/06-test/actions/01-test.md index a289c330f..87f38db7e 100644 --- a/plugins/aidd-dev/skills/06-test/actions/01-test.md +++ b/plugins/aidd-dev/skills/06-test/actions/01-test.md @@ -1,24 +1,30 @@ # 01 - Test - Identify untested behaviors in the target, then write and iterate tests until they pass with modern testing practices. ## Input - The scope to cover, a feature, module, or file glob, from the arguments. ## Output - -The behaviors found and, per behavior, a passing test with its file path, or a `pending` entry with a one-line reason when deliberately skipped. +The behaviors found and, per behavior, a passing test with its file path and the break it was confirmed to fail under, or a `pending` entry with a one-line reason when deliberately skipped. ## Process - 1. **List.** Enumerate the untested behaviors in the target area. Reason from the existing ones, score each from 0 (not needed) to 5 (critical core flow), group by section, prioritize by score and impact, and show a minimal bullet list. 2. **Approve.** Wait for user approval before generating any test. A behavior the user declines becomes a `pending` entry with a one-line reason, not tested. 3. **Generate.** Write the initial test for the highest-priority behavior, applying current testing practices and project conventions. -4. **Iterate.** Run the test. On failure, analyze it, improve the test, and repeat. On pass, check it against the quality criteria and improve when it falls short. +4. **Iterate.** Run the test. On failure, analyze it, improve the test, and repeat. On pass, check it against every quality criterion below and improve it until it meets them all. 5. **Next.** Move to the next behavior and repeat from Generate until the list is exhausted. -## Test +## Quality criteria +The project's testing commands, frameworks and conventions stay authoritative; these criteria hold within them, whatever the language. + +- **Observable behavior.** Assert what a caller or user can observe: a returned value, a state change, an output, an error. A test that checks only internal structure or which collaborators were called is not done. +- **Real subject.** The behavior under test runs for real. Never replace it, or any part of it, with a mock, stub, fake or spy. +- **Doubles at seams.** Introduce a double only at a boundary: network, filesystem, clock, randomness, a third-party service, or a seam the project already defines. Everywhere else, prefer real domain and pure objects. +- **Interaction as behavior.** Assert that a collaborator was called only when that call is the behavior itself, such as sending a message or emitting an event, and let the test's name say so. +- **No tautology.** Reject a test whose expected value only echoes what a double was configured to return, or whose only assertion is that a double was called. It passes whatever the subject does. +- **Effectiveness.** Name the break that would defeat the intended behavior, such as a removed branch, an inverted condition, or an input returned unchanged, and confirm the test fails under it. When it is cheap, apply the break, run the test, then revert it. A test no break can fail protects nothing: strengthen it. Use mutation testing only when the project already configures it; never install a tool for this. +## Test - Every behavior in the approved list has a corresponding test in the project suite that passes, with its file path recorded. +- Every such test meets each quality criterion and names the break it was confirmed to fail under. - Each deliberately skipped behavior has a `pending` entry with a one-line reason. From a2b69cef315ea6ac03bbbab52d9652c62a4e28d2 Mon Sep 17 00:00:00 2001 From: Black Sun <18490995+BlandineRdl@users.noreply.github.com> Date: Fri, 23 Oct 2026 10:27:36 +0200 Subject: [PATCH 2/6] fix(aidd-dev): reconcile interaction tests with the test quality criteria Observable behavior and no-tautology now carve out the call that is the behavior itself, the subject keeps its own logic real while its boundaries follow the seam rule, and each test records whether its break was run or only reasoned, with a rerun after the revert. Co-Authored-By: Claude Opus 5.5 --- plugins/aidd-dev/skills/06-test/actions/01-test.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/plugins/aidd-dev/skills/06-test/actions/01-test.md b/plugins/aidd-dev/skills/06-test/actions/01-test.md index 87f38db7e..c9471aa58 100644 --- a/plugins/aidd-dev/skills/06-test/actions/01-test.md +++ b/plugins/aidd-dev/skills/06-test/actions/01-test.md @@ -5,7 +5,7 @@ Identify untested behaviors in the target, then write and iterate tests until th The scope to cover, a feature, module, or file glob, from the arguments. ## Output -The behaviors found and, per behavior, a passing test with its file path and the break it was confirmed to fail under, or a `pending` entry with a one-line reason when deliberately skipped. +The behaviors found and, per behavior, a passing test with its file path and the break it fails under, marked `run` or `reasoned`, or a `pending` entry with a one-line reason when deliberately skipped. ## Process 1. **List.** Enumerate the untested behaviors in the target area. Reason from the existing ones, score each from 0 (not needed) to 5 (critical core flow), group by section, prioritize by score and impact, and show a minimal bullet list. @@ -17,14 +17,14 @@ The behaviors found and, per behavior, a passing test with its file path and the ## Quality criteria The project's testing commands, frameworks and conventions stay authoritative; these criteria hold within them, whatever the language. -- **Observable behavior.** Assert what a caller or user can observe: a returned value, a state change, an output, an error. A test that checks only internal structure or which collaborators were called is not done. -- **Real subject.** The behavior under test runs for real. Never replace it, or any part of it, with a mock, stub, fake or spy. -- **Doubles at seams.** Introduce a double only at a boundary: network, filesystem, clock, randomness, a third-party service, or a seam the project already defines. Everywhere else, prefer real domain and pure objects. +- **Observable behavior.** Assert what a caller or user can observe: a returned value, a state change, an output, an error. A test that checks only internal structure, or only which collaborators were called when that call is not the behavior, is not done. +- **Real subject.** The behavior under test runs for real. Never replace its own logic with a mock, stub, fake or spy; the boundaries it crosses follow Doubles at seams. +- **Doubles at seams.** Introduce a double only at a boundary: network, filesystem, clock, randomness, a third-party service, or a seam the project already defines. Everywhere else, use real domain and pure objects. - **Interaction as behavior.** Assert that a collaborator was called only when that call is the behavior itself, such as sending a message or emitting an event, and let the test's name say so. -- **No tautology.** Reject a test whose expected value only echoes what a double was configured to return, or whose only assertion is that a double was called. It passes whatever the subject does. -- **Effectiveness.** Name the break that would defeat the intended behavior, such as a removed branch, an inverted condition, or an input returned unchanged, and confirm the test fails under it. When it is cheap, apply the break, run the test, then revert it. A test no break can fail protects nothing: strengthen it. Use mutation testing only when the project already configures it; never install a tool for this. +- **No tautology.** Reject a test whose expected value only echoes what a double was configured to return, or whose only assertion is that a double was called when that call is not the behavior. It passes whatever the subject does. +- **Effectiveness.** Name the break that would defeat the intended behavior, such as a removed branch, an inverted condition, or an input returned unchanged, and check that the test fails under it. When it is cheap, apply the break, watch the test fail, revert it, then rerun the test and confirm it passes on the unchanged subject; this is a `run` check, anything less is `reasoned`. A test no break can fail protects nothing: strengthen it. Use mutation testing only when the project already configures it; never install a tool for this. ## Test - Every behavior in the approved list has a corresponding test in the project suite that passes, with its file path recorded. -- Every such test meets each quality criterion and names the break it was confirmed to fail under. +- Every such test meets each quality criterion and names the break it fails under, marked `run` or `reasoned`. - Each deliberately skipped behavior has a `pending` entry with a one-line reason. From 5cdc0eb877e6d286cc886d440bbe085b3a921193 Mon Sep 17 00:00:00 2001 From: Black Sun <18490995+BlandineRdl@users.noreply.github.com> Date: Fri, 23 Oct 2026 10:47:10 +0200 Subject: [PATCH 3/6] fix(aidd-dev): sharpen test doubles and effectiveness rules Doubles are split by role, stubs never asserted and mocks asserted as the command they receive. A boundary with no seam leaves the behavior pending instead of patching the subject. Critical behaviors must run their break, with the restore checked against a copy. Co-Authored-By: Claude Opus 5.5 --- .../aidd-dev/skills/06-test/actions/01-test.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/plugins/aidd-dev/skills/06-test/actions/01-test.md b/plugins/aidd-dev/skills/06-test/actions/01-test.md index c9471aa58..bdd3d5d94 100644 --- a/plugins/aidd-dev/skills/06-test/actions/01-test.md +++ b/plugins/aidd-dev/skills/06-test/actions/01-test.md @@ -5,7 +5,7 @@ Identify untested behaviors in the target, then write and iterate tests until th The scope to cover, a feature, module, or file glob, from the arguments. ## Output -The behaviors found and, per behavior, a passing test with its file path and the break it fails under, marked `run` or `reasoned`, or a `pending` entry with a one-line reason when deliberately skipped. +The behaviors found and, per behavior, a passing test with its file path and its effectiveness check, `run` or `reasoned` with its reason, or a `pending` entry with a one-line reason when deliberately skipped or blocked by a missing seam. ## Process 1. **List.** Enumerate the untested behaviors in the target area. Reason from the existing ones, score each from 0 (not needed) to 5 (critical core flow), group by section, prioritize by score and impact, and show a minimal bullet list. @@ -17,14 +17,14 @@ The behaviors found and, per behavior, a passing test with its file path and the ## Quality criteria The project's testing commands, frameworks and conventions stay authoritative; these criteria hold within them, whatever the language. -- **Observable behavior.** Assert what a caller or user can observe: a returned value, a state change, an output, an error. A test that checks only internal structure, or only which collaborators were called when that call is not the behavior, is not done. -- **Real subject.** The behavior under test runs for real. Never replace its own logic with a mock, stub, fake or spy; the boundaries it crosses follow Doubles at seams. -- **Doubles at seams.** Introduce a double only at a boundary: network, filesystem, clock, randomness, a third-party service, or a seam the project already defines. Everywhere else, use real domain and pure objects. -- **Interaction as behavior.** Assert that a collaborator was called only when that call is the behavior itself, such as sending a message or emitting an event, and let the test's name say so. -- **No tautology.** Reject a test whose expected value only echoes what a double was configured to return, or whose only assertion is that a double was called when that call is not the behavior. It passes whatever the subject does. -- **Effectiveness.** Name the break that would defeat the intended behavior, such as a removed branch, an inverted condition, or an input returned unchanged, and check that the test fails under it. When it is cheap, apply the break, watch the test fail, revert it, then rerun the test and confirm it passes on the unchanged subject; this is a `run` check, anything less is `reasoned`. A test no break can fail protects nothing: strengthen it. Use mutation testing only when the project already configures it; never install a tool for this. +- **Observable behavior.** Assert what a caller or user can observe: a returned value, a state change, an output, an error. A test that checks only internal structure is not done; which calls may be asserted is settled by Stub queries, mock commands. +- **Real subject.** The behavior under test runs for real. Never replace its own logic with a double; the boundaries it crosses follow Doubles at seams. +- **Doubles at seams.** Introduce a double only at a boundary: network, filesystem, clock, randomness, a third-party service, or a seam the project already defines, such as a port. Replacing the whole module that owns a boundary is such a seam; replacing part of the subject is not. Everywhere else, use real domain and pure objects. When a boundary offers no seam, never patch the subject to make one: the behavior becomes `pending` with `missing seam at ` as its reason, and the design change is left to its owner. +- **Stub queries, mock commands.** A double that feeds the subject data is a stub: never assert on it. A double that receives the subject's outgoing command, such as sending a message or writing a record, is a mock: asserting that call is the behavior, and the test's name says so. +- **No tautology.** Reject a test whose expected value only echoes what a stub was configured to return. It passes whatever the subject does. +- **Effectiveness.** Name the break that would defeat the intended behavior, such as a removed branch, an inverted condition, or an input returned unchanged. For a behavior scored 4 or 5, the check is `run`: copy the files the break touches, apply it and watch the test fail, restore them, confirm they match the copy, then rerun and watch the test pass. Below 4, a `reasoned` check may replace it, stating why the test fails under the break. A test no break can fail protects nothing: strengthen it. Use mutation testing only when the project already configures it; never install a tool for this. ## Test - Every behavior in the approved list has a corresponding test in the project suite that passes, with its file path recorded. -- Every such test meets each quality criterion and names the break it fails under, marked `run` or `reasoned`. -- Each deliberately skipped behavior has a `pending` entry with a one-line reason. +- Every such test meets each quality criterion and records its effectiveness check: `run` for a behavior scored 4 or 5, `run` or `reasoned` with its reason below. +- Each behavior deliberately skipped or blocked by a missing seam has a `pending` entry with a one-line reason. From d3a4feba13d39d921988024fda03c8127c168cc5 Mon Sep 17 00:00:00 2001 From: Black Sun <18490995+BlandineRdl@users.noreply.github.com> Date: Fri, 23 Oct 2026 10:49:56 +0200 Subject: [PATCH 4/6] fix(aidd-dev): close the loopholes in the test quality criteria A seam is a module with no domain logic or the framework's own interception, a missing seam is skipped only with the user's word, scores are approved with the list, a break must keep the subject running, and a convention that overrides a criterion is noted. Mocks of in-process collaborators stay allowed when the interaction is the behavior. Co-Authored-By: Claude Opus 5.5 --- .../skills/06-test/actions/01-test.md | 24 ++++++++++++------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/plugins/aidd-dev/skills/06-test/actions/01-test.md b/plugins/aidd-dev/skills/06-test/actions/01-test.md index bdd3d5d94..47e9a11e8 100644 --- a/plugins/aidd-dev/skills/06-test/actions/01-test.md +++ b/plugins/aidd-dev/skills/06-test/actions/01-test.md @@ -1,30 +1,36 @@ # 01 - Test + Identify untested behaviors in the target, then write and iterate tests until they pass with modern testing practices. ## Input + The scope to cover, a feature, module, or file glob, from the arguments. ## Output -The behaviors found and, per behavior, a passing test with its file path and its effectiveness check, `run` or `reasoned` with its reason, or a `pending` entry with a one-line reason when deliberately skipped or blocked by a missing seam. + +The behaviors found and, per behavior, a passing test with its file path and its effectiveness check, `run` or `reasoned` with its reason, or a `pending` entry with a one-line reason when deliberately skipped or blocked by a missing seam. Any deviation from a quality criterion that a project convention imposes is noted beside its test. ## Process -1. **List.** Enumerate the untested behaviors in the target area. Reason from the existing ones, score each from 0 (not needed) to 5 (critical core flow), group by section, prioritize by score and impact, and show a minimal bullet list. -2. **Approve.** Wait for user approval before generating any test. A behavior the user declines becomes a `pending` entry with a one-line reason, not tested. + +1. **List.** Enumerate the untested behaviors in the target area. Reason from the existing ones, score each from 0 (not needed) to 5 (critical core flow), group by section, prioritize by score and impact, and show a minimal bullet list with each score. +2. **Approve.** Wait for user approval of the list and its scores before generating any test. A behavior the user declines becomes a `pending` entry with a one-line reason, not tested. 3. **Generate.** Write the initial test for the highest-priority behavior, applying current testing practices and project conventions. 4. **Iterate.** Run the test. On failure, analyze it, improve the test, and repeat. On pass, check it against every quality criterion below and improve it until it meets them all. 5. **Next.** Move to the next behavior and repeat from Generate until the list is exhausted. ## Quality criteria -The project's testing commands, frameworks and conventions stay authoritative; these criteria hold within them, whatever the language. -- **Observable behavior.** Assert what a caller or user can observe: a returned value, a state change, an output, an error. A test that checks only internal structure is not done; which calls may be asserted is settled by Stub queries, mock commands. +The project's testing commands, frameworks and conventions stay authoritative: where one conflicts with a criterion, follow it and note the deviation. A convention is a documented rule or the project's test configuration, never a pattern found in existing tests. Otherwise every criterion holds, whatever the language. + +- **Observable behavior.** Assert what a caller or user can observe: a returned value, a state change, an output, an error, or an outgoing command as Stub queries, mock commands defines it. A test that checks only internal structure, calls to a stub, or which collaborators were called without the values the subject computed is not done. - **Real subject.** The behavior under test runs for real. Never replace its own logic with a double; the boundaries it crosses follow Doubles at seams. -- **Doubles at seams.** Introduce a double only at a boundary: network, filesystem, clock, randomness, a third-party service, or a seam the project already defines, such as a port. Replacing the whole module that owns a boundary is such a seam; replacing part of the subject is not. Everywhere else, use real domain and pure objects. When a boundary offers no seam, never patch the subject to make one: the behavior becomes `pending` with `missing seam at ` as its reason, and the design change is left to its owner. -- **Stub queries, mock commands.** A double that feeds the subject data is a stub: never assert on it. A double that receives the subject's outgoing command, such as sending a message or writing a record, is a mock: asserting that call is the behavior, and the test's name says so. +- **Doubles at seams.** Introduce a double only at a boundary, or as a mock when the interaction is the behavior, as Stub queries, mock commands defines it; there, a domain collaborator may be mocked only for the call under test. A boundary is network, filesystem, clock, randomness, a third-party service, or a seam the project already defines, such as a port. A module whose sole job is that boundary, such as an adapter or client with no domain logic, is such a seam, and so is the test framework's or runtime's own interception of it, such as fake timers or an HTTP interceptor. A collaborator carrying domain logic is not: use the real one, as with domain and pure objects. When a boundary offers no seam, never patch the subject to make one: show the behavior to the user with its score and `missing seam at `, and once they agree, it becomes `pending` with that reason and the design change is left to its owner. +- **Stub queries, mock commands.** A double that feeds the subject data is a stub: never assert on it. A double whose call changes something outside the subject, such as sending a message, writing a record or emitting an event, is a mock: asserting that call, with the values the subject computed, is the behavior, and the test's name says so. A call that only fetches data is a stub's, whatever the double is called. - **No tautology.** Reject a test whose expected value only echoes what a stub was configured to return. It passes whatever the subject does. -- **Effectiveness.** Name the break that would defeat the intended behavior, such as a removed branch, an inverted condition, or an input returned unchanged. For a behavior scored 4 or 5, the check is `run`: copy the files the break touches, apply it and watch the test fail, restore them, confirm they match the copy, then rerun and watch the test pass. Below 4, a `reasoned` check may replace it, stating why the test fails under the break. A test no break can fail protects nothing: strengthen it. Use mutation testing only when the project already configures it; never install a tool for this. +- **Effectiveness.** Name the break that would defeat the intended behavior, such as a removed branch, an inverted condition, or an input returned unchanged. The break keeps the subject running and targets the behavior's own logic; deleting the subject or making it throw does not count. For a behavior scored 4 or 5, the check is `run`: copy the files the break touches, apply it and watch the test fail, restore them, confirm they match the copy, then rerun and watch the test pass. Below 4, a `reasoned` check may replace it, stating why the test fails under the break. A test no break can fail protects nothing: strengthen it. Use mutation testing only when the project already configures it; never install a tool for this. ## Test + - Every behavior in the approved list has a corresponding test in the project suite that passes, with its file path recorded. -- Every such test meets each quality criterion and records its effectiveness check: `run` for a behavior scored 4 or 5, `run` or `reasoned` with its reason below. +- Every such test meets each quality criterion, or notes the convention it defers to, and records its effectiveness check: `run` for a behavior scored 4 or 5, `run` or `reasoned` with its reason below. - Each behavior deliberately skipped or blocked by a missing seam has a `pending` entry with a one-line reason. From 560831d5cd77ac70054cf13a0c1772a4cd1aebf6 Mon Sep 17 00:00:00 2001 From: Black Sun <18490995+BlandineRdl@users.noreply.github.com> Date: Fri, 23 Oct 2026 11:29:00 +0200 Subject: [PATCH 5/6] fix(aidd-dev): order the test-double cases so no two rules conflict The doubles criterion is now a first-match decision list: the subject, an outgoing command, a seamed boundary, a missing seam, then everything real. A domain collaborator could previously be both mocked and kept real; the order settles it. Co-Authored-By: Claude Opus 5.5 --- plugins/aidd-dev/skills/06-test/actions/01-test.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/plugins/aidd-dev/skills/06-test/actions/01-test.md b/plugins/aidd-dev/skills/06-test/actions/01-test.md index 47e9a11e8..2f88ea6bc 100644 --- a/plugins/aidd-dev/skills/06-test/actions/01-test.md +++ b/plugins/aidd-dev/skills/06-test/actions/01-test.md @@ -24,7 +24,12 @@ The project's testing commands, frameworks and conventions stay authoritative: w - **Observable behavior.** Assert what a caller or user can observe: a returned value, a state change, an output, an error, or an outgoing command as Stub queries, mock commands defines it. A test that checks only internal structure, calls to a stub, or which collaborators were called without the values the subject computed is not done. - **Real subject.** The behavior under test runs for real. Never replace its own logic with a double; the boundaries it crosses follow Doubles at seams. -- **Doubles at seams.** Introduce a double only at a boundary, or as a mock when the interaction is the behavior, as Stub queries, mock commands defines it; there, a domain collaborator may be mocked only for the call under test. A boundary is network, filesystem, clock, randomness, a third-party service, or a seam the project already defines, such as a port. A module whose sole job is that boundary, such as an adapter or client with no domain logic, is such a seam, and so is the test framework's or runtime's own interception of it, such as fake timers or an HTTP interceptor. A collaborator carrying domain logic is not: use the real one, as with domain and pure objects. When a boundary offers no seam, never patch the subject to make one: show the behavior to the user with its score and `missing seam at `, and once they agree, it becomes `pending` with that reason and the design change is left to its owner. +- **Doubles at seams.** For each dependency of the subject, take the first case that applies: + 1. It is the behavior under test: never double it. + 2. The subject sends it a command whose call is the behavior: mock it for that call only, as Stub queries, mock commands says, where the test can substitute it without patching the subject. + 3. It crosses a boundary (network, filesystem, clock, randomness, a third-party service) through a seam: a seam the project defines such as a port, a module whose sole job is that boundary, or the test framework's own interception of it such as fake timers. Double it at that seam. + 4. It receives such a command or crosses such a boundary, but no seam lets the test substitute it: never patch the subject. Show the behavior to the user with its score and `missing seam at `; once they agree, it becomes `pending` with that reason and the design change is left to its owner. + 5. Anything else, including a collaborator carrying domain logic: use the real one. - **Stub queries, mock commands.** A double that feeds the subject data is a stub: never assert on it. A double whose call changes something outside the subject, such as sending a message, writing a record or emitting an event, is a mock: asserting that call, with the values the subject computed, is the behavior, and the test's name says so. A call that only fetches data is a stub's, whatever the double is called. - **No tautology.** Reject a test whose expected value only echoes what a stub was configured to return. It passes whatever the subject does. - **Effectiveness.** Name the break that would defeat the intended behavior, such as a removed branch, an inverted condition, or an input returned unchanged. The break keeps the subject running and targets the behavior's own logic; deleting the subject or making it throw does not count. For a behavior scored 4 or 5, the check is `run`: copy the files the break touches, apply it and watch the test fail, restore them, confirm they match the copy, then rerun and watch the test pass. Below 4, a `reasoned` check may replace it, stating why the test fails under the break. A test no break can fail protects nothing: strengthen it. Use mutation testing only when the project already configures it; never install a tool for this. From 70d4db63c2b410a29bc737830ccda5b2639ba071 Mon Sep 17 00:00:00 2001 From: Black Sun <18490995+BlandineRdl@users.noreply.github.com> Date: Fri, 23 Oct 2026 12:01:49 +0200 Subject: [PATCH 6/6] fix(aidd-dev): let real resources stand in for doubles and bar any install A seamed boundary may use an isolated real resource instead of a double, so a behavior goes pending only when neither fits. The ban on installing a dependency now covers mocking and coverage tools, not only mutation testing. A tautology is judged by whether the test can fail, so a stubbed value routed into a command stays testable. Co-Authored-By: Claude Opus 5.5 --- plugins/aidd-dev/skills/06-test/actions/01-test.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/plugins/aidd-dev/skills/06-test/actions/01-test.md b/plugins/aidd-dev/skills/06-test/actions/01-test.md index 2f88ea6bc..8095f3561 100644 --- a/plugins/aidd-dev/skills/06-test/actions/01-test.md +++ b/plugins/aidd-dev/skills/06-test/actions/01-test.md @@ -20,19 +20,19 @@ The behaviors found and, per behavior, a passing test with its file path and its ## Quality criteria -The project's testing commands, frameworks and conventions stay authoritative: where one conflicts with a criterion, follow it and note the deviation. A convention is a documented rule or the project's test configuration, never a pattern found in existing tests. Otherwise every criterion holds, whatever the language. +The project's testing commands, frameworks and conventions stay authoritative: where one conflicts with a criterion, follow it and note the deviation. A convention is a documented rule or the project's test configuration, never a pattern found in existing tests. Never install a mocking, coverage or mutation dependency to meet these criteria. Otherwise every criterion holds, whatever the language. - **Observable behavior.** Assert what a caller or user can observe: a returned value, a state change, an output, an error, or an outgoing command as Stub queries, mock commands defines it. A test that checks only internal structure, calls to a stub, or which collaborators were called without the values the subject computed is not done. - **Real subject.** The behavior under test runs for real. Never replace its own logic with a double; the boundaries it crosses follow Doubles at seams. - **Doubles at seams.** For each dependency of the subject, take the first case that applies: 1. It is the behavior under test: never double it. 2. The subject sends it a command whose call is the behavior: mock it for that call only, as Stub queries, mock commands says, where the test can substitute it without patching the subject. - 3. It crosses a boundary (network, filesystem, clock, randomness, a third-party service) through a seam: a seam the project defines such as a port, a module whose sole job is that boundary, or the test framework's own interception of it such as fake timers. Double it at that seam. - 4. It receives such a command or crosses such a boundary, but no seam lets the test substitute it: never patch the subject. Show the behavior to the user with its score and `missing seam at `; once they agree, it becomes `pending` with that reason and the design change is left to its owner. + 3. It crosses a boundary (network, filesystem, clock, randomness, a third-party service) through a seam: a seam the project defines such as a port, a module whose sole job is that boundary, or the test framework's own interception of it such as fake timers. Double it at that seam, or use an isolated real resource that exposes the effect, such as a temporary directory or an in-memory store. + 4. It receives such a command or crosses such a boundary, but no seam lets the test substitute it and no isolated real resource exposes the effect: never patch the subject. Show the behavior to the user with its score and `missing seam at `; once they agree, it becomes `pending` with that reason and the design change is left to its owner. 5. Anything else, including a collaborator carrying domain logic: use the real one. - **Stub queries, mock commands.** A double that feeds the subject data is a stub: never assert on it. A double whose call changes something outside the subject, such as sending a message, writing a record or emitting an event, is a mock: asserting that call, with the values the subject computed, is the behavior, and the test's name says so. A call that only fetches data is a stub's, whatever the double is called. -- **No tautology.** Reject a test whose expected value only echoes what a stub was configured to return. It passes whatever the subject does. -- **Effectiveness.** Name the break that would defeat the intended behavior, such as a removed branch, an inverted condition, or an input returned unchanged. The break keeps the subject running and targets the behavior's own logic; deleting the subject or making it throw does not count. For a behavior scored 4 or 5, the check is `run`: copy the files the break touches, apply it and watch the test fail, restore them, confirm they match the copy, then rerun and watch the test pass. Below 4, a `reasoned` check may replace it, stating why the test fails under the break. A test no break can fail protects nothing: strengthen it. Use mutation testing only when the project already configures it; never install a tool for this. +- **No tautology.** Reject a test that passes whatever the subject does, such as one whose expected value only echoes what a stub was configured to return. +- **Effectiveness.** Name the break that would defeat the intended behavior, such as a removed branch, an inverted condition, or an input returned unchanged. The break keeps the subject running and targets the behavior's own logic; deleting the subject or making it throw does not count. For a behavior scored 4 or 5, the check is `run`: copy the files the break touches, apply it and watch the test fail, restore them, confirm they match the copy, then rerun and watch the test pass. Below 4, a `reasoned` check may replace it, stating why the test fails under the break. A test no break can fail protects nothing: strengthen it. Use mutation testing only when the project already configures it. ## Test