Skip to content

feat(aidd-dev): require effective tests and disciplined test doubles - #1082

Open
BlandineRdl wants to merge 6 commits into
ai-driven-dev:nextfrom
BlandineRdl:feat/test-effectiveness-doubles
Open

BlandineRdl wants to merge 6 commits into
ai-driven-dev:nextfrom
BlandineRdl:feat/test-effectiveness-doubles

Conversation

@BlandineRdl

@BlandineRdl BlandineRdl commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

🎯 What & why

aidd-dev:06-test told the agent to check a passing test "against the quality criteria" without defining them. A green test that only echoes a mock's return value, or only asserts a mock was called, satisfied the contract while protecting nothing.

🛠️ How it works

plugins/aidd-dev/skills/06-test/actions/01-test.md gains a ## Quality criteria section, language-independent:

  • Project conventions win: a documented rule or the test configuration, never a pattern found in existing tests; any deviation is noted beside the test.
  • Observable behavior: no test that checks only internal structure, calls to a stub, or a collaboration graph without computed values.
  • Real subject: the logic under test is never doubled.
  • Doubles at seams: a first-match decision list per dependency, so no two rules conflict: never double the subject; mock an outgoing command only for that call; double a boundary only through a seam (port, adapter with no domain logic, or the framework's own interception such as fake timers); with no seam, never patch the subject, the behavior goes pending with missing seam at <dependency> once the user agrees; everything else, domain collaborators included, stays real.
  • Stub queries, mock commands: never assert on a stub; asserting a mock's call, with computed values, is legitimate when the call is the behavior.
  • No tautology: mock-return/assert arrangements are rejected.
  • Effectiveness: each test names a break that keeps the subject running. Behaviors scored 4-5 must run it (apply, watch it fail, restore and compare against a copy, rerun green); below 4 a reasoned check with its reason is accepted. Mutation testing only if the project already configures it; nothing is installed.

List and Approve now show and approve each behavior's score, since the score decides run vs reasoned. SKILL.md and 02-test-journey are untouched.

🧪 How to verify

⚠️ Heads-up

  • No structural test guards this text; nothing in the repository requires one.
  • A blind old-vs-new A/B (Sonnet, Haiku) ran on an earlier revision of this action and showed no measurable gain on a fixture with clean injected seams. A fixture with hard-wired dependencies is still needed to exercise the missing-seam rules.

🔗 Linked issue

Closes #952

✅ I certify

  • I DO CERTIFY I READ EACH LINE OF THE PULL REQUEST BECAUSE I AM A SOFTWARE ENGINEER, NOT A AI PUPPY.

🤖 Generated with Claude Code

BlandineRdl and others added 6 commits October 23, 2026 10:25
The test action now defines its quality criteria: assert observable
behavior, keep the subject real, place doubles only at seams, justify
interaction mocks, reject tautological mock-return assertions, and
confirm each test fails under a named break before calling it done.

Closes ai-driven-dev#952

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eria

Observable behavior and no-tautology now carve out the call that is
the behavior itself, the subject keeps its own logic real while its
boundaries follow the seam rule, and each test records whether its
break was run or only reasoned, with a rerun after the revert.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Doubles are split by role, stubs never asserted and mocks asserted as
the command they receive. A boundary with no seam leaves the behavior
pending instead of patching the subject. Critical behaviors must run
their break, with the restore checked against a copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A seam is a module with no domain logic or the framework's own
interception, a missing seam is skipped only with the user's word,
scores are approved with the list, a break must keep the subject
running, and a convention that overrides a criterion is noted.
Mocks of in-process collaborators stay allowed when the interaction
is the behavior.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The doubles criterion is now a first-match decision list: the subject,
an outgoing command, a seamed boundary, a missing seam, then everything
real. A domain collaborator could previously be both mocked and kept
real; the order settles it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stall

A seamed boundary may use an isolated real resource instead of a
double, so a behavior goes pending only when neither fits. The ban on
installing a dependency now covers mocking and coverage tools, not
only mutation testing. A tautology is judged by whether the test can
fail, so a stubbed value routed into a command stays testable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@BlandineRdl
BlandineRdl marked this pull request as ready for review October 23, 2026 11:30
@BlandineRdl
BlandineRdl requested a review from a team as a code owner October 23, 2026 11:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant