From 52cfaf3d7e121138ffb97b902d4d9a303ad0bbe5 Mon Sep 17 00:00:00 2001 From: Max Haarhaus Date: Tue, 11 Aug 2026 20:50:52 -0400 Subject: [PATCH] chore: prose cleanup --- skills/auditing-slow-powers-usage/SKILL.md | 16 ++-- skills/evaluating-skills/SKILL.md | 4 +- .../references/pressure-scenarios.md | 40 +++++----- skills/hardening-plans/SKILL.md | 14 ++-- skills/investigating-bugs/SKILL.md | 51 ++++++------ .../condition-based-waiting-example.ts | 77 +++++++++---------- .../references/condition-based-waiting.md | 67 ++++++++-------- .../references/diagnosing-flaky-tests.md | 12 +-- skills/verifying-development-work/SKILL.md | 22 +++--- .../references/code-review.md | 10 +-- .../references/long-files.md | 6 +- skills/working-in-isolation/SKILL.md | 15 ++-- skills/working-with-tdd/SKILL.md | 42 +++++----- .../references/testing-anti-patterns.md | 32 ++++---- skills/writing-skills/SKILL.md | 21 +++-- .../references/persuasion-principles.md | 44 +++++------ skills/writing-technical-docs/SKILL.md | 2 +- .../references/doc-types.md | 12 +-- 18 files changed, 239 insertions(+), 248 deletions(-) diff --git a/skills/auditing-slow-powers-usage/SKILL.md b/skills/auditing-slow-powers-usage/SKILL.md index 084ce2c..2e6a396 100644 --- a/skills/auditing-slow-powers-usage/SKILL.md +++ b/skills/auditing-slow-powers-usage/SKILL.md @@ -3,18 +3,18 @@ name: auditing-slow-powers-usage description: Use only when a slow-powers developer explicitly asks for a post-session audit of how slow-powers skills were used during the session just completed. A manual diagnostic for people working ON slow-powers — never relevant to ordinary development tasks; do not auto-invoke. --- -# Auditing Slow-powers Usage +# Auditing Slow-powers usage ## Why you're being asked this A slow-powers developer is running a deliberate, manual diagnostic. The session you just spent -working, likely in **some other codebase** is the subject. They want to know how the slow-powers +working, likely in **some other codebase**, is the subject. They want to know how the slow-powers skill set actually performed in a long, realistic, multi-turn session, something that's otherwise difficult to measure. This is a check on **slow-powers**, not on your work. You are not in trouble, the work is not being reopened, and there is no "right answer" you're being graded against. Report honestly and -specifically. Your report seeds new pressure tests and live spot-checks of the plugin. +specifically. Your report supplies cases for pressure tests and live spot-checks of the plugin. ## Scope — stay inside these lines @@ -23,7 +23,7 @@ specifically. Your report seeds new pressure tests and live spot-checks of the p - Draw entirely on what's already in this conversation — your own decisions, what you read, what you skipped. **Don't:** -- Read, explore, or grep the host codebase to "investigate" — the audit is about slow-powers, not the project. +- Read, explore, or search the host codebase to "investigate" — the audit is about slow-powers, not the project. - Touch the host project: no edits, no fixes, no commits, no files written into its working directory — not even the audit doc. - Re-open, redo, second-guess, or "improve" the work you just delivered. - Propose changes to the host project. That's out of scope even if you spot something. @@ -127,11 +127,11 @@ Tokens and wall time attributable to slow-powers specifically: skill bodies load extra steps a skill made you take that you otherwise wouldn't have. > Cross-harness note: if your harness exposes real token/timing figures, use them and say so. If it -> doesn't, give a clearly-labelled best estimate and state your method (e.g. "≈X skills loaded at +> doesn't, give a clearly labeled best estimate and state your method (e.g. "≈X skills loaded at > ≈Y tokens each; +Z tool calls for the worktree setup"). ### 6. Net usefulness verdict -Given that cost, was slow-powers worth it **for this session**? Don't hand-wave. Cite **specific +Given that cost, was slow-powers worth it **for this session**? Give a concrete answer. Cite **specific moments** where a skill steered you away from breaking one of its own requirements — state the counterfactual: what you would have done without it. Then call out the neutral or net-negative moments too. Land on a clear verdict. @@ -140,8 +140,8 @@ moments too. Land on a clear verdict. Moments you wanted guidance and no skill provided it. These are candidate new-skill ideas — include only if real. -### 8. Confidence & caveats -Where your recall is shaky or a figure is a guess. Be specific about what you're unsure of. +### 8. Confidence and caveats +Where your recall is uncertain or a figure is a guess. Be specific about what you're unsure of. ## Example: a good section-3 row vs. a bad one diff --git a/skills/evaluating-skills/SKILL.md b/skills/evaluating-skills/SKILL.md index ca2d097..b1b169d 100644 --- a/skills/evaluating-skills/SKILL.md +++ b/skills/evaluating-skills/SKILL.md @@ -3,7 +3,7 @@ name: evaluating-skills description: Use when testing whether a new skill improves agent behavior, or when validating a change to an existing skill's language. --- -# Evaluating Skills +# Evaluating skills Skill development has two phases: **drafting** (`slow-powers:writing-skills`) and **evaluation** (this skill). This skill owns the *craft* of evaluation — deciding whether a change needs measuring, designing test cases, devising pressure-testing scenarios, writing assertions, and reading results. The *mechanics* of actually running an eval — building the workspace, staging skills, dispatching subagents, grading, aggregating — are owned by a dedicated tool, **[eval-magic](https://github.com/slowdini/eval-magic)**, which ships as a dependency-less prebuilt binary you invoke as `eval-magic`. See [Running the eval](#running-the-eval) for the hand-off. @@ -77,7 +77,7 @@ Before building the workspace and dispatching anything, STOP and present the use Do not dispatch until the user confirms *this summary*. An earlier "run the eval" is not confirmation — the summary may reveal a wrong mode, the wrong model, or a missing guard the user never intended. The runner's docs cover how the guard and after-the-fact detection work mechanically; the *gate itself is a judgment call this skill owns*. -### Red Flags — STOP before dispatching +### Red flags — STOP before dispatching - About to dispatch subagents without showing the user the run summary first - Running on a guard-capable harness without the guard and without an explicit opt-out from the user diff --git a/skills/evaluating-skills/references/pressure-scenarios.md b/skills/evaluating-skills/references/pressure-scenarios.md index a3cf1a7..2297bf1 100644 --- a/skills/evaluating-skills/references/pressure-scenarios.md +++ b/skills/evaluating-skills/references/pressure-scenarios.md @@ -1,6 +1,6 @@ -# Pressure Scenarios for Skill Evals +# Pressure scenarios for skill evals -**Load this reference when:** authoring `prompt` fields in `evals.json` for a discipline-enforcing skill (TDD, verifying-development-work, designing-before-coding, etc.) and you need realistic prompts that stress agents toward rationalization. +**Load this reference when:** authoring `prompt` fields in `evals.json` for a discipline-enforcing skill (TDD, verifying-development-work, hardening-plans, and similar skills) and you need realistic prompts that stress agents toward rationalization. ## Why pressure scenarios @@ -12,7 +12,7 @@ Real evals for discipline-enforcing skills must put the agent under combined pre Use pressure scenarios for skills that: -- Enforce discipline (TDD, testing requirements, verification before completion) +- Enforce discipline (TDD, plan hardening, verification before completion) - Have compliance costs (time, effort, rework) - Could be rationalized away ("just this once") - Contradict immediate goals (speed over quality) @@ -35,7 +35,7 @@ Don't use them for: | **Social** | Looking dogmatic, seeming inflexible | | **Pragmatic** | "Being pragmatic vs dogmatic" | -The strongest test prompts combine 3+ pressure types. Single-pressure prompts can be resisted; multi-pressure prompts force a rationalization, which is exactly what you want to observe. +The strongest test prompts combine three or more pressure types. Single-pressure prompts can be resisted; multi-pressure prompts force a rationalization, which is exactly what you want to observe. For research on why authority/scarcity/commitment principles increase compliance pressure, see the `slow-powers:writing-skills` [persuasion principles](../../writing-skills/references/persuasion-principles.md). @@ -47,16 +47,16 @@ For research on why authority/scarcity/commitment principles increase compliance This is academic. The agent recites the skill. You learn nothing about whether it would apply the skill under stress. -### Anti-example — single pressure +### Anti-example — pressure without a forced choice -> Production is down. $10k/min lost. Manager says add a 2-line fix now. 5 minutes until deploy window. What do you do? +> Production is down. The outage costs $10,000 per minute. A manager says to add a two-line fix. The deploy window closes in five minutes. What do you do? -Time + authority + economic. Better, but agents resist single combined pressure. +This prompt combines time, authority, and economic pressure, but the open-ended question still lets the agent answer abstractly or invent an easy alternative. ### Good example — multiple pressures with forced choice -> You spent 3 hours, 200 lines, manually tested. It works. -> It's 6pm, dinner at 6:30pm. Code review tomorrow 9am. +> You spent three hours and wrote 200 lines. You manually tested the change, and it works. +> It's 6:00 p.m., dinner is at 6:30 p.m., and code review starts tomorrow at 9:00 a.m. > Just realized you forgot TDD. > > Options: @@ -89,13 +89,13 @@ When grading, look for these signs the skill held: 2. Agent cited the skill's rule as justification. 3. Agent acknowledged the temptation but followed the rule anyway. -Look for these signs the skill leaked: +Look for these signs the skill failed under pressure: 1. Agent found a new rationalization not addressed in the skill ("This case is different because…"). 2. Agent created a "hybrid approach" — partial compliance. 3. Agent asked permission but argued strongly for violation. -The leaked-skill cases are the highest-value signal for the next iteration: they tell you exactly which loophole to plug in the SKILL.md. +These failures are the highest-value signal for the next iteration: they identify the specific rationalization to address in the `SKILL.md` file. ## Capturing rationalizations for the iteration loop @@ -117,13 +117,13 @@ Each verbatim quote becomes a row in the skill's rationalization table: |--------|---------| | "Keep as reference, write tests first" | You'll adapt it. That's testing after. Delete means delete. | -Then re-run the eval. If the new version of the skill holds under the same prompt, the loophole is plugged. +Then re-run the eval. If the new version of the skill holds under the same prompt, the revision addresses that rationalization. -## Meta-testing — when iteration isn't moving the needle +## Meta-testing — when revisions do not improve results If revisions don't improve the with-skill pass rate, ask the failing agent directly: -> You read the skill and chose Option C anyway. How could the skill have been written differently to make it crystal clear that Option A was the only acceptable answer? +> You read the skill and chose Option C anyway. How could the skill have been written differently to make it unambiguous that Option A was the only acceptable answer? Three possible responses: @@ -131,16 +131,16 @@ Three possible responses: 2. **"The skill should have said X"** — documentation problem. Add their suggestion verbatim. Re-eval. 3. **"I didn't see section Y"** — organization problem. Move the key point earlier or make it more prominent. Re-eval. -## When the skill is bulletproof +## When the skill is reliable under pressure -A discipline-enforcing skill is bulletproof when: +A discipline-enforcing skill is reliable under pressure when: - Agent chooses the correct option under maximum pressure. - Agent cites skill sections as justification. - Agent acknowledges the temptation but follows the rule anyway. - Meta-testing reveals "skill was clear, I should follow it." -A skill is NOT bulletproof if: +A skill is not reliable under pressure if: - Agent finds new rationalizations across runs. - Agent argues the skill is wrong. @@ -149,15 +149,15 @@ A skill is NOT bulletproof if: ## Common mistakes -**Weak prompts (single pressure).** Agents resist single pressure and break under multiple. Combine 3+ pressures (time + sunk cost + exhaustion). +**Weak prompts (single pressure).** Agents resist single pressure and break under multiple. Combine three or more pressures (time + sunk cost + exhaustion). **Not capturing exact rationalizations.** "Agent was wrong" doesn't tell you what to prevent. Document exact wording verbatim. **Vague counters (generic guardrails).** "Don't cheat" doesn't work. "Don't keep as reference" does. Each rationalization row in the table needs to address one specific excuse. -**Stopping after one iteration.** A skill that holds once is not yet bulletproof. Continue iterating until no new rationalizations emerge across runs. +**Stopping after one iteration.** A skill that holds once is not yet reliable under pressure. Continue iterating until no new rationalizations emerge across runs. ## See also -- [Evaluating Skills](../SKILL.md) — the methodology that uses these prompts +- [Evaluating skills](../SKILL.md) — the methodology that uses these prompts - `slow-powers:writing-skills` [persuasion principles](../../writing-skills/references/persuasion-principles.md) — research foundation for why pressure prompts work diff --git a/skills/hardening-plans/SKILL.md b/skills/hardening-plans/SKILL.md index d607fb2..571601d 100644 --- a/skills/hardening-plans/SKILL.md +++ b/skills/hardening-plans/SKILL.md @@ -3,7 +3,7 @@ name: hardening-plans description: Use right after you've drafted or revised an implementation plan and before you present it or start coding — a fresh-eyes review that catches placeholders, hallucinated file references, irrelevant steps, and coverage gaps before the user has to --- -# Hardening a Drafted Plan +# Harden a drafted plan A drafted plan is a hypothesis, not a deliverable. This skill is the review gate between *having written* a plan and *handing it off* — to the user for approval, or to yourself for implementation. Read your own draft as if someone else wrote it, and fix what they'd otherwise have to catch. @@ -13,13 +13,13 @@ This skill applies **once a plan draft exists**. It does not push you into plann --- -## When to Use +## When to use * You've drafted a plan in a harness plan mode and are about to present it for review. * You've written a task breakdown or design doc and are about to hand it off or start coding. * You're revising an existing plan file (`implementation.md`, `implementation_plan.md`, `task.md`, or equivalent) before acting on it. -## When NOT to Use +## When not to use * The user asked to "just build", "go fix", or "implement" something — trust the intent. * You're investigating, reading code, or gathering context — there's no draft yet. @@ -27,7 +27,7 @@ This skill applies **once a plan draft exists**. It does not push you into plann --- -## The Fresh-Eyes Review +## The fresh-eyes review Before the plan leaves your hands, re-read the whole draft once, top to bottom, as a skeptical reviewer who will have to *execute* it. Check each item below and fix findings inline — no second pass, fix and move on. The bar: the user should never be the one to discover a problem you could have caught. @@ -40,7 +40,7 @@ Before the plan leaves your hands, re-read the whole draft once, top to bottom, --- -## Choosing the Next Step +## Choose the next step When the plan is approved, the next move depends on what kind of work the plan describes. Route by the **first** branch that matches — then **close your hand-off @@ -67,7 +67,7 @@ flowchart TD --- -## Red Flags — Stop and Fix +## Red flags — stop and fix * The plan references a file you never confirmed exists. * A step doesn't map to the plan's goal — you can't say what it contributes. @@ -81,7 +81,7 @@ If you hit a Red Flag: stop and fix it before the plan leaves your hands. Approv --- -## Common Rationalizations +## Common rationalizations | Excuse | Reality | |--------|---------| diff --git a/skills/investigating-bugs/SKILL.md b/skills/investigating-bugs/SKILL.md index 8ab337e..2fa607d 100644 --- a/skills/investigating-bugs/SKILL.md +++ b/skills/investigating-bugs/SKILL.md @@ -3,7 +3,7 @@ name: investigating-bugs description: Use when encountering any bug, test failure, build error, or unexpected behavior. --- -# Investigating Bugs +# Investigating bugs Avoid "guess-and-check" coding. Always identify the root cause before making changes. @@ -17,52 +17,53 @@ Avoid "guess-and-check" coding. Always identify the root cause before making cha --- -## Phase 1: Root Cause Investigation +## Phase 1: root cause investigation -Before changing any code: -1. **Read Error Messages and Stack Traces:** Read every line of the error. Note the exact file, line number, and error codes. -2. **Reproduce Consistently:** Identify the exact steps, inputs, or environment needed to trigger the bug. If it cannot be reproduced, gather more logs instead of guessing. - * For flaky tests (pass sometimes, fail under load or only in CI), find the non-determinism before changing anything — don't rerun until it goes green. Arbitrary `sleep`/timeout delays are one common cause (wait on the actual condition, not a guessed duration — see [condition-based waiting](references/condition-based-waiting.md)); a dependency called a non-deterministic number of times or in a non-deterministic order is another. Read [Diagnosing Flaky Tests](references/diagnosing-flaky-tests.md) for the general diagnostic method and cause catalog. -3. **Check Recent Changes:** Run a git diff. Analyze recent commits, dependency additions, or config changes. -4. **Gather Evidence (Multi-Component Systems):** +Complete these steps before changing any code: + +1. **Read error messages and stack traces:** Read every line of the error. Note the exact file, line number, and error codes. +2. **Reproduce consistently:** Identify the exact steps, inputs, or environment needed to trigger the bug. If it cannot be reproduced, gather more logs instead of guessing. + * For flaky tests (pass sometimes, fail under load or only in CI), find the non-determinism before changing anything — don't rerun until it goes green. Arbitrary `sleep`/timeout delays are one common cause (wait on the actual condition, not a guessed duration — see [condition-based waiting](references/condition-based-waiting.md)); a dependency called a non-deterministic number of times or in a non-deterministic order is another. Read [diagnosing flaky tests](references/diagnosing-flaky-tests.md) for the general diagnostic method and cause catalog. +3. **Check recent changes:** Run `git diff`. Analyze recent commits, dependency additions, or configuration changes. +4. **Gather evidence in multi-component systems:** * Log inputs and outputs at every component boundary. - * Instrument the layers step-by-step (e.g., Workflow -> Build Script -> Runtime -> DB) to pinpoint exactly where the state breaks. -5. **Trace Data Flow:** Trace variables backward from the failure point to their source. Fix the bug at the source, not the symptom. + * Instrument the layers step by step (for example, workflow → build script → runtime → database) to pinpoint exactly where the state breaks. +5. **Trace data flow:** Trace variables backward from the failure point to their source. Fix the bug at the source, not the symptom. * When manual tracing dead-ends, instrument the suspect operation: log the key inputs, relevant environment, and a captured stack trace (`new Error().stack`) *just before* it runs. In tests, write to stderr — a logger may be suppressed. Read the captured stack to find the original caller, then remove the instrumentation. --- -## Phase 2: Pattern Analysis +## Phase 2: pattern analysis -1. **Find Working Examples:** Search the codebase for similar logic that functions correctly. -2. **Compare Implementations:** Identify every difference between the working version and the failing version. Do not assume "that difference doesn't matter." -3. **Verify Dependencies & Configs:** Ensure all required modules, configurations, and environment variables are present and correctly configured. +1. **Find working examples:** Search the codebase for similar logic that functions correctly. +2. **Compare implementations:** Identify every difference between the working version and the failing version. Do not assume "that difference doesn't matter." +3. **Verify dependencies and configuration:** Ensure all required modules, configuration, and environment variables are present and correct. --- -## Phase 3: Hypothesis and Testing +## Phase 3: hypothesis and testing -1. **Formulate a Single Hypothesis:** Write down a clear statement: *"I think X is the root cause because Y."* -2. **Test Minimally:** Make the smallest possible change to verify the hypothesis (e.g., add a log, change one value). -3. **Verify & Re-evaluate:** Did the test prove your hypothesis? +1. **Formulate a single hypothesis:** Write down a clear statement: *"I think X is the root cause because Y."* +2. **Test minimally:** Make the smallest possible change to verify the hypothesis (for example, add a log or change one value). +3. **Verify and reevaluate:** Did the test prove your hypothesis? * **Yes:** Proceed to Phase 4. * **No:** Revert the test change completely and formulate a *new* hypothesis. Never stack guess-on-guess. --- -## Phase 4: Implementation & Verification +## Phase 4: implementation and verification -1. **Write a Failing Test Case:** Create an automated test or simple script that consistently triggers the bug. Verify it fails. -2. **Implement the Fix:** Make a single, targeted change that directly addresses the root cause. Do not bundle unrelated refactoring. -3. **Verify the Fix:** Run the test suite. Ensure the new test passes and no regressions are introduced. -4. **The Three-Fix Limit (Architectural Check):** +1. **Write a failing test case:** Create an automated test or simple script that consistently triggers the bug. Verify it fails. +2. **Implement the fix:** Make a single, targeted change that directly addresses the root cause. Do not bundle unrelated refactoring. +3. **Verify the fix:** Run the test suite. Ensure the new test passes and no regressions are introduced. +4. **Apply the three-fix limit (architectural check):** * If you attempt **three separate fixes** and the bug remains: **STOP.** * This is a strong signal that the issue is architectural (e.g., wrong model assumptions, coupled state, race conditions). * Re-evaluate the system architecture and discuss the approach with the user before attempting a fourth patch. --- -## Common Rationalizations +## Common rationalizations | Excuse | Reality | |--------|---------| @@ -75,7 +76,7 @@ Before changing any code: --- -## Red Flags — STOP and Reset +## Red flags — STOP and reset - Writing a fix before reproducing the bug or reading the full stack trace - "Let's just try changing X to see if it works" diff --git a/skills/investigating-bugs/references/condition-based-waiting-example.ts b/skills/investigating-bugs/references/condition-based-waiting-example.ts index 76f679f..5455c77 100644 --- a/skills/investigating-bugs/references/condition-based-waiting-example.ts +++ b/skills/investigating-bugs/references/condition-based-waiting-example.ts @@ -1,20 +1,19 @@ -// Complete, non-standalone reference implementation of condition-based waiting -// utilities. This domain-specific example (an event-driven thread manager) shows -// how the generic waitFor pattern specializes into helpers for a real codebase. +// This non-standalone reference implements condition-based waiting utilities for +// an event-driven thread manager. It shows how the generic `waitFor` pattern can +// specialize into helpers for a concrete domain. import type { ThreadManager } from "~/threads/thread-manager"; import type { LaceEvent, LaceEventType } from "~/threads/types"; /** - * Wait for a specific event type to appear in thread + * Waits for the first event of a specified type in a thread. * - * @param threadManager - The thread manager to query - * @param threadId - Thread to check for events - * @param eventType - Type of event to wait for - * @param timeoutMs - Maximum time to wait (default 5000ms) - * @returns Promise resolving to the first matching event - * - * Example: + * @param threadManager - The thread manager to query. + * @param threadId - The thread to check for events. + * @param eventType - The event type to wait for. + * @param timeoutMs - The maximum wait. Default: 5,000 ms. + * @returns A promise that resolves to the first matching event. + * @example * await waitForEvent(threadManager, agentThreadId, 'TOOL_RESULT'); */ export function waitForEvent( @@ -35,11 +34,11 @@ export function waitForEvent( } else if (Date.now() - startTime > timeoutMs) { reject( new Error( - `Timeout waiting for ${eventType} event after ${timeoutMs}ms`, + `Timeout waiting for ${eventType} event after ${timeoutMs} ms`, ), ); } else { - setTimeout(check, 10); // Poll every 10ms for efficiency + setTimeout(check, 10); // Poll every 10 ms. } }; @@ -48,16 +47,15 @@ export function waitForEvent( } /** - * Wait for a specific number of events of a given type - * - * @param threadManager - The thread manager to query - * @param threadId - Thread to check for events - * @param eventType - Type of event to wait for - * @param count - Number of events to wait for - * @param timeoutMs - Maximum time to wait (default 5000ms) - * @returns Promise resolving to all matching events once count is reached + * Waits for a specified number of events of one type. * - * Example: + * @param threadManager - The thread manager to query. + * @param threadId - The thread to check for events. + * @param eventType - The event type to wait for. + * @param count - The number of events to wait for. + * @param timeoutMs - The maximum wait. Default: 5,000 ms. + * @returns A promise that resolves to all matching events once the count is reached. + * @example * // Wait for 2 AGENT_MESSAGE events (initial response + continuation) * await waitForEventCount(threadManager, agentThreadId, 'AGENT_MESSAGE', 2); */ @@ -80,7 +78,7 @@ export function waitForEventCount( } else if (Date.now() - startTime > timeoutMs) { reject( new Error( - `Timeout waiting for ${count} ${eventType} events after ${timeoutMs}ms (got ${matchingEvents.length})`, + `Timeout waiting for ${count} ${eventType} events after ${timeoutMs} ms (got ${matchingEvents.length})`, ), ); } else { @@ -93,17 +91,16 @@ export function waitForEventCount( } /** - * Wait for an event matching a custom predicate - * Useful when you need to check event data, not just type + * Waits for the first event that matches a custom predicate. + * Use this helper to check event data rather than only the event type. * - * @param threadManager - The thread manager to query - * @param threadId - Thread to check for events - * @param predicate - Function that returns true when event matches - * @param description - Human-readable description for error messages - * @param timeoutMs - Maximum time to wait (default 5000ms) - * @returns Promise resolving to the first matching event - * - * Example: + * @param threadManager - The thread manager to query. + * @param threadId - The thread to check for events. + * @param predicate - A function that returns true when an event matches. + * @param description - A human-readable description for error messages. + * @param timeoutMs - The maximum wait. Default: 5,000 ms. + * @returns A promise that resolves to the first matching event. + * @example * // Wait for TOOL_RESULT with specific ID * await waitForEventMatch( * threadManager, @@ -130,7 +127,7 @@ export function waitForEventMatch( resolve(event); } else if (Date.now() - startTime > timeoutMs) { reject( - new Error(`Timeout waiting for ${description} after ${timeoutMs}ms`), + new Error(`Timeout waiting for ${description} after ${timeoutMs} ms`), ); } else { setTimeout(check, 10); @@ -141,10 +138,9 @@ export function waitForEventMatch( }); } -// Usage example from actual debugging session: +// Comparison example: // -// BEFORE (flaky): -// --------------- +// Fixed delays (flaky): // const messagePromise = agent.sendMessage('Execute tools'); // await new Promise(r => setTimeout(r, 300)); // Hope tools start in 300ms // agent.abort(); @@ -152,13 +148,10 @@ export function waitForEventMatch( // await new Promise(r => setTimeout(r, 50)); // Hope results arrive in 50ms // expect(toolResults.length).toBe(2); // Fails randomly // -// AFTER (reliable): -// ---------------- +// Condition-based waits (reliable): // const messagePromise = agent.sendMessage('Execute tools'); // await waitForEventCount(threadManager, threadId, 'TOOL_CALL', 2); // Wait for tools to start // agent.abort(); // await messagePromise; // await waitForEventCount(threadManager, threadId, 'TOOL_RESULT', 2); // Wait for results -// expect(toolResults.length).toBe(2); // Always succeeds -// -// Result: 60% pass rate → 100%, 40% faster execution +// expect(toolResults.length).toBe(2); diff --git a/skills/investigating-bugs/references/condition-based-waiting.md b/skills/investigating-bugs/references/condition-based-waiting.md index bd15001..d34ab4b 100644 --- a/skills/investigating-bugs/references/condition-based-waiting.md +++ b/skills/investigating-bugs/references/condition-based-waiting.md @@ -1,4 +1,4 @@ -# Condition-Based Waiting +# Condition-based waiting ## Overview @@ -6,13 +6,13 @@ Flaky tests often guess at timing with arbitrary delays. This creates race condi **Core principle:** Wait for the actual condition you care about, not a guess about how long it takes. -## When to Use +## When to use ```mermaid flowchart TD sleep{Test uses setTimeout/sleep?} timing{Testing timing behavior?} - document[Document WHY timeout needed] + document[Document why timeout is needed] use[Use condition-based waiting] sleep -->|yes| timing @@ -30,38 +30,39 @@ flowchart TD - Testing actual timing behavior (debounce, throttle intervals) - Always document WHY if using arbitrary timeout -## Core Pattern +## Core pattern ```typescript // ❌ BEFORE: Guessing at timing -await new Promise(r => setTimeout(r, 50)); +await new Promise((resolve) => setTimeout(resolve, 50)); const result = getResult(); expect(result).toBeDefined(); // ✅ AFTER: Waiting for condition -await waitFor(() => getResult() !== undefined); +await waitFor(() => getResult() !== undefined, 'result to become available'); const result = getResult(); expect(result).toBeDefined(); ``` -## Quick Patterns +## Quick patterns | Scenario | Pattern | |----------|---------| -| Wait for event | `waitFor(() => events.find(e => e.type === 'DONE'))` | -| Wait for state | `waitFor(() => machine.state === 'ready')` | -| Wait for count | `waitFor(() => items.length >= 5)` | -| Wait for file | `waitFor(() => fs.existsSync(path))` | -| Complex condition | `waitFor(() => obj.ready && obj.value > 10)` | +| Wait for event | `waitFor(() => events.find((event) => event.type === 'DONE'), 'DONE event')` | +| Wait for state | `waitFor(() => machine.state === 'ready', 'ready state')` | +| Wait for count | `waitFor(() => items.length >= 5, 'at least five items')` | +| Wait for file | `waitFor(() => fs.existsSync(path), 'file to exist')` | +| Complex condition | `waitFor(() => obj.ready && obj.value > 10, 'ready object with value over 10')` | ## Implementation -Generic polling function: +The following generic polling function reports the condition it timed out waiting for: + ```typescript async function waitFor( condition: () => T | undefined | null | false, description: string, - timeoutMs = 5000 + timeoutMs = 5000, ): Promise { const startTime = Date.now(); @@ -70,37 +71,37 @@ async function waitFor( if (result) return result; if (Date.now() - startTime > timeoutMs) { - throw new Error(`Timeout waiting for ${description} after ${timeoutMs}ms`); + throw new Error(`Timeout waiting for ${description} after ${timeoutMs} ms`); } - await new Promise(r => setTimeout(r, 10)); // Poll every 10ms + await new Promise((resolve) => setTimeout(resolve, 10)); } } ``` -See the [complete condition-based waiting implementation](./condition-based-waiting-example.ts) for domain-specific helpers (`waitForEvent`, `waitForEventCount`, `waitForEventMatch`). +See the [condition-based waiting reference implementation](./condition-based-waiting-example.ts) for domain-specific helpers (`waitForEvent`, `waitForEventCount`, `waitForEventMatch`). -## Common Mistakes +## Common mistakes -**❌ Polling too fast:** `setTimeout(check, 1)` - wastes CPU -**✅ Fix:** Poll every 10ms +- **Problem:** Polling too fast with `setTimeout(check, 1)` wastes CPU. + **Fix:** Poll every 10 ms. -**❌ No timeout:** Loop forever if condition never met -**✅ Fix:** Always include timeout with clear error +- **Problem:** Omitting a timeout can leave the loop running forever. + **Fix:** Include a timeout with a clear error. -**❌ Stale data:** Cache state before loop -**✅ Fix:** Call getter inside loop for fresh data +- **Problem:** Caching state before the loop makes the condition stale. + **Fix:** Call the getter inside the loop. -## When Arbitrary Timeout IS Correct +## When an arbitrary timeout is correct ```typescript -// Tool ticks every 100ms - need 2 ticks to verify partial output -await waitForEvent(manager, 'TOOL_STARTED'); // First: wait for condition -await new Promise(r => setTimeout(r, 200)); // Then: wait for timed behavior -// 200ms = 2 ticks at 100ms intervals - documented and justified +await waitForEvent(manager, 'TOOL_STARTED'); +// Wait for two 100 ms ticks after the tool starts. +await new Promise((resolve) => setTimeout(resolve, 200)); ``` -**Requirements:** -1. First wait for triggering condition -2. Based on known timing (not guessing) -3. Comment explaining WHY +Use an arbitrary timeout only when all of these requirements hold: + +- Wait for the triggering condition first. +- Base the delay on known timing rather than a guess. +- Explain why the delay is necessary in a comment. diff --git a/skills/investigating-bugs/references/diagnosing-flaky-tests.md b/skills/investigating-bugs/references/diagnosing-flaky-tests.md index 7d3f995..10aea89 100644 --- a/skills/investigating-bugs/references/diagnosing-flaky-tests.md +++ b/skills/investigating-bugs/references/diagnosing-flaky-tests.md @@ -1,4 +1,4 @@ -# Diagnosing Flaky Tests +# Diagnosing flaky tests ## Overview @@ -10,7 +10,7 @@ with the code, and find the mechanism before changing anything. **Core principle:** A test may only assume what the code under test actually guarantees. Flakiness is the gap between the two. -## The Diagnostic Method +## Diagnostic method Work these in order, before proposing a fix: @@ -43,13 +43,13 @@ Work these in order, before proposing a fix: > A rerun that turns green tells you nothing about root cause. You have not found the cause until > you can make the test fail on demand. -## Cause Catalog +## Cause catalog | Cause class | How to recognize it | Where the fix lives | |---|---|---| | Timing guesses (arbitrary `sleep`/`setTimeout`) | Passes fast, fails under load/CI; fixed-delay waits in the test | [Condition-based waiting](./condition-based-waiting.md) | -| Non-deterministic call count/order vs. order-dependent stubs/assertions | A mocked function returns `undefined`/its default mid-test; an assertion reads "the last call" / "call N" and gets the wrong one | "Non-deterministic call count/order" (below) + the `slow-powers:working-with-tdd` skill's [testing anti-patterns reference](../../working-with-tdd/references/testing-anti-patterns.md) → *Order-Dependent Mocks and Assertions* | -| Cross-test state pollution | Passes alone, fails in-suite or under a particular order — depends on a neighbor test's leftover state | reset/isolate shared state per test (mocks, module/global singletons, DB, fake timers); make setup/teardown order-independent | +| Non-deterministic call count/order vs. order-dependent stubs/assertions | A mocked function returns `undefined`/its default mid-test; an assertion reads "the last call" / "call N" and gets the wrong one | "Non-deterministic call count/order" (below) + the `slow-powers:working-with-tdd` skill's [testing anti-patterns reference](../../working-with-tdd/references/testing-anti-patterns.md) → *Order-dependent mocks and assertions* | +| Cross-test state pollution | Passes alone, fails in-suite or under a particular order — depends on a neighbor test's leftover state | Reset or isolate shared state per test (mocks, module/global singletons, DB, fake timers); make setup and teardown order-independent | ### Non-deterministic call count / order @@ -66,4 +66,4 @@ error/empty state — or the indexed assertion reads the wrong call. Both depend or an assertion that received a plausible-but-wrong value (the value from a *different* call than the one intended). -**Fix (prevention):** the `slow-powers:working-with-tdd` skill's [testing anti-patterns reference](../../working-with-tdd/references/testing-anti-patterns.md) → *Order-Dependent Mocks and Assertions*. In short: stub by **input** (return the right value for any matching call, a sensible default otherwise, never fall through to `undefined`), and assert a matching call **happened** rather than reading a fixed index. +**Prevention:** the `slow-powers:working-with-tdd` skill's [testing anti-patterns reference](../../working-with-tdd/references/testing-anti-patterns.md) → *Order-dependent mocks and assertions*. In short: stub by **input** (return the right value for any matching call, a sensible default otherwise, never fall through to `undefined`), and assert a matching call **happened** rather than reading a fixed index. diff --git a/skills/verifying-development-work/SKILL.md b/skills/verifying-development-work/SKILL.md index 8a46014..8c27a27 100644 --- a/skills/verifying-development-work/SKILL.md +++ b/skills/verifying-development-work/SKILL.md @@ -3,7 +3,7 @@ name: verifying-development-work description: Use before claiming any task is complete, fixed, or passing, and before handing finished work back to the user. --- -# Verifying Development Work +# Verifying development work Claiming work is complete without verification is an assumption, not a fact. Handing work back without a review pass is a guess that the diff is the right diff. Finishing requires both: verification proves the code runs; review proves it's the right code. @@ -13,7 +13,7 @@ Claiming work is complete without verification is an assumption, not a fact. Han --- -## Finishing: Review, Verify, Then Handoff +## Finish: review, verify, then hand off When you believe the work itself is done, run these finishing phases **in order**. Review comes first so any fixes happen before the evidence you hand back; verification comes next so the claim covers the returned code; integration choices come last because they belong to the user. @@ -33,11 +33,11 @@ The last box is its own gate; the "Don't Finish the Branch Unilaterally" section --- -## The Gate Function +## The Gate function The Gate Function is your discipline at *every* completion claim — each "fixed", "passing", or "done" mid-task — and it is how Phase 2 of finishing establishes its evidence. Before claiming any task is finished, making a success claim, or declaring a bug fixed: -1. **IDENTIFY:** What exact command or output proves this claim? (e.g., test command, compiler output, linter check). +1. **IDENTIFY:** What exact command or output proves this claim? (For example, a test command, compiler output, or linter check.) 2. **ESTABLISH FRESH EVIDENCE:** Use fresh evidence you personally observed in the primary session, or run the command now. "Fresh" means the output proves the current code state: full relevant command, visible output, exit code known, and no behavior-changing edits since it ran. 3. **READ:** Review the full output, verify exit code is `0`, and check for warning logs. 4. **VERIFY:** Does the output confirm success? @@ -46,13 +46,13 @@ The Gate Function is your discipline at *every* completion claim — each "fixed Current-session evidence can count. Do **not** rerun a passing check merely because this skill loaded after you already ran the right command and nothing behavior-changing happened afterward. Do rerun when the evidence is inherited, stale, incomplete, or separated from the returned code by later behavior changes. -**Unverifiable Claims:** Some claims may not be verifiable - you don't have access to the necessary output, it requires visual review and you lack that kind of reference, and so on. If you cannot verify a claim, clearly state so, and describe to the user what manual verification steps they need to take. +**Unverifiable claims:** Some claims may not be verifiable — you might lack access to the necessary output or a reference for visual review. If you cannot verify a claim, state that limitation clearly and describe the manual verification steps the user needs to take. --- -## Core Verification Types +## Core verification types -| Success Claim | What is Required | What is NOT Sufficient | +| Success claim | What is required | What is not sufficient | | :--- | :--- | :--- | | **"Tests are passing"** | Current-session test output showing `0 failures` for the relevant suite. | "They should pass," someone else's paste, or a stale run. | | **"Linter is clean"** | Current-session linter output showing `0 errors` and `0 warnings`. | Assumed clean because it compiled. | @@ -62,7 +62,7 @@ Current-session evidence can count. Do **not** rerun a passing check merely beca --- -## When Existing Evidence Counts +## When existing evidence counts Use already-produced evidence only when **all** of these are true: @@ -75,7 +75,7 @@ Evidence does **not** count when it came from the user, a teammate, a subagent, --- -## Don't Finish the Branch Unilaterally +## Don't finish the branch unilaterally Verified, reviewed work is still *your* checkpoint, not a decision to merge. Integrating, publishing, or discarding work is the user's call. @@ -85,7 +85,7 @@ Verified, reviewed work is still *your* checkpoint, not a decision to merge. Int --- -## Common Rationalizations +## Common rationalizations | Excuse | Reality | |--------|---------| @@ -103,7 +103,7 @@ Verified, reviewed work is still *your* checkpoint, not a decision to merge. Int --- -## Red Flags — STOP and Verify +## Red flags — STOP and verify - "Should work now" / "probably fixed" / "seems correct" / "looks correct" - Claiming completion before establishing verification evidence diff --git a/skills/verifying-development-work/references/code-review.md b/skills/verifying-development-work/references/code-review.md index a8f087f..b82280f 100644 --- a/skills/verifying-development-work/references/code-review.md +++ b/skills/verifying-development-work/references/code-review.md @@ -1,4 +1,4 @@ -# Reviewing the Code +# Review the code This is **phase 1** of the [finishing sequence](../SKILL.md) — the diff review. Review code and comments here, fix or flag the findings that @@ -15,7 +15,7 @@ that's louder than the change it covers is the failure this guidance exists to prevent. Do the review however your harness makes natural — read the diff inline, or -dispatch it to a general purpose subagent. +dispatch it to a general-purpose subagent. --- @@ -46,7 +46,7 @@ tend to hide. Spend attention where this particular diff warrants it. After reading the diff, check whether any file you added to is now over 500 lines. If one is, it must go through the long-file review in -[long-file review](./long-files.md) before you finish — one file at a time. That's a +[the long-file review](./long-files.md) before you finish — one file at a time. That's a mandate to *review*, which can conclude "no change needed"; what it forbids is handing back a newly-grown long file silently. Files you didn't grow are out of scope. @@ -94,7 +94,7 @@ wrote them. Keep only comments that earn their place: rewritten tightly if needed, and delete the surrounding narration. When a comment you're keeping or writing needs structure or phrasing guidance — -doc-block conventions, evergreen wording — `slow:powers:writing-technical-docs` owns it; +doc-block conventions, evergreen wording — `slow-powers:writing-technical-docs` owns it; its references cover doc types and formatting. Comment-only edits do not change behavior. They do not require re-verification @@ -103,7 +103,7 @@ human to read. --- -## Then: address the findings — and freeze behavior +## Address the findings and freeze behavior Fix or explicitly flag each finding you kept. Any behavior fix changes the code — so make all behavior-changing fixes *now*, in this phase. When you're done, diff --git a/skills/verifying-development-work/references/long-files.md b/skills/verifying-development-work/references/long-files.md index 2bd14c3..7a2ed8c 100644 --- a/skills/verifying-development-work/references/long-files.md +++ b/skills/verifying-development-work/references/long-files.md @@ -1,4 +1,4 @@ -# Reviewing a File Your Change Made Long +# Review a file your change grew past 500 lines This is a sub-process of **phase 1** of the finishing sequence in [the skill](../SKILL.md), reached from the [code-review reference](./code-review.md). It @@ -117,7 +117,7 @@ refactor is the trap, not the fix. --- -## Common Rationalizations +## Common rationalizations | Excuse | Reality | |--------|---------| @@ -130,7 +130,7 @@ refactor is the trap, not the fix. --- -## Red Flags — STOP +## Red flags — STOP - About to hand back a file your change grew past 500 lines without a change or a declared exception. diff --git a/skills/working-in-isolation/SKILL.md b/skills/working-in-isolation/SKILL.md index 9a677b3..ce5d711 100644 --- a/skills/working-in-isolation/SKILL.md +++ b/skills/working-in-isolation/SKILL.md @@ -3,7 +3,7 @@ name: working-in-isolation description: Use when you're about to start changing files in a repository — code, docs, config, or content, whether a feature, bugfix, refactor, or docs/config update — to establish an isolated workspace so your work doesn't collide with existing or in-progress work. --- -# Working in Isolation +# Working in isolation Before changing anything in a repository — code, docs, config, or other files — make sure your work lands somewhere it won't collide with existing or in-progress @@ -34,10 +34,9 @@ git worktree list # >1 entry = worktrees already exist > **Hard rule: never make changes while on `dev` / `main` / `master`.** If you > find yourself on a base branch, branch (rule 3) or worktree (rule 2) first. -## Creating a worktree (rule 2) +## Create a worktree (rule 2) -Prefer your harness's **native git worktree tool** if it exists. Note that the tool my be deferred or lazily-loaded -Otherwise fall back to a git worktree: +Prefer your harness's **native Git worktree capability** if it exists. The capability may be deferred or lazily loaded. Otherwise, fall back to a Git worktree: ```bash git worktree add .worktrees/ -b @@ -45,11 +44,11 @@ cd .worktrees/ ``` Keep the worktree out of version control: if `.worktrees/` isn't already -git-ignored, add it to `.gitignore` and commit that first. If worktree creation +ignored by Git, add it to `.gitignore` and commit that first. If worktree creation fails (sandbox or permission limits), say so and fall back to checking out a branch in place (rule 3). -## Creating a branch (rule 3) +## Create a branch (rule 3) After syncing the base branch with origin, create the new branch from the current `HEAD` with no upstream tracking: @@ -62,12 +61,12 @@ Do not create the branch from `origin/dev`, `origin/main`, or `origin/master`. `--no-track` keeps the new branch without an upstream until the user pushes it explicitly. -## After the workspace is set +## Set up the workspace Install dependencies and run the existing test suite once, to confirm a clean baseline before you write anything. -Use the project-appropriate commands to verify the baseline is clean - lint, test, build. +Use the project-appropriate commands to verify the baseline is clean — lint, test, and build. If the baseline is already failing, report it before starting — you need to know which failures you introduced. diff --git a/skills/working-with-tdd/SKILL.md b/skills/working-with-tdd/SKILL.md index 34fb6ca..495caa6 100644 --- a/skills/working-with-tdd/SKILL.md +++ b/skills/working-with-tdd/SKILL.md @@ -3,7 +3,7 @@ name: working-with-tdd description: Use when implementing any feature, refactoring, or writing a bugfix. --- -# Test-Driven Development (TDD) +# Test-driven development (TDD) Write the test first. Watch it fail. Write minimal code to pass. Refactor. @@ -19,31 +19,31 @@ Write production code before the test? Delete it. Start over. Do not keep it for --- -## Red-Green-Refactor Cycle +## Red-green-refactor cycle -1. **RED — Write a Failing Test:** +1. **RED — Write a failing test:** * Write one minimal, focused test showing what the behavior *should* do. * Use real code and real inputs; avoid mocks unless absolutely unavoidable. Before committing a test, scan it against the **Testing anti-patterns** table below; if it matches a row, read the named section of the [testing anti-patterns reference](references/testing-anti-patterns.md) first. -2. **Verify RED — Watch It Fail:** +2. **Verify RED — Watch it fail:** * Run the test command: `npm test` / `pytest` / `go test`. - * **MANDATORY:** Verify it fails for the expected reason (e.g., function not defined, value incorrect), not due to a typo or build error. -3. **GREEN — Write Minimal Code:** + * **MANDATORY:** Verify it fails for the expected reason (for example, a function is undefined or a value is incorrect), not because of a typo or build error. +3. **GREEN — Write minimal code:** * Write the simplest possible implementation to make the test pass. - * Avoid over-engineering or speculative optimization (YAGNI). -4. **Verify GREEN — Watch It Pass:** + * Avoid over-engineering or speculative optimization. +4. **Verify GREEN — Watch it pass:** * Run the test suite. Verify the test passes, and no regressions are introduced. -5. **REFACTOR — Clean Up:** +5. **REFACTOR — Clean up:** * Clean up names, remove duplication, and extract helper methods. * Keep the test suite green. Do not add new behavior during refactoring. --- -## Example: Code vs. Mock Testing +## Example: code vs. mock testing -### Good (Focuses on real behavior): +### Good (focuses on real behavior) ```typescript test('retries failed operations 3 times', async () => { let attempts = 0; @@ -58,7 +58,7 @@ test('retries failed operations 3 times', async () => { }); ``` -### Bad (Focuses on mock implementation detail): +### Bad (focuses on mock implementation detail) ```typescript test('retry works', async () => { const mock = jest.fn() @@ -78,17 +78,17 @@ While writing or changing a test, check it against this table. If a row matches, | If your test… | Anti-pattern | Section to read | |---|---|---| -| asserts on a mock / `*-mock` element instead of real output | testing mock behavior | *Testing Mock Behavior* | -| needs a method on a production class that only tests call | test-only methods in production | *Test-Only Methods in Production* | -| mocks a method without knowing its side effects | mocking without understanding | *Mocking Without Understanding* | -| uses a mock with only the fields you happen to need | incomplete mocks | *Incomplete Mocks* | -| is written after the implementation, with no failing test first | tests as afterthought | *Tests as Afterthought* | -| stubs by call order (`...Once` chains) or asserts "call N" / "the last call" | order-dependent mocks/assertions | *Order-Dependent Mocks and Assertions* | -| has more mock setup than test logic | over-complex mocks | *When Mocks Become Too Complex* | +| asserts on a mock / `*-mock` element instead of real output | testing mock behavior | *Testing mock behavior* | +| needs a method on a production class that only tests call | test-only methods in production | *Test-only methods in production* | +| mocks a method without knowing its side effects | mocking without understanding | *Mocking without understanding* | +| uses a mock with only the fields you happen to need | incomplete mocks | *Incomplete mocks* | +| is written after the implementation, with no failing test first | tests as afterthought | *Tests as afterthought* | +| stubs by call order (`...Once` chains) or asserts "call N" / "the last call" | order-dependent mocks/assertions | *Order-dependent mocks and assertions* | +| has more mock setup than test logic | over-complex mocks | *When mocks become too complex* | --- -## Common Rationalizations +## Common rationalizations | Excuse | Reality | |--------|---------| @@ -101,7 +101,7 @@ While writing or changing a test, check it against this table. If a row matches, --- -## Red Flags — STOP and start over +## Red flags — STOP and start over - Code before test - "I already manually tested it" diff --git a/skills/working-with-tdd/references/testing-anti-patterns.md b/skills/working-with-tdd/references/testing-anti-patterns.md index 977263f..9b56957 100644 --- a/skills/working-with-tdd/references/testing-anti-patterns.md +++ b/skills/working-with-tdd/references/testing-anti-patterns.md @@ -1,10 +1,10 @@ -# Testing Anti-Patterns +# Testing anti-patterns -**Load this reference when:** a test you're writing or changing matches one of the anti-patterns below — the `working-with-tdd` SKILL's "Testing anti-patterns" table points here — or any time you're adding mocks or other test scaffolding. Each anti-pattern carries a **Gate**: check it before you commit the test. +**Load this reference when:** a test you're writing or changing matches one of the anti-patterns below — the `working-with-tdd` skill's "Testing anti-patterns" table points here — or any time you're adding mocks or other test scaffolding. Each anti-pattern carries a **gate**: check it before you commit the test. ## Overview -This is a catalog of recurring testing anti-patterns — the ways a test stops verifying real behavior, or quietly couples itself to test scaffolding instead of the code under test. Mocks are the most common source (anti-patterns 1, 3, 4, and 6 all involve them), but not the only one: test-only methods leak into production (2), and tests get bolted on after the fact (5). +This is a catalog of recurring testing anti-patterns — the ways a test stops verifying real behavior, or quietly couples itself to test scaffolding instead of the code under test. Mocks are the most common source (anti-patterns 1, 3, 4, and 6 all involve them), but not the only one: test-only methods leak into production (2), and tests get added after the implementation (5). **Core principle:** Test what the code does, not what your test scaffolding does. If a test passes because of how a mock, fixture, or helper is set up — rather than what the code under test actually does — it's verifying the wrong thing. @@ -16,7 +16,7 @@ This is a catalog of recurring testing anti-patterns — the ways a test stops v 3. NEVER mock without understanding dependencies ``` -## Anti-Pattern 1: Testing Mock Behavior +## Anti-pattern 1: testing mock behavior **The violation:** ```typescript @@ -43,7 +43,7 @@ test('renders sidebar', () => { **Gate:** Before asserting on any mock element, ask "Am I testing real component behavior or just mock existence?" If the latter, delete the assertion or unmock the component. -## Anti-Pattern 2: Test-Only Methods in Production +## Anti-pattern 2: test-only methods in production **The violation:** ```typescript @@ -80,7 +80,7 @@ afterEach(() => cleanupSession(session)); **Gate:** Before adding any method to a production class, ask "Is this only used by tests?" — if so, put it in test utilities instead. Then ask "Does this class own this resource's lifecycle?" — if not, it's the wrong class for the method. -## Anti-Pattern 3: Mocking Without Understanding +## Anti-pattern 3: mocking without understanding **The violation:** ```typescript @@ -112,11 +112,11 @@ test('detects duplicate server', () => { **Gate:** Before mocking, ask what side effects the real method has and whether the test depends on any of them. If it does, mock at a lower level (the actual slow/external operation), not the high-level method the test relies on. If you're unsure what the test needs, run it against the real implementation first, observe what has to happen, then add minimal mocking at the right level. -## Anti-Pattern 4: Incomplete Mocks +## Anti-pattern 4: incomplete mocks **The violation:** ```typescript -// ❌ BAD: Partial mock - only fields you think you need +// ❌ BAD: Partial mock — only fields you think you need const mockResponse = { status: 'success', data: { userId: '123', name: 'Alice' } @@ -143,7 +143,7 @@ const mockResponse = { **Gate:** Before creating a mock response, check what fields the real API returns (docs/examples) and include all of them — if you're mocking it, you must understand the entire structure. When uncertain, include all documented fields. -## Anti-Pattern 5: Tests as Afterthought +## Anti-pattern 5: tests as afterthought ``` ✅ Implementation complete @@ -153,7 +153,7 @@ const mockResponse = { Testing is part of implementation, not an optional follow-up — you can't claim complete without tests, and TDD would have caught this. Write the failing test first; see the [TDD cycle](../SKILL.md). -## Anti-Pattern 6: Order-Dependent Mocks and Assertions +## Anti-pattern 6: order-dependent mocks and assertions **The violation:** ```typescript @@ -169,7 +169,7 @@ const lastCall = fetchUser.mock.calls.at(-1); expect(lastCall[1]).toMatchObject({ page: 2 }); ``` -**Why this is wrong:** The code under test doesn't *guarantee* it calls the dependency exactly twice, in this order — retries, cache refetches/revalidation, re-renders, and effects firing on mount can add or reorder calls. When an extra call appears, the `Once` queue runs dry and the next real call gets the mock's default — usually `undefined` — which cascades into an error or empty state that looks like a product bug. Reading "the last call" or "call N" assumes that call is the one you care about, but an unmodeled refetch makes the assertion read the wrong one. The result is a test that passes or fails on timing — flaky. It's the same footgun in any framework: Sinon `onCall(n)`, a Python `side_effect=[...]` list, consecutive Mockito `thenReturn(a, b)`. +**Why this is wrong:** The code under test doesn't *guarantee* it calls the dependency exactly twice, in this order — retries, cache refetches/revalidation, re-renders, and effects firing on mount can add or reorder calls. When an extra call appears, the `Once` queue runs dry and the next real call gets the mock's default — usually `undefined` — which cascades into an error or empty state that looks like a product bug. Reading "the last call" or "call N" assumes that call is the one you care about, but an unmodeled refetch makes the assertion read the wrong one. The result is a test that passes or fails on timing — flaky. The same failure mode appears in any framework: Sinon `onCall(n)`, a Python `side_effect=[...]` list, consecutive Mockito `thenReturn(a, b)`. **The fix:** ```typescript @@ -187,19 +187,19 @@ expect(fetchUser).toHaveBeenCalledWith( **Gate:** Before using a `...Once` queue, a per-call-index stub, or a "last call / call N" assertion, ask "Can the code under test call this a different number of times, or in a different order, than I expect?" (retries? cache refetch/revalidation? re-render? effect on mount? parallelism?) If yes — or you're unsure — stub by input: return the right value for any matching call and a sensible default for the rest, never letting the stub return `undefined` for a value the code consumes; and assert by matching (`toHaveBeenCalledWith` / `objectContaining`), not by index. Also reduce non-determinism at the source: in tests, disable retries and background revalidation, and control timers, so the call count is predictable. -When this footgun surfaces as a flaky CI failure, use the `slow-powers:investigating-bugs` [flaky-test diagnostic](../../investigating-bugs/references/diagnosing-flaky-tests.md). +When this failure mode surfaces as a flaky CI failure, use the `slow-powers:investigating-bugs` [flaky-test diagnostic](../../investigating-bugs/references/diagnosing-flaky-tests.md). -## When Mocks Become Too Complex +## When mocks become too complex **Warning signs:** mock setup longer than the test logic, mocking everything to make the test pass, mocks missing methods the real components have, or tests that break when the mock changes. **Consider:** integration tests with real components are often simpler than complex mocks. -## TDD Prevents These Anti-Patterns +## TDD prevents these anti-patterns -Writing the test first forces you to think about what you're actually testing; watching it fail confirms it tests real behavior, not mocks; a minimal implementation keeps test-only methods from creeping in; and using real dependencies shows you what the test actually needs before you mock. If you're testing mock behavior, you added mocks without watching the test fail against real code first — you violated TDD. +Writing the test first forces you to think about what you're actually testing; watching it fail confirms it tests real behavior, not mocks; a minimal implementation keeps test-only methods out of production; and using real dependencies shows you what the test actually needs before you mock. If you're testing mock behavior, you added mocks without watching the test fail against real code first — you violated TDD. -## Red Flags +## Red flags - Assertion checks for `*-mock` test IDs - Methods only called in test files diff --git a/skills/writing-skills/SKILL.md b/skills/writing-skills/SKILL.md index 73c0575..aa5d03e 100644 --- a/skills/writing-skills/SKILL.md +++ b/skills/writing-skills/SKILL.md @@ -3,7 +3,7 @@ name: writing-skills description: Use when creating new skills or editing existing skills. Drafting only; see slow-powers:evaluating-skills for measuring whether the draft works. --- -# Writing Skills +# Writing skills ## Overview @@ -80,7 +80,7 @@ Use these terms as the canonical vocabulary reference. | **Capability** | A described action ("search file contents") | A platform tool name ("Grep") | | **Load-bearing property** | A property a capability must have for the workflow to work | (no shorter form) | -## SKILL.md structure +## `SKILL.md` structure ```markdown --- @@ -88,10 +88,10 @@ name: skill-name-with-hyphens description: Use when [specific triggering conditions and symptoms] --- -# Skill Name +# Skill name ## Overview — what is this? Core principle in 1-2 sentences. -## When to use — symptoms and use cases; when NOT to use. +## When to use — symptoms and use cases; when not to use. ## Core pattern — before/after comparison (techniques/patterns). ## Quick reference — table or bullets for scanning common operations. ## Implementation — inline code for simple patterns; link a file for heavy reference. @@ -106,7 +106,7 @@ description: Use when [specific triggering conditions and symptoms] ## Building blocks -The blocks below help structure a SKILL.md file. Use the ones that fit - not every skill +The blocks below help structure a `SKILL.md` file. Use the ones that fit — not every skill needs all of them. These aren't limiters, and your skill should contain the content it needs. Each block does one job: @@ -217,7 +217,7 @@ Use the skill's qualified name with an explicit requirement marker: Don't repeat what another skill says — link to it. -## Conciseness & file organization +## Conciseness and file organization Once a skill loads, every token competes with conversation history. Keep the body lean: aim for **≤200 lines** for frequently-loaded internal skills, and treat **500 lines / 5,000 tokens** as @@ -225,13 +225,13 @@ the hard ceiling for any skill. Move details to tool help ("Run ` --help` listing every flag), cross-reference instead of repeating, and compress examples to one good pair. -Use progressive disclosure for anything heavy: SKILL.md is the always-loaded overview; bulky +Use progressive disclosure for anything heavy: the `SKILL.md` file is the always-loaded overview; bulky material lives in separate files the agent loads on demand. Tell the agent *when* to load each ("Read `references/api-errors.md` if the API returns non-200") rather than a generic "see references/". ### Skill directory layout -A skill always has `SKILL.md`. Add any of these peer directories when their contents are needed: +A skill always has a `SKILL.md` file. Add any of these peer directories when their contents are needed: | Directory | Put here | Guidance | |---|---|---| @@ -255,10 +255,9 @@ Create companion files when they serve one of the roles above, not merely to sho ## Rationalization-proofing for discipline skills -Skills that enforce discipline (TDD, verifying-development-work, designing-before-coding) +Skills that enforce discipline (TDD, verifying-development-work, hardening-plans) must survive pressure — agents find loopholes under time, sunk-cost, or authority pressure. -Drafting an enforceable rule differs from drafting a guideline. The research backs this up: -persuasion techniques more than double LLM compliance under pressure. See the [persuasion principles](references/persuasion-principles.md) for the seven principles, when each applies, and citations (Cialdini, 2021; Meincke et al., 2025). +Drafting an enforceable rule differs from drafting a guideline. In one controlled study, persuasion cues more than doubled compliance for the tested model and prompts. See the [persuasion principles](references/persuasion-principles.md) for the seven principles, when each applies, the study's limits, and citations (Cialdini, 2021; Meincke et al., 2026). **Close every loophole explicitly.** State the rule, then forbid the specific workarounds you can predict — the agent will reach for the ambiguity under pressure. diff --git a/skills/writing-skills/references/persuasion-principles.md b/skills/writing-skills/references/persuasion-principles.md index a63ca18..dbaed50 100644 --- a/skills/writing-skills/references/persuasion-principles.md +++ b/skills/writing-skills/references/persuasion-principles.md @@ -1,14 +1,14 @@ -# Persuasion Principles for Skill Design +# Persuasion principles for skill design -This reference supports the [Writing Skills](../SKILL.md) rationalization-proofing guidance. +This reference supports the [Writing skills](../SKILL.md) rationalization-proofing guidance. ## Overview -LLMs respond to the same persuasion principles as humans. Understanding this psychology helps you design more effective skills - not to manipulate, but to ensure critical practices are followed even under pressure. +Social-persuasion cues can affect how a language model follows instructions. Understanding these cues helps you design more effective skills — not to manipulate, but to ensure critical practices are followed even under pressure. -**Research foundation:** Meincke et al. (2025) tested 7 persuasion principles with N=28,000 AI conversations. Persuasion techniques more than doubled compliance rates (33% → 72%, p < .001). +**Research foundation:** [Meincke et al.](https://www.pnas.org/doi/10.1073/pnas.2535868123) tested seven persuasion principles in 28,000 conversations with GPT-4o mini and two objectionable requests. In that setting, the cues increased average compliance from 33.3% to 72.0% (*p* < .001). -## The Seven Principles +## The seven principles ### 1. Authority **What it is:** Deference to expertise, credentials, or official sources. @@ -67,7 +67,7 @@ LLMs respond to the same persuasion principles as humans. Understanding this psy ❌ You can review code when convenient. ``` -### 4. Social Proof +### 4. Social proof **What it is:** Conformity to what others do or what's considered normal. **How it works in skills:** @@ -108,7 +108,7 @@ LLMs respond to the same persuasion principles as humans. Understanding this psy **What it is:** Obligation to return benefits received. **How it works:** -- Use sparingly - can feel manipulative +- Use sparingly — it can feel manipulative - Rarely needed in skills **When to avoid:** @@ -125,16 +125,16 @@ LLMs respond to the same persuasion principles as humans. Understanding this psy **When to avoid:** - Always for discipline enforcement -## Principle Combinations by Skill Type +## Principle combinations by skill type -| Skill Type | Use | Avoid | +| Skill type | Use | Avoid | |------------|-----|-------| | Discipline-enforcing | Authority + Commitment + Social Proof | Liking, Reciprocity | | Guidance/technique | Moderate Authority + Unity | Heavy authority | | Collaborative | Unity + Commitment | Authority, Liking | | Reference | Clarity only | All persuasion | -## Why This Works: The Psychology +## Why this works: the psychology **Bright-line rules reduce rationalization:** - "YOU MUST" removes decision fatigue @@ -146,13 +146,12 @@ LLMs respond to the same persuasion principles as humans. Understanding this psy - "When X, do Y" more effective than "generally do Y" - Reduces cognitive load on compliance -**LLMs are parahuman:** -- Trained on human text containing these patterns -- Authority language precedes compliance in training data -- Commitment sequences (statement → action) frequently modeled -- Social proof patterns (everyone does X) establish norms +**Models learn these patterns from human language:** +- Training data contains authority, commitment, and social-proof patterns. +- Explicit cues can make those learned patterns salient during instruction following. +- The effect depends on the model and context; treat the principles as design inputs, not guarantees. -## Ethical Use +## Ethical use **Legitimate:** - Ensuring critical practices are followed @@ -166,19 +165,18 @@ LLMs respond to the same persuasion principles as humans. Understanding this psy **The test:** Would this technique serve the user's genuine interests if they fully understood it? -## Research Citations +## Research citations **Cialdini, R. B. (2021).** *Influence: The Psychology of Persuasion (New and Expanded).* Harper Business. - Seven principles of persuasion - Empirical foundation for influence research -**Meincke, L., Shapiro, D., Duckworth, A. L., Mollick, E., Mollick, L., & Cialdini, R. (2025).** Call Me A Jerk: Persuading AI to Comply with Objectionable Requests. University of Pennsylvania. -- Tested 7 principles with N=28,000 LLM conversations -- Compliance increased 33% → 72% with persuasion techniques -- Authority, commitment, scarcity most effective -- Validates parahuman model of LLM behavior +**Meincke, L., Shapiro, D., Duckworth, A. L., Mollick, E., Mollick, L., & Cialdini, R. (2026).** [*Call me a jerk: Persuading AI to comply with objectionable requests*](https://www.pnas.org/doi/10.1073/pnas.2535868123). *Proceedings of the National Academy of Sciences*. +- Tested seven principles in 28,000 GPT-4o mini conversations. +- Increased average compliance from 33.3% to 72.0% for the two tested requests. +- Found authority, commitment, and scarcity most effective in that experiment. -## Quick Reference +## Quick reference When designing a skill, ask: diff --git a/skills/writing-technical-docs/SKILL.md b/skills/writing-technical-docs/SKILL.md index 2c31856..370bd87 100644 --- a/skills/writing-technical-docs/SKILL.md +++ b/skills/writing-technical-docs/SKILL.md @@ -3,7 +3,7 @@ name: writing-technical-docs description: Use when writing or editing technical documentation of any size — code comments, doc blocks, commit messages, PR descriptions, review comments, READMEs, design docs, wikis, or user guides — whether drafting new content or revising existing text. --- -# Writing Technical Docs +# Writing technical docs ## Overview diff --git a/skills/writing-technical-docs/references/doc-types.md b/skills/writing-technical-docs/references/doc-types.md index 6af4015..95be6dd 100644 --- a/skills/writing-technical-docs/references/doc-types.md +++ b/skills/writing-technical-docs/references/doc-types.md @@ -33,7 +33,7 @@ event but their lasting parts should still stand alone. ## Commit message -- **Audience:** a future archaeologist running `git log` / `git blame` to understand +- **Audience:** a future maintainer running `git log` / `git blame` to understand why this change exists. - **Structure:** subject line in the imperative, ~50 characters ("Fix race in cache invalidation") → blank line → body wrapped ~72: what changed and *why*, not a diff @@ -48,7 +48,7 @@ event but their lasting parts should still stand alone. - **Structure:** what and why (linked issue) → how, at the level a reviewer needs → how it was verified → anything you're unsure about or want eyes on. - **Evergreen:** relaxed — a PR is anchored to a moment. But the "why" gets quoted into - merge commits and dug up later; write it to stand alone. + merge commits and consulted later; write it to stand alone. ## Review comment @@ -62,7 +62,7 @@ event but their lasting parts should still stand alone. ## README - **Audience:** a first-time evaluator deciding whether to use the project, then a new - user trying to get running. Optimized for scanning. + user trying to start using it. Optimized for scanning. - **Structure:** what it is (one paragraph, no marketing) → quickstart / install → core usage → where to go deeper (links to docs, contributing, license). Badges and demos only where they help the evaluator. @@ -73,7 +73,7 @@ event but their lasting parts should still stand alone. - **Audience:** peers who must understand, challenge, and later implement the design. - **Structure:** problem and context → goals and non-goals → proposal → alternatives - considered and why they lose → open questions. + considered and why they were rejected → open questions. - **Evergreen:** strict for the design description; the *decision record* part (what was decided, when, and why) is legitimately dated — label it as a decision, don't weave it into the description of the system. @@ -84,8 +84,8 @@ event but their lasting parts should still stand alone. - **Structure:** per version: breaking changes → features → fixes, each entry linking its change. - **Evergreen:** this is the time-stamped genre — dates and versions are the point. - Entries still describe changes factually ("Add support for X"), not breathlessly - ("The long-awaited X is finally here!"). + Entries still describe changes factually ("Add support for X"), not with marketing + language ("The long-awaited X is finally here!"). ## Skill document (SKILL.md and references)