Claude/repo analysis brainstorming e nf uw

CHANGELOG.md

@@ -4,6 +4,29 @@ All notable changes to the Iterative Planner project will be documented in this
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
+## [2.18.0] - 2026-05-19
+
+### Added
+- **Ideation Gate inside PLAN** — `ip-plan-writer` now produces a written `ideation.md` (≥3 candidate approaches with hard-constraint check, "X at the cost of Y" trade-off, and top risk; plus a Selection; plus one-line rejection rationales) **before** writing `plan.md`. Replaces the prior self-assessed Solutions Exploration Confidence dimension with a durable artifact. Single-Path Escape Hatch covers genuinely single-path tasks (mechanical renames, deterministic migrations) — 1 candidate is permitted if both "Why no alternatives" and a Falsification trigger are populated.
+- **PC-IDEATION** — 7th Presentation Contract. Emitted by the orchestrator during PLAN after `ideation.md` is written and **before** `plan.md` is generated. Renders candidates + Selection + Rejected one-liners verbatim; user can approve, redirect to a rejected candidate, or push for more options. Defined in `references/file-formats.md` "Presentation Contracts" section, inlined in `agents/orchestrator.md` PLAN dispatch.
+- **Two-phase ip-plan-writer dispatch** — orchestrator now spawns `ip-plan-writer` twice in a PLAN cycle: phase 1 writes `ideation.md` only, phase 2 writes `plan.md` after the user approves the Selection via PC-IDEATION. Avoids burning cycles writing `plan.md` when ideation might get redirected.
+- **Viability-flipping constraint rule** — before generating candidates, `ip-plan-writer` scans classified constraints for any whose reclassification (hard ↔ soft ↔ ghost) would flip a candidate's viability. Uncertain on a viability-flipping constraint → signal `NEEDS_USER_CLARIFICATION:<question>` to the orchestrator instead of guessing. Orchestrator asks the user, updates `findings.md` (with `[CORRECTED iter-N]` if applicable), and re-spawns the writer. Same rule fires if a misclassification is discovered mid-generation.
+- **`ideation.md` artifact** — new per-plan file. Created by `bootstrap.mjs new` with a stub template. Read during PLAN (by ip-plan-writer before writing `plan.md`) and PIVOT ghost-constraint scan. Not merged into consolidated `plans/FINDINGS.md` or `plans/DECISIONS.md` on close — stays plan-local.
+- **`validate-plan.mjs checkIdeation`** — new check enforces the gate at state ≥ PLAN: `ideation.md` exists, has `## Candidates` with ≥3 `### C-N` headings (or both fields of the Single-Path Escape Hatch populated), and a non-placeholder `## Selection`. Partial escape hatch (only "Why no alternatives" OR only "Falsification") produces a precise error message. ERRORs on miss; no-op during EXPLORE.
+- **`ideation.md` row in File Lifecycle Matrix** — W in PLAN by `ip-plan-writer`, R in EXECUTE, R+W in PIVOT (for `[REACTIVATED iter-N]` markers when a rejected candidate becomes viable after a ghost constraint is found).
+- **Ideation Discipline section in `planning-rigor.md`** — motivates divergence-before-convergence; explains why three candidates (not two), why a written artifact, how the escape hatch prevents friction on single-path tasks, and the viability-flipping ask-don't-guess rule.
+- **12 new tests** in `bootstrap.test.mjs` — bootstrap creates `ideation.md`, validator no-op in EXPLORE, ERROR on missing/missing-Candidates/few-candidates/empty-Selection/partial-escape-hatch at PLAN, passes with 3 candidates + Selection, passes with populated escape hatch, close succeeds with `ideation.md`, ideation not merged into `plans/FINDINGS.md` or `plans/DECISIONS.md` on close.
+
+### Changed
+- **PLAN dispatch in `agents/orchestrator.md`** — spawns `ip-plan-writer` in two phases with PC-IDEATION emitted between them. Approve → spawn phase 2 (writes `plan.md`); redirect → re-spawn phase 1 with new direction; request more options → re-spawn phase 1 with "add candidates" directive.
+- **`agents/ip-plan-writer.md` Output Format** — describes both phase 1 (ideation) and phase 2 (plan) outputs. Phase 1 returns the PC-IDEATION digest (candidates, selection, rejected); phase 2 returns the PC-PLAN digest (path + section anchors + one-paragraph summary).
+- **PLAN gate check** in `SKILL.md` now reads `ideation.md` alongside the other plan files.
+- **PLAN decisions.md D-001** is now the carried-forward Selection from `ideation.md` (same Trade-off, with rejected candidates referenced).
+- **PIVOT ghost-constraint scan** now includes a 4th question: re-read `ideation.md` Rejected list for candidates rejected against constraints that became ghosts; mark resurrected entries `[REACTIVATED iter-N]`.
+- **Solutions Exploration Confidence dimension** in `planning-rigor.md` demoted to point at `ideation.md` — the dimension is now materialized as a written artifact rather than a self-assessment.
+- **Mandatory Re-reads** table now includes `ideation.md` in the "Before PLAN or PIVOT" row.
+- **User Interaction table in `SKILL.md`** now distinguishes PLAN (ideation) → PC-IDEATION from PLAN (plan) → PC-PLAN; both contracts emitted during a single PLAN cycle.
+
 ## [2.17.0] - 2026-05-07
 
 ### Added

CLAUDE.md

@@ -24,14 +24,14 @@ iterative-planner/
     ├── agents/                       # Sub-agent definitions (installed to ~/.claude/agents/)
     │   ├── orchestrator.md           # State machine owner, spawns all other agents
     │   ├── ip-explorer.md            # Read-only codebase research (EXPLORE phase)
-    │   ├── ip-plan-writer.md         # Plan generation (PLAN phase)
+    │   ├── ip-plan-writer.md         # Plan generation (PLAN phase) — writes ideation.md then plan.md
     │   ├── ip-executor.md            # Code execution (EXECUTE phase)
     │   ├── ip-verifier.md            # Verification checks (REFLECT phase)
     │   ├── ip-reviewer.md            # Adversarial review (REFLECT phase, iteration >= 2)
     │   └── ip-archivist.md           # CLOSE phase housekeeping
     ├── scripts/
     │   ├── bootstrap.mjs             # Initializes plans/plan_YYYY-MM-DD_XXXXXXXX/ directory (Node.js 18+)
-    │   ├── bootstrap.test.mjs        # Test suite (node:test, 102 tests)
+    │   ├── bootstrap.test.mjs        # Test suite (node:test, 134 tests)
     │   └── validate-plan.mjs         # Protocol compliance validator (Node.js 18+)
     └── references/                   # Knowledge base documents
         ├── code-hygiene.md           # Change manifest format, revert procedures, forbidden leftovers

README.md

@@ -1,7 +1,7 @@
 # Iterative Planner
 
 [![License](https://img.shields.io/badge/License-GPLv3-blue.svg)](LICENSE)
-[![Skill](https://img.shields.io/badge/Skill-v2.17.0-green.svg)](CHANGELOG.md)
+[![Skill](https://img.shields.io/badge/Skill-v2.18.0-green.svg)](CHANGELOG.md)
 [![Tests](https://img.shields.io/badge/tests-122%20passing-brightgreen.svg)](src/scripts/bootstrap.test.mjs)
 [![Sponsored by Electi](https://img.shields.io/badge/Sponsored%20by-Electi-red.svg)](https://www.electiconsulting.com)
 
@@ -216,6 +216,7 @@ Each state embeds domain-agnostic thinking tools:
 
 | Framework | What it does |
 |-----------|-------------|
+| **Divergent ideation** | Before locking the plan, `ip-plan-writer` writes `ideation.md` with ≥3 candidate approaches — each with a hard-constraint check, "X at the cost of Y" trade-off, and top risk — then a Selection. Rejected candidates persist as one-liners for later PIVOT ghost-constraint scans. Validator-enforced at state ≥ PLAN; surfaced to the user as PC-IDEATION before `plan.md` is written. Single-Path Escape Hatch (with "Why no alternatives" + Falsification trigger) covers genuinely single-path tasks. |
 | **Problem decomposition** | Understand the whole, find natural boundaries, minimize dependencies, start with the riskiest part. |
 | **Assumption tracking** | Every assumption traced to a finding, linked to dependent steps. When one breaks, you know what's invalidated. |
 | **Pre-mortem and falsification** | Assume the plan failed. Why? Extract concrete STOP IF triggers. Prevents confirmation bias. |
@@ -298,6 +299,7 @@ plans/
     ├── decisions.md               # append-only log of every decision and pivot
     ├── findings.md                # index of discoveries (corrected when wrong)
     ├── findings/                  # detailed research files (one per topic)
+    ├── ideation.md                # candidate approaches + Selection (ip-plan-writer, before plan.md)
     ├── progress.md                # done vs in-progress vs remaining
     ├── verification.md            # verification results per REFLECT cycle
     ├── changelog.md               # per-edit ledger (one line per file edit)

VERSION

@@ -1 +1 @@
-2.17.0
+2.18.0

src/SKILL.md

@@ -45,7 +45,7 @@ stateDiagram-v2
 
 | From → To | Trigger |
 |-----------|---------|
-| EXPLORE → PLAN | Sufficient context. ≥3 indexed findings in `findings.md`. |
+| EXPLORE → PLAN | Sufficient context. ≥3 indexed findings in `findings.md`. (Ideation Gate then fires inside PLAN: `ip-plan-writer` writes `ideation.md` before `plan.md`.) |
 | PLAN → EXPLORE | Can't state problem, can't list files, or insufficient findings. |
 | PLAN → PLAN | User rejects plan. Revise and re-present. |
 | PLAN → EXECUTE | User explicitly approves. |
@@ -73,7 +73,7 @@ These files are active working memory. Re-read during the conversation, not just
 | Before any EXECUTE step | `state.md`, `plan.md`, `progress.md` | Confirm step, manifest, fix attempts, progress sync |
 | Before writing a fix | `decisions.md` | Don't repeat failed approaches. Check 3-strike. |
 | Before modifying `DECISION`-commented code | Referenced `decisions.md` entry | Understand why before changing |
-| Before PLAN or PIVOT | `decisions.md`, `findings.md`, `findings/*`, `plans/LESSONS.md`, `plans/SYSTEM.md` | Ground plan in known facts + institutional memory + system atlas |
+| Before PLAN or PIVOT | `decisions.md`, `findings.md`, `findings/*`, `ideation.md` (if exists), `plans/LESSONS.md`, `plans/SYSTEM.md` | Ground plan in known facts + institutional memory + system atlas; don't reinvent rejected candidates |
 | Before any REFLECT | `plan.md` (criteria + verification strategy + assumptions), `progress.md`, `verification.md`, `findings.md`, `checkpoints/*`, `decisions.md` | Phase 1 Gate-In: full context before evaluating |
 | Every 10 tool calls | `state.md` | Reorient. Right step? Scope crept? |
 
@@ -113,6 +113,7 @@ plans/
     ├── decisions.md               # Append-only decision/pivot log
     ├── findings.md                # Summary + index of findings
     ├── findings/                  # Detailed finding files (subagents write here)
+    ├── ideation.md                # Candidate approaches + Selection (written by ip-plan-writer before plan.md)
     ├── progress.md                # Done vs remaining
     ├── verification.md            # Verification results per REFLECT cycle
     ├── changelog.md               # Per-edit ledger (one line per file edit, append-only)
@@ -136,6 +137,7 @@ R = read only | W = update (implicit read + write) | R+W = distinct read and wri
 | decisions.md | — | R+W | R | R+W | R+W | R |
 | findings.md | W | R | — | R | R+W | R |
 | findings/* | W | R | — | R | R+W | R |
+| ideation.md | — | W | R | — | R+W | R |
 | progress.md | — | W | R+W | R+W | W | R |
 | verification.md | — | W | W | W | R | R |
 | changelog.md | — | — | W (append) | R | W (append REVERT) | R |
@@ -221,6 +223,8 @@ Institutional memory across plans. Unlike FINDINGS.md and DECISIONS.md which gro
 
 ### PLAN
 - **Gate check**: read `state.md`, `plan.md`, `findings.md`, `findings/*`, `decisions.md`, `progress.md`, `verification.md`, `plans/FINDINGS.md` (limit: 600), `plans/DECISIONS.md` (limit: 600), `plans/LESSONS.md` before writing anything. If not read → read now. No exceptions. If `findings.md` has <3 indexed findings → go back to EXPLORE.
+- **Ideation Gate** *(CORE — applies to iter-1)* — `ip-plan-writer` produces `ideation.md` **before** `plan.md`. Required: ≥3 candidate approaches (`### C-1`, `### C-2`, `### C-3`, …) with Sketch, Hard-constraint check (refs `findings.md`/`findings/*`), Trade-off in **"X at the cost of Y"** form, Top risk; plus a Selection (picked candidate, criteria, confidence); plus one-line Rejected rationales. Single-Path Escape Hatch covers genuinely single-path tasks (mechanical rename, deterministic migration) — 1 candidate permitted if both "Why no alternatives" and "Falsification" are populated. The orchestrator emits **PC-IDEATION** before `plan.md` is written — user can approve, redirect to a rejected candidate, or push for more options. The picked candidate becomes the first PLAN entry in `decisions.md` (e.g., `plan_YYYY-MM-DD_xxxxxxxx/D-001`). Validator (`validate-plan.mjs`) ERRORs at state ≥ PLAN if `ideation.md` is missing, empty, lacks ≥3 candidates (without populated escape hatch), or has no Selection. See `references/file-formats.md` `ideation.md` template and `references/planning-rigor.md` Ideation Discipline.
+- **Viability-flipping constraint check** *(CORE)* — before generating candidates, `ip-plan-writer` scans `findings.md` constraint classifications. A constraint is *viability-flipping* if reclassifying it (hard ↔ soft ↔ ghost) would change which candidates are viable. If classification is uncertain on a viability-flipping constraint → **signal `NEEDS_USER_CLARIFICATION:<question>`** to the orchestrator instead of guessing. The orchestrator asks the user, updates `findings.md` (with `[CORRECTED iter-N]` if applicable), and re-spawns `ip-plan-writer`. Same rule fires if `ip-plan-writer` discovers mid-generation that a constraint was misclassified.
 - **Problem Statement first** — before designing steps, write in `plan.md`: (1) what behavior is expected, (2) invariants — what must always be true, (3) edge cases at boundaries. Can't state the problem clearly → go back to EXPLORE.
 - Write `plan.md`: problem statement, steps (with risk/dependency annotations), assumptions, failure modes, pre-mortem & falsification signals, success criteria, verification strategy, complexity budget.
 - **Decomposition** — when breaking the goal into steps:
@@ -233,7 +237,7 @@ Institutional memory across plans. Unlike FINDINGS.md and DECISIONS.md which gro
 - **Assumptions** — bullet list in plan.md: what you assume, which finding grounds it, which steps depend on it. On surprise discovery during EXECUTE → check this list first. See `references/planning-rigor.md`.
 - **Failure Mode Analysis** — for each external dependency or integration point in the plan, answer: what if slow? returns garbage? is down? What's the blast radius? Write to plan.md `Failure Modes` section. No dependencies → write "None identified" (proves you checked).
 - **Pre-Mortem & Falsification Signals** — assume the plan failed. 2-3 scenarios with concrete STOP IF triggers. If a trigger fires during EXECUTE → stop and REFLECT. Covers approach validity (distinct from Failure Modes which cover dependencies, and Autonomy Leash which covers step failure). See `references/planning-rigor.md`.
-- Write `decisions.md`: log chosen approach + why (mandatory even for first plan). **Trade-off rule** — phrase every decision as **"X at the cost of Y"**. Never recommend without stating what it costs.
+- Write `decisions.md`: log chosen approach + why (mandatory even for first plan). The first PLAN entry (D-001) carries forward the picked candidate from `ideation.md` Selection — same Trade-off, with a brief reference to the rejected candidates. **Trade-off rule** — phrase every decision as **"X at the cost of Y"**. Never recommend without stating what it costs.
 - Read then write `verification.md` with initial template (criteria table populated from success criteria, methods from verification strategy, results pending).
 - Read then write `state.md` + `progress.md`.
 - List **every file** to modify/create. Can't list them → go back to EXPLORE.
@@ -315,7 +319,7 @@ All seven reads are CORE. Do not evaluate until all are complete.
 ### PIVOT
 - Read `decisions.md`, `findings.md`, relevant `findings/*`, `plans/LESSONS.md`.
 - Read `checkpoints/*` — decide keep vs revert. Default: if unsure, revert to latest checkpoint. See `references/code-hygiene.md` for full decision framework.
-- **Ghost constraint scan** *(EXTENDED — skip for iteration 1)* — before designing a new approach, ask: (1) Is the constraint that led to the failed approach still valid? (2) Are we inheriting environmental constraints that are actually preferences? (3) Did an early finding become stale? Log ghost constraints found in `decisions.md`. See `references/planning-rigor.md`.
+- **Ghost constraint scan** *(EXTENDED — skip for iteration 1)* — before designing a new approach, ask: (1) Is the constraint that led to the failed approach still valid? (2) Are we inheriting environmental constraints that are actually preferences? (3) Did an early finding become stale? (4) Re-read `ideation.md` Rejected list — was a viable candidate rejected against a constraint that turned out to be ghost? If so, mark the entry `[REACTIVATED iter-N]` in `ideation.md` and append a new Selection rationale (don't rewrite the original — keep the audit trail). Log ghost constraints found in `decisions.md`. See `references/planning-rigor.md`.
 - If earlier findings proved wrong or incomplete → update `findings.md` + `findings/*` with corrections. Mark corrections: `[CORRECTED iter-N]` + what changed and why. Append, don't delete original text.
 - **Momentum check** *(EXTENDED — 2nd PIVOT onward)* — log pivot direction, check for oscillation. Momentum < 0.3 → recommend decomposition. See `references/convergence-metrics.md`.
 - Write `decisions.md`: log pivot + mandatory Complexity Assessment (+ pivot direction log if EXTENDED).
@@ -394,7 +398,8 @@ Sub-agents are invisible to the user — only the orchestrator's chat text reach
 | State | Contract | Behavior |
 |-------|----------|----------|
 | EXPLORE | **PC-EXPLORE** (Findings Digest) | Ask focused questions, one at a time. At handoff, emit findings index + key constraints (HARD/SOFT/GHOST) verbatim, plus exploration confidence and a synthesis paragraph. |
-| PLAN | **PC-PLAN** (Plan Presentation) | Render `plan.md` verbatim. Floor (always render): Steps, Success Criteria, Verification Strategy, Failure Modes, Assumptions. Wait for approval. Re-present same contract if modified. |
+| PLAN (ideation) | **PC-IDEATION** (Candidate Approaches Digest) | After `ip-plan-writer` writes `ideation.md` and before `plan.md` is generated, render candidates + Selection + Rejected one-liners verbatim. Wait for user to approve, redirect to a rejected candidate, or request more options. |
+| PLAN (plan) | **PC-PLAN** (Plan Presentation) | Render `plan.md` verbatim. Floor (always render): Steps, Success Criteria, Verification Strategy, Failure Modes, Assumptions. Wait for approval. Re-present same contract if modified. |
 | EXECUTE | **PC-EXECUTE-STEP** (Per-Step Status) / **PC-EXECUTE-LEASH** (Leash Failure) | After each successful step: 5 fields (step + files + commit + surprises + next-preview). On leash hit: 5 fields (step intent + 2 attempts + root-cause guess + checkpoint registry + prompt). |
 | REFLECT | **PC-REFLECT** (Phase-3 Gate-Out 5-Item Block) | Exactly 5 items: completed / remaining / verification table verbatim / issues + reviewer concerns / recommendation + prompt. **Ask** user: close, pivot, or explore. Never auto-close. |
 | PIVOT | **PC-PIVOT** (Pivot Options) | Pivot reason + checkpoint registry (verbatim) + ghost constraints + 1-3 candidate directions ("X at the cost of Y") + explicit prompt for direction and keep-vs-revert. |