Skip to content

planning: redesign /planning:interview (stable ids, reopen, detail levels, gap triage, one home per rule) #17964

Open
6 of 6 issues completed

Description

@kyle-sexton

Problem

/planning:interview starts most human-in-the-loop sessions, and its rounds had drifted: question
ids jumped or were renumbered (#7541), a contradicted answer had no way back except a new question,
round 1 put recommended answers in the user's mouth before the problem was known, every round
printed every evidence line, and several rules were stated two ways across the skill's files.

Owner decisions this work implements

Acceptance

  • Question ids come from round.sh next-id and are never reused or renumbered; a contradicted
    answer reopens under its own id with every dependent flagged for recheck.
  • Rounds print at a question_detail level (default standard); the saved round file keeps the
    full block; the round-file check fails a question with no recommendation-type line and a header
    with no Scope check: line.
  • Round 1 of a request with unknowns asks only open discovery questions; a question a running
    lookup could change is held; one blind recommender drafts recommendations.
  • The eval gate: every results file VALID and not partial; no kept or out-of-tag case regresses
    (fails on the new skill while passing on the old one); every rewritten case fails on the old skill
    and passes on the new one; every new case passes on the new skill; CI Status green.

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 21, 2026
  2. github-actions commented on Nov 21, 2026

    @github-actions
    Contributor

    Size: Large
    Complexity: High
    Uncertainty: Knowable
    Flags: none
    Blocked by: none
    Blast radius: the planning plugin's interview skill files, surface/round.sh and its tests, the interview evals and the plugin CHANGELOG; everyone who runs /planning:interview.
    Reversible: yes; revert the change, though ledger files already written with - gap | lines would keep them.
    Open questions: none
    Understood: redesign /planning:interview so ids are stable and never reused, a contradicted answer reopens under its own id, rounds print at a question_detail level, the round-file check is stricter, round 1 asks only discovery questions when unknowns exist, and each rule lives in one file. The body records the owner decisions and gives an exact done-when, so there is no intent question. The work is Large and could be split into the id/ledger scripts, the print level and round-file check, and the round-1 rules plus evals. Related open issues: #17736 (may touch the same skill files), #14878 and #16626 (interview evals already failing on main, which affects the eval gate baseline). No duplicate and no merged fix found.

    Automated triage. A maintainer reviews it.

  3. removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 21, 2026
  4. melodic-automation-lanes commented on Nov 21, 2026

    @melodic-automation-lanes
    Contributor

    Not admitted to the issue lane: Size large: every sub-issue is closed, but it is still open, so an item no sub-issue covers or a sub-issue's close keeps it from closing as a finished split (owner Q93). Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).

  5. melodic-automation-lanes commented on Nov 21, 2026

    @melodic-automation-lanes
    Contributor

    Refinement. A read-only run explored this repository, searched its issues and pull requests, and read the allowed docs to draft what follows. On an issue a listed human filed, it counts as the issue's spec once posted, while unedited; it is still model text, so check it.

    Intended outcome: Redesign /planning:interview so question ids come from round.sh next-id and are never reused, a contradicted answer reopens under its own id, rounds print at a question_detail level, the round-file check is stricter, round 1 is discovery-only when unknowns exist, and each rule has one home.

    Acceptance criteria

    • plugins/planning/surface/round.sh has a next-id subcommand, and a test in surface/test_round.py shows it never returns an id a ledger row or page question already held
    • plugins/planning/skills/interview/context/loop.md and SKILL.md state that ids are never reused or renumbered, and that a contradicted answer reopens under its own id with every dependent flagged for recheck
    • A test shows reopening a contradicted answer keeps its id and marks each dependent for recheck
    • A question_detail setting exists with default standard and is documented once in context/session-config.md; a printed round at standard omits evidence lines, while the saved round file keeps the full block
    • The round-file check fails a question with no recommendation-type line and a header with no Scope check: line, each covered by a test that fails before the change
    • The register gate passes an id gap only where a - gap | line declares it; export-ledger and sync-ledger write - gap | Q<a>-Q<b> | never assigned only for ids no ledger row and no page question ever held, and a test shows an id the ledger held is never declared
    • The per-round contradiction pass is removed from context/assumption-sweep.md (the 'Pre-round contradiction pass' section) and context/product.md, and legacy eval case 36 is removed from evals/evals.json
    • A new eval case interview-sweep-reports-contradiction exists and the end sweep carries the contradiction check
    • Round 1 of a request with unknowns asks only open discovery questions; a question a running lookup could change is held; one blind recommender drafts the recommendations; each is stated in the skill files and covered by an eval case
    • Each rule changed here is stated in exactly one skill file, and the other files point to it
    • Eval gate: every results file is VALID and not partial; no kept or out-of-tag case regresses; every rewritten case fails on the old skill and passes on the new one; every new case passes on the new skill
    • CI Status is green on the pull request, and plugins/planning/CHANGELOG.md records the change

    External state: none

    Dependencies

    • #​17736 (remaining interview friction fixes from the #​17279 audit, open): may touch the same skill files, so check overlap and order the work to avoid conflicts
    • #​14878 and #​16626 (open, interview evals failing on main): the eval gate's no-regression check needs a known baseline, so note which cases already fail on the old skill
    • Split suggestion: this is Large work; the id/ledger scripts, the print level and round-file check, and the round-1 rules plus evals could be separate runs

    Related work

    Sources read

    • plugins/planning/skills/interview/context/assumption-sweep.md:148: has the 'Pre-round contradiction pass' section the issue removes
    • plugins/planning/skills/interview/context/product.md:19: mentions the pre-round contradiction pass in the pacing rule
    • plugins/planning/skills/interview/context/surface.md:148: reopen decision exists on the surface and clears an answer while keeping the note
    • plugins/planning/skills/interview/context/loop.md:403: restates a contradicted recommendation as revised under the same question
    • plugins/planning/surface/round.sh: no next-id subcommand exists yet
    • plugins/planning/skills/interview: no question_detail or Scope check: text exists yet
    • plugins/planning/skills/interview/evals/evals.json: holds the interview eval cases, including legacy case 36

    Automated refinement by the lanes App. Intake triages the issue again with it.

  6. melodic-automation-lanes commented on Nov 21, 2026

    @melodic-automation-lanes
    Contributor

    Split at intake (owner Q93). This issue is Size Large, so its done-when is split into 6 sub-issues, each holding its task-list lines as written.

    Automated split by the lanes App. Admission waits on this issue while a sub-issue is open; work-unblock closes it as completed once every sub-issue is closed as completed by a merge or a listed human and its done-when is covered.

  7. kyle-sexton commented on Nov 21, 2026

    @kyle-sexton
    MemberAuthor

    This issue is delivered as one change: PR #17965 (branch feat/interview-redesign) builds every acceptance item and closes it; the merge is gated on the eval comparison in that PR. The six intake parts (#17971-#17976) are closed as duplicates so no lane rebuilds the same files in parallel. The split was triggered by Size Large at intake; the filing session should have marked the issue as session-owned and the repository has no such label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Fields

    Work status

    confirmed

    Priority

    medium

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions