Skip to content

Two script test suites fail on a clean main: test_ai_session (4 cases) and test_compare_shell_forms (1 case) #1382

Description

@d-morrison

Noticed while running the full script test suite as part of the pre-push checklist on #1381. Both failures are pre-existing on origin/main and unrelated to that PR's diff; filing per report-mistakes-proactively rather than fixing, since neither is in a file #1381 authors.

Reproduction

Measured at origin/main = 2ed74b89 in a detached worktree, so nothing local is in play. Exit codes read without a pipeline, since $? after | tail -1 reports tail's status, not the script's (errexit-is-not-uniform.md):

git worktree add --detach /tmp/wt-chk origin/main
( cd /tmp/wt-chk && python3 scripts/test_ai_session.py >/tmp/o.txt 2>&1 ); echo $?
( cd /tmp/wt-chk && python3 scripts/test_compare_shell_forms.py >/tmp/o.txt 2>&1 ); echo $?

Both return exit 1, identically at 2ed74b89 and at #1381's head 4694e47b.

scripts/test_ai_session.py — 4 failing cases

4 FAILED:
  - a stale session exits 2, not 1: rc=0
  - a stale session says the session is the problem: mwc is active for session sess-1279
  - a stale session names the recovery command: mwc is active for session sess-1279
  - prune sweeps a stale session's marker

All four concern stale-session detection. The observed behaviour is that a session marker for sess-1279 is treated as live (rc=0, "mwc is active for session sess-1279") where the tests expect it to be recognized as stale, exit 2, name the session as the problem, and be swept by prune.

This is the more concerning of the two: ai-session.py backs the mwc grant's session scoping, so a stale marker reading as active means a merge-authority grant can outlive the session that was given it. Worth confirming whether the tests encode the intended contract and the script regressed, or the contract changed and the tests were not updated — c3a01e0f ("hooks: make the merge and unreviewed-PR guards satisfiable again", #1279/#1287) is the most recent commit touching this area.

scripts/test_compare_shell_forms.py — 1 failing case

FAIL: two backslashes produce the SyntaxWarning
22 passed, 1 failed

Already observed independently and noted as pre-existing in #1366's body; this issue records it so it is tracked rather than only mentioned in a PR description. Most recent commit touching the script is d2f35c0c (#1309).

Why this is worth tracking

validate is green on both, so neither failure gates anything — these suites appear not to be wired into CI. That is the shape fail-fast.md warns about: a check that cannot fail carries no signal, and a locally-run suite that has been red for some time trains readers to skim its output. Two follow-ups worth separating from the fixes themselves:

  1. Fix (or re-baseline) the two suites.
  2. Decide whether scripts/test_*.py should run in validate, so a red suite is visible rather than discovered by whoever happens to run it.

Cross-ref: #1381 (where this surfaced), #1366 (notes the compare_shell_forms failure in passing).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions