Skip to content

Nightly regression: tests/integration/test_sdlc_multi_lineage.py::TestConcurrentMultiLineageContention::test_second_lineage_inherits_owner_run_id_via_named_refusal #3326

Description

@valorengels

Failing node

tests/integration/test_sdlc_multi_lineage.py::TestConcurrentMultiLineageContention::test_second_lineage_inherits_owner_run_id_via_named_refusal

Surfaced by nightly regression triage (lane nightly-triage-f7ec48e1).

Observed failure

Reproduced locally with ./scripts/pytest-clean.sh <node>:

tests/integration/test_sdlc_multi_lineage.py:106: in test_second_lineage_inherits_owner_run_id_via_named_refusal
    owner_run_id = supervisor["run_id"]
                   ^^^^^^^^^^^^^^^^^^^^
E   KeyError: 'run_id'

The supervisor's ensure returned an error dict with no run_id. The guard on line 105 (assert not supervisor.get("blocked")) did not catch it, because a RUN_BIND_FAILED result carries error, not blocked.

Likely cause (shared across all four nodes in this class)

Stale test double, not broken production behavior. The readback in _acquire_run_lock_and_bind was migrated to a resolver the test's AgentSession mock does not stub.

  • tools/sdlc_session_ensure.py:613-647 — after saving active_run_id, the bind does a post-save readback:
    from models.agent_session import AgentSession
    fresh = AgentSession.newest_for_session_id(session_id)
    readback_run_id = getattr(fresh, "active_run_id", None) if fresh is not None else None
    ...
    if readback_run_id != candidate:
        release_issue_lock(issue_number, candidate)
        return None, {"error": "RUN_BIND_FAILED", "reason": "post-save readback mismatch", ...}
  • tests/integration/test_sdlc_multi_lineage.py:36-40 — the test's _readback_as() helper stubs only the old access path:
    def _readback_as(session: MagicMock) -> MagicMock:
        mock_as = MagicMock()
        mock_as.query.filter.return_value = [session]
        return mock_as
    newest_for_session_id is never configured, so it returns an auto-generated MagicMock, whose .active_run_id is another auto-MagicMock that can never equal candidate. Every bind therefore fails the readback, releases the lock via compare-and-delete, and returns RUN_BIND_FAILED — so no lineage ever mints and no lease is ever held.
  • The readback moved to AgentSession.newest_for_session_id (models/agent_session.py:1232-1235) in commit 3c77e1eab ("AgentSession: one newest-wins resolver for every session_id read"). The mock helper was not updated with it.

All four tests in TestConcurrentMultiLineageContention share this single cause; they fail at four different assertions because each consumes the broken result differently.

A secondary, independent weakness this exposed: assert not supervisor.get("blocked") (lines 105, 122, 142) is satisfied by a RUN_BIND_FAILED dict, which carries error but no blocked key. The guard passes on an error result and the test then dies on a bare KeyError: 'run_id' instead of reporting the actual failure.

Suggested next steps

  1. Fix _readback_as (tests/integration/test_sdlc_multi_lineage.py:36-40) to stub the resolver the code under test actually calls — configure mock_as.newest_for_session_id.return_value = session (keeping query.filter only if some other path still needs it). Because session.active_run_id is assigned by the bind before the readback, a mock returning the same object naturally reads back the candidate.
  2. Strengthen the supervisor guards at lines 105, 122 and 142 to assert not supervisor.get("blocked") and not supervisor.get("error"), supervisor so an error result fails loudly at its origin instead of as a downstream KeyError.
  3. Consider whether the readback deserves a mock-independent guard: this class is the acceptance test for exactly-one-owner semantics (Reliability risk: /do-sdlc fork vs supervisor — forked executions merge past blocked gates, record unearned/contradictory verdicts, and lease TTL churn deadlocks the router #2026 WS1), and it silently stopped exercising the lock contest the moment the mock drifted — every lineage was refused for the wrong reason while the assertions still looked meaningful.

Related

All four nodes in TestConcurrentMultiLineageContention failed in the same nightly batch and share the root cause above:

  • test_concurrent_bare_ensures_mint_exactly_one_owner
  • test_second_lineage_inherits_owner_run_id_via_named_refusal
  • test_release_frees_issue_for_a_fresh_lineage
  • test_non_owner_lineage_cannot_pass_single_owner_merge_gate

Cluster siblings already filed: #3323, #3324, #3325.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    testingRelated to the test suite (tests/)

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions