You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Surfaced by nightly regression triage (lane nightly-triage-f7ec48e1).
Observed failure
Reproduced locally with ./scripts/pytest-clean.sh <node>:
tests/integration/test_sdlc_multi_lineage.py:106: in test_second_lineage_inherits_owner_run_id_via_named_refusal
owner_run_id = supervisor["run_id"]
^^^^^^^^^^^^^^^^^^^^
E KeyError: 'run_id'
The supervisor's ensure returned an error dict with no run_id. The guard on line 105 (assert not supervisor.get("blocked")) did not catch it, because a RUN_BIND_FAILED result carries error, not blocked.
Likely cause (shared across all four nodes in this class)
Stale test double, not broken production behavior. The readback in _acquire_run_lock_and_bind was migrated to a resolver the test's AgentSession mock does not stub.
tools/sdlc_session_ensure.py:613-647 — after saving active_run_id, the bind does a post-save readback:
newest_for_session_id is never configured, so it returns an auto-generated MagicMock, whose .active_run_id is another auto-MagicMock that can never equal candidate. Every bind therefore fails the readback, releases the lock via compare-and-delete, and returns RUN_BIND_FAILED — so no lineage ever mints and no lease is ever held.
The readback moved to AgentSession.newest_for_session_id (models/agent_session.py:1232-1235) in commit 3c77e1eab ("AgentSession: one newest-wins resolver for every session_id read"). The mock helper was not updated with it.
All four tests in TestConcurrentMultiLineageContention share this single cause; they fail at four different assertions because each consumes the broken result differently.
A secondary, independent weakness this exposed: assert not supervisor.get("blocked") (lines 105, 122, 142) is satisfied by a RUN_BIND_FAILED dict, which carries error but no blocked key. The guard passes on an error result and the test then dies on a bare KeyError: 'run_id' instead of reporting the actual failure.
Suggested next steps
Fix _readback_as (tests/integration/test_sdlc_multi_lineage.py:36-40) to stub the resolver the code under test actually calls — configure mock_as.newest_for_session_id.return_value = session (keeping query.filter only if some other path still needs it). Because session.active_run_id is assigned by the bind before the readback, a mock returning the same object naturally reads back the candidate.
Strengthen the supervisor guards at lines 105, 122 and 142 to assert not supervisor.get("blocked") and not supervisor.get("error"), supervisor so an error result fails loudly at its origin instead of as a downstream KeyError.
Failing node
Surfaced by nightly regression triage (lane
nightly-triage-f7ec48e1).Observed failure
Reproduced locally with
./scripts/pytest-clean.sh <node>:The supervisor's ensure returned an error dict with no
run_id. The guard on line 105 (assert not supervisor.get("blocked")) did not catch it, because aRUN_BIND_FAILEDresult carrieserror, notblocked.Likely cause (shared across all four nodes in this class)
Stale test double, not broken production behavior. The readback in
_acquire_run_lock_and_bindwas migrated to a resolver the test'sAgentSessionmock does not stub.tools/sdlc_session_ensure.py:613-647— after savingactive_run_id, the bind does a post-save readback:tests/integration/test_sdlc_multi_lineage.py:36-40— the test's_readback_as()helper stubs only the old access path:newest_for_session_idis never configured, so it returns an auto-generatedMagicMock, whose.active_run_idis another auto-MagicMockthat can never equalcandidate. Every bind therefore fails the readback, releases the lock via compare-and-delete, and returnsRUN_BIND_FAILED— so no lineage ever mints and no lease is ever held.AgentSession.newest_for_session_id(models/agent_session.py:1232-1235) in commit3c77e1eab("AgentSession: one newest-wins resolver for every session_id read"). The mock helper was not updated with it.All four tests in
TestConcurrentMultiLineageContentionshare this single cause; they fail at four different assertions because each consumes the broken result differently.A secondary, independent weakness this exposed:
assert not supervisor.get("blocked")(lines 105, 122, 142) is satisfied by aRUN_BIND_FAILEDdict, which carrieserrorbut noblockedkey. The guard passes on an error result and the test then dies on a bareKeyError: 'run_id'instead of reporting the actual failure.Suggested next steps
_readback_as(tests/integration/test_sdlc_multi_lineage.py:36-40) to stub the resolver the code under test actually calls — configuremock_as.newest_for_session_id.return_value = session(keepingquery.filteronly if some other path still needs it). Becausesession.active_run_idis assigned by the bind before the readback, a mock returning the same object naturally reads back the candidate.assert not supervisor.get("blocked") and not supervisor.get("error"), supervisorso an error result fails loudly at its origin instead of as a downstreamKeyError.Related
All four nodes in
TestConcurrentMultiLineageContentionfailed in the same nightly batch and share the root cause above:test_concurrent_bare_ensures_mint_exactly_one_ownertest_second_lineage_inherits_owner_run_id_via_named_refusaltest_release_frees_issue_for_a_fresh_lineagetest_non_owner_lineage_cannot_pass_single_owner_merge_gateCluster siblings already filed: #3323, #3324, #3325.