Skip to content

Commit e2bc66c

Browse files
test(runtime-host): assert owned Host exit against the kernel's shutdown contract
Both owned-lifecycle tests asserted that a Host process had exited against a 5 s wall clock, which is shorter than the exit the kernel guarantees, and one of them started that clock from an event the Host does not control. `an authority-supervised Candidate exits if its launch owner is killed` waited on `connected.connection.closed` and then gave the process 5 s. That promise is the Client's own transport, and the Client aborts the transport when a liveness probe goes unanswered for two seconds, so a Host that is merely busy resolves it while still running and the exit budget starts at an unrelated moment. The Host also cannot exit as early as the bound assumed: the owner-loss close is armed by `launchOwnerGuard.bind(...)` only after `startExecutionRuntimeHostCandidate` resolves, so composition startup and recovery must finish first, and the shutdown that follows is bounded by `shutdownGraceMs` (10 s). Wait for the process instead, with a 20 s bound that sits above that grace and below the launcher fixture's idle grace, and keep the connection-closed assertion after the exit, where it is a consequence rather than a gate. The fixture's idle grace moves from 10 s to 60 s so a Candidate that exits because it went idle can never satisfy an owner-loss bound, with an explicit 10 s first-connection deadline so a Candidate no Client reaches still exits. `owned Host exits promptly after its first connection closes` allowed the whole shutdown 5 s against a kernel that allows itself 10 s, and conflated the two claims in its name. The promptness claim is the owned launch's idleGraceMs of 0 against a 30 s default, which the draining registration the kernel publishes as its first shutdown step reports directly; the exit is a second claim, now bounded by the kernel's grace. Refs #4776 Generated-by: Claude Code
1 parent a9307d0 commit e2bc66c

3 files changed

Lines changed: 70 additions & 8 deletions

File tree

packages/runtime-host/src/__tests__/fixtures/owned-authority-launcher.ts

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,13 @@ const attempt = await launchOwnedRuntimeHostCandidate({
3232
rootPath,
3333
expectedRootId,
3434
entrypoint: new URL('../../execution-candidate-main.js', import.meta.url),
35-
idleGraceMs: 10_000,
35+
// The idle grace only has to outlast the test, and it has to stay clear of
36+
// any bound a test puts on an owner-loss exit: a Candidate that exits
37+
// because it went idle must never be mistaken for one that exited because
38+
// its launch owner died. The first-connection deadline stays short so a
39+
// Candidate no Client ever reaches still exits on its own.
40+
idleGraceMs: 60_000,
41+
initialConnectionTimeoutMs: 10_000,
3642
inheritableAuthorityLeaseFd: leaseFd,
3743
launchOwnerClientInstanceId: clientInstanceId,
3844
}).spawned;

packages/runtime-host/src/__tests__/host-kernel.test.ts

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1916,12 +1916,26 @@ describe('non-serving Runtime Host kernel', () => {
19161916

19171917
launcher.kill('SIGKILL');
19181918
await waitForExit(launcher);
1919+
// The process is the only thing that reports the claim. A Client's
1920+
// `connection.closed` does not: it is that Client's own transport, and
1921+
// the Client aborts it after its liveness probe goes unanswered for two
1922+
// seconds. A Host that is merely busy therefore resolves it while still
1923+
// running, and gating the exit assertion on it starts the exit budget at
1924+
// a moment that has nothing to do with the Host's shutdown.
1925+
//
1926+
// The bound comes from the kernel's contract rather than from an
1927+
// interval this test could predict. Owner loss cannot close a
1928+
// composition before its startup settles, and the shutdown that follows
1929+
// is bounded by `shutdownGraceMs` (10 s), after which the kernel
1930+
// force-terminates. Twenty seconds therefore sits above every
1931+
// legitimate exit and below the launcher's 60 s idle grace, so it cannot
1932+
// be satisfied by a Candidate that merely went idle.
1933+
await waitForProcessExit(launchedPid, 20_000);
19191934
await withTimeout(
19201935
connected.connection.closed,
19211936
5_000,
1922-
'authority-supervised Candidate survived its launch owner',
1937+
'authority-supervised Candidate exited without closing its Client connection',
19231938
);
1924-
await waitForProcessExit(launchedPid);
19251939
paths.resources.forgetPid(launchedPid);
19261940
});
19271941
});

packages/runtime-host/src/__tests__/owned-candidate.test.ts

Lines changed: 47 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,11 @@ import { mkdtemp, readdir, rm } from 'node:fs/promises';
2323
import { tmpdir } from 'node:os';
2424
import { join } from 'node:path';
2525
import test from 'node:test';
26+
import {
27+
resolveExistingStorageRoot,
28+
resolveExistingStorageRootControlDirectory,
29+
} from '@maka/storage/root-authority';
30+
import { readHostRegistration } from '../control/registration.js';
2631
import {
2732
INTERACTIVE_RUNTIME_HOST_COMPOSITION_ID,
2833
RUNTIME_HOST_PROTOCOL_VERSION,
@@ -257,12 +262,20 @@ test('owned Host exits promptly after its first connection closes', async () =>
257262

258263
assert.equal(result.kind, 'connected', connectFailure(result));
259264
if (result.kind !== 'connected') return;
265+
const controlDirectory = await resolveHostControlDirectory(rootPath, result.connection.rootId);
260266
await result.connection.close();
261-
// Prompt means the owned launch's idleGraceMs of 0, as opposed to the 30 s
262-
// default grace, so the bound only has to sit well below that. Shutdown takes
263-
// about 30 ms on an idle machine and stretches past 500 ms under a full CI
264-
// suite while still exiting cleanly: the Host is starved, not stuck.
265-
assert.equal(await result.host.settle(5_000), true);
267+
// Promptness is when the Host starts shutting down, not how long shutting
268+
// down takes: the owned launch's idleGraceMs is 0 against a 30 s default.
269+
// The kernel publishes its draining registration as the first step of
270+
// shutdown, so the registration reports the idle grace directly. Reading it
271+
// from `settle` alone could not separate the two, which is why a loaded
272+
// machine that only made the shutdown itself slow failed this assertion.
273+
await waitForHostShutdownStart(controlDirectory, 10_000);
274+
// The exit is a second claim with a bound of its own, and the kernel sets
275+
// it: shutdown gets `shutdownGraceMs` (10 s) to close every resource before
276+
// the kernel force-terminates the process. Anything below that fails a Host
277+
// that is starved rather than stuck.
278+
assert.equal(await result.host.settle(15_000), true);
266279
});
267280

268281
test('an exited owned Candidate permits one real successor in the same election', {
@@ -436,6 +449,35 @@ test('pre-cancelled hosted execution does not start a Runtime Host', async () =>
436449
assert.deepEqual(await readdir(rootPath), []);
437450
});
438451

452+
async function resolveHostControlDirectory(rootPath: string, rootId: string): Promise<string> {
453+
const capability = await resolveExistingStorageRoot({
454+
path: rootPath,
455+
kind: 'interactive',
456+
expectedRootId: rootId,
457+
});
458+
const { controlDirectory } = await resolveExistingStorageRootControlDirectory(capability);
459+
return controlDirectory;
460+
}
461+
462+
/**
463+
* Resolves once the Host has begun shutting down. `draining` is the state the
464+
* kernel publishes before it does any shutdown work, and the registration is
465+
* removed near the end of that work, so either observation proves shutdown
466+
* started; the Host was serving this Client, so its registration existed.
467+
*/
468+
async function waitForHostShutdownStart(
469+
controlDirectory: string,
470+
timeoutMs: number,
471+
): Promise<void> {
472+
const deadline = Date.now() + timeoutMs;
473+
while (Date.now() < deadline) {
474+
const registration = await readHostRegistration(controlDirectory).catch(() => undefined);
475+
if (!registration || registration.state === 'draining') return;
476+
await new Promise((resolve) => setTimeout(resolve, 10));
477+
}
478+
throw new Error('owned Host did not begin shutting down after its first connection closed');
479+
}
480+
439481
function connectFailure(
440482
result:
441483
| Awaited<ReturnType<typeof connectOwnedRuntimeHostWithDependencies>>

0 commit comments

Comments
 (0)