Config: app.command, readyUrl and port 0 (refs #645)

COMPATIBILITY.md

@@ -154,7 +154,7 @@ Ported in part (the test name carries `Partial_` or a `PARTIAL` suffix):
 
 Not ported, because the feature is not in the port:
 
-- No `ListReporter`, run header, or live window; no JUnit or Markdown report renderer; no serial groups, workers, shards, runner, `e2e run`, `e2e init`, or `e2e list`; no JavaScript config loader; no `test.extend`; no managed process; no telemetry; no wordmark; no `projectId`; no `runWithRetries`: the `report-list`, `report-junit`, `report-markdown`, `cli-run`, `cli-boot`, `config-load`, `parallel-workers`, `managed-process`, `extended-fixtures`, `telemetry-*`, `wordmark`, `target-identity`, and `retry` tests, and the three `report-status` tests about the last attempt of a serial group
+- No `ListReporter`, run header, or live window; no JUnit or Markdown report renderer; no serial groups, workers, shards, runner, `e2e run`, `e2e init`, or `e2e list`; no JavaScript config loader; no `test.extend`; no telemetry; no wordmark; no `projectId`; no `runWithRetries`: the `report-list`, `report-junit`, `report-markdown`, `cli-run`, `cli-boot`, `config-load`, `parallel-workers`, `extended-fixtures`, `telemetry-*`, `wordmark`, `target-identity`, and `retry` tests, and the three `report-status` tests about the last attempt of a serial group
 - No value expectations (`expect-values` `truthy/falsy/null/undefined fail on the wrong value, and pass negated`), no `toHaveClass`, `frameLocator`, `*css=` captures, or click-modifier tests; the `waitForResponse`, `route`, and `unroute` predicate tests and the `an option ... does not take is refused` tests cannot happen, because C# overloads are typed
 - No run-wide usage accumulator (the port has `ModelUsage`, but no overflow-capping total or usage events), `withStallGuard`, `SessionStore`, `realmSlot`, or `verifyEndState` with `waitMs`: the `model-usage`, `tool-loop-stall`, `session-envelope`, `realm-slot`, and `trace-replay` end-wait tests
 - No `pixelFallback` observe option, `tap_at` or `select_at`, upload tool, or `schema` and `vision` options on `act`: the `agent-fallback-handoff`, `agent-act-pixel-tools`, `agent-act-verbs` hidden-file, `act-validation`, and `agent-screen-update` tests
@@ -227,7 +227,7 @@ Upstream's `4982d8d` (#1333) documents that the browser engine's semantic tree l
 
 `skills/create-verification-skill/` ports upstream's skill of the same name (commit `d85702a2`, `feat(skills): create-verification-skill generates a verify-<app> skill on e2e (#981)`). Like upstream, it is not bundled in `E2E.Cli` and `e2e guide` does not read it; install it with `npx skills add hardkoded/e2e-dotnet --skill create-verification-skill`. Differences:
 
-- The steps use what the port has. The launch is the app started by hand (the port has no `app.command`), the doctor is a smoke test run with `dotnet test --filter`, and the evidence is the `dotnet test` result and the test's own checks, since the port writes no report or trace page, and a video only when `video` is on.
+- The steps use what the port has. The launch is the target's `app.command` or the app started by hand, the doctor is a smoke test run with `dotnet test --filter`, and the evidence is the `dotnet test` result and the test's own checks, since the port writes no report or trace page, and a video only when `video` is on.
 - Not ported: the generated bug bash fan-out. It depends on `e2e.bugbash.config.ts`, which the port lacks, and runs many `e2e explore` commands at once, so a feature's charters are goals for a live MCP walk, and a finding is a bug once a repro test fails with `ASSERTION_FAILED`.
 - Not ported: `npx e2e init` as the setup step, `e2e list` as a config check, `--output` for proofs, and the `.gitignore` lines for `.e2e/bugbash/` and `.e2e/proof/`. The repo already ignores `.e2e/`.
 - The example under `references/example/` is `verify-todomvc` for `samples/TodoMvc`, in place of `verify-playground` for upstream's `apps/testbed`, which the port does not have. It has three features (adding, completing, filtering), not the playground's three (todos, forms, login).
@@ -249,7 +249,7 @@ Not ported: `e2e init`, which scaffolds the config, installs the skill into `.ag
 
 `E2E.Mobile.NUnit.MobileAppTest` and `E2E.Mobile.XUnit.V3.MobileAppTest` derive from `E2ETest` and make a device test one `using` and one config file. Upstream declares a device in `e2e.config.ts` and its runner owns the engine and the hosted device. The port's adapters do, so these classes do it. Every symbol here is .NET-only. Differences:
 
-- `e2e.config.json` accepts `"platform": "ios"` and `"android"`. A device target takes `app.bundleId` and `app.appPath`, as upstream's `app`. It does not take the other upstream app keys (`identity`, `environment`, `launchArguments`, `permissions`). The port still runs one target.
+- `e2e.config.json` accepts `"platform": "ios"` and `"android"`. A device target takes `app.bundleId` and `app.appPath`, as upstream's `app`. It takes `app.command` and `app.readyUrl` too (see [Config](#config)). It does not take the other upstream app keys (`identity`, `environment`, `launchArguments`, `permissions`). The port still runs one target.
 - `E2E_MOBILE_PLATFORM` and `E2E_MOBILE_APP_PATH` win over the config, so one project can run on iOS and Android. `E2E_MOBILE_DEVICE`, `E2E_MOBILE_PROVIDER` (`local`, `limrun`, `devflow`), `E2E_DEVFLOW_PORT`, and `E2E_DEVFLOW_APP` are .NET-only.
 - A hosted run leases one device on the first fixture and releases it when the process ends, because the adapters have no hook after the last test. A killed process is covered by the provider's hard timeout (15 minutes for Limrun).
 - `MobileAppHost` is one source file, compiled into both packages. `HostedDeviceProviders` (the provider each `E2E_MOBILE_PROVIDER` name makes) is a second, compiled into both packages and into `E2E.Cli`. `CreateMobileOptions`, `AppId`, `AppPath`, `Platform`, and `AttachesToRunningApp` are the members a test project overrides or reads.
@@ -487,9 +487,18 @@ Not ported:
 
 ## Config
 
-`E2ETest` loads the nearest `e2e.config.json` above the test assembly directory, then above the working directory, once per run. Supported keys: `targets` (one entry with `name`, `platform`, and `app.url`), `timeout`, `launchTimeout`, `actionTimeout`, `assertionTimeout`, `cleanupTimeout`, `retries`, `agents`, `cache` (`mode`, `dir`, `strict`), and `secrets`. Defaults, the CI cache demotion to `read-only`, `E2E_SECRET_<NAME>` overrides, the 6 code point secret minimum, and `INVALID_CONFIG` for unknown keys follow upstream.
+`E2ETest` loads the nearest `e2e.config.json` above the test assembly directory, then above the working directory, once per run. Supported keys: `targets` (one entry with `name`, `platform`, and `app` with `url`, `command`, and `readyUrl`), `timeout`, `launchTimeout`, `actionTimeout`, `assertionTimeout`, `cleanupTimeout`, `retries`, `agents`, `cache` (`mode`, `dir`, `strict`), and `secrets`. Defaults, the CI cache demotion to `read-only`, `E2E_SECRET_<NAME>` overrides, the 6 code point secret minimum, and `INVALID_CONFIG` for unknown keys follow upstream.
 
-`app.url` is normalized as upstream does, with `System.Uri` instead of WHATWG: a missing scheme becomes `https://`, or `http://` for a loopback host. Userinfo, a query, a fragment, a scheme other than http(s), and plain HTTP on a non-loopback host are `INVALID_APP_URL`, with upstream's messages. A base URL set in code (`E2EFixture.BaseUrl`, `E2ESessionOptions.BaseUrl`) follows the same rule. Port `0` on `127.0.0.1` or `[::1]` is accepted by the parser, but the port has no `app.command` to start the app on a free port, so it is `INVALID_CONFIG`; any other host with port `0` is `INVALID_APP_URL`. Keys of the old .NET shape (`app`, `engine`, `agent`, `timeouts`) fail with a hint to the upstream key.
+`app.url` is normalized as upstream does, with `System.Uri` instead of WHATWG: a missing scheme becomes `https://`, or `http://` for a loopback host. Userinfo, a query, a fragment, a scheme other than http(s), and plain HTTP on a non-loopback host are `INVALID_APP_URL`, with upstream's messages. A base URL set in code (`E2EFixture.BaseUrl`, `E2ESessionOptions.BaseUrl`) follows the same rule. Port `0` on `127.0.0.1` or `[::1]` asks for a free port and needs an `app.command` to start the app on it (`INVALID_CONFIG` without one, or with `reuseExisting`); any other host with port `0` is `INVALID_APP_URL`. A base URL set in code with port `0` is `INVALID_CONFIG`, since only `app.command` starts an app on a free port. Keys of the old .NET shape (`app`, `engine`, `agent`, `timeouts`) fail with a hint to the upstream key.
+
+`app.command` and `app.readyUrl` port upstream's `config/command.ts` and `config/app.ts` (`CommandConfig`, `Command.Normalize`, `TargetApps.ResolveTargetApp`), and `ManagedProcess` ports `run/managed-process.ts` (readiness polling, the preflight probe and `reuseExisting`, the startup-failure tail of `command.log`, `inheritEnv`). `executable` is started with an argument list, never a shell. `{port}` expands in `args`, in `env` values, and in `readyUrl`, and a `url` on port `0` takes a free loopback port (`AppPorts.AllocateTargetPorts`, upstream's `allocateTargetPorts`). The first test, `e2e explore`, or `e2e mcp` session that needs the command starts it (`E2EFixture.StartSessionAsync`, `Explore`, and the MCP session call `RunApps.StartAsync`) and it runs until the process ends, where `ManagedProcess.KillManagedProcessGroups` takes it down on the way out; on Windows the process runs in its own job object (`ProcessJob`, .NET-only), which ends it and what it started when it is stopped, and when this process is killed outright, as a test host is once its run ends. On Unix a descendant is read with `pgrep -P` when the app is stopped, so one that outlives a leader that exited earlier is not found. The app command differs from upstream's in these ways:
+
+- There is no runner, so a command starts once per test process (a .NET-only holder, `RunApps`), not once per run. Two processes that declare one command, such as `dotnet test` and `e2e mcp`, each start their own, so give a shared app a fixed port and `reuseExisting`. Upstream's `SharedAppProcesses` pool, `declaredCommands`, and the engine `prepare` hooks that `startDeclaredProcesses` shares a provision with are not ported (one target, no pool).
+- A started process is stopped with `SIGTERM` and then `Process.Kill(entireProcessTree: true)` after `shutdownTimeout`. On Windows, which has no signal to ask a process to stop, the first step is the kill. There are no process groups, so a descendant that outlives its leader is not signalled; the two `managed-process` integration tests about that (`stops a surviving child after the command leader exits on SIGTERM`, `lets a child finish its shutdown after the leader exits`) are not ported. `stdout` and `stderr` are read together into `log` (or dropped when there is none), line by line as the process writes them.
+- A cancelled token ends `ManagedProcess.StartAsync` by throwing `OperationCanceledException` after taking the process down, where upstream's `start` returns and the caller reads the signal. `startupTimeout` and `shutdownTimeout` are `TimeSpan`s (`CommandConfig.StartupTimeout`, `ShutdownTimeout`), and a value over 2147483647 milliseconds, which a .NET timer cannot wait, is `INVALID_CONFIG`. A readiness probe still waiting when the process exits ends there.
+- JSON has no `secrets.get()` handle, so the message that names one for a string value of `args`, `env`, or `url` is not reachable, and an `env` entry of `null` stands for upstream's `undefined`. `command.log` containment resolves symlinks as `relativeToProjectRoot` does (`Paths.RelativeToProjectRoot`). An app key outside the port's set is named in the port's own words (`unknown targets[0].app key "readyURL"`), where upstream says `target "web" app has unknown key`; the command's own keys use upstream's message.
+- The resolved app has `Base`, `PortRequest`, `Command`, and `ReadyUrl`; `site`, `environment`, and `identity` are not ported, and neither is the config digest (`digestCommand`, `digestTargetApp`), so the `config.test.ts` tests that digest an app, and the `identity` and `environment` tests, are not ported.
+- Tests: `ManagedProcess/` ports the `it` of `tests/unit/managed-process.test.ts` and `ManagedProcessIntegration/` those of `tests/integration/managed-process.test.ts`, with the program they start (`E2E.Tests.App`) where upstream runs `node -e`; the notice tests wait out a real budget where upstream moves a fake clock. `AppPorts/` ports `app-ports.test.ts` (partial: one target, and no `identity` or digest). `Config/` ports the command and `readyUrl` tests of `config.test.ts`. `RunApps/` is .NET-only. Not ported: `target-processes.test.ts` and `shared-app-processes.test.ts`, which drive the runner.
 
 Differences:
 

E2E.slnx

@@ -36,6 +36,7 @@
     <Project Path="tests/E2E.NUnit.Tests/E2E.NUnit.Tests.csproj" />
     <Project Path="tests/E2E.TestMu.Mobile.Tests/E2E.TestMu.Mobile.Tests.csproj" />
     <Project Path="tests/E2E.TestMu.Tests/E2E.TestMu.Tests.csproj" />
+    <Project Path="tests/E2E.Tests.App/E2E.Tests.App.csproj" />
     <Project Path="tests/E2E.Tests.Schema/E2E.Tests.Schema.csproj" />
     <Project Path="tests/E2E.Tests/E2E.Tests.csproj" />
     <Project Path="tests/E2E.WinApp.Tests/E2E.WinApp.Tests.csproj" />

docs-site/public/api/E2E.CommandConfig.html

@@ -0,0 +1,13 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<link rel="canonical" href="https://hardkoded.github.io/e2e-dotnet/api/E2E/CommandConfig/">
+<meta http-equiv="refresh" content="0; url=/e2e-dotnet/api/E2E/CommandConfig/">
+<title>Redirecting…</title>
+<script>location.replace("/e2e-dotnet/api/E2E/CommandConfig/" + (location.hash || ""));</script>
+</head>
+<body>
+<p><a href="/e2e-dotnet/api/E2E/CommandConfig/">Continue</a></p>
+</body>
+</html>

docs-site/src/content/docs/api/E2E/AppConfig.md

@@ -21,7 +21,7 @@ public sealed class AppConfig
 
 **Inheritance:** [object](https://learn.microsoft.com/dotnet/api/system.object)
 
-[View source](https://github.com/hardkoded/e2e-dotnet/blob/main/src/E2E/Config/E2EConfig.cs#L700)
+[View source](https://github.com/hardkoded/e2e-dotnet/blob/main/src/E2E/Config/E2EConfig.cs#L708)
 
 ## Properties
 
@@ -40,7 +40,7 @@ The build a device target installs, relative to the folder of `e2e.config.json`.
 
 **Value:** [string](https://learn.microsoft.com/dotnet/api/system.string)
 
-[View source](https://github.com/hardkoded/e2e-dotnet/blob/main/src/E2E/Config/E2EConfig.cs#L712)
+[View source](https://github.com/hardkoded/e2e-dotnet/blob/main/src/E2E/Config/E2EConfig.cs#L720)
 
 <a id="E2E_AppConfig_BundleId_"></a>
 <a id="E2E_AppConfig_BundleId"></a>
@@ -57,7 +57,43 @@ The bundle id (iOS) or package (Android) of an installed app a device target lau
 
 **Value:** [string](https://learn.microsoft.com/dotnet/api/system.string)
 
-[View source](https://github.com/hardkoded/e2e-dotnet/blob/main/src/E2E/Config/E2EConfig.cs#L709)
+[View source](https://github.com/hardkoded/e2e-dotnet/blob/main/src/E2E/Config/E2EConfig.cs#L717)
+
+<a id="E2E_AppConfig_Command_"></a>
+<a id="E2E_AppConfig_Command"></a>
+
+### Command
+
+
+```csharp
+public CommandConfig? Command { get; init; }
+```
+
+
+The command that starts the app (upstream's `app.command`), started before the first test and stopped when the process ends.
+`{port}` in [Args](/api/E2E/CommandConfig/#E2E_CommandConfig_Args) and [Env](/api/E2E/CommandConfig/#E2E_CommandConfig_Env) is the port the app is served on. A
+[Url](#E2E_AppConfig_Url) with port 0 asks for a free one, and needs this.
+
+**Value:** [CommandConfig](/api/E2E/CommandConfig/)
+
+[View source](https://github.com/hardkoded/e2e-dotnet/blob/main/src/E2E/Config/E2EConfig.cs#L727)
+
+<a id="E2E_AppConfig_ReadyUrl_"></a>
+<a id="E2E_AppConfig_ReadyUrl"></a>
+
+### ReadyUrl
+
+
+```csharp
+public string? ReadyUrl { get; init; }
+```
+
+
+The URL that answers once [Command](#E2E_AppConfig_Command) has started the app (upstream's `app.readyUrl`). Unset, [Url](#E2E_AppConfig_Url).
+
+**Value:** [string](https://learn.microsoft.com/dotnet/api/system.string)
+
+[View source](https://github.com/hardkoded/e2e-dotnet/blob/main/src/E2E/Config/E2EConfig.cs#L730)
 
 <a id="E2E_AppConfig_Url_"></a>
 <a id="E2E_AppConfig_Url"></a>
@@ -75,4 +111,4 @@ Relative URLs resolve against it.
 
 **Value:** [string](https://learn.microsoft.com/dotnet/api/system.string)
 
-[View source](https://github.com/hardkoded/e2e-dotnet/blob/main/src/E2E/Config/E2EConfig.cs#L706)
+[View source](https://github.com/hardkoded/e2e-dotnet/blob/main/src/E2E/Config/E2EConfig.cs#L714)