Rewrite text downloads through the secret ledger (#682)

COMPATIBILITY.md

@@ -553,7 +553,7 @@ The walk goes through open shadow roots and closed ones that page script attache
 - A runner calls `WebEngine.PrepareAsync`, `Init`, `FinishAsync`, and `DisposeAsync` as upstream's `prepare`, `init`, `finish`, and `dispose`, with `EnginePrepareInfo`, `EnginePrepareResult`, `EngineInitInfo`, and `EngineFinishInfo`; an attempt is `StartAsync` and the dispose of its session (`startAttempt` and `endAttempt`). The hand-off to a worker travels in the same `E2E_WEB_BROWSERS_<TARGET>_<DIGEST>` environment variable, capped at 16384 bytes. The port has no runner, so an engine whose `Init` never ran leases for the one attempt it starts, as upstream's standalone run does: it prepares one slot of a run of its own, reading `EngineStartOptions.Env` (the process environment by default), and gives the lease back, and sweeps, when the session is disposed, with the default cleanup timeout for each step (the engine does not see the session's). `EngineStartOptions.AttemptId` names the attempt to a per-attempt lease
 - Differences from upstream's lease flow, all from the port's engine. It keeps no browser connection between attempts, so a worker dials the endpoint of the lease it holds at every attempt, and leases a replacement only after an attempt attached to a browser that was no longer connected when the attempt ended (upstream: when the next attempt finds the connection dropped). A start that fails gives its lease back itself, since there is no `endAttempt` to call after it. A lease that arrives too late, after the attempt was cancelled or the worker disposed, is given back under a cleanup budget of its own (upstream passes the signal that already aborted). Upstream's `state` capability, which an attempt-scoped provider drops, does not exist in the port, so only `app.clearState()` is refused (`UNSUPPORTED_CAPABILITY`, naming the scope)
 - The lease tests are `BrowserProvider/` in `E2E.Tests`, from `browser-provider.test.ts`. Not ported: "rejects a sweep that is not a function" and "rejects a record that is not a function" (both are methods of the interface), "releases the replacement on dispose even when ending the attempt failed" (the port's `DisposeAsync` does not end an attempt), and "refuses, at the start and named, a recording without a start time and a stop" (a recording is typed). The other tests of the recording group are ported in full, on a real Chromium the provider leases (upstream drives a scripted page), through `StartVideoAsync` and `StopVideoAsync` of the engine session. Also not ported: "knows a recording by a parseable start time and a stop" of `provider-recording.test.ts`. The rest of that file is `ProviderRecording/`, and `provider-downloads.test.ts` is `ProviderDownloads/`, on a real Chrome over CDP, with the `Browser` fixture over the engine session
-- `Browser.OnDialogAsync` and `Browser.WaitForDownloadAsync` are upstream's `onDialog` and `waitForDownload`. `DialogHandler` is upstream's `DialogHandler` (`DialogHandler.Accept` and `Dismiss` are its `'accept'` and `'dismiss'`, and a `Func<Dialog, Task>` converts to it), `Dialog.Type` is a `DialogType` (`Alert`, `Confirm`, `Prompt`, `BeforeUnload`) where upstream's is a string union, and `OnDialogAsync` returns `Func<Task>` for upstream's unsubscribe. The engine side is `DialogRouter` (`dialogs.ts`) and `ErrorLatch` and `WebSupport.SanitizeFilename` (`support.ts`), and `WebSession.ArtifactPath` is the surface's `artifactPath`. Differences: the engine session has `OnDialogAsync` and `WaitForDownloadAsync` (default members of `IBrowserSession`), and the `Browser` fixture attaches the file to the step (`attachArtifact`), since the port's fixture holds no context; `BrowserDownload` is the result upstream types inline. A dialog handler's failure is latched and fails the next step, as a route handler's does, but the port has no `settleAttempt`, so one that no step follows is dropped (upstream fails the attempt with it). A test framework's assertion is not an `E2EException`, so one thrown in a handler fails the next step as `ENGINE_FAILURE`, `dialog handler failed: ...`, where upstream's `expect` failure keeps its `ASSERTION_FAILED`. A download is a `download` artifact of the `browser.waitForDownload` step, with the media type of its extension. The port has no artifact store, so upstream's rewrite of a text-like download through the secret ledger has no port: the record says `incomplete`, as upstream's does for a session without secrecy. Not ported: "a dialog handler assertion that no step follows" and "fails a %s handler assertion at the attempt end when no step follows it" of `web-platform.test.ts` (no `settleAttempt`), and nothing else. The frame tests of `dialogs-frames.e2e.ts` are `DialogsFrames/FramesTests`, and the download tests trigger the link with `browser.locator('a[download]')` as upstream does
+- `Browser.OnDialogAsync` and `Browser.WaitForDownloadAsync` are upstream's `onDialog` and `waitForDownload`. `DialogHandler` is upstream's `DialogHandler` (`DialogHandler.Accept` and `Dismiss` are its `'accept'` and `'dismiss'`, and a `Func<Dialog, Task>` converts to it), `Dialog.Type` is a `DialogType` (`Alert`, `Confirm`, `Prompt`, `BeforeUnload`) where upstream's is a string union, and `OnDialogAsync` returns `Func<Task>` for upstream's unsubscribe. The engine side is `DialogRouter` (`dialogs.ts`) and `ErrorLatch` and `WebSupport.SanitizeFilename` (`support.ts`), and `WebSession.ArtifactPath` is the surface's `artifactPath`. Differences: the engine session has `OnDialogAsync` and `WaitForDownloadAsync` (default members of `IBrowserSession`), and the `Browser` fixture attaches the file to the step (`attachArtifact`), since the port's fixture holds no context; `BrowserDownload` is the result upstream types inline. A dialog handler's failure is latched and fails the next step, as a route handler's does, but the port has no `settleAttempt`, so one that no step follows is dropped (upstream fails the attempt with it). A test framework's assertion is not an `E2EException`, so one thrown in a handler fails the next step as `ENGINE_FAILURE`, `dialog handler failed: ...`, where upstream's `expect` failure keeps its `ASSERTION_FAILED`. A download is a `download` artifact of the `browser.waitForDownload` step, with the media type of its extension. The port has no artifact store (`ArtifactStore`, `createAttemptArtifacts`), so `AttemptScope.AttachArtifact` does what upstream's `sink.register` does for a download: when the session has a secret value and the media type is text-like (`text/*` or `application/json`), it rewrites the file through the redactor (`ArtifactPaths.RedactDownload`, upstream's `redactDownload`) and the record says `complete`; any other download, one it cannot decode as UTF-8, one containing NUL bytes, or a session without a secret value stays as served and the record says `incomplete`. Unlike upstream, the port rejects NUL-containing text conservatively so BOM-less UTF-16 cannot hide secrets between NULs and still be marked `complete`. `DownloadRedactionTests` extends the binary-download case with a BOM-less UTF-16LE CSV regression. `AttemptScope.ArtifactsDir` is upstream's `ArtifactSink.dir`. Not ported: the two download tests of `filled-secrets.test.ts` and the download case of `engine-secrets.test.ts` (no credentials config, runner fixtures, or store). Not ported: "a dialog handler assertion that no step follows" and "fails a %s handler assertion at the attempt end when no step follows it" of `web-platform.test.ts` (no `settleAttempt`), and nothing else. The frame tests of `dialogs-frames.e2e.ts` are `DialogsFrames/FramesTests`, and the download tests trigger the link with `browser.locator('a[download]')` as upstream does
 - With `ReconnectEndpoint`, as upstream, each attempt rides the default context of a fresh browser from `CdpEndpoint`, refuses a browser an earlier attempt of the same `WebEngine` rode, and reconnects once to the same browser and page after a transport drop. `Headers`, `BasicAuth`, `UserAgent`, `Locale`, and `TimezoneId` are `INVALID_CONFIG`, and `app.clearState()` is `UNSUPPORTED_CAPABILITY`. After a reconnect, `PerformAsync`, the agent's key press, and the viewport swipe fail with `NODE_STALE` until the next observation; the browser fixture's keyboard and mouse are direct input and keep working. As upstream, the reconnect is spent from the budget of the operation that found the transport dropped, both for the CDP dial and for the operation itself
 - `BasicAuth` takes a `Secret` password. As upstream, the attempt redacts the password and the `Authorization` credential the browser sends with it, padded and not, as text only: it does not withhold screenshot pixels, and the agent cannot fill it by name. The port has no engine secrets to declare, so the engine resolves the password through the attempt when it starts. A plain string password is not redacted
 - An `InitScripts` entry is a `WebInitScript`: source (a string converts to one), `WebInitScript.FromPath`, or `WebInitScript.FromFunction`. A C# lambda cannot run in the page, so a function is its JavaScript source. A relative path resolves against the project root (`EngineStartOptions.ProjectRoot`, which `E2ESession` passes from `E2ESessionOptions.ProjectRoot`). Each attempt reads the files when its session starts; upstream reads them once per process
@@ -673,6 +673,8 @@ Upstream writes `report.json`, a `report-1` document of the whole run (`report/b
 
 ## App and browser fixtures
 
+Text downloads are rescanned through the final secrecy ledger before their report records are read, including values filled after attachment. Fragment scans also recognize ASCII secrets written with JSON Unicode escapes, once or twice quoted. These safeguards extend upstream's attachment-time scan and ASCII spelling shortcut; stream redaction keeps its existing buffering. The .NET-only regressions are `DownloadRedactionTests.Rescans_downloads_with_values_registered_after_attachment`, `Reports_a_previously_complete_download_incomplete_when_the_final_scan_fails`, and `Redacts_JSON_Unicode_escaped_ASCII_secrets_before_reporting_complete`.
+
 `App.RestartAsync` closes the tab and opens a new one in the same browser context, so cookies and storage survive. `App.ClearStateAsync` replaces the context with a clean one. Both then reopen the app at its base URL, as upstream, and keep a viewport set with `Browser.SetViewportAsync`. Without a base URL they leave a blank page.
 
 Differences from upstream:

docs-site/src/content/docs/get-started/web.mdx

@@ -107,6 +107,9 @@ The first run asks the model to perform each `ActAsync`. A passing run records e
 
 ## Next
 
+Text downloads are rescanned before reporting with all secrets known by then,
+including later fills and ASCII secrets written as JSON Unicode escapes.
+
 <CardGrid>
   <LinkCard title="Dariten sample" href={`${import.meta.env.BASE_URL}samples/dariten/`} description="A locator test and an agent test against a live public app." />
   <LinkCard title="TodoMVC sample" href={`${import.meta.env.BASE_URL}samples/todomvc/`} description="Playwright's TodoMVC suite rewritten as agent steps." />

skills/e2e-dotnet/SKILL.md

@@ -11,6 +11,9 @@ judge the screen. `Screen`, `App`, `Browser`, and `Expect` make exact
 interactions and checks without a model. A verified agent step is recorded
 and replayed on the next run with no model call; judgments still run live.
 
+Text downloads are rescanned before reporting with all secrets known by then,
+including later fills and JSON Unicode escapes of ASCII secrets.
+
 ## Start here: pick the reference for the job
 
 Read the one reference that matches the request before doing anything else.
@@ -38,6 +41,8 @@ trip. The installed tool prints the same text with `e2e guide <topic>`
 
 Two sibling skills install with `npx skills add hardkoded/e2e-dotnet --skill <name>`. `create-verification-skill` writes a project-local `verify-<app>` skill. `migrate-to-e2e` moves an existing Playwright .NET, Selenium .NET, Appium, Cypress, Detox, or Maestro suite to e2e one slice at a time, with the old suite running beside the new one and a `MIGRATION.md` progress file. Hosted Limrun devices are covered in topic `mobile`.
 
+Text downloads are redacted only when they are UTF-8 without NUL bytes; other files stay as served and `incomplete`. See `writing-tests` for download handling.
+
 ## Quick start
 
 A config with one web target and a model, and a test that drives one goal

skills/e2e-dotnet/references/writing-tests.md

@@ -453,6 +453,9 @@ await Browser.OnDialogAsync(async dialog =>
 
 ### Download a file
 
+Before reporting, text downloads are scanned again with the final secrecy ledger,
+including secrets filled later and ASCII secrets written as JSON Unicode escapes.
+
 ```csharp
 var file = await Browser.WaitForDownloadAsync(() =>
     Screen.GetByRole("link", "Export CSV").TapAsync());
@@ -463,7 +466,7 @@ The file is a `download` artifact of the step, saved under the attempt's
 `downloads` folder with a counted name (`downloads/001-report.csv`). Read it
 through `AbsolutePath`, which works with a hosted browser whose provider
 serves `Downloads` too; `Path` is relative to the attempt's artifact
-directory, not to the project. `SuggestedFilename` is the name the page or
+directory, not to the project. A text download (`text/*` or `application/json`) is rewritten in place when the session holds a secret value: every value, fragment, and encoded form of it becomes `<secret:name>`, a byte order mark stays, and the report records the artifact as `complete`; a binary file, a file that is not UTF-8 or contains NUL bytes (including BOM-less UTF-16), or a session with no secret leaves it as served and `incomplete`. `SuggestedFilename` is the name the page or
 server gave it. A trigger that throws fails the call with its own error. One
 that returns and starts no download within the timeout fails with `ACTION_FAILED`, naming how long it
 waited and how long the trigger took. Without `Downloads` on the provider,

src/E2E/Agent/Agent.cs

@@ -12,6 +12,7 @@
 using E2E.Cache;
 using E2E.Engine;
 using E2E.Internal;
+using E2E.Reports;
 
 namespace E2E;
 
@@ -1298,18 +1299,36 @@ public Redactor Redactor
 
     public List<PendingAct> Acts { get; } = [];
 
-    private readonly List<(string Id, string Kind, string Path)> _attached = [];
+    private readonly List<(string Id, string Kind, string Path, ArtifactRedaction? Redaction)> _attached = [];
+
+    /// <summary>The attempt's artifact directory (upstream's <c>ArtifactSink.dir</c>); null when the scope has none, as in a unit test.</summary>
+    public string? ArtifactsDir { get; init; }
 
     /// <summary>
-    /// The files fixtures kept for the attempt, such as a download, as the id the step names, the kind, and the path under the attempt's
-    /// artifact directory (upstream's <c>attachArtifact</c>).
+    /// The files fixtures kept for the attempt, such as a download, as the id the step names, the kind, the path under the attempt's
+    /// artifact directory (upstream's <c>attachArtifact</c>), and the redaction the runner claims for it when it rewrote it (null when
+    /// the kind decides).
     /// </summary>
-    public IReadOnlyList<(string Id, string Kind, string Path)> Attached
+    public IReadOnlyList<(string Id, string Kind, string Path, ArtifactRedaction? Redaction)> Attached
     {
         get
         {
             lock (_attached)
             {
+                // Reports must use the final ledger, including values a later fill registered.
+                if (ArtifactsDir is { } directory && Secrecy.RedactsDownloads(Redactor))
+                {
+                    var redactor = Redactor;
+                    for (var index = 0; index < _attached.Count; index++)
+                    {
+                        var entry = _attached[index];
+                        if (entry.Kind == "download" && ArtifactPaths.IsTextLike(ArtifactPaths.MediaTypeFor(entry.Path)))
+                        {
+                            _attached[index] = (entry.Id, entry.Kind, entry.Path, ArtifactPaths.RedactDownload(Path.Combine(directory, entry.Path), redactor));
+                        }
+                    }
+                }
+
                 return [.. _attached];
             }
         }
@@ -1318,11 +1337,24 @@ public Redactor Redactor
     /// <summary>Keeps <paramref name="relative"/> as an artifact of the running step, or of the last one when none runs.</summary>
     public void AttachArtifact(string kind, string relative)
     {
+        // A text-like download is rewritten through the session's ledger before it is measured, so the record claims what that did
+        // (upstream's `sink.register`, `run/artifacts.ts`). Any other download is bytes the app served: incomplete.
+        ArtifactRedaction? redaction = null;
+        if (kind == "download" && ArtifactsDir is { } directory)
+        {
+            redaction = ArtifactRedaction.Incomplete;
+            var redactor = Redactor;
+            if (Secrecy.RedactsDownloads(redactor) && ArtifactPaths.IsTextLike(ArtifactPaths.MediaTypeFor(relative)))
+            {
+                redaction = ArtifactPaths.RedactDownload(Path.Combine(directory, relative), redactor);
+            }
+        }
+
         string id;
         lock (_attached)
         {
             id = kind + "-" + (_attached.Count(entry => entry.Kind == kind)).ToString(System.Globalization.CultureInfo.InvariantCulture);
-            _attached.Add((id, kind, relative));
+            _attached.Add((id, kind, relative, redaction));
         }
 
         Steps.AttachArtifact(id);