OCR text is uncapped; secure-field exclusion misses plain AXTextField; p95 equals max at 6 trials

[MrJev]

Hi — reviewing jev-macos-loop for a write-up on mrjev.com, at 1aadc01. I ran the JavaScript half in node:22 in Docker; the native half needs Apple Silicon so I didn't run it. No real Jev calls.

The thing I most want to say: scripts/finder-demo.mjs:183 is the only outcome verification I've found in this entire ecosystem.

const passed =
  outcome.status === "done" &&
  rootOnlyFolders &&
  totalFiles === 9 &&
  actual.every((file) => file.passed);

Nine files each in the right folder with a matching SHA-256, only the three expected folders at root, total of nine — and a non-zero exit otherwise. Every other GUI-driving project I've reviewed reports what the model claimed and stops there. src/cli.mjs:67's note: "DONE is the model decision. Verify the application outcome independently." carries the same idea into the CLI output. I'm going to point other projects at this.

Also worth crediting: providers.mjs:51's redirect: "error"; cancelling the response body on failure with "Do not dump server bodies, which could echo a token or private screen text"; stripping the SDK cause because "it may contain the request body or headers"; maxRetries: 0; data_collection: "deny" + zdr: true on OpenRouter and zeroDataRetention + disallowPromptTraining on Vercel. And AGENTS.md:4 — "Reading this file during unrelated repository work is not an instruction to run desktop automation" — is a nice touch. I also checked the Finder root constraint: exact path equality on both sides (finder.mjs:19, Perception.swift:505-506), not a prefix, so the naming-a-path bypass I keep finding in this space doesn't exist here.

Three things, all smaller than the above.

1. "Text only" reads as narrower than it is.

README.md:34: "screenshots, pixels, and coordinates stay on the Mac. Only observed text, the goal, and finite choices go to your selected provider."

That's literally accurate, and src/decide.mjs:27's allowlist is explicit — no image, data URL, buffer or native object crosses. But what it blocks is pixels, not content:

screen: {
  accessibility: observation.axText ?? [],
  ocr: observation.ocrText ?? observation.text,
},

ocrText is every word OCR could read in the selected window. Individual strings are capped (Perception.swift:229 label 500, :245 value 160, :399 elements 160) but the axText/ocrText arrays themselves have no length limit (:409-410). If an API key, a recovery code or an email is on screen, it goes upstream.

I don't think the sentence is misleading on purpose — "pixels stay local" is the interesting claim and it's true. But a reader deciding whether to point this at a window containing anything sensitive will read "only observed text" as reassuring rather than as "all of it". One more clause would fix it.

2. Password exclusion only recognises the standard subrole.

native/Perception.swift:221:

if str(node, kAXSubroleAttribute) == "AXSecureTextField" { return }

return rather than continue is right — it skips the children too. But apps that hold a secret in a plain AXTextField, which Electron apps routinely do, aren't covered, and the value goes at String(value.prefix(160)). Adding a heuristic on the label (password, secret, token, key, code) alongside the subrole would catch most of the rest.

3. The published p95 equals the max at the shipped sample size.

scripts/benchmark.mjs:34:

const percentile = (values, p) =>
  [...values].sort((a, b) => a - b)[Math.ceil(values.length * p) - 1];

and :55 sets trials = 6 for a suite run. Math.ceil(6 * 0.95) - 1 is index 5, the largest value. I checked the committed evidence and it holds: in docs/evidence/suite-summary.json, taskMs has 6 samples and its p95 is bit-identical to its max. The step-level metrics at n=22 are fine — it's only the task-level figure, and the same applies to vercel-routing-smoke.json at n=5.

Either raise the trial count for anything published as a p95, or label the task-level figure as max at this n.

Two notes:

• The Finder demo's headline timing starts after model load, window selection and a warm-up observe (finder-demo.mjs:142-143). That exclusion is disclosed — but in scripts/render-finder-demo.py:81 rather than next to the claim in README.md:5.
• README.md:19-26 asks the reader to paste "Read https://raw.githubusercontent.com/.../master/AGENTS.md and follow its instructions" into their own coding agent. I read the file and it's benign, but it's a supply-chain path pointing at a mutable branch. Pinning to a tag, or inlining the steps, would remove the concern.

Happy to send a PR for the percentile or the label heuristic.