diff --git a/docs/research/issue-26-electron-control.md b/docs/research/issue-26-electron-control.md new file mode 100644 index 0000000..1176549 --- /dev/null +++ b/docs/research/issue-26-electron-control.md @@ -0,0 +1,215 @@ +# Issue #26: one-off Electron control without macOS Accessibility permission + +## Finding + +The safest practical one-off path is a disposable detached worktree, a fresh +Electron profile, and a loopback-only Chromium DevTools Protocol (CDP) port. +Launch the built Electron entrypoint with `--remote-debugging-port=`, attach a temporary Node/TypeScript Playwright harness with +`chromium.connectOverCDP()`, and drive the renderer through semantic locators. +This uses Chromium's renderer protocol, so it does not require macOS +Accessibility permission or synthetic OS mouse/keyboard events. + +This is suitable for live renderer evidence: the harness attaches to the real +Electron process and performs real DOM actions, while collecting screenshots, +ARIA snapshots, and renderer console/page errors. It is not full desktop/UI +automation evidence: native macOS dialogs and other OS-level surfaces remain +outside the renderer. It also does not, by itself, observe Electron main-process +JavaScript; collect that separately from Electron's log file. + +## Disposable runbook + +Run from the repository, preserving the existing checkout and its data: + +```sh +AUDIT_WT="../local-anara-issue-26-audit" +git worktree add --detach "$AUDIT_WT" HEAD +cd "$AUDIT_WT" +npm ci +npm run build +mkdir -p .audit/{artifacts,user-data} +PORT=$(node -e "const net=require('node:net');const s=net.createServer();s.listen(0,'127.0.0.1',()=>{console.log(s.address().port);s.close()})") +ELECTRON_ENABLE_LOGGING=1 ELECTRON_LOG_FILE="$PWD/.audit/artifacts/electron.log" \ + ./node_modules/.bin/electron out/main/index.js \ + --remote-debugging-port="$PORT" \ + --user-data-dir="$PWD/.audit/user-data" \ + >.audit/artifacts/electron-stdout.log 2>.audit/artifacts/electron-stderr.log & +ELECTRON_PID=$! +for attempt in {1..100}; do + kill -0 "$ELECTRON_PID" 2>/dev/null || break + curl --silent --fail "http://127.0.0.1:$PORT/json/version" >/dev/null && break + sleep 0.1 +done +kill -0 "$ELECTRON_PID" +curl --silent --fail "http://127.0.0.1:$PORT/json/version" >/dev/null +lsof -t -iTCP:"$PORT" -sTCP:LISTEN | grep -Fx "$ELECTRON_PID" +``` + +Use a free high port selected for this run, verify that the CDP endpoint belongs +to this process, and never expose it beyond loopback. The Electron command-line +switch documentation says `--remote-debugging-port` enables remote debugging +over HTTP and that `--enable-logging`/`--log-file` persist Chromium logging: +[Electron supported command-line switches](https://www.electronjs.org/docs/latest/api/command-line-switches). +The same documentation shows that command-line switches are Chromium controls, +not application arguments. `--inspect` is a different V8 inspector for the main +process, as documented in [Electron's main-process debugging guide](https://www.electronjs.org/docs/latest/tutorial/debugging-main-process). + +Before closing, record the launcher's current descendant PIDs from the process +tree. Close the app through the harness, wait boundedly for the launcher and +every recorded descendant to exit, and verify the loopback endpoint is closed. +If graceful close fails, stop the cleanup and inspect the recorded numeric PIDs; +do not guess at a broader kill target or remove the worktree: + +```sh +AUDIT_DESCENDANT_PIDS=$(AUDIT_PARENT_PID="$ELECTRON_PID" node -e ' +const {execFileSync}=require("node:child_process"); +const root=Number(process.env.AUDIT_PARENT_PID); +const rows=execFileSync("ps",["-axo","pid=,ppid="]).toString().trim().split("\n").map(line=>line.trim().split(/\s+/).map(Number)); +const children=new Map(); for(const [pid,ppid] of rows) children.set(ppid,[...(children.get(ppid)||[]),pid]); +const found=[]; const visit=pid=>{for(const child of children.get(pid)||[]){found.push(child);visit(child)}}; visit(root); +process.stdout.write(found.join(" ")); +') +# Have the Playwright/CDP harness close Electron here. +for attempt in {1..100}; do kill -0 "$ELECTRON_PID" 2>/dev/null || break; sleep 0.1; done +! kill -0 "$ELECTRON_PID" 2>/dev/null +for attempt in {1..100}; do + audit_descendants_alive=0 + for pid in $AUDIT_DESCENDANT_PIDS; do kill -0 "$pid" 2>/dev/null && audit_descendants_alive=1; done + [ "$audit_descendants_alive" -eq 0 ] && break + sleep 0.1 +done +for pid in $AUDIT_DESCENDANT_PIDS; do ! kill -0 "$pid" 2>/dev/null; done +wait "$ELECTRON_PID" +! curl --silent --fail "http://127.0.0.1:$PORT/json/version" +cd - +git worktree remove "$AUDIT_WT" +``` + +Copy `.audit/artifacts` out before removal. A non-force worktree removal should +refuse if untracked or modified audit material remains; inspect rather than +bypassing that refusal. Do not use a personal profile or the normal repository's +`data/` directory. Vellum resolves paper files and SQLite relative to its current +working directory, so the detached worktree also isolates those files from the +user's checkout. + +## Temporary Playwright harness + +Install Playwright only in the disposable worktree (do not commit the package +or harness), then use the CDP endpoint: + +```ts +import { chromium } from 'playwright' + +const browser = await chromium.connectOverCDP(`http://127.0.0.1:${port}`) +const context = browser.contexts()[0] +const page = context.pages()[0] + +page.on('console', message => appendJson('renderer-console.jsonl', { + type: message.type(), text: message.text(), location: message.location(), +})) +page.on('pageerror', error => appendJson('renderer-errors.jsonl', { + message: error.message, stack: error.stack, +})) +page.on('requestfailed', request => appendJson('request-failures.jsonl', { + url: request.url(), method: request.method(), failure: request.failure(), +})) + +await page.locator('body').ariaSnapshot({ mode: 'default' }) + .then(snapshot => writeText('aria-before.yml', snapshot)) +await page.screenshot({ path: 'before.png', fullPage: true }) + +// Prefer role/name locators and ordinary actions; these exercise the real +// renderer path instead of calling React handlers or Electron internals. +await page.getByRole('button', { name: 'Library' }).click() +await page.getByRole('textbox', { name: /search/i }).fill('target paper') +await page.screenshot({ path: 'library-search.png', fullPage: true }) + +await page.locator('body').ariaSnapshot({ mode: 'default' }) + .then(snapshot => writeText('aria-after.yml', snapshot)) +await browser.close() +``` + +The `appendJson` and `writeText` helpers above are intentionally omitted from +the production tree; the audit harness should implement them with Node's +filesystem APIs and write only under `.audit/artifacts`. Playwright documents +`connectOverCDP()` as attaching to an existing Chromium instance, while warning +that CDP has lower fidelity than Playwright's own protocol: +[BrowserType.connectOverCDP](https://playwright.dev/docs/api/class-browsertype#browser-type-connect-over-cdp). +Its Electron API documents screenshots, renderer console forwarding, and clicks +for a Playwright-launched Electron process: +[Playwright Electron API](https://playwright.dev/docs/api/class-electron). +For an already-running process, the CDP `Page` object provides the equivalent +renderer surface. Playwright's locator API documents semantic, auto-waiting +locators, clicks, and ARIA snapshots: +[Locator](https://playwright.dev/docs/api/class-locator). + +## What the evidence proves + +- `*.png`: visible renderer state at checkpoints; Playwright's screenshot API + supports full-page and path-based capture ([screenshots](https://playwright.dev/docs/screenshots)). +- `aria-*.yml`: accessibility-tree representation exposed by the renderer, not + a macOS Accessibility Inspector dump. +- `renderer-console.jsonl`, `renderer-errors.jsonl`, and request failures: + renderer diagnostics. The CDP Runtime domain defines `consoleAPICalled` for + console API calls ([CDP Runtime](https://chromedevtools.github.io/devtools-protocol/v8/Runtime/)). +- `electron.log`, stdout, and stderr: main/Chromium process diagnostics. The + Electron logging switches above do not guarantee application-level structured + logs unless the app writes them. +- The audit journal: action, locator, timestamp, result, artifact filename, + and whether the action reached a loading/error/empty state. + +Use `page.getByRole()`/`getByLabel()` and `.click()`/`.fill()` for ordinary +renderer actions. Avoid `page.evaluate()` to invoke application functions, +direct IPC calls, or DOM mutation: those are useful diagnostics but weaken the +claim that the user journey was exercised. + +## Security and limitations + +1. CDP is a control channel. Anyone who can reach its endpoint can inspect and + control the renderer. Use a fresh profile, a random unused port, loopback + only, no shared Wi-Fi exposure, and stop Electron immediately after capture. +2. Never attach to a daily-driver browser, another Electron instance, or a + profile containing credentials. Playwright explicitly warns that connecting + to an existing browser is lower fidelity; its `connectOverCDP` endpoint is + therefore an intentional, tightly scoped audit seam, not a general browser + automation service. +3. Playwright's Electron docs state that native Electron dialogs + (`dialog.showOpenDialog`, `showMessageBox`, etc.) are not intercepted because + they execute in the main process and go to OS APIs. File-picker or native + dialog acceptance needs a separate permitted OS-control method, a product + test seam, or an explicitly unverified path. +4. The CDP endpoint controls renderer targets. To debug main-process code, use + Electron's separate `--inspect`/V8 inspector path; do not confuse its logs or + protocol with renderer evidence. +5. CDP attachment may not reproduce every Playwright-launched-browser feature, + and a screenshot/ARIA snapshot can miss transient states. Record the exact + app commit, Electron version, port, command line, harness version, and + artifact timestamps. + +## Escalation / uncertainty + +The recommendation assumes the built entrypoint accepts Chromium switches and +that Electron 32 exposes its renderer target on the selected port; verify with +`curl http://127.0.0.1:$PORT/json/version` and Playwright's page list before +calling the run live evidence. If `electron-vite` development mode is required +for a feature, use the same switch at the underlying Electron launch boundary; +the repository currently has no documented dev-script contract for forwarding +arbitrary Electron switches. Do not edit product code solely to add a debug +hook without a separate issue and security review. + +If CDP cannot attach, the honest fallback is an evidence gap—not a claim that +Accessibility permission was bypassed. Escalate for an approved OS-level +control tool or a narrowly scoped, reviewed test-only launch seam. Do not use +credentials, a personal browser profile, or third-party subscription/API-key +bridges to make the audit work. + +## Sources + +- [Electron supported command-line switches](https://www.electronjs.org/docs/latest/api/command-line-switches) +- [Electron main-process debugging](https://www.electronjs.org/docs/latest/tutorial/debugging-main-process) +- [Playwright Electron API](https://playwright.dev/docs/api/class-electron) +- [Playwright BrowserType.connectOverCDP](https://playwright.dev/docs/api/class-browsertype#browser-type-connect-over-cdp) +- [Playwright Locator API](https://playwright.dev/docs/api/class-locator) +- [Playwright screenshots](https://playwright.dev/docs/screenshots) +- [Chrome DevTools Protocol Page domain](https://chromedevtools.github.io/devtools-protocol/tot/Page/) +- [Chrome DevTools Protocol Runtime domain](https://chromedevtools.github.io/devtools-protocol/v8/Runtime/) diff --git a/docs/verification/electron-core-journey-2026-08-22.md b/docs/verification/electron-core-journey-2026-08-22.md new file mode 100644 index 0000000..6971b37 --- /dev/null +++ b/docs/verification/electron-core-journey-2026-08-22.md @@ -0,0 +1,127 @@ +# Electron core-journey audit — 2026-08-22 + +Issue: [#26](https://github.com/dkritarth/Vellum/issues/26) + +Commit under test: `a21d596e3fca6de7edef1e63576a8beb00183e5e` + +Result: **failed at boot and entry-to-ingest boundaries; downstream journey blocked** + +## Test isolation and control + +- Ran from detached worktree `/tmp/vellum-issue-26-audit.ikQaIL`; its `data/` + started absent. Normal checkout `data/` was neither read nor mutated. +- macOS Accessibility permission was unavailable. Renderer checks used a + temporary Playwright/CDP controller against the real Electron process. + Main-process output came from `ELECTRON_ENABLE_LOGGING=1` PTY capture. +- Temporary worktree-only bypasses were used to expose later failures: + preload path changed from missing `index.js` to emitted `preload.mjs`, and + `better-sqlite3` was externalized. None exists in this branch. +- Electron control limits and primary-source basis are recorded in + [`../research/issue-26-electron-control.md`](../research/issue-26-electron-control.md). + +## Real paper fixture + +Planned input: `1706.03762` / `arXiv:1706.03762v7`. + +Expected metadata from the [official arXiv record](https://arxiv.org/abs/1706.03762): + +- Title: *Attention Is All You Need* +- Authors: Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, + Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin +- First submitted: 12 June 2017; tested record version: v7, 2 August 2023 +- Length: 15 pages, 5 figures +- Key factual grounding checks planned: Transformer uses attention without + recurrence/convolution; reported WMT 2014 scores are 28.4 BLEU for + English-to-German and 41.8 BLEU for English-to-French. + +The app never exposed a working ingest path, so it did not download or persist +this paper. Metadata above records expected values; it is not claimed as Vellum +output. + +## Results matrix + +| Core step | Result | Observed evidence | +|---|---|---| +| Clean isolated start | Pass | Detached worktree began without `data/`; normal checkout stayed untouched. | +| Production/dev preload bridge | **Fail** | Build emitted `out/preload/preload.mjs`; Electron requested `out/preload/index.js`; status remained `bridge: …`. Filed [#51](https://github.com/dkritarth/Vellum/issues/51). Screenshot: [`issue-26/01-preload-failure.png`](issue-26/01-preload-failure.png). | +| Empty reader state | Pass after temporary preload bypass | Visible `No paper open` and `Open a paper from Library to start reading`; status `bridge: pong`. Screenshot: [`issue-26/02-empty-state.png`](issue-26/02-empty-state.png). | +| Create / ingest entry | **Fail** | Semantic click on **Create** produced no visible or accessible-tree change; controller recorded `didCreate: true` and `createChanged: false`. Filed [#53](https://github.com/dkritarth/Vellum/issues/53). | +| Input classification | **Fail — blocked** | No user-facing ingest input exists. Direct IPC was not substituted for the required user journey; #53. | +| Real arXiv ingest | **Fail — blocked** | #53 prevents initiation; #52 prevents first DB-backed operation. | +| Re-ingest / idempotency | **Fail — blocked** | No first ingest completed because of #52 and #53. | +| Library empty state | **Fail** | Unmodified app emptied the renderer and logged `window.vellum`/`listPapers` TypeError (#51). With preload temporarily bypassed, it showed `Could not load your library.` and main threw the native-binding error (#52). | +| Library loading state | **Fail — evidence unavailable** | Audit could not verify this state before #52. This is an audit-coverage failure, not evidence that the loading UI itself malfunctions. | +| Library search and sort | **Fail — blocked** | #52 prevents a successful Library query. | +| Open paper tab | **Fail — blocked** | #52 and #53 prevent creation of a paper record. | +| PDF render and page navigation | **Fail — blocked** | #52 and #53 prevent ingest/open. | +| Zoom, TOC, document search | **Fail — blocked** | #52 and #53 prevent Reader from receiving a paper. | +| Details and generated summary | **Fail — blocked** | #52 and #53 prevent paper record/markdown creation. | +| Model selector | **Fail — blocked** | #52 and #53 prevent opening a paper/session. | +| Codex factual question and grounding check | **Fail — blocked** | No paper markdown/session is reachable; prior #25 adapter smoke is not substituted for this journey. | +| Codex interpretive question | **Fail — blocked** | Same #52/#53 blocker. | +| New chat | **Fail — blocked** | #52/#53 prevent a paper chat from opening. | +| Chat persistence across reload | **Fail — blocked** | #52/#53 prevent chat creation. | +| App restart persistence | **Fail / blocked** | Repeated fresh launches reproduced #51. Temporary-bypass launches reproduced #52; no durable paper/chat state could be created to test round trip. | +| Failure state | Pass | Library rendered `Could not load your library.` after #52. | +| Renderer console | Pass for captured post-load window | No renderer page errors were observed after the temporary preload bypass; [`issue-26/last-renderer-console.json`](issue-26/last-renderer-console.json). This does not cover preload-time errors. | +| Main-process console | **Fail** | Exact preload and SQLite errors below. | +| Keyboard/accessibility labels | Partial pass | Empty-state accessibility tree exposed named navigation buttons, `Paper view`, right-panel tabs, `Highlight`, `Skills`, `Context`, and disabled `Ask a question`. Focus reached clicked `Library`. Full reader/chat order was blocked. | + +Machine-readable action/result transcript: [`issue-26/interaction-journal.json`](issue-26/interaction-journal.json). +Controller console transcript: [`issue-26/controller-console.txt`](issue-26/controller-console.txt). + +## Exact console failures + +Unmodified preload startup: + +```text +Unable to load preload script: .../out/preload/index.js +Error: Cannot find module '.../out/preload/index.js' +``` + +Library after temporary preload bypass: + +```text +Error occurred in handler for 'vellum:list-papers': Error: Could not dynamically require +".../out/build/Release/better_sqlite3.node". Please configure the +dynamicRequireTargets or/and ignoreDynamicRequires option of +@rollup/plugin-commonjs appropriately for this require call to work. +``` + +Generated `out/main/index.js` contained `commonjsRequire(filename)` for the +native addon and resolved it under `out/build/Release/`, where no addon exists. +Captured excerpts: [`issue-26/main-console.txt`](issue-26/main-console.txt). + +## Filed failures + +1. [#51 — Electron dev boot cannot load preload bridge](https://github.com/dkritarth/Vellum/issues/51) +2. [#52 — Bundled better-sqlite3 cannot load native binding in Electron](https://github.com/dkritarth/Vellum/issues/52) +3. [#53 — Create control cannot start paper ingest](https://github.com/dkritarth/Vellum/issues/53) + +Each issue includes severity, expected/actual behavior, exact reproduction, +suspected ownership, evidence, and explicit non-goals. All remain blocked; this +audit made no product fix. + +## Automated verification + +- `npm run typecheck`: passed. +- `npm run build`: passed. Build output itself reproduces #51 by emitting + `out/preload/preload.mjs` while unmodified main expects `index.js`. +- Focused `npx vitest run core/acp/stdio-client.test.ts`: 18/18 passed. +- Final `npm test`: 147/147 passed across 24 files. Initial full run was + 146/147 because `dispose() does not resolve until the adapter subprocess + exits` failed once; focused and final full reruns passed. Existing jsdom + canvas-not-implemented stderr remained non-failing test noise and is not PDF + rendering proof. +- `git diff --check master...HEAD`: passed. + +## Limitations + +- Native macOS dialogs were not exercised because Accessibility permission was + unavailable and no dialog was reachable. +- Loading-state screenshots, real-paper reader evidence, signed-in Codex Ask, + and persistence evidence are absent because boot-critical defects prevented + those states. These are recorded as blocked, not passed. +- Temporary package/install experiments in the detached worktree were audit + tooling only. They are not implementation recommendations or acceptance + evidence for any filed defect. diff --git a/docs/verification/issue-26/01-preload-failure.png b/docs/verification/issue-26/01-preload-failure.png new file mode 100644 index 0000000..ce19754 Binary files /dev/null and b/docs/verification/issue-26/01-preload-failure.png differ diff --git a/docs/verification/issue-26/02-empty-state.png b/docs/verification/issue-26/02-empty-state.png new file mode 100644 index 0000000..0135be8 Binary files /dev/null and b/docs/verification/issue-26/02-empty-state.png differ diff --git a/docs/verification/issue-26/controller-console.txt b/docs/verification/issue-26/controller-console.txt new file mode 100644 index 0000000..13ab69f --- /dev/null +++ b/docs/verification/issue-26/controller-console.txt @@ -0,0 +1,15 @@ +Issue #26 renderer-control output captured after the detached-worktree-only +preload bypass. This is audit-controller console evidence, not application +renderer console output. + +{ + "didCreate": true, + "createChanged": false, + "didLibrary": true, + "active": "Library" +} + +Body text immediately before and after the Create click retained: + +No paper open +Open a paper from Library to start reading. diff --git a/docs/verification/issue-26/interaction-journal.json b/docs/verification/issue-26/interaction-journal.json new file mode 100644 index 0000000..31ad95c --- /dev/null +++ b/docs/verification/issue-26/interaction-journal.json @@ -0,0 +1,46 @@ +{ + "commit": "a21d596e3fca6de7edef1e63576a8beb00183e5e", + "date": "2026-08-22", + "workspace": "/tmp/vellum-issue-26-audit.ikQaIL", + "normalDataTouched": false, + "steps": [ + { + "id": "boot-unmodified", + "action": "Launch Electron from the unmodified production/dev output contract", + "actual": "Renderer appeared, status stayed at bridge: ..., and main logged missing out/preload/index.js", + "result": "fail", + "evidence": "01-preload-failure.png", + "issue": 51 + }, + { + "id": "empty-after-preload-bypass", + "action": "Launch with detached-worktree-only preload path pointed at emitted preload.mjs", + "actual": "Empty reader appeared and status reached bridge: pong", + "result": "pass", + "evidence": "02-empty-state.png" + }, + { + "id": "create", + "action": "Click visible Create button through the real Electron renderer", + "beforeText": "No paper open\nOpen a paper from Library to start reading.", + "afterText": "No paper open\nOpen a paper from Library to start reading.", + "changed": false, + "result": "fail", + "issue": 53 + }, + { + "id": "library", + "action": "Click visible Library button after the preload-only bypass", + "actual": "Could not load your library.", + "mainError": "Could not dynamically require out/build/Release/better_sqlite3.node", + "result": "fail", + "issue": 52 + }, + { + "id": "restart", + "action": "Stop and relaunch isolated Electron process multiple times", + "actual": "Unmodified launch reproduced issue 51; preload-bypass launch reproduced issue 52; no paper or chat state could be created", + "result": "fail" + } + ] +} diff --git a/docs/verification/issue-26/last-renderer-console.json b/docs/verification/issue-26/last-renderer-console.json new file mode 100644 index 0000000..d42d7c9 --- /dev/null +++ b/docs/verification/issue-26/last-renderer-console.json @@ -0,0 +1,4 @@ +{ + "consoleEntries": [], + "pageErrors": [] +} \ No newline at end of file diff --git a/docs/verification/issue-26/main-console.txt b/docs/verification/issue-26/main-console.txt new file mode 100644 index 0000000..971b72c --- /dev/null +++ b/docs/verification/issue-26/main-console.txt @@ -0,0 +1,22 @@ +Issue #26 main-process console excerpts captured from the isolated detached worktree. +Commit under test: a21d596e3fca6de7edef1e63576a8beb00183e5e + +UNMODIFIED PRELOAD CONTRACT + +Unable to load preload script: /private/tmp/vellum-issue-26-audit.ikQaIL/out/preload/index.js +Error: Cannot find module '/private/tmp/vellum-issue-26-audit.ikQaIL/out/preload/index.js' + +LIBRARY AFTER TEMPORARY PRELOAD-ONLY BYPASS + +Error occurred in handler for 'vellum:list-papers': Error: Could not dynamically require "/private/tmp/vellum-issue-26-audit.ikQaIL/out/build/Release/better_sqlite3.node". Please configure the dynamicRequireTargets or/and ignoreDynamicRequires option of @rollup/plugin-commonjs appropriately for this require call to work. + at commonjsRequire (file:///private/tmp/vellum-issue-26-audit.ikQaIL/out/main/index.js:10035:9) + at getBinding (file:///private/tmp/vellum-issue-26-audit.ikQaIL/out/main/index.js:10063:28) + at new Database2 (file:///private/tmp/vellum-issue-26-audit.ikQaIL/out/main/index.js:9979:19) + at openDb (file:///private/tmp/vellum-issue-26-audit.ikQaIL/out/main/index.js:10159:15) + at getDb (file:///private/tmp/vellum-issue-26-audit.ikQaIL/out/main/index.js:10210:17) + +Independent verifier also observed this renderer manifestation without bypassing +the preload defect: clicking Library emptied the page and emitted +`TypeError: Cannot read properties of undefined (reading 'listPapers')` because +`window.vellum` was undefined. This is attributed to issue #51, not a separate +root defect.