Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions technical-documentation/testing/manual-e2e-checklist.md
Original file line number Diff line number Diff line change
Expand Up @@ -976,3 +976,4 @@ These checks are the human slice: an agent cannot click Apple's picker (see abov
| 2026-10-15 | installed `v2.1.0` candidate, `release/v2.1.0` @ `912b44800`, CI-built DMG `Openscreen-macOS-Apple-Silicon-2.0.0.dmg` from build run 38370716184 (`openscreen-mac-arm64`), Developer ID signed and notarized (`spctl`: accepted, Notarized Developer ID, M4LK7C6S84), installed over `/Applications/Openscreen.app`. `release/v2.1.0` has since moved to `a6cd941f3` with the shader-effects integration, so this build is not its current head | macOS 26.5 (25F71), Mac mini M1 (arm64), one 3840×2160 display at 2×, no camera (MDM), BlackHole 2ch as the only input | Pass, scoped to the fixes listed; no new defect | **Targeted at the fixes that reached the release after `bd04d3a7`**, real OS mouse and keyboard through computer-use, launched with `OPENSCREEN_DIAGNOSTIC=1` and `OPENSCREEN_MAC_SOURCE_PICKER=legacy`; numbers are ffprobe/ffmpeg on the files. **Passed:** #1136, a HUD dropped at the top (bar y≈60) moves down while the language menu is open and returns to y≈60 when it closes; #1135, typing in Open project until one result is left highlights it and Enter opens it; #1138, with the project open in the editor stdio `listProjects` reports `open:true` and an `addTrim` after `createCheckpoint` is refused ("this project is open in the OpenScreen editor ..."); #1133/#1144, an identical 1.80x zoom dragged into another stops flush (16947 ms against 16947 ms, length 1727 ms kept), and changing the level of one half changes only that one and splits the pill (#1017); #1134, the first message of a new conversation shows 11:46:52 AM, not a raw ISO time; #1137/#1145, the running `Openscreen mcp` holds its own `cli-sessions/<pid>-.../Local Storage` lock while only the GUI holds the GUI's, and a GUI switched to German then relaunched with that server still running comes back in German with no language prompt; #1140/#1150, `startRecording` on a missing window answers that minimized, hidden, other-Space and behind-full-screen windows are not listed; #1141, the shipped `stopRecording` description reads "Not one per click ..."; `2c978f3c`, the 7-trim speech project exports 3651 frames at 60 fps against an exact 3652.7 for its kept 60.879 s (a per-span ceil would give 3657), audio and video within 16 ms. **Known, already fixed:** while `Openscreen mcp` ran, opening OpenScreen from the Dock or Finder (`open -a`) activated the headless MCP process (bare "Quit openscreen" menu, no HUD); that is #1214 and #1233, fixed by `394aebef9`, `a999816b3` and `4f4bb997c`, which landed after `912b44800` and are in `release/v2.1.0` @ `a6cd941f3`, not in this build. **Observations:** the agent's `setZoom` stretching one of two touching identical zooms into the other merges them (the twin is removed, one undo restores it), which reads as a merge of one pill rather than a defect; the `startRecording` refusal carries a JavaScript stack trace after its message; the export above is 1.7 frames (29 ms) short of the exact kept duration. **Skipped:** `13366eba` and `85aeaf08` (Windows WGC helper only); the rest of the file (rows above and the 2026-10-14 macOS row on `main`). |
| 2026-10-15 | `release/v2.1.0` @ `a6cd941f3` (advanced to `main`: shader effects integration #1210, MCP unification and editor parity #1175 to #1186, fixes #1192, #1194, #1197, #1205, #1222), CI artifact `openscreen-windows-x64` from build run 38714540858. NSIS sha512 matched `update-info-win-x64.json`; installed silently per user over the previous build; About still `2.0.0` (intermediate build). | Windows 11 Home 26200 (x64), Ryzen 7 5800X, RTX 4070 Ti, 1920×1080 @ 100 %; Logitech StreamCam | **Pass — new features first (shader effects, MCP), plus a short regression slice; 1 minor** | Real OS input through computer-use. MCP calls come from an SDK client over stdio (`Openscreen.exe mcp`, forwarded to the running app) and over the in-app HTTP server, whose token was copied from *AI settings*. **Tools:** 55, including `addEffect`, `setEffect`, `setEffectDefinition` and `getEffect`. **Recording:** a stdio `startRecording` on a Notepad window, with `camera: "StreamCam"`, system audio and real clicks, gives an 890×596 60 fps take of 35.8 s, with a webcam file, system audio at −3.6 dB and 1 zoom. **Shader effects over MCP:** - a WGSL grayscale with an `amount` parameter compiles, at cost tier `light` (1 frame read); - an effect overlapping another is refused ("one effect runs at a time"); - code that does not compile, in `addEffect` and in `setEffectDefinition`, is refused with naga's error at the column; - `setEffect` values apply; - a second region can reuse the effect by `effectId`; - with no export yet, `getFrames` renders from the document (#1230): grey inside the region, webcam included, colour outside; - a `ctx.clicks` tint is red 0.96 s after a click and absent 2.96 s after, as documented (the last 2 s only), both in `getFrames` and in an MP4 export, where the frame centre at 28 s reads RGB 135/5/19. **Shader effects in the editor:** the Effects lane shows each region. An effect added over MCP to the open project lands live and comes off with one Ctrl+Z. The effect inspector shows the parameter slider, the Image / Text / Both target, the cost and the code; moving its slider to the middle desaturates the preview by half, and `getEffect` reads it back as `amount: 0.48`. **Captions:** `transcribe` finds 25 English words; `putCaptionTranslation` writes 4 French units; `setCaptionSettings` (`bold`, `fr`) shows them live in the editor preview. **In-app server:** with *MCP edits* turned on, `startRecording` answers `recording: true` in 4 s through the recording bar, with its countdown and only the app's own `wgc-capture.exe` running (#1205). `stopRecording` returns the 24.1 s take with webcam and system audio, open in the editor under the requested title. **Regression:** the Export dialog's MP4 at 1080p and 30 fps, saved through the native save dialog, gives 1506×1080 at 30 fps, 722 frames for 24.07 s. The MCP export of the same project carries the click sound at both clicks. **Minor:** each `getFrames` rendered from the document logs `Failed to read the click telemetry of <take> ReferenceError: window is not defined` (from `src/lib/ai-edition/clickSound.ts`, run in the main process). The frames are right and the export still has its click sounds, so only the log is affected. Not run: the microphone (no working input on this machine); HUD-started takes, tray, settings, i18n and persistence sections of the checklist beyond the slice above; macOS and Linux. |
| 2026-10-15 | `v2.1.0-rc.1`, `release/v2.1.0` @ `cd88be2`, CI artifact `openscreen-windows-x64` from build run 38735485661. NSIS sha512 matched `update-info-win-x64.json`; installed over the previous build; About reads `2.1.0-rc.1` (Electron 41.2.1, Chromium 146.0.7680.188, Node 24.14.1, `win32 x64 · nsis`). | Windows 11 Home 26200 (x64), Ryzen 7 5800X, RTX 4070 Ti, 1920×1080 @ 100 %; Logitech StreamCam | **Partial — whole checklist attempted for the release candidate; 5 P2 failures, plus 1 known P2 that predates the candidate** | Real OS input through computer-use; MCP checks from an SDK client over stdio and the in-app HTTP server; audio judged on exports with a 200→2600 Hz sweep as the track (frequency = track time). **Failures:** - *Hide desktop icons*, a known defect from before this candidate (P2, not a regression; `desktop_icon_cover.cpp` has not changed since 2.0.0): the icons come back at the first click on another window, so a take shows them from then on. A take with no click keeps them hidden. The cover is placed above the icon host once and never placed again. - Full Camera preview (P2, preview only): the webcam picture is torn (a washed-out band over the bottom third) at 31.6 s and 47.5 s, and smeared over one frame at a trim cut; `getFrames` and the webcam file are clean. - Voiceover export (P2): over a 2× speed region followed by a cut, the piece before the cut runs 0.9 s into the next one and both play at once. Also no fade-out at the take's end (P3). - AI chat rewind (P2): the rewound user message stays in the conversation, and the next turn (*Smart cuts*) executed it again, bringing the rewound annotation and zoom back. - AI chat (P2, 2 turns of 5): operations reported as "applied" never reached the document (an annotation; *Smart cuts*' "added 3 trims"). *Smart cuts* took 3.5 min with no stop control. - A text annotation keeps its box as a share of the frame (P2): after a frame-style change it wraps and is clipped, in the preview and the export render. **Passed:** - HUD: layout toggle, drag, language, camera and mic toggles, *Device settings*, *Camera quality* (4K default; 1080p persists; a hand-edited 720p shows 4K), hide and tray refocus, quit. - Source picker; record, pause, restart, cancel, stop. A webcam take on a sub-4K camera records 1920×1080 at 30 fps. - *Record mode* rows, hand-over, auto-zoom On/Off. *Recordings folder*: pick, persist, reset. - Editor: Media tab, clips (reorder, *Edit clip* grips, ratio lock and *Free*, trim-bar playhead, the rail's clip picker), copy/paste of a clip carrying four modifiers (#1011), transport, end-of-timeline hold (#997). - Regions: every kind; zoom level row on a 1080p take ("Zoom goes from 1× to 2.5×"); 3D camera presets; orbit offering rule; focus drag; *Automatic zooms* (one undo); delete; speed playback. - Modifiers under a trim, and a trim inside Full Camera. Zoom trails with trim and junction hatching, pan trail, #1017, #1008. Clip-anchored modifiers through reorder, split and rejoin, clip delete, *Edit clip* clamp, region copy/paste. - Imported audio: label, inspector, -18 dB / 1 s defaults and reset. Ghost and `in → out / length` readout on edge drags. Mute absent from the file, loop fills its span, ~10 dB duck under speech. A music bed plays through a cut, by design (#560). A voiceover cut removes its words. It still loads after a restart. - Transcript and captions: seek, cut and undo (#1012), silence trim and restore, word correction surviving *Regenerate*, *Names and terms*, styles, customize, translate and delete, anchors. Caption plate measured on a 1752×1080 export: left edge 174–176 px for 175.2 expected. The packaged-build insertion gate holds. - MCP: edit, record and export are refused without *MCP edits*; edits to the open project land live. - Export dialog. GIF: cancel without a partial file; 1222 frames at 25 fps; plays once. MP4: 1466 packets = ⌈(48.866 − 0.001) × 30⌉, audio in sync, true peak −2.0 dBFS. - Composition, camera layouts, camera background (Cutout/Blur/Custom, in the export), Glass Lens, theme, French UI. - Shortcuts dialog (rebind and reset), wordmark menu, About and Copy, Ctrl+S/Ctrl+O, persistence (document identical after reopen), editor bounds after restart. - Usage ping: `usage-ping.json` written once on the day despite six relaunches. CLI `help` lists `mcp`; a held `mcp` process owns no window. - Takes: H.264 High, no B-frames, bt709 tags, AAC 48 kHz, hardware encoder. **Not run:** - Every tray context-menu check, the usage-ping checkbox included: right-click on the notification area is refused to computer-use. - Preview audio by ear; the microphone with speech. - Ctrl/Shift+wheel; MP4 cancel (the export finished first); the in-app `startRecording` flow and stdio quit/relaunch (passed on build 38714540858). - Migrated projects, ONNX removal, data-testid checks. The macOS-only CLI menu fixes (c309edd, 40c01bf, 2d4ac75) have nothing to observe here. - Hardware-specific Windows rows: displays, 96 kHz, C920, Windows 10, ARM. Evidence and the full list: the run's notes. |
| 2026-10-15 | installed `v2.1.0-rc.1`, `release/v2.1.0` @ `cd88be2` — `.deb` (`2.1.0~rc.1`) from build run 38735485661, artifact `openscreen-linux` (AppImage sha512 matches `latest-linux.yml`), installed over 2.0.0 with apt. The run's first attempt failed on `macOS x64 DMG` ("Validate stapled DMG"); the second published the pre-release without re-running the Linux job, so the published Linux packages are the ones tested here | Ubuntu 24.04, GNOME Shell 46 **Wayland**, AMD Radeon 610M (Vulkan/VAAPI), 1920×1080 @ 100%, USB2.0 HD UVC webcam | **Partial — 6 defects** (all Linux/Wayland; 5 of 6 fixed for rc.2, and the countdown placement in (1) is still open) | **Driver: `agent-sh/computer-use-linux` 0.7.16 MCP, not the computer-use MCP** (no Claude Desktop computer-use on Linux). Pointer input goes through the RemoteDesktop portal (compositor-level, real OS events), screenshots through the screenshot portal. Its keyboard needs a *second* portal grant that was never answered, so **no key was sent all run**; its drag is press → one jump → release in 70 ms, so **HTML5 drag-and-drop cannot be driven**. Screenshots lag one frame, and **every portal screenshot flashes the GNOME screen white — flashes taken during a take are baked into the capture** (onsets measured in the source); the washed frames in the exports below are those, not an export defect. **Defects:** (1) **the countdown's last digit "1" is recorded in frame 0 of full-screen takes** (2/2 checked), and the overlay is placed by mutter, not centred (top-left on one take, lower-middle on another); window takes are clean. (2) **Tray → Open does not bring back a minimized HUD**: GNOME only shows "“openscreen” is ready" (`hud-overlay-hide` → `minimize()`, and xdg-shell has no unminimize); the dock icon restores it. (3) **Vertical HUD runs off the screen**: the window grows to 648×1080 but keeps its top-left (~150,186), so record sits under the dock and Studio/language/hide/quit are below the screen edge until dragged. (4) **HUD cannot be dragged into the top of the screen**: x follows the pointer exactly and the drop point holds, but the bar stops at y≈681 (horizontal) / 593 (vertical) — the transparent reserve is part of the window and mutter keeps the window's top under the top bar. (5) **Editor restore button does nothing** (2 tries; stays maximized) and `editor-window.json` is written with impossible normal bounds — `{x:256,y:-10,1440×1121}` on a 1080-tall display. (6) **Save Diagnostics bundle is nearly empty on Linux**: `recentLogs` and `mainProcessLogs` `[]` despite plenty of main-process output, `projectState: null` with a project open, `helperOutput` has only `windows`/`mac`. Minor: the first frame of a VAAPI take is soft/blocky in its lower half (I-frame 54 KB, crisp by frame 5) and it is the frame the editor opens on; window captures keep the CSD shadow margin as an opaque black border around the window; window takes write no `.cursor.json` (cursor facet absent). **Since the pass** (merged to `main` and cherry-picked to `release/v2.1.0` for rc.2, not yet re-verified on a build): (1) the digit in frame 0 → #1267; (2) and (3) → #1272 (on Linux the HUD is hidden instead of minimized, and the vertical bar is kept on screen); (5) → #1271 (works around electron/electron#53063 and stops the saved bounds drifting); (6) → #1268 (main-process logs from launch and the Linux helper's output; the open project stays out of the bundle on purpose); the soft first VAAPI frame → #1269 (QP capped). The shadow border and the missing cursor of window takes are documented as limits in #1270. Still open: (4), which needs a HUD design decision, and the countdown's placement under mutter from (1). **Passed:** single instance (second launch exits 0 in 0.7 s); layout toggle both ways; language menu; minimize keeps the app alive; quit clean; tray menu (Open / Check for Updates / Update Settings ▸ with the 3 modes / Send Anonymous Usage Count / About / Save Diagnostics / Quit), recording state = red icon + only "Stop Recording", Check for Updates back after the take; HUD device panel (mics, live level, camera + preview, `Version 2.1.0-rc.1`). **Capture:** no in-app source button; the portal picker comes before the countdown, from the HUD and from the Rec stage alike, and **every** start asks again (5×); screen take 1920×1080, 3008 frames / 50.42 s, 0 dropped, `vaapi`, AAC 48k stereo, cursor + session sidecars, webcam webm 1280×720 30 fps (−73 ms offset); **window take 866×628 = the window, not the monitor**; pause freezes the timer and the file (~66 s wall − ~16 s paused ≈ 50.4 s); restart discards and re-asks the portal; cancel discards the file, mints no project, opens no editor, and leaves no OpenScreen PipeWire screencast node (`pw-dump`); system-audio-only take carries exactly the 6.04 s test tone (silencedetect); mic-only take (`role:microphone` only) picks the tone up acoustically, +10.4 dB in the 1 kHz band; capture prefs survive a restart. **Editor:** modes, play/pause, ruler seek, playback skips a trim (0:17 → 0:27.1 over ~5 s), trim edge drags, zoom levels / 3D presets / manual focus drag, speed presets, text / blur (mosaic, smooth, oval) / arrow annotations, annotation drag persists, undo/redo, wallpaper / gradient / colour / 1:1 frame, Dual-frame layout, **AI camera Cutout works** (ONNX Runtime bundled), theme light↔dark, locale fr↔en, wordmark menu (About row `2.1.0-rc.1`, About box with Electron 41.2.1 / Chromium 146 / Node 24.14.1 / `linux x64 · deb`, Keyboard Shortcuts, AI settings with the provider list and MCP server section), Open project dialog, reopen keeps every modifier, a 2.0.0-era project opens without schema error. **Transcription** auto-runs after stop on `whispercpp-vulkan` (50.5 s in 6.0 s); a TTS take transcribed word-perfect with silence spans and durations; word click seeks monotonically (0:04.5 → 0:12.5); captions on, Bold, top position; **translation via the configured OpenAI-compatible provider** → "Ceci est un test de transcription locale", back to Original. **Export MP4 1080p60** via `h264_vaapi` (hardware, dmabuf): 1752×1080, 2613 frames / 43.533 s, AAC 43.550 s (Δ 16.7 ms = 1 frame @ 60), decodes clean, ~33 s for 43.5 s; progress predicted 2617 = (50.474667 clip − 5.584431 trim − 2.577 × ½ speed) × 60, and 2613 = the same with the video stream's 50.4167 s end — the clip is sized to the audio, 3.5 frames longer than the video. Frames show background, card, webcam, cursor, the dragged arrow and the 2.2× auto-focus zoom. **GIF** 20 fps Small, loop off: 778×480, 871 frames / 43.55 s, decoded by ffmpeg and PIL, no NETSCAPE loop block. **Not run:** everything that needs a key (Esc, shortcuts incl. Ctrl+S/Ctrl+O, renaming, custom values, transcript word edit/insert/Backspace, chat and chat-driven editing), HTML5 drag-and-drop (media → timeline, clip reorder, therefore the whole modifier-anchoring section), imported audio / voice-over, unsaved-changes prompts, Check for Updates flows, About → Copy (an unfocused process cannot read the Wayland clipboard), mic + system balance, the CPU-notice path (GPU present). Observations, not filed: no H.264/H.265 choice in the export dialog and no blur intensity / block size / freehand in the inspector (checklist drift?); AI settings shows "–" for the OpenAI-compatible model although the config names one; `Ctrl+Shift+O` cannot register (GNOME 46 has no GlobalShortcuts portal). |
Loading