Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions technical-documentation/testing/manual-e2e-checklist.md
Original file line number Diff line number Diff line change
Expand Up @@ -978,3 +978,4 @@ These checks are the human slice: an agent cannot click Apple's picker (see abov
| 2026-10-15 | `v2.1.0-rc.1`, `release/v2.1.0` @ `cd88be2`, CI artifact `openscreen-windows-x64` from build run 38735485661. NSIS sha512 matched `update-info-win-x64.json`; installed over the previous build; About reads `2.1.0-rc.1` (Electron 41.2.1, Chromium 146.0.7680.188, Node 24.14.1, `win32 x64 Β· nsis`). | Windows 11 Home 26200 (x64), Ryzen 7 5800X, RTX 4070 Ti, 1920Γ—1080 @ 100 %; Logitech StreamCam | **Partial β€” whole checklist attempted for the release candidate; 5 P2 failures, plus 1 known P2 that predates the candidate** | Real OS input through computer-use; MCP checks from an SDK client over stdio and the in-app HTTP server; audio judged on exports with a 200β†’2600 Hz sweep as the track (frequency = track time). **Failures:** - *Hide desktop icons*, a known defect from before this candidate (P2, not a regression; `desktop_icon_cover.cpp` has not changed since 2.0.0): the icons come back at the first click on another window, so a take shows them from then on. A take with no click keeps them hidden. The cover is placed above the icon host once and never placed again. - Full Camera preview (P2, preview only): the webcam picture is torn (a washed-out band over the bottom third) at 31.6 s and 47.5 s, and smeared over one frame at a trim cut; `getFrames` and the webcam file are clean. - Voiceover export (P2): over a 2Γ— speed region followed by a cut, the piece before the cut runs 0.9 s into the next one and both play at once. Also no fade-out at the take's end (P3). - AI chat rewind (P2): the rewound user message stays in the conversation, and the next turn (*Smart cuts*) executed it again, bringing the rewound annotation and zoom back. - AI chat (P2, 2 turns of 5): operations reported as "applied" never reached the document (an annotation; *Smart cuts*' "added 3 trims"). *Smart cuts* took 3.5 min with no stop control. - A text annotation keeps its box as a share of the frame (P2): after a frame-style change it wraps and is clipped, in the preview and the export render. **Passed:** - HUD: layout toggle, drag, language, camera and mic toggles, *Device settings*, *Camera quality* (4K default; 1080p persists; a hand-edited 720p shows 4K), hide and tray refocus, quit. - Source picker; record, pause, restart, cancel, stop. A webcam take on a sub-4K camera records 1920Γ—1080 at 30 fps. - *Record mode* rows, hand-over, auto-zoom On/Off. *Recordings folder*: pick, persist, reset. - Editor: Media tab, clips (reorder, *Edit clip* grips, ratio lock and *Free*, trim-bar playhead, the rail's clip picker), copy/paste of a clip carrying four modifiers (#1011), transport, end-of-timeline hold (#997). - Regions: every kind; zoom level row on a 1080p take ("Zoom goes from 1Γ— to 2.5Γ—"); 3D camera presets; orbit offering rule; focus drag; *Automatic zooms* (one undo); delete; speed playback. - Modifiers under a trim, and a trim inside Full Camera. Zoom trails with trim and junction hatching, pan trail, #1017, #1008. Clip-anchored modifiers through reorder, split and rejoin, clip delete, *Edit clip* clamp, region copy/paste. - Imported audio: label, inspector, -18 dB / 1 s defaults and reset. Ghost and `in β†’ out / length` readout on edge drags. Mute absent from the file, loop fills its span, ~10 dB duck under speech. A music bed plays through a cut, by design (#560). A voiceover cut removes its words. It still loads after a restart. - Transcript and captions: seek, cut and undo (#1012), silence trim and restore, word correction surviving *Regenerate*, *Names and terms*, styles, customize, translate and delete, anchors. Caption plate measured on a 1752Γ—1080 export: left edge 174–176 px for 175.2 expected. The packaged-build insertion gate holds. - MCP: edit, record and export are refused without *MCP edits*; edits to the open project land live. - Export dialog. GIF: cancel without a partial file; 1222 frames at 25 fps; plays once. MP4: 1466 packets = ⌈(48.866 βˆ’ 0.001) Γ— 30βŒ‰, audio in sync, true peak βˆ’2.0 dBFS. - Composition, camera layouts, camera background (Cutout/Blur/Custom, in the export), Glass Lens, theme, French UI. - Shortcuts dialog (rebind and reset), wordmark menu, About and Copy, Ctrl+S/Ctrl+O, persistence (document identical after reopen), editor bounds after restart. - Usage ping: `usage-ping.json` written once on the day despite six relaunches. CLI `help` lists `mcp`; a held `mcp` process owns no window. - Takes: H.264 High, no B-frames, bt709 tags, AAC 48 kHz, hardware encoder. **Not run:** - Every tray context-menu check, the usage-ping checkbox included: right-click on the notification area is refused to computer-use. - Preview audio by ear; the microphone with speech. - Ctrl/Shift+wheel; MP4 cancel (the export finished first); the in-app `startRecording` flow and stdio quit/relaunch (passed on build 38714540858). - Migrated projects, ONNX removal, data-testid checks. The macOS-only CLI menu fixes (c309edd, 40c01bf, 2d4ac75) have nothing to observe here. - Hardware-specific Windows rows: displays, 96 kHz, C920, Windows 10, ARM. Evidence and the full list: the run's notes. |
| 2026-10-15 | installed `v2.1.0-rc.1`, `release/v2.1.0` @ `cd88be2` β€” `.deb` (`2.1.0~rc.1`) from build run 38735485661, artifact `openscreen-linux` (AppImage sha512 matches `latest-linux.yml`), installed over 2.0.0 with apt. The run's first attempt failed on `macOS x64 DMG` ("Validate stapled DMG"); the second published the pre-release without re-running the Linux job, so the published Linux packages are the ones tested here | Ubuntu 24.04, GNOME Shell 46 **Wayland**, AMD Radeon 610M (Vulkan/VAAPI), 1920Γ—1080 @ 100%, USB2.0 HD UVC webcam | **Partial β€” 6 defects** (all Linux/Wayland; 5 of 6 fixed for rc.2, and the countdown placement in (1) is still open) | **Driver: `agent-sh/computer-use-linux` 0.7.16 MCP, not the computer-use MCP** (no Claude Desktop computer-use on Linux). Pointer input goes through the RemoteDesktop portal (compositor-level, real OS events), screenshots through the screenshot portal. Its keyboard needs a *second* portal grant that was never answered, so **no key was sent all run**; its drag is press β†’ one jump β†’ release in 70 ms, so **HTML5 drag-and-drop cannot be driven**. Screenshots lag one frame, and **every portal screenshot flashes the GNOME screen white β€” flashes taken during a take are baked into the capture** (onsets measured in the source); the washed frames in the exports below are those, not an export defect. **Defects:** (1) **the countdown's last digit "1" is recorded in frame 0 of full-screen takes** (2/2 checked), and the overlay is placed by mutter, not centred (top-left on one take, lower-middle on another); window takes are clean. (2) **Tray β†’ Open does not bring back a minimized HUD**: GNOME only shows "β€œopenscreen” is ready" (`hud-overlay-hide` β†’ `minimize()`, and xdg-shell has no unminimize); the dock icon restores it. (3) **Vertical HUD runs off the screen**: the window grows to 648Γ—1080 but keeps its top-left (~150,186), so record sits under the dock and Studio/language/hide/quit are below the screen edge until dragged. (4) **HUD cannot be dragged into the top of the screen**: x follows the pointer exactly and the drop point holds, but the bar stops at yβ‰ˆ681 (horizontal) / 593 (vertical) β€” the transparent reserve is part of the window and mutter keeps the window's top under the top bar. (5) **Editor restore button does nothing** (2 tries; stays maximized) and `editor-window.json` is written with impossible normal bounds β€” `{x:256,y:-10,1440Γ—1121}` on a 1080-tall display. (6) **Save Diagnostics bundle is nearly empty on Linux**: `recentLogs` and `mainProcessLogs` `[]` despite plenty of main-process output, `projectState: null` with a project open, `helperOutput` has only `windows`/`mac`. Minor: the first frame of a VAAPI take is soft/blocky in its lower half (I-frame 54 KB, crisp by frame 5) and it is the frame the editor opens on; window captures keep the CSD shadow margin as an opaque black border around the window; window takes write no `.cursor.json` (cursor facet absent). **Since the pass** (merged to `main` and cherry-picked to `release/v2.1.0` for rc.2, not yet re-verified on a build): (1) the digit in frame 0 β†’ #1267; (2) and (3) β†’ #1272 (on Linux the HUD is hidden instead of minimized, and the vertical bar is kept on screen); (5) β†’ #1271 (works around electron/electron#53063 and stops the saved bounds drifting); (6) β†’ #1268 (main-process logs from launch and the Linux helper's output; the open project stays out of the bundle on purpose); the soft first VAAPI frame β†’ #1269 (QP capped). The shadow border and the missing cursor of window takes are documented as limits in #1270. Still open: (4), which needs a HUD design decision, and the countdown's placement under mutter from (1). **Passed:** single instance (second launch exits 0 in 0.7 s); layout toggle both ways; language menu; minimize keeps the app alive; quit clean; tray menu (Open / Check for Updates / Update Settings β–Έ with the 3 modes / Send Anonymous Usage Count / About / Save Diagnostics / Quit), recording state = red icon + only "Stop Recording", Check for Updates back after the take; HUD device panel (mics, live level, camera + preview, `Version 2.1.0-rc.1`). **Capture:** no in-app source button; the portal picker comes before the countdown, from the HUD and from the Rec stage alike, and **every** start asks again (5Γ—); screen take 1920Γ—1080, 3008 frames / 50.42 s, 0 dropped, `vaapi`, AAC 48k stereo, cursor + session sidecars, webcam webm 1280Γ—720 30 fps (βˆ’73 ms offset); **window take 866Γ—628 = the window, not the monitor**; pause freezes the timer and the file (~66 s wall βˆ’ ~16 s paused β‰ˆ 50.4 s); restart discards and re-asks the portal; cancel discards the file, mints no project, opens no editor, and leaves no OpenScreen PipeWire screencast node (`pw-dump`); system-audio-only take carries exactly the 6.04 s test tone (silencedetect); mic-only take (`role:microphone` only) picks the tone up acoustically, +10.4 dB in the 1 kHz band; capture prefs survive a restart. **Editor:** modes, play/pause, ruler seek, playback skips a trim (0:17 β†’ 0:27.1 over ~5 s), trim edge drags, zoom levels / 3D presets / manual focus drag, speed presets, text / blur (mosaic, smooth, oval) / arrow annotations, annotation drag persists, undo/redo, wallpaper / gradient / colour / 1:1 frame, Dual-frame layout, **AI camera Cutout works** (ONNX Runtime bundled), theme light↔dark, locale fr↔en, wordmark menu (About row `2.1.0-rc.1`, About box with Electron 41.2.1 / Chromium 146 / Node 24.14.1 / `linux x64 Β· deb`, Keyboard Shortcuts, AI settings with the provider list and MCP server section), Open project dialog, reopen keeps every modifier, a 2.0.0-era project opens without schema error. **Transcription** auto-runs after stop on `whispercpp-vulkan` (50.5 s in 6.0 s); a TTS take transcribed word-perfect with silence spans and durations; word click seeks monotonically (0:04.5 β†’ 0:12.5); captions on, Bold, top position; **translation via the configured OpenAI-compatible provider** β†’ "Ceci est un test de transcription locale", back to Original. **Export MP4 1080p60** via `h264_vaapi` (hardware, dmabuf): 1752Γ—1080, 2613 frames / 43.533 s, AAC 43.550 s (Ξ” 16.7 ms = 1 frame @ 60), decodes clean, ~33 s for 43.5 s; progress predicted 2617 = (50.474667 clip βˆ’ 5.584431 trim βˆ’ 2.577 Γ— Β½ speed) Γ— 60, and 2613 = the same with the video stream's 50.4167 s end β€” the clip is sized to the audio, 3.5 frames longer than the video. Frames show background, card, webcam, cursor, the dragged arrow and the 2.2Γ— auto-focus zoom. **GIF** 20 fps Small, loop off: 778Γ—480, 871 frames / 43.55 s, decoded by ffmpeg and PIL, no NETSCAPE loop block. **Not run:** everything that needs a key (Esc, shortcuts incl. Ctrl+S/Ctrl+O, renaming, custom values, transcript word edit/insert/Backspace, chat and chat-driven editing), HTML5 drag-and-drop (media β†’ timeline, clip reorder, therefore the whole modifier-anchoring section), imported audio / voice-over, unsaved-changes prompts, Check for Updates flows, About β†’ Copy (an unfocused process cannot read the Wayland clipboard), mic + system balance, the CPU-notice path (GPU present). Observations, not filed: no H.264/H.265 choice in the export dialog and no blur intensity / block size / freehand in the inspector (checklist drift?); AI settings shows "–" for the OpenAI-compatible model although the config names one; `Ctrl+Shift+O` cannot register (GNOME 46 has no GlobalShortcuts portal). |
| 2026-10-16 | installed `v2.1.0-rc.1` (`cd88be2b1`), CI-built DMG `Openscreen-macOS-Apple-Silicon-2.1.0-rc.1.dmg` from build run 38735485661 (the macOS x64 job failed once on Apple's ticket service, error 68, and passed on rerun; the pre-release then published), Developer ID signed and notarized (`spctl`: accepted, Notarized Developer ID, M4LK7C6S84); About reports `2.1.0-rc.1 (dmg)`, Electron 41.2.1, Chromium 146.0.7680.188, Node 24.14.1, darwin arm64 | macOS 26.5 (25F71), Mac mini M1 (arm64), one 3840Γ—2160 display at 2Γ—, no camera (MDM), BlackHole 2ch as the only input | Partial, no defect | **Scoped to what the RC adds over the builds already passed** (rows 2026-10-12 and 2026-10-15), real OS mouse and keyboard through computer-use, launched with `OPENSCREEN_DIAGNOSTIC=1` and `OPENSCREEN_MAC_SOURCE_PICKER=legacy`; numbers are ffprobe/ffmpeg on the files. **Usage ping (`b5ff5a348`):** under `DO_NOT_TRACK=1` no ping (`usage-ping.json` unchanged after 10 s) and no tray item; a normal launch pings within 2 s (`day` moves to 2026-10-16, week and month unchanged); the tray's *Send Anonymous Usage Count* is checked by default, unchecking writes `enabled:false`, checking it again restores `true`. **Shader effects (Metal):** the chat agent adds a "Black & White" effect on 0:03–0:08, shown as its own pill with an inspector (Image/Text/Both, cost Light, Code, Delete region); the preview is grey at 5 s and in colour at 9.7 s; the 30 fps export measures SATAVG 0 inside the span and about 21.5 outside. **MCP:** with the project open in the editor, a stdio `addTrim` shows on the timeline at once and Cmd+Z removes it (#1174); `createProject` from the speech fixture, `transcribe` (fr, 122 words), `setCaptionSettings` bold, `exportVideo` 720p: the captions are burnt in ("VoilΓ , alors je fais un petit enregistrement," at 3 s). **Recording:** HUD take of the screen, stop, editor opens; window take of System Settings through Record mode with system audio: 1446Γ—1920 H.264 + AAC 48 kHz stereo, a 1 kHz tone on the speakers is in the take at βˆ’34.6 dB band level (`aa17b10a9`). **Export:** GIF with Loop off carries no NETSCAPE block (`173f615d0`). The CLI menu, #1214 and #1233 were passed the day before on builds with the same CLI code. **Skipped:** the in-app HTTP MCP server's new record and export paths (no HTTP MCP client here); Apple's picker slice; camera checks (MDM); the rest of the file (rows above, including the 2026-10-14 full pass on `main`). **Checklist gap:** no section covers shader effects or the usage ping yet. |
| 2026-10-16 | `v2.1.0-rc.2`, `release/v2.1.0` @ `c5bc8be` (main + bump), CI artifact `openscreen-windows-x64` from build run 38868631147. NSIS sha512 matched `update-info-win-x64.json`; installed silently over rc.1. | Windows 11 Home 26200 (x64), Ryzen 7 5800X, RTX 4070 Ti, 1920Γ—1080 @ 100 %; Logitech StreamCam | **Partial β€” targeted pass on the rc.1 fixes; all 7 checked items pass, preview smoothness measured with one known gap** | Real OS input through computer-use; MCP edits and takes from an SDK client over stdio. Each item was rerun on the scenario that failed on rc.1 (#1258). **Passed:** - *Hide desktop icons* with `wgc-capture.exe` granted to computer-use (#1263): a 77 s full-screen take with clicks and typing in Notepad shows no icon in any frame checked (5, 40, 58, 66, 72, 76 s). - Chat rewind (#1259): the rewound message leaves the conversation and `chat.json`. Its text goes back to the composer. The next turn ("add a zoom from 30 to 32 s") brings back nothing from the rewound request. - Chat operations (#1261): a two-operation turn (annotation + 2Γ— speed) and *Smart cuts* on a 37 s TTS-narrated take. Every "applied" line is in the document: 4 trims announced, 4 trims present, none over a transcribed word. The *Smart cuts* turn took about 1 min. - Text annotation refit (#1262): "Refit check" stays on one line, unclipped, with the Screen frame style and with 9:16, in the preview and in the export render (`getFrames`). - Full Camera at a cut (#1266): three plays across a 1 s trim inside a Full Camera segment, 13 frames sampled, no trail or ghost. The export render at 27.00 and 27.02 s, just after the cut, is clean. Parked inside the trim's hole, the preview shows the webcam in its corner: the region is split at the trim, as #1266 describes. - `getFrames` rendered from the document (#1253): no `window is not defined` and no click-telemetry error in the main-process logs. - *Save Diagnostics* from the tray, clicked by Etienne (right-click is refused to computer-use): the bundle carries `2.1.0-rc.2`, OS and Electron versions, the Windows helper output and 54 main-process log lines. `projectState` is null and the open project is absent. **Preview smoothness (#1189), measured with a reservation:** the perf bench was running its own OpenScreen instance at the same time, with the screen probably at 60 Hz instead of 280 Hz; treat these figures as indicative. DXGI capture of the preview area at 60 fps during playback, counting runs of identical frames. Reference: an ffplay 30 fps pattern, captured the same way, shows one 67 ms gap in 10 s. - Normal take (60 fps source, webcam, two zooms one in 3D, a shader effect): no gap above 83 ms. 22 gaps of 67–83 ms in 14 s, grouped around the zoom transitions (6.0 s and 11.5 s). - Held-frame take (webcam, shader effect, orbit zoom, Full Camera): during Full Camera, recurring 67–100 ms gaps. This matches the known webcam + held-frame defect slated for rc.3. Long freezes elsewhere fall on static content and cannot be judged this way. - Cursor recoil: not judged. **Also seen:** - `startRecording` with `camera: true` over MCP picked the system default camera, a virtual "VCam Camera", and the take came out with no webcam. - The torn webcam frames of #1284 are in the recorded webcam file itself (plain ffmpeg decode), not in the preview. The preview and `getFrames` pick different frames at the same second, which is why the export looked clean on rc.1. **Not run:** the EXDEV fallback (#1256, Store MSIX only); the doubled voiceover (#1195, not fixed); the rest of the checklist (rc.1 row). |
Loading