From 40b3e08e9ca56edd5676e275af36a33943a15e1e Mon Sep 17 00:00:00 2001 From: Etienne Lescot Date: Sat, 3 Oct 2026 09:52:54 +0200 Subject: [PATCH] docs(testing): log the v2.0.0-rc.13 computer-use e2e pass --- technical-documentation/testing/manual-e2e-checklist.md | 1 + 1 file changed, 1 insertion(+) diff --git a/technical-documentation/testing/manual-e2e-checklist.md b/technical-documentation/testing/manual-e2e-checklist.md index a95488104..5cae7e374 100644 --- a/technical-documentation/testing/manual-e2e-checklist.md +++ b/technical-documentation/testing/manual-e2e-checklist.md @@ -845,3 +845,4 @@ On macOS 15.2+ it opens at launch, until it has been closed once, while one of i | 2026-10-01 | installed `v2.0.0-rc.6` (CI artifact, sha512 matched `latest.yml`), installed over rc.1 | Windows 11 26200, 1920×1080 @ 125 %, AMD Radeon iGPU | Partial — 1 defect (transcript) | Targeted at the Windows changes since rc.1. **#951:** two vertical↔horizontal round trips with real clicks, state read after each, HUD back at the same bounds (`302,149`). **Recording** (full screen + mic + system audio + webcam, instant pause/resume mid-take): screen and webcam H.264 **High**, BT.709 primaries/transfer/matrix, TV range; audio runs 28 ms past video (rc.1: 0.87 s); stop→editor ~1 s. Screen at 57.2 fps average — 181 missed 60 Hz ticks over 52 s, timestamps stay on the clock (video 52.17 s vs audio 52.20 s) — known, #945. **STT:** helper boots on Vulkan, VAD on CPU, no crash; on a ground-truth clip (12.12 s silence then speech), first word at 12.10 s and every sentence boundary within 0.12 s; the "en" CTC aligner downloads in the background and only applies from the next transcription. **Defect:** the last word before one pause ("happened.") lands after it as a 20 ms word, its audio given to the previous word; without the aligner the display order also inverts ("nothing The happened."). Reported to #948. **Export:** 3-clip MP4 720p30, progress reaches 100 % (4200/4203), 4200 frames, video 139.97 s / audio 140.00 s, each clip's speech at the expected offset, decodes clean. **Mixer (#911, instant resume):** a second take with system audio only (mic off) while a 26 s TTS WAV played at 76 % volume (output peak meter 0.985 during the take). No punched hole anywhere in the voice: an alignment-free scan for 2–12 ms dips below 5 % between loud neighbours found none, and the same scan finds 10 ms holes injected into the take. Instant pause/resume (0.32 s button to button) skips 0.330 s of the source, so no voice before the pause is lost (the bug was 80–90 ms). Take is +3 dB over the source, peak −0.1 dBFS, 0 full-scale samples. The first take of this pass had the speakers muted at 0 %, which silences WASAPI loopback, so its audio was not usable. **Not covered:** preview limiter; MCP server; keep-awake; tray; GIF; macOS/Linux. | | 2026-10-01 | installed `v2.0.0-rc.8` (build run 36883700385, tag `33a16fe1`, helper from build-whisper-stt run 36830983393; NSIS sha512 matched `latest.yml`), installed over rc.5 for all users, About reports `2.0.0-rc.8` (`win32 x64 · nsis`, Electron 41.2.1) | Windows 11 Home 26200, 1920×1080 @ 125 %, Logitech StreamCam (sub-4K, MF) | **Partial — 2 defects** | **Webcam:** torn and mis-exposed frames in the webcam file (some frames split horizontally into two pictures, others blue-dark or washed out). The maintainer attributes them to the camera itself, not OpenScreen; the same frames are in the rc.1 take of 2026-09-30. **Defect 1 (#960): captions in the preview are stale after switching projects.** Two clips of one asset, captions Bold top-left: after opening another project and reopening this one, clip 1 shows no caption or the wrong line (`0:06` showed "we are recording the screen…", `0:12` and `1:05` nothing) until *Show captions* is toggled; saved data is intact. **Defect 2 (#961): after a working session, closing the editor leaves OpenScreen running with no window** (6 processes, tray only), seen twice: once after takes (HUD, Record mode, restart, cancel), once after only the project dialog, *Edit clip* and a chat turn. An instance that only opened the editor and closed it quit as `window-all-closed` intends, so some hidden window outlives those surfaces. **Loudness in spec:** MP4 export at the Audio facet's default output level (never changed in this project), −15.8 LUFS integrated, −1.4 dBFS sample and −1.4 dBTP true peak; the check asked for a sample peak under −1.5 dBFS, which the AAC overshoot past the −1.5 dBFS limiter cannot meet, so its wording is corrected in this change. **Passed:** HUD layout toggle both ways, tooltips idle and recording (with continuous real moves; a teleported cursor shows the previous control's tooltip, a driver artefact), language menu, *Device settings* (devices, live preview, *Camera quality* 1080p/1440p/4K, 4K default, 1080p persisted across quit/relaunch as `camQuality`, hand-edited `720p` reads back as 4K, device switch persisted), *Hide recording bar* → tray click brings the HUD back, *Quit OpenScreen* (0 processes), second launch exits 0 with one HUD. Source selector (Screens/Windows, Cancel), record with no source opens it and starts after the pick, countdown, pause (span absent from the file) / resume, restart (new helper process, timer reset), cancel (no editor, no file), gear locked during a take. Stop → editor, 53.0 s take with 3 automatic zooms at the clicks, desktop icons hidden in the second take for its whole length. Files: screen and webcam `profile=High`, `has_b_frames=0`, `tv/bt709×3`, AAC 48 kHz, `encoder-selection` hardware. Auto-transcription after the take; *Regenerate as* English. Transcript: clip order, silence spans with durations, monotonic word seeks (4.0 → 11.2 → 16.4 → 21.3 → 21.8 → 27.2 → 56.9 s), live word highlight, Backspace marks skipped + restore, silence-span trim and restore, **#959 checked: sentence-final "time." cut 11.91–12.51 against a speech offset of 12.44 (silencedetect −40 dB) and next speech at 16.46**; typing between words does nothing (packaged gate). Transport, Left/Right 1/60 s, stop at end, navigator narrow/pan. Regions by T/S/A/C, trim edges, *Bring this part back*, undo/redo; zoom levels, custom 1.25 kept, 9 → "Zoom goes from 1× to 2.86×", 3D Orbit/left/right; speed presets, custom 3×, 20 → "Speed can't go higher than 16×"; annotation text/size/plate; Full Camera. Clip reorder carries every modifier; Edit clip grip (0:10.7–0:53.0) and 1:1 crop, Cancel leaves it. Export dialog v2 layout and sizes (1224×720 / 1838×1080 / 1910×1122); MP4 1838×1080 30 fps, video 92.033 s vs audio 92.066 s (1 frame), 2762 frames vs 2760.7 computed (105.941 − 10.312 trim − 5.41 × 2/3 for 3×), frames match the preview at 16.5/28/40 s; GIF cancel (no partial file) then full GIF 816×480, 1383 frames at 15 fps, loop, ffmpeg and GDI+ agree. Imported audio: −18 dB, 1 s fades, ramps at the track's own edges in the export (3.0→4.0 s, 22.0→23.0 s). Captions toggle, Bold, top/left anchors and *Distance from left*, captions unscaled under a zoom. Camera layout presets with Auto reshaping, shape, mirror, Cutout. Cursor facet style and *Show cursor*. Wordmark menu, About (version, Copy to clipboard), light/dark theme, French and back, shortcut rebind (Q adds a zoom) and reset, Ctrl+S toast, Ctrl+O, an older project opens without prompt, reopen keeps regions/settings/audio track. Editor bounds and maximized state restored. Chat (deepseek-v4-flash): "Add a 2x zoom from 0:07 to 0:09" applied as 2.20× with a reason, one undo/redo unit. **Not run:** HUD drag (the HUD window now ends ~45 px above the bar, any longer move lands on the click-tier desktop; short in-window moves did not move it, inconclusive); `Esc` (Claude Desktop swallows it); Ctrl/Shift+wheel (the scroll tool holds no modifier); tray right-click menu (Stop Recording, Save Diagnostics, Update Settings, About from tray) and the Alt-held Help menu; level meter and mic-only/system-only takes, music ducking (nobody spoke); "No camera found"; 4K-camera, no-30-fps, DirectShow, covered-lens and sharpness checks (no such camera / setup); window source and 24H2 context menu; DPI and multi-display (single display, scaling not changed); voice-over recording; region copy/paste and merge/repel, clip duplicate/delete; translation; most AI-settings and chat-history checks; camera background export of Cutout/Custom; caption plate measurement; legacy-project migration checks beyond opening one; macOS and Linux. Also seen: an English text read by the French SAPI voice auto-transcribed as French (detection, not app). | | 2026-10-03 | `v2.0.0-rc.13` CI artifact (build run 37072118778, tag `018d22ab`, same `crates/` `electron/` `src/` `scripts/` as `main` at `0624c148`) against `v2.0.0-rc.12` and `v1.11.0-rc.1`, each run from its installer's extracted app tree; editor checks on a dev build of `main` with rc.13's whole `bin/win32-x64` | Windows 11 Home 26200, Ryzen 5 7520U, AMD Radeon iGPU (D3D11, `h264_amf`) | Partial — no defect | **Export-path slice for #982/#983/#984.** `cargo test -p openscreen-compositor --lib --tests` on the real GPU: 456 passed, including the D3D11 render tests for the device frame, cursor model, click impact and cursor tap. Benchmark A/B in one session: rc.12 4.78× its floor, rc.13 1.71×, 1.11.0-rc.1 1.72× (numbers in `rendering-performance.md`, "The same fix on Windows"). rc.13's scoring exports are byte-identical to rc.12's. Twelve-second variants exported by both builds: the 3D cursor with click impact, the laptop frame **with its shadow**, the window frame, and a colour with Aurora are byte-identical; the two blurred-background variants differ invisibly (PSNR ≥ 67 dB). Frames extracted with ffmpeg show the device frame, 3D cursor, window chrome and rounded corners. Preview while the background is cached: image/colour/gradient switches, blur 100→0→50 and a window resize are all followed, Aurora keeps moving, None stays still, and playback frames show no leftover trail rectangle. **Not run:** real capture and HUD (computer-use screenshots came back uniformly grey, so the editor was driven over CDP, with injected input rather than OS input), Radeon GPU Analyzer register counts, Intel iGPU. | +| 2026-10-03 | installed `v2.0.0-rc.13` (NSIS from the release, sha512 matched `latest.yml`), installed over rc.12 for all users, About reports `2.0.0-rc.13` | Windows 11 Home 26200, 1920×1080 @ 125 %, AMD Radeon iGPU, built-in USB2.0 HD UVC WebCam | Partial — 1 defect | **Scoped to the app changes between rc.8, the last computer-use pass, and rc.13**; the export path itself is covered by the rc.13 row above. Real OS mouse and keyboard input throughout; computer-use screenshots came back uniformly grey again, so the app was observed through `PrintWindow` on its own windows and DOM reads over CDP. **Defect: the webcam shows its first frame when a clip is entered past the webcam's last frame from another clip.** Two clips of one take (screen video 50.98 s, webcam 51.24 s, audio and clip 51.31 s): scrubbing from clip 2 to 51.2 s in clip 1 shows the screen's last frame, as 227b9b84 intends, but the webcam's frame 0 (2 of 2; ground truth from ffmpeg: an empty chair from 18.7 s to the end). Scrubbing to the same point from inside clip 1, or to 51.0 s from clip 2, shows the right webcam frame. Cause: `open_and_seek_clip` and `seek_pair` in `crates/compositor/src/live.rs` seek the webcam with `seek_to` and fall back to `seek_to(0.0)`, where `present_frame` uses `seek_to_or_last`; unchanged on `main`. Paused scrub only, in a ~70 ms window at the end of each clip. **Passed:** #961: one take with the countdown overlay, Record mode, editor work and an export, then closing the editor left 0 processes. #966: after rebinding *Add zoom* to Q and *Add audio* to J, the empty-lane hints read "Appuyez sur Q pour ajouter un zoom" and "Appuyez sur J pour ajouter un audio, V pour enregistrer une voix off", and Q adds a zoom. Wordmark menu in French: every row 31 px (one line), menu 261 px wide, "2.0.0-rc.13" in one piece. Preview: playback from 0:12 to 0:45.8 across a trim (0:15.0 → 0:18.5) and a 1.5× region with no canvas stall over 150 ms, 32–50 picture changes a second, wall time matching the edit; stops at 0:51.3 on the screen's and the webcam's last frames. #960 (#964): A → B → A with the playhead at 0:06, the preview is pixel-identical to before (mean difference 0, against 77.9 for project B). Background image → gradient → colour followed in the preview. #968: a media file whose read the OS refused (`icacls` deny on the test file) — *Regenerate* shows the translated "Impossible de lire l'audio de ce média…" hint inline and in the "Échec de la transcription" toast, without the IPC wrapper; the main process logs `MediaUnreadableError … Permission denied`. MP4 export from the GUI, 1752×1080 30 fps: 2930 frames, video 97.633 s against audio 97.666 s, zoom, background, webcam and cursor present in extracted frames, −18.3 LUFS. **Also seen:** the green "Aucune parole détectée" chip from the earlier successful run stays next to the unreadable hint; each clip of the take still carries 0.32 s of audio past its video (what is left of #942), and the export ends each clip at its video (97.63 s, against 98.22 s from the clip lengths the timeline shows). **Not run:** the HUD with no camera (#967; a camera is attached), the macOS HUD fix (c2a8559e), tray, microphone and system-audio content, GIF, macOS and Linux. |