Repository navigation
agent: expose the streamer's wait, record and screenshot - #17
Merged
Merged
Conversation
The console can already say when the screen settles, save an MP4 from the
encode it is already running, and hand out its last frame. None of that was
reachable from the agent API — which is where it is worth the most, because a
computer-use loop could only sleep and hope the UI had caught up.
- GET /api/desktops/{id}/wait — returns when the screen stops changing (or at
the first change with ?change=1). Damage-driven inside the guest, so waiting
costs nothing while the screen is still: no polling, no screenshots.
- POST /api/desktops/{id}/record?seconds= — an MP4 written from the live H.264,
so no second encoder and no extra CPU. Clamped to 300s.
- GET /api/desktops/{id}/screenshot — the streamer's frame when there is one
(a copy, not a process spawn), the agent's ImageMagick capture otherwise.
An image built before voxl keeps working: /screenshot falls back to the agent
and /wait answers 501 with a reason. Audio now switches on by itself for an
image that ships a PulseAudio socket, instead of being hardcoded off.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The last PR made voxl the console. It also handed us three capabilities warmbox never had — and the agent API, which is where they matter most, could not reach any of them. A computer-use loop had exactly one tool for "did my click work?": sleep, then screenshot.
What
GET /api/desktops/{id}/wait?change=1for the first change).idle,region,since,timeoutare passed through.POST /api/desktops/{id}/record?seconds=GET /api/desktops/{id}/screenshot/waitis damage-driven inside the guest (XDamage), so it costs nothing while the screen is still. It is the difference between act → sleep → hope and act → wait → look.Compatibility
An image built before voxl behaves exactly as it did:
/screenshotfalls back to the agent's own capture, and/waitanswers501 stream_unavailablewith a reason rather than hanging.?classic=1still forces noVNC.Audio: the guest used to hardcode
--audio-enabled=false. It now turns on by itself when the image has a PulseAudio socket. Our images still ship no sound stack, so nothing changes today — but an image that adds one gets audio without a code change.Test plan
Against a real
xfce-voxldesktop, through the daemon:Against the published
xfceimage (no voxl), the same binary:go build,go vet,go test ./...pass;recordSecondsis unit-tested at the bounds.