Skip to content

Use a tablet as one more screen from the window - #134

Open
avillagran wants to merge 60 commits into
omacom:mainfrom
avillagran:feat/tablet-screen-pr
Open

avillagran wants to merge 60 commits into
omacom:mainfrom
avillagran:feat/tablet-screen-pr

Conversation

@avillagran

@avillagran avillagran commented Nov 11, 2026 •

Copy link
Copy Markdown

Stacked PR. This builds on the gliff-server --vnc PR (#133) and on #106, #108 and #109, for the window's header buttons. Only the last two commits are new here.

The problem

gliff-server --vnc makes a tablet one more screen, but setting it up means a terminal, a command line and a VNC app on the tablet. An old iPad cannot install a VNC app any more, and it has a web browser.

What this adds

  • A tablet button in the window's header opens a dialog. It sets:

    • where the screen stands next to this computer's, and its size;
    • colours: 256 colours send a byte a pixel instead of four;
    • a frame rate of 15, 10 or 5 a second, which leaves an old tablet time to answer touches.

    The dialog starts and stops the screen, and copies the address to open on the tablet.

  • bin/install-tablet sets up two user units:

    • gliff-tablet-vnc serves the screen with gliff-server --vnc on 127.0.0.1:5900 only.
    • gliff-tablet-web serves a page on port 6080 that shows the screen through noVNC and a websockify bridge, so the tablet needs nothing but a browser.

    noVNC 0.6.2 is the last release that runs on iOS 8's Safari. The installer downloads it and checks its checksum.

  • The page's bar:

    • Full screen.
    • Disconnect and Connect.
    • Remove Screen and Put Screen Back, which move the windows away from the tablet's screen and back.
  • The page reconnects by itself after the tablet sleeps. Colours and frame rate apply without restarting the screen; only a new side, alignment or size restarts it.

  • docs/tablet.md covers set-up, use and security.

Screenshots

Taken from a test laptop running Hyprland. The part of the address after the port is covered, since it carries the password.

The tablet dialog from the window's header: where the screen stands, its size, colours and frames, and the address to open on the tablet.

The tablet dialog

The tablet page in a browser, opened without the password in its address, so it asks for it. Its bar has Full Screen, Disconnect and Remove Screen.

The tablet page asking for the password

Security

  • The address carries the VNC password after #pw=, so a Home Screen icon opens without asking. The part after # is not sent over the network, but anyone who sees the address can use the screen.
  • The page and VNC are not encrypted. The docs say to open port 6080 only to your own devices.

Testing

  • scripts/check.sh passes. The tablet module has unit tests for the address and the settings file.
  • By hand, with an iPad 2 on iOS 8.4.1 beside an aarch64 laptop running Hyprland, in daily use:
    • The screen was placed and moved with the dialog: below/start at 1024x768 landed where expected.
    • The page connected from Safari and as a Home Screen web app.
    • Switching to 256 colours made the page reconnect at 8 bits a pixel without restarting the screen.
    • The newest viewer took the screen over, and the one it replaced stopped reconnecting.

- The manifest is written atomically before and after each file, with
  the hash of the file being written as pending. An install that stops part
  way leaves only files gliff recognises as its own and is marked
  unfinished, so the next attempt finishes it even though gliff-server may
  already be on PATH.
- Selecting the machine's tab while gliff is being installed no longer
  starts a connection, which made the install's reconnect stale and could
  offer a second install over the first.
- The desktop file's Exec path is quoted and escaped by the desktop entry
  rules, so a home directory with spaces, quotes or % still launches.
A headless output always went where Hyprland found room. --position puts
it right, left, above or below this machine's screens, and --align lines
it up with their top or left edges (start), their middles, or their
bottom or right edges (end). The position is worked out from the other
screens' layout each time the output's mode is set, so a resize keeps
it in place beside them.
The client offers to install gliff only while ssh cannot find
gliff-server. With gliff-server written second, an install that failed
on the desktop file or the icon left a working server on PATH, so the
offer never came back and the rest stayed missing. It is now written
last, so until everything else is in place the next connection offers
the install again, and the script finishes it over its own files.

A file that cannot be written now reports which one and why, instead of
the temporary file's name, and an unreadable manifest says how to
recover.
gliff --stdio shows the gliff-server whose protocol arrives on stdin and
leaves on stdout. Another machine runs that server and starts this over
ssh with the server's stdout as its stdin, to use this screen as one of
its own; only that machine needs ssh access to this one.

- stdout is kept for the protocol and fd 1 points at stderr, so nothing
  that prints can corrupt the stream.
- Started by ssh outside the desktop session, it finds the user's
  Hyprland session the way gliff-server does.
- The window shows only that session and remembers nothing; it closes
  when the stream ends, since only the other machine can start it again.
- --fullscreen opens the window fullscreen; the header no longer warns
  when the window starts that way.
A headless output always went where Hyprland found room. --position puts
it right, left, above or below this machine's screens, and --align lines
it up with their top or left edges (start), their middles, or their
bottom or right edges (end). The position is worked out from the other
screens' layout each time the output's mode is set, so a resize keeps
it in place beside them.
The header gets a button that adds a headless output here and shows it
on the selected machine with `gliff --stdio --fullscreen` over ssh, and
a settings button for which side of this computer's screens it stands
on and how it lines up. A gliff there without `--stdio` still shows it
through a relay in place of its ssh, with Hyprland making its window
fullscreen. Stopping, the other window closing or this window closing
gives the screen back.
The position comes from the output's logical size, which resize
computed with the requested scale. When Hyprland applies a rounded
scale instead, place the output again at that scale, so it still meets
the other screens on the chosen side and lines up as asked.
The position comes from the output's logical size, which resize
computed with the requested scale. When Hyprland applies a rounded
scale instead, place the output again at that scale, so it still meets
the other screens on the chosen side and lines up as asked.
The window closes when that stream ends, so a machine opened from its
address bar would close with it. A --stdio window shows neither the +
nor the address bar, and connecting from the bar does nothing there.
A --stdio window that closes on a connection error or an incompatible
server now makes gliff exit with a failure, with the reason as its last
stderr line, so the machine that started it can say why. A window the
user closes, or a stream the other side ends, still exits successfully.
Showing another machine forgets the lent screen's tab as the selection,
and selecting that tab again did not bring it back, so it lost its
highlight and the screen buttons asked for a tab once the screen
stopped. Selecting a tab whose screen is in use now makes it the
selected one, as starting the screen does.
Hyprland places a screen whose position is `auto`, as Omarchy's default
configuration does, after every screen with a fixed one. A fixed
position for the headless output therefore pushed such a screen aside:
on a laptop asked for `--position right`, the laptop screen moved right
of the new output. When placing the output moves another screen, the
placement goes to Hyprland's `auto-right`, `auto-center-right` and
related positions instead. They line up at the start or the center of a
side, so `--align end` is centered there, with a warning.
CaptureConfig::paint_cursors asks the compositor to draw cursors onto
the captured frames (ext-image-copy-capture's paint_cursors option), for
a viewer that cannot draw a cursor of its own. It is off by default, so
the gliff session, which sends the cursor separately, is unchanged.
gliff-vnc holds the RFB protocol for serving a Hyprland output to VNC
clients, kept apart from any I/O beyond the stream it is given so each
part can be tested on its own:

- the handshake for RFB 3.3, 3.7 and 3.8, with VNC Authentication
  (DES challenge-response, checked against OpenSSL) or none;
- every pixel format a client may ask for: 8, 16 and 32-bit true colour
  in either byte order, and 8-bit colour maps, which get a 3-3-2 map;
- the Raw, Hextile and ZRLE encodings, decoded back in the tests by a
  separate decoder over desktop-like and random pictures;
- which 16x16 tiles changed since the client was last sent them;
- a keymap of the server's own on which each keysym a client sends has a
  key, with modifiers kept as modifiers and keys reused least recently
  typed first, so any character can be typed whatever the local layout.
`gliff-server --vnc ADDR --headless --size WxH` adds a screen of that
size beside the others (`--scale`, `--position` and `--align` as for a
gliff session) and serves it to VNC clients until stopped, so a device
with any VNC viewer, however old, becomes one more screen of this
machine. `--output` serves an existing screen instead.

The screen lives as long as the server: a client that sleeps or drops
comes back to the same windows. One client is served at a time, and one
that logs in takes over from the previous one; a wrong password waits
two seconds before it is refused. A password is required, from
--vnc-password-file or GLIFF_VNC_PASSWORD, unless --vnc-no-password
says otherwise.

Frames are captured with the cursor painted in, only while a client is
connected, and only once the screen changes; a client's first update
waits for a frame newer than its connection. Updates carry the tiles
that changed. Key events type keysyms through gliff-vnc's keymap, and
pointer events move, click and scroll on that screen.

setup_output now takes the mode to start in, so the VNC screen starts at
its size and scale, and places the output again when Hyprland rounds
the scale. SIGINT, SIGTERM and SIGHUP give the screen back.
Hyprland places a screen whose position is `auto`, as Omarchy's default
configuration does, after every screen with a fixed one. A fixed
position for the headless output therefore pushed such a screen aside:
on a laptop asked for `--position right`, the laptop screen moved right
of the new output. When placing the output moves another screen, the
placement goes to Hyprland's `auto-right`, `auto-center-right` and
related positions instead. They line up at the start or the center of a
side, so `--align end` is centered there, with a warning.
gliff 0.4.1 opens the GPU decoder whenever VA-API lists H.264, and NVIDIA's
VA-API driver does, but every picture then fails to export, so the screen
lent to such a machine stayed black. That gliff cannot switch to the CPU on
its own, so the relay asks for it with GLIFF_VIDEO=cpu when every render
node belongs to the nvidia driver, unless GLIFF_VIDEO is already set.
A new crate puts a one-pixel layer surface along chosen outer edges of the
layout. When the pointer touches one it locks the pointer there, hides it,
takes the keyboard and inhibits shortcuts, and reports relative motion,
buttons, the wheel, keys, modifiers and the keymap until told to release.
Releasing puts the pointer back just inside that edge, through a new
hypr-ipc move_cursor that speaks both config formats.
--control SIDE skips the picture: after the usual Hello it binds a virtual
keyboard and pointer, with the client's keymap, to the screen on SIDE of the
layout, answers with that screen's size, and injects the client's keys,
motion, buttons and wheel until the client goes. The handshake and the
setup error report are the session's own.
gliff --control SIDE:HOST, given once for each machine, runs with no window:
it keeps a gliff-server --control connection to each machine, reconnecting
with backoff, and arms that side's edge only while the machine answers.
Pushing the pointer past an armed edge carries it onto that machine's
screen, entering at the facing edge where it left this one. Moving back
across that edge, or the release hotkey, returns it at the matching point;
a held button keeps the pointer there. Keys and buttons still held are let
go on the other machine on every return. The release hotkey is matched by
keysym in the local keymap, so it means the same keys in any layout.
# Conflicts:
#	crates/gliff/src/main.rs
A mouse button beside the screen buttons drives the selected machine with
this computer's pointer and keyboard across the edge on its side: the same
side the screen settings keep, asked for the first time as they are, without
the alignment, which only a screen in use needs. Each driven machine runs
its control on a thread of its own, so one machine on each side can be
driven at once; a second machine on a side already taken is refused.

The machine's tab stays selected without its session, says how to cross
over or that it is connecting, and its dot shows whether the machine
answers. The tab's button or the mouse button, now a stop button, ends the
control, which gives the pointer back. A machine is not driven and used as
a screen at once. Closing the window ends every control.

The control loop now stops on a signal from its owner and reports when a
machine connects, drops and why, and when it ends.
While a machine is driven or its screen is in use, the window has nothing
to show, so a button beside the screen buttons hides it. A tray icon (a
StatusNotifierItem over the session bus, which Omarchy's bar and Waybar
show) stands in for it, and a click on the icon brings the window back.
Without a tray to come back from, the window stays. The window also comes
back by itself once nothing runs in the background any more.
A speaker button, shown while the selected machine is driven, moves that
machine's sound to this one: over ssh, pw-record takes what its default
output plays, before its volume, as raw 48 kHz stereo, and pw-play plays it
here. The other machine's output is muted meanwhile and unmuted however the
stream ends. It is muted only once the capture has woken the output,
because WirePlumber restores a suspended output's saved mute as it wakes,
and once more a second later. The choice is kept per machine and applies
the next time it is driven. Stopping the control, or the window, stops the
sound too.
A compiled remote_install.py was committed by mistake with the
recoverable install. Remove it and ignore __pycache__ directories.
The client works out the pointer's extent from the size in HelloAck, but
it got the mode as Hyprland reports it, unrotated, while the server bound
the pointer to the rotated screen. On a portrait screen the pointer could
not reach its bottom. Swap the size for a 90 or 270 degree transform, as
the video session does.
The pointer returned to the edge's screen as it stood when control
started. A screen moved or rescaled meanwhile put it somewhere else,
possibly on the edge again. Read the layout when releasing, and keep the
old one only when it cannot be read.
The script unmuted whichever output was the default when the stream
ended, and read the mute of the default rather than of the output it
muted. If a headset became the default meanwhile, the speakers stayed
muted and the headset was unmuted. Keep the default output at the start
and use that one throughout.
A sound stopped and started again, or stopped with its control and
started with the next one, ends after the new sound began. Its end
cleared the machine's sound by name, which dropped the new one and killed
its ssh. Each sound now has an id, and an end from another run is
ignored.
An edge's screen and rectangle were chosen once, so a screen added
farther out left the edge on what was now an inner side, and a moved or
rescaled screen kept the old coordinates. hypr-wl's Outputs now binds
and forgets outputs that come and go, and hypr-edge reads the layout
again after each output's burst, when one goes away, and when the
pointer is given back: an edge whose outermost screen changed is put on
the new one once Wayland has announced it, or after the pointer it holds
is given back. Whether an edge is armed is now kept apart from its
surface, so a surface the compositor closes with its screen is given
again, and events from an edge's previous surface are ignored.
Keymaps and locks reach a link by separate ways, so a layout selected in
a keymap that had just gained it could arrive first, be dropped by a
server still on the old keymap, and not be sent again. The link now
writes the last locks again right after each keymap.
An edge whose screen changed while it held the pointer moved only when
the pointer was given back by a release. When the compositor ended the
capture instead, by leaving the edge or unlocking the pointer, the edge
stayed on the old screen until the next layout change. Losing the
capture now reads the layout again, as a release does.
The Omarchy package has no --control yet, so a machine on it failed the
connection and retried for ever, with no hint why. Ask the machine before
driving it, and offer to install this computer's server for that user. The
upload has to run there before it replaces gliff's own file.
The window's clipboard stops at its sessions, so a machine driven from an
edge could neither give nor take a copy. Move the server's bridge to a crate
of its own and run it on both ends of --control. One clipboard here serves
every driven machine: a copy here is offered to them all, and a copy there
is pasted from the machine that made it.
The stream arrives in bursts, and over WiFi one is sometimes 70 ms late,
which left the 30 ms player with nothing to play.
With a machine on each side the screen between them is narrow enough for a
fast return to reach the other edge and leave again.
Installing this computer's gliff-server in ~/.local/bin on a machine whose
system package is older did not help when that machine's ssh PATH puts
/usr/bin first: `gliff-server` still ran the old one, and the update was
offered again every time.

The installer now reports whether `gliff-server` on that machine's PATH is
the one it wrote, both for a fresh install and for an update. The control
flow tries the user install by its own path when the one on PATH cannot be
controlled, before offering the update and again after it, so an older
package first on PATH no longer blocks control, and a machine already
updated is driven without asking again.
A config reload drops the monitor rule that set a headless output's mode, and Omarchy reloads when a monitor comes or goes, which left a tablet's 1024x720 screen at 1920x1080 scale 2. The VNC server checks the mode every two seconds and sets it back.
# Conflicts:
#	README.md
#	crates/gliff-server/src/main.rs
#	crates/gliff-server/src/session.rs
A header button opens a dialog that places the screen, sizes it, starts or stops it, and copies the address to open on the tablet. Two user units, set up by bin/install-tablet, serve the screen over VNC on localhost and a noVNC page that shows it on port 6080, so an old iPad needs no app.
256 colors send a byte a pixel instead of four, and a frame rate spaces out the page's update requests, both easier on an old tablet. The bridge serves them to the page, which connects again when they change; only a new side, alignment or size restarts the screen.
@greptile-apps

greptile-apps Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Critical impact] The changes appear safe to merge; the reported viewer handoff race is addressed.

Summary

Adds a tablet screen through a browser, with a settings dialog, installer, and user services.

  • VNC viewers can show and control a screen from a Hyprland desktop.
  • A tablet can show a new desktop screen in its web browser.
  • Another machine's screen can appear beside the screens on this computer.
  • The pointer and keyboard can cross a screen edge to control another machine.
  • Gliff can install a compatible server for the current user on another machine.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Dialog[Tablet dialog] --> Settings[Saved settings]
  Dialog --> Units[User services]
  Settings --> Bridge[Web bridge]
  Units --> Bridge
  Units --> VNC[VNC server]
  Tablet[Tablet browser] <--> Bridge
  Bridge <--> VNC
  VNC <--> Screen[Headless screen]
Loading

Reviews (3) · Last reviewed commit: "Let a replaced tablet viewer wait a mome..." · Reviewed by Greptile

Comment thread crates/gliff-server/src/vnc.rs Resolved
Comment thread crates/gliff-vnc/src/proto.rs Resolved
Comment thread tablet/tablet-web.py Resolved
Comment thread crates/gliff-server/src/vnc.rs Resolved
Comment thread bin/install-tablet Resolved
- Updates go to a writer task per client through a queue of four, each
  write with a 10-second limit; a client that stops reading is dropped
  instead of blocking logins, frames and shutdown. Client messages are
  read ahead through a bounded queue, so reading waits instead of
  growing without end.
- A client's reader and writer tasks stop when it is dropped or
  replaced.
- The client's version line is read as bytes, so a line that is not
  ASCII is refused instead of panicking, and a login slot is given back
  by a guard however the login ends.
- SetEncodings changes only the encoding: the encoder, and the ZRLE
  stream that runs for the whole connection, stay.
- A headless screen keeps the mode it was made with as the one to
  restore after a reload; frames of another size no longer replace it.
- --scale takes a finite value from 0.5 to 4.
…G dirs

The bridge named each new websocket the newest viewer as it opened, so
anyone who could reach port 6080 could make the tablet's page close as
replaced and stop reconnecting, without the password. The bridge now
reads the server's side of the login as it passes (version, security
type, challenge, result) and names the viewer only once gliff-server
has accepted it; a viewer still logging in is never taken as replaced.

install-tablet honoured XDG_DATA_HOME and XDG_CONFIG_HOME but the units
and the bridge assumed the defaults. It now writes the directories it
used into both units, and passes gliff's config directory to the bridge.
@avillagran

Copy link
Copy Markdown
Author

All five findings are addressed. The three in the VNC server are fixed in #133 and merged here (84cd30a). The bridge's newest-viewer check and the XDG directories are fixed in 6b5551c. scripts/check.sh passes (338 tests).

Comment thread tablet/tablet-web.py Resolved
The headless screen's requested mode stays apart, but the pointer's
extent now follows the frame size, so clicks land where the viewer sees
them during the moment a reload holds the screen at another size.
gliff-server can drop the older viewer before the newer bridge has read
its login reply and named itself, and the older page then reconnected
and took the screen back. On a close, the older bridge now waits up to a
second for a newer viewer's name before reporting a plain close.
avillagran added a commit to avillagran/gliff that referenced this pull request Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant