Skip to content

Show and drive machines that run a VNC server - #135

Open
avillagran wants to merge 62 commits into
omacom:mainfrom
avillagran:feat/vnc-client-pr
Open

avillagran wants to merge 62 commits into
omacom:mainfrom
avillagran:feat/vnc-client-pr

Conversation

@avillagran

@avillagran avillagran commented Nov 11, 2026 •

Copy link
Copy Markdown

Stacked PR. This builds on #131 (RDP and RustDesk machines). Only a40af4b is new here. Once #131 lands, the diff shrinks to it.

The problem

#131 lets gliff show and drive machines that run RDP or RustDesk instead of gliff. Many machines offer only VNC: a Mac's screen sharing, wayvnc on another Wayland desktop, TigerVNC or x11vnc on a server.

What this adds

  • vnc://[user@]host[:port] machines, with VNC in the address bar's protocol badge. A port below 100 is a display number, so :1 means port 5901.
    • A tab shows the screen.
    • The mouse button drives the machine from the screen edge, as it does for RDP and RustDesk.
    • A new machine that ssh fails on is also checked for VNC on port 5900.
  • The client (gliff-foreign/src/vnc):
    • Protocol: RFB 3.3, 3.7 and 3.8.
    • Logins:
      • none;
      • the VNC password (DES over the server's challenge);
      • VeNCrypt under TLS with a certificate, with a user and password (wayvnc's enable_auth), the VNC password, or nothing.
    • Encodings: ZRLE, CopyRect and Raw, plus the cursor, DesktopSize, ExtendedDesktopSize and LastRect pseudo-encodings.
    • Keys: sent by position through QEMU's extended key events when the server takes them, as gliff's own sessions do. Otherwise gliff sends the keysym a US keyboard types. Input waits for the first update so every key goes the same way.
    • Driving from the edge asks for a single pixel, once, so no picture is streamed.
    • The login has the same deadline and cancellation as RDP and RustDesk from Show and drive machines that run RDP or RustDesk #131: it stops when the tab or the control stops.
  • Certificates: TLS certificate pinning now lives in a small tls module that RDP and VNC share. A certificate is kept on first use and a different one refused later, as ssh does with host keys.

Screenshots

Taken from gliff running on a test laptop.

VNC in the protocol badge.

The badge's menu with VNC

Logging in to wayvnc, which asks for a user as well as the password, under TLS with its certificate kept on first use.

The VNC login dialog

A VNC desktop in a tab. This is a bare TigerVNC session (vnc://127.0.0.1:5, VNC password) with no programs running.

A TigerVNC desktop shown over VNC

Security

  • VNC password: it never travels in the clear, but the session is not encrypted and only the first eight characters count. docs/foreign.md says so.
  • Not used: VeNCrypt's anonymous TLS, which rustls does not do and which authenticates nothing, and VeNCrypt's user and password without TLS. A server that offers only those gets a clear error that says to give it a certificate.

Testing

  • scripts/check.sh passes. The new unit tests cover:
    • the DES answer against an OpenSSL vector;
    • version agreement and the VeNCrypt choice;
    • a scripted 3.8 server: login, wrong password, and Raw, ZRLE palette and solid tiles across one zlib stream, and CopyRect;
    • ZRLE runs, cursor masks, and the input encoding.
  • By hand against real servers:
    • wayvnc 0.10.2 without authentication. Picture, pointer, and typing both by keysym and by key position.
    • wayvnc with enable_auth and a certificate. User and password; a wrong password is asked for again; the certificate is kept and a changed one refused.
    • TigerVNC 1.16, with the VNC password on its own and inside VeNCrypt.
    • From the window, over the LAN, a wayvnc desktop shown and driven.

- The manifest is written atomically before and after each file, with
  the hash of the file being written as pending. An install that stops part
  way leaves only files gliff recognises as its own and is marked
  unfinished, so the next attempt finishes it even though gliff-server may
  already be on PATH.
- Selecting the machine's tab while gliff is being installed no longer
  starts a connection, which made the install's reconnect stale and could
  offer a second install over the first.
- The desktop file's Exec path is quoted and escaped by the desktop entry
  rules, so a home directory with spaces, quotes or % still launches.
The client offers to install gliff only while ssh cannot find
gliff-server. With gliff-server written second, an install that failed
on the desktop file or the icon left a working server on PATH, so the
offer never came back and the rest stayed missing. It is now written
last, so until everything else is in place the next connection offers
the install again, and the script finishes it over its own files.

A file that cannot be written now reports which one and why, instead of
the temporary file's name, and an unreadable manifest says how to
recover.
gliff --stdio shows the gliff-server whose protocol arrives on stdin and
leaves on stdout. Another machine runs that server and starts this over
ssh with the server's stdout as its stdin, to use this screen as one of
its own; only that machine needs ssh access to this one.

- stdout is kept for the protocol and fd 1 points at stderr, so nothing
  that prints can corrupt the stream.
- Started by ssh outside the desktop session, it finds the user's
  Hyprland session the way gliff-server does.
- The window shows only that session and remembers nothing; it closes
  when the stream ends, since only the other machine can start it again.
- --fullscreen opens the window fullscreen; the header no longer warns
  when the window starts that way.
A headless output always went where Hyprland found room. --position puts
it right, left, above or below this machine's screens, and --align lines
it up with their top or left edges (start), their middles, or their
bottom or right edges (end). The position is worked out from the other
screens' layout each time the output's mode is set, so a resize keeps
it in place beside them.
The header gets a button that adds a headless output here and shows it
on the selected machine with `gliff --stdio --fullscreen` over ssh, and
a settings button for which side of this computer's screens it stands
on and how it lines up. A gliff there without `--stdio` still shows it
through a relay in place of its ssh, with Hyprland making its window
fullscreen. Stopping, the other window closing or this window closing
gives the screen back.
The position comes from the output's logical size, which resize
computed with the requested scale. When Hyprland applies a rounded
scale instead, place the output again at that scale, so it still meets
the other screens on the chosen side and lines up as asked.
The window closes when that stream ends, so a machine opened from its
address bar would close with it. A --stdio window shows neither the +
nor the address bar, and connecting from the bar does nothing there.
A --stdio window that closes on a connection error or an incompatible
server now makes gliff exit with a failure, with the reason as its last
stderr line, so the machine that started it can say why. A window the
user closes, or a stream the other side ends, still exits successfully.
Showing another machine forgets the lent screen's tab as the selection,
and selecting that tab again did not bring it back, so it lost its
highlight and the screen buttons asked for a tab once the screen
stopped. Selecting a tab whose screen is in use now makes it the
selected one, as starting the screen does.
Hyprland places a screen whose position is `auto`, as Omarchy's default
configuration does, after every screen with a fixed one. A fixed
position for the headless output therefore pushed such a screen aside:
on a laptop asked for `--position right`, the laptop screen moved right
of the new output. When placing the output moves another screen, the
placement goes to Hyprland's `auto-right`, `auto-center-right` and
related positions instead. They line up at the start or the center of a
side, so `--align end` is centered there, with a warning.
gliff 0.4.1 opens the GPU decoder whenever VA-API lists H.264, and NVIDIA's
VA-API driver does, but every picture then fails to export, so the screen
lent to such a machine stayed black. That gliff cannot switch to the CPU on
its own, so the relay asks for it with GLIFF_VIDEO=cpu when every render
node belongs to the nvidia driver, unless GLIFF_VIDEO is already set.
A new crate puts a one-pixel layer surface along chosen outer edges of the
layout. When the pointer touches one it locks the pointer there, hides it,
takes the keyboard and inhibits shortcuts, and reports relative motion,
buttons, the wheel, keys, modifiers and the keymap until told to release.
Releasing puts the pointer back just inside that edge, through a new
hypr-ipc move_cursor that speaks both config formats.
--control SIDE skips the picture: after the usual Hello it binds a virtual
keyboard and pointer, with the client's keymap, to the screen on SIDE of the
layout, answers with that screen's size, and injects the client's keys,
motion, buttons and wheel until the client goes. The handshake and the
setup error report are the session's own.
gliff --control SIDE:HOST, given once for each machine, runs with no window:
it keeps a gliff-server --control connection to each machine, reconnecting
with backoff, and arms that side's edge only while the machine answers.
Pushing the pointer past an armed edge carries it onto that machine's
screen, entering at the facing edge where it left this one. Moving back
across that edge, or the release hotkey, returns it at the matching point;
a held button keeps the pointer there. Keys and buttons still held are let
go on the other machine on every return. The release hotkey is matched by
keysym in the local keymap, so it means the same keys in any layout.
# Conflicts:
#	crates/gliff/src/main.rs
A mouse button beside the screen buttons drives the selected machine with
this computer's pointer and keyboard across the edge on its side: the same
side the screen settings keep, asked for the first time as they are, without
the alignment, which only a screen in use needs. Each driven machine runs
its control on a thread of its own, so one machine on each side can be
driven at once; a second machine on a side already taken is refused.

The machine's tab stays selected without its session, says how to cross
over or that it is connecting, and its dot shows whether the machine
answers. The tab's button or the mouse button, now a stop button, ends the
control, which gives the pointer back. A machine is not driven and used as
a screen at once. Closing the window ends every control.

The control loop now stops on a signal from its owner and reports when a
machine connects, drops and why, and when it ends.
While a machine is driven or its screen is in use, the window has nothing
to show, so a button beside the screen buttons hides it. A tray icon (a
StatusNotifierItem over the session bus, which Omarchy's bar and Waybar
show) stands in for it, and a click on the icon brings the window back.
Without a tray to come back from, the window stays. The window also comes
back by itself once nothing runs in the background any more.
A speaker button, shown while the selected machine is driven, moves that
machine's sound to this one: over ssh, pw-record takes what its default
output plays, before its volume, as raw 48 kHz stereo, and pw-play plays it
here. The other machine's output is muted meanwhile and unmuted however the
stream ends. It is muted only once the capture has woken the output,
because WirePlumber restores a suspended output's saved mute as it wakes,
and once more a second later. The choice is kept per machine and applies
the next time it is driven. Stopping the control, or the window, stops the
sound too.
A compiled remote_install.py was committed by mistake with the
recoverable install. Remove it and ignore __pycache__ directories.
The client works out the pointer's extent from the size in HelloAck, but
it got the mode as Hyprland reports it, unrotated, while the server bound
the pointer to the rotated screen. On a portrait screen the pointer could
not reach its bottom. Swap the size for a 90 or 270 degree transform, as
the video session does.
The pointer returned to the edge's screen as it stood when control
started. A screen moved or rescaled meanwhile put it somewhere else,
possibly on the edge again. Read the layout when releasing, and keep the
old one only when it cannot be read.
The script unmuted whichever output was the default when the stream
ended, and read the mute of the default rather than of the output it
muted. If a headset became the default meanwhile, the speakers stayed
muted and the headset was unmuted. Keep the default output at the start
and use that one throughout.
A sound stopped and started again, or stopped with its control and
started with the next one, ends after the new sound began. Its end
cleared the machine's sound by name, which dropped the new one and killed
its ssh. Each sound now has an id, and an end from another run is
ignored.
If the tray's connection failed, its thread ended and the icon went with
it, but the window stayed hidden, leaving no way to stop what ran. A
closed channel from the tray now shows the window, as a click does.
The other machine's keyboard started from a fresh state and followed only
the keys typed into it, so a second layout selected here, or Caps Lock or
Num Lock already on, did not apply there. The client now sends its
layout in effect and locked modifiers, by name, on entering and whenever
they change while captured, in a new KeyboardLocks message. The control
server lists the keyboard-locks feature and applies them to its virtual
keyboard; a server that does not list it is not sent them, and the video
session ignores the message.
Hotkey::parse accepts names such as exclam, but matching looked only at
each key's unshifted symbol, so shift+exclam was accepted and never
matched. A key now matches by its unshifted symbol or by the one it
produces with the modifiers held.
A link whose network went away without closing the connection stayed
up: nothing read had a deadline, so its edge kept taking the pointer to
a machine that was gone, and input queued for a write that never ended.
The link now pings every 2 seconds and is dropped after 6 seconds
without hearing from the other machine, or when a write takes that long;
it then disarms its edge and reconnects as after any drop.
An edge's screen and rectangle were chosen once, so a screen added
farther out left the edge on what was now an inner side, and a moved or
rescaled screen kept the old coordinates. hypr-wl's Outputs now binds
and forgets outputs that come and go, and hypr-edge reads the layout
again after each output's burst, when one goes away, and when the
pointer is given back: an edge whose outermost screen changed is put on
the new one once Wayland has announced it, or after the pointer it holds
is given back. Whether an edge is armed is now kept apart from its
surface, so a surface the compositor closes with its screen is given
again, and events from an edge's previous surface are ignored.
The Omarchy package has no --control yet, so a machine on it failed the
connection and retried for ever, with no hint why. Ask the machine before
driving it, and offer to install this computer's server for that user. The
upload has to run there before it replaces gliff's own file.
The window's clipboard stops at its sessions, so a machine driven from an
edge could neither give nor take a copy. Move the server's bridge to a crate
of its own and run it on both ends of --control. One clipboard here serves
every driven machine: a copy here is offered to them all, and a copy there
is pasted from the machine that made it.
The stream arrives in bursts, and over WiFi one is sometimes 70 ms late,
which left the 30 ms player with nothing to play.
With a machine on each side the screen between them is narrow enough for a
fast return to reach the other edge and leave again.
Installing this computer's gliff-server in ~/.local/bin on a machine whose
system package is older did not help when that machine's ssh PATH puts
/usr/bin first: `gliff-server` still ran the old one, and the update was
offered again every time.

The installer now reports whether `gliff-server` on that machine's PATH is
the one it wrote, both for a fresh install and for an update. The control
flow tries the user install by its own path when the one on PATH cannot be
controlled, before offering the update and again after it, so an older
package first on PATH no longer blocks control, and a machine already
updated is driven without asking again.
gliff-foreign shows and drives a machine through its Remote Desktop
server (IronRDP: TLS, NLA with NTLM, the first self-signed certificate
kept as ssh keeps host keys) or through RustDesk by direct IP (its
encrypted handshake and salted password challenge, VP8 and VP9 through
the system libvpx). Input comes as gliff's own messages and leaves as
each protocol's events, keys by position for the other machine's
platform.
rdp://user@host and rustdesk://host are machines like any other: a tab
shows the screen, and the mouse button drives it across a screen edge,
on the side picked for it. The login is asked for once and kept in the
desktop's keyring; a refused one is asked for again, and an RDP server's
first self-signed certificate is kept like an ssh host key.

A machine added by its ssh name that does not let ssh in is checked for
RDP and RustDesk, and gliff offers to reach it through what answers.
…out a key

The 10-second limit covered only the TCP connection: an RDP server that
accepted it and then stopped answering left the tab connecting, and closing
the tab did not end the wait. The whole login now has a deadline (30 s for
RDP, RustDesk's two minutes for a person to accept), and it is dropped as
soon as the tab or the control stops. Input sent meanwhile stays queued.

A RustDesk machine whose identity carries no key, or that opens with the
login challenge, would have taken the password proof, the screen and the
keys over plain TCP. gliff now refuses it.

RustDesk's own encryption uses one key in both directions with a counter
on each side, so a message each way shares a nonce; gliff cannot change
that and stay compatible. docs/foreign.md says so and points to a VPN or
an ssh tunnel on networks you do not trust.
…er runs

Driving an RDP machine from the edge before ever showing it started with
no saved certificate and dropped the one it saw, so each reconnect took
whatever certificate came as the first. The control now holds the server
to the first one it sees and saves it in the config, as the window does.

The config is written beside and renamed over, so the window and a
control saving at once never leave half a file.
Every writer loaded the config, changed it and saved it on its own, so a
control saving a certificate while the window saved its tabs could undo
one or the other, and threads of one process shared a temporary name.
Config::update now loads, changes and saves under a lock, a mutex within
the process and an exclusive flock on config.toml.lock between gliff
processes, and every save uses a temporary file of its own. All the
window's saves and the control's go through it.

A control keeps a new certificate as soon as it is seen, rather than when
the session ends, so stopping it mid-session still leaves it saved.
@greptile-apps

greptile-apps Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[High impact] The PR appears safe to merge with the documented limits of RustDesk direct-IP connections.

Summary

Adds VNC machines alongside RDP and RustDesk:

  • Gliff can open and control VNC, RDP, and RustDesk machines.
  • The pointer and keyboard can cross a screen edge to another machine.
  • Gliff offers to install its server when a remote machine lacks it.
  • Remote screens can join the desktop layout and stay in their tabs.
  • Sound from a controlled machine plays through this computer.
  • Gliff can keep running in the tray after its window closes.

All five earlier, unnumbered findings are addressed: saved VNC certificates require TLS, controls save certificates immediately, config updates are serialized, VNC key releases reuse the pressed symbol, and display zero maps to port 5900.

avillagran explicitly documents plaintext RustDesk traffic and reused encryption counters as accepted protocol limitations. No new actionable issue was found. Tests were inspected, not run.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Machine address] --> B{Protocol}
  B -->|SSH| C[gliff server]
  B -->|RDP| D[RDP client]
  B -->|VNC| E[VNC client]
  B -->|RustDesk| F[RustDesk client]
  D --> G[Shared TLS certificate checks]
  E --> G
  C --> H[Tab or screen-edge control]
  D --> H
  E --> H
  F --> H
Loading

Reviews (4) · Last reviewed commit: "Merge logging in to RustDesk's direct IP..." · Reviewed by Greptile

Comment thread crates/gliff-foreign/src/vnc/mod.rs Resolved
Comment thread crates/gliff/src/control.rs Resolved
Comment thread crates/gliff/src/recent.rs Resolved
Comment thread crates/gliff-foreign/src/vnc/mod.rs Resolved
Comment thread crates/gliff-foreign/src/lib.rs Resolved
- A machine whose certificate is saved logs in through VeNCrypt with TLS
  that shows it, or not at all: a server that now offers no security, the
  plain VNC password or VeNCrypt without TLS is refused, since it may be
  another machine posing as it. Without a saved certificate, VeNCrypt
  comes before no security where both are offered.
- A key goes up as the keysym it went down as, so Shift let go in between
  no longer leaves the shifted symbol held.
- vnc://host:0 is display 0, port 5900; port 0 stays refused otherwise.
@avillagran

Copy link
Copy Markdown
Author

All five findings are addressed: the saved-certificate bypass, held shifted keys and display 0 in 42106e3; the concurrent config saves and the certificate lost on stop in #131's 957b5dc, merged here in 661162f. scripts/check.sh passes (355 tests), with new tests for each.

Refusing a RustDesk machine that offers no key turned away every direct IP
connection: RustDesk's direct server never makes the key exchange (it
creates the connection with secure set to false) and opens with the login
challenge. Checked against RustDesk 1.5.1, which gliff could no longer
reach. Such a login goes ahead again, and the login dialog, the offer to
switch and docs/foreign.md now say plainly that RustDesk does not encrypt
direct IP connections, so it belongs on your own network or behind a VPN.
The password still never travels, only a salted hash.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant