Skip to content

M9: Docs rewrite, release automation, v0.2.0 version bump #23

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
whoisaldo merged 4 commits into from
Aug 26, 2026
Merged

M9: Docs rewrite, release automation, v0.2.0 version bump #23

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
0821255
Docs and release automation for v0.2.0
whoisaldo Aug 26, 2026
8d993a7
Docs prose pass
whoisaldo Aug 26, 2026
e388a03
Merge main (M8 polish) into the docs branch
whoisaldo Aug 26, 2026
b1c323d
Version 0.2.0 (host + wire crate)
whoisaldo Aug 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 116 additions & 0 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
name: Release

# Tagging vX.Y.Z builds the Windows installer and publishes a GitHub release
# with its SHA-256 in the body (docs/download.html reads both from the
# releases API). workflow_dispatch runs the same build WITHOUT publishing —
# the dry-run mode for validating this pipeline.
on:
push:
tags: ["v*"]
workflow_dispatch:

permissions:
contents: write

env:
CARGO_TERM_COLOR: always
# The bundled Virtual Display Driver (VirtualDrivers/Virtual-Display-Driver,
# MIT). Pinned: the installer's scheduled tasks and lifecycle handling are
# verified against this exact version.
VDD_RELEASE_TAG: "25.5.2"
VDD_ASSET: "Virtual.Display.Driver-v25.05.03-setup-x64.exe"

jobs:
windows-installer:
name: Windows installer
runs-on: windows-2022
timeout-minutes: 60
env:
LIBCLANG_PATH: C:\Program Files\LLVM\bin
steps:
- uses: actions/checkout@v4

- name: Cache FFmpeg 7.1.1 shared SDK
id: ffmpeg-cache
uses: actions/cache@v4
with:
path: C:\ffmpeg
key: ffmpeg-7.1.1-full_build-shared

- name: Download FFmpeg 7.1.1 shared SDK
if: steps.ffmpeg-cache.outputs.cache-hit != 'true'
shell: pwsh
run: |
$url = "https://github.com/GyanD/codexffmpeg/releases/download/7.1.1/ffmpeg-7.1.1-full_build-shared.7z"
Invoke-WebRequest -Uri $url -OutFile ffmpeg.7z
7z x ffmpeg.7z -oC:\ffmpeg-extract | Out-Null
Move-Item C:\ffmpeg-extract\ffmpeg-7.1.1-full_build-shared C:\ffmpeg

- name: Configure FFmpeg environment
shell: pwsh
run: |
"FFMPEG_DIR=C:\ffmpeg" | Out-File -FilePath $env:GITHUB_ENV -Append
"C:\ffmpeg\bin" | Out-File -FilePath $env:GITHUB_PATH -Append

- uses: Swatinem/rust-cache@v2

- name: Download the bundled virtual display driver
shell: pwsh
run: |
$dest = "installer\vendor\vdd"
$url = "https://github.com/VirtualDrivers/Virtual-Display-Driver/releases/download/$env:VDD_RELEASE_TAG/$env:VDD_ASSET"
Invoke-WebRequest -Uri $url -OutFile (Join-Path $dest $env:VDD_ASSET)
# Its MIT license text, redistributed alongside the binary.
Invoke-WebRequest -Uri "https://raw.githubusercontent.com/VirtualDrivers/Virtual-Display-Driver/$env:VDD_RELEASE_TAG/LICENSE" `
-OutFile (Join-Path $dest "LICENSE-VirtualDisplayDriver.txt")

- name: Verify the driver's Authenticode signature
shell: pwsh
run: |
$sig = Get-AuthenticodeSignature "installer\vendor\vdd\$env:VDD_ASSET"
Write-Host "Signature status: $($sig.Status) Signer: $($sig.SignerCertificate.Subject)"
if ($sig.Status -ne "Valid") {
throw "Driver signature is '$($sig.Status)' — refusing to release an unverified driver."
}

- name: Install Inno Setup
shell: pwsh
run: choco install innosetup -y --no-progress

- name: Build the installer (strict driver signature)
shell: pwsh
run: .\scripts\build-installer.ps1 -StrictSignature

- name: Compute SHA-256
id: hash
shell: pwsh
run: |
$hash = (Get-FileHash "build\out\EternalMonitor-Setup.exe" -Algorithm SHA256).Hash
"sha256=$hash" | Out-File -FilePath $env:GITHUB_OUTPUT -Append
Write-Host "SHA256: $hash"

- name: Upload installer artifact (dry runs and debugging)
uses: actions/upload-artifact@v4
with:
name: EternalMonitor-Setup
path: build\out\EternalMonitor-Setup.exe

- name: Publish GitHub release
if: startsWith(github.ref, 'refs/tags/v')
shell: pwsh
env:
GH_TOKEN: ${{ github.token }}
run: |
$tag = $env:GITHUB_REF_NAME
$sha = "${{ steps.hash.outputs.sha256 }}"
$lines = @(
"EternalMonitor $tag — see [RELEASE_NOTES.md](https://github.com/whoisaldo/EternalMonitor/blob/main/RELEASE_NOTES.md) for what's new.",
"",
"**EternalMonitor-Setup.exe SHA-256:**",
'```',
$sha,
'```'
)
gh release create $tag "build\out\EternalMonitor-Setup.exe" `
--title "EternalMonitor $tag" `
--notes ($lines -join "`n")
262 changes: 129 additions & 133 deletions ARCHITECTURE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,138 +1,134 @@
# EternalMonitor — Architecture
# EternalMonitor architecture

## Current pipeline
Accurate as of v0.2.0 (protocol v2).

## The pipeline

```text
[Windows desktop]
|
v
+---------------------------------------------+
| host/ (Rust Windows app) |
| |
| DXGI Desktop Duplication |
| -> primary display capture |
| -> CPU-readable BGRA frame |
| |
| ffmpeg-next + NVENC |
| -> BGRA to YUV420P |
| -> H.264 encode |
| |
| UDP transport |
| -> FlatBuffer FramePacket |
| -> custom fragmentation header |
| -> mDNS advertisement |
+---------------------------------------------+
|
v
+---------------------------------------------+
| ios/ (Swift iPad app) |
| |
| Connect UI |
| -> manual IP entry |
| -> Bonjour scan attempt |
| |
| UDP receiver |
| -> fragment reassembly |
| -> FramePacket parse |
| |
| VideoToolbox |
| -> H.264 decode |
| |
| Metal MTKView |
| -> render latest decoded frame |
+---------------------------------------------+
[Windows desktop] [iPad]
| ^
v | Metal (NV12 + BT.601/709 shader,
+-------------------------- host/ ---------------|-- aspect-fit, draw-on-demand)
| capture thread encode thread | VideoToolbox (H.264/HEVC, hw on
| DXGI duplication --> BGRA->YUV420P --> | device, sw in the simulator)
| (or synthetic) slot swscale, then | FrameAssembler (per-frame
| cursor composite NVENC/AMF/QSV/x264/ | fragment reassembly, caps)
| x265/VideoToolbox | UDPReceiver (ephemeral port,
| | | media/control demux)
| v channel |
| transport task (tokio) -----+--> media datagrams (UDP)
| v2 fragmentation, pacing, <--- control datagrams (same socket)
| session, heartbeats, ABR |
+-------------------------------------------------+
supervised by supervisor.rs (health, watchdogs, backoff restarts)
```

## What is implemented

### Capture

- API: DXGI Desktop Duplication
- Current behavior: enumerates all adapters/outputs and duplicates a **selectable** output
(primary by default), copying it into a CPU-readable staging texture
- The Settings tab exposes a capture-display picker; choosing a virtual output created by
an Indirect Display Driver turns the iPad into an extended desktop instead of a mirror.
The capture adapter follows the chosen output; encoder selection stays vendor-based.
- The managed virtual display is brought up **on demand, only once an iPad has connected**, and
is disabled on exit / target change / startup (and via a panic hook), so it never lingers as a
phantom monitor. If the captured display is idle, the loop resends the last frame so the iPad
still receives a startup keyframe.
- Output format passed downstream: BGRA frame buffer plus frame metadata

This is functional but not yet the final zero-copy path described in earlier docs.

### Encode

- Crate: `ffmpeg-next`
- Codec path: `h264_nvenc`
- Current codec settings:
- H.264
- `baseline` profile
- `gop=30`
- `max_b_frames=0`
- `zerolatency=1`
- `rc=cbr`

The encoder emits Annex B H.264 byte streams that are wrapped in `FramePacket`.

### Transport

- Current transport: WiFi/local-network UDP only
- Registration handshake: iPad sends `ETERNALHELLO` plus its listen port
- Packetization:
- each encoded frame becomes one FlatBuffer `FramePacket`
- payload is fragmented into MTU-sized UDP datagrams
- fragment header is currently `16` bytes
- fragment index and fragment count are `u16`
- the header's final 4 bytes carry a per-pipeline-run `stream_epoch`, so the receiver drops
stale reassembly state immediately on a stream restart (seq reset) instead of inferring it
from a sequence gap. These bytes were previously reserved/zero, so older receivers that
ignore them stay wire-compatible.

The `u16` fragment-count change is important. Older host and iPad builds are not wire-compatible with the current transport fix.

### Discovery

- Host side: advertises `_eternaldisplay._udp.local.` over mDNS/DNS-SD
- iPad side: scans via `NetServiceBrowser`

This exists in code, but it is not reliable enough to treat as finished. Direct IP connect is the known-good path.

### iPad receive/decode/render

- UDP datagrams are received with `NWConnection`
- Fragments are reassembled by sequence number
- `FramePacket` is parsed manually from FlatBuffers
- H.264 is decoded with `VTDecompressionSession`
- Frames are rendered with `MTKView`

The renderer currently keeps the latest available decoded texture and draws that.

## Protocol

Implemented message shape in active use:

- `FramePacket`
- `seq`
- `timestamp_us`
- `data`
- `width`
- `height`
- `is_keyframe`

Not all planned protocol families are implemented yet. `InputEvent`, richer control messages, and display configuration exchange are still roadmap items.

## Known-good state

- Working end-to-end UDP stream: commit `bc44770`
- Manual IP connect works
- Network scan/discovery may still fail even when streaming works

## Not implemented yet

- First-party signed display driver (the host currently drives a third-party signed
virtual display driver; a first-party in-tree IDD is a v0.2.0 goal)
- USB transport
- Reverse input channel
- Reliability controls such as selective NACK
- Dynamic transport switching
Host stages are dedicated OS threads. Capture hands frames to the encoder
through a latest-wins slot: an unconsumed frame gets displaced, so the
encoder always works on the freshest picture. The encoder hands access units
to transport through a lossless channel, because dropping an encoded P-frame
would corrupt the GOP. Frame pixels travel in `Arc<Vec<u8>>` buffers that
get recycled; steady state does one full-frame copy, the DXGI staging
readback.

The supervisor owns the pipeline. Stage threads report their exits, wedge
watchdogs fire on silent stalls (loop heartbeat stale 3 s, no frame for 5 s
with a client connected, encoder flat 3 s), and restarts back off
exponentially with a restart-storm brake. The client session lives outside
the pipeline, so a crash-restart resumes streaming to the same session. The
client sees a new stream epoch, resets reassembly, and never re-handshakes.

## Wire protocol v2

One UDP socket carries both media and control. Every datagram starts with an
8-byte prefix: magic `"EM"`, version `2`, packet type, flags (media bit 0 =
keyframe), a reserved byte, and a strict payload length. Legacy v1 datagrams
began with `"ET"`, so the two never get confused. v2 is otherwise a clean
break; each side tells the user to update the other on contact with v1.

Media (type 0x01) is a 32-byte header followed by a raw Annex B chunk. The
header carries session id, stream epoch, frame sequence, fragment
index/count (up to 3066, about 4 MiB per frame), and the capture timestamp
in microseconds on the host process clock. There is no serialization
framework; width, height, and codec travel in the control plane.

Control messages share a 16-byte header (session id, message sequence,
type):

| Message | Direction | Purpose |
| --- | --- | --- |
| HELLO2 / HELLO_ACK | C→H / H→C | Session establishment. Capability bits (H.264/HEVC decode, wants-input), screen size and refresh, a nonce-idempotent ACK carrying the session id, host-dictated timing, and the stream config |
| HEARTBEAT | H→C | 1 Hz liveness plus the embedded stream config, which self-heals lost config changes |
| RECEIVER_REPORT | C→H | Every 500 ms: loss, completion, jitter, queue depths. Feeds ABR and doubles as client liveness |
| KEYFRAME_REQUEST | C→H | Recovery after loss or a decode error; the host rate-limits to one per 500 ms |
| PING / PONG | C→H→C | NTP-style clock sync (min-RTT offset) behind the HUD's measured end-to-end latency |
| STREAM_CONFIG | H→C | Immediate notify on a bitrate, codec, or resolution change |
| INPUT_EVENT | C→H | Input relay (below) |
| BYE | both | Clean teardown with a reason (user, backgrounded, shutdown) |

Session rules, implemented as a pure state machine on the host: one client
at a time, and a second device gets `busy`. The same device reconnecting
supersedes in place with a fresh session id. Duplicate HELLO2 nonces get an
identical ACK, which makes handshake retransmits harmless. Liveness expires
3 s after the last report or input event, and expiry also tears down the
virtual display.

## Reliability

- ABR: a bitrate ladder (4 to 20 Mbps, capped by the GUI "Max bitrate"
slider) driven by receiver reports. Loss or keyframe-request pressure
steps down with a 3 s cooldown; 15 s of clean reports steps back up. A
bitrate change reopens the encoder session (50 to 200 ms, same epoch) and
forces an IDR.
- Pacing: keyframe bursts go out in batches with microsleeps under a hard
3 ms wall-clock budget, on a socket with a 4 MiB kernel send buffer. This
smooths WiFi loss without adding latency.
- Recovery: the client requests keyframes on gaps and decode errors; the
host honors at most two per second. The app shows "SIGNAL LOST" after 3 s
of silence and reconnects with backoff.

## Input relay

The iPad normalizes touches over the displayed video (letterbox-corrected)
to a 0–65535 grid. A pure gesture machine turns raw touches into events: tap
means click at the touch-down point, a drag commits after slop, two fingers
scroll in the direct-manipulation direction, a 500 ms hold right-clicks, and
Apple Pencil presses immediately with pressure. Press/release edges are sent
twice with one event id; the host dedupes, maps through the captured
output's desktop rectangle onto the Windows virtual screen, and injects with
`SendInput`. Sessions that didn't set the wants-input capability never get
injected for.

## Codecs

H.264 is the default everywhere. With the host's "Prefer HEVC" setting on
and a client that advertises HEVC decode, the encoder live-switches to the
HEVC sibling (NVENC/AMF/QSV/VideoToolbox/x265) through the same reopen
mechanism. The iPad decoder detects the codec from the bitstream itself (an
HEVC VPS in a keyframe), so the switch has no ordering race between config
and media. HEVC encoders repeat VPS/SPS/PPS in-band; H.264 keyframes get
cached SPS/PPS prepended. The AMF path carries extra guards developed
against real hardware.

## Discovery

The host advertises `_eternaldisplay._udp` over mDNS with `version`,
`proto=2`, and `platform` TXT records. The advertisement re-upserts every
60 s without an unregister gap and goes out with goodbye packets on exit.
Manual IP entry and the QR code remain the fallback for networks that
filter multicast.

## Portability and testing

Everything protocol- or logic-shaped lives in the pure `eternal-wire` crate
(v2 codecs, H.264/HEVC bitstream helpers, reassembly) or in host modules
with injected clocks (session, ABR, pacer, supervisor, input mapping), all
tested on every platform. Windows-only code (DXGI, SendInput, VDD control)
sits behind `cfg(windows)`. A synthetic capture source with a
machine-readable frame counter lets the full host run on macOS and CI,
where the Rust E2E suite drives a fake receiver through handshake, loss,
crash-recovery, input, and HEVC scenarios, and `scripts/e2e_ios.sh` streams
into the real iPad app in the simulator. Golden wire vectors are parsed
byte-for-byte by both languages. What only hardware can prove (GPU
encoders, the display driver, real WiFi) is enumerated in
HARDWARE_VERIFICATION.md.
2 changes: 1 addition & 1 deletion Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading