Skip to content

Row-change tracking (ContentVersion, CopyChangedRows) and Terminal.EncodeKey for headless hosts - #193

Merged
tomlm merged 3 commits into
tomlm:mainfrom
confetty-sh:stack/xterm-net-row-claude
Nov 9, 2026
Merged

tomlm merged 3 commits into
tomlm:mainfrom
confetty-sh:stack/xterm-net-row-claude

Conversation

@JohnCampionJr

@JohnCampionJr JohnCampionJr commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

Why

A headless host is any program that drives a Terminal without a renderer: a test harness, a recorder, or an agent that runs a TUI and reads its screen back as data. Two things are missing for one, and today every such host writes its own copy of them:

  1. Which rows changed. There is no dirty-row or version API, so after every Write a host has to diff every cell of every row to find out what moved.
  2. Which key encoder to use. Choosing between Win32 input records, the Kitty protocol and the legacy encodings (honouring DECCKM, DECKPAM and the Kitty flags) lives in the Avalonia TerminalView, so any other host has to copy that logic.

The use case behind this PR is an agent driving a terminal session: it sends keys, waits for the screen to settle, and asks for just the rows that changed since its last look.

API

Row-change tracking (Terminal.RowChanges.cs, Buffer/BufferLine.Version.cs)

long Terminal.ContentVersion { get; }                         // the version as of the last look; a pure read
long Terminal.RefreshContentVersion()                         // looks at the rows, returns the up-to-date version
long Terminal.GetRowVersion(int row)                          // the version a viewport row last changed at
int  Terminal.CopyChangedRows(long sinceVersion, Span<bool> changed)  // marks the rows changed since then, returns the count
long BufferLine.Version { get; }                              // per-line write counter
  • What counts as a change: a visible row's content (cells, attributes, line attribute, images, sized text), and which buffer line the row shows. That second part covers scrolls (every row shifts), viewport scrolling, resize and the alternate-screen switch, even though none of them write to the lines they move. Writes, erases (EL/ECH/ED), clears and attribute-only rewrites all count. A write that puts back the same content still counts, because nothing compares cells.
  • What doesn't count: cursor moves, which change no row (a host reads Buffer.X/Y and CursorVisible), and screen-wide state like the palette, ReverseVideo and the title. A bare SGR or an empty Write doesn't move the version either.
  • How it works: each place in BufferLine that already cleared Cache (the invalidation the renderer depends on) now calls MarkChanged(), which clears Cache and increments _version. So the two can't disagree, and the only cost on the print path is one increment. The comparison happens when a host asks: each call looks at the visible rows once, comparing each row's (line, version) with what the previous call saw, and gives the rows that differ a new ContentVersion. Asking costs O(rows), never O(cells), and nothing allocates. The per-row versions are stored on the terminal, so any number of observers can each keep their own last-seen version.
  • Threads: RefreshContentVersion, CopyChangedRows and GetRowVersion follow the same contract as Write: call them on the thread that writes, or serialized with it. ContentVersion reads one field and writes nothing, so it's safe from any thread. What it reports is the last look, and it's 0 before the first one.
  • The usual loop: CopyChangedRows(since, rows); since = ContentVersion;. CopyChangedRows looks first, so the loop needs no separate refresh.

ColorSnapshot.ResolveUnderline (Common/ColorPalette.cs)

int? ColorSnapshot.ResolveUnderline(in AttributeData attributes)

This is the underline counterpart of ResolveBackground. SGR 58;5;n resolves through the palette, and 58;2 RGB comes through as named. Bold changes neither. It returns null when the underline should follow the text colour: SGR 59, no SGR 58, or an index past the palette. The host then draws it in the colour it used for the text, after its own inverse and dim.

Terminal.EncodeKey (Terminal.KeyEncoding.cs)

string? Terminal.EncodeKey(KeyEvent ev, KittyKeyboardEventType eventType = KittyKeyboardEventType.Press)
  • Precedence: the same order TerminalView uses. Win32InputMode comes first (it's a transport, not a competing encoding), then KittyKeyboardActive through GenerateKittyKeyInput, then the legacy generators, which honour DECCKM and DECKPAM.
  • Key model: the browser-style KeyEvent the Kitty path already uses. Key is a functional-key name ("ArrowUp", "F5", "Shift") or the typed text. Code is the physical key, and it's the only thing that separates the keypad from the main keys. KeyCode, when set, is taken as the Windows virtual-key code.
  • Modifiers: Shift, Alt and Ctrl map to every protocol. MetaKey is Super: Kitty encodes it (bit 8), while legacy and Win32 can't, so they drop it. That's why KeyModifiers doesn't need a Super member. AltGr should be reported as the composed character with Ctrl and Alt clear.
  • Returns null when nothing should be sent: KAM is set, a bare modifier outside Win32/Kitty, a release that legacy can't express, or an unknown key.
  • Left to the host: key rewriting (Alt+Left → ESC b, Cmd shortcuts, selection edits), Win32 lock-key state (the event doesn't carry it), and ConPTY's bare-Escape-as-a-Win32-record rule, which is a fact about that transport rather than the protocol.
  • One deliberate difference from the control: the keypad (by Code) goes through Keypad*, so DECKPAM is honoured. With DECKPAM off the output is the same as today, the literal character. F13–F20 are covered in legacy mode as well.

Performance

MarkChanged() replaces Cache = null in the same places, so the print path gains one field increment per line write and nothing else: no branch, no allocation. I measured it with the bench's own ci and compare modes, two runs of each side alternated on one machine (arm64 macOS, 60M chars per corpus). Allocation is unchanged, and every timing difference is inside the noise for its corpus:

corpus ns/char base → head Δ noise
scroll-ascii 1.71 → 1.72 +0.6% ±3%
sgr-churn 4.42 → 4.44 +0.3% ±9%
truecolor 4.65 → 4.35 -6.4% ±10%
alt-redraw 5.73 → 5.60 -2.3% ±0%
unicode 13.87 → 13.77 -0.7% ±2%
flood 43.48 → 42.70 -1.8% ±4%

CI's perf gate is the real check.

Tests

  • RowChangeTrackingTests (22):
    • the version moves on print, wrap, attribute-only rewrite, EL, ECH, ED 2, a line feed that scrolls (including with scrollback 0, where the top line is recycled), SU, viewport scroll, resize and the alt-screen switch in both directions;
    • it doesn't move on cursor-only sequences, empty writes or a bare SGR;
    • two observers each see their own changes;
    • OSC 66 sized text marks every row it covers when written and when erased, including from an anchor above the viewport;
    • a zero-row terminal can be observed before layout;
    • ContentVersion is a pure read of the last look, and the usual loop works without a refresh.
  • EncodeKeyTests (73): a table of expected bytes per mode (legacy, DECCKM, DECKPAM, Kitty disambiguate, Kitty with event types, Win32, Win32+Kitty), written as literals that match what TerminalView sends today, plus releases, repeats, KeyCode and KAM. The Win32 rows include NumLock-off keypad navigation and shifted OEM keys.
  • ColorPaletteTests: ResolveUnderline for every SGR 58 encoding (plus 59 and none), the live palette, and an index past it.
  • Mutation check: 25 mutants, all 25 killed. Each one removes the version bump, ignores line identity, ignores resize, reorders Kitty ahead of Win32, encodes legacy releases, drops the Win32 Ctrl fold, the enhanced-key flag, the keypad mapping, the KAM check, or right-Alt handedness; or, after review, drops the sized-text reach or the old reach, stops watching above the viewport, always uses VK_NUMPADn, or drops the OEM code or US-layout fallback; or makes the getter compare again, stops the refresh from looking, or breaks the underline's RGB, palette, default or bounds rules.
  • The whole XTerm.NET.Tests project: 2642 passed, 1 skipped, 0 failed.

🤖 Generated with Claude Code

@JohnCampionJr JohnCampionJr left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-merge review (Codex), for the PR author

Reviewed 9c47782. Three should-fix findings, detailed inline: shifted OEM keys fall out of Win32 encoding; NumLock-off keypad navigation gets the numeric VK; tall OSC 66 changes omit the other visible rows they cover. No blocking findings or nits.

Checked cache invalidation replacements, line identity/version tracking across scroll/resize/alternate screen, independent observers, print-path work, and mode selection against the actual control. The deliberate DECKPAM difference is accounted for. Baseline: 335 focused tests passed; PR: 413 existing/new tests passed. Four additional reproductions fail as described (two cover the sized-text finding); both Win32 mismatches also fail comparisons against TerminalView's actual encoder. Current CI including the perf gate is green. No separate local benchmark was run.

Comment thread src/XTerm.NET/Terminal.KeyEncoding.cs Resolved
Comment thread src/XTerm.NET/Terminal.KeyEncoding.cs Resolved
Comment thread src/XTerm.NET/Terminal.RowChanges.cs Resolved
@JohnCampionJr
JohnCampionJr force-pushed the stack/xterm-net-row-claude branch from 9c47782 to 85f7c9b Compare November 6, 2026 03:06

@JohnCampionJr JohnCampionJr left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-merge review follow-up (Codex), for the PR author

Rechecked 85f7c9b: all three original findings are resolved. The four unchanged core reproductions and both comparisons against the actual control now pass, as do all 434 filtered core checks. The new tests cover shifted OEM release/Win32 precedence, NumLock-off navigation and old/new tall-run coverage including anchors above the viewport. The documented non-enhanced keypad navigation flag is appropriate.

One new should-fix regression in the sized-row delta is detailed inline: first observation of a terminal constructed with Rows = 0 throws IndexOutOfRangeException. My focused test fails on this head and passes with the previous row tracker. Please initialize the watched arrays independently of the visible row count. No other new findings in the delta. Original threads left for the author/maintainer to resolve.

Comment thread src/XTerm.NET/Terminal.RowChanges.cs Resolved
@JohnCampionJr
JohnCampionJr force-pushed the stack/xterm-net-row-claude branch from 85f7c9b to 6aa52d4 Compare November 6, 2026 03:10

@JohnCampionJr JohnCampionJr left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-merge review follow-up (Codex), for the PR author

Rechecked 6aa52d4 (stack fix ec4bffc): the zero-row finding is resolved. My unchanged construction → ContentVersion → empty CopyChangedRows → resize-to-five reproduction passes. The new author regression also confirms GetRowVersion(0) throws ArgumentOutOfRangeException at zero rows and that all five rows are reported after resize.

All 101 filtered row-tracking/key-encoding checks pass, including the retained original reproductions. Keying initialization on the watched-array size fixes first observation without changing the normal resize behavior. No new findings in this delta. Thread resolution left to the author/maintainer.

@JohnCampionJr
JohnCampionJr force-pushed the stack/xterm-net-row-claude branch from 6aa52d4 to cecb21d Compare November 6, 2026 03:23

@JohnCampionJr JohnCampionJr left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-merge review follow-up (Codex), for the PR author

Reviewed cecb21d (stack 7aff34e). No findings in this delta.

ContentVersion is a pure Volatile.Read of the cached counter; it neither inspects the buffer nor updates tracking state. RefreshContentVersion performs the comparison explicitly. CopyChangedRows and GetRowVersion still refresh themselves, and their docs describe that behavior. Independent observers retain their own last-seen versions without consuming each other's changes.

ResolveUnderline preserves the null foreground-fallback sentinel for default/reset, resolves indexed colors through the snapshot palette, and distinguishes RGB (including black and numerically-small values) from indices. Explicit underline color stays independent of inverse/bold; null lets the host use its final inverse/dim foreground, as documented. Out-of-range and negative direct-set indices safely fall back.

Baseline color/attribute checks: 115 passed. Current focused color/row/attribute suite, including 13 additional observer and inverse/underline checks: 166 passed. No public test or runtime limitation encountered for these cases.

@JohnCampionJr
JohnCampionJr force-pushed the stack/xterm-net-row-claude branch from 3b84116 to 234e45b Compare November 9, 2026 16:00
@tomlm
tomlm merged commit b7a711f into tomlm:main Nov 9, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants