This repository was archived by the owner on Nov 12, 2026. It is now read-only.
Repository navigation
feat(mac): the document as a text area VoiceOver and agents read and edit as text - #24
Merged
Merged
Conversation
… Editor::describe The editor draws its document as rows of runs, which Blitz's tree gives as static text, with no value, selection or caret. `Editor::describe` makes the editor's node (the element `TEXT_ID` names) an AccessKit `MultilineTextInput`: its value is the whole text, and its children are text runs, one per visual line (and at most 255 characters, chained on the line), with each character's length, position and width, where its words start, and the selection. The text is the text as shown in the rendered view, its rows of drawn glyphs, a line per row; in the source view it is the Markdown. A character is a grapheme cluster. Positions map to the source through the view's own (row, column) map, a range's end just past its last character, so selecting a bold word does not take its closing `**`. `Editor::accessibility_action` carries out what assistive tech asks of the text area or a run, through the document as typing does: SetTextSelection selects the range's source, ReplaceSelectedText types over the selection, and SetValue types the stretch that differs over it (in the source view the value is written exactly, by `replace_source`). The text and its runs are kept while the document, the caret and the rows' layouts are unchanged. Behavioural-change: the element the editor draws its rows in has the DOM id `leaf-text`.
…edit as text Every accessibility tree built for a window passes through a hook (blitz-shell's `View::set_accessibility_tree_hook`, from diaryx-org/blitz's branch `a11y-tree-hook` until diaryx-org/blitz#15 merges), which hands the editor's element to `Editor::describe` and puts its text runs in place of the nodes under it. The editor has the focus in the tree unless one of the page's controls has it. A request for the text area or one of its runs goes to `Editor::accessibility_action`; a press still becomes a click. Checked on the app with Cua Driver and the AX API, in a background window: AXValue, AXNumberOfCharacters, AXSelectedText(Range), AXInsertionPointLineNumber, AXRangeForLine, AXLineForIndex, AXStringForRange and AXBoundsForRange answer, a line a visual line; setting AXSelectedTextRange moves the drawn selection, and setting AXValue edits the document. Behavioural-change: the document is an AXTextArea whose value is its text, and setting its value or selected range edits and selects in it.
…e hook has merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Mac app's document reached the accessibility tree only as
AXStaticText: no value, selection, caret or insertion through AX, so an agent could type only with raw key events and could not check them. Now it is anAXTextArea.Uses diaryx-org/blitz#15 (merged into
leaf): a hook in blitz-shell through which the embedder changes each tree built.apps/leaf-dioxustakes Blitz frombranch = "leaf", locked at 9ee8db17.What it does
Editor::describe(crates/leaf-dioxus/src/access.rs) makes the editor's element (id="leaf-text") aMultilineTextInput: its value is the whole text, and its children are text runs, one per visual line (at most 255 characters, chained on the line), with character lengths (grapheme clusters), positions, widths, word starts, and the selection.**.Editor::accessibility_actioncarries outSetTextSelection,ReplaceSelectedTextandSetValuethrough the document as typing does (select_range+insert, the path that ends insplice_exact). A new value replaces only the stretch that differs; in the source view it is written exactly (replace_source).apps/leaf-dioxus/src/access.rs) installs the hook on every window, replaces the nodes under the editor with the runs, gives the editor the focus unless a page control has it, and routes requests for the text area or its runs to the editor.Checked on the app (background window, Cua Driver and the AX API)
AXTextArea; AXValue, AXNumberOfCharacters, AXSelectedText, AXSelectedTextRange, AXInsertionPointLineNumber, AXRangeForLine, AXLineForIndex, AXStringForRange and AXBoundsForRange all answer. Each soft-wrapped visual line is its own line. Per-character bounds are exact (bold→ 33.7 pt wide).set_valueon the text area's element token edited the document, and reading the value back showed the edit. Toolbar presses still work.Tests
12 unit tests in
crates/leaf-dioxus/src/access.rs: shown and source text, grapheme clusters, selection mapping, caret at a line break, soft wraps and long lines, select, replace and set-value edits, and an empty document.ship ci dioxusandship ci versionspass.