Skip to content

feat(navigator): add keyboard hints, semantic actions and decision gateway - #1058

Draft
Phil-Fan wants to merge 18 commits into
mainfrom
feat/1054-e123
Draft

Phil-Fan wants to merge 18 commits into
mainfrom
feat/1054-e123

Conversation

@Phil-Fan

Copy link
Copy Markdown
Contributor

摘要

按 RFC #1054 落地 Navigator 的键盘导航、语义动作与受控决策切片,并吸收 #1038 的 C1/C2 验收要求:

  • P0/P1:统一 keyboard ownership,PDF Vim 位移键,全局 F 元素提示。
  • P2:paper / file / pdf-anchor / selection 语义目标,复用现有 Command Registry 的动作适配器,执行前重新校验目标与上下文。
  • P3:受限 pdf.selection.intent Decision Gateway,规则优先、System One 兜底,模型只能返回白名单内动作。
  • P4(部分):规则型 Suggestion Coordinator,处理去重、TTL、冷却、dismiss / snooze;提示层与接受后的 UI 执行闭环暂缓,不产生 Toast 或阻塞弹窗。
  • 附带一个独立的导入修复:arXiv e-print 返回 404/410 时记为终态 unavailable,不再按网络错误循环重试,也不阻断本地 PDF 正文解析。

Refs #1054、Refs #1038。

类型

  • feat:新功能
  • fix:修复
  • ui:界面
  • refactor
  • docs
  • chore / ci / test

场景

键盘归属与输入保护

  • src/lib/keyboard/ownership.ts 与 src/lib/pdf/keyboard-ownership.ts 统一判断当前键盘归属:最上层 modal、输入框 / 编辑器 / IME composing、PDF 当前 active surface 与 split hover,避免 Hint 和 PDF Vim 键抢走输入。
  • PDF 箭头键与 Vim 键共享同一套归属规则,现有方向键行为保持不变。

PDF 位移键

  • 新增 j / k / d / u / h / l / J / K / gg / {n}G / n / N,复用现有 PDF 滚动、翻页与搜索能力,不另造滚动实现。
  • PdfVimKeyHandler 同时接入 PDF reader 与翻译 PDF reader;gg、{n}G 由可单测的 key sequence parser 处理。
  • 设置中新增 pdfVimKeys 开关,持久化并做默认值归一化。

全局 F Hint

  • HintModeProvider 挂载到主窗口、设置窗口与各 popout window root;F 只作用于当前 webview/document。
  • 显式 data-hint 优先,并覆盖 button / tab / menuitem / treeitem / option / link 等通用目标;论文行、文件行、标签页由容器接管子树,优先标出论文名 / 文件名。
  • 标签使用主键区小写字母,f 保留给进入 / 退出;支持前缀输入、Backspace、Esc,标签输入大小写不敏感,非拉丁布局回退物理 KeyA–KeyZ。
  • 激活目标时依次派发 pointerdown / mousedown / pointerup / mouseup / click,修复 Radix / Base UI 只在 pointerdown 打开的下拉与菜单。
  • 列表上的 j / k / h / l:打开的下拉映射为上下方向键,聚焦树映射为上下左右;输入框、编辑器与带 Shift 的组合不生效。

语义目标与动作

  • SemanticTarget 使用 discriminated union,区分 paper、file、pdf-anchor、selection;UIHintTarget 仅限当前窗口、仅存内存,不序列化为语义目标。
  • registerNavigatorAction / executeNavigatorAction 复用 Command Registry 中的业务 handler;执行前检查 target 支持、可用性、context fingerprint 与 resource revision,失败返回明确的 unsupported-target / unavailable / stale-context / unregistered-action。
  • 首批动作覆盖论文打开 / 引用,以及 PDF 选区翻译 / 交给 Agent / 高亮。

Decision Gateway

  • 新增 decision_gateway_decide IPC:固定 decisionId=pdf.selection.intent、schema version、requestId、target/context identity 与 allowedActionIds。
  • Host 侧把 allowedActionIds 收窄到已注册的 translate / explain / highlight / add-to-agent,模型只能返回其中之一;返回值会再次按白名单过滤。
  • 规则优先、System One 兜底;8 秒超时、并发信号量、错误归一,失败时返回无建议而不是阻塞交互。
  • 前端 requestPdfSelectionIntent 在非 Tauri 环境返回安全空建议。

规则型建议

  • 订阅 paper:assets-ready 与 job:completed,生成低敏的内存建议;按 workflowId + target + action + resourceRevision 去重,带 TTL、冷却、dismiss / snooze 与执行状态。
  • 目前没有提示 UI,也不会自动执行动作;这是 P4 的下一步,不在本 PR 内扩大范围。

设置与文档

  • 新增 pdfVimKeys、hintModeEnabled 设置与键盘设置页开关,en / zh-CN 文案同步。
  • 更新 docs/development/rfc-keyboard-navigation-layer.md 的落地状态、过滤规则、标签生成和窗口挂载要求;更新 docs/backend/paper-import.md 的 arXiv 源码不可得语义。

验证

  • pnpm typecheck 通过。
  • pnpm vitest run test/keyboard-navigation.test.ts test/file-tree-keyboard.test.ts test/navigator-actions.test.ts test/navigator-suggestions.test.ts test/decision-gateway.test.ts:5 个文件、20 个测试通过。
  • pnpm biome check(本次改动文件)通过。
  • pnpm deps:check 通过(本机默认 Node 25 不受 dependency-cruiser 支持,使用 Node 22 运行)。
  • cargo test -p agentero export_typescript_bindings 通过,src/lib/core/bindings.ts 与 Rust 契约一致。
  • cargo test -p agentero app::events_contract:9 个测试通过。

未覆盖:桌面端手动回归与 UI 录屏尚未执行,见下方待补项。

截图(UI / 功能改动必填)

文本对比:

场景 之前 之后
键盘导航 主要依赖鼠标 / 命令面板 F 显示元素标签,输入标签直接激活;PDF 支持 Vim 位移键
下拉与菜单 键盘聚焦后不一定展开 Hint 激活派发完整指针序列,Radix / Base UI 菜单可打开
列表与文件树 无 j / k / h / l 打开的下拉与聚焦树支持 Vim 方向键
论文行提示 通用按钮各自出标签 论文行整体接管,优先标出论文名
设置 无相关开关 键盘设置页可分别关闭 PDF Vim 键与 F Hint

截图 / GIF 待补充:当前执行环境没有桌面显示,无法录制。合并前需在 pnpm tauri dev 中补「之前 / 之后」截图,并在窄窗口下补一张布局对比。

Demo(feat 必填)

10–60 秒演示待补充,建议路径:打开 PDF → j / k / gg / {n}G → F 进入 Hint → 标签打开论文 / 菜单 → 设置中切换开关。

自查清单

  • 本地 pre-commit 通过(未使用 --no-verify)
  • 已运行与改动对应的检查(命令与结果见「验证」)
  • CI 全绿
  • UI 文案已走 t() 并同步 zh-CN
  • 相关文档已更新
  • 手动验证:待补桌面端实际交互截图与录屏

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Cursor Bugbot was not present after the first check poll, so that automated-review signal was skipped. No applicable approval policy required human review, and this automation had no prior approval to re-evaluate.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Router and Approver

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Deploying agentero-doc with  Cloudflare Pages  Cloudflare Pages

Latest commit: f2546df
Status:🚫  Build failed.

View logs

@Phil-Fan

Copy link
Copy Markdown
Contributor Author

F and jk 操作

UU.mp4

@Phil-Fan
Phil-Fan marked this pull request as draft November 10, 2026 15:31
@Phil-Fan

Copy link
Copy Markdown
Contributor Author
UU.mp4

d/u/gg/G 操作

Shift arrives as its own keydown before Shift+A, and treating that
modifier press as unknown input closed the whole hint session. Ignore
modifier-only and IME keys, resolve label characters case-insensitively
(with a physical KeyA-KeyZ fallback), and render badges in the case the
user actually types so Shift is never implied.

Refs #1054
Library rows carried no hint target, so F filled its 60-slot budget
with per-cell copy/tag buttons and skipped most paper names. Mark each
row as a priority-1 hint (one badge per visible paper), let an explicit
data-hint container own its subtree so nested icon buttons collapse into
it, and pick up menuitem/option roles inside floating popups.

Floating cards that declare aria-modal="false" no longer become the
only scan scope, so hovering a preview keeps the rest of the window
hintable.

Refs #1054
Hint activation only called element.click(), so Radix / Base UI menus and
selects that open on pointerdown (Agent switcher, title-bar settings gear,
layout menu) just took focus and never opened. Dispatch the pointer ->
mouse -> click sequence a real pointer produces, which keeps click-only
triggers working.

Refs #1054
Hint mode listens on the window of the document it is mounted in, and
only the main window (App.tsx) mounted it. Settings, feature and doc
windows are separate webviews, so F did nothing there. Wrap those roots
and note the per-window requirement at the root dispatch.

Refs #1054
Dropdowns already implement arrow navigation, so j/k re-dispatch
ArrowDown/ArrowUp to the focused menu/listbox (Radix menus, selects,
context menus) instead of reimplementing focus movement.

The file tree had no arrow navigation at all. Add the WAI-ARIA tree
behaviour (up/down move focus + selection, right expands or enters the
first child, left collapses or steps to the parent) and map j/k/h/l onto
it while a tree row is focused. h/l stay inert inside menus.

Refs #1054
Single-key navigation had no off switch: j/k/h/l moved the file tree and
dropdowns unconditionally. Add vimMode (default on) under General →
Keyboard as the master switch. It gates list navigation and the PDF Vim
keys; the PDF-specific toggle stays but is disabled and explains itself
while Vim mode is off.

Refs #1054
…ture

The pane now renders the element hint switch from AppSettings, so the
fixture must provide it; cover the toggle so the setting stays wired.
The viewer now mounts PdfVimKeyHandler, which reads the settings store and
EmbedPDF capabilities, so the partial react-i18next and scroll mocks must
cover them.
The Vim mode switch snaps back on when the Host binary predates the
field: settingsSet drops vimMode, the echoed settings normalise it to the
default true. Guard the camelCase round-trip so this cannot regress
silently.

Refs #1054
The master switch only gated list navigation, and it read a render-time
ref, so a cross-window settings:changed could leave the main window
running the old value. F stayed on regardless because hint mode had its
own switch.

Read the settings module cache at keydown time (it is updated
synchronously by broadcasts), gate the F hint entry on vimMode too, and
close an open hint session when either switch turns off. The Keyboard /
PDF sub-switches now disable themselves with a "Requires Vim mode"
note while the master is off.

Refs #1054
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant