Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-27-web-file-upload-by-path.md
2026-08-27-web-file-upload-by-path.md: 584d4095b00fb3f3abfc2781bb33765151c8d0bd
2026-08-27-web-file-upload-by-path.zh.md: 98251922eaa047094789c0befd967448ca71c6f5
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Agent Note: Web GUI file upload sends host paths, not staged bytes

Status: implemented

English | [中文](2026-08-27-web-file-upload-by-path.zh.md)

## Problem

The composer only accepted images, and only as base64 inlined into the prompt (`createDraftImages` rejected every non-`png/jpeg/webp/gif` MIME with `UnsupportedImageMediaTypeError`). A user could not hand the agent an arbitrary local file without either staging a copy on disk or inflating the conversation with its bytes. The browser can never read a dragged file's real host path — `File` exposes only `name`, `size`, and content — so "send the path and let the agent read it" has no pure-browser route.

## Decision

Two complementary host capabilities, plus a composer split, all path-only:

- **`file-picker` / `file-picker-native`** (`ctx.filePicker`): a native-only capability seam whose single interaction `pickFiles({ multiple }, signal)` opens one OS chooser on the host display and returns the selected absolute paths. Native-only because a remote client has no display to open a chooser on; unlike `directory-picker` there is no browse twin. The gateway serves it as `host.pickFiles`, answering `file-picker-unavailable` under any non-native backend (fail loud, never fake a pick). macOS uses `osascript choose file`; Linux uses Zenity with a KDialog fallback; Windows fails loud until the koffi `IFileOpenDialog` file multi-select conversation is built.
- **Basename location** (`file-picker`'s `./locate`): `locateByName(root, name)` walks a tree for exact-basename matches and returns absolute paths — the zero-byte, zero-write answer for a drag, whose only input is a `name`. It accepts an optional `systemSearch` delegate for anything wider than the workspace walk (spotlight/`find`), which this increment does not wire; the gateway's `host.locateFiles` walks the session workspace only.
- **Composer split** (`InputBar`): a dropped/pasted file is classified by MIME — multi-modal images join the image rail unchanged; everything else resolves through `locateFiles` and inserts a `@path` mention (reusing `formatFileMention`) — never staged bytes. A new paperclip button drives `pickFiles` and inserts every selected path as a mention. A basename with zero or several matches is announced ("use the add-file button"), not silently dropped.

Why these shapes won:

- **Path-only beats staging.** Staging a copy into the workspace would satisfy "the agent reads it by path" but spends disk and risks clobbering the user's tree; a content-addressed object store (like images) sidesteps clobbering but yields no readable path at all. Returning the real path is the only option that is simultaneously zero-copy, zero-context, and agent-readable. Model-visible ⟺ logged holds trivially: the mention is ordinary prompt text.
- **Drag gets a name, not a path.** The `locateByName` exact-basename walk is precise and unambiguous at the workspace tier; the system-wide tier stays an injected delegate because its cost and coverage are the caller's policy, and this increment has no workspace-external consumer yet.
- **Native-only, discriminated anyway.** The `FilePickerCapabilities` map is declared merge-extensible from day one, mirroring `directory-picker`, so a future backend adds its shape without editing this package. The failure mode for an unsupported interaction is a typed `file-picker-unavailable` code, matching `directory-picker-unavailable`.
- **MIME decides image vs file in the composer.** The image rail's own admission keys on MIME too, so a suffix-only match would be refused there; routing on MIME alone keeps one rule and lets an unknown-type file fall through to path location (the agent picks the reading tool).

## Alternatives considered

- **Stage-then-mention and submit-time atomic staging** — both write bytes to disk (rejected; the operator asked for zero landing), and submit-time staging hides the path from the draft.
- **Bundle a system-wide search now** — `mdfind`/`find` per platform adds an OS-command surface and cost policy this increment has no consumer for; the `systemSearch` delegate keeps the seam ready without committing those choices.
- **Reuse `@file`'s `WorkspaceFileSearch`** — that index returns workspace-relative paths for autocomplete ranking, not absolute paths from a bare basename; a separate exact-basename walk is a different, smaller query.
- **Extend `directory-picker` with a `pickFiles` method** — rejected: the seam is named for directory selection and its browse/native contract is about levels and creation, not file selection; a sibling package keeps each contract's consumer set honest.

## Consequences

- Two new host packages (`file-picker`, `file-picker-native`), two gateway methods (`host.pickFiles`, `host.locateFiles`), and one error code (`file-picker-unavailable`) widen the web GUI's host surface; each mirrors the `directory-picker` seam and rides the same apiproxy/rpc/schema layers.
- The composer splits on the `image/*` MIME family, so a deployment's narrowed `imageLimits.mediaTypes` no longer changes the image-vs-file split (an `image/*` type keeps the image rail's authoritative "only PNG/JPG/WebP/GIF" refusal); an empty-MIME file always resolves by path.
- Windows file selection is not shipped: `pickNativeFiles` fails loud on win32 until the koffi `IFileOpenDialog` file multi-select conversation lands. Remote clients likewise have no display to open a chooser on.
- `locateByName` walks the workspace tree only; a wider `systemSearch` tier is an injected delegate left unwired until a workspace-external consumer needs it. Dragged basenames with zero or several matches announce "use the add-file button" rather than guessing.
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Agent Note:Web GUI 文件上传发送宿主路径,而非暂存字节

Status: implemented

[English](2026-08-27-web-file-upload-by-path.md) | 中文

## Problem

对话框只接受图片,而且只以 base64 内嵌进提示文本(`createDraftImages` 对非 `png/jpeg/webp/gif` 的 MIME 一律抛 `UnsupportedImageMediaTypeError`)。用户在既不暂存磁盘副本、也不膨胀对话上下文的前提下,无法把任意本地文件交给智能体。浏览器永远读不到拖拽文件的真实宿主路径——`File` 只暴露 `name`、`size` 和内容——因此「发送路径、让智能体去读」没有纯浏览器路线。

## Decision

两个互补的宿主能力,加上对话框的分流,全程只发路径:

- **`file-picker` / `file-picker-native`**(`ctx.filePicker`):一个仅 native 的能力缝隙,其唯一交互 `pickFiles({ multiple }, signal)` 在宿主显示器上打开一个操作系统文件选择框并返回选中的绝对路径。只做 native,因为远程客户端没有可打开选择框的显示器;与 `directory-picker` 不同,没有 browse 孪生形态。网关以 `host.pickFiles` 提供它,在非 native 后端下回答 `file-picker-unavailable`(大声失败,绝不假装已选)。macOS 用 `osascript choose file`;Linux 用 Zenity、以 KDialog 回退;Windows 在建成 koffi `IFileOpenDialog` 文件多选对话之前大声失败。
- **Basename 定位**(`file-picker` 的 `./locate`):`locateByName(root, name)` 遍历目录树做 basename 精确匹配并返回绝对路径——这是拖拽的零字节、零写入答案,其唯一输入就是 `name`。它接受可选的 `systemSearch` 委托来处理任何比工作区遍历更宽的搜索(spotlight/`find`),本增量不接线;网关的 `host.locateFiles` 只遍历会话工作区。
- **对话框分流**(`InputBar`):对拖入/粘贴的文件按 MIME 分类——多模态图片照旧进图片轨道;其余全部经 `locateFiles` 解析并插入 `@path` 引用(复用 `formatFileMention`)——绝不暂存字节。新增的回形针按钮驱动 `pickFiles` 并把每个选中路径插为引用。basename 命中零个或多个时给出提示(「请用添加文件按钮」),而不是静默丢弃。

这些取舍胜出的原因:

- **只发路径胜过暂存。** 把副本暂存进工作区能满足「智能体按路径读」,但消耗磁盘并可能污染用户的目录树;内容寻址对象存储(如图片)能避开污染,却根本拿不出可读路径。返回真实路径是唯一同时做到零复制、零上下文、且智能体可读的方案。「模型可见 ⟺ 已入日志」天然成立:引用就是普通提示文本。
- **拖拽拿到的是名字,不是路径。** `locateByName` 的 basename 精确遍历在工作区这一层是精确且无歧义的;系统级层保持为注入式委托,因为其开销与覆盖范围属于调用方策略,而本增量尚无工作区之外的消费者。
- **仅 native,但仍然判别式。** `FilePickerCapabilities` 映射从第一天起就声明为可合并扩展,镜像 `directory-picker`,因此未来后端无需修改本包即可增加其形态。不支持的交互以大写的 `file-picker-unavailable` 码失败,与 `directory-picker-unavailable` 一致。
- **对话框里由 MIME 决定图片还是文件。** 图片轨道自身的准入同样以 MIME 为准,因此仅按后缀匹配会被拒;只按 MIME 路由保持单一规则,并让未知类型的文件落入路径定位(由智能体选择读取工具)。

## Alternatives considered

- **先暂存再引用、以及提交时原子暂存**——两者都写字节到磁盘(被拒;操作者要求零落地),且提交时暂存会把路径藏出草稿。
- **现在就内置系统级搜索**——按平台 `mdfind`/`find` 会引入操作系统命令面与开销策略,而本增量没有消费者;`systemSearch` 委托让缝隙保持就绪而不必现在拍板这些选择。
- **复用 `@file` 的 `WorkspaceFileSearch`**——那个索引为自动补全排序返回工作区相对路径,而不是从裸 basename 得到绝对路径;单独的 basename 精确遍历是另一个更小的查询。
- **给 `directory-picker` 加 `pickFiles` 方法**——被拒:该缝隙以目录选择命名,其 browse/native 契约关乎层级与创建,而非文件选择;兄弟包让每个契约的消费者集合保持诚实。

## Consequences

- 新增两个宿主包(`file-picker`、`file-picker-native`)、两个网关方法(`host.pickFiles`、`host.locateFiles`)和一个错误码(`file-picker-unavailable`),扩大了 Web GUI 的宿主面;各自镜像 `directory-picker` 缝隙,并受同一套 apiproxy/rpc/schema 层覆盖。
- 对话框按 `image/*` MIME 族分流,因此部署收窄 `imageLimits.mediaTypes` 不再改变「图片 vs 文件」的划分(`image/*` 类型保留图片轨道权威的「仅 PNG/JPG/WebP/GIF」拒绝);空 MIME 文件始终按路径解析。
- Windows 文件选择未随此交付:在 koffi `IFileOpenDialog` 文件多选对话落地之前,`pickNativeFiles` 在 win32 上大声失败。远程客户端同样没有可打开选择框的显示器。
- `locateByName` 只遍历工作区树;更宽的 `systemSearch` 层级是注入式委托,留待出现工作区之外消费者时再接线。basename 命中零个或多个时给出「请用添加文件按钮」提示,而非猜测。
2 changes: 2 additions & 0 deletions apps/desktop/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,8 @@
"@deepseek-ai/dsh-host-apiproxy": "workspace:^",
"@deepseek-ai/dsh-host-directory-picker-browse": "workspace:^",
"@deepseek-ai/dsh-host-directory-picker-native": "workspace:^",
"@deepseek-ai/dsh-host-file-picker": "workspace:^",
"@deepseek-ai/dsh-host-file-picker-native": "workspace:^",
"@deepseek-ai/dsh-host-plugin-control": "workspace:^",
"@deepseek-ai/dsh-host-plugin-installer": "workspace:^",
"@deepseek-ai/dsh-host-plugin-inventory": "workspace:^",
Expand Down
10 changes: 5 additions & 5 deletions apps/web/tests/image-display.snapshot.ts
Original file line number Diff line number Diff line change
Expand Up @@ -134,19 +134,19 @@ it('accepts pasted images into the composer rail in order and removes them', asy
expect(document.querySelector('[role="group"][aria-label="Pending images"]')).toBeNull()
})

// An unsupported file announces a transient toast (the inline strip is
// gone) and the banner dismisses itself after its hold-and-fade lifetime.
// A non-image file resolves by basename; an empty match announces the
// add-file fallback (the inline strip is already gone).
fireEvent.paste(textarea, {
clipboardData: {
items: [{ kind: 'file', type: 'text/plain', getAsFile: () => new File(['x'], 'notes.txt', { type: 'text/plain' }) }],
getData: () => '',
},
})
const unsupportedMessage = 'Only PNG, JPG, WebP, and GIF images are supported'
const toast = await screen.findByText(unsupportedMessage)
const locateMessage = 'Cannot locate notes.txt; use the add-file button'
const toast = await screen.findByText(locateMessage)
expect(toast.closest('[role="alert"]')).not.toBeNull()
await waitFor(() => {
expect(screen.queryByText(unsupportedMessage)).toBeNull()
expect(screen.queryByText(locateMessage)).toBeNull()
}, { timeout: 6_000 })
})

Expand Down
2 changes: 2 additions & 0 deletions apps/web/tests/snapshots/bash-abort-row/ui.expected.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@
- img
- text: "Failed Bash Error: tool call aborted before dispatch"
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
2 changes: 2 additions & 0 deletions apps/web/tests/snapshots/code-mode-round/ui.expected.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@
- img
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
2 changes: 2 additions & 0 deletions apps/web/tests/snapshots/cordis-tool-round/ui.expected.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,8 @@
- img
- text: {{clock}} Ran for {{duration}}
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
2 changes: 2 additions & 0 deletions apps/web/tests/snapshots/feedback-command/ack.expected.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@
- img
- text: "feedback Feedback recorded for session session-{{uuid}} Anonymous user: {{uuid}}. Session sharing is enabled."
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
2 changes: 2 additions & 0 deletions apps/web/tests/snapshots/fresh-round-trip/ui.expected.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@
- img
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@
- img
- text: "goal No goal is currently set. Usage: /goal [<objective>|clear|edit <objective>|pause|resume]"
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,8 @@
- tooltip "Branch into a new conversation"
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
2 changes: 2 additions & 0 deletions apps/web/tests/snapshots/lifecycle-chrome/hero.expected.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@
- text: Standard mode
- img
- textbox "Describe what you want to build"
- button "Add files":
- img
- button "Commands":
- img
- tooltip "Commands"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@
- text: Standard mode
- img
- textbox "Describe what you want to build"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@
- img
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
2 changes: 2 additions & 0 deletions apps/web/tests/snapshots/live-interactions/cancel.expected.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@
- img
- text: {{clock}} Ran for {{duration}} TTFT {{duration}}
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@
- text: This turn failedAPI key is invalid
- code: AUTH
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@
- paragraph: partial
- status: Deep diving...
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@
- text: This turn failedupstream 503
- code: SERVER
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
2 changes: 2 additions & 0 deletions apps/web/tests/snapshots/live-interactions/retry.expected.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@
- img
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
2 changes: 2 additions & 0 deletions apps/web/tests/snapshots/markdown-cjk-strong/ui.expected.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@
- img
- text: {{clock}} Ran for {{duration}}
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
2 changes: 2 additions & 0 deletions apps/web/tests/snapshots/markdown-images/ui.expected.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@
- img
- text: {{clock}} Ran for {{duration}}
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@
- img
- text: {{clock}} Ran for {{duration}}
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
2 changes: 2 additions & 0 deletions apps/web/tests/snapshots/math-rendering/ui.expected.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@
- img
- text: {{clock}} Ran for {{duration}}
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
2 changes: 2 additions & 0 deletions apps/web/tests/snapshots/message-actions/delete.expected.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@
- button "Edit":
- img
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
2 changes: 2 additions & 0 deletions apps/web/tests/snapshots/message-actions/ui.expected.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@
- img
- text: 7/25 {{clock}} Ran for {{duration}}
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
2 changes: 2 additions & 0 deletions apps/web/tests/snapshots/plan-review/approved.expected.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,8 @@
- img
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@
- img
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
2 changes: 2 additions & 0 deletions apps/web/tests/snapshots/queue-actions/collapsed.expected.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
- status: Deep diving...
- button "2 queued messages"
- textbox "Cmd/Ctrl+Enter steers all queued messages"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
2 changes: 2 additions & 0 deletions apps/web/tests/snapshots/queue-actions/editing.expected.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@
- button "Cancel editing":
- img
- textbox "Cmd/Ctrl+Enter steers all queued messages"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
2 changes: 2 additions & 0 deletions apps/web/tests/snapshots/queue-actions/layout.expected.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@
- img
- button "2 queued messages"
- textbox "Cmd/Ctrl+Enter steers all queued messages"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
2 changes: 2 additions & 0 deletions apps/web/tests/snapshots/queue-actions/preserved.expected.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@
- button "Steer queued message" [disabled]:
- img
- textbox "Message the agent"
- button "Add files":
- img
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
Expand Down
Loading
Loading