Skip to content

fix(vision): preserve native image blocks during handoff - #2

Merged
Gu-ZT merged 1 commit into
dsh-plugins:mainfrom
christoph-hess-itemis:fix/native-first-image-handoff
Sep 6, 2026
Merged

Gu-ZT merged 1 commit into
dsh-plugins:mainfrom
christoph-hess-itemis:fix/native-first-image-handoff

Conversation

@christoph-hess-itemis

Copy link
Copy Markdown
Contributor

Summary

  • preserve original ImageBlocks for exact request routes that declare image input
  • rewrite images to durable describe_image references only for text-only or undeclared routes
  • make handoff re-entry request-local and teardown/reconfiguration safe
  • keep the auxiliary vision route’s capability metadata honest
  • add focused Node regression coverage and bilingual documentation

Root cause

The v0.6.4 handoff listener treated every non-auxiliary route as text-only. It never resolved the exact request route’s real modalities, so native vision models were downgraded to text references. Its global re-entry flag and resolver replacement also made concurrent streams and teardown fragile, while inflating the configured auxiliary route could hide an invalid vision selection.

Validation

  • npm run typecheck
  • npm test (14/14)
  • npm pack --dry-run --json --cache .npm-cache
  • git diff --check

Scope / follow-up

This remains a plugin-only fix and uses no private agent/request-surface API. The existing skipWhenMainModelSupportsImage option remains a generic prompt-assembly approximation.

Exact per-attempt Auto/Always/Disabled visibility needs a Harness core contract exposing the prepared route/capabilities and request-lifetime tool restriction/guard lifecycle.

Because the released llm/stream contract describes loop request content as read-only, a fully first-class handoff should also gain a core-owned, session-logged image-reference projection/admission hook. This PR hardens the compatibility shim already shipped in v0.6.4.

@Gu-ZT
Gu-ZT merged commit e497aa5 into dsh-plugins:main Sep 6, 2026
5 checks passed
@Gu-ZT

Gu-ZT commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator

已审查并以 squash 合并(e497aa5)。感谢贡献!

评审确认的三处真实修复:

  1. 原生支持图片的路由不再被无条件降级为文本引用 —— 按请求用原始 resolveModelInfo 探测真实模态,仅纯文本/未声明路由回退到 [image: …] 引用,这正是 v0.6.4 行为里最需要修的一点。
  2. admission 包装不再虚增辅助视觉路由自身的能力,误选纯文本视觉模型时 askVision 的守卫现在会正确报错。
  3. 重入守卫从全局布尔改为请求局部 WeakSet,并发流互不干扰;disposer 幂等且不会覆盖其他插件后装的包装,拆除语义干净。

本地复核: npm test 14/14 通过、typecheckbuild 干净;PR CI 三平台全绿。与我这边 v0.6.4 引入的 skipWhenMainModelSupportsImage(提示词装配层)无冲突 —— 两者作用层级不同,组合语义一致。

两个非阻塞的后续建议:

  • CI 工作流目前只跑 typecheck/build/pack,新增的 npm test 没有接入 CI,建议后续在 quality 作业里加一步,避免回归测试腐化。
  • 能力解析失败现在会让含图请求以流错误失败(此前是无条件改写)。agent-loop 场景下解析在请求准备阶段已成功,实际无影响;如想更保守,可在解析异常时回退到安全改写。

关于 PR 描述里提到的核心契约(把准备后的路由/能力发布给提示词装配、请求生命周期的工具限制、session-logged 图片引用投影挂接点):方向认同,欢迎到 dsh 主仓开 issue 讨论。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants