Conversation
…ction tool Collapse the three flat tool registrations into a single sema(action=search|stats|reindex) tool. The execute and render logic of the three former tools is kept verbatim; only the tool surface changes, so three per-round schema payloads become one. The open output schema root records the producing sub-tool via __act so render dispatches; an explicit additionalProperties value is required by the dsh schema compiler. 【中文】将 sema_search/sema_reindex/sema_stats 三件平铺工具聚合为 sema(action=search|stats|reindex) 单件。三件原工具的 execute 与 render 逻辑逐字保留,仅收敛工具面——每轮注入 schema 从三件降为 一件。输出 schema 根保持开放(additionalProperties 显式声明—— dsh schema 编译器拒省略态),内部 __act 字段记录产出子工具供 render 分发。
The 16000-char default assumes roughly 0.25 tokens per character, which holds for English but not for CJK text (close to one token per character). A default-sized CJK chunk exceeds embedding endpoints' per-item token limit, every batch is rejected with HTTP 400, and the index silently degrades to the local lexical provider. Lower the default to 3000 chars, which keeps CJK chunks within typical per-item limits at a small cost for ASCII-heavy corpora. 【中文】默认 maxCharsPerText=16000 按英文 token 比标定;中文接近 1 字≈1 token,默认尺寸中文 chunk 超出 embedding 端点单条 token 上限——整批 400、allowFallback 下静默永久降级为本地 lexical。 默认降为 3000 字符:中文安全,英文语料损失小。
A batch-level HTTP 400 (parameter-error family) can be triggered by a single rejected chunk. Previously the whole embed call failed, and after the first boot-build failure the index silently degraded to the local lexical provider forever. Retry a rejected batch item-by-item: good items keep their vectors, rejected items become zero-vector Float32Array placeholders so row alignment survives persistence, and if every item fails the original batch error is kept. Per-item retries share the batch timeout budget through AbortSignal.any so a slow endpoint cannot hang the salvage loop. 【中文】批级 HTTP 400(参数误族)常由单条坏 chunk 触发——原先整批 embed 失败,boot 首败+allowFallback 即永久降级 lexical。改为逐条 重试:好条保留向量,坏条以零向量 Float32Array 占位保行对齐 (persist 层依赖 subarray,普通 Array 会炸持久化致整次构建丢失); 全批皆坏时保留原批错误语义。单条重试经 AbortSignal.any 共享 timeoutMs 预算,防慢端点无限挂起。
Add an optional resolveKey hook to the OpenAI-compatible provider and thread it through createProvider / SearchIndex. When provided, the key is resolved on every embed operation, so key rotation applies without recreating the index, and hosts that keep credentials in a managed store can feed the provider without environment variables. Also surface safePersist failures through log.error in addition to the errors array: callers that never read the array (e.g. the CLI) would otherwise miss a silent persist failure that discards a long build while the on-disk index stays unchanged. 【中文】为 OpenAI 兼容 provider 增加可选 resolveKey 钩子并经 createProvider / SearchIndex 透传:每次 embed 操作现场解析 key—— 轮换无需重建索引,凭证托管型宿主无需向环境变量导出明文。 另:safePersist 失败在 errors 数组之外补发 log.error——不读该数组的 调用方(如 CLI)此前对「长时构建向量丢失、盘面零变化」完全无感。
lexical fallback apply() starts the background boot (load -> build -> embed) while the credentials service may still be loading its store. If the embedding key resolves to empty at that moment, the first embed fails and the index degrades to the local lexical provider permanently, even though the credential appears moments later. When the openai provider is configured, resolve the key per operation through ctx.credentials (provider.apiKeyEnv names a stored credential reference), retrying until the credential resolves (30s cap; once the store is ready the first iteration resolves immediately). Key rotation now applies without a restart, and no key material has to be exported into the environment. 【中文】apply() 启动即跑后台 boot(装载→构建→embed),此刻宿主 credentials 服务可能仍在读库——key 偶发为空则首败固化,allowFallback 下索引永久降级本地 lexical(实测:配置全对、凭证数分钟后已在库、 stats 仍报 lexical)。openai 线改经 ctx.credentials 每操作解析 (apiKeyEnv 作凭证引用名),boot 期重试等待就绪(帽 30s;就绪后 首循环即命中)。密钥零入 process.env,轮换热生效。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Retry credential resolution during boot to avoid permanent lexical fallback
Base: on top of pr-5 (resolveKey plumbing) · stacked series: pr-1 → pr-2 → pr-4 → pr-5 → this PR
Motivation
apply()starts the background boot immediately: load persisted state →build('full')→ embed →resolveKey. At that moment the host'scredentialsservice may still be loading its store. If the key resolves to empty just once, the first embed fails, and withallowFallbackthe index pins itself to the local lexical provider permanently — observed in production: provider config entirely correct,sema(action=stats)still reporting the lexical provider after the credential had been available for minutes.The service contract explicitly supports this pattern: resolution is per call, and consumers re-resolve at each operation.
What changes
injectgainscredentials.openaiandprovider.apiKeyEnvnames a credential reference,applybuilds aresolveKeythat resolves throughctx.credentials.resolve(credentialRef(...))per operation — with a boot-race retry: up to 30 attempts, 1s apart, until the credential resolves. Once the store is ready the first iteration resolves immediately; the cap bounds the wait when the reference is genuinely unconfigured.package.json:@deepseek-ai/dsh-credentialsadded to peerDependencies (>=0.1.0-rc.6 <0.2.0) and devDependencies;package-lock.jsonintentionally not included (regenerate in CI).process.env, and rotation applies without a restart.Before / after
Testing
tsc --noEmitclean; full suite 85/85 pass, including a new integration test: mock credential service empty for two calls, then ready — boot build waits through both retry intervals, the embed request carries the late key (Bearer sk-late), no boot errors logged.stats()reports the openai provider, not lexical.