From ac8874914a4458b26574302c0b658321ab78b354 Mon Sep 17 00:00:00 2001 From: A-znk <145305286+Yuki-zik@users.noreply.github.com> Date: Wed, 8 Jul 2026 11:07:01 +0800 Subject: [PATCH] fix(ripgrep): return partial results instead of failing when output exceeds maxBuffer On large repos ripgrep stdout could exceed the 200 KB maxBuffer, which rejected the whole search and left the tree with zero todos. Raise the default taskvision.ripgrep.ripgrepMaxBuffer to 20480 (20 MB) and, when the limit is still hit, flag the results as truncated, stop the process, and resolve with the partial matches so the tree still populates. A one-time non-fatal warning advises raising the limit; the SIGINT interrupted path is preserved. Fixes #4 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- agent/tasks.md | 10 ++++----- agent/timeline.md | 1 + package.json | 2 +- package.nls.json | 2 +- package.nls.zh-cn.json | 2 +- src/extension.js | 18 +++++++++++++++ src/ripgrep.js | 32 +++++++++++++++++++++++--- test/ripgrep.tests.js | 51 ++++++++++++++++++++++++++++++++++++++++++ 8 files changed, 107 insertions(+), 11 deletions(-) diff --git a/agent/tasks.md b/agent/tasks.md index a208c23..47325aa 100644 --- a/agent/tasks.md +++ b/agent/tasks.md @@ -9,11 +9,11 @@ ## Session Summary -- Active focus: Reanalyzed the AI context feature's data flow, product logic, and agent handoff contract after sidecar write hardening. -- Verification: read-only source/docs/tests review plus `git --no-pager ls-files .taskvision .gitignore`; no test suite was run because no runtime code changed. +- Active focus: Fixed GitHub issue #4 — `ripgrep.search` now returns the partial results it collected (with a one-time warning) instead of rejecting the whole search when stdout exceeds `maxBuffer`, and raised the default `taskvision.ripgrep.ripgrepMaxBuffer` from 200 KB to 20480 KB (20 MB). +- Verification: `npm test` (120 passing, including a new truncation/partial-results regression test and the unchanged SIGINT interrupted test) and `npm run webpack` (build succeeded). ## Active Session Task -| Priority | Task | Status | Owner | Due | -| -------- | ----------------------------- | --------- | ----- | ---------- | -| P1 | Reanalyze AI context logic | Completed | AI | 2026-04-29 | +| Priority | Task | Status | Owner | Due | +| -------- | ----------------------------------------------------------------- | --------- | ----- | ---------- | +| P1 | Fix issue #4: return partial ripgrep results instead of failing | Completed | AI | 2026-07-08 | diff --git a/agent/timeline.md b/agent/timeline.md index 55e172f..50f09a4 100644 --- a/agent/timeline.md +++ b/agent/timeline.md @@ -2,6 +2,7 @@ | 日期时间 | 任务/变更 | 修改文件 | 实现逻辑 | 修改动机 | 结果/备注 | | ---------------- | --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-07-08 11:05 | 修复 issue #4：ripgrep 输出超过 maxBuffer 时返回部分结果而非整体失败 | src/ripgrep.js, src/extension.js, package.json, package.nls.json, package.nls.zh-cn.json, test/ripgrep.tests.js, agent/tasks.md, agent/timeline.md | 将 `taskvision.ripgrep.ripgrepMaxBuffer` 默认值从 200 提升到 20480（20 MB）并同步中英文 markdownDescription；`ripgrep.search` 超过 maxBuffer 时不再 reject，而是置 `truncated` 标记、SIGINT 停止进程，并在 close 处理器优先 `resolve` `formatResults` 的部分结果（数组附带 `truncated`/`maxBuffer`），保留真实 SIGINT interrupted 与退出码分支；extension.js `search()` 检测 `matches.truncated` 后调用一次性 `notifyBufferTruncated` 警告（复用 `OPEN_SETTINGS_BUTTON` 模式），并在 `rebuild()` 重置该一次性标记；新增部分结果回归测试 | 大型仓库因 ripgrep stdout 超过 200 KB 默认缓冲导致整体 reject、树视图零结果（GitHub issue #4），需要在提高上限的同时对超限做优雅降级，保证已收集的标签仍能展示 | 已完成；验证执行 `npm test`（120 passing，含新增 truncation 用例且既有 `ripgrep.kill forwards SIGINT` interrupted 用例未回归）、`npm run webpack`（构建成功） | | 2026-04-29 20:45 | 重新分析 AI 上下文功能逻辑与协作思路 | agent/tasks.md, agent/timeline.md | 只读梳理 `aiContext` 导出器、扩展命令链路、task/context/session sidecar store、稳定 ID 生成、README 协议说明、配置 schema 与测试覆盖，并确认 `.taskvision` 生成文件当前被 Git 跟踪 | 用户要求重新检查 AI 上下文功能部分的逻辑与思路是否合理，需要在继续实现前判断产品定位、数据源边界、会话语义和协作契约是否自洽 | 已完成分析；结论为“外部 agent 交接包”的核心方向合理，但 implementation session 未进入导出、sidecar/source 权责边界、跟踪生成文件、状态报告语义和 Extension Host 集成测试仍需收敛；验证执行只读源码/文档/测试核查与 `git --no-pager ls-files .taskvision .gitignore`，未运行测试套件 | | 2026-04-29 20:18 | 加固 AI sidecar JSON 写入与损坏恢复 | src/jsonStore.js, src/taskMetaStore.js, src/contextStore.js, src/changeSessionStore.js, test/dataModel.tests.js, test/extension.tests.js, agent/tasks.md, agent/timeline.md | 新增共享 `jsonStore`，用同目录临时文件加 `rename` 替换写入 JSON；读取 malformed JSON 时先重命名为 `.invalid` 备份再恢复为空 store；任务与上下文 store 在写入失败后失效内存 cache，避免未落盘变更被下一次保存带回；补充 malformed 备份、原子替换失败和 cache 丢弃回归测试 | 用户要求开始实现 AI 可靠性改进，需要优先降低 `.taskvision` sidecar 被半写、损坏或静默覆盖时的协作风险 | 已完成；验证执行 `npx qunit test/dataModel.tests.js test/aiContext.tests.js test/extension.tests.js`、`npm test`（119 passing）、`npm run webpack`、`git diff --check -- agent/tasks.md agent/timeline.md src/jsonStore.js src/taskMetaStore.js src/contextStore.js src/changeSessionStore.js test/dataModel.tests.js test/extension.tests.js` | | 2026-04-29 19:00 | 修复深度 review 中确认的问题 | buildCodiconNames.js, package.json, package.nls.json, package.nls.zh-cn.json, src/config.js, src/extension.js, src/icons.js, src/taskMetaStore.js, test/aiContext.tests.js, test/extension.tests.js, test/schemes.tests.js, agent/tasks.md, agent/timeline.md | 为 AI 输出目录增加 workspace 内约束与 schema 校验，修正 `file excludes` 枚举判断，移除 regex 配置变更早退并让 opacity 变更清高亮缓存，合并字体样式配置写入，修正 TaskVision 内置图标校验条件，补齐命令 NLS key，并让 codicon 生成失败时中止构建而不是写入极小 fallback；同时补充对应回归测试 | 用户要求开始修复全系统深度 review 中已确认的问题，需要用测试先行方式把行为锁住并避免误改未验证项 | 已完成；验证执行 `npx qunit test/aiContext.tests.js test/extension.tests.js test/schemes.tests.js`、`npm test`（112 passing）、`npm run webpack`、`git diff --check -- agent/tasks.md agent/timeline.md buildCodiconNames.js package.json package.nls.json package.nls.zh-cn.json src/config.js src/extension.js src/icons.js src/taskMetaStore.js test/aiContext.tests.js test/extension.tests.js test/schemes.tests.js` | diff --git a/package.json b/package.json index f74812c..3098bf4 100644 --- a/package.json +++ b/package.json @@ -1716,7 +1716,7 @@ "type": "object", "properties": { "taskvision.ripgrep.ripgrepMaxBuffer": { - "default": 200, + "default": 20480, "markdownDescription": "%taskvision.configuration.ripgrep.ripgrepMaxBuffer.markdownDescription%", "type": "integer" }, diff --git a/package.nls.json b/package.nls.json index 5592a7d..3c57b45 100644 --- a/package.nls.json +++ b/package.nls.json @@ -106,7 +106,7 @@ "taskvision.configuration.ripgrep": "Ripgrep", "taskvision.configuration.ripgrep.ripgrep.markdownDescription": "Location of your version of ripgrep.", "taskvision.configuration.ripgrep.ripgrepArgs.markdownDescription": "Additional arguments to pass to ripgrep (Use with caution!).", - "taskvision.configuration.ripgrep.ripgrepMaxBuffer.markdownDescription": "Size of the buffer to use for retrieving output from ripgrep (kilobytes).", + "taskvision.configuration.ripgrep.ripgrepMaxBuffer.markdownDescription": "Size of the buffer to use for retrieving output from ripgrep (kilobytes). Defaults to `20480` (20 MB). If a large repository still exceeds this limit, TaskVision shows the partial results it collected and warns you to raise this value further.", "taskvision.configuration.ripgrep.usePatternFile.markdownDescription": "A pattern file is used with ripgrep by default. If you experience issues with deleting the pattern file, set this to false to use the legacy method of providing the regex to ripgrep.", "taskvision.configuration.tree": "Tree", "taskvision.configuration.tree.autoRefresh.markdownDescription": "Refresh the tree when files are opened or saved.", diff --git a/package.nls.zh-cn.json b/package.nls.zh-cn.json index cdbdaf6..e0ff637 100644 --- a/package.nls.zh-cn.json +++ b/package.nls.zh-cn.json @@ -106,7 +106,7 @@ "taskvision.configuration.ripgrep": "Ripgrep", "taskvision.configuration.ripgrep.ripgrep.markdownDescription": "通常此拓展在需要时能自行定位 ripgrep。若要使用 ripgrep 的其他版本，在此处设置其安装位置。", "taskvision.configuration.ripgrep.ripgrepArgs.markdownDescription": "配置要传递给 ripgrep 的额外参数（谨慎使用）。", - "taskvision.configuration.ripgrep.ripgrepMaxBuffer.markdownDescription": "配置 ripgrep 缓冲区的大小（KB）。当目前缓冲区大小不足以查看所有标签时，可适当增加缓冲区大小。", + "taskvision.configuration.ripgrep.ripgrepMaxBuffer.markdownDescription": "配置 ripgrep 缓冲区的大小（KB）。默认值为 `20480`（20 MB）。当超大仓库仍超出此限制时，TaskVision 会展示已收集到的部分结果，并提示你进一步增大该值。", "taskvision.configuration.ripgrep.usePatternFile.markdownDescription": "默认情况下，模板文件与 ripgrep 一起使用。当删除模板文件时遇到问题，请将此项设置为 false，即可使用向 ripgrep 提供正则表达式的旧方法。", "taskvision.configuration.tree": "树", "taskvision.configuration.tree.autoRefresh.markdownDescription": "打开或保存文件时自动刷新树状图。", diff --git a/src/extension.js b/src/extension.js index e993fdd..6701cc6 100644 --- a/src/extension.js +++ b/src/extension.js @@ -58,6 +58,7 @@ var OK_BUTTON = "OK"; function activate(context) { var outputChannel; + var bufferTruncationWarningShown = false; function settingLocation(setting) { var current = vscode.workspace.getConfiguration('taskvision').inspect(setting); @@ -294,10 +295,26 @@ function activate(context) { } } + function notifyBufferTruncated(maxBuffer) { + debug("Search output exceeded maxBuffer of " + maxBuffer + " KB - showing partial results"); + if (bufferTruncationWarningShown) { + return; + } + bufferTruncationWarningShown = true; + vscode.window.showWarningMessage("TaskVision: Search output exceeded the buffer limit (" + maxBuffer + " KB), so only partial results are shown. Increase 'taskvision.ripgrep.ripgrepMaxBuffer' to see all tags.", OPEN_SETTINGS_BUTTON, OK_BUTTON).then(function (button) { + if (button === OPEN_SETTINGS_BUTTON) { + vscode.commands.executeCommand('workbench.action.openSettings', 'taskvision.ripgrep.ripgrepMaxBuffer'); + } + }); + } + function search(options) { debug("Searching " + options.filename + "..."); return ripgrep.search("/", options).then(matches => { + if (matches && matches.truncated === true) { + notifyBufferTruncated(matches.maxBuffer || options.maxBuffer); + } if (matches.length > 0) { matches.forEach(match => { match.uri = vscode.Uri.file(match.fsPath); @@ -547,6 +564,7 @@ function activate(context) { provider.clear(vscode.workspace.workspaceFolders); interrupted = false; + bufferTruncationWarningShown = false; statusBarIndicator.text = "TaskVision: Scanning..."; statusBarIndicator.show(); diff --git a/src/ripgrep.js b/src/ripgrep.js index 39260fc..4f28a45 100644 --- a/src/ripgrep.js +++ b/src/ripgrep.js @@ -234,10 +234,12 @@ module.exports.search = function ripGrep( cwd, options ) return new Promise( function( resolve, reject ) { - const maxBuffer = ( options.maxBuffer || 200 ) * 1024; + const maxBufferKB = ( options.maxBuffer || 200 ); + const maxBuffer = maxBufferKB * 1024; var results = ""; var errors = ""; var hasCompleted = false; + var truncated = false; currentProcess = child_process.spawn( rgPath, args, { cwd: cwd, @@ -262,12 +264,23 @@ module.exports.search = function ripGrep( cwd, options ) currentProcess.stdout.on( 'data', function( data ) { + if( truncated ) + { + return; + } debug( "Search results:\n" + data ); results += data; if( Buffer.byteLength( results, 'utf8' ) > maxBuffer ) { - currentProcess.kill( 'SIGINT' ); - fail( "Search output exceeded maxBuffer of " + ( options.maxBuffer || 200 ) + " KB", errors ); + // Rather than rejecting (which would leave the tree empty on + // large repos), flag the result as truncated, stop the search + // and let the 'close' handler resolve with the partial matches. + truncated = true; + debug( "Search output exceeded maxBuffer of " + maxBufferKB + " KB - returning partial results. Raise 'taskvision.ripgrep.ripgrepMaxBuffer' to collect more tags." ); + if( currentProcess !== undefined ) + { + currentProcess.kill( 'SIGINT' ); + } } } ); @@ -293,6 +306,19 @@ module.exports.search = function ripGrep( cwd, options ) cleanupPatternFile( options.patternFilePath ); currentProcess = undefined; + if( truncated ) + { + // Partial results collected before the buffer limit are still + // useful. Resolve them with a marker so callers can warn the + // user instead of discarding every tag. This takes priority + // over the SIGINT we sent to stop the search. + var partial = formatResults( results, options.multiline ); + partial.truncated = true; + partial.maxBuffer = maxBufferKB; + resolve( partial ); + return; + } + if( signal ) { reject( new RipgrepError( "Search interrupted (" + signal + ")", errors ) ); diff --git a/test/ripgrep.tests.js b/test/ripgrep.tests.js index 5d1b3a4..6490b46 100644 --- a/test/ripgrep.tests.js +++ b/test/ripgrep.tests.js @@ -83,3 +83,54 @@ QUnit.test("ripgrep.kill forwards SIGINT to active process", function (assert) { ripgrep.kill(); }); + +QUnit.test("ripgrep.search returns partial results when output exceeds maxBuffer", function (assert) { + var done = assert.async(); + var childProcess = require('child_process'); + var originalSpawn = childProcess.spawn; + + var fakeProcess = new events.EventEmitter(); + fakeProcess.stdout = new events.EventEmitter(); + fakeProcess.stderr = new events.EventEmitter(); + fakeProcess.stdout.setEncoding = function () { }; + fakeProcess.stderr.setEncoding = function () { }; + var killSignal; + fakeProcess.kill = function (signal) { + killSignal = signal; + // Emulate ripgrep stopping after being asked to stop. + this.emit('close', null, 'SIGINT'); + }; + + childProcess.spawn = function () { + return fakeProcess; + }; + + // Build more than one KB of valid vimgrep output so the 1 KB buffer overflows. + var line = 'src/file.js:1:1:// TODO something that needs doing\n'; + var bigChunk = ''; + while (bigChunk.length < 2 * 1024) { + bigChunk += line; + } + + ripgrep.search(process.cwd(), { + rgPath: process.execPath, + regex: '(TODO)', + unquotedRegex: '(TODO)', + additional: '', + globs: [], + maxBuffer: 1 + }).then(function (matches) { + childProcess.spawn = originalSpawn; + assert.strictEqual(matches.truncated, true, "results are flagged as truncated"); + assert.strictEqual(matches.maxBuffer, 1, "the buffer size is reported back to the caller"); + assert.ok(matches.length > 0, "partial matches are still returned instead of rejecting"); + assert.equal(killSignal, 'SIGINT', "the search process is stopped once the buffer is exceeded"); + done(); + }).catch(function (e) { + childProcess.spawn = originalSpawn; + assert.ok(false, "search should resolve with partial results, but rejected: " + (e && e.message)); + done(); + }); + + fakeProcess.stdout.emit('data', bigChunk); +});