diff --git a/CHANGELOG.md b/CHANGELOG.md index a7ffb19..4eeb862 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,38 @@ ## Unreleased +## 2.0.6 - 2026-09-05 + +This is a small reliability release on the 2.0 TypeScript runtime. +`web_search` now queries Wikipedia's open MediaWiki API, then Bing HTML, then +DuckDuckGo HTML. Wikipedia is encyclopedia-only; Bing is the no-key web +fallback when Wikipedia is unreachable. Bundled ripgrep is used for `rg_*` +tools so Windows no longer depends on a system `rg`. Array tool results such as +`skill_list` keep their payloads in the model-facing short view. Permission +mode ids are unchanged. + +### Fixed + +- `web_search` merges Wikipedia, Bing HTML, and DuckDuckGo HTML (optional + SearXNG still works). Wikipedia requests are capped at about eight seconds. +- `rg_search`, `rg_files`, and `rg_count` prefer the bundled ripgrep binary. +- `skill_list`, `todo_read`, `mcp_list`, and `rg_count` short views no longer + collapse arrays to `{}`. + +### Upgrade + +```sh +git pull +npm ci +npm run verify:install +npm link +ant-code --version +``` + +`ant-code --version` should print `2.0.6`. Restart a running Dashboard so it +loads this runtime. Gateway config and `.lab-agent` sessions do not need to +be recreated. + ## 2.0.5 - 2026-09-04 This is a small reliability release on the 2.0 TypeScript runtime. Tool diff --git a/README.md b/README.md index 8ad169f..265aabd 100644 --- a/README.md +++ b/README.md @@ -113,7 +113,7 @@ ant-code doctor ant-code ``` -`ant-code --version` should print `2.0.5`. If you previously linked a +`ant-code --version` should print `2.0.6`. If you previously linked a JavaScript install, run `npm link` again so the global command points at `src/cli/index.ts`. diff --git a/config/skills/web-research/SKILL.md b/config/skills/web-research/SKILL.md index 674fd1e..ec028b5 100644 --- a/config/skills/web-research/SKILL.md +++ b/config/skills/web-research/SKILL.md @@ -12,7 +12,7 @@ argument_hint: 简述要调查的问题、时间范围、可信来源偏好和 ## 工作流 1. 明确用户要查什么、是否需要最新信息、是否有指定来源。 -2. 优先使用 `web_search` 搜索 3-5 个候选来源（内置 DuckDuckGo HTML）。只有配置了自建 SearXNG 时才走搜索 MCP。 +2. 优先使用 `web_search` 搜索 3-5 个候选来源（维基开放接口、Bing HTML、DuckDuckGo HTML）。只有配置了自建 SearXNG 时才走搜索 MCP。 3. 对关键来源使用 `web_fetch` 抓取正文，优先 `markdown`。当前 Ant Code 默认让 `web_fetch` 先走 `fetch` MCP；如果 MCP 不可用，再回退到内置抓取器。 - 如果抓取被权限策略、反爬或 HTML 噪声阻断，先改用搜索摘要、官方 API/raw 文件。 - 若配置允许，可尝试 reader mirror：`https://r.jina.ai/http://r.jina.ai/http://<原始URL>`，并在 caveats 中说明使用了镜像读取。 diff --git a/docs/releases/2.0.6-web-search-rg-tool-views_zh.md b/docs/releases/2.0.6-web-search-rg-tool-views_zh.md new file mode 100644 index 0000000..afac1dc --- /dev/null +++ b/docs/releases/2.0.6-web-search-rg-tool-views_zh.md @@ -0,0 +1,40 @@ +# Ant Code v2.0.6：无 key 搜索渠道与随包代码搜索 + +发布日期：2026-09-05 + +这是 2.0 TypeScript 运行时上的小版本。`web_search` 增加维基百科开放接口，并在维基不可达时继续抓取 Bing HTML，再与 DuckDuckGo HTML 去重合并。Windows 上的 `rg_*` 改为使用随包 ripgrep，不再依赖系统 PATH 是否安装了 `rg`。技能列表、待办和 MCP 列表发给模型时保留实际条目，不再显示成空的 `{}`。权限模式 id 不变。本说明处于公开发布候选阶段，不表示 CI、tag 或 GitHub Release 已经完成。 + +## 对用户工作流的直接变化 + +- `web_search` 先查 Wikipedia/Wikimedia 只读 API（查询含汉字则用中文维基，最多约 8 秒），再抓 Bing HTML，最后才是 DuckDuckGo HTML/lite。可选自建 SearXNG 仍可用。维基覆盖百科词条，不是 Google 式全网检索。 +- 国内直连维基常会超时；Ant Code 会读取 Windows 系统代理做 HTTPS CONNECT。不必在配置里再填代理端口。Bing 在直连可达时可作为无 key 的全网补充。 +- `rg_search` / `rg_files` / `rg_count` 优先使用随包 `rg.exe`，避免 `ENOENT: spawn rg`。 +- `skill_list`、`todo_read`、`mcp_list`、`rg_count` 的模型短视图会写出条数和名称，不再把数组结果收成 `{}`。 + +## 配置、数据与安全边界 + +- 配置格式、session / transcript、凭据存储和网关协议没有迁移。现有配置和 `.lab-agent` 会话可继续使用。 +- 默认允许主机增加 `en.wikipedia.org`、`zh.wikipedia.org`、`bing.com`、`www.bing.com`。`approved-web` 下若允许列表里缺其中任何一个，整次 `web_search` 可能被询问或拦住。 +- 这次没有改权限模式 id。搜索结果必须引用，不能当成最终事实。 +- 自动化测试不能代表所有私有研究数据、外部模型供应商或现场网络环境都已得到验证。 + +## 升级与产物位置 + +需要 Node.js 22.18+。从源码更新： + +```sh +git pull +npm ci +npm run verify:install +npm link +ant-code --version +``` + +`ant-code --version` 应显示 `2.0.6`。若以前 link 过旧安装，需要重新 `npm link`。正在跑的 Dashboard 需要重启。现有网关配置和会话不必重建。本次没有新的安装包要求。 + +## 验证、限制与待完成门禁 + +- 开发仓 `npm test`（`node:test` 口径）：tests 1273，pass 1272，fail 1。失败项是 `MCP request timeout is a failed request, not a user interrupt`，与本次搜索/rg 改动无关。定向套件 `tests/unit/tools.test.ts` 与 `tests/unit/tool-result-view.test.ts` 105 项全部通过。完整 `npm run verify:release` 交给 GitHub CI。 +- DuckDuckGo HTML 仍可能被反爬；维基在未走系统代理时可能超时；Bing HTML 刮取没有长期保证。 +- 未把 TaxaMask 内嵌主题、`dashboard-embed` 或源码写入守卫带入独立版。 +- 候选阶段最终结论以正式仓同一提交的 CI 和 Release 门禁为准。 diff --git a/lab-agent.config.json b/lab-agent.config.json index bb25033..4393da0 100644 --- a/lab-agent.config.json +++ b/lab-agent.config.json @@ -29,7 +29,14 @@ ], "networkMode": "lab-only", "allowedHosts": [ - "gateway.example.com" + "gateway.example.com", + "en.wikipedia.org", + "zh.wikipedia.org", + "bing.com", + "www.bing.com", + "duckduckgo.com", + "html.duckduckgo.com", + "lite.duckduckgo.com" ], "lab": { "gatewayUrl": "https://gateway.example.com/v1/chat/completions", diff --git a/npm-shrinkwrap.json b/npm-shrinkwrap.json index f35cabb..4b9fadf 100644 --- a/npm-shrinkwrap.json +++ b/npm-shrinkwrap.json @@ -1,12 +1,12 @@ { "name": "@ant-code/cli", - "version": "2.0.5", + "version": "2.0.6", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@ant-code/cli", - "version": "2.0.5", + "version": "2.0.6", "dependencies": { "@vscode/ripgrep": "1.18.0", "ink": "^6.8.0", diff --git a/package-lock.json b/package-lock.json index f35cabb..4b9fadf 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@ant-code/cli", - "version": "2.0.5", + "version": "2.0.6", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@ant-code/cli", - "version": "2.0.5", + "version": "2.0.6", "dependencies": { "@vscode/ripgrep": "1.18.0", "ink": "^6.8.0", diff --git a/package.json b/package.json index 72dc5dc..56a86fe 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@ant-code/cli", - "version": "2.0.5", + "version": "2.0.6", "description": "AGPL-licensed local coding agent with a terminal UI, local dashboard, tool permissions, skills, MCP integration, and model gateway adapters.", "type": "module", "private": true, diff --git a/src/tools/definitions.ts b/src/tools/definitions.ts index 60597bd..d479ecb 100644 --- a/src/tools/definitions.ts +++ b/src/tools/definitions.ts @@ -522,7 +522,7 @@ export const BUILT_IN_TOOLS = Object.freeze([ }, { name: "web_search", - description: "Search the public web through a configured SearXNG endpoint or DuckDuckGo HTML fallback. Results are best-effort and must be cited.", + description: "Search the public web through Wikipedia, Bing HTML, and DuckDuckGo HTML, or a configured SearXNG endpoint. Results are best-effort and must be cited.", risk: "network", supportsAbort: true, inputSchema: { diff --git a/src/tools/result-view.ts b/src/tools/result-view.ts index 1314544..5eed4d6 100644 --- a/src/tools/result-view.ts +++ b/src/tools/result-view.ts @@ -62,9 +62,12 @@ function bodyForTool(name: string, execution: ToolResultValue, result: Record): ViewDraft { }; } +function formatRgCount(result: Record): ViewDraft { + const count = Number.isFinite(Number(result.count)) ? Number(result.count) : 0; + const mode = stringField(result.mode) || "matches"; + return { + text: [ + `count=${count}`, + `mode=${mode}` + ].join("\n"), + truncated: result.truncated === true + }; +} + +function formatSkillList(execution: ToolResultValue): ViewDraft { + const skills = Array.isArray(execution.result) ? execution.result : asArray(asRecord(execution.result).skills); + const shown = skills.slice(0, SEARCH_VIEW_MATCHES); + const truncated = skills.length > shown.length; + const lines = [ + `skills=${skills.length}${truncated ? " truncated=true" : ""}`, + ...shown.map((item) => { + const skill = asRecord(item); + const name = stringField(skill.name) || "?"; + const description = stringField(skill.description); + return `- ${name}${description ? `: ${truncateClean(description, SEARCH_LINE_CHARS)}` : ""}`; + }) + ]; + return { text: lines.join("\n"), truncated }; +} + +function formatTodoList(execution: ToolResultValue): ViewDraft { + const todos = Array.isArray(execution.result) ? execution.result : asArray(asRecord(execution.result).todos); + const shown = todos.slice(0, SEARCH_VIEW_MATCHES); + const truncated = todos.length > shown.length; + const lines = [ + `todos=${todos.length}${truncated ? " truncated=true" : ""}`, + ...shown.map((item) => { + const todo = asRecord(item); + const status = stringField(todo.status) || "unknown"; + const content = stringField(todo.content) || stringField(todo.id) || "?"; + return `- [${status}] ${truncateClean(content, SEARCH_LINE_CHARS)}`; + }) + ]; + return { text: lines.join("\n"), truncated }; +} + function formatSearch(result: Record): ViewDraft { const matches = asArray(result.matches); const shown = matches.slice(0, SEARCH_VIEW_MATCHES); @@ -331,6 +384,21 @@ function formatAgent(execution: ToolResultValue, result: Record }; } +function formatMcpList(name: string, execution: ToolResultValue, result: Record): ViewDraft { + if (name === "mcp_list" && Array.isArray(execution.result)) { + const servers = execution.result; + const shown = servers.slice(0, SEARCH_VIEW_MATCHES); + return { + text: [ + `servers=${servers.length}`, + ...shown.map((item) => `- ${typeof item === "string" ? item : stringifyCompact(item)}`) + ].join("\n"), + truncated: servers.length > shown.length + }; + } + return formatMcp(result); +} + function formatMcp(result: Record): ViewDraft { const payload = result.content ?? result.result ?? result; const omitted = omitImagePayloads(payload); diff --git a/src/tools/rg-tools.ts b/src/tools/rg-tools.ts index fbdee79..8169bdd 100644 --- a/src/tools/rg-tools.ts +++ b/src/tools/rg-tools.ts @@ -2,10 +2,12 @@ import { spawn } from "node:child_process"; import fs from "node:fs/promises"; import path from "node:path"; import { createRequire } from "node:module"; +import { fileURLToPath } from "node:url"; import { isInside } from "../permissions/policy-engine.ts"; import { normalizeToolPath } from "../permissions/path-utils.ts"; const require = createRequire(import.meta.url); +const PACKAGE_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..", ".."); const DEFAULT_TIMEOUT_MS = 30_000; const DEFAULT_MAX_RESULTS = 100; const DEFAULT_MAX_OUTPUT_BYTES = 1024 * 1024; @@ -286,8 +288,9 @@ async function runRg(cwd: string, args: string[], input: Record const startedAt = Date.now(); const timeoutMs = positiveInteger(input.timeoutMs, DEFAULT_TIMEOUT_MS); const maxOutputBytes = positiveInteger(input.maxOutputBytes, DEFAULT_MAX_OUTPUT_BYTES); + const spawnArgs = usesWindowsExecutable(executable) ? args.map((arg) => toWindowsPathIfWslMount(arg)) : args; return new Promise((resolve) => { - const child = spawn(executable, args, { cwd, windowsHide: true }); + const child = spawn(executable, spawnArgs, { cwd, windowsHide: true }); const stdout: Buffer[] = []; const stderr: Buffer[] = []; let stdoutBytes = 0; @@ -387,26 +390,113 @@ async function findRgExecutable(input: Record) { const candidates = [ input.rgPath, process.env.ANT_CODE_RG_PATH, - bundledRgPath(), - "rg" + ...bundledRgCandidates(), + ...pathRgCandidates(), + ...wslWindowsRgCandidates() ].filter(Boolean).map(String); + const seen = new Set(); for (const candidate of candidates) { - if (candidate === "rg") { - return candidate; - } const resolved = path.resolve(candidate); - if (await fs.access(resolved).then(() => true).catch(() => false)) { + const key = process.platform === "win32" ? resolved.toLowerCase() : resolved; + if (seen.has(key)) { + continue; + } + seen.add(key); + if (await pathExists(resolved)) { return resolved; } } return null; } -function bundledRgPath() { +export function bundledRgCandidates(env: NodeJS.ProcessEnv = process.env) { + const binary = process.platform === "win32" ? "rg.exe" : "rg"; + const platformPkg = `@vscode/ripgrep-${process.platform}-${process.arch}`; + const roots = [ + env.LAB_AGENT_PACKAGE_ROOT, + PACKAGE_ROOT + ].filter(Boolean).map((root) => path.resolve(String(root))); + const candidates: string[] = []; + try { + const fromModule = require("@vscode/ripgrep").rgPath; + if (typeof fromModule === "string" && fromModule) { + candidates.push(fromModule); + } + } catch { + // Fall through to filesystem lookup. Dashboard cwd is the project, not + // the package root, so module resolution can miss the optional platform + // package even when rg.exe is installed beside it. + } + for (const root of roots) { + candidates.push(path.join(root, "node_modules", platformPkg, "bin", binary)); + candidates.push(path.join(root, "node_modules", "@vscode", "ripgrep", "bin", binary)); + } + return candidates; +} + +export function wslWindowsRgCandidates(env: NodeJS.ProcessEnv = process.env) { + if (!isWslEnvironment(env)) { + return []; + } + const roots = [ + env.LAB_AGENT_PACKAGE_ROOT, + PACKAGE_ROOT + ].filter(Boolean).map((root) => path.resolve(String(root))); + const candidates: string[] = []; + for (const root of roots) { + candidates.push(path.join(root, "node_modules", "@vscode", "ripgrep-win32-x64", "bin", "rg.exe")); + candidates.push(path.join(root, "node_modules", "@vscode", "ripgrep-win32-arm64", "bin", "rg.exe")); + } + return candidates; +} + +export function isWslEnvironment(env: NodeJS.ProcessEnv = process.env) { + if (env.WSL_DISTRO_NAME || env.WSL_INTEROP) { + return true; + } + return false; +} + +export function toWindowsPathIfWslMount(value: string) { + const text = String(value ?? ""); + const match = text.match(/^\/mnt\/([a-zA-Z])\/(.*)$/); + if (!match) { + return text; + } + return `${match[1].toUpperCase()}:\\${match[2].replace(/\//g, "\\")}`; +} + +function usesWindowsExecutable(executable: string) { + return process.platform !== "win32" && String(executable ?? "").toLowerCase().endsWith(".exe"); +} + +function pathRgCandidates() { + const names = process.platform === "win32" ? ["rg.exe", "rg"] : ["rg"]; + const extensions = process.platform === "win32" + ? String(process.env.PATHEXT || ".EXE;.CMD;.BAT").split(";").filter(Boolean) + : [""]; + const dirs = String(process.env.PATH || "").split(path.delimiter).filter(Boolean); + const candidates: string[] = []; + for (const dir of dirs) { + for (const name of names) { + if (process.platform === "win32" && !path.extname(name)) { + for (const ext of extensions) { + candidates.push(path.join(dir, `${name}${ext}`)); + } + } else { + candidates.push(path.join(dir, name)); + } + } + } + return candidates; +} + +async function pathExists(filePath: string) { try { - return require("@vscode/ripgrep").rgPath; + await fs.access(filePath); + return true; } catch { - return null; + return false; } } diff --git a/src/tools/runtime.ts b/src/tools/runtime.ts index 6a0bc08..4011182 100644 --- a/src/tools/runtime.ts +++ b/src/tools/runtime.ts @@ -437,6 +437,11 @@ export function createToolRuntime(options: ToolRuntimeOptions) { approvedOutsideWorkspace: approvedByUser && asResultRecord(decision).outsideWorkspace === true } }); + if (isPlainObject(rawResult) && rawResult.ok === false) { + return finishTool(options, name, input, definition, options.signal?.aborted + ? interruptedToolExecution(name, input, definition, rawResult) + : rawResult); + } const result = Array.isArray(rawResult) ? rawResult : asResultRecord(rawResult); const resultFields = asResultRecord(Array.isArray(rawResult) ? EMPTY_RECORD : rawResult); if (name === "background_shell" && resultFields.started === true) { diff --git a/src/tools/web-tools.ts b/src/tools/web-tools.ts index 84706ec..b4850ca 100644 --- a/src/tools/web-tools.ts +++ b/src/tools/web-tools.ts @@ -12,6 +12,12 @@ const SEARCH_HEADERS = Object.freeze({ accept: "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "accept-language": "en-US,en;q=0.9" }); +const WIKIMEDIA_USER_AGENT = "Ant-Code/2.0.5 (https://github.com/wicm84266964/Ant-Code; local research agent)"; +const WIKIMEDIA_HEADERS = Object.freeze({ + "user-agent": WIKIMEDIA_USER_AGENT, + accept: "application/json" +}); +const WIKIPEDIA_TIMEOUT_MS = 8_000; const RAW_HTTP_HEADER_MAX_BYTES = 64 * 1024; const MAX_SEARCH_RESULTS = 10; const MAX_REDIRECTS = 5; @@ -100,24 +106,41 @@ export async function webSearchTool(input: Record) { const searxngUrl = normalizeOptionalUrl(input.searxngUrl ?? asRecord(config.web).searxngUrl ?? env?.LAB_AGENT_SEARXNG_URL); const errors: string[] = []; - const attempts: Array<[string, () => Promise<{ provider: string; query: string; url: string; results: SearchResult[]; truncated: boolean }>]> = []; + const batches: Array<{ provider: string; query: string; url: string; results: SearchResult[]; truncated: boolean }> = []; + const attempts: Array<[string, () => Promise<{ provider: string; query: string; url: string; results: SearchResult[]; truncated: boolean }>]> = [ + ["wikipedia", () => searchWikipedia({ query, maxResults, timeoutMs, config, env, signal })] + ]; if (searxngUrl) { attempts.push(["searxng", () => searchSearxng({ query, maxResults, timeoutMs, searxngUrl, config, env, signal })]); } + attempts.push(["bing", () => searchBing({ query, maxResults, timeoutMs, config, env, signal })]); attempts.push(["duckduckgo", () => searchDuckDuckGo({ query, maxResults, timeoutMs, config, env, signal })]); for (const [label, run] of attempts) { try { const result = await run(); if (result.results.length > 0) { - return result; + batches.push(result); + } else { + errors.push(`${label}: empty`); } - errors.push(`${label}: empty`); } catch (error) { errors.push(`${label}: ${error instanceof Error ? error.message : String(error)}`); } } + const merged = dedupeResults(batches.flatMap((batch) => batch.results)).slice(0, maxResults); + if (merged.length > 0) { + const total = batches.reduce((count, batch) => count + batch.results.length, 0); + return { + provider: batches.map((batch) => batch.provider).join("+"), + query, + url: batches[0]?.url ?? "", + results: merged, + truncated: batches.some((batch) => batch.truncated) || total > maxResults + }; + } + throw Object.assign(new Error(`Web search fetch failed (${errors.join("; ")})`), { code: "WEB_SEARCH_FETCH_FAILED" }); @@ -132,6 +155,9 @@ export function networkHostsForWebTool(name: string, input: Record; env?: NodeJS.ProcessEnv; signal?: AbortSignal }) { + const lang = wikipediaLanguageForQuery(query); + const endpoint = new URL(`https://${lang}.wikipedia.org/w/api.php`); + endpoint.searchParams.set("action", "query"); + endpoint.searchParams.set("list", "search"); + endpoint.searchParams.set("srsearch", query); + endpoint.searchParams.set("srlimit", String(maxResults)); + endpoint.searchParams.set("srprop", "snippet"); + endpoint.searchParams.set("format", "json"); + endpoint.searchParams.set("utf8", "1"); + const { body } = await fetchTextWithTimeout(endpoint.href, { + timeoutMs: Math.min(timeoutMs, WIKIPEDIA_TIMEOUT_MS), + maxBytes: DEFAULT_SEARCH_MAX_BYTES, + truncateOnLimit: true, + headers: WIKIMEDIA_HEADERS, + config, + env, + signal + }); + let json: unknown = {}; + try { + json = JSON.parse(body.text); + } catch { + json = {}; + } + const results = parseWikipediaQueryJson(json, lang).slice(0, maxResults); + return { + provider: `wikipedia-${lang}`, + query, + url: endpoint.href, + results, + truncated: body.truncated || results.length >= maxResults + }; +} + +export function wikipediaLanguageForQuery(query: string) { + return /[\u3400-\u9fff]/.test(query) ? "zh" : "en"; +} + +export function parseWikipediaQueryJson(json: unknown, lang: string) { + const search = asRecord(asRecord(json).query).search; + const rows = Array.isArray(search) ? search : []; + const results: SearchResult[] = []; + for (const item of rows) { + const row = asRecord(item); + const title = cleanText(row.title); + if (!title) { + continue; + } + results.push({ + title, + url: wikipediaArticleUrl(lang, title), + snippet: cleanText(stripHtml(row.snippet ?? "")), + engine: "wikipedia" + }); + } + return dedupeResults(results); +} + +function wikipediaArticleUrl(lang: string, title: string) { + return `https://${lang}.wikipedia.org/wiki/${encodeURIComponent(title.replace(/ /g, "_"))}`; +} + +function bingResultUrl(href: string, citeHtml: string) { + const direct = decodeHtml(href); + if (/^https?:\/\//i.test(direct) && !/\bbing\.com\b/i.test(direct)) { + return direct; + } + const cite = cleanText(stripHtml(citeHtml)).split(/[›·|]/)[0].trim(); + if (/^https?:\/\//i.test(cite)) { + return cite; + } + if (/^(www\.)?[\w.-]+\.[a-z]{2,}(\/.*)?$/i.test(cite)) { + return `https://${cite.replace(/^\/\//, "")}`; + } + return ""; +} + +async function searchBing({ query, maxResults, timeoutMs, config, env, signal }: { query: string; maxResults: number; timeoutMs: number; config?: Record; env?: NodeJS.ProcessEnv; signal?: AbortSignal }) { + const endpoint = new URL("https://www.bing.com/search"); + endpoint.searchParams.set("q", query); + if (wikipediaLanguageForQuery(query) === "zh") { + endpoint.searchParams.set("setlang", "zh-CN"); + } + const { body } = await fetchTextWithTimeout(endpoint.href, { + timeoutMs, + maxBytes: DEFAULT_SEARCH_MAX_BYTES, + truncateOnLimit: true, + headers: { + ...SEARCH_HEADERS, + "accept-language": wikipediaLanguageForQuery(query) === "zh" ? "zh-CN,zh;q=0.9,en;q=0.5" : SEARCH_HEADERS["accept-language"] + }, + config, + env, + signal + }); + const results = parseBingHtml(body.text).slice(0, maxResults); + return { + provider: "bing-html", + query, + url: endpoint.href, + results, + truncated: body.truncated || results.length >= maxResults + }; +} + +export function parseBingHtml(html: unknown) { + const text = String(html ?? ""); + const results: SearchResult[] = []; + const blocks = Array.from(text.matchAll(/]*class=["'][^"']*b_algo[^"']*["'][^>]*>([\s\S]*?)<\/li>/gi)); + for (const block of blocks) { + const body = block[1]; + const link = body.match(/]*>\s*]+href=["']([^"']+)["'][^>]*>([\s\S]*?)<\/a>/i); + if (!link) { + continue; + } + const cite = body.match(/]*>([\s\S]*?)<\/cite>/i); + const url = bingResultUrl(link[1], cite?.[1] ?? ""); + const title = cleanText(stripHtml(link[2])); + const snippetMatch = body.match(/]*>([\s\S]*?)<\/p>/i); + if (!title || !url) { + continue; + } + results.push({ + title, + url, + snippet: cleanText(stripHtml(snippetMatch?.[1] ?? "")), + engine: "bing" + }); + } + return dedupeResults(results); +} + async function searchSearxng({ query, maxResults, timeoutMs, searxngUrl, config, env, signal }: { query: string; maxResults: number; timeoutMs: number; searxngUrl: string; config?: Record; env?: NodeJS.ProcessEnv; signal?: AbortSignal }) { const endpoint = new URL("/search", searxngUrl); endpoint.searchParams.set("q", query); diff --git a/src/version.ts b/src/version.ts index 654c0d1..32502ed 100644 --- a/src/version.ts +++ b/src/version.ts @@ -2,7 +2,7 @@ import fs from "node:fs/promises"; import path from "node:path"; import { fileURLToPath } from "node:url"; -const FALLBACK_VERSION = "2.0.5"; +const FALLBACK_VERSION = "2.0.6"; /** * @param {NodeJS.ProcessEnv} [env] diff --git a/tests/unit/tool-result-view.test.ts b/tests/unit/tool-result-view.test.ts index 1ef9dbd..3e22f27 100644 --- a/tests/unit/tool-result-view.test.ts +++ b/tests/unit/tool-result-view.test.ts @@ -103,6 +103,36 @@ test("agent_run model view keeps the report and drops nested tool dumps", () => assert.doesNotMatch(view.text, /xxxx/); }); +test("skill_list, todo_read, rg_count, and empty mcp_list keep array payloads", () => { + const skills = renderToolResultView("skill_list", { + ok: true, + result: [ + { name: "codebase-orientation", description: "orient" }, + { name: "web-research", description: "search" } + ] + }); + assert.match(skills.text, /skills=2/); + assert.match(skills.text, /codebase-orientation/); + assert.equal(skills.text.includes("\n{}"), false); + + const todos = renderToolResultView("todo_read", { + ok: true, + result: [{ id: "1", status: "pending", content: "inspect workspace" }] + }); + assert.match(todos.text, /todos=1/); + assert.match(todos.text, /inspect workspace/); + + const count = renderToolResultView("rg_count", { + ok: true, + result: { command: "rg --count-matches", mode: "matches", count: 9 } + }); + assert.match(count.text, /count=9/); + assert.equal(count.text.includes("matches=0"), false); + + const mcp = renderToolResultView("mcp_list", { ok: true, result: [] }); + assert.match(mcp.text, /servers=0/); +}); + test("hard safety valve still truncates a huge view", () => { const serialized = formatToolResultForModel("bash", { ok: true, diff --git a/tests/unit/tools.test.ts b/tests/unit/tools.test.ts index 8917be5..1512b2b 100644 --- a/tests/unit/tools.test.ts +++ b/tests/unit/tools.test.ts @@ -8,7 +8,7 @@ import path from "node:path"; import test from "node:test"; import { promisify } from "node:util"; import { editFileTool, globTool, grepTool, listFilesTool, readFileTool, writeFileTool } from "../../src/tools/file-tools.ts"; -import { formatRgCloseResult, rgCountTool, rgFilesTool, rgFilesWithMatchesTool, rgSearchTool, windowsReservedDeviceGlobArgs } from "../../src/tools/rg-tools.ts"; +import { bundledRgCandidates, formatRgCloseResult, rgCountTool, rgFilesTool, rgFilesWithMatchesTool, rgSearchTool, windowsReservedDeviceGlobArgs } from "../../src/tools/rg-tools.ts"; import { tsDiagnosticsTool, tsFindDefinitionTool, tsFindReferencesTool, tsSymbolsTool } from "../../src/tools/semantic-tools.ts"; import { scrubEnvironment } from "../../src/tools/env-scrubber.ts"; import { createMcpRuntime } from "../../src/mcp/runtime.ts"; @@ -17,7 +17,10 @@ import { serializeToolResult } from "../../src/tools/result.ts"; import { createToolRuntime } from "../../src/tools/runtime.ts"; import { collectRawHttpResponse, + parseBingHtml, parseDuckDuckGoHtml, + parseWikipediaQueryJson, + wikipediaLanguageForQuery, readResponseText, WEB_FETCH_DEFAULT_MAX_BYTES, webFetchTool @@ -205,7 +208,9 @@ test("rg_search can use bundled ripgrep when PATH is unavailable", async () => { const previousPath = process.env.PATH; try { process.env.PATH = ""; + assert.equal(bundledRgCandidates().some((item) => item.endsWith("rg.exe") || item.endsWith(`${path.sep}rg`)), true); const result = await rgSearchTool({ cwd, pattern: "marker", glob: ["*.ts"] }); + assert.equal("error" in result, false); assert.equal(result.matches.some((item) => item.path === "sample.ts"), true); } finally { process.env.PATH = previousPath; @@ -1902,6 +1907,47 @@ test("duckduckgo html parser extracts result titles and URLs", () => { assert.match(results[0].snippet, /Official documentation/); }); +test("wikipedia search parser maps MediaWiki JSON to article URLs", () => { + const results = parseWikipediaQueryJson({ + query: { + search: [ + { + title: "Camponotus", + snippet: "A genus of ants in the subfamily Formicinae." + } + ] + } + }, "en"); + + assert.equal(wikipediaLanguageForQuery("Camponotus japonicus"), "en"); + assert.equal(wikipediaLanguageForQuery("日本弓背蚁"), "zh"); + assert.equal(results.length, 1); + assert.equal(results[0].title, "Camponotus"); + assert.equal(results[0].url, "https://en.wikipedia.org/wiki/Camponotus"); + assert.equal(results[0].engine, "wikipedia"); + assert.match(results[0].snippet, /genus of ants/); +}); + +test("bing html parser extracts result titles and URLs", () => { + const results = parseBingHtml(` +
  • +

    Solenopsis invicta

    + https://www.antweb.org › description +

    Red imported fire ant species page.

    +
    • +
  • +

    Tracker

    + www.example.com/fire-ants +

    Wrapped result.

    +
    • + `); + + assert.equal(results.length, 2); + assert.equal(results[0].url, "https://www.antweb.org/description.do?genus=solenopsis"); + assert.equal(results[0].engine, "bing"); + assert.equal(results[1].url, "https://www.example.com/fire-ants"); +}); + test("document_intake extracts markdown from local HTML documents", async () => { const cwd = await makeTempWorkspace(); await fs.writeFile(path.join(cwd, "report.html"), "

    Report

    Alpha beta.

    ", "utf8");