grep: restore as an eval-only builtin backed by an in-process native engine with six-host prebuilds

[code-yeongyu]

Summary

Restore grep as an eval-only builtin backed by an in-process native engine, with six-host prebuilds for supported release artifacts.

Reproduction

In an eval cell, tool.grep returns Unknown tool grep; tool_schema("grep") returns not found.

Expected

Ideal state = Scope Must have, sanitized: eval can call tool.grep with a stable schema, the builtin remains eval-only, the native engine is in-process, and release artifacts provide the required host prebuilds.

Actual

The grep builtin is withheld from the exposed tool set, so eval callers cannot discover or invoke it.

Evidence

• packages/coding-agent/src/core/tools/index.ts:107-114
• packages/coding-agent/src/core/agent-session.ts:7548-7551,7612-7618
• PR feat(coding-agent): temporarily withhold grep and reframe the eval graph section as workflow #1186

Root cause

The current exposure and workflow guidance intentionally withhold grep from eval, leaving no native eval-only implementation or packaging path.

Scope

Must have:

• expose grep only to eval with a stable schema and direct-call guidance;
• add an in-process native engine with an engine-agnostic layer;
• add six-host prebuild packaging and release verification;
• preserve existing permission, output, pagination, and cancellation contracts.

Acceptance criteria:

• eval discovery and invocation succeed while non-eval surfaces remain unchanged;
• native and fallback engines pass parity tests for matching, limits, errors, and cancellation;
• release artifacts contain the required host prebuilds and package verification passes;
• documentation and CI checks cover the contract.

Related

• feat(coding-agent): temporarily withhold grep and reframe the eval graph section as workflow #1186 (withhold grep and reframe the eval graph section as workflow)
• feat(coding-agent): add experimental.bashEvalOnly to route shell tools through eval #1192 (bashEvalOnly)
• feat(coding-agent): add experimental.workflowEvalOnly to route the workflow tool through eval #1198 (workflowEvalOnly)

[code-yeongyu]

S1 progress: restored eval-only surface verified

PR: #1684

Root cause confirmed: the temporary grep filter removed it from the executable/discovery catalog, so eval and schema discovery could not reach it. S1 removes that filter and uses declared exposure: "eval" instead of another hard-coded tool name.

RED/GREEN evidence:

• Declared exposure and grep schema restoration have passing lane regressions.
• Final SDK QA found that the terminal bash replacement also needed the declaration. The regression failed with expected eval, received direct, then passed after the one-field correction.
• Static checks and the coding-agent build pass. Targeted results: coding-agent/docs 147 tests, codemode 33 tests, terminal plus grep/bash policy 31 tests.

Real SDK captures on darwin-arm64:

• return await tool.grep({ pattern: "temporarilyDisabled", path: "." }) returned repository match rows through actual codemode and grep.
• A scripted model-issued direct grep call returned the tool.grep(...) hint. The grep execution-hook count stayed at 1 before and after the direct attempt, proving that it did not execute.
• A child session with allowlist ["read","grep"] kept grep active without eval and returned No matches found on a direct search.

Receipts are in local-ignore/qa-evidence/task-8/ in the task worktree. The PR is open; current base changes require branch integration before the required CI jobs can run. The broader native engine and packaging work remains tracked here, so this issue stays open.

[code-yeongyu]

S1 CI failures diagnosed and repaired

PR #1684's first full CI run passed static checks, workspace tests, all Windows jobs, changelog and review gates, but failed the coding-agent shards. The failures were caused by stale pre-restoration expectations, not runner instability.

The seven failing files reproduced locally: 18 failures, 42 passes. They now pass all 60 tests:

• Exact default-tool arrays now include grep and preserve its order across model transitions, lifecycle payloads, RPC model switches and exclusions.
• Dedicated promotion fixtures explicitly start without grep, so adding it still exercises a real activation rather than an already-active default.
• Budget assertions check the updated numeric projection and exact live-context subtraction, not brittle error prose or a widened numeric band.
• The MCP resident-cost assertion retains its original bound and measures the MCP tool set, as intended, rather than counting unrelated core tool schemas.

The follow-up is test-only; production sources are identical to the independently verified SDK snapshot. npm run check and the coding-agent build pass again. The updated head is undergoing a fresh full CI run; no blind rerun and no CI exception was used.

[code-yeongyu]

S1 merged and verified

PR #1684 merged into main at 2026-09-14T06:07:34Z.

Merge commit: 864052294263c2da09ad956c4b9b45fbdf097833.
Verified branch head: 030ec5270998e9e085f4377895b7f8d2b4f7a817.

All required checks passed: Check and test, Changelog gate, and Review claim gate. All three coding-agent shards, workspace/script tests, static checks, Windows RPC/trust/compiled-extension checks, and the platform terminal/inspector jobs are green. Normal merge succeeded; no admin bypass or CI exception was needed.

Independent darwin-arm64 SDK QA confirmed:

• tool.grep executes through a real eval cell and returns match rows.
• A direct model grep call returns the eval-form hint without executing; the execution-hook count remains unchanged.
• A child allowlist without eval keeps grep active and directly executable.

Docs, release notes and fork trackers describe the declared eval-only policy. The terminal replacement preserves bash's eval exposure. The initial CI failures were reproduced and fixed without changing runtime behavior, widening bounds or skipping tests.

Receipts: local-ignore/qa-evidence/task-8/ in the task worktree. The independent QA snapshot was removed after verification. This completes S1 and unblocks the dependent engine-layer integration; the broader grep restoration/native-engine issue remains open for the remaining lanes.

[code-yeongyu]

S2 native crate: prebuild matrix and downloaded-artifact QA complete

PR-S2: #1689 (feat(native): add the senpi-grep in-process search crate).

Native Prebuilds run: https://github.com/code-yeongyu/senpi/actions/runs/34813670341

All six Build rows succeeded, including best-effort win32-arm64. No target was dropped, no compile fix was needed, and no Windows ARM follow-up is necessary.

Downloaded artifact listing (each directory also contains manifest.txt):

Artifact directory	PTY addon	Grep addon	Architecture verified by file
native-prebuild-darwin-arm64	senpi_pty.darwin-arm64.node	senpi_grep.darwin-arm64.node	Mach-O arm64
native-prebuild-darwin-x64	senpi_pty.darwin-x64.node	senpi_grep.darwin-x64.node	Mach-O x86_64
native-prebuild-linux-arm64	senpi_pty.linux-arm64-gnu.2.17.node	senpi_grep.linux-arm64-gnu.2.17.node	ELF aarch64
native-prebuild-linux-x64	senpi_pty.linux-x64-gnu.2.17.node	senpi_grep.linux-x64-gnu.2.17.node	ELF x86-64
native-prebuild-win32-arm64	senpi_pty.win32-arm64-msvc.node	senpi_grep.win32-arm64-msvc.node	PE32+ Aarch64
native-prebuild-win32-x64	senpi_pty.win32-x64-msvc.node	senpi_grep.win32-x64-msvc.node	PE32+ x86-64

Verified exactly two .node files in each directory and that both file_senpi_pty= and file_senpi_grep= name the actual files. Platform-tag filenames are preserved; release staging owns renaming.

Validation:

• npm run check: exit 0; npm run build: exit 0.
• cargo test -p senpi-grep --locked: 21 passed, zero failed or ignored.
• node --test scripts/probe-native-grep.test.mjs scripts/bench-grep.test.mjs: eight passed, zero failed or skipped.
• node scripts/probe-native-grep.mjs native-prebuild-darwin-arm64/senpi_grep.darwin-arm64.node: exit 0, sentinel=1 matches=3 aborted=ABORTED timeout=timedOut probeFiles=5000 on two darwin-arm64 hosts, including independent QA with matching SHA-256 digests.
• Wrong-architecture control: the same probe against native-prebuild-darwin-x64/senpi_grep.darwin-x64.node exits 1 with ERR_DLOPEN_FAILED and an incompatible-architecture diagnostic on both hosts.

PR #1689 merged at 2026-09-14T06:39:13Z as b9b258d38087909df0dcd9702ceb3f83c4eb174f. All required checks passed: Check and test, Changelog gate, and Review claim gate. The full CI run succeeded. Normal merge was used, with no admin bypass or CI exception and no extra task commit.

[code-yeongyu]

S3 landed: engine-backed grep with a both-engine CI gate

PR #1693 is merged (merge commit 56a09310). The TS engine layer, the rebuilt tool, the renderers, the Cursor bridge mapping, and the CI contract job all ship in it.

New CI job grep-native-contract (ubuntu-latest), required through Check and test

It reads the toolchain channel from rust-toolchain.toml, caches cargo state with a pinned Swatinem/rust-cache, runs cargo build -p senpi-grep --release --locked plus napi build --platform --release, resolves the produced addon by glob (ls crates/senpi-grep/senpi_grep.*.node - the host triple is never hardcoded), and runs every test/grep file twice in the same job. The native leg writes a vitest JSON report, and the job fails if the native contract file is missing, has a non-passing case, or reports a skip.

Both-engine evidence from the merged run:

engine=native
 Test Files  9 passed (9)
      Tests  128 passed (128)
engine=native engine-contract.native.test.ts: 20 passed, 0 skipped
engine=rg
 Test Files  9 passed (9)
      Tests  128 passed (128)

Job duration 2m7s; every required check passed on the merged head, including the three coding-agent shards and all six native prebuild jobs. No CI exception was used.

Three defects this job surfaced on a clean runner, fixed in the same PR

1. The fixture corpus ran git commit with no identity, so every corpus-backed contract case failed on a runner (fatal: empty ident name). Identity now travels through GIT_AUTHOR_*/GIT_COMMITTER_*, which outrank -c user.*, with commit signing disabled. Control captured: the old command fails, the new one commits.
2. grep-eval-only still pinned the pre-engine No matches found text; it now matches the frozen footer grammar.
3. The model usability budget projection moved with the rebuilt grep schema (240 schema units instead of 256).

Scope note: the Windows jobs still run enumerated files only and gained no test/grep coverage; that follow-up is tracked separately. Windows users still receive the win32-x64 native prebuild through the release.