fix: atomic incremental snapshots and reliable CLI output (1.17.1)

.claude-plugin/marketplace.json

@@ -13,7 +13,7 @@
       "name": "sensegrep",
       "source": "./plugin/sensegrep-plugin",
       "description": "Semantic code search for AI agents. Search code by meaning, not text patterns. Adds sensegrep MCP tools + smart usage instructions to Claude Code.",
-      "version": "1.17.0",
+      "version": "1.17.1",
       "author": {
         "name": "sensegrep"
       },

.cursor-plugin/marketplace.json

@@ -13,7 +13,7 @@
       "name": "sensegrep",
       "source": "plugin/sensegrep-cursor",
       "description": "Semantic code search for AI agents. Search code by meaning, not text patterns.",
-      "version": "1.17.0"
+      "version": "1.17.1"
     }
   ]
 }

docs/reliability-release-1.17.1.md

@@ -0,0 +1,18 @@
+# Reliability fixes validated on Windows
+
+## Behavior
+
+- Incremental CLI indexing copies existing vectors in Arrow batches into a staging generation, applies replacements/removals, verifies its count, and atomically switches the metadata pointer. Failure before activation leaves the previous generation intact. No-change runs do not copy the table. This adds local disk work for changed indexes, but does not re-embed unchanged chunks. Single-file watcher updates are outside this change.
+- CLI duplicate JSON now applies `--limit` in minimal, diagnostic and full projections. `summary.total` is the number found; `summary.returned` is the number emitted (`totalDuplicates`/`returnedDuplicates` in full). Output truncation sets `status: incomplete`, `truncated` and `outputTruncated`; it does not invent a continuation cursor. A scan cursor, when present, is preserved. Raise `--limit` to see more groups from the same scan.
+- Default search returns at most one result per file. Explicit `--max-per-file` overrides this, and `--exact` retains the two-per-file default. Context selection is unchanged and may select several relevant helpers per file.
+- Group labels retain up to seven symbol words instead of four and remove dangling English prepositions/conjunctions.
+
+## Validation
+
+- Typecheck passed; full suite passed with 238 tests, followed by the added no-op regression test (14 indexer tests passed). Release CI runs the complete final suite of 239 tests.
+- A real child CLI using Ollama was terminated during incremental persistence. Strict verification found stale source files but **no chunk mismatch**, and the active table name was unchanged. The next incremental run activated a new table; strict verification and exact lookup of the added symbol passed.
+- Repeated the 16 CuraAI query cases in default and explicitly diverse modes. Default expected-file top-five coverage improved from 13/16 to 15/16; top-ten remained 16/16. The voice concurrency case improved from rank 10 to 6, so broad semantic retrieval is still not exhaustive.
+- Exercised all three duplicate JSON projections with `--limit 1`, checking one emitted group and explicit output truncation.
+- Rechecked small context budgets, real call references, cluster labels and index health using the built CLI before release. The installed npm CLI is checked again after publication.
+
+The initial source corpus changed during earlier testing; use these measurements as targeted acceptance evidence, not a universal accuracy claim. `--exact` still prefers exact symbols rather than imposing a strict filter; use `literal` to establish exhaustive textual absence.

packages/cli/CHANGELOG.md

@@ -1,5 +1,14 @@
 # @sensegrep/cli
 
+## 1.17.1
+
+### Patch Changes
+
+- [`063d867`](https://github.com/Stahldavid/sensegrep/commit/063d867e85b36f7e3153f376fe9fceba0d1e50ad) Thanks [@Stahldavid](https://github.com/Stahldavid)! - Keep the active index consistent when incremental indexing is interrupted by staging vector changes before atomically switching metadata. Preserve the no-change fast path and reuse existing embeddings. Honor duplicate result limits in every CLI JSON projection with explicit output truncation. Improve default search file diversity and retain meaningful words in cluster labels.
+
+- Updated dependencies [[`063d867`](https://github.com/Stahldavid/sensegrep/commit/063d867e85b36f7e3153f376fe9fceba0d1e50ad)]:
+  - @sensegrep/core@1.17.1
+
 ## 1.17.0
 
 ### Minor Changes

packages/cli/README.md

@@ -51,3 +51,11 @@ structured warning instead of failing silently.
 - CLI reference: https://github.com/Stahldavid/sensegrep/blob/main/docs/cli-reference.md
 - Getting started: https://github.com/Stahldavid/sensegrep/blob/main/docs/getting-started.md
 - Issues: https://github.com/Stahldavid/sensegrep/issues
+
+### Reliability in 1.17.1
+
+Search defaults to one result per file; use `--max-per-file 2` for more snippets from each file. `--exact` preserves its two-per-file default and prefers exact symbols rather than excluding approximate matches.
+
+Duplicate JSON respects `--limit`. `summary.outputTruncated` means more groups were found than emitted; raise `--limit` to expose them. A continuation cursor advances the candidate scan, not output pagination. Candidate caps still require raising `--max-candidates` for full coverage.
+
+Incremental `index --no-watch` stages updates before atomically activating the new snapshot, preserving the old index if the process is interrupted. No-change runs avoid copying vectors. This protection does not yet extend to individual watcher updates.

packages/cli/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@sensegrep/cli",
-  "version": "1.17.0",
+  "version": "1.17.1",
   "type": "module",
   "bin": {
     "sensegrep": "dist/main.js"
@@ -25,7 +25,7 @@
     "node": ">=20"
   },
   "dependencies": {
-    "@sensegrep/core": "^1.17.0"
+    "@sensegrep/core": "^1.17.1"
   },
   "publishConfig": {
     "access": "public"

packages/cli/src/main.ts

@@ -860,7 +860,7 @@ async function run() {
 
     if (flags.json) {
       const detail = resolveJsonProjection(flags["json-detail"] ?? flags.jsonDetail, flags.diagnostic === true)
-      writeJson(projectDuplicateResponse(result, detail, showCode || fullCode))
+      writeJson(projectDuplicateResponse(result, detail, showCode || fullCode, limit))
       return
     }
 

packages/cli/src/search-commands.test.ts

@@ -120,6 +120,19 @@ describe("search command agent JSON contracts", () => {
     expect(projectDuplicateResponse(result, "minimal", true).duplicates[0].instances[0]).toHaveProperty("content", "secret code")
   })
 
+  it("limits every duplicate JSON projection without losing scan continuation", () => {
+    const raw = { status: "complete", summary: { totalDuplicates: 3, returnedDuplicates: 3, resumeCursor: "next" }, duplicates: [{ instances: [] }, { instances: [] }, { instances: [] }] }
+    for (const detail of ["minimal", "diagnostic", "full"] as const) {
+      const result = projectDuplicateResponse(raw, detail, false, 1)
+      expect(result.duplicates).toHaveLength(1)
+      expect(result.status).toBe("incomplete")
+      expect(result.summary.outputTruncated).toBe(true)
+      expect(detail === "full" ? result.summary.returnedDuplicates : result.summary.returned).toBe(1)
+      expect(detail === "full" ? result.summary.resumeCursor : result.continuation.cursor).toBe("next")
+    }
+    expect(raw.duplicates).toHaveLength(3)
+  })
+
   it("projects literal, graph, and show with canonical locations", () => {
     const literal = projectLiteralResponse({
       command: "literal",

packages/cli/src/search-commands.ts

@@ -41,7 +41,11 @@ export function projectSearchResponse(res: any, detail: JsonProjection, includeR
   })
 }
 
-export function projectDuplicateResponse(res: any, detail: JsonProjection, includeCode: boolean): any {
+export function projectDuplicateResponse(res: any, detail: JsonProjection, includeCode: boolean, limit?: number): any {
+  if (limit !== undefined && res.duplicates.length > limit) {
+    res = { ...res, status: "incomplete", duplicates: res.duplicates.slice(0, limit),
+      summary: { ...res.summary, returnedDuplicates: limit, truncated: true, outputTruncated: true } }
+  }
   return projectDuplicateAgentResponse(res, { detail, diagnostics: detail === "diagnostic", includeCode })
 }
 

packages/cli/src/usage.ts

@@ -40,7 +40,7 @@ Search options:
   --min-complexity <n>      Minimum cyclomatic complexity
   --max-complexity <n>      Maximum cyclomatic complexity
   --min-score <n>           Minimum relevance score 0-1
-  --max-per-file <n>        Max results per file (default: 2)
+  --max-per-file <n>        Max results per file (search default: 1; --exact: 2)
   --max-per-symbol <n>      Max results per symbol (default: 2)
   --has-docs <true|false>   Require documentation
   --language <lang>         typescript|javascript|python|java|vue (comma-separated for multiple)

packages/core/CHANGELOG.md

@@ -1,5 +1,11 @@
 # @sensegrep/core
 
+## 1.17.1
+
+### Patch Changes
+
+- [`063d867`](https://github.com/Stahldavid/sensegrep/commit/063d867e85b36f7e3153f376fe9fceba0d1e50ad) Thanks [@Stahldavid](https://github.com/Stahldavid)! - Keep the active index consistent when incremental indexing is interrupted by staging vector changes before atomically switching metadata. Preserve the no-change fast path and reuse existing embeddings. Honor duplicate result limits in every CLI JSON projection with explicit output truncation. Improve default search file diversity and retain meaningful words in cluster labels.
+
 ## 1.17.0
 
 ### Minor Changes

packages/core/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@sensegrep/core",
-  "version": "1.17.0",
+  "version": "1.17.1",
   "type": "module",
   "main": "./dist/index.js",
   "types": "./dist/index.d.ts",

packages/core/src/semantic/indexer.test.ts

@@ -1,5 +1,5 @@
 import path from "node:path"
-import { mkdir, rm, writeFile } from "node:fs/promises"
+import { mkdir, rm, writeFile, stat } from "node:fs/promises"
 import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"
 
 const TEST_DIR = path.join(process.cwd(), ".test-indexer")
@@ -18,6 +18,7 @@ const replaceFileDocuments = vi.fn()
 const updateDocuments = vi.fn()
 const writeIndexMeta = vi.fn()
 const deleteCollection = vi.fn()
+const copyCollection = vi.fn()
 const createStagingCollection = vi.fn()
 const openCollectionReadOnly = vi.fn()
 const dropCollectionTable = vi.fn()
@@ -57,6 +58,7 @@ vi.mock("./lancedb.js", () => ({
     writeIndexMeta,
     deleteCollection,
     createStagingCollection,
+    copyCollection,
     openCollectionReadOnly,
     dropCollectionTable,
     cleanupInactiveTables,
@@ -210,11 +212,42 @@ describe("Indexer incremental updates", () => {
     expect(updateDocuments).not.toHaveBeenCalled()
     expect(embedDocumentsReusingFile).toHaveBeenCalledTimes(1)
     expect(embedDocuments).toHaveBeenCalledTimes(1)
-    expect(replaceFileDocuments).toHaveBeenCalledWith({}, "src/a.ts", expect.any(Array))
+    expect(replaceFileDocuments).toHaveBeenCalledWith({ staging: true }, "src/a.ts", expect.any(Array))
     expect(writeIndexMeta).toHaveBeenCalledTimes(1)
     expect(writeIndexMeta.mock.calls[0][1].chunking).toEqual(testChunkingSignature)
   })
 
+  it("keeps the active snapshot untouched when staged persistence fails", async () => {
+    replaceFileDocuments.mockRejectedValueOnce(new Error("disk failure"))
+    const { Indexer } = await import("./indexer.js")
+    await expect(Indexer.indexProjectIncremental()).rejects.toThrow("disk failure")
+    expect(copyCollection).toHaveBeenCalledWith({}, { staging: true }, expect.anything())
+    expect(replaceFileDocuments).toHaveBeenCalledWith({ staging: true }, "src/a.ts", expect.any(Array))
+    expect(writeIndexMeta).not.toHaveBeenCalled()
+    expect(dropCollectionTable).toHaveBeenCalledWith(TEST_DIR, "chunks_staging")
+  })
+
+  it("does not activate a generation if its metadata cannot be committed", async () => {
+    writeIndexMeta.mockRejectedValueOnce(new Error("metadata failure"))
+    const { Indexer } = await import("./indexer.js")
+    await expect(Indexer.indexProjectIncremental()).rejects.toThrow("metadata failure")
+    expect(dropCollectionTable).toHaveBeenCalledWith(TEST_DIR, "chunks_staging")
+    expect(clearProjectCache).not.toHaveBeenCalled()
+  })
+
+  it("keeps no-op indexing on the active generation without copying vectors", async () => {
+    const meta = await readIndexMeta()
+    const current = await stat(path.join(TEST_DIR, "src/a.ts"))
+    meta.files["src/a.ts"].size = current.size
+    meta.files["src/a.ts"].mtimeMs = current.mtimeMs
+    const { Indexer } = await import("./indexer.js")
+    const result = await Indexer.indexProjectIncremental()
+    expect(result).toMatchObject({ files: 0, skipped: 1 })
+    expect(createStagingCollection).not.toHaveBeenCalled()
+    expect(copyCollection).not.toHaveBeenCalled()
+    expect(replaceFileDocuments).not.toHaveBeenCalled()
+  })
+
   it("plans embedding work without calling the provider or mutating the index", async () => {
     const { Indexer } = await import("./indexer.js")