release: v1.4.0 - #261
Merged
Merged
release: v1.4.0#261
Conversation
This repo lives at codesearch.git as a bare+working-tree hybrid (full checked-out tree + .git/index, but core.bare=true in .git/config). core.bare intermittently resets to true — VS Code's git integration rewrites .git/config on ref changes — and when it does, cargo's source fingerprinting aborts every build with 'did not expect repo ...\.git to be bare', breaking copy-to-common.ps1 -> build.ps1 -> cargo build. build.ps1 now forces core.bare=false right after Set-Location, before any cargo invocation. Idempotent and harmless for a normal (truly non-bare) checkout; non-fatal if git is unreachable.
fix(build): build.ps1 self-heals core.bare before cargo (no more 'did not expect repo to be bare')
) The 8GB hard cap (MAX_LMDB_MAP_SIZE_MB=8192) was too low for very large corpora — GitHub issue #189 shows a 1GB / 53k-file cargo-registry source with >1.2M chunks legitimately exceeding it, crashing after auto-resize exhausts ("already at max size 8192MB" → fatal MDB_MAP_FULL). - constants.rs: raise MAX_LMDB_MAP_SIZE_MB 8192→32768 (32GB). On 64-bit Linux/macOS the mapsize is just a VA reservation (free until written); on Windows the file may be pre-allocated but only to the grown size, which only happens on demand when MDB_MAP_FULL bites. - constants.rs: add max_lmdb_map_size_mb() reading the new CODESEARCH_MAX_LMDB_MAP_SIZE_MB env var (clamped to >= default), so operators with extreme corpora or Windows instances can tune the cap without rebuilding. - store.rs: route the 5 runtime cap comparisons (pin_map_size, resize_environment check+message, build_index, delete_chunks, insert_chunks_with_ids) through max_lmdb_map_size_mb(). The cap test is now env-aware (asserts against the resolved fn, not the const). Stage 1 of 3 for #189. Stage 2 adds the same auto-resize to PersistentEmbeddingCache (currently hardcoded 512MB, no resize).
🐛 fix: raise LMDB mapsize cap 8GB→16GB + PersistentEmbeddingCache auto-resize on MDB_MAP_FULL (#189)
…ed error messages
When the LMDB mapsize auto-resize cap is reached (7 sites: 4 in
vectordb/store.rs, 3 in embed/cache.rs), the error/warn messages said
'already at max size {}MB' or 'exceeds MAX_LMDB_MAP_SIZE_MB {}MB' but
never told the operator how to raise the cap. Appended '(set
CODESEARCH_MAX_LMDB_MAP_SIZE_MB to raise this cap)' to all 7 messages,
using backslash line-continuation per the repo's caller-facing-literal
convention (validated by tests/caller_facing_literals.rs - all 4 pass).
- CHANGELOG: renamed unreleased section 1.2.1 -> 1.2.4 (current pending version) and added the #189 fix entry (cap raise + cache auto-resize + error-message hint), alongside the existing build.ps1 entry. - README: documented the new CODESEARCH_MAX_LMDB_MAP_SIZE_MB env var in the Environment Variables table. PRs #187 and #176 (also merged since v1.2.0) were docs/AGENTS.md-only changes with no user-facing code impact, so intentionally have no CHANGELOG entries.
…remove dead PR trigger - test-linux: remove `cargo build --release` (nothing consumes it; release.yml builds isolated per-platform artifacts at tag time). Was pure wasted CI minutes on every feature/develop push. - Add `master` to push triggers: the develop->master release squash-merge previously got ZERO CI build/test coverage (protect-master.yml only checks the source-branch name, not code quality). - Remove the pull_request: branches: [main] trigger: this repo has no `main` branch (default is `master`), and PRs target develop/master anyway — the trigger was permanently dead. push already covers every commit that matters.
master only ever advances via a squash-merged release PR (develop/release/* -> master, per RELEASING.md). A direct git push to master is always either a mistake or should go through that same PR path anyway. GitHub's branch ruleset already blocks non-admin pushes, but the repo owner's own bypass privilege makes that toothless against exactly this kind of accidental local push -- which happened this session (a commit landed on master while HEAD was there for the release flow, caught only by manual inspection). Adds scripts/pre-push (tracked) with the master-push guard + QC gate, and installs the equivalent (hand-maintained superset, customer-ref-leak check stays local-only per existing convention) to .git/hooks/pre-push. Verified: simulated stdin push to refs/heads/master exits 1 with the guard message; refs/heads/develop passes through unaffected.
…s path gate
Two changes to .github/workflows/ci.yml, both reducing wasted CI minutes
without losing coverage:
1. concurrency: ci-${{ github.ref }}, cancel-in-progress: true — a stale
in-flight run on the same branch (e.g. a stage commit immediately
followed by a review-fix amend + re-push, which happened repeatedly
this session) now gets cancelled instead of running to completion for
a commit that is already superseded.
2. csharp-integration-tests now checks (via git diff on the pushed range,
not a fragile commit-message convention) whether helpers/csharp/**
actually changed before running its dotnet publish + cargo test cycle.
That job was previously running unconditionally on every single push
regardless of relevance — a full self-contained dotnet publish plus a
cargo test compile for code nobody touched.
Not changed (deliberately): test-lib running on both linux and windows is
cross-platform coverage, not duplication — kept as-is. fmt/clippy running
both locally (pre-push hook) and in CI is intentional defense-in-depth
(CI cannot trust a hook that can be bypassed via --no-verify).
Diagnosed a user report: a local 'codesearch serve' instance was silently keeping a mounted cloud federation peer warm, defeating its scale-to-zero, with zero trace in the logs. Traced and ruled out TUI federated polling (correctly gated behind --no-tui already) and explicit federated tool calls (none logged). Root cause: the cloud keep-warm task (CODESEARCH_KEEP_WARM_URL / --keep-warm-url) is not gated by --no-tui at all, has no restriction that its target must be 'self', and every ping was completely silent (success and failure both discarded with zero log line) -- so a keep-warm URL accidentally pointing at another peer (e.g. copy-pasted from a cloud deployment env into a local shell profile) would silently generate periodic outbound traffic every KEEP_WARM_INTERVAL_SECS (120s -- matches the reported 'every 2 minutes') with no way to see it in the local logs. Fixes: - Per-ping logging: debug! on success, warn! on failure (was: silently discarded). - Startup sanity check: new extract_host_from_url() helper (no new crate dependency) compares the keep-warm target's host against this server's own effective bind host; a mismatch (and not localhost/127.0.0.1/::1) now fires a loud warn! naming both hosts, explicit that keep-warm exists to self-ping THIS replica, not another peer. Tests: 6 new cases for extract_host_from_url (plain http, https with real hostname, no-scheme input, IPv6 literal bracket-preserving, query/fragment stripping, empty-host edge case). Full serve:: suite green (104 passed). Diagnosis write-up: docs/diagnose-federated-keep-warm.md
The embedded serve TUI ran a background `/status` fan-out to every mounted federated peer on a cadence equal to the LOCAL serve's idle-suspend window (2h by default). Each such poll WOKE the peer's scale-to-zero replica, which then held itself warm for its own full idle window (~1h on the cloud deploy) — roughly a 50% duty cycle on a peer nobody had queried. Ground truth from Azure Log Analytics: wakes exactly 120/121/120 minutes apart, each warm period ~67 min, with zero federated searches. The design spec was that background polling of LOCAL repos is fine but a FEDERATED peer must never be polled in the background. "Cannot keep a peer awake past the host's own suspend term" is a strictly weaker property than "never wakes it", and the two windows were unrelated values besides (local host vs. remote peer). - tui.rs: `spawn_remote_discovery` no longer polls on a timer. The periodic tick is now CONFIG-ONLY (`REMOTE_ROW_REFRESH_SECS` = 5s, zero HTTP): it rebuilds rows from the `remote_mounts` allowlist so mount/unmount edits and `l` reloads surface promptly. The activity poke (a real federated tool call landed on that peer) remains the only thing that ever contacts a peer, plus the explicit `i` info-overlay keypress. The `initial_cycle` startup gate is gone — every cycle is now config-only, so it had nothing left to gate. - serve/mod.rs: drop `ServeState::idle_suspend_secs` (field, env init, getter and the `--idle-suspend-secs` override). It existed solely to feed the TUI poll cadence and is now write-only. The keep-warm task reads the flag/env directly, so `--idle-suspend-secs` behaviour is unchanged. - Doc comments record WHY there is no baseline poll, to stop the reasoning from being reintroduced. Local repos are entirely unaffected. Review-fixes: - [Important] tui_common.rs `activity_stale` doc still referenced "the slow baseline poll hasn't fired" as the reason a remote row goes stale → rewritten to state there is no background poll and that `-` on an idle mount is the normal steady state, not a fault. - [Important] AGENTS.md and CHANGELOG.md still asserted the removed `idle_suspend_secs` cadence as current design (and named a field this commit deletes) → corrected, but deliberately NOT folded in here: both files carry a large unrelated pending doc-cleanup rewrite that must not enter a source commit. They land in the docs commit later on this same unpushed branch. - [Pre-existing, fixed opportunistically] the discovery snapshot was gated on `!cfg.remotes.is_empty()`, so removing the last peer from repos.json left its rows rendered forever with no snapshot to clear them. The emit is now unconditional; with no peers `build_remote_rows` yields an empty vec, which clears them. Still zero HTTP. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Second half of the federated scale-to-zero defect. Removing the TUI's timer
poll (previous commit) stops the peer being woken; this stops a wake that
does happen from costing a full warm hour.
The keep-warm loop computed its idle check as:
let last = kw_state.most_recent_tool_call().unwrap_or(start);
`/status` and `/healthz` have their own handlers and never call
`record_tool_call`, so a replica woken by anything other than a genuine tool
call found no recorded call, fell back to the process start time, and
self-pinged its own ingress every 120s for the entire idle window.
The fallback was unreachable in the case it was written for ("a freshly
deployed replica stays warm for the full idle window before first use"): a
real tool call always records itself, so the fallback could ONLY ever fire
when the wake was not real work. Its entire practical effect was rewarding
spurious wakes — ~67 min warm instead of the ~6 min a bare wake costs,
roughly 11x amplification.
Keep-warm now requires a real recorded tool call; with none it does not ping
and lets the host suspend the replica, which the next real request wakes.
Verified this preserves the legitimate path: an inbound federated search hits
SEARCH_PATH -> crate::mcp::rest_search_handler -> the MultiStoreContext path
that calls record_tool_call, so after real use the peer keeps itself warm
exactly as before. A peer staying warm for an hour after real use is correct
behaviour and is unchanged.
Also fixes the startup "target isn't self" warning shipped in 55fa36b, which
false-positived on the ONLY deployment where keep-warm is correct: on Azure
Container Apps the process binds 0.0.0.0 while keep_warm_url is the ingress
FQDN, so the host comparison failed and the warning fired on every cold
start. A wildcard bind means our externally-visible host is genuinely
unknown, so the comparison cannot conclude anything and must stay silent — a
check that cries wolf on the correct configuration trains operators to ignore
the case that matters. The rule moved into a testable `keep_warm_foreign_target`
helper, covered by 5 new tests (wildcard binds, genuine foreign host, matching
host, loopback targets, unparseable URL).
cargo fmt / clippy -D warnings clean; 1134 passed, 42 ignored.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The diagnosis document's root cause was a hypothesis that turned out to be wrong, and two project docs still described the removed poll cadence as the current design — which is how this same behaviour got re-introduced twice (PR #181, #184). Corrected against the code now on this branch. DIAGNOSE_FEDERATED_KEEP_WARM.md (moved from docs/, rewritten): - The old root cause blamed a misconfigured local CODESEARCH_KEEP_WARM_URL. Disproven on four independent grounds, now recorded under "What was ruled out": the env var is set nowhere locally (process env, HKCU, HKLM, all shell profiles); the one-time "keep-warm enabled" line appears in zero logs from 2026-04-26 on; that absence is meaningful because init_serve_logger is always file-only in serve mode and those logs do carry other INFO lines; and no local process held a :443 connection. - Replaced with the confirmed two-defect root cause plus the Azure Log Analytics ground truth (wakes 120/121/120 min apart, ~67 min warm each, zero searches). - Records the requirement being violated ("poll LOCAL repos, never federated") and the rejected reasoning, so it is not re-litigated. - Notes one residual, not currently exploitable: the MCP `status` TOOL, when project-scoped, does record a tool call — unlike the HTTP /status endpoint that every known poller actually uses. CHANGELOG.md: - The consolidated fix entry goes under [1.2.4] (unreleased). An earlier draft wrongly rewrote the [1.2.0] section — v1.2.0 is a real tag and #181/#184 shipped in it, so editing it would have made released notes claim a fix that is not in that release. Both original 1.2.0 entries are restored verbatim as historical record, each marked superseded. - Older version entries compressed to one-liners (existing convention). AGENTS.md: - The "Scale-to-zero-safe federated polling" bullet asserted the removed cadence as current and named ServeState::idle_suspend_secs, a field that no longer exists. Rewritten as an explicit design constraint with the rejected reasoning attached. - Keep-warm bullet updated for the tool-call requirement and the wildcard-bind carve-out. - Completed TODO sections removed, Implemented Features compressed. README.md: - The grep-guard bullet still described the 5-minute retry-unblock that 1.2.0 replaced with a /healthz liveness probe. Rewritten to match. - Verified NOT stale and left alone: "17 languages" (supported_languages() returns 17 — the table's 18th row, Jupyter, is JSON-parsed rather than tree-sitter) and the web-guard's 5-minute retry, which still exists. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Per-feature work log covering the whole cycle: the disproven original hypothesis, the Azure Log Analytics ground truth, the two defects, the three stages with their commit SHAs and review outcomes, and the open follow-ups. Records why this behaviour took three attempts across PR #181, #184 and this branch, so the rejected reasoning is not re-litigated a fourth time. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`ci.yml`'s push trigger is a prefix allowlist that never included `fix/**`, and there is no `pull_request` trigger. Every `fix/...` branch therefore merged into develop having never run fmt, clippy or a single test in CI. This is the repo's own documented naming convention — the comment directly above the branch list even says "feature/fix -> develop". Only the prefix was missing. It was invisible because CodeQL is a separate, pull_request-triggered workflow, so the PR still showed a green check. On PR #192, `gh pr checks` listed CodeQL and nothing else; the local pre-push QC gate was the only thing validating the branch. Adds `fix/**` and documents the footgun plus how to verify coverage (`gh pr checks <n>` should list the CI jobs, not just CodeQL). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
🐛 fix: stop waking the scale-to-zero federated peer (and stop it self-warming afterwards)
… duplicates
All hooks now live in .githooks/ and are enabled with a single
`git config core.hooksPath .githooks`. Removes the copy-into-.git/hooks step
that had let the tracked template and the installed hook drift apart.
Why this was broken:
- scripts/pre-push documented its own install as `cp scripts/pre-push
.git/hooks/pre-push`, but the tracked copy had no customer-reference leak
scan. Following the documented install silently removed that guard.
- .githooks/post-checkout documented `git config core.hooksPath .githooks`,
which would have made git look only in .githooks/ — a directory that held
neither pre-commit nor pre-push. Enabling one hook would have disabled the
two that matter. core.hooksPath was in fact unset, so post-checkout had
never run at all.
Changes:
- .githooks/{pre-commit,pre-push,post-checkout} + README.md; scripts/pre-commit
and scripts/pre-push deleted (scripts/qc.ps1 stays).
- Customer patterns move to untracked .githooks/customer-patterns.local. The
list is itself the thing being hidden, so it must never be tracked. When it
is missing, pre-push prints a loud "SKIPPED — nothing was checked" warning
instead of passing silently: a guard that quietly stops running is worse
than no guard, because it is still trusted.
- pre-push skips the Rust QC gate when the branch changes no .rs/Cargo.* files
vs origin/develop. Compared against origin/develop rather than the pushed
ref range because stdin is already consumed by the master guard, and this
avoids the all-zeros remote_sha case for a new branch. Every uncertain case
runs the gate rather than skipping it.
- Leak scan uses `git grep -lIE` instead of `git ls-files | xargs grep`:
space-safe paths, no argument-length limit, no stderr to suppress.
- ci.yml: add chore/** to the push branch allowlist — same prefix-allowlist
footgun that left fix/** with no CI at all until PR #192.
- .gitattributes: drop the now-dead scripts/pre-commit eol=lf rule.
Master-push guard is unchanged.
Verified by running the hook directly: master push blocked (exit 1); normal
push passes with QC correctly skipped (exit 0); missing pattern file warns
loudly and exits 0; a planted customer reference in a tracked file blocks
(exit 1); the Rust-path filter matches src/foo.rs and Cargo.toml and not
docs/x.md, proving the skip can actually un-skip.
Review-fixes:
- [Important] `|| true` on the leak scan turned a git grep *failure* into a
clean pass, and the comment-stripper could manufacture that failure: an
inline `#` in a valid pattern like `Acme(NV#|BV)` was truncated to `Acme(NV`,
producing an invalid ERE that exits 128. One bad line poisoned the whole
alternation, so a real leak sailed through with "No customer references
detected." Now: only whole-line comments are stripped (a mid-line `#` stays
part of the pattern), and the git grep exit status is branched on — 0 blocks,
1 is clean, anything else prints the loud SKIPPED banner. One rule overall:
configuration problems warn, actual leaks block.
- [Important] RELEASING.md still documented `cp scripts/pre-commit
.git/hooks/pre-commit` — a copy of a file this commit deletes, into a
directory core.hooksPath makes git ignore. Replaced with the single
`git config core.hooksPath .githooks` install.
- [Important] The QC-skip regex missed `rust-toolchain.toml` and `.cargo/`,
both tracked here: a toolchain bump changes no .rs file yet flips fmt,
clippy and tests, so the gate was skipped for exactly the change most likely
to need it. Widened to cover rust-toolchain, .cargo/, scripts/qc.*, and
rustfmt/clippy.toml (the latter two not tracked today, listed so adding one
later cannot silently reopen the gap).
- [doc] README.md said `hooks git install` writes to `.git/hooks/`. It resolves
the target with `git rev-parse --git-path hooks`, so it honours
core.hooksPath and chains into an existing hook as a marker-delimited block.
Corrected; no code change needed.
Re-verified after the fixes: master push blocked (exit 1); normal push passes
with QC skipped (exit 0); an invalid regex in the pattern file now warns
loudly and exits 0 instead of reporting clean; a mid-line `#` survives
parsing; rust-toolchain.toml, .cargo/config.toml, scripts/qc.ps1, src/a.rs and
Cargo.lock all trigger the gate while docs-only paths do not.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
🔧 chore: unify git hooks in .githooks/ — one location, no hand-copied duplicates
The worklog still said "Not pushed. Three commits sit locally" while the work was merged (PR #192) and running in production. A change record that is wrong is worse than none, so this brings it to ground truth. - Header: status is now shipped + verified, with the merge SHA and the deployed revision; test line records CI green, not just the local run. - Stage 4 added: the branch had no CI at all until `fix/**` was added to ci.yml's prefix allowlist, with the self-test that proved it. - Stage 5 added: deployment of both sides (local via copy-to-common, cloud as revision --0000024), config verified intact across the image update, and the scale-to-zero verification — replicas held at 0 across six consecutive checks with no traffic. Recorded as positive evidence: under the old binary the keep-warm fallback made reaching 0 at ten minutes impossible. - Build note: two az acr build runs failed at the identical step with "layer does not exist" on a byte-identical Dockerfile; a local docker build succeeded. Root cause is the ACR Tasks agent, not this repo. - Follow-ups: dropped the stale "not pushed" item; the production check is now a 24h Log Analytics sample rather than "unverified"; added the ACR Tasks failure as a blocker for any automated cloud deploy. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
📝 docs: worklog — record merge, deploy and production verification
…e forever A repo whose database failed to open (typically a transient write lock - an indexing run holding the DB when a query arrived) was cached as RepoState::Conflicted, and the fast path in get_or_open_stores replayed that error on every later call without ever retrying the open. The state's only documented exit was idle eviction, and it was unreachable: evict_idle_repos iterates last_access, but both paths that mark a repo Conflicted (warmup_repo and the get_or_open_stores slow path) propagate the failure with `?` before reaching their touch_access call. A repo that conflicts on first open therefore never gets a last_access entry and is never considered for eviction, however long it sits idle. Querying it did not help either: the fast path replayed the cached error while calling touch_access on the way, so the only queries that would have registered it for eviction were also the ones resetting its idle timer. Net effect: a momentary lock was indistinguishable from permanent corruption and could only be cleared by restarting serve - while the error text promised "the next query will retry automatically". A cached conflict is now dropped on next access and the open genuinely retried. Retrying is cheap when it still fails (a refused file lock), and this mirrors the missing-DB path, which already refuses to cache Conflicted for the same reason (missing_db_not_cached_as_conflicted). Regression test asserts recovery without a restart and without an idle wait. It carries two preconditions so it cannot pass vacuously (the first open must genuinely fail, and that failure must actually be cached as Conflicted), and was confirmed to FAIL with the fix neutralised - reproducing the exact user-visible error string. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Review-fixes: - [Important] Non-atomic get()+remove() on the Conflicted cache entry could race with a concurrent insert (e.g. add_repo_handler or force-reindex installing a fresh RepoState::Write) and delete that live entry instead, dropping its cancel_token without cancelling it. Fixed by using DashMap::remove_if with a matches!(v, RepoState::Conflicted) predicate — same atomic check-and-remove primitive already used by is_indexing() in this file — so removal can only ever affect an entry still Conflicted at removal time.
thiserror 1.0 → 2.0 — drop-in, zero call-site changes
axum 0.7 → 0.8 — path params :param → {param}
Debouncer absorbed Watcher (debouncer.watch()/unwatch()); root cache tracking is automatic (cache().add_root() removed). mio 0.8.11 fully out of the lock (inotify 0.9→0.11 on Linux; Windows path never used it). Validation: fmt + clippy -D warnings + 1386 tests green.
notify 6.1 → 8.2 + notify-debouncer-full 0.3 → 0.7
Zero call-site changes: Parser/Language surface stable, grammars ride tree-sitter-language (all 17 grammar crates stay pinned). Validation: fmt + clippy -D warnings + 1386 tests green.
tree-sitter 0.26 → 0.27 + tree-sitter-proto 0.4 → 0.6
…sinfo 0.39; tower/tower-http direct deps verwijderd (dood) sha2 0.11: digest arrays implen LowerHex niet meer — twee hex-sites encoderen expliciet (output ongewijzigd). dirs/scip/sysinfo drop-in. tower + tower-http waren nergens geïmporteerd: directe entries weg (blijven transitief via axum/reqwest/hf-hub). Validation: fmt + clippy -D warnings + 1386 tests green.
Small majors batch: dirs 7, sha2 0.11, scip 0.10, sysinfo 0.39; remove dead tower/tower-http
The multi-repo MCP service built its shared embedder from ModelType::default() (384-dim MiniLM) and ignored both --model and the model_short_name every index records. On indexes rebuilt with a 768-dim model (EmbeddingGemma) every semantic query failed with "Query embedding dimension mismatch: expected 768, got 384"; a same-dimension mismatch would have silently compared incomparable vector spaces. The single-repo stdio path already resolved the model from metadata.json (issue #118) — the serve path never did. - ModelType::from_index_metadata: reader counterpart to write_metadata_fields, used by every query path. - EmbeddingServicePool: lazily loads one EmbeddingService per model, shared across MCP sessions and REST handlers, so a hub can hold mixed-model indexes and each query uses its target's model. - CodesearchService::query_model / embedding_service_for replace the single shared Option<EmbeddingService>. - semantic_search and semantic_search_multi resolve the model per repo; the fan-out helper now passes the alias to its closure so each store is searched with its own model's query embedding. - CLI warns that serve --model is inert; README documents per-repo resolution. Tests: model_for_alias_* and serve_service_uses_repo_model_not_default (serve), test_model_type_round_trips_through_index_metadata (embed).
…ulting `codesearch index` used `model.unwrap_or_default()`, so re-indexing an existing index built with a non-default model (EmbeddingGemma, 768-dim) picked the 384-dim default: `FileMetaStore` reported "Model changed, full re-index required" and 384-dim vectors were embedded against a 768-dim store. It now resolves the recorded model through the same helper the serve/watcher paths use; an explicit `--model` that disagrees is rejected with a pointer to `--force`. `codesearch stats`, `get_db_stats` and the repo listing opened the vector store with a hardcoded 384, so `Dimensions:` always read 384 for every index — they now read the recorded dimensions. `status(kind="index")` reported the service's own model (the hardcoded default in serve mode) while `dimensions` came from live stats; it now resolves per repo, and reports the common model or "mixed" for a group. Tests: index_model_resolution_tests (recorded_dimensions and resolve_index_model precedence/mismatch/force/fresh), group_status_model_label_is_common_or_mixed.
`codesearch index add --model embeddinggemma-q4` delegates to `POST /repos` when serve is running. The handler opened the store with the default 384-dim dimension and applied the model override to `metadata.json` only afterwards, so the background reindex embedded 768-dim vectors into a 384-dim store and indexed nothing — the `.codesearch.db` directory was created, `stats` reported the new dimension, and zero files were indexed. `try_open_stores` now takes an optional dimension override, and `add_repo_handler` passes the request model's dimensions when one is given (the model parse moved before the store open). All other call sites pass `None` and keep reading the dimension from `metadata.json`. Test: `try_open_stores_honours_dimension_override_for_a_fresh_repo` (observes 384 vs 768 when the override is ignored).
…s built `status(kind="index")` derived readiness only from `total_chunks > 0`, so a repo mid-rebuild — chunks inserted but `build_index()` not yet run — reported `status: "ready"` / "Index is ready for searching." while every search failed with "Index not built. Call build_index() after inserting chunks." Observed live on a 27-repo hub partway through a model migration: `py-opa` had 403 chunks, `indexed: false`, and still read as ready. - Single-store path: new `single_index_status(total_chunks, indexed)`. - Group path: `index_status_summary` takes `all_indexed`; a store that failed to report stats is tracked separately so a failure is never misread as "not built" (it still surfaces as degraded-ready with `warnings`). Tests: `index_status_summary_reports_building_when_chunks_exist_but_graph_is_not_built` and `single_index_status_requires_a_built_graph_to_report_ready` (both fail `ready` vs `building` when the built-graph signal is ignored).
`codesearch serve --model X` now sets a serve-wide default embedding model for indexes created through serve. `POST /repos` (including `index add` delegated to a running serve) uses it when the request carries no explicit model and the index records none; an existing index keeps its recorded model, and an explicit `model` in the request still wins. Serve reports the default as `default_model` in `GET /status`, at startup, and as the fallback query model for a repo whose metadata records none. Adds resolve_add_repo_model() to pin the precedence, with tests covering the new-index default, the recorded-model-wins case, and the service fallback.
`serve --model X` was used as the query fallback for a repo whose `metadata.json` records no model. That overrode a legacy index the moment the flag was set: a 384-dim index queried with a 768-dim model failed every search with "Query embedding dimension mismatch: expected 384, got 768", and a same-dimension model compared incomparable vector spaces without erroring. Resolve such an index to the built-in default instead: that is both the historical behaviour and the value every other reader assumes for metadata without a `model_short_name`. `serve --model` keeps doing what its name says, selecting the model for newly created indexes. Surface the assumption to the caller as a search warning naming the repo, the assumed model and the re-index command, and log it once per repo so a busy hub does not repeat the same line. The scope-free status summary still reports the serve default.
fix: resolve each index's own embedding model (serve queries, CLI index/stats/status)
… + rebuild Symbol-rebuild failures of the MDB_BAD_VALSIZE class (data written by an older storage-major, observed on all C# repos after the arroy 0.8/heed 0.22 upgrades) now queue the repo for automatic recovery: evict stores (remove_repo's sequence, minus unregister), wipe the DB dir with the bounded lock-retry, and force-reindex through the TUI machinery whose store-open path recreates fresh formats. Recoveries are processed strictly one repo at a time (single worker, flag+queue under one lock so no lost wake-up); read-only repos are skipped with a pointer to the owning writer. Detection helper is unit-tested; queue dedup + worker start pinned by test.
Serve: auto-recover LMDB format corruption (sequential wipe + rebuild)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release v1.4.0 — squashed release delta from develop (v1.3.14 → v1.4.0).
Highlights (full details in CHANGELOG.md):
MSRV/protocol: behavior-identical MCP (legacy lifecycle); release binaries via release.yml on tag.