diff --git a/.github/workflows/changelog.yml b/.github/workflows/changelog.yml index 9fbcc76..e8a3bab 100644 --- a/.github/workflows/changelog.yml +++ b/.github/workflows/changelog.yml @@ -4,8 +4,7 @@ on: push: branches: [main] -permissions: - contents: write +permissions: {} concurrency: group: changelog @@ -13,12 +12,28 @@ concurrency: jobs: regenerate: - if: github.actor != 'github-actions[bot]' + if: ${{ !endsWith(github.actor, '[bot]') }} runs-on: ubuntu-latest + permissions: {} # all repo access comes from the minted App token below steps: + - name: Mint a changelog-bot token + id: app-token + uses: actions/create-github-app-token@v3 + with: + client-id: ${{ vars.CHANGELOG_APP_CLIENT_ID }} + private-key: ${{ secrets.CHANGELOG_APP_PRIVATE_KEY }} + permission-contents: write + permission-pull-requests: write + - name: Get changelog-bot user id + id: app-user + env: + GH_TOKEN: ${{ steps.app-token.outputs.token }} + APP_SLUG: ${{ steps.app-token.outputs.app-slug }} + run: echo "id=$(gh api "/users/${APP_SLUG}[bot]" --jq .id)" >> "$GITHUB_OUTPUT" - uses: actions/checkout@v5 with: fetch-depth: 0 # surface diff worktree + previous-tag lookup need history + token: ${{ steps.app-token.outputs.token }} - uses: dtolnay/rust-toolchain@stable # rustdoc JSON for the API surface diff is nightly-only - run: rustup toolchain install nightly --profile minimal @@ -34,11 +49,22 @@ jobs: [ -n "$PREV" ] && AGAINST="--against-ref $PREV" # shellcheck disable=SC2086 # deliberate flag list cargo soothfast report changelog -p scry-core -p scry-server --baseline base $AGAINST - - name: Commit if changed - run: | - git diff --quiet CHANGELOG.md && exit 0 - git config user.name "github-actions[bot]" - git config user.email "github-actions[bot]@users.noreply.github.com" - git add CHANGELOG.md - git commit -m "docs: regenerate CHANGELOG.md" - git push + - uses: peter-evans/create-pull-request@v8 + id: cpr + with: + token: ${{ steps.app-token.outputs.token }} + add-paths: CHANGELOG.md + commit-message: "docs: regenerate CHANGELOG.md" + title: "docs: regenerate CHANGELOG.md" + body: Automated CHANGELOG regeneration. + branch: bot/changelog-update + delete-branch: true + author: >- + ${{ steps.app-token.outputs.app-slug }}[bot] + <${{ steps.app-user.outputs.id }}+${{ steps.app-token.outputs.app-slug }}[bot]@users.noreply.github.com> + - name: Merge pull request + if: steps.cpr.outputs.pull-request-number + env: + GH_TOKEN: ${{ steps.app-token.outputs.token }} + PR_NUMBER: ${{ steps.cpr.outputs.pull-request-number }} + run: gh pr merge --auto --squash --delete-branch "$PR_NUMBER" diff --git a/CHANGELOG.md b/CHANGELOG.md index f506811..648ade4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,17 +1,40 @@ # Changelog -## Unreleased (draft vs v0.1.0) +## 0.2.0 - 2026-09-05 - - +### Overview + +Retrieval quality release. Tree-sitter chunks now keep the doc comment +and attributes above each definition, and spans under four lines merge +into a neighbour, so a bare `#[cfg]` line or a one-line `mod` declaration +never surfaces as a hit on its own; on finance-query that removed a fifth +of all chunks and lifted recall@10 on the golden set from 0.717 to 0.833. +Answers go through the same rerank as search and say what the sources do +not cover instead of guessing. The reranker reads at most `max_chars` +(3000) of each document, which is both faster and more accurate than full +length, and the server caches the last 256 query vectors so a repeated +query skips HyDE and embedding. Files marked `@generated` are no longer +indexed. A cross-encoder is still optional; nothing changes for a server +without `[rerank]`. + +### Upgrade notes - - +Run `scry index --full` once per repo after upgrading the server. Files +are only re-chunked when their content changes, so an existing index +keeps the old short chunks until every file is re-uploaded; a plain +`scry index` reports everything unchanged. The full pass re-embeds the +repo, which took eight minutes for a 1,200-file repo on an iGPU. The new +`[rerank] max_chars` key defaults to 3000 and the `rerank` field on +`/v1/answer` defaults to true, so no config change is required. ### ✨ Features +- Rerank answer sources and admit gaps +- Add scry index --full to re-chunk a repo +- Skip @generated files when chunking +- Keep trivia with defs and merge short chunks - Rerank stage fused as a weighted RRF leg (#4) - Greedy chat completions, HyDE capped at 120 tokens - Eval --limit for recall at deeper cutoffs @@ -28,6 +51,8 @@ ### ⚡ Performance +- Cache query vectors on the server +- Cut reranker input to max_chars - One allocation for the rescore id list, exact alloc claim - Allocation-free exact scan and schema guards - Float vectors in a plain table, coarse pass rescored from it @@ -35,11 +60,14 @@ ### 📝 Documentation +- Release notes for 0.2.0 - Allocation claim for the dense search path - Add CLAUDE.md ### 🔧 Internal +- Bump version to 0.2.0 +- Open a PR for CHANGELOG regeneration - Tie-aware recall and recall@10 in refiner_curve - Store pragmas, optimize on shutdown, vacuum on prune - Store-layer benches and refiner recall harness @@ -50,13 +78,17 @@ ``` # scry-core +ADDED scry_core::chunker::is_generated ADDED scry_core::config::RerankConfig ADDED scry_core::rerank ADDED scry_core::rerank::RerankClient ADDED scry_core::rerank::RerankResult ADDED scry_core::rerank::fuse ADDED scry_core::search::RRF_K +REMOVED scry_core::chunker::line_window::chunk_lines CHANGED scry_core::Error (body) +CHANGED scry_core::chunker::chunk_file (body) +CHANGED scry_core::chunker::line_window::chunk (body) CHANGED scry_core::config::Config (body) CHANGED scry_core::error::Error (body) CHANGED scry_core::search::expand_symbols (body) @@ -64,7 +96,10 @@ CHANGED scry_core::search::fts_query (body) CHANGED scry_core::search::query_vector (body) # scry-server +ADDED scry_server::QueryCache +ADDED scry_server::query_cache::QueryCache CHANGED scry_server::AppState (body) +CHANGED scry_server::api::AnswerRequest (body) CHANGED scry_server::api::SearchRequest (body) CHANGED scry_server::serve (body) ``` @@ -73,7 +108,7 @@ CHANGED scry_server::serve (body) | item | metric | was | now | delta | |---|---|---:|---:|---:| -| `scry_core::bench_expand_symbols` | instructions | 58528.0 | 31276.0 | -46.6% | +| `scry_core::bench_expand_symbols` | instructions | 50170.0 | 27518.0 | -45.2% | | `scry_core::bench_expand_symbols` | allocs | 88.0 | 55.0 | -37.5% | diff --git a/CLAUDE.md b/CLAUDE.md index 288024d..3cc8dac 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -51,6 +51,11 @@ edits elsewhere in a file, but renaming a file or changing the repo key deliberately re-embeds. Directories at or above `$HOME` are never treated as repos; search there goes cross-repo (`repo_id = None` end to end). +**Chunking happens server-side** in `prepare_file`, and a file is only +re-chunked when its content hash changes. After touching `chunker/`, run +`scry index --full` in a test repo to see the effect; plain `scry index` +will report everything unchanged. `@generated` files yield no chunks. + **One SQLite file** holds everything: repos/files/chunks + FTS5 + sqlite-vec tables + memories. The vec tables are created at `Store::open` because their DDL carries the embedding dimension; `meta` stamps the diff --git a/Cargo.lock b/Cargo.lock index da137b2..e4af64b 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1444,7 +1444,7 @@ dependencies = [ [[package]] name = "scry" -version = "0.1.0" +version = "0.2.0" dependencies = [ "anyhow", "clap", @@ -1462,7 +1462,7 @@ dependencies = [ [[package]] name = "scry-core" -version = "0.1.0" +version = "0.2.0" dependencies = [ "async-trait", "ignore", @@ -1498,7 +1498,7 @@ dependencies = [ [[package]] name = "scry-server" -version = "0.1.0" +version = "0.2.0" dependencies = [ "axum", "reqwest", diff --git a/Cargo.toml b/Cargo.toml index 72e2c37..80996e6 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -3,7 +3,7 @@ resolver = "3" members = ["crates/scry-core", "crates/scry-server", "crates/scry"] [workspace.package] -version = "0.1.0" +version = "0.2.0" edition = "2024" license = "MIT" repository = "https://github.com/Verdenroz/scry" diff --git a/crates/scry-core/src/chunker/coalesce.rs b/crates/scry-core/src/chunker/coalesce.rs new file mode 100644 index 0000000..38a6d4f --- /dev/null +++ b/crates/scry-core/src/chunker/coalesce.rs @@ -0,0 +1,106 @@ +//! Merges undersized spans into a neighbour so a bare attribute, a `use` +//! group, or a one-line `mod` declaration never embeds on its own. + +use super::Span; + +/// Spans shorter than this merge forward into the next span; a short +/// trailing span merges back into the previous one. +pub(crate) const MIN_LINES: usize = 4; + +const SEPARATOR: &str = " > "; + +pub(crate) fn coalesce(spans: Vec) -> Vec { + let mut out: Vec = Vec::new(); + let mut pending: Option = None; + for span in spans { + let span = match pending.take() { + Some(short) => merge(short, span), + None => span, + }; + if span.end - span.start < MIN_LINES { + pending = Some(span); + } else { + out.push(span); + } + } + if let Some(short) = pending { + let merged = match out.pop() { + Some(last) => merge(last, short), + None => short, + }; + out.push(merged); + } + out +} + +fn merge(first: Span, second: Span) -> Span { + Span { + start: first.start.min(second.start), + end: first.end.max(second.end), + symbol: shared_symbol(first.symbol.as_deref(), second.symbol.as_deref()), + } +} + +/// `S > a` and `S > b` merge under `S`; unrelated symbols merge under none. +fn shared_symbol(a: Option<&str>, b: Option<&str>) -> Option { + match (a, b) { + (None, other) | (other, None) => other.map(str::to_string), + (Some(a), Some(b)) => { + let shared = a + .split(" > ") + .zip(b.split(" > ")) + .take_while(|(x, y)| x == y) + .fold(0, |len, (x, _)| len + x.len() + SEPARATOR.len()); + (shared > 0).then(|| a[..shared - SEPARATOR.len()].to_string()) + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn span(start: usize, end: usize, symbol: Option<&str>) -> Span { + Span { + start, + end, + symbol: symbol.map(str::to_string), + } + } + + #[test] + fn short_spans_merge_forward_and_keep_the_definition_symbol() { + let out = coalesce(vec![span(0, 2, None), span(2, 10, Some("alpha"))]); + assert_eq!(out, vec![span(0, 10, Some("alpha"))]); + } + + #[test] + fn short_trailing_span_merges_back() { + let out = coalesce(vec![span(0, 10, Some("alpha")), span(10, 11, None)]); + assert_eq!(out, vec![span(0, 11, Some("alpha"))]); + } + + #[test] + fn runs_of_one_liners_accumulate() { + let out = coalesce(vec![ + span(0, 1, Some("a")), + span(1, 2, Some("b")), + span(2, 3, Some("c")), + span(3, 4, Some("d")), + span(4, 12, Some("e")), + ]); + assert_eq!(out, vec![span(0, 4, None), span(4, 12, Some("e"))]); + } + + #[test] + fn sibling_methods_merge_under_their_parent() { + let out = coalesce(vec![span(0, 2, Some("S > a")), span(2, 5, Some("S > b"))]); + assert_eq!(out, vec![span(0, 5, Some("S"))]); + } + + #[test] + fn long_spans_pass_through() { + let spans = vec![span(0, 10, Some("a")), span(10, 20, Some("b"))]; + assert_eq!(coalesce(spans.clone()), spans); + } +} diff --git a/crates/scry-core/src/chunker/line_window.rs b/crates/scry-core/src/chunker/line_window.rs index bdfcf2d..d1a20cf 100644 --- a/crates/scry-core/src/chunker/line_window.rs +++ b/crates/scry-core/src/chunker/line_window.rs @@ -1,4 +1,4 @@ -use super::{Chunk, lines_chunk}; +use super::{Chunk, Span, lines_chunk}; pub const WINDOW: usize = 40; pub const OVERLAP: usize = 10; @@ -6,13 +6,16 @@ const SNAP: usize = 5; pub fn chunk(content: &str) -> Vec { let lines: Vec<&str> = content.lines().collect(); - chunk_lines(&lines, 0, lines.len(), None) + spans(&lines, 0, lines.len()) + .into_iter() + .map(|span| lines_chunk(&lines, span)) + .collect() } -/// Windows `lines[from..to]` into chunks, snapping each cut back to the -/// nearest blank line within [`SNAP`] lines. -pub fn chunk_lines(lines: &[&str], from: usize, to: usize, symbol: Option<&str>) -> Vec { - let mut chunks = Vec::new(); +/// Windows `lines[from..to]`, snapping each cut back to the nearest blank +/// line within [`SNAP`] lines. Windows holding only blank lines are dropped. +pub(crate) fn spans(lines: &[&str], from: usize, to: usize) -> Vec { + let mut spans = Vec::new(); let mut start = from; while start < to { let mut end = (start + WINDOW).min(to); @@ -29,14 +32,18 @@ pub fn chunk_lines(lines: &[&str], from: usize, to: usize, symbol: Option<&str>) end = (start + WINDOW).min(to); } if lines[start..end].iter().any(|line| !line.trim().is_empty()) { - chunks.push(lines_chunk(lines, start, end, symbol.map(str::to_string))); + spans.push(Span { + start, + end, + symbol: None, + }); } if end >= to { break; } start = end.saturating_sub(OVERLAP).max(start + 1); } - chunks + spans } #[cfg(test)] diff --git a/crates/scry-core/src/chunker/mod.rs b/crates/scry-core/src/chunker/mod.rs index cff57ab..d794017 100644 --- a/crates/scry-core/src/chunker/mod.rs +++ b/crates/scry-core/src/chunker/mod.rs @@ -3,11 +3,15 @@ //! falls back to blank-line-snapped windows. Line numbers are 1-based and //! inclusive so hits print as `path:start-end`. +mod coalesce; pub mod line_window; mod treesitter; pub use treesitter::Language; +const GENERATED_MARKER: &str = "@generated"; +const GENERATED_HEAD_LINES: usize = 5; + #[derive(Debug, Clone, PartialEq, Eq)] pub struct Chunk { pub start_line: u32, @@ -16,8 +20,16 @@ pub struct Chunk { pub content: String, } +/// Half-open line range `[start, end)` into a file, 0-based. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct Span { + pub start: usize, + pub end: usize, + pub symbol: Option, +} + pub fn chunk_file(relpath: &str, content: &str) -> Vec { - if content.contains('\0') { + if content.contains('\0') || is_generated(content) { return Vec::new(); } match Language::from_path(relpath) { @@ -28,16 +40,39 @@ pub fn chunk_file(relpath: &str, content: &str) -> Vec { } } -pub(crate) fn lines_chunk( - lines: &[&str], - start: usize, - end: usize, - symbol: Option, -) -> Chunk { +/// The `@generated` header convention GitHub collapses diffs on; such +/// files duplicate their sources and crowd out hand-written code. +pub fn is_generated(content: &str) -> bool { + content + .lines() + .take(GENERATED_HEAD_LINES) + .any(|line| line.contains(GENERATED_MARKER)) +} + +pub(crate) fn lines_chunk(lines: &[&str], span: Span) -> Chunk { Chunk { - start_line: start as u32 + 1, - end_line: end as u32, - symbol, - content: lines[start..end].join("\n"), + start_line: span.start as u32 + 1, + end_line: span.end as u32, + content: lines[span.start..span.end].join("\n"), + symbol: span.symbol, + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn generated_files_yield_no_chunks() { + let source = "// @generated by `cargo soothfast docs gen-tests`\nfn main() {}\n"; + assert!(chunk_file("examples/gen.rs", source).is_empty()); + assert!(!chunk_file("src/lib.rs", "fn main() {}\n").is_empty()); + } + + #[test] + fn generated_marker_only_counts_in_the_head() { + let mut lines = vec!["fn main() {}"; 10]; + lines.push("// mentions @generated much later"); + assert!(!chunk_file("src/lib.rs", &lines.join("\n")).is_empty()); } } diff --git a/crates/scry-core/src/chunker/treesitter.rs b/crates/scry-core/src/chunker/treesitter.rs index d9f1d83..a330aa3 100644 --- a/crates/scry-core/src/chunker/treesitter.rs +++ b/crates/scry-core/src/chunker/treesitter.rs @@ -1,6 +1,7 @@ use tree_sitter::{Node, Parser}; -use super::{Chunk, line_window, lines_chunk}; +use super::coalesce::coalesce; +use super::{Chunk, Span, line_window, lines_chunk}; /// Definitions longer than this are not emitted whole; the walk descends /// into them so nested definitions become their own chunks. @@ -208,28 +209,49 @@ fn def_name(language: Language, node: Node<'_>, source: &str) -> Option named.utf8_text(source.as_bytes()).ok().map(str::to_string) } +/// Comments, attributes, and decorators sitting directly above a +/// definition belong to it; the definition's span grows upward over them. +fn is_trivia(kind: &str) -> bool { + kind.contains("comment") || kind.contains("attribute") || kind.contains("decorator") +} + struct Walker<'a> { language: Language, source: &'a str, - defs: Vec<(usize, usize, Option)>, + defs: Vec, } impl Walker<'_> { fn walk(&mut self, node: Node<'_>, path: &[String]) { let mut cursor = node.walk(); + let mut trivia_start: Option = None; + let mut last_end_row = 0; for child in node.named_children(&mut cursor) { + let start = child.start_position().row; + let end = child.end_position().row + 1; + let adjacent = trivia_start.is_some() && last_end_row >= start; + last_end_row = end; + if is_trivia(child.kind()) { + if !adjacent { + trivia_start = Some(start); + } + continue; + } + let span_start = if adjacent { trivia_start.take() } else { None }; + trivia_start = None; if !self.language.is_def(child.kind()) { self.walk(child, path); continue; } - let start = child.start_position().row; - let end = child.end_position().row + 1; let name = def_name(self.language, child, self.source); if end - start <= MAX_DEF_LINES { let mut symbol = path.to_vec(); symbol.extend(name); - self.defs - .push((start, end, (!symbol.is_empty()).then(|| symbol.join(" > ")))); + self.defs.push(Span { + start: span_start.unwrap_or(start), + end, + symbol: (!symbol.is_empty()).then(|| symbol.join(" > ")), + }); } else { let mut inner = path.to_vec(); inner.extend(name); @@ -251,29 +273,32 @@ pub fn chunk(language: Language, content: &str) -> Option> { }; walker.walk(tree.root_node(), &[]); let mut defs = walker.defs; - defs.sort_by_key(|(start, _, _)| *start); + defs.sort_by_key(|def| def.start); let lines: Vec<&str> = content.lines().collect(); - let mut chunks = Vec::new(); + let mut spans = Vec::new(); let mut covered = 0; - for (start, end, symbol) in defs { - if start < covered { + for def in defs { + let start = def.start.max(covered); + let end = def.end.min(lines.len()); + if start >= end { continue; } - if start > covered { - chunks.extend(line_window::chunk_lines(&lines, covered, start, None)); - } - let end = end.min(lines.len()); - if start < end { - chunks.push(lines_chunk(&lines, start, end, symbol)); - } - covered = covered.max(end); + spans.extend(line_window::spans(&lines, covered, start)); + spans.push(Span { + start, + end, + symbol: def.symbol, + }); + covered = end; } - if covered < lines.len() { - chunks.extend(line_window::chunk_lines(&lines, covered, lines.len(), None)); - } - chunks.sort_by_key(|chunk| chunk.start_line); - Some(chunks) + spans.extend(line_window::spans(&lines, covered, lines.len())); + Some( + coalesce(spans) + .into_iter() + .map(|span| lines_chunk(&lines, span)) + .collect(), + ) } #[cfg(test)] @@ -284,17 +309,50 @@ mod tests { chunks.iter().filter_map(|c| c.symbol.as_deref()).collect() } + fn by_symbol<'a>(chunks: &'a [Chunk], symbol: &str) -> &'a Chunk { + chunks + .iter() + .find(|c| c.symbol.as_deref() == Some(symbol)) + .unwrap() + } + #[test] fn rust_functions_become_symbol_chunks() { - let source = "use std::io;\n\nfn alpha() {\n println!(\"a\");\n}\n\nstruct Beta {\n field: u32,\n}\n"; + let source = "use std::io;\n\nfn alpha() {\n let a = 1;\n let b = 2;\n println!(\"{a}{b}\");\n}\n\nstruct Beta {\n field: u32,\n other: u32,\n third: u32,\n}\n"; let chunks = chunk(Language::Rust, source).unwrap(); assert_eq!(symbols(&chunks), ["alpha", "Beta"]); - let alpha = chunks - .iter() - .find(|c| c.symbol.as_deref() == Some("alpha")) - .unwrap(); - assert_eq!((alpha.start_line, alpha.end_line), (3, 5)); + let alpha = by_symbol(&chunks, "alpha"); + assert_eq!((alpha.start_line, alpha.end_line), (1, 7)); assert!(alpha.content.contains("println!")); + assert_eq!(by_symbol(&chunks, "Beta").start_line, 9); + } + + #[test] + fn doc_comments_and_attributes_stay_with_their_definition() { + let source = "fn filler() {\n 1;\n 2;\n 3;\n}\n\n/// Documents alpha.\n/// Second line.\n#[cfg(feature = \"x\")]\nfn alpha() {\n let a = 1;\n let b = 2;\n a + b;\n}\n"; + let chunks = chunk(Language::Rust, source).unwrap(); + let alpha = by_symbol(&chunks, "alpha"); + assert_eq!((alpha.start_line, alpha.end_line), (7, 14)); + assert!(alpha.content.starts_with("/// Documents alpha.")); + assert_eq!(by_symbol(&chunks, "filler").end_line, 5); + } + + #[test] + fn detached_comments_are_not_pulled_in() { + let source = + "// unrelated note\n\nfn alpha() {\n let a = 1;\n let b = 2;\n a + b;\n}\n"; + let chunks = chunk(Language::Rust, source).unwrap(); + assert_eq!(chunks.len(), 1); + assert_eq!(chunks[0].start_line, 1); + } + + #[test] + fn one_line_declarations_merge_into_one_chunk() { + let source = "pub(crate) mod cftc;\n#[cfg(feature = \"crypto\")]\npub(crate) mod coingecko;\npub(crate) mod edgar;\n#[cfg(feature = \"finra\")]\npub(crate) mod finra;\n"; + let chunks = chunk(Language::Rust, source).unwrap(); + assert_eq!(chunks.len(), 1); + assert_eq!((chunks[0].start_line, chunks[0].end_line), (1, 6)); + assert_eq!(chunks[0].symbol, None); } #[test] @@ -315,17 +373,19 @@ mod tests { .collect(); let source = format!("struct S;\n\nimpl S {{\n{body}}}\n"); let chunks = chunk(Language::Rust, &source).unwrap(); - assert!(chunks.iter().any(|c| c.symbol.as_deref() == Some("S > m0"))); assert!( chunks .iter() - .any(|c| c.symbol.as_deref() == Some("S > m69")) + .all(|c| c.symbol.as_deref().is_some_and(|s| s.starts_with("S"))) ); + assert!(chunks.iter().any(|c| c.content.contains("fn m0("))); + assert!(chunks.iter().any(|c| c.content.contains("fn m69("))); + assert!(chunks.iter().all(|c| c.end_line - c.start_line + 1 >= 4)); } #[test] fn python_class_and_decorated_defs() { - let source = "import os\n\nclass Greeter:\n def hello(self):\n return 'hi'\n\n@cached\ndef top():\n return 1\n"; + let source = "import os\n\nclass Greeter:\n def hello(self):\n return 'hi'\n\n def bye(self):\n return 'bye'\n\n@cached\ndef top():\n a = 1\n b = 2\n return a + b\n"; let chunks = chunk(Language::Python, source).unwrap(); assert!(symbols(&chunks).contains(&"Greeter")); assert!(symbols(&chunks).contains(&"top")); @@ -333,7 +393,7 @@ mod tests { #[test] fn typescript_interfaces_and_functions() { - let source = "export interface Config {\n port: number;\n}\n\nexport function load(): Config {\n return { port: 1 };\n}\n"; + let source = "export interface Config {\n port: number;\n host: string;\n tls: boolean;\n}\n\nexport function load(): Config {\n const port = 1;\n const host = 'x';\n return { port, host, tls: false };\n}\n"; let chunks = chunk(Language::Typescript, source).unwrap(); assert!(symbols(&chunks).contains(&"Config")); assert!(symbols(&chunks).contains(&"load")); @@ -341,7 +401,7 @@ mod tests { #[test] fn c_functions_resolve_names_through_declarators() { - let source = "#include \n\nint add(int a, int b) {\n return a + b;\n}\n"; + let source = "#include \n\nint add(int a, int b) {\n int c = a + b;\n printf(\"%d\", c);\n return c;\n}\n"; let chunks = chunk(Language::C, source).unwrap(); assert!(symbols(&chunks).contains(&"add")); } @@ -355,7 +415,7 @@ mod tests { #[test] fn kotlin_functions_and_classes() { - let source = "class Greeter {\n fun hello(): String {\n return \"hi\"\n }\n}\n\nfun top(): Int = 1\n"; + let source = "class Greeter {\n fun hello(): String {\n return \"hi\"\n }\n}\n\nfun top(): Int {\n val a = 1\n val b = 2\n return a + b\n}\n"; let chunks = chunk(Language::Kotlin, source).unwrap(); assert!(symbols(&chunks).contains(&"Greeter")); assert!(symbols(&chunks).contains(&"top")); @@ -370,19 +430,17 @@ mod tests { #[test] fn bash_functions() { - let source = "#!/bin/bash\n\ndeploy() {\n echo deploying\n}\n"; + let source = "#!/bin/bash\n\ndeploy() {\n echo deploying\n echo again\n echo done\n}\n"; let chunks = chunk(Language::Bash, source).unwrap(); assert!(symbols(&chunks).contains(&"deploy")); } #[test] fn gap_lines_still_covered() { - let source = "const A: u32 = 1;\nconst B: u32 = 2;\n\nfn used() -> u32 {\n A + B\n}\n"; + let source = "const A: u32 = 1;\nconst B: u32 = 2;\n\nfn used() -> u32 {\n let a = A;\n let b = B;\n a + b\n}\n"; let chunks = chunk(Language::Rust, source).unwrap(); - assert!( - chunks - .iter() - .any(|c| c.symbol.is_none() && c.content.contains("const A")) - ); + assert_eq!(chunks.len(), 1); + assert!(chunks[0].content.contains("const A")); + assert_eq!(chunks[0].symbol.as_deref(), Some("used")); } } diff --git a/crates/scry-core/src/config.rs b/crates/scry-core/src/config.rs index d0829ed..a9e29e1 100644 --- a/crates/scry-core/src/config.rs +++ b/crates/scry-core/src/config.rs @@ -118,12 +118,20 @@ pub struct RerankConfig { pub top_n: usize, #[serde(default = "default_rerank_weight")] pub weight: f64, + /// Each document is cut here before scoring; the header and the start + /// of a chunk carry the signal the cross-encoder needs. + #[serde(default = "default_rerank_max_chars")] + pub max_chars: usize, } fn default_rerank_top_n() -> usize { 20 } +fn default_rerank_max_chars() -> usize { + 3000 +} + fn default_rerank_weight() -> f64 { 1.0 } diff --git a/crates/scry-core/src/rerank.rs b/crates/scry-core/src/rerank.rs index f25e06b..8bb316d 100644 --- a/crates/scry-core/src/rerank.rs +++ b/crates/scry-core/src/rerank.rs @@ -4,6 +4,7 @@ use serde::Deserialize; use crate::config::RerankConfig; +use crate::index::embed_input; use crate::search::{RRF_K, SearchHit}; use crate::{Error, Result}; @@ -42,6 +43,18 @@ impl RerankClient { self.config.weight } + /// The text scored for a hit: its embedding input cut to `max_chars`. + pub fn document(&self, hit: &SearchHit) -> String { + let mut text = embed_input( + &hit.repo_key, + &hit.relpath, + hit.symbol.as_deref(), + &hit.content, + ); + text.truncate(text.floor_char_boundary(self.config.max_chars)); + text + } + /// Scores every document against the query; results come back best /// first with the index into `documents`. pub async fn rerank(&self, query: &str, documents: &[String]) -> Result> { @@ -137,6 +150,23 @@ mod tests { assert_eq!(out[0].score, 0.5); } + #[test] + fn document_keeps_the_header_and_cuts_on_a_char_boundary() { + let client = RerankClient::new(RerankConfig { + base_url: String::new(), + api_key: String::new(), + model: String::new(), + top_n: 20, + weight: 1.0, + max_chars: 12, + }); + let mut long = hit("a.rs"); + long.content = "éééééééééé".to_string(); + let document = client.document(&long); + assert_eq!(document, "r > a.rs\né"); + assert!(document.len() <= 12); + } + #[test] fn fuse_with_zero_weight_keeps_the_pool_order() { let hits = vec![hit("a"), hit("b")]; diff --git a/crates/scry-core/tests/pipeline.rs b/crates/scry-core/tests/pipeline.rs index 5dbf488..fcb0753 100644 --- a/crates/scry-core/tests/pipeline.rs +++ b/crates/scry-core/tests/pipeline.rs @@ -18,7 +18,7 @@ fn fixture_repo(root: &Path) { write( root, "src/config.rs", - "pub struct Settings {\n pub port: u16,\n}\n\npub fn load_configuration(path: &str) -> Settings {\n let text = std::fs::read_to_string(path).unwrap();\n parse_settings(&text)\n}\n\nfn parse_settings(text: &str) -> Settings {\n Settings { port: text.trim().parse().unwrap() }\n}\n", + "pub struct Settings {\n pub port: u16,\n pub host: String,\n pub verbose: bool,\n}\n\npub fn load_configuration(path: &str) -> Settings {\n let text = std::fs::read_to_string(path).unwrap();\n parse_settings(&text)\n}\n\nfn parse_settings(text: &str) -> Settings {\n let port = text.trim().parse().unwrap();\n let host = String::from(\"localhost\");\n Settings { port, host, verbose: false }\n}\n", ); write( root, @@ -72,7 +72,7 @@ async fn index_search_and_incremental_reindex() { write( dir.path(), "src/config.rs", - "pub struct Settings {\n pub port: u16,\n}\n\npub fn load_configuration(path: &str) -> Settings {\n let text = std::fs::read_to_string(path).unwrap();\n parse_settings(&text)\n}\n\nfn parse_settings(text: &str) -> Settings {\n Settings { port: text.trim().parse().unwrap_or(8080) }\n}\n", + "pub struct Settings {\n pub port: u16,\n pub host: String,\n pub verbose: bool,\n}\n\npub fn load_configuration(path: &str) -> Settings {\n let text = std::fs::read_to_string(path).unwrap();\n parse_settings(&text)\n}\n\nfn parse_settings(text: &str) -> Settings {\n let port = text.trim().parse().unwrap_or(8080);\n let host = String::from(\"localhost\");\n Settings { port, host, verbose: false }\n}\n", ); let third = index_repo(&mut store, &embedder, REPO_KEY, dir.path(), &config) .await diff --git a/crates/scry-server/src/api.rs b/crates/scry-server/src/api.rs index 6c4693b..47c1974 100644 --- a/crates/scry-server/src/api.rs +++ b/crates/scry-server/src/api.rs @@ -203,6 +203,9 @@ pub struct AnswerRequest { pub repo_key: Option, #[serde(default)] pub web: bool, + /// Applies the server's `[rerank]` stage to the local sources. + #[serde(default = "default_true")] + pub rerank: bool, } #[derive(Debug, Serialize, Deserialize)] diff --git a/crates/scry-server/src/lib.rs b/crates/scry-server/src/lib.rs index 85888bd..a898fb8 100644 --- a/crates/scry-server/src/lib.rs +++ b/crates/scry-server/src/lib.rs @@ -4,10 +4,12 @@ pub mod api; mod auth; mod error; +mod query_cache; mod routes; mod store_actor; pub mod tavily; +pub use query_cache::QueryCache; pub use store_actor::StoreHandle; use std::sync::Arc; @@ -23,6 +25,7 @@ use scry_core::rerank::RerankClient; use scry_core::store::Store; const MAX_SYNC_BODY_BYTES: usize = 64 * 1024 * 1024; +const QUERY_CACHE_ENTRIES: usize = 256; pub struct AppState { pub store: StoreHandle, @@ -34,6 +37,7 @@ pub struct AppState { pub memory_config: MemoryConfig, pub tavily: Option, pub rerank: Option, + pub query_cache: QueryCache, } impl AppState { @@ -57,6 +61,7 @@ impl AppState { memory_config: config.memory.clone(), tavily: config.tavily.clone().map(tavily::TavilyClient::new), rerank: config.rerank.clone().map(RerankClient::new), + query_cache: QueryCache::new(QUERY_CACHE_ENTRIES), } } } diff --git a/crates/scry-server/src/query_cache.rs b/crates/scry-server/src/query_cache.rs new file mode 100644 index 0000000..239e393 --- /dev/null +++ b/crates/scry-server/src/query_cache.rs @@ -0,0 +1,72 @@ +//! Query vectors keyed by query text. A vector depends only on the +//! embedder and the HyDE setting, both fixed for a server's lifetime, so +//! a repeat query skips the chat and embedding round trips. + +use std::collections::{HashMap, VecDeque}; +use std::sync::Mutex; + +pub struct QueryCache { + capacity: usize, + inner: Mutex, +} + +#[derive(Default)] +struct Entries { + vectors: HashMap>, + order: VecDeque, +} + +impl QueryCache { + pub fn new(capacity: usize) -> Self { + Self { + capacity, + inner: Mutex::new(Entries::default()), + } + } + + pub fn get(&self, query: &str) -> Option> { + self.inner + .lock() + .expect("query cache") + .vectors + .get(query) + .cloned() + } + + pub fn insert(&self, query: &str, vector: Vec) { + let mut entries = self.inner.lock().expect("query cache"); + if entries.vectors.insert(query.to_string(), vector).is_none() { + entries.order.push_back(query.to_string()); + } + while entries.order.len() > self.capacity { + let oldest = entries.order.pop_front().expect("non-empty"); + entries.vectors.remove(&oldest); + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn returns_inserted_vectors_and_evicts_oldest() { + let cache = QueryCache::new(2); + cache.insert("a", vec![1.0]); + cache.insert("b", vec![2.0]); + cache.insert("c", vec![3.0]); + assert_eq!(cache.get("a"), None); + assert_eq!(cache.get("b"), Some(vec![2.0])); + assert_eq!(cache.get("c"), Some(vec![3.0])); + } + + #[test] + fn reinserting_a_key_does_not_grow_the_order() { + let cache = QueryCache::new(2); + cache.insert("a", vec![1.0]); + cache.insert("a", vec![1.5]); + cache.insert("b", vec![2.0]); + assert_eq!(cache.get("a"), Some(vec![1.5])); + assert_eq!(cache.get("b"), Some(vec![2.0])); + } +} diff --git a/crates/scry-server/src/routes/answer.rs b/crates/scry-server/src/routes/answer.rs index 4cff703..2571bc8 100644 --- a/crates/scry-server/src/routes/answer.rs +++ b/crates/scry-server/src/routes/answer.rs @@ -2,7 +2,7 @@ use std::sync::Arc; use axum::Json; use axum::extract::State; -use scry_core::search::{SearchOptions, query_vector, search_with_vector}; +use scry_core::search::{SearchOptions, search_with_vector}; use crate::AppState; use crate::api::{AnswerRequest, AnswerResponse, Citation}; @@ -10,7 +10,7 @@ use crate::error::ApiError; const LOCAL_SOURCES: usize = 8; const WEB_SOURCES: usize = 5; -const SNIPPET_CHARS: usize = 1200; +const SNIPPET_CHARS: usize = 1600; struct Source { label: String, @@ -29,15 +29,10 @@ pub async fn answer( let mut sources: Vec = Vec::new(); if let Some(repo_key) = &request.repo_key { - let vector = query_vector( - state.embedder.as_ref(), - state.chat.as_ref(), - state.hyde, - &request.query, - ) - .await?; + let vector = super::search::query_vector_cached(&state, &request.query).await?; let repo_key = repo_key.clone(); let query = request.query.clone(); + let pool = super::search::pool_size(&state, request.rerank, LOCAL_SOURCES); let hits = state .store .call(move |store| { @@ -45,12 +40,15 @@ pub async fn answer( return Ok(Vec::new()); }; let options = SearchOptions { - limit: LOCAL_SOURCES, + limit: pool, path_prefix: None, }; search_with_vector(store, Some(repo_id), &query, &vector, &options) }) .await?; + let hits = + super::search::reranked(&state, request.rerank, &request.query, hits, LOCAL_SOURCES) + .await; sources.extend(hits.into_iter().map(|hit| Source { label: format!("{}:{}-{}", hit.relpath, hit.start_line, hit.end_line), text: truncate(&hit.content, SNIPPET_CHARS), @@ -72,8 +70,11 @@ pub async fn answer( } let mut prompt = String::from( - "Answer the question using only the numbered sources below. Cite every \ - claim with [N] markers referring to source numbers. Be concise.\n\n", + "Answer the question using only the numbered sources below: excerpts from \ + the repository's files, labelled path:start-end, or web pages labelled by \ + URL. Cite every claim with [N] markers referring to source numbers. If the \ + sources do not settle the question, say what is missing instead of \ + guessing. Be concise.\n\n", ); for (i, source) in sources.iter().enumerate() { prompt.push_str(&format!( diff --git a/crates/scry-server/src/routes/search.rs b/crates/scry-server/src/routes/search.rs index fc72061..cbc4ec4 100644 --- a/crates/scry-server/src/routes/search.rs +++ b/crates/scry-server/src/routes/search.rs @@ -2,9 +2,8 @@ use std::sync::Arc; use axum::Json; use axum::extract::State; -use scry_core::index::embed_input; use scry_core::rerank::fuse; -use scry_core::search::{SearchOptions, query_vector, search_with_vector}; +use scry_core::search::{SearchHit, SearchOptions, query_vector, search_with_vector}; use crate::AppState; use crate::api::{Hit, SearchRequest, SearchResponse}; @@ -19,20 +18,67 @@ fn truncated(mut hits: Vec, limit: usize) -> Vec { hits } -pub async fn search( - State(state): State>, - Json(request): Json, -) -> Result, ApiError> { +/// The query's vector, from the cache when this server has embedded the +/// same text before. +pub(super) async fn query_vector_cached( + state: &AppState, + query: &str, +) -> scry_core::Result> { + if let Some(vector) = state.query_cache.get(query) { + return Ok(vector); + } let vector = query_vector( state.embedder.as_ref(), state.chat.as_ref(), state.hyde, - &request.query, + query, ) .await?; - let rerank = state.rerank.as_ref().filter(|_| request.rerank); - let pool = rerank.map_or(request.limit, |client| request.limit.max(client.top_n())); - let (query, limit) = (request.query.clone(), request.limit); + state.query_cache.insert(query, vector.clone()); + Ok(vector) +} + +/// How many fused candidates to retrieve so the reranker, when it runs, +/// sees its full `top_n` pool. +pub(super) fn pool_size(state: &AppState, rerank: bool, limit: usize) -> usize { + state + .rerank + .as_ref() + .filter(|_| rerank) + .map_or(limit, |client| limit.max(client.top_n())) +} + +pub(super) async fn reranked( + state: &AppState, + rerank: bool, + query: &str, + hits: Vec, + limit: usize, +) -> Vec { + let Some(client) = state.rerank.as_ref().filter(|_| rerank) else { + return hits; + }; + let documents: Vec = hits.iter().map(|h| client.document(h)).collect(); + match tokio::time::timeout(RERANK_BUDGET, client.rerank(query, &documents)).await { + Ok(Ok(ranked)) => fuse(hits, &ranked, client.weight(), limit), + Ok(Err(error)) => { + tracing::warn!("rerank failed, returning fused order: {error}"); + truncated(hits, limit) + } + Err(_) => { + tracing::warn!("rerank exceeded {RERANK_BUDGET:?}, returning fused order"); + truncated(hits, limit) + } + } +} + +pub async fn search( + State(state): State>, + Json(request): Json, +) -> Result, ApiError> { + let vector = query_vector_cached(&state, &request.query).await?; + let pool = pool_size(&state, request.rerank, request.limit); + let (query, limit, rerank) = (request.query.clone(), request.limit, request.rerank); let hits = state .store .call(move |store| { @@ -51,26 +97,7 @@ pub async fn search( }) .await? .ok_or_else(|| ApiError::NotFound("repo not indexed".to_string()))?; - let hits = match rerank { - Some(client) => { - let documents: Vec = hits - .iter() - .map(|h| embed_input(&h.repo_key, &h.relpath, h.symbol.as_deref(), &h.content)) - .collect(); - match tokio::time::timeout(RERANK_BUDGET, client.rerank(&query, &documents)).await { - Ok(Ok(ranked)) => fuse(hits, &ranked, client.weight(), limit), - Ok(Err(error)) => { - tracing::warn!("rerank failed, returning fused order: {error}"); - truncated(hits, limit) - } - Err(_) => { - tracing::warn!("rerank exceeded {RERANK_BUDGET:?}, returning fused order"); - truncated(hits, limit) - } - } - } - None => hits, - }; + let hits = reranked(&state, rerank, &query, hits, limit).await; Ok(Json(SearchResponse { hits: hits .into_iter() diff --git a/crates/scry-server/tests/http.rs b/crates/scry-server/tests/http.rs index 33694e1..27b9c98 100644 --- a/crates/scry-server/tests/http.rs +++ b/crates/scry-server/tests/http.rs @@ -9,7 +9,7 @@ use scry_server::api::{ FileUpload, ManifestRequest, SearchRequest, SearchResponse, StatusResponse, SyncRequest, SyncResponse, }; -use scry_server::{AppState, StoreHandle, router}; +use scry_server::{AppState, QueryCache, StoreHandle, router}; const TOKEN: &str = "test-token"; const REPO: &str = "github.com/test/http"; @@ -34,6 +34,7 @@ async fn spawn_server_with(rerank: Option) -> String { memory_config: config.memory.clone(), tavily: None, rerank, + query_cache: QueryCache::new(8), }; let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); let base = format!("http://{}", listener.local_addr().unwrap()); diff --git a/crates/scry/src/cli.rs b/crates/scry/src/cli.rs index 96c33ec..be7f6ba 100644 --- a/crates/scry/src/cli.rs +++ b/crates/scry/src/cli.rs @@ -14,7 +14,8 @@ const USAGE: &str = "scry - self-hosted semantic code search USAGE: scry \"natural language query\" [path] search the current repo scry serve run the index server - scry index sync this repo to the server + scry index [--full] sync this repo to the server + (--full re-chunks every file) scry watch sync continuously while you work scry status show server index counts scry eval [--runs N] [--limit N] score retrieval against a golden set @@ -98,7 +99,7 @@ pub async fn dispatch(args: &[String]) -> Result<()> { Ok(()) } Some("serve") => commands::serve::run().await, - Some("index") => commands::index::run().await, + Some("index") => commands::index::run(&args[1..]).await, Some("watch") => commands::watch::run().await, Some("status") => commands::status::run().await, Some("eval") => commands::eval::run(&args[1..]).await, diff --git a/crates/scry/src/commands/index.rs b/crates/scry/src/commands/index.rs index 709efa5..0b5cad5 100644 --- a/crates/scry/src/commands/index.rs +++ b/crates/scry/src/commands/index.rs @@ -11,7 +11,8 @@ use super::repo_context; const BATCH_FILES: usize = 16; const BATCH_BYTES: usize = 2 * 1024 * 1024; -pub async fn run() -> Result<()> { +pub async fn run(args: &[String]) -> Result<()> { + let full = args.iter().any(|a| a == "--full"); let ctx = repo_context()?; if super::at_or_above_home(&ctx.identity.root) { bail!( @@ -19,7 +20,7 @@ pub async fn run() -> Result<()> { ctx.identity.root.display() ); } - let outcome = sync_repo(&ctx).await?; + let outcome = sync_repo(&ctx, full).await?; println!( "indexed {} files ({} embedded, {} reused chunks), deleted {}, unchanged {}", outcome.indexed_files, @@ -40,7 +41,9 @@ pub struct SyncOutcome { pub unchanged: usize, } -pub async fn sync_repo(ctx: &RepoContext) -> Result { +/// `full` re-uploads every file regardless of the manifest, which is how +/// a chunker change reaches files whose content has not changed. +pub async fn sync_repo(ctx: &RepoContext, full: bool) -> Result { let entries = walk_repo(&ctx.identity.root, ctx.config.index.max_file_size)?; if entries.len() > ctx.config.index.max_file_count { bail!( @@ -72,7 +75,7 @@ pub async fn sync_repo(ctx: &RepoContext) -> Result { for entry in &entries { let path = ctx.identity.root.join(&entry.relpath); let hash = hashing::hex(hashing::hash_file(&path)?); - if manifest.get(&entry.relpath) == Some(&hash) { + if !full && manifest.get(&entry.relpath) == Some(&hash) { outcome.unchanged += 1; continue; } diff --git a/crates/scry/src/commands/search.rs b/crates/scry/src/commands/search.rs index c7a6054..7b967c3 100644 --- a/crates/scry/src/commands/search.rs +++ b/crates/scry/src/commands/search.rs @@ -60,6 +60,7 @@ pub async fn run(args: SearchArgs) -> Result<()> { query: args.query, repo_key: scope.repo_key, web: args.web, + rerank: args.rerank, }) .await?; println!("{}\n", response.answer); diff --git a/crates/scry/src/commands/watch.rs b/crates/scry/src/commands/watch.rs index 70254bd..8bcd55d 100644 --- a/crates/scry/src/commands/watch.rs +++ b/crates/scry/src/commands/watch.rs @@ -15,7 +15,7 @@ pub async fn run() -> Result<()> { ); } - let outcome = sync_repo(&ctx).await?; + let outcome = sync_repo(&ctx, false).await?; println!( "watching {} (key {}): {} files indexed, {} unchanged", ctx.identity.root.display(), @@ -38,7 +38,7 @@ pub async fn run() -> Result<()> { .watch(&ctx.identity.root, RecursiveMode::Recursive)?; while rx.recv().await.is_some() { - match sync_repo(&ctx).await { + match sync_repo(&ctx, false).await { Ok(outcome) if outcome.indexed_files + outcome.deleted_files > 0 => { println!( "synced: {} files indexed, {} deleted", diff --git a/docs/search.md b/docs/search.md index 5b4f401..1966f5f 100644 --- a/docs/search.md +++ b/docs/search.md @@ -17,14 +17,26 @@ a greedy near-duplicate filter before the final ranking. With a `[rerank]` endpoint configured, the fused top `top_n` candidates are scored by a cross-encoder and its ranking joins the fusion as a -third reciprocal-rank leg with `weight` (k=60, default 1.0). The -reranker changes order only; the displayed score stays the dense cosine. -A reranker that errors or takes longer than six seconds is skipped for -that query, and `--no-rerank` skips it per query. +third reciprocal-rank leg with `weight` (k=60, default 1.0). Each +document is cut to `max_chars` (default 3000) before scoring; the header +and the start of a chunk carry the signal, and shorter inputs are both +faster and, measured on the finance-query golden set, better on recall +than full-length ones. The reranker changes order only; the displayed +score stays the dense cosine. A reranker that errors or takes longer than +six seconds is skipped for that query, and `--no-rerank` skips it per +query. + +The server keeps the last 256 query vectors, so a repeated query skips +HyDE and embedding and pays only for retrieval and the reranker. Chunks are function-level where a tree-sitter grammar exists (16 languages), blank-line-snapped windows elsewhere, and each chunk is -embedded with a `repo > path > symbol` header for context. +embedded with a `repo > path > symbol` header for context. Comments and +attributes directly above a definition stay with it, and spans under four +lines merge into a neighbour, so a bare `#[cfg]` line or a one-line `mod` +declaration never becomes its own hit. Files whose first lines carry the +`@generated` marker are not chunked at all. A chunker change reaches +already-indexed files with `scry index --full`. Measured hot paths are gated in CI by soothfast: diff --git a/soothfast.toml b/soothfast.toml index 818b894..74b5ba7 100644 --- a/soothfast.toml +++ b/soothfast.toml @@ -1,6 +1,6 @@ [site] name = "scry" -version = "0.1.0" +version = "0.2.0" repo = "https://github.com/Verdenroz/scry" # Brand seeds; soothfast derives the full light+dark Material role set.