diff --git a/CLAUDE.md b/CLAUDE.md index c621f83..6782c3a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -141,10 +141,25 @@ rest; a file matching no rule and no fallback **fails the build**. `aiIndex.plan Markdown plan (file → rule id → prompt → rough time estimate, summed per model and overall) and stops (no model loaded). The plan also computes each routed file's **context-window fit** up front (via `AiInputWindowCalculator`, the same threshold the run uses to trim): a file larger than its routed -model's window is flagged in the plan and **always fails the build** (a hard abort). The resolution is -**configuration only** — the plugin never auto-picks a model: the user must add a `` -rule with a size `` that routes oversized files to a model with a large enough context window -(see the big-window fallback example in the POM). See `AiFieldGenerationSelector` (selection + validation), `AiCondition`/ +model's window is flagged in the plan and, **by default (`onOversize=fail`), fails the build** (a hard +abort) — the plugin never auto-picks a model. The resolution is **configuration only**, via the rule's +**``** strategy (`AiOversizeStrategy`): `fail` (default), `sample` (trim to the window head, +one call), `mapReduce` (chunk at line boundaries → summarize each → combine via a **hierarchical reduce** +that batches partials to fit the window so whole-file `=0` coverage isn't trimmed away; +`` bounds the time — `AiSourceChunker`; the chunk window carries a token-safety headroom so a +token-dense chunk can't overflow the model context), or `deterministic` (model-free metadata+sample body +— `AiDeterministicSummary`). +So oversized files are either routed to a larger-context model or handled by a strategy; only `fail` +entries abort. A rule may also carry an orthogonal, language-agnostic **``** list (`AiFactCounter` ++ `AiFactExtractor`): each `{label, pattern}` reports its regex match count over the **whole** source, +prepended to the body of **every** file the rule matches (oversize or not) — exact structural counts +(SQL `INSERT` rows / tables / views, Java type declarations / `\bboolean\b` fields, …) that give +downstream agents authoritative numbers the (possibly sampled) AI prose cannot reliably produce. A fact +set can be defined once in a shared `` group and referenced from many rules by +`` (`AiFactDefinition` + `AiFactDefinitionSupport`, resolved onto each rule before indexing), +instead of repeating the `` block inline. Generation is also guarded by a **retry-on-overflow**: +if the provider rejects a prompt as exceeding the context (the char-based trim under-counted tokens on +dense content), the call is re-trimmed to a smaller window and retried rather than failing the build. See `AiFieldGenerationSelector` (selection + validation), `AiCondition`/ `AiConditionEvaluator` (the tree), and `AiIndexPlan` (the rendered plan). This is how one run can index different file kinds/sizes with different models *and* prompts. @@ -189,6 +204,15 @@ the top of `execute()`. Covered by `MojoPhaseSkipTest`. | `AiCondition` / `AiConditionEvaluator` | Composable and/or/not condition tree (leaves: extensions/size/lines/modifiedAfter/modifiedBefore/pathGlob) + its evaluator (`matches`/`validate`/`usesLines`) | | `AiIndexPlan` | The routing plan: routes grouped by model id, skips, unmatched, per-file context-window fit; renders the up-front tree | | `AiInputWindowCalculator` | Pure calculator for the input window (max source chars before trimming + whether a source exceeds it); single source of truth shared by the run-time trim (`AiFieldGenerationSupport`) and the plan-time over-window check (`SourceFileIndexer.classify`) | +| `AiOversizeStrategy` | Enum of the per-rule `` strategies (`fail`/`sample`/`mapReduce`/`deterministic`) + case-insensitive parser; default `fail` | +| `AiSourceChunker` | Pure line-boundary chunker for `mapReduce` (overlap + `maxChunks` representative sampling) | +| `AiFactCounter` | Maven `@Parameter` POJO for one `` — a `{label, pattern}` deterministic counter | +| `AiFactExtractor` | Pure: counts each `` regex over the whole source → the exact "facts" block prepended to the body (+ fail-fast pattern validation) | +| `AiFactDefinition` | Maven `@Parameter` POJO for a named, reusable `` group `{key, facts}` | +| `AiFactDefinitionSupport` | Key-indexed lookup: resolves each rule's `` to its shared group's counters (copies onto the rule) | +| `CalibrateMojo` / `AiCalibrationRunner` | `ai-index:calibrate` goal (thin orchestration) + the indexer-layer measurer (warmup + two sized generations, prefers the binding's reported throughput, else a wall-clock differential) → `AiCalibrationMeasurement` | +| `AiCalibration` | `` `@Parameter` POJO on `` (`prefillTokensPerSecond`/`decodeTokensPerSecond`/`charsPerToken`); makes `AiGenerationTimeEstimator` use measured per-machine rates instead of the built-in reference-CPU model | +| `AiDeterministicSummary` | Pure model-free body builder for `deterministic` (size, line count, head/tail sample) | | `AiProgressBar` | Pure ASCII progress-bar renderer (`[##### ] 42%`); `GenerateMojo` logs it after each file, advancing by the running sum of per-file plan estimates over the grand total (with estimated time left + actual elapsed) | | `PackageIndexer` | Creates `package.ai.md` files with contents listings, calls AI to fill the document body | | `ProjectIndexer` | Phase 3: harvests each package's lead + relative link into one `project.ai.md`; deterministic listing, with an optional one-call AI `#### Overview` from the leads | @@ -265,6 +289,7 @@ header block, so a `- F:` line in the body is never read as a link. | `ai-index:generate` | Phase 1: index source files and fill the AI-generated document body | | `ai-index:aggregate-packages` | Phase 2: aggregate package index files and fill the AI-generated document body | | `ai-index:aggregate-project` | Phase 3: harvest per-package leads into one `project.ai.md` (deterministic listing; optional one-call AI `#### Overview` when a `` is configured) | +| `ai-index:calibrate` | Preflight + per-machine timing (between `planOnly` and a real run): loads each routed model once (catches bad path / OOM / wrong native), measures prefill/decode throughput, and prints a paste-ready `` block per `` | ### Key Parameters (`GenerateMojo`) @@ -355,7 +380,7 @@ Immutable value types are implemented as Java `record` types (e.g., `AiMdDocumen | Dependency | Version | Purpose | |---|---|---| -| `net.ladenthin:llama` | 5.0.3 | llama.cpp JNI binding (GGUF inference); released on Maven Central, carries the prompt-cache/slot APIs + GPU classifier jars. Brings `slf4j-api` transitively, converged to 2.0.18 via ``. | +| `net.ladenthin:llama` | 5.0.4 | llama.cpp JNI binding (GGUF inference); released on Maven Central, carries the prompt-cache/slot APIs + GPU classifier jars. Brings `slf4j-api` transitively, converged to 2.0.18 via ``. | | `org.apache.maven:maven-plugin-api` | 3.9.13 | Maven plugin API (provided) | | `org.apache.maven.plugin-tools:maven-plugin-annotations` | 3.15.1 | `@Mojo`, `@Parameter` annotations (provided) | diff --git a/README.md b/README.md index d81a99c..f3e4bab 100644 --- a/README.md +++ b/README.md @@ -204,14 +204,40 @@ The plugin is configured from three building blocks, declared on the plugin insi `-DaiIndex.planOnly=true` to print the routing plan (a copy-pasteable Markdown table: file → rule id → prompt → context-window fit → rough time estimate, summed per model and overall) and stop before loading any model. The plan also checks each file against its routed model's **context window**: a - file too large for the window would lose content if trimmed, so the build **always fails** (a hard - abort). The fix is **configuration only** — the plugin never picks a model for you: add a - `` rule with a size `` that routes oversized files to a model with a large - enough window (see the `granite-4.0-h-tiny-bigwindow` definition + the `big-window-java` / `big-window-sql` rules in the POM — - IBM Granite 4.0-H-Tiny, Apache-2.0, a hybrid Mamba model whose KV cache grows only linearly, configured - at a 384K window to cover files up to ~1 MB; verified summarizing a ~995 KB / ~268K-token file on an - 8 GB GPU with no OOM. Quality is best within Granite's validated 128K (~500 KB) and degrades gradually - beyond it — a whole-file summary still beats a trimmed one). + file too large for the window would lose content if trimmed. What happens then is **configuration + only** — the plugin never picks a model for you — and is chosen per rule with ``: + - **`fail`** *(default)* — abort the build (a hard fail). The fix is to add a `` rule + with a size `` that routes oversized files to a model with a large enough window (see the + `granite-4.0-h-tiny-bigwindow` definition + the `big-window-java` / `big-window-sql` rules in the + POM — IBM Granite 4.0-H-Tiny, Apache-2.0, a hybrid Mamba model whose KV cache grows only linearly, + configured at a 384K window to cover files up to ~1 MB; verified summarizing a ~995 KB / ~268K-token + file on an 8 GB GPU with no OOM). + - **`sample`** — feed only the head of the file (trimmed to the window) in a single call. Fast and + bounded; good for repetitive data where the head represents the whole. + - **`mapReduce`** — split the file into window-sized chunks at line boundaries (with overlap so + records are never torn mid-syntax), summarize each chunk, then combine the partial summaries in one + final call. A `` cap bounds the time (a representative subset — always head + tail, + evenly spaced — is summarized when the file would exceed the cap), so an arbitrarily large file + (e.g. 7 MB) stays within a chosen per-file budget. **Route oversized files to a *small*, fast model + for this — not the big-window one.** Prefill is `O(n²)` in prompt length, so chunks should be small: + many cheap small-window passes are far faster than a few giant ones (one 384K-token pass alone can + dwarf a whole run). E.g. a 16K-window model with `maxChunks=6` is ~7 calls (~1 h order on a + reference CPU; less on GPU) and samples a representative slice — the right trade-off for repetitive + data. mapReduce *on* a big window is the slowest possible combination; avoid it. + - **`deterministic`** — no model at all: emit a deterministic body (size, line count, head/tail + sample). Instant; for pure data where no AI analysis is needed. + + Quality from a real model is best within Granite's validated 128K (~500 KB) and degrades gradually + beyond it — a whole-file (or chunked) summary still beats a trimmed one. + + **Exact counts with `` (optional, every file — not just oversize).** A sampled/chunked AI + summary can't reliably *count* things (no single call sees the whole file — it will guess "25 rows"). + Add a `` list to a rule and each `{label, pattern}` reports its regex match count over the + **whole** source, prepended to the body of every file the rule matches as an exact facts line — so + downstream agents get authoritative structural counts in every summary. It's fully generic — the + meaning is in the regex, so the same mechanism counts SQL `INSERT` rows or Java `\bboolean\b` fields; + multi-line matching is opt-in via the inline `(?m)` flag. Keep patterns robust (a fact that miscounts + is worse than none). Example: `**Facts (exact, whole file):** INSERT rows: 36738; tables: 122; views: 4`. ```xml @@ -376,6 +402,18 @@ and a fallback for everything else: 49152 + + java-hugefile-body-javagpt-oss-20B-c16k + mapReduce6 + + \bboolean\b + (?m)^\s+(public|private|protected).*\( + + + .java + 1048576 + + fallbacktrue file-body-fallbackgpt-oss-20B-c96k @@ -385,7 +423,10 @@ and a fallback for everything else: Notes: size bounds are **min-exclusive / max-inclusive** so `band2.min == band1.max` is non-overlapping; `` works the same way; `2026-01-01T00:00:00Z` only (re)indexes -recently changed files. Preview the mapping without running a model: +recently changed files. `` (`fail` *(default)* / `sample` / `mapReduce` / `deterministic` — +see the context-window note above) chooses what a rule does when a file is still larger than its routed +model's window; `` caps how many chunks `mapReduce` summarizes. Preview the mapping without +running a model: ``` mvn ai-index:generate -DaiIndex.planOnly=true diff --git a/TODO.md b/TODO.md index b48da88..645add7 100644 --- a/TODO.md +++ b/TODO.md @@ -15,8 +15,6 @@ recorded in git history and `crossrepostatus.md`, not here. - **Expand PIT mutation scope (optional).** `pom.xml` wires `100` over an explicit `` list (config / document / prompt / provider / support value+logic classes, plus `indexer.AiInputWindowCalculator` and `support.AiProgressBar`), all killed at 100%. Still out (optional, need careful fixtures): `document.AiMdDocumentCodec` / `AiMdHeaderCodec`, `prompt.AiPromptPreparationSupport`, and the newer `indexer.AiIndexPlan` / `config.AiConditionGroup`. The orchestration layers (`indexer.*` walk, `mojo.*`) and the JNI provider stay out of PIT — they need a Maven/native context rather than pure-unit mutation (see crossrepostatus "Deliberate non-parity"). -- **Big-window fallback beyond ~1 MB.** The `granite-4.0-h-tiny-bigwindow` preset (384K context) covers source files up to ~1 MB; larger files hard-fail by design. If genuinely needed, add a still-larger preset (Granite allocates up to its 512K training limit, with further quality loss) or document a file-splitting workflow. - - **Cross-repo code-quality TODOs** — see [`../workspace/policies/code-quality-todos.md`](../workspace/policies/code-quality-todos.md) for the canonical `@VisibleForTesting` design-fit review, package hierarchy review, and class/method naming review. This repo has no `@VisibleForTesting` usages today; the package and naming reviews are still open here. - **No LogCaptor smoke test needed (note, not an action).** This module has no logging code (`org.slf4j.*` not used in `src/main/java/`); production uses Maven's `Log` interface. If SLF4J logging is ever introduced, add a LogCaptor smoke test at the same time so the binding/configuration is exercised in tests. diff --git a/pom.xml b/pom.xml index 079ea44..850a5fb 100644 --- a/pom.xml +++ b/pom.xml @@ -62,7 +62,7 @@ SPDX-License-Identifier: Apache-2.0 - 5.0.3 + 5.0.4 org.pitest pitest-maven @@ -714,6 +714,7 @@ SPDX-License-Identifier: Apache-2.0 + net.ladenthin.maven.llamacpp.aiindex.config.AiCalibration net.ladenthin.maven.llamacpp.aiindex.config.AiCondition net.ladenthin.maven.llamacpp.aiindex.config.AiConditionEvaluator net.ladenthin.maven.llamacpp.aiindex.config.AiFieldGenerationConfig @@ -723,6 +724,7 @@ SPDX-License-Identifier: Apache-2.0 net.ladenthin.maven.llamacpp.aiindex.config.AiRangeCondition net.ladenthin.maven.llamacpp.aiindex.config.AiModelDefinition net.ladenthin.maven.llamacpp.aiindex.config.AiModelDefinitionSupport + net.ladenthin.maven.llamacpp.aiindex.indexer.AiCalibrationMeasurement net.ladenthin.maven.llamacpp.aiindex.indexer.AiInputWindowCalculator net.ladenthin.maven.llamacpp.aiindex.document.AiGenerationRequest net.ladenthin.maven.llamacpp.aiindex.document.AiGenerationResult @@ -736,10 +738,17 @@ SPDX-License-Identifier: Apache-2.0 net.ladenthin.maven.llamacpp.aiindex.prompt.AiPromptPreparationSupport net.ladenthin.maven.llamacpp.aiindex.prompt.AiPromptSupport net.ladenthin.maven.llamacpp.aiindex.provider.AiCompletionParser + net.ladenthin.maven.llamacpp.aiindex.provider.AiGenerationTimings net.ladenthin.maven.llamacpp.aiindex.provider.AiGenerationProvider net.ladenthin.maven.llamacpp.aiindex.provider.AiGenerationProviderFactory net.ladenthin.maven.llamacpp.aiindex.provider.MockAiGenerationProvider + net.ladenthin.maven.llamacpp.aiindex.config.AiOversizeStrategy + net.ladenthin.maven.llamacpp.aiindex.config.AiFactCounter + net.ladenthin.maven.llamacpp.aiindex.config.AiFactDefinition + net.ladenthin.maven.llamacpp.aiindex.config.AiFactDefinitionSupport + net.ladenthin.maven.llamacpp.aiindex.config.AiFactExtractor net.ladenthin.maven.llamacpp.aiindex.support.AiChecksumSupport + net.ladenthin.maven.llamacpp.aiindex.support.AiDeterministicSummary net.ladenthin.maven.llamacpp.aiindex.support.AiGenerationTimeEstimator net.ladenthin.maven.llamacpp.aiindex.support.AiPathSupport net.ladenthin.maven.llamacpp.aiindex.support.AiProgressBar @@ -854,7 +863,7 @@ SPDX-License-Identifier: Apache-2.0 @@ -1111,7 +1120,7 @@ SPDX-License-Identifier: Apache-2.0 @@ -1133,7 +1142,7 @@ SPDX-License-Identifier: Apache-2.0 @@ -1157,7 +1166,7 @@ SPDX-License-Identifier: Apache-2.0 Same model as Q4_K_M above; Q6_K_P is a custom lossless quantization by HauhauCS that preserves quality where it matters most (effectively 1-2 quant levels higher at only 5-15 % larger file size). - Temperature 1.0 is critical for Gemma 4 — lowering degrades quality. + Temperature 1.0 is critical for Gemma 4 - lowering degrades quality. 32k context is a practical sweet spot for code summarization on local hardware. https://huggingface.co/HauhauCS/Gemma-4-E4B-Uncensored-HauhauCS-Aggressive --> @@ -1177,7 +1186,7 @@ SPDX-License-Identifier: Apache-2.0 <end_of_turn> - @@ -1314,10 +1323,10 @@ SPDX-License-Identifier: Apache-2.0 1.05 gpt-oss-20B-c16k @@ -1560,9 +1569,11 @@ SPDX-License-Identifier: Apache-2.0 window at run time); the cpt-3 check admits up to ~1.16 MB. reasoningEffort is empty: Granite has no gpt-oss harmony channels, so no reasoning kwarg is sent. The size-routing rules that target this key live in the ai-generate - below (ids big-window-java and big-window-sql). For >~1 MB files, split them or - add a still-larger preset (Granite allocates up to its 512K training limit - here, with further quality loss). --> + below (ids big-window-java and big-window-sql), for the + ~275 KB .. ~1 MB band (one pass). Files > ~1 MB are NOT summarized in one + giant pass here (O(n^2) prefill makes that far too slow); they route to the + small-window granite-4.0-h-1b-fastchunk preset via the huge-java/huge-sql + rules and are map-reduced (cheap small chunks, maxChunks-bounded). --> granite-4.0-h-tiny-bigwindow X:/Modelle/granite-4.0-h-tiny-Q4_K_M.gguf @@ -1582,6 +1593,43 @@ SPDX-License-Identifier: Apache-2.0 + + + granite-4.0-h-1b-fastchunk + X:/Modelle/granite-4.0-h-1b-Q4_K_M.gguf + 16384 + ${ai.gpuLayers} + ${ai.mainGpu} + ${ai.devices} + 1536 + 0.2 + 8 + 3 + true + 1.0 + 0 + 0.05 + 1.0 + + + EXP-Qwen3-Coder-30B X:/Modelle/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf @@ -1596,7 +1644,7 @@ SPDX-License-Identifier: Apache-2.0 1.05 @@ -1665,7 +1713,7 @@ SPDX-License-Identifier: Apache-2.0 You are a Java code indexer. Produce a dense, structured markdown summary of ONE Java source file for a searchable code index. Prioritize core logic, types, and data flow. Avoid filler words. Write in English. ## Output: start with the lead, then the sections below. -Begin with a single blockquote line — one sentence capturing what this file does in DOMAIN terms: the business/functional purpose a developer would search for (what it accomplishes), not how it is named or structured. Prefer the words a human would search ("Calculates VAT for invoices") over class mechanics ("Service class with three methods"): +Begin with a single blockquote line - one sentence capturing what this file does in DOMAIN terms: the business/functional purpose a developer would search for (what it accomplishes), not how it is named or structured. Prefer the words a human would search ("Calculates VAT for invoices") over class mechanics ("Service class with three methods"): > Then output these sections as markdown. Omit any section that does not apply. Use '####' for section labels. Never use '###' or higher. @@ -1717,7 +1765,7 @@ line, then a blank line, then the source. Index only what is literally present t You are a SQL schema indexer. Produce a dense, structured markdown summary of ONE SQL source file (DDL/DML: tables, views, procedures, functions, triggers) for a searchable code index. Prioritize the schema shape, the tables read vs written, columns, and relationships. Avoid filler words. Write in English. ## Output: start with the lead, then the sections below. -Begin with a single blockquote line — one sentence capturing what this file does in DOMAIN terms: the business/functional purpose a developer would search for (what data it models or what operation it performs), not how it is named: +Begin with a single blockquote line - one sentence capturing what this file does in DOMAIN terms: the business/functional purpose a developer would search for (what data it models or what operation it performs), not how it is named: > Then output these sections as markdown. Omit any section that does not apply. Use '####' for section labels. Never use '###' or higher. @@ -1765,7 +1813,7 @@ line, then a blank line, then the source. Index only what is literally present t You are a multi-language code indexer. Produce a dense, structured markdown summary of ONE source file (Java, C/C++, OpenCL, SQL, or similar) for a searchable code index. Prioritize core logic and data flow. Avoid filler words. Write in English. ## Output: start with the lead, then the sections below. -Begin with a single blockquote line — one sentence capturing what this file does in DOMAIN terms: the business/functional purpose a developer would search for (what it accomplishes), not how it is named or structured. Prefer the words a human would search ("Calculates VAT for invoices") over class mechanics ("Service class with three methods"): +Begin with a single blockquote line - one sentence capturing what this file does in DOMAIN terms: the business/functional purpose a developer would search for (what it accomplishes), not how it is named or structured. Prefer the words a human would search ("Calculates VAT for invoices") over class mechanics ("Service class with three methods"): > Then output these sections as markdown. Omit any section that does not apply. Use '####' for section labels. Never use '###' or higher. @@ -1810,10 +1858,10 @@ line, then a blank line, then the source. Index only what is literally present t package-body