From c7932e7af8a991767fd2d07c3c027db8a20a1d7f Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Tue, 30 Jun 2026 22:30:25 +0200 Subject: [PATCH 01/18] feat: configurable onOversize handling for files beyond the model window Adds a per-rule strategy so a file larger than its routed model's context window is handled deliberately instead of always aborting: - fail (default) hard abort; route oversized files to a bigger window - sample feed only the head (trimmed) in one call - mapReduce chunk at line boundaries (+overlap), summarize each, combine in a final call; -capped (head+tail+ evenly-spaced representative subset) - deterministic model-free body (size, line count, head/tail sample) New: AiOversizeStrategy (parsing/validation), AiSourceChunker (line-boundary chunking + representative selection), AiDeterministicSummary (model-free body). AiFieldGenerationConfig gains onOversize/maxChunks; AiFieldGenerationSupport branches on the strategy and implements the map-reduce loop; SourceFileIndexer estimates oversize routes (0 for deterministic, (chunks+1)*window for mapReduce); AiIndexPlan/GenerateMojo only hard-fail when a still-over-window file is routed with onOversize=fail. Selector validates onOversize fail-fast. Quality gates: mvn verify BUILD SUCCESS, SpotBugs clean (scoped IMPROPER_UNICODE suppression for the ASCII config-token match in fromConfig), PIT 498/498 (100%) on the gated classes. Docs updated (CLAUDE.md, README.md, TODO.md). Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd --- CLAUDE.md | 14 +- README.md | 41 ++++-- TODO.md | 2 +- pom.xml | 18 ++- spotbugs-exclude.xml | 14 ++ .../config/AiFieldGenerationConfig.java | 61 +++++++++ .../config/AiFieldGenerationSelector.java | 2 + .../aiindex/config/AiOversizeStrategy.java | 85 +++++++++++++ .../indexer/AiFieldGenerationSupport.java | 116 +++++++++++++++++ .../llamacpp/aiindex/indexer/AiIndexPlan.java | 52 +++++++- .../aiindex/indexer/SourceFileIndexer.java | 59 +++++++-- .../llamacpp/aiindex/mojo/GenerateMojo.java | 25 ++-- .../support/AiDeterministicSummary.java | 94 ++++++++++++++ .../aiindex/support/AiSourceChunker.java | 98 ++++++++++++++ .../config/AiFieldGenerationConfigTest.java | 24 ++++ .../config/AiFieldGenerationSelectorTest.java | 14 ++ .../config/AiOversizeStrategyTest.java | 55 ++++++++ .../indexer/AiFieldGenerationSupportTest.java | 120 ++++++++++++++++++ .../support/AiDeterministicSummaryTest.java | 73 +++++++++++ .../aiindex/support/AiSourceChunkerTest.java | 78 ++++++++++++ 20 files changed, 997 insertions(+), 48 deletions(-) create mode 100644 src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiOversizeStrategy.java create mode 100644 src/main/java/net/ladenthin/maven/llamacpp/aiindex/support/AiDeterministicSummary.java create mode 100644 src/main/java/net/ladenthin/maven/llamacpp/aiindex/support/AiSourceChunker.java create mode 100644 src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiOversizeStrategyTest.java create mode 100644 src/test/java/net/ladenthin/maven/llamacpp/aiindex/support/AiDeterministicSummaryTest.java create mode 100644 src/test/java/net/ladenthin/maven/llamacpp/aiindex/support/AiSourceChunkerTest.java diff --git a/CLAUDE.md b/CLAUDE.md index c621f83..fe73439 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -141,10 +141,13 @@ rest; a file matching no rule and no fallback **fails the build**. `aiIndex.plan Markdown plan (file → rule id → prompt → rough time estimate, summed per model and overall) and stops (no model loaded). The plan also computes each routed file's **context-window fit** up front (via `AiInputWindowCalculator`, the same threshold the run uses to trim): a file larger than its routed -model's window is flagged in the plan and **always fails the build** (a hard abort). The resolution is -**configuration only** — the plugin never auto-picks a model: the user must add a `` -rule with a size `` that routes oversized files to a model with a large enough context window -(see the big-window fallback example in the POM). See `AiFieldGenerationSelector` (selection + validation), `AiCondition`/ +model's window is flagged in the plan and, **by default (`onOversize=fail`), fails the build** (a hard +abort) — the plugin never auto-picks a model. The resolution is **configuration only**, via the rule's +**``** strategy (`AiOversizeStrategy`): `fail` (default), `sample` (trim to the window head, +one call), `mapReduce` (chunk at line boundaries → summarize each → combine; `` bounds the +time — `AiSourceChunker`), or `deterministic` (model-free metadata+sample body — `AiDeterministicSummary`). +So oversized files are either routed to a larger-context model or handled by a strategy; only `fail` +entries abort. See `AiFieldGenerationSelector` (selection + validation), `AiCondition`/ `AiConditionEvaluator` (the tree), and `AiIndexPlan` (the rendered plan). This is how one run can index different file kinds/sizes with different models *and* prompts. @@ -189,6 +192,9 @@ the top of `execute()`. Covered by `MojoPhaseSkipTest`. | `AiCondition` / `AiConditionEvaluator` | Composable and/or/not condition tree (leaves: extensions/size/lines/modifiedAfter/modifiedBefore/pathGlob) + its evaluator (`matches`/`validate`/`usesLines`) | | `AiIndexPlan` | The routing plan: routes grouped by model id, skips, unmatched, per-file context-window fit; renders the up-front tree | | `AiInputWindowCalculator` | Pure calculator for the input window (max source chars before trimming + whether a source exceeds it); single source of truth shared by the run-time trim (`AiFieldGenerationSupport`) and the plan-time over-window check (`SourceFileIndexer.classify`) | +| `AiOversizeStrategy` | Enum of the per-rule `` strategies (`fail`/`sample`/`mapReduce`/`deterministic`) + case-insensitive parser; default `fail` | +| `AiSourceChunker` | Pure line-boundary chunker for `mapReduce` (overlap + `maxChunks` representative sampling) | +| `AiDeterministicSummary` | Pure model-free body builder for `deterministic` (size, line count, head/tail sample) | | `AiProgressBar` | Pure ASCII progress-bar renderer (`[##### ] 42%`); `GenerateMojo` logs it after each file, advancing by the running sum of per-file plan estimates over the grand total (with estimated time left + actual elapsed) | | `PackageIndexer` | Creates `package.ai.md` files with contents listings, calls AI to fill the document body | | `ProjectIndexer` | Phase 3: harvests each package's lead + relative link into one `project.ai.md`; deterministic listing, with an optional one-call AI `#### Overview` from the leads | diff --git a/README.md b/README.md index d81a99c..a4d1fd5 100644 --- a/README.md +++ b/README.md @@ -204,14 +204,26 @@ The plugin is configured from three building blocks, declared on the plugin insi `-DaiIndex.planOnly=true` to print the routing plan (a copy-pasteable Markdown table: file → rule id → prompt → context-window fit → rough time estimate, summed per model and overall) and stop before loading any model. The plan also checks each file against its routed model's **context window**: a - file too large for the window would lose content if trimmed, so the build **always fails** (a hard - abort). The fix is **configuration only** — the plugin never picks a model for you: add a - `` rule with a size `` that routes oversized files to a model with a large - enough window (see the `granite-4.0-h-tiny-bigwindow` definition + the `big-window-java` / `big-window-sql` rules in the POM — - IBM Granite 4.0-H-Tiny, Apache-2.0, a hybrid Mamba model whose KV cache grows only linearly, configured - at a 384K window to cover files up to ~1 MB; verified summarizing a ~995 KB / ~268K-token file on an - 8 GB GPU with no OOM. Quality is best within Granite's validated 128K (~500 KB) and degrades gradually - beyond it — a whole-file summary still beats a trimmed one). + file too large for the window would lose content if trimmed. What happens then is **configuration + only** — the plugin never picks a model for you — and is chosen per rule with ``: + - **`fail`** *(default)* — abort the build (a hard fail). The fix is to add a `` rule + with a size `` that routes oversized files to a model with a large enough window (see the + `granite-4.0-h-tiny-bigwindow` definition + the `big-window-java` / `big-window-sql` rules in the + POM — IBM Granite 4.0-H-Tiny, Apache-2.0, a hybrid Mamba model whose KV cache grows only linearly, + configured at a 384K window to cover files up to ~1 MB; verified summarizing a ~995 KB / ~268K-token + file on an 8 GB GPU with no OOM). + - **`sample`** — feed only the head of the file (trimmed to the window) in a single call. Fast and + bounded; good for repetitive data where the head represents the whole. + - **`mapReduce`** — split the file into window-sized chunks at line boundaries (with overlap so + records are never torn mid-syntax), summarize each chunk, then combine the partial summaries in one + final call. A `` cap bounds the time (a representative subset — always head + tail, + evenly spaced — is summarized when the file would exceed the cap), so an arbitrarily large file + (e.g. 7 MB) stays within a chosen per-file budget. + - **`deterministic`** — no model at all: emit a deterministic body (size, line count, head/tail + sample). Instant; for pure data where no AI analysis is needed. + + Quality from a real model is best within Granite's validated 128K (~500 KB) and degrades gradually + beyond it — a whole-file (or chunked) summary still beats a trimmed one. ```xml @@ -376,6 +388,14 @@ and a fallback for everything else: 49152 + + java-hugefile-body-javagranite-4.0-h-tiny-bigwindow + mapReduce6 + + .java + 1048576 + + fallbacktrue file-body-fallbackgpt-oss-20B-c96k @@ -385,7 +405,10 @@ and a fallback for everything else: Notes: size bounds are **min-exclusive / max-inclusive** so `band2.min == band1.max` is non-overlapping; `` works the same way; `2026-01-01T00:00:00Z` only (re)indexes -recently changed files. Preview the mapping without running a model: +recently changed files. `` (`fail` *(default)* / `sample` / `mapReduce` / `deterministic` — +see the context-window note above) chooses what a rule does when a file is still larger than its routed +model's window; `` caps how many chunks `mapReduce` summarizes. Preview the mapping without +running a model: ``` mvn ai-index:generate -DaiIndex.planOnly=true diff --git a/TODO.md b/TODO.md index b48da88..d58953b 100644 --- a/TODO.md +++ b/TODO.md @@ -15,7 +15,7 @@ recorded in git history and `crossrepostatus.md`, not here. - **Expand PIT mutation scope (optional).** `pom.xml` wires `100` over an explicit `` list (config / document / prompt / provider / support value+logic classes, plus `indexer.AiInputWindowCalculator` and `support.AiProgressBar`), all killed at 100%. Still out (optional, need careful fixtures): `document.AiMdDocumentCodec` / `AiMdHeaderCodec`, `prompt.AiPromptPreparationSupport`, and the newer `indexer.AiIndexPlan` / `config.AiConditionGroup`. The orchestration layers (`indexer.*` walk, `mojo.*`) and the JNI provider stay out of PIT — they need a Maven/native context rather than pure-unit mutation (see crossrepostatus "Deliberate non-parity"). -- **Big-window fallback beyond ~1 MB.** The `granite-4.0-h-tiny-bigwindow` preset (384K context) covers source files up to ~1 MB; larger files hard-fail by design. If genuinely needed, add a still-larger preset (Granite allocates up to its 512K training limit, with further quality loss) or document a file-splitting workflow. +- **Files beyond the big-window preset — handled via ``.** The `granite-4.0-h-tiny-bigwindow` preset (384K context) covers source files up to ~1 MB. Larger files (or any file over its routed model's window) are handled per rule by ``: `fail` *(default — hard abort)*, `sample` (head only), `mapReduce` (chunk at line boundaries + combine, ``-capped), or `deterministic` (model-free metadata body). See `AiOversizeStrategy` / `AiSourceChunker` / `AiDeterministicSummary` and the README context-window note. Optional follow-up: a real end-to-end Granite `mapReduce` smoke test (orchestration is unit-tested + planOnly-verified today), and a still-larger 512K preset is wired in the POM only as an unused example. - **Cross-repo code-quality TODOs** — see [`../workspace/policies/code-quality-todos.md`](../workspace/policies/code-quality-todos.md) for the canonical `@VisibleForTesting` design-fit review, package hierarchy review, and class/method naming review. This repo has no `@VisibleForTesting` usages today; the package and naming reviews are still open here. diff --git a/pom.xml b/pom.xml index 079ea44..57d666c 100644 --- a/pom.xml +++ b/pom.xml @@ -739,7 +739,9 @@ SPDX-License-Identifier: Apache-2.0 net.ladenthin.maven.llamacpp.aiindex.provider.AiGenerationProvider net.ladenthin.maven.llamacpp.aiindex.provider.AiGenerationProviderFactory net.ladenthin.maven.llamacpp.aiindex.provider.MockAiGenerationProvider + net.ladenthin.maven.llamacpp.aiindex.config.AiOversizeStrategy net.ladenthin.maven.llamacpp.aiindex.support.AiChecksumSupport + net.ladenthin.maven.llamacpp.aiindex.support.AiDeterministicSummary net.ladenthin.maven.llamacpp.aiindex.support.AiGenerationTimeEstimator net.ladenthin.maven.llamacpp.aiindex.support.AiPathSupport net.ladenthin.maven.llamacpp.aiindex.support.AiProgressBar @@ -2036,13 +2038,23 @@ message: the path on the first line, then a blank line, then the summaries to sy the plain rule. Without rules like these, a file larger than the default model's window hard-fails the build (the plugin never auto-selects a model) - this is how YOU satisfy that check. Threshold - is conservative; tune to your model's window. (A file beyond Granite's - window fails too: split it or add a still-larger fallback.) --> + is conservative; tune to your model's window. Files larger than even + Granite's window do NOT fail here: these rules set onOversize=mapReduce + (chunk + summarize + combine, maxChunks-bounded), so arbitrarily large + data files are handled. (Default onOversize=fail still aborts for any + rule that does not opt into sample/mapReduce/deterministic.) --> big-window-java file-body-java granite-4.0-h-tiny-bigwindow 100 + + mapReduce + 6 @@ -2061,6 +2073,8 @@ message: the path on the first line, then a blank line, then the summaries to sy file-body-sql granite-4.0-h-tiny-bigwindow 100 + mapReduce + 6 diff --git a/spotbugs-exclude.xml b/spotbugs-exclude.xml index 1961980..b5437b4 100644 --- a/spotbugs-exclude.xml +++ b/spotbugs-exclude.xml @@ -428,4 +428,18 @@ SPDX-License-Identifier: Apache-2.0 + + + + + + + diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfig.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfig.java index 6f9929f..b372408 100644 --- a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfig.java +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfig.java @@ -75,6 +75,21 @@ public AiFieldGenerationConfig() { */ private boolean skip; + /** + * What to do when a matched file is larger than its routed model's context window. One of + * {@code fail} (default — abort the build), {@code sample} (trim to the window, summarize the head), + * {@code mapReduce} (chunk + summarize each + combine), {@code deterministic} (model-free metadata + * body). Parsed by {@link AiOversizeStrategy#fromConfig(String)}; {@code null}/blank = {@code fail}. + */ + private @Nullable String onOversize; + + /** + * For {@link AiOversizeStrategy#MAP_REDUCE}: the maximum number of chunks to summarize. {@code 0} + * (default) = unbounded (process every chunk). A positive value bounds the run time by sampling that + * many representative chunks (head + evenly spaced + tail) across the file. + */ + private int maxChunks; + /** * Returns the optional rule id (label), or {@code null} when not set. * @@ -200,4 +215,50 @@ public boolean isSkip() { public void setSkip(final boolean skip) { this.skip = skip; } + + /** + * Returns the raw {@code onOversize} config token, or {@code null} when not set. + * + * @return the oversize-strategy token, or {@code null} + */ + public @Nullable String getOnOversize() { + return onOversize; + } + + /** + * Sets the {@code onOversize} config token (one of {@code fail}/{@code sample}/{@code mapReduce}/{@code deterministic}). + * + * @param onOversize the oversize-strategy token + */ + public void setOnOversize(final @Nullable String onOversize) { + this.onOversize = onOversize; + } + + /** + * Returns the parsed oversize strategy ({@link AiOversizeStrategy#FAIL} when unset/blank). + * + * @return the oversize strategy + * @throws IllegalArgumentException if {@code onOversize} is non-blank and matches no strategy + */ + public AiOversizeStrategy getOversizeStrategy() { + return AiOversizeStrategy.fromConfig(onOversize); + } + + /** + * Returns the map-reduce chunk cap ({@code 0} = unbounded). + * + * @return the maximum number of chunks + */ + public int getMaxChunks() { + return maxChunks; + } + + /** + * Sets the map-reduce chunk cap ({@code 0} = unbounded). + * + * @param maxChunks the maximum number of chunks + */ + public void setMaxChunks(final int maxChunks) { + this.maxChunks = maxChunks; + } } diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationSelector.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationSelector.java index d2cf2a7..b8dec2b 100644 --- a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationSelector.java +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationSelector.java @@ -79,6 +79,8 @@ public void validate(final Iterable configs) { if (config == null) { continue; } + // Fail fast on an unknown onOversize token (throws IllegalArgumentException naming the value). + config.getOversizeStrategy(); if (config.isFallback()) { fallbackCount++; if (config.isSkip()) { diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiOversizeStrategy.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiOversizeStrategy.java new file mode 100644 index 0000000..f20d728 --- /dev/null +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiOversizeStrategy.java @@ -0,0 +1,85 @@ +// @formatter:off +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: Apache-2.0 +// @formatter:on +package net.ladenthin.maven.llamacpp.aiindex.config; + +import org.jspecify.annotations.Nullable; + +/** + * What to do when a source file is larger than its routed model's context window. + * + *

Configured per routing rule via {@code } (a string, parsed by + * {@link #fromConfig(String)}). The default is {@link #FAIL}: the build aborts and the user must route + * oversized files to a larger-context model (no automatic model choice). The other strategies let a rule + * deliberately handle oversized files:

+ * + *
    + *
  • {@link #SAMPLE} — feed only the head of the file (trimmed to the window) in one call; fast, + * bounded, good for repetitive data where the head represents the whole.
    • + *
  • {@link #MAP_REDUCE} — split the file into window-sized chunks at line boundaries, summarize each, + * then combine the partial summaries in one final call; processes far more of the file than a + * single window, with a {@code maxChunks} cap to bound the time.
    • + *
  • {@link #DETERMINISTIC} — no model at all: emit a deterministic body (size, line count, head/tail + * sample); instant, for pure data where no AI analysis is needed.
    • + *
+ */ +public enum AiOversizeStrategy { + + /** Abort the build; the user must configure a larger-context model (default). */ + FAIL("fail"), + + /** Trim the source to the window and summarize the head in a single call. */ + SAMPLE("sample"), + + /** Chunk at line boundaries, summarize each chunk, then combine the partial summaries. */ + MAP_REDUCE("mapReduce"), + + /** Emit a model-free deterministic body (metadata + head/tail sample). */ + DETERMINISTIC("deterministic"); + + /** The strategy applied when none is configured. */ + public static final AiOversizeStrategy DEFAULT = FAIL; + + /** The {@code } config token for this strategy. */ + private final String configValue; + + AiOversizeStrategy(final String configValue) { + this.configValue = configValue; + } + + /** + * Returns the {@code } config token (e.g. {@code "mapReduce"}). + * + * @return the config token + */ + public String configValue() { + return configValue; + } + + /** + * Parses a {@code } config value (case-insensitive). {@code null} or blank yields + * {@link #DEFAULT}. + * + * @param value the configured value, or {@code null} + * @return the matching strategy + * @throws IllegalArgumentException if {@code value} is non-blank and matches no strategy + */ + public static AiOversizeStrategy fromConfig(final @Nullable String value) { + if (value == null) { + return DEFAULT; + } + final String trimmed = value.trim(); + if (trimmed.isEmpty()) { + return DEFAULT; + } + for (final AiOversizeStrategy strategy : values()) { + if (strategy.configValue.equalsIgnoreCase(trimmed)) { + return strategy; + } + } + throw new IllegalArgumentException( + "Unknown onOversize strategy: '" + value + "' (expected one of fail/sample/mapReduce/deterministic)"); + } +} diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupport.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupport.java index e92d328..d359623 100644 --- a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupport.java +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupport.java @@ -12,13 +12,16 @@ import net.ladenthin.maven.llamacpp.aiindex.config.AiFieldGenerationConfig; import net.ladenthin.maven.llamacpp.aiindex.config.AiGenerationConfig; import net.ladenthin.maven.llamacpp.aiindex.config.AiModelDefinitionSupport; +import net.ladenthin.maven.llamacpp.aiindex.config.AiOversizeStrategy; import net.ladenthin.maven.llamacpp.aiindex.document.AiGenerationRequest; import net.ladenthin.maven.llamacpp.aiindex.document.AiGenerationResult; import net.ladenthin.maven.llamacpp.aiindex.document.AiMdHeader; import net.ladenthin.maven.llamacpp.aiindex.prompt.AiPreparedPrompt; import net.ladenthin.maven.llamacpp.aiindex.prompt.AiPromptPreparationSupport; import net.ladenthin.maven.llamacpp.aiindex.provider.AiGenerationProvider; +import net.ladenthin.maven.llamacpp.aiindex.support.AiDeterministicSummary; import net.ladenthin.maven.llamacpp.aiindex.support.AiGenerationTimeEstimator; +import net.ladenthin.maven.llamacpp.aiindex.support.AiSourceChunker; import net.ladenthin.maven.llamacpp.aiindex.support.Java8CompatibilityHelper; import org.apache.maven.plugin.logging.Log; @@ -111,6 +114,18 @@ public class AiFieldGenerationSupport { /** Nanoseconds per second, used to convert the measured generation duration to whole seconds. */ private static final long NANOS_PER_SECOND = 1_000_000_000L; + /** Leading/trailing lines shown in a {@link AiOversizeStrategy#DETERMINISTIC} body. */ + private static final int DETERMINISTIC_SAMPLE_LINES = 20; + + /** Upper bound on the character overlap between map-reduce chunks. */ + private static final int MAP_REDUCE_OVERLAP_CHARS = 2000; + + /** Fraction (1/n) of the chunk size used as overlap when smaller than {@link #MAP_REDUCE_OVERLAP_CHARS}. */ + private static final int MAP_REDUCE_OVERLAP_DIVISOR = 20; + + /** Minimum per-chunk source budget, so a tiny window never yields zero-length chunks. */ + private static final int MIN_CHUNK_CHARS = 1000; + // Maven plugin Log — its default toString prints implementation details // (logger configuration, output stream state); excluded from the rendered output. @ToString.Exclude @@ -205,6 +220,25 @@ public AiGenerationResult processFieldGenerations( final AiPreparedPrompt preparedPrompt = promptPreparationSupport.preparePrompt(request, effectiveMaxInputChars); + // An oversized file (the source did not fit the window) is handled per the rule's onOversize + // strategy. DETERMINISTIC/MAP_REDUCE replace the single generation entirely; FAIL/SAMPLE fall + // through to the single-shot path below (the source was already trimmed to the window head). + if (preparedPrompt.trimmed()) { + final AiOversizeStrategy oversize = fieldGeneration.getOversizeStrategy(); + if (oversize == AiOversizeStrategy.DETERMINISTIC) { + body = AiDeterministicSummary.body( + sourceText, contextName(contextFile), DETERMINISTIC_SAMPLE_LINES); + log.info("Oversize " + contextType + " '" + contextFile + "' -> deterministic body (no model), " + + sourceText.length() + " chars"); + continue; + } + if (oversize == AiOversizeStrategy.MAP_REDUCE) { + body = mapReduceSummarize( + fieldGeneration, contextFile, sourceText, baseHeader, effectiveMaxInputChars); + continue; + } + } + if (preparedPrompt.trimmed() && generationConfig.isWarnOnTrim()) { log.warn("Trimmed AI input for " + contextType + TRIM_WARN_FIELD_LABEL + "body': " + contextFile + " (source chars " + preparedPrompt.originalSourceLength() @@ -250,6 +284,88 @@ public AiGenerationResult processFieldGenerations( return new AiGenerationResult(body); } + /** + * Summarizes an oversized source via map-reduce: split it into window-sized chunks at line boundaries + * (with overlap), summarize each chunk with the rule's prompt, then summarize the concatenated partial + * summaries into the final body. The {@code maxChunks} cap (when set) bounds the run time by sampling + * representative chunks across the file. + * + * @param fieldGeneration the routing rule (prompt key + {@code maxChunks}) + * @param contextFile the source file path + * @param sourceText the full source text + * @param baseHeader the header passed through to each request + * @param effectiveMaxInputChars the per-call window budget + * @return the combined summary body + * @throws IOException if the provider throws during generation + */ + private String mapReduceSummarize( + final AiFieldGenerationConfig fieldGeneration, + final Path contextFile, + final String sourceText, + final AiMdHeader baseHeader, + final int effectiveMaxInputChars) + throws IOException { + final int basePromptLength = + promptPreparationSupport.getBasePromptLength(fieldGeneration.getPromptKey(), contextFile); + final int sourceBudget = Math.max(MIN_CHUNK_CHARS, effectiveMaxInputChars - basePromptLength); + final int overlap = Math.min(MAP_REDUCE_OVERLAP_CHARS, sourceBudget / MAP_REDUCE_OVERLAP_DIVISOR); + final List chunks = + AiSourceChunker.chunk(sourceText, sourceBudget, overlap, fieldGeneration.getMaxChunks()); + log.info("Oversize file '" + contextFile + "' -> mapReduce: " + sourceText.length() + " chars in " + + chunks.size() + " chunk(s)"); + final StringBuilder partials = new StringBuilder(); + int index = 0; + for (final String chunk : chunks) { + index++; + final String partial = summarizeChunk( + fieldGeneration.getPromptKey(), contextFile, chunk, baseHeader, effectiveMaxInputChars); + partials.append("Chunk ") + .append(index) + .append(":\n") + .append(partial.trim()) + .append("\n\n"); + log.info(" mapReduce chunk " + index + "/" + chunks.size() + " summarized (" + chunk.length() + " chars)"); + } + // Reduce: summarize the partial summaries (trimmed to the window) into the final body. + return summarizeChunk( + fieldGeneration.getPromptKey(), contextFile, partials.toString(), baseHeader, effectiveMaxInputChars); + } + + /** + * Prepares the prompt for one chunk (trimming to the window) and returns the provider's summary. + * + * @param promptKey the prompt template key + * @param contextFile the source file path + * @param chunk the chunk (or concatenated partials) to summarize + * @param baseHeader the header passed through to the request + * @param effectiveMaxInputChars the per-call window budget + * @return the provider's summary for the chunk + * @throws IOException if the provider throws during generation + */ + private String summarizeChunk( + final String promptKey, + final Path contextFile, + final String chunk, + final AiMdHeader baseHeader, + final int effectiveMaxInputChars) + throws IOException { + final AiGenerationRequest request = new AiGenerationRequest(promptKey, contextFile, chunk, baseHeader); + final AiPreparedPrompt prepared = promptPreparationSupport.preparePrompt(request, effectiveMaxInputChars); + return generationProvider.generate( + new AiGenerationRequest(promptKey, contextFile, prepared.sourceText(), baseHeader)); + } + + /** + * Returns a short display name for a context file (its file name, or the full path as a fallback). + * + * @param contextFile the file path + * @return the display name + */ + private static String contextName(final Path contextFile) { + final Path fileName = contextFile.getFileName(); + return fileName != null ? fileName.toString() : contextFile.toString(); + } + /** * Returns the effective {@code maxInputChars} for the given field generation entry. * diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiIndexPlan.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiIndexPlan.java index c0f56b2..1d1b7c0 100644 --- a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiIndexPlan.java +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiIndexPlan.java @@ -9,6 +9,7 @@ import java.util.List; import java.util.Map; import net.ladenthin.maven.llamacpp.aiindex.config.AiFieldGenerationConfig; +import net.ladenthin.maven.llamacpp.aiindex.config.AiOversizeStrategy; import net.ladenthin.maven.llamacpp.aiindex.support.AiGenerationTimeEstimator; /** @@ -247,6 +248,36 @@ public int windowExceededCount() { return total; } + /** + * Returns the number of over-window files whose rule's {@code onOversize} strategy is + * {@link AiOversizeStrategy#FAIL} — i.e. the ones that abort the build. Over-window files with a + * {@code sample}/{@code mapReduce}/{@code deterministic} strategy are handled at run time and do not + * count here. + * + * @return the count of over-window files that fail the build + */ + public int windowFailCount() { + int total = 0; + for (final List entries : routesByModel.values()) { + for (final Entry entry : entries) { + if (entry.exceedsWindow() && strategyOf(entry) == AiOversizeStrategy.FAIL) { + total++; + } + } + } + return total; + } + + /** + * Returns the oversize strategy configured on an entry's rule. + * + * @param entry the plan entry + * @return the parsed oversize strategy + */ + private static AiOversizeStrategy strategyOf(final Entry entry) { + return entry.rule().getOversizeStrategy(); + } + /** * Returns the summed estimated seconds for one model's entries. * @@ -299,7 +330,9 @@ public String renderMarkdown(final Path baseDir) { .append(unmatched.size()) .append(" unmatched, ") .append(windowExceededCount()) - .append(" over window _(time is a rough estimate)_\n"); + .append(" over window (") + .append(windowFailCount()) + .append(" will fail) _(time is a rough estimate)_\n"); for (final Map.Entry> group : routesByModel.entrySet()) { final List entries = group.getValue(); sb.append("\n### Model `") @@ -360,6 +393,7 @@ private void appendWindowExceededSection(final StringBuilder sb, final Path base for (final Map.Entry> group : routesByModel.entrySet()) { for (final Entry entry : group.getValue()) { if (entry.exceedsWindow()) { + final AiOversizeStrategy strategy = strategyOf(entry); sb.append("- ") .append(relativize(baseDir, entry.file())) .append(" -> model `") @@ -368,13 +402,18 @@ private void appendWindowExceededSection(final StringBuilder sb, final Path base .append(entry.sourceChars()) .append(" chars > window ") .append(entry.availableSourceChars()) - .append(" chars)\n"); + .append(" chars) onOversize=") + .append(strategy.configValue()) + .append(strategy == AiOversizeStrategy.FAIL ? " -> FAILS BUILD" : " -> handled") + .append('\n'); } } } - sb.append("\nThis fails the build. Add a rule with a size that routes") - .append(" these files to a model with a large enough context window (config only; the build") - .append(" never picks a model for you).\n"); + if (windowFailCount() > 0) { + sb.append("\nThe onOversize=fail entries fail the build. Either route them to a model with a large") + .append(" enough context window, or set onOversize on the rule (sample/mapReduce/deterministic)") + .append(" to handle oversized files (config only; the build never picks a model for you).\n"); + } } /** @@ -389,7 +428,8 @@ private String windowCell(final Entry entry) { return "-"; } if (entry.exceedsWindow()) { - return "(!) " + entry.sourceChars() + ">" + entry.availableSourceChars(); + return "(!) " + entry.sourceChars() + ">" + entry.availableSourceChars() + " " + + strategyOf(entry).configValue(); } return "ok"; } diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/SourceFileIndexer.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/SourceFileIndexer.java index 8067f52..0b57b82 100644 --- a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/SourceFileIndexer.java +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/SourceFileIndexer.java @@ -19,6 +19,7 @@ import net.ladenthin.maven.llamacpp.aiindex.config.AiFileContext; import net.ladenthin.maven.llamacpp.aiindex.config.AiGenerationConfig; import net.ladenthin.maven.llamacpp.aiindex.config.AiModelDefinitionSupport; +import net.ladenthin.maven.llamacpp.aiindex.config.AiOversizeStrategy; import net.ladenthin.maven.llamacpp.aiindex.document.AiGenerationResult; import net.ladenthin.maven.llamacpp.aiindex.document.AiMdDocument; import net.ladenthin.maven.llamacpp.aiindex.document.AiMdDocumentCodec; @@ -222,27 +223,59 @@ public AiIndexPlan classify( plan.addUnmatched(file); } else if (rule.isSkip()) { plan.addSkipped(file); + } else if (modelDefinitionSupport != null && promptPreparationSupport != null) { + final AiGenerationConfig modelConfig = modelDefinitionSupport.getConfig(rule.getAiDefinitionKey()); + final int basePromptLength = promptPreparationSupport.getBasePromptLength(rule.getPromptKey(), file); + final long availableSourceChars = + AiInputWindowCalculator.availableSourceChars(modelConfig, basePromptLength); + final long sourceChars = compatibilityHelper.readString(file).length(); + final long routeSeconds = estimateRouteSeconds(rule, fileSizeBytes, sourceChars, availableSourceChars); + plan.addRoute(rule.getAiDefinitionKey(), file, rule, routeSeconds, sourceChars, availableSourceChars); } else { final long estimatedSeconds = timeEstimator.estimateSeconds((int) Math.min(fileSizeBytes, Integer.MAX_VALUE)); - if (modelDefinitionSupport != null && promptPreparationSupport != null) { - final AiGenerationConfig modelConfig = modelDefinitionSupport.getConfig(rule.getAiDefinitionKey()); - final int basePromptLength = - promptPreparationSupport.getBasePromptLength(rule.getPromptKey(), file); - final long availableSourceChars = - AiInputWindowCalculator.availableSourceChars(modelConfig, basePromptLength); - final long sourceChars = - compatibilityHelper.readString(file).length(); - plan.addRoute( - rule.getAiDefinitionKey(), file, rule, estimatedSeconds, sourceChars, availableSourceChars); - } else { - plan.addRoute(rule.getAiDefinitionKey(), file, rule, estimatedSeconds); - } + plan.addRoute(rule.getAiDefinitionKey(), file, rule, estimatedSeconds); } } return plan; } + /** + * Estimates the generation time for one routed file, accounting for the rule's oversize strategy so + * the plan ETA stays realistic for huge files (the per-call cost is bounded by the model window, not + * the raw file size). A file that fits the window uses the normal size-based estimate; an over-window + * file is estimated per strategy: {@code deterministic} is free, {@code sample}/{@code fail} cost one + * window pass, and {@code mapReduce} costs (chunks + reduce) window passes (capped by {@code maxChunks}). + * + * @param rule the routing rule + * @param fileSizeBytes the file size in bytes + * @param sourceChars the source length in characters + * @param availableSourceChars the per-call source-character budget of the routed model+prompt + * @return the estimated seconds + */ + private long estimateRouteSeconds( + final AiFieldGenerationConfig rule, + final long fileSizeBytes, + final long sourceChars, + final long availableSourceChars) { + if (sourceChars <= availableSourceChars) { + return timeEstimator.estimateSeconds((int) Math.min(fileSizeBytes, Integer.MAX_VALUE)); + } + final long windowSeconds = + timeEstimator.estimateSeconds((int) Math.min(availableSourceChars, Integer.MAX_VALUE)); + final AiOversizeStrategy strategy = rule.getOversizeStrategy(); + if (strategy == AiOversizeStrategy.DETERMINISTIC) { + return 0L; + } + if (strategy == AiOversizeStrategy.MAP_REDUCE) { + final long denom = Math.max(1L, availableSourceChars); + final long naturalChunks = Math.max(1L, (sourceChars + denom - 1) / denom); + final long chunks = rule.getMaxChunks() > 0 ? Math.min(rule.getMaxChunks(), naturalChunks) : naturalChunks; + return (chunks + 1) * windowSeconds; + } + return windowSeconds; + } + /** * Writes the {@code .ai.md} for one source file using the given rule and a provider-bearing * support. Honours the incremental-skip rule: an up-to-date target (matching source checksum, no diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/mojo/GenerateMojo.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/mojo/GenerateMojo.java index e00bbab..0f07522 100644 --- a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/mojo/GenerateMojo.java +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/mojo/GenerateMojo.java @@ -196,19 +196,18 @@ public void execute() throws MojoExecutionException { + "add a rule or a matching rule (see the plan above)."); } - // 3b. A file larger than its routed model's window would lose content if trimmed. This is - // ALWAYS a hard failure (planOnly and real run alike). The fix is user configuration, - // never an automatic model choice: the user must add a routing rule ( - // with a size ) that sends oversized files to a model with a large enough - // context window. The plugin deliberately does not pick a model for you. - final int overWindowCount = plan.windowExceededCount(); - if (overWindowCount > 0) { - throw new MojoExecutionException(overWindowCount - + " source file(s) exceed their routed model's context window (see the 'Over window' " - + "section in the plan above). Add a rule with a size " - + "that routes these files to a model with a large enough context window (see the " - + "big-window fallback example in the POM). The build does not pick a model for you; " - + "this is configuration only."); + // 3b. A file larger than its routed model's window would lose content if trimmed. By default + // (onOversize=fail) this is a hard failure: the fix is user configuration, never an + // automatic model choice — route oversized files to a larger-context model, or set the + // rule's onOversize (sample/mapReduce/deterministic) to handle them at run time. Only the + // fail entries abort here; the handled ones are processed during generation. + final int overWindowFailCount = plan.windowFailCount(); + if (overWindowFailCount > 0) { + throw new MojoExecutionException(overWindowFailCount + + " source file(s) exceed their routed model's context window with onOversize=fail " + + "(see the 'Over window' section in the plan above). Route them to a model with a " + + "large enough context window, or set onOversize=sample|mapReduce|deterministic on " + + "the rule. The build does not pick a model for you; this is configuration only."); } if (planOnly) { diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/support/AiDeterministicSummary.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/support/AiDeterministicSummary.java new file mode 100644 index 0000000..aeb4897 --- /dev/null +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/support/AiDeterministicSummary.java @@ -0,0 +1,94 @@ +// @formatter:off +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: Apache-2.0 +// @formatter:on +package net.ladenthin.maven.llamacpp.aiindex.support; + +/** + * Builds a model-free, deterministic Markdown body for files too large (or too uninteresting) to feed to + * an AI model — e.g. huge repetitive data files. The body states size and line count and shows a head + * (and, when the file is long enough, a tail) sample, so the index still carries a useful, instant, + * zero-cost description. Pure; no I/O. + */ +public final class AiDeterministicSummary { + + /** Newline used to split and count lines. */ + private static final char NEWLINE = '\n'; + + /** Utility class; not instantiable. */ + private AiDeterministicSummary() { + // no-op + } + + /** + * Builds the deterministic body. + * + * @param source the full source text + * @param contextName a human-readable name for the source (e.g. the file name) + * @param sampleLines number of leading (and, when the file is long, trailing) lines to show; values + * {@code <= 0} show no sample, only the metadata line + * @return the Markdown body + */ + public static String body(final String source, final String contextName, final int sampleLines) { + final int chars = source.length(); + final int lines = countLines(source); + final StringBuilder sb = new StringBuilder(256); + sb.append("> Large file summarized deterministically (no AI): `") + .append(contextName) + .append("` - ") + .append(chars) + .append(" chars, ") + .append(lines) + .append(" lines.\n"); + if (sampleLines <= 0 || lines == 0) { + return sb.toString(); + } + final String[] all = source.split("\n", -1); + if (lines <= sampleLines * 2) { + appendFenced(sb, "Content", all, 0, all.length); + } else { + appendFenced(sb, "First " + sampleLines + " lines", all, 0, sampleLines); + appendFenced(sb, "Last " + sampleLines + " lines", all, all.length - sampleLines, all.length); + } + return sb.toString(); + } + + /** + * Appends a labelled fenced block with {@code lines[from..toExclusive)}. + * + * @param sb the buffer + * @param label the section label + * @param lines the split lines + * @param from start index (inclusive) + * @param toExclusive end index (exclusive) + */ + private static void appendFenced( + final StringBuilder sb, final String label, final String[] lines, final int from, final int toExclusive) { + sb.append('\n').append(label).append(":\n```\n"); + for (int i = from; i < toExclusive; i++) { + sb.append(lines[i]).append('\n'); + } + sb.append("```\n"); + } + + /** + * Counts lines: zero for empty input, otherwise the number of newlines plus one when the text does not + * end with a newline. + * + * @param source the text + * @return the line count + */ + private static int countLines(final String source) { + if (source.isEmpty()) { + return 0; + } + int newlines = 0; + for (int i = 0; i < source.length(); i++) { + if (source.charAt(i) == NEWLINE) { + newlines++; + } + } + return source.charAt(source.length() - 1) == NEWLINE ? newlines : newlines + 1; + } +} diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/support/AiSourceChunker.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/support/AiSourceChunker.java new file mode 100644 index 0000000..c5884b5 --- /dev/null +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/support/AiSourceChunker.java @@ -0,0 +1,98 @@ +// @formatter:off +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: Apache-2.0 +// @formatter:on +package net.ladenthin.maven.llamacpp.aiindex.support; + +import java.util.ArrayList; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Set; + +/** + * Splits a source string into chunks for map-reduce summarization of very large files. + * + *

Each chunk is at most {@code maxChars} long and ends at a line boundary (the last + * {@code \n} within the window) so records/lines are never torn in the middle; if a single line is longer + * than {@code maxChars} the chunk is hard-cut. Consecutive chunks may {@code overlap} by a number of + * characters so context is not lost at the seam. When the file would produce more than {@code maxChunks} + * chunks (and {@code maxChunks > 0}), a representative subset is selected — always including the first and + * last chunk and evenly spacing the rest — so the summary samples the whole file, not just its + * head. Pure; no I/O.

+ */ +public final class AiSourceChunker { + + /** Utility class; not instantiable. */ + private AiSourceChunker() { + // no-op + } + + /** + * Chunks {@code source} per the class contract. + * + * @param source the full source text + * @param maxChars maximum characters per chunk (must be {@code >= 1}) + * @param overlapChars characters of overlap between consecutive chunks (clamped to {@code [0, maxChars - 1]}) + * @param maxChunks maximum number of chunks to return; {@code <= 0} means unbounded (return all) + * @return the chunks in document order (a representative subset when capped); empty if {@code source} is empty + * @throws IllegalArgumentException if {@code maxChars < 1} + */ + public static List chunk( + final String source, final int maxChars, final int overlapChars, final int maxChunks) { + if (maxChars < 1) { + throw new IllegalArgumentException("maxChars must be >= 1 but was " + maxChars); + } + final int length = source.length(); + final List chunks = new ArrayList<>(length / maxChars + 1); + if (length == 0) { + return chunks; + } + final int overlap = Math.max(0, Math.min(overlapChars, maxChars - 1)); + + int pos = 0; + while (pos < length) { + int end = Math.min(pos + maxChars, length); + if (end < length) { + final int lastNewline = source.lastIndexOf('\n', end - 1); + if (lastNewline > pos) { + end = lastNewline + 1; + } + } + chunks.add(source.substring(pos, end)); + if (end >= length) { + break; + } + pos = Math.max(pos + 1, end - overlap); + } + return select(chunks, maxChunks); + } + + /** + * Returns all chunks, or a representative subset of exactly the distinct indices spanning first..last + * when there are more than {@code maxChunks} and {@code maxChunks > 0}. + * + * @param chunks the natural chunk list + * @param maxChunks the cap ({@code <= 0} = unbounded) + * @return the (possibly reduced) chunk list + */ + private static List select(final List chunks, final int maxChunks) { + final int total = chunks.size(); + if (maxChunks <= 0 || total <= maxChunks) { + return chunks; + } + final Set indices = new LinkedHashSet<>(); + if (maxChunks == 1) { + indices.add(0); + } else { + for (int i = 0; i < maxChunks; i++) { + indices.add((int) Math.round((double) i * (total - 1) / (maxChunks - 1))); + } + } + final List selected = new ArrayList<>(indices.size()); + for (final int index : indices) { + selected.add(chunks.get(index)); + } + return selected; + } +} diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfigTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfigTest.java index 9282747..bceb28e 100644 --- a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfigTest.java +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfigTest.java @@ -68,4 +68,28 @@ public void skip_defaultsFalseAndTogglesTrue() { c.setSkip(true); assertThat(c.isSkip(), is(true)); } + + @Test + public void onOversize_defaultsNullAndRoundTrips() { + final AiFieldGenerationConfig c = new AiFieldGenerationConfig(); + assertThat(c.getOnOversize(), is(nullValue())); + c.setOnOversize("mapReduce"); + assertThat(c.getOnOversize(), is(equalTo("mapReduce"))); + } + + @Test + public void oversizeStrategy_defaultsFailAndReflectsToken() { + final AiFieldGenerationConfig c = new AiFieldGenerationConfig(); + assertThat(c.getOversizeStrategy(), is(AiOversizeStrategy.FAIL)); + c.setOnOversize("deterministic"); + assertThat(c.getOversizeStrategy(), is(AiOversizeStrategy.DETERMINISTIC)); + } + + @Test + public void maxChunks_defaultsZeroAndRoundTrips() { + final AiFieldGenerationConfig c = new AiFieldGenerationConfig(); + assertThat(c.getMaxChunks(), is(equalTo(0))); + c.setMaxChunks(7); + assertThat(c.getMaxChunks(), is(equalTo(7))); + } } diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationSelectorTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationSelectorTest.java index fa3d415..dcf1e4b 100644 --- a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationSelectorTest.java +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationSelectorTest.java @@ -161,6 +161,20 @@ public void validate_fallbackThatIsAlsoSkip_throws() { assertThrows(IllegalArgumentException.class, () -> selector.validate(Collections.singletonList(fb))); } + @Test + public void validate_invalidOnOversize_throws() { + final AiFieldGenerationConfig r = route("java", extCond(".java"), 0); + r.setOnOversize("nonsense"); + assertThrows(IllegalArgumentException.class, () -> selector.validate(Collections.singletonList(r))); + } + + @Test + public void validate_validOnOversize_passes() { + final AiFieldGenerationConfig r = route("java", extCond(".java"), 0); + r.setOnOversize("mapReduce"); + assertDoesNotThrow(() -> selector.validate(Collections.singletonList(r))); + } + @Test public void validate_routeMissingPromptKey_throws() { final AiFieldGenerationConfig bad = new AiFieldGenerationConfig(); diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiOversizeStrategyTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiOversizeStrategyTest.java new file mode 100644 index 0000000..d38f946 --- /dev/null +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiOversizeStrategyTest.java @@ -0,0 +1,55 @@ +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: Apache-2.0 +package net.ladenthin.maven.llamacpp.aiindex.config; + +import static org.hamcrest.CoreMatchers.containsString; +import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.MatcherAssert.assertThat; +import static org.junit.jupiter.api.Assertions.assertThrows; + +import org.junit.jupiter.api.Test; + +public class AiOversizeStrategyTest { + + @Test + public void defaultIsFail() { + assertThat(AiOversizeStrategy.DEFAULT, is(AiOversizeStrategy.FAIL)); + } + + @Test + public void configValuesAreTheExpectedTokens() { + assertThat(AiOversizeStrategy.FAIL.configValue(), is("fail")); + assertThat(AiOversizeStrategy.SAMPLE.configValue(), is("sample")); + assertThat(AiOversizeStrategy.MAP_REDUCE.configValue(), is("mapReduce")); + assertThat(AiOversizeStrategy.DETERMINISTIC.configValue(), is("deterministic")); + } + + @Test + public void fromConfig_nullOrBlank_isDefault() { + assertThat(AiOversizeStrategy.fromConfig(null), is(AiOversizeStrategy.FAIL)); + assertThat(AiOversizeStrategy.fromConfig(""), is(AiOversizeStrategy.FAIL)); + assertThat(AiOversizeStrategy.fromConfig(" "), is(AiOversizeStrategy.FAIL)); + } + + @Test + public void fromConfig_eachToken_parses() { + assertThat(AiOversizeStrategy.fromConfig("fail"), is(AiOversizeStrategy.FAIL)); + assertThat(AiOversizeStrategy.fromConfig("sample"), is(AiOversizeStrategy.SAMPLE)); + assertThat(AiOversizeStrategy.fromConfig("mapReduce"), is(AiOversizeStrategy.MAP_REDUCE)); + assertThat(AiOversizeStrategy.fromConfig("deterministic"), is(AiOversizeStrategy.DETERMINISTIC)); + } + + @Test + public void fromConfig_isCaseInsensitiveAndTrimmed() { + assertThat(AiOversizeStrategy.fromConfig(" MAPREDUCE "), is(AiOversizeStrategy.MAP_REDUCE)); + assertThat(AiOversizeStrategy.fromConfig("Sample"), is(AiOversizeStrategy.SAMPLE)); + } + + @Test + public void fromConfig_unknown_throwsWithValue() { + final IllegalArgumentException ex = + assertThrows(IllegalArgumentException.class, () -> AiOversizeStrategy.fromConfig("nonsense")); + assertThat(ex.getMessage(), containsString("nonsense")); + } +} diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupportTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupportTest.java index 47ac267..767e366 100644 --- a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupportTest.java +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupportTest.java @@ -11,9 +11,14 @@ import java.nio.file.Files; import java.nio.file.Path; import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collections; import java.util.List; import java.util.concurrent.atomic.AtomicInteger; import net.ladenthin.maven.llamacpp.aiindex.CommonTestFixtures; +import net.ladenthin.maven.llamacpp.aiindex.config.AiFieldGenerationConfig; +import net.ladenthin.maven.llamacpp.aiindex.config.AiModelDefinition; +import net.ladenthin.maven.llamacpp.aiindex.config.AiModelDefinitionSupport; import net.ladenthin.maven.llamacpp.aiindex.document.AiGenerationRequest; import net.ladenthin.maven.llamacpp.aiindex.document.AiGenerationResult; import net.ladenthin.maven.llamacpp.aiindex.document.AiMdHeader; @@ -288,4 +293,119 @@ public String generate(final AiGenerationRequest request) { } } // + + // + + /** A model whose tiny static input window forces every non-trivial source to be over-window. */ + private static AiModelDefinitionSupport tinyWindowModel(final String key) { + final AiModelDefinition def = new AiModelDefinition(); + def.setKey(key); + def.setModelPath("unused.gguf"); + // contextSize 1 x charsPerToken 1 minus overhead -> computed maxInputChars clamps to 0, so any + // non-empty source is over-window and triggers the onOversize strategy. + def.setContextSize(1); + def.setCharsPerToken(1); + return new AiModelDefinitionSupport(Arrays.asList(def)); + } + + private AiFieldGenerationSupport supportWith(final AiGenerationProvider provider, final String modelKey) { + final AiPromptSupport promptSupport = new AiPromptSupport(CommonTestFixtures.createFilePromptDefinitions()); + return new AiFieldGenerationSupport( + capturingLog, provider, new AiPromptPreparationSupport(promptSupport), tinyWindowModel(modelKey)); + } + + private static AiFieldGenerationConfig oversizeRule( + final String modelKey, final String onOversize, final int maxChunks) { + final AiFieldGenerationConfig rule = new AiFieldGenerationConfig(); + rule.setPromptKey(CommonTestFixtures.PROMPT_KEY_FILE_BODY); + rule.setAiDefinitionKey(modelKey); + rule.setOnOversize(onOversize); + rule.setMaxChunks(maxChunks); + return rule; + } + + private static String largeSource(final int lines) { + final StringBuilder sb = new StringBuilder(); + for (int i = 0; i < lines; i++) { + sb.append("line ").append(i).append('\n'); + } + return sb.toString(); + } + + @Test + public void onOversize_deterministic_skipsModelAndEmitsMetadataBody() throws Exception { + final Path contextFile = Files.createTempFile("Huge", ".java"); + final AtomicInteger calls = new AtomicInteger(); + final AiGenerationProvider provider = request -> { + calls.incrementAndGet(); + return "MODEL OUTPUT"; + }; + final AiFieldGenerationSupport support = supportWith(provider, "m"); + + final AiGenerationResult result = support.processFieldGenerations( + Collections.singletonList(oversizeRule("m", "deterministic", 0)), + contextFile, + "file", + largeSource(500), + anyHeader()); + + assertThat(calls.get(), is(equalTo(0))); + assertThat(result.body(), containsString("deterministically (no AI)")); + } + + @Test + public void onOversize_mapReduce_callsModelPerChunkPlusReduce() throws Exception { + final Path contextFile = Files.createTempFile("Huge", ".java"); + final AtomicInteger calls = new AtomicInteger(); + final AiGenerationProvider provider = request -> { + calls.incrementAndGet(); + return "PARTIAL"; + }; + final AiFieldGenerationSupport support = supportWith(provider, "m"); + + // ~3600 chars > 1000-char chunk budget -> several chunks; maxChunks=2 caps to 2 mapped + 1 reduce. + final AiGenerationResult result = support.processFieldGenerations( + Collections.singletonList(oversizeRule("m", "mapReduce", 2)), + contextFile, + "file", + largeSource(600), + anyHeader()); + + assertThat(calls.get(), is(equalTo(3))); + assertThat(result.body(), is(equalTo("PARTIAL"))); + } + + @Test + public void onOversize_sample_singleCallOnTrimmedHead() throws Exception { + final Path contextFile = Files.createTempFile("Huge", ".java"); + final AtomicInteger calls = new AtomicInteger(); + final AiGenerationProvider provider = request -> { + calls.incrementAndGet(); + return "SAMPLED"; + }; + final AiFieldGenerationSupport support = supportWith(provider, "m"); + + final AiGenerationResult result = support.processFieldGenerations( + Collections.singletonList(oversizeRule("m", "sample", 0)), + contextFile, + "file", + largeSource(500), + anyHeader()); + + assertThat(calls.get(), is(equalTo(1))); + assertThat(result.body(), is(equalTo("SAMPLED"))); + } + + private static AiMdHeader anyHeader() { + return new AiMdHeader( + "Huge.java", + "1.0", + "ABCD1234", + "2026-01-01T00:00:00Z", + "2026-01-01T00:01:00Z", + "0.1.0", + "0.0.0", + AiMdHeaderCodec.NODE_TYPE_FILE); + } + // } diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/support/AiDeterministicSummaryTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/support/AiDeterministicSummaryTest.java new file mode 100644 index 0000000..34ecbfa --- /dev/null +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/support/AiDeterministicSummaryTest.java @@ -0,0 +1,73 @@ +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: Apache-2.0 +package net.ladenthin.maven.llamacpp.aiindex.support; + +import static org.hamcrest.CoreMatchers.containsString; +import static org.hamcrest.CoreMatchers.not; +import static org.hamcrest.MatcherAssert.assertThat; + +import org.junit.jupiter.api.Test; + +public class AiDeterministicSummaryTest { + + @Test + public void emptySource_metadataOnly_zeroCounts() { + final String body = AiDeterministicSummary.body("", "Empty.java", 5); + assertThat(body, containsString("`Empty.java`")); + assertThat(body, containsString("0 chars, 0 lines")); + assertThat(body, not(containsString("```"))); + } + + @Test + public void sampleLinesZero_metadataOnly() { + final String body = AiDeterministicSummary.body("a\nb\n", "F.java", 0); + assertThat(body, containsString("4 chars, 2 lines")); + assertThat(body, not(containsString("```"))); + } + + @Test + public void smallFile_showsAllContentOnce() { + // 3 lines <= 2*5 -> a single "Content" block with every line, no First/Last split. + final String body = AiDeterministicSummary.body("L1\nL2\nL3", "S.java", 5); + assertThat(body, containsString("8 chars, 3 lines")); + assertThat(body, containsString("Content:")); + assertThat(body, containsString("L1")); + assertThat(body, containsString("L3")); + assertThat(body, not(containsString("First"))); + } + + @Test + public void largeFile_showsHeadAndTailOnly() { + // 10 lines > 2*2 -> First 2 + Last 2; the middle (ln5) is omitted. + final String body = + AiDeterministicSummary.body("ln0\nln1\nln2\nln3\nln4\nln5\nln6\nln7\nln8\nln9", "Big.java", 2); + assertThat(body, containsString("First 2 lines:")); + assertThat(body, containsString("Last 2 lines:")); + assertThat(body, containsString("ln0")); + assertThat(body, containsString("ln1")); + assertThat(body, containsString("ln8")); + assertThat(body, containsString("ln9")); + assertThat(body, not(containsString("ln5"))); + } + + @Test + public void exactlyDoubleSampleLines_stillShowsContentNotHeadTail() { + // lines == 2*sampleLines is the boundary: <= keeps one "Content" block (not First/Last). + final String body = AiDeterministicSummary.body("a\nb\nc\nd", "B.java", 2); + assertThat(body, containsString("Content:")); + assertThat(body, not(containsString("First"))); + } + + @Test + public void lineCount_handlesTrailingNewlineAndNoTrailing() { + assertThat(AiDeterministicSummary.body("a", "f", 0), containsString("1 lines")); + assertThat(AiDeterministicSummary.body("a\n", "f", 0), containsString("1 lines")); + assertThat(AiDeterministicSummary.body("a\nb", "f", 0), containsString("2 lines")); + } + + @Test + public void charCountReflectsSourceLength() { + assertThat(AiDeterministicSummary.body("abcde", "f", 0), containsString("5 chars")); + } +} diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/support/AiSourceChunkerTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/support/AiSourceChunkerTest.java new file mode 100644 index 0000000..21d5895 --- /dev/null +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/support/AiSourceChunkerTest.java @@ -0,0 +1,78 @@ +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: Apache-2.0 +package net.ladenthin.maven.llamacpp.aiindex.support; + +import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.MatcherAssert.assertThat; +import static org.junit.jupiter.api.Assertions.assertThrows; + +import java.util.Arrays; +import java.util.List; +import org.junit.jupiter.api.Test; + +public class AiSourceChunkerTest { + + @Test + public void maxCharsBelowOne_throws() { + assertThrows(IllegalArgumentException.class, () -> AiSourceChunker.chunk("abc", 0, 0, 0)); + } + + @Test + public void emptySource_returnsEmptyList() { + assertThat(AiSourceChunker.chunk("", 10, 0, 0), is(Arrays.asList())); + } + + @Test + public void sourceFitsInOneChunk() { + assertThat(AiSourceChunker.chunk("abc", 10, 0, 0), is(Arrays.asList("abc"))); + } + + @Test + public void splitsAtLineBoundaries() { + // "L1\nL2\nL3\n" with maxChars 4 -> each chunk ends at a newline; rejoined == source. + final List chunks = AiSourceChunker.chunk("L1\nL2\nL3\n", 4, 0, 0); + assertThat(chunks, is(Arrays.asList("L1\n", "L2\n", "L3\n"))); + assertThat(String.join("", chunks), is("L1\nL2\nL3\n")); + } + + @Test + public void hardCutsWhenNoNewlineInWindow() { + // No newline anywhere -> hard cut at maxChars. + assertThat(AiSourceChunker.chunk("ABCDEFGH", 3, 0, 0), is(Arrays.asList("ABC", "DEF", "GH"))); + } + + @Test + public void overlapIsClampedAndProducesOverlappingChunks() { + // overlap 100 with maxChars 4 -> clamped to 3; consecutive chunks share 3 chars. + final List chunks = AiSourceChunker.chunk("ABCDEFGH", 4, 100, 0); + assertThat(chunks, is(Arrays.asList("ABCD", "BCDE", "CDEF", "DEFG", "EFGH"))); + } + + @Test + public void maxChunksZero_returnsAll() { + assertThat(AiSourceChunker.chunk("ABCDEFGHIJ", 2, 0, 0), is(Arrays.asList("AB", "CD", "EF", "GH", "IJ"))); + } + + @Test + public void maxChunksAtOrAboveCount_returnsAll() { + assertThat(AiSourceChunker.chunk("ABCDEFGHIJ", 2, 0, 10), is(Arrays.asList("AB", "CD", "EF", "GH", "IJ"))); + } + + @Test + public void maxChunksOne_keepsTheFirst() { + assertThat(AiSourceChunker.chunk("ABCDEFGHIJ", 2, 0, 1), is(Arrays.asList("AB"))); + } + + @Test + public void maxChunksSamplesFirstSpreadAndLast() { + // 5 chunks, cap 3 -> indices round(i*4/2) = 0,2,4 -> head/middle/tail. + assertThat(AiSourceChunker.chunk("ABCDEFGHIJ", 2, 0, 3), is(Arrays.asList("AB", "EF", "IJ"))); + } + + @Test + public void maxChunksUsesRoundingNotFloorForSpread() { + // 4 chunks, cap 3 -> indices round(i*3/2) = 0, round(1.5)=2, 3 -> AB, EF, GH (floor would pick CD). + assertThat(AiSourceChunker.chunk("ABCDEFGH", 2, 0, 3), is(Arrays.asList("AB", "EF", "GH"))); + } +} From 2145a4fdac6f6e0a9cfd057fd034ee78b3148871 Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Tue, 30 Jun 2026 22:43:47 +0200 Subject: [PATCH 02/18] fix(config): map-reduce huge files over a small fast window (~70 min/file) Prefill is O(n^2) in prompt length, so map-reducing a multi-MB file AT the big window (granite-bigwindow, ~1.16M-char chunks x maxChunks) was the slowest possible combination (~6500 min/file in the plan estimate). The fix is the opposite: chunk with a SMALL window so each prefill is cheap, and cap maxChunks. - New aiDefinition `granite-4.0-h-tiny-fastchunk` (same Granite model file as the big-window preset, contextSize 16384) for the map-reduce of very large files. - New size-tiered rules huge-java / huge-sql (priority 200, size > ~1 MB) route to fastchunk with onOversize=mapReduce, maxChunks=6. The big-window rules drop back to a single-pass 275 KB..1 MB band (priority 100); >1 MB no longer map-reduces on the 384K window. - README: route oversized files to a *small* fast model for mapReduce, with the O(n^2) rationale; example java-huge now uses gpt-oss-20B-c16k. Verified with a standalone planOnly repro on a 7.3 MB .java file: routes to granite-fastchunk (48,175-char chunks), onOversize=mapReduce -> handled, per-file estimate ~70 min (within the 60-90 min ceiling), BUILD SUCCESS. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd --- README.md | 13 ++++-- pom.xml | 132 ++++++++++++++++++++++++++++++++++++++++++------------ 2 files changed, 112 insertions(+), 33 deletions(-) diff --git a/README.md b/README.md index a4d1fd5..9aa15c1 100644 --- a/README.md +++ b/README.md @@ -218,7 +218,12 @@ The plugin is configured from three building blocks, declared on the plugin insi records are never torn mid-syntax), summarize each chunk, then combine the partial summaries in one final call. A `` cap bounds the time (a representative subset — always head + tail, evenly spaced — is summarized when the file would exceed the cap), so an arbitrarily large file - (e.g. 7 MB) stays within a chosen per-file budget. + (e.g. 7 MB) stays within a chosen per-file budget. **Route oversized files to a *small*, fast model + for this — not the big-window one.** Prefill is `O(n²)` in prompt length, so chunks should be small: + many cheap small-window passes are far faster than a few giant ones (one 384K-token pass alone can + dwarf a whole run). E.g. a 16K-window model with `maxChunks=6` is ~7 calls (~1 h order on a + reference CPU; less on GPU) and samples a representative slice — the right trade-off for repetitive + data. mapReduce *on* a big window is the slowest possible combination; avoid it. - **`deterministic`** — no model at all: emit a deterministic body (size, line count, head/tail sample). Instant; for pure data where no AI analysis is needed. @@ -388,9 +393,9 @@ and a fallback for everything else: 49152 - - java-hugefile-body-javagranite-4.0-h-tiny-bigwindow - mapReduce6 + + java-hugefile-body-javagpt-oss-20B-c16k + mapReduce6 .java 1048576 diff --git a/pom.xml b/pom.xml index 57d666c..94a33b1 100644 --- a/pom.xml +++ b/pom.xml @@ -1562,9 +1562,11 @@ SPDX-License-Identifier: Apache-2.0 window at run time); the cpt-3 check admits up to ~1.16 MB. reasoningEffort is empty: Granite has no gpt-oss harmony channels, so no reasoning kwarg is sent. The size-routing rules that target this key live in the ai-generate - below (ids big-window-java and big-window-sql). For >~1 MB files, split them or - add a still-larger preset (Granite allocates up to its 512K training limit - here, with further quality loss). --> + below (ids big-window-java and big-window-sql), for the + ~275 KB .. ~1 MB band (one pass). Files > ~1 MB are NOT summarized in one + giant pass here (O(n^2) prefill makes that far too slow); they route to the + small-window granite-4.0-h-tiny-fastchunk preset via the huge-java/huge-sql + rules and are map-reduced (cheap small chunks, maxChunks-bounded). --> granite-4.0-h-tiny-bigwindow X:/Modelle/granite-4.0-h-tiny-Q4_K_M.gguf @@ -1584,6 +1586,35 @@ SPDX-License-Identifier: Apache-2.0 + + + granite-4.0-h-tiny-fastchunk + X:/Modelle/granite-4.0-h-tiny-Q4_K_M.gguf + 16384 + ${ai.gpuLayers} + ${ai.mainGpu} + ${ai.devices} + 1536 + 0.2 + 8 + 3 + true + 1.0 + 0 + 0.05 + 1.0 + + + EXP-Qwen3-Coder-30B X:/Modelle/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf @@ -2026,35 +2057,80 @@ message: the path on the first line, then a blank line, then the summaries to sy .sql - + + + huge-java + file-body-java + granite-4.0-h-tiny-fastchunk + 200 + + mapReduce + 6 + + + + + .java + + + 1000000 + + + + + + + huge-sql + file-body-sql + granite-4.0-h-tiny-fastchunk + 200 + mapReduce + 6 + + + + + .sql + + + 1000000 + + + + + big-window-java file-body-java granite-4.0-h-tiny-bigwindow 100 - - mapReduce - 6 + @@ -2073,8 +2149,6 @@ message: the path on the first line, then a blank line, then the summaries to sy file-body-sql granite-4.0-h-tiny-bigwindow 100 - mapReduce - 6 From 16e0b7acf51ef62dfa39a7c76f71b475f3c861ef Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Wed, 1 Jul 2026 08:58:35 +0200 Subject: [PATCH 03/18] feat(config): default the oversize fallback to Granite 4.0 H-1B (1.5B, CPU-fast) Switches the fastchunk preset (the huge-java/huge-sql onOversize=mapReduce fallback) from the 7B h-tiny to the smallest Granite 4.0 hybrid, H-1B (~1.5B, Apache-2.0). Rationale: for the oversize map-reduce path, throughput matters more than raw quality, and the 1.5B roughly doubles CPU prefill throughput. Measured on this machine (CPU-only, 16 threads, Q4_K_M) over a 1 MB SQL file, maxChunks=6: - 3B h-micro : 16m39s (~90 tok/s) - 1.5B h-1b : 8m27s (~175 tok/s) <- new default GPU (RTX 3070) does the same run in ~40s. Per-file time is bounded by maxChunks (~8.5 min CPU regardless of file size); a 10 MB file summarizes in ~31s on GPU. Renamed the preset key granite-4.0-h-tiny-fastchunk -> granite-4.0-h-1b-fastchunk and updated both rule references; model at X:/Modelle/granite-4.0-h-1b-Q4_K_M.gguf loads on the existing llama 5.0.3 native (granitehybrid arch, no rebuild). Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd --- pom.xml | 38 +++++++++++++++++++++++--------------- 1 file changed, 23 insertions(+), 15 deletions(-) diff --git a/pom.xml b/pom.xml index 94a33b1..375d9c4 100644 --- a/pom.xml +++ b/pom.xml @@ -1565,7 +1565,7 @@ SPDX-License-Identifier: Apache-2.0 below (ids big-window-java and big-window-sql), for the ~275 KB .. ~1 MB band (one pass). Files > ~1 MB are NOT summarized in one giant pass here (O(n^2) prefill makes that far too slow); they route to the - small-window granite-4.0-h-tiny-fastchunk preset via the huge-java/huge-sql + small-window granite-4.0-h-1b-fastchunk preset via the huge-java/huge-sql rules and are map-reduced (cheap small chunks, maxChunks-bounded). --> granite-4.0-h-tiny-bigwindow @@ -1586,19 +1586,27 @@ SPDX-License-Identifier: Apache-2.0 - - granite-4.0-h-tiny-fastchunk - X:/Modelle/granite-4.0-h-tiny-Q4_K_M.gguf + granite-4.0-h-1b-fastchunk + X:/Modelle/granite-4.0-h-1b-Q4_K_M.gguf 16384 ${ai.gpuLayers} ${ai.mainGpu} @@ -2068,7 +2076,7 @@ message: the path on the first line, then a blank line, then the summaries to sy Prefill is O(n^2) in prompt length, so ONE big 384K pass over a multi-MB file is catastrophically slow, and map-reducing AT the big window (6 x 384K passes) is worse still. The fast path is the opposite: chunk with a SMALL window - (granite-4.0-h-tiny-fastchunk, 16K) so each prefill is cheap, and cap + (granite-4.0-h-1b-fastchunk, 16K) so each prefill is cheap, and cap so a 7 MB file stays a handful of calls (~1 h order on the reference CPU; less on GPU) - sampling a representative head/tail/spread, which is the right trade-off for repetitive data needing no precise whole-file read. @@ -2081,7 +2089,7 @@ message: the path on the first line, then a blank line, then the summaries to sy huge-java file-body-java - granite-4.0-h-tiny-fastchunk + granite-4.0-h-1b-fastchunk 200 java-hugefile-body-javagpt-oss-20B-c16k mapReduce6 + + \bboolean\b + (?m)^\s+(public|private|protected).*\( + .java 1048576 diff --git a/pom.xml b/pom.xml index 375d9c4..20f6d2f 100644 --- a/pom.xml +++ b/pom.xml @@ -740,6 +740,8 @@ SPDX-License-Identifier: Apache-2.0 net.ladenthin.maven.llamacpp.aiindex.provider.AiGenerationProviderFactory net.ladenthin.maven.llamacpp.aiindex.provider.MockAiGenerationProvider net.ladenthin.maven.llamacpp.aiindex.config.AiOversizeStrategy + net.ladenthin.maven.llamacpp.aiindex.config.AiFactCounter + net.ladenthin.maven.llamacpp.aiindex.config.AiFactExtractor net.ladenthin.maven.llamacpp.aiindex.support.AiChecksumSupport net.ladenthin.maven.llamacpp.aiindex.support.AiDeterministicSummary net.ladenthin.maven.llamacpp.aiindex.support.AiGenerationTimeEstimator @@ -2117,6 +2119,17 @@ message: the path on the first line, then a blank line, then the summaries to sy 200 mapReduce 6 + + + (?m)^\s*INSERT\b + (?m)^\s*CREATE TABLE\b + (?m)^\s*CREATE VIEW\b + diff --git a/spotbugs-exclude.xml b/spotbugs-exclude.xml index b5437b4..a76d8d6 100644 --- a/spotbugs-exclude.xml +++ b/spotbugs-exclude.xml @@ -360,6 +360,7 @@ SPDX-License-Identifier: Apache-2.0 + @@ -413,6 +414,7 @@ SPDX-License-Identifier: Apache-2.0 + @@ -421,6 +423,7 @@ SPDX-License-Identifier: Apache-2.0 + diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFactCounter.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFactCounter.java new file mode 100644 index 0000000..0741395 --- /dev/null +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFactCounter.java @@ -0,0 +1,79 @@ +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: Apache-2.0 +package net.ladenthin.maven.llamacpp.aiindex.config; + +import lombok.ToString; + +/** + * One deterministic "fact" counter for an oversize routing rule: a human-readable {@link #label} paired + * with a {@link #pattern} (a Java regular expression). The number of non-overlapping matches of + * {@code pattern} across the whole source is reported next to {@code label} in the generated + * body, giving exact counts the AI cannot reliably produce (it never sees the whole file when the source + * is chunked/sampled). + * + *

Language-agnostic by design: the meaning is entirely in the regex the operator writes, so the same + * mechanism counts {@code (?m)^\s*INSERT\b} rows in a SQL dump, {@code \bboolean\b} occurrences in Java, + * or {@code (?m)^\s*def\b} functions in Python. Multi-line matching is opt-in via the inline flag + * {@code (?m)} in the pattern; no flags are applied implicitly.

+ * + *

Note: this class must remain a mutable JavaBean with setters because the Maven + * plugin framework populates it via reflection from the {@code } configuration list.

+ * + *

{@code toString} is generated by Lombok for build-log diagnostics; {@code equals}/{@code hashCode} + * are intentionally NOT generated (instances are configuration data managed by position in a list, and + * the mutable JavaBean shape makes value semantics unsafe).

+ * + * @see AiFactExtractor + */ +@SuppressWarnings({"NullAway.Init", "initialization.fields.uninitialized"}) +@ToString +public class AiFactCounter { + + /** Creates a new {@link AiFactCounter}. */ + public AiFactCounter() { + // no-op + } + + /** Human-readable label shown next to the count (e.g. {@code "INSERT rows"}). */ + private String label; + + /** Java regular expression whose non-overlapping match count over the whole source is reported. */ + private String pattern; + + /** + * Returns the human-readable label. + * + * @return the label + */ + public String getLabel() { + return label; + } + + /** + * Sets the human-readable label. + * + * @param label the label + */ + public void setLabel(final String label) { + this.label = label; + } + + /** + * Returns the counting regular expression. + * + * @return the pattern + */ + public String getPattern() { + return pattern; + } + + /** + * Sets the counting regular expression. + * + * @param pattern the pattern + */ + public void setPattern(final String pattern) { + this.pattern = pattern; + } +} diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFactExtractor.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFactExtractor.java new file mode 100644 index 0000000..95f714e --- /dev/null +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFactExtractor.java @@ -0,0 +1,115 @@ +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: Apache-2.0 +package net.ladenthin.maven.llamacpp.aiindex.config; + +import java.util.List; +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import java.util.regex.PatternSyntaxException; +import org.jspecify.annotations.Nullable; + +/** + * Computes a deterministic, language-agnostic "facts" block from a list of {@link AiFactCounter}s. + * + *

Each counter contributes one {@code label: count} entry, where {@code count} is the number of + * non-overlapping matches of the counter's regex across the whole source. This gives exact, + * model-free counts (rows, tables, boolean fields, functions, …) that a chunked/sampled AI summary + * cannot reliably produce, and is prepended to the AI body on the oversize path. Pure; no I/O.

+ */ +public final class AiFactExtractor { + + /** Header prefixing the facts line, marking the counts as exact and whole-file (not AI-estimated). */ + public static final String FACTS_HEADER = "**Facts (exact, whole file):** "; + + /** Separator between {@code label} and its count. */ + public static final String LABEL_COUNT_SEPARATOR = ": "; + + /** Separator between consecutive {@code label: count} entries. */ + public static final String ENTRY_SEPARATOR = "; "; + + /** Trailing separator between the facts line and the AI body that follows it. */ + public static final String FACTS_SUFFIX = "\n\n"; + + /** Utility class; not instantiable. */ + private AiFactExtractor() { + // no-op + } + + /** + * Builds the facts block for the given counters over {@code source}, or an empty string when there are + * no counters. Counters with a {@code null} label or pattern are skipped (they are rejected up front by + * {@link #validate(List)}). + * + * @param counters the configured fact counters; {@code null} or empty yields an empty string + * @param source the full source text the regexes are counted over + * @return the facts block (ending in a blank line), or {@code ""} when there is nothing to report + */ + public static String factsBlock(final @Nullable List counters, final String source) { + if (counters == null || counters.isEmpty()) { + return ""; + } + final StringBuilder sb = new StringBuilder(64); + for (final AiFactCounter counter : counters) { + if (counter == null || counter.getLabel() == null || counter.getPattern() == null) { + continue; + } + if (sb.length() > 0) { + sb.append(ENTRY_SEPARATOR); + } + sb.append(counter.getLabel()) + .append(LABEL_COUNT_SEPARATOR) + .append(countMatches(counter.getPattern(), source)); + } + if (sb.length() == 0) { + return ""; + } + return FACTS_HEADER + sb + FACTS_SUFFIX; + } + + /** + * Validates that every counter has a non-null label and a compilable pattern, throwing + * {@link IllegalArgumentException} so the build fails fast on a misconfigured {@code } list. + * + * @param counters the configured fact counters; {@code null} or empty is valid (nothing to check) + * @throws IllegalArgumentException if a counter is missing its label/pattern or has an invalid regex + */ + public static void validate(final @Nullable List counters) { + if (counters == null) { + return; + } + for (final AiFactCounter counter : counters) { + if (counter == null) { + continue; + } + if (counter.getLabel() == null) { + throw new IllegalArgumentException("A fact counter must have a label: " + counter); + } + if (counter.getPattern() == null) { + throw new IllegalArgumentException("A fact counter must have a pattern: " + counter); + } + try { + Pattern.compile(counter.getPattern()); + } catch (final PatternSyntaxException e) { + throw new IllegalArgumentException( + "Invalid fact pattern for label '" + counter.getLabel() + "': " + e.getMessage(), e); + } + } + } + + /** + * Counts the non-overlapping matches of {@code pattern} in {@code source}. + * + * @param pattern the regular expression + * @param source the text to search + * @return the number of matches (zero-width matches, e.g. {@code (?m)^}, count once per position) + */ + private static int countMatches(final String pattern, final String source) { + final Matcher matcher = Pattern.compile(pattern).matcher(source); + int count = 0; + while (matcher.find()) { + count++; + } + return count; + } +} diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfig.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfig.java index b372408..27cead6 100644 --- a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfig.java +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfig.java @@ -3,6 +3,7 @@ // SPDX-License-Identifier: Apache-2.0 package net.ladenthin.maven.llamacpp.aiindex.config; +import java.util.List; import lombok.ToString; import org.jspecify.annotations.Nullable; @@ -90,6 +91,15 @@ public AiFieldGenerationConfig() { */ private int maxChunks; + /** + * Optional deterministic "fact" counters ({@code }). When set and the file takes an oversize + * path (sample/mapReduce/deterministic), each counter's {@code label: } is prepended to the generated body — exact, language-agnostic counts (e.g. SQL {@code INSERT} + * rows, Java {@code boolean} fields) the sampled AI summary cannot reliably produce. {@code null}/empty + * = no facts block. See {@link AiFactExtractor}. + */ + private @Nullable List facts; + /** * Returns the optional rule id (label), or {@code null} when not set. * @@ -261,4 +271,22 @@ public int getMaxChunks() { public void setMaxChunks(final int maxChunks) { this.maxChunks = maxChunks; } + + /** + * Returns the optional deterministic fact counters, or {@code null} when none are configured. + * + * @return the fact counters, or {@code null} + */ + public @Nullable List getFacts() { + return facts; + } + + /** + * Sets the deterministic fact counters. + * + * @param facts the fact counters + */ + public void setFacts(final @Nullable List facts) { + this.facts = facts; + } } diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationSelector.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationSelector.java index b8dec2b..059df68 100644 --- a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationSelector.java +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationSelector.java @@ -81,6 +81,8 @@ public void validate(final Iterable configs) { } // Fail fast on an unknown onOversize token (throws IllegalArgumentException naming the value). config.getOversizeStrategy(); + // Fail fast on a malformed counter (missing label/pattern or an invalid regex). + AiFactExtractor.validate(config.getFacts()); if (config.isFallback()) { fallbackCount++; if (config.isSkip()) { diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupport.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupport.java index 4178b27..0047856 100644 --- a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupport.java +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupport.java @@ -10,6 +10,7 @@ import java.util.List; import java.util.Map; import lombok.ToString; +import net.ladenthin.maven.llamacpp.aiindex.config.AiFactExtractor; import net.ladenthin.maven.llamacpp.aiindex.config.AiFieldGenerationConfig; import net.ladenthin.maven.llamacpp.aiindex.config.AiGenerationConfig; import net.ladenthin.maven.llamacpp.aiindex.config.AiModelDefinitionSupport; @@ -247,23 +248,31 @@ public AiGenerationResult processFieldGenerations( final AiPreparedPrompt preparedPrompt = promptPreparationSupport.preparePrompt(request, effectiveMaxInputChars); + // Deterministic, whole-file "facts" (exact counts the sampled AI body cannot produce) are + // prepended to the body on the oversize path; computed once here over the FULL source. + String factsPrefix = ""; + // An oversized file (the source did not fit the window) is handled per the rule's onOversize // strategy. DETERMINISTIC/MAP_REDUCE replace the single generation entirely; FAIL/SAMPLE fall // through to the single-shot path below (the source was already trimmed to the window head). if (preparedPrompt.trimmed()) { + final String factsBlock = AiFactExtractor.factsBlock(fieldGeneration.getFacts(), sourceText); final AiOversizeStrategy oversize = fieldGeneration.getOversizeStrategy(); if (oversize == AiOversizeStrategy.DETERMINISTIC) { - body = AiDeterministicSummary.body( - sourceText, contextName(contextFile), DETERMINISTIC_SAMPLE_LINES); + body = factsBlock + + AiDeterministicSummary.body( + sourceText, contextName(contextFile), DETERMINISTIC_SAMPLE_LINES); log.info("Oversize " + contextType + " '" + contextFile + "' -> deterministic body (no model), " + sourceText.length() + " chars"); continue; } if (oversize == AiOversizeStrategy.MAP_REDUCE) { - body = mapReduceSummarize( - fieldGeneration, contextFile, sourceText, baseHeader, effectiveMaxInputChars); + body = factsBlock + + mapReduceSummarize( + fieldGeneration, contextFile, sourceText, baseHeader, effectiveMaxInputChars); continue; } + factsPrefix = factsBlock; } if (preparedPrompt.trimmed() && generationConfig.isWarnOnTrim()) { @@ -293,8 +302,9 @@ public AiGenerationResult processFieldGenerations( + generationConfig.getTemperature() + ", maxInputChars=" + effectiveMaxInputChars); final long generationStartNanos = System.nanoTime(); - body = generationProvider.generate(generationRequest); + final String generated = generationProvider.generate(generationRequest); final long actualSeconds = (System.nanoTime() - generationStartNanos) / NANOS_PER_SECOND; + body = factsPrefix + generated; // Log the MEASURED wall-clock duration next to the estimate so real runs are directly // comparable across models/quants without a separate benchmark harness. @@ -302,7 +312,7 @@ public AiGenerationResult processFieldGenerations( + timeEstimator.formatDuration(actualSeconds) + GENERATED_LOG_ACTUAL_INFIX + timeEstimator.formatDuration(estimatedSeconds) + GENERATED_LOG_SUFFIX); - if (compatibilityHelper.isBlank(body)) { + if (compatibilityHelper.isBlank(generated)) { log.warn(EMPTY_OUTPUT_WARN_PREFIX + contextType + TRIM_WARN_FIELD_LABEL + fieldGeneration.getPromptKey() + "': " + contextFile); } diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFactCounterTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFactCounterTest.java new file mode 100644 index 0000000..93d4b3d --- /dev/null +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFactCounterTest.java @@ -0,0 +1,41 @@ +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: Apache-2.0 +package net.ladenthin.maven.llamacpp.aiindex.config; + +import static org.hamcrest.CoreMatchers.containsString; +import static org.hamcrest.CoreMatchers.equalTo; +import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.CoreMatchers.nullValue; +import static org.hamcrest.MatcherAssert.assertThat; + +import org.junit.jupiter.api.Test; + +public class AiFactCounterTest { + + @Test + public void label_defaultsNullAndRoundTrips() { + final AiFactCounter c = new AiFactCounter(); + assertThat(c.getLabel(), is(nullValue())); + c.setLabel("INSERT rows"); + assertThat(c.getLabel(), is(equalTo("INSERT rows"))); + } + + @Test + public void pattern_defaultsNullAndRoundTrips() { + final AiFactCounter c = new AiFactCounter(); + assertThat(c.getPattern(), is(nullValue())); + c.setPattern("(?m)^INSERT"); + assertThat(c.getPattern(), is(equalTo("(?m)^INSERT"))); + } + + @Test + public void toString_includesBothFields() { + final AiFactCounter c = new AiFactCounter(); + c.setLabel("tables"); + c.setPattern("(?m)^CREATE TABLE"); + final String text = c.toString(); + assertThat(text, containsString("tables")); + assertThat(text, containsString("(?m)^CREATE TABLE")); + } +} diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFactExtractorTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFactExtractorTest.java new file mode 100644 index 0000000..e2cf199 --- /dev/null +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFactExtractorTest.java @@ -0,0 +1,127 @@ +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: Apache-2.0 +package net.ladenthin.maven.llamacpp.aiindex.config; + +import static org.hamcrest.CoreMatchers.containsString; +import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.CoreMatchers.startsWith; +import static org.hamcrest.MatcherAssert.assertThat; +import static org.junit.jupiter.api.Assertions.assertDoesNotThrow; +import static org.junit.jupiter.api.Assertions.assertThrows; + +import java.util.Arrays; +import java.util.Collections; +import org.junit.jupiter.api.Test; + +public class AiFactExtractorTest { + + private static AiFactCounter counter(final String label, final String pattern) { + final AiFactCounter c = new AiFactCounter(); + c.setLabel(label); + c.setPattern(pattern); + return c; + } + + @Test + public void factsBlock_nullCounters_returnsEmpty() { + assertThat(AiFactExtractor.factsBlock(null, "anything"), is("")); + } + + @Test + public void factsBlock_emptyCounters_returnsEmpty() { + assertThat(AiFactExtractor.factsBlock(Collections.emptyList(), "anything"), is("")); + } + + @Test + public void factsBlock_countsOccurrencesAcrossTheWholeSource() { + // \bboolean\b matches each occurrence (2), regardless of line. + final String block = AiFactExtractor.factsBlock( + Collections.singletonList(counter("boolean fields", "\\bboolean\\b")), "boolean a; int i; boolean b;"); + assertThat(block, containsString("boolean fields: 2")); + } + + @Test + public void factsBlock_multilineAnchorCountsPerLine() { + // (?m)^INSERT counts lines starting with INSERT (2), not the CREATE line. + final String block = AiFactExtractor.factsBlock( + Collections.singletonList(counter("rows", "(?m)^INSERT")), "INSERT a\nINSERT b\nCREATE c\n"); + assertThat(block, containsString("rows: 2")); + } + + @Test + public void factsBlock_zeroMatches_reportsZero() { + final String block = AiFactExtractor.factsBlock( + Collections.singletonList(counter("views", "(?m)^CREATE VIEW")), "no views here"); + assertThat(block, containsString("views: 0")); + } + + @Test + public void factsBlock_multipleCounters_joinedWithSeparatorHeaderAndSuffix() { + final String block = AiFactExtractor.factsBlock( + Arrays.asList(counter("tables", "(?m)^CREATE TABLE"), counter("rows", "(?m)^INSERT")), + "CREATE TABLE t\nINSERT a\nINSERT b\n"); + assertThat(block, startsWith(AiFactExtractor.FACTS_HEADER)); + assertThat(block, containsString("tables" + AiFactExtractor.LABEL_COUNT_SEPARATOR + "1")); + assertThat(block, containsString(AiFactExtractor.ENTRY_SEPARATOR + "rows: 2")); + assertThat(block.endsWith(AiFactExtractor.FACTS_SUFFIX), is(true)); + } + + @Test + public void factsBlock_singleCounter_hasNoEntrySeparator() { + final String block = + AiFactExtractor.factsBlock(Collections.singletonList(counter("rows", "(?m)^INSERT")), "INSERT a\n"); + assertThat(block.contains(AiFactExtractor.ENTRY_SEPARATOR), is(false)); + } + + @Test + public void factsBlock_counterWithNullLabelOrPattern_isSkipped() { + // Both counters malformed -> nothing to report -> empty string. + assertThat(AiFactExtractor.factsBlock(Arrays.asList(counter(null, "x"), counter("y", null)), "xxxx"), is("")); + } + + @Test + public void factsBlock_mixOfValidAndSkipped_reportsOnlyValid() { + final String block = AiFactExtractor.factsBlock( + Arrays.asList(counter(null, "x"), counter("rows", "(?m)^INSERT")), "INSERT a\n"); + assertThat(block, startsWith(AiFactExtractor.FACTS_HEADER)); + assertThat(block, containsString("rows: 1")); + } + + @Test + public void validate_null_isAllowed() { + assertDoesNotThrow(() -> AiFactExtractor.validate(null)); + } + + @Test + public void validate_validCounters_doesNotThrow() { + assertDoesNotThrow(() -> AiFactExtractor.validate(Collections.singletonList(counter("rows", "(?m)^INSERT")))); + } + + @Test + public void validate_nullLabel_throws() { + assertThrows( + IllegalArgumentException.class, + () -> AiFactExtractor.validate(Collections.singletonList(counter(null, "x")))); + } + + @Test + public void validate_nullPattern_throws() { + assertThrows( + IllegalArgumentException.class, + () -> AiFactExtractor.validate(Collections.singletonList(counter("x", null)))); + } + + @Test + public void validate_invalidRegex_throwsNamingTheLabel() { + final IllegalArgumentException ex = assertThrows( + IllegalArgumentException.class, + () -> AiFactExtractor.validate(Collections.singletonList(counter("bad", "[")))); + assertThat(ex.getMessage(), containsString("bad")); + } + + @Test + public void validate_nullEntryInList_isSkipped() { + assertDoesNotThrow(() -> AiFactExtractor.validate(Collections.singletonList(null))); + } +} diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfigTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfigTest.java index bceb28e..ab8802d 100644 --- a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfigTest.java +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfigTest.java @@ -92,4 +92,14 @@ public void maxChunks_defaultsZeroAndRoundTrips() { c.setMaxChunks(7); assertThat(c.getMaxChunks(), is(equalTo(7))); } + + @Test + public void facts_defaultsNullAndRoundTrips() { + final AiFieldGenerationConfig c = new AiFieldGenerationConfig(); + assertThat(c.getFacts(), is(nullValue())); + final AiFactCounter counter = new AiFactCounter(); + c.setFacts(java.util.Collections.singletonList(counter)); + assertThat(c.getFacts(), is(notNullValue())); + assertThat(c.getFacts().size(), is(equalTo(1))); + } } diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationSelectorTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationSelectorTest.java index dcf1e4b..10f08c5 100644 --- a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationSelectorTest.java +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationSelectorTest.java @@ -209,5 +209,25 @@ public void validate_invalidConditionInRule_throws() { final AiFieldGenerationConfig bad = route("p", new AiCondition(), 0); assertThrows(IllegalArgumentException.class, () -> selector.validate(Collections.singletonList(bad))); } + + @Test + public void validate_invalidFactPattern_throws() { + final AiFieldGenerationConfig r = route("java", extCond(".java"), 0); + final AiFactCounter bad = new AiFactCounter(); + bad.setLabel("broken"); + bad.setPattern("["); + r.setFacts(Collections.singletonList(bad)); + assertThrows(IllegalArgumentException.class, () -> selector.validate(Collections.singletonList(r))); + } + + @Test + public void validate_validFactPattern_passes() { + final AiFieldGenerationConfig r = route("java", extCond(".java"), 0); + final AiFactCounter ok = new AiFactCounter(); + ok.setLabel("rows"); + ok.setPattern("(?m)^INSERT"); + r.setFacts(Collections.singletonList(ok)); + assertDoesNotThrow(() -> selector.validate(Collections.singletonList(r))); + } // } diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupportTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupportTest.java index 6ad8d35..28563fb 100644 --- a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupportTest.java +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupportTest.java @@ -6,6 +6,7 @@ import static org.hamcrest.CoreMatchers.containsString; import static org.hamcrest.CoreMatchers.equalTo; import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.CoreMatchers.startsWith; import static org.hamcrest.MatcherAssert.assertThat; import java.nio.file.Files; @@ -16,6 +17,8 @@ import java.util.List; import java.util.concurrent.atomic.AtomicInteger; import net.ladenthin.maven.llamacpp.aiindex.CommonTestFixtures; +import net.ladenthin.maven.llamacpp.aiindex.config.AiFactCounter; +import net.ladenthin.maven.llamacpp.aiindex.config.AiFactExtractor; import net.ladenthin.maven.llamacpp.aiindex.config.AiFieldGenerationConfig; import net.ladenthin.maven.llamacpp.aiindex.config.AiModelDefinition; import net.ladenthin.maven.llamacpp.aiindex.config.AiModelDefinitionSupport; @@ -381,6 +384,28 @@ public void onOversize_mapReduce_callsModelPerChunkPlusReduce() throws Exception assertThat(result.body(), is(equalTo("PARTIAL"))); } + @Test + public void onOversize_mapReduce_withFacts_prependsExactWholeFileCountsToBody() throws Exception { + final Path contextFile = Files.createTempFile("Huge", ".java"); + final AiGenerationProvider provider = request -> "PARTIAL"; + final AiFieldGenerationSupport support = supportWith(provider, "m"); + + final AiFieldGenerationConfig rule = oversizeRule("m", "mapReduce", 2); + final AiFactCounter counter = new AiFactCounter(); + counter.setLabel("lines"); + counter.setPattern("(?m)^line "); + rule.setFacts(Collections.singletonList(counter)); + + // largeSource(600) has exactly 600 lines beginning with "line "; the AI (mock) never sees them all, + // but the deterministic facts block counts every one over the FULL source and is prepended. + final AiGenerationResult result = support.processFieldGenerations( + Collections.singletonList(rule), contextFile, "file", largeSource(600), anyHeader()); + + assertThat(result.body(), startsWith(AiFactExtractor.FACTS_HEADER)); + assertThat(result.body(), containsString("lines: 600")); + assertThat(result.body(), containsString("PARTIAL")); + } + @Test public void onOversize_mapReduce_unbounded_reducesHierarchicallyInMultipleRounds() throws Exception { final Path contextFile = Files.createTempFile("Huge", ".java"); From edf98f26b54befde1ee3d8806548950d5b363bc1 Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Wed, 1 Jul 2026 11:06:27 +0200 Subject: [PATCH 06/18] feat(facts): apply to every file (not just oversize) + curated fact sets Follow-up to the feature so it serves downstream agents on ALL summaries: - Facts now apply to every file a rule matches, not only oversize ones. The facts block is computed once over the full source and prepended whether the body comes from a normal single-shot generation or an oversize strategy. Small files get exact structural counts too (the AI could count them, but the facts line is the authoritative, greppable number an agent can trust without re-reading the source). - Curated, robust fact sets added to the real self-test rules: java / huge-java : type declarations, public declarations, TODO/FIXME, @Override sql / huge-sql : INSERT rows, tables, views, indexes, functions/procedures Patterns are line-anchored / unambiguous (verified against real repo files: AiFieldGenerationSelector -> 1 type, 4 public; a sample SQL -> 2/1/1/1/1). New test facts_prependedOnNonOversizeFile covers the always-on path (small source that fits the window still gets the facts block). mvn verify green (362 tests, 1 skipped), SpotBugs clean; PIT unaffected (no gated-class logic change). Docs (CLAUDE.md, README) updated to "every file, not just oversize". Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd --- CLAUDE.md | 5 ++- README.md | 14 ++++--- pom.xml | 33 +++++++++++++-- .../config/AiFieldGenerationConfig.java | 11 ++--- .../indexer/AiFieldGenerationSupport.java | 13 +++--- .../indexer/AiFieldGenerationSupportTest.java | 42 ++++++++++++++++++- 6 files changed, 94 insertions(+), 24 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 7e091fe..ed0802e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -152,8 +152,9 @@ token-dense chunk can't overflow the model context), or `deterministic` (model-f So oversized files are either routed to a larger-context model or handled by a strategy; only `fail` entries abort. A rule may also carry an orthogonal, language-agnostic **``** list (`AiFactCounter` + `AiFactExtractor`): each `{label, pattern}` reports its regex match count over the **whole** source, -prepended to the body on the oversize path — exact counts (SQL `INSERT` rows, Java `\bboolean\b` fields, -…) the sampled AI prose cannot reliably produce. See `AiFieldGenerationSelector` (selection + validation), `AiCondition`/ +prepended to the body of **every** file the rule matches (oversize or not) — exact structural counts +(SQL `INSERT` rows / tables / views, Java type declarations / `\bboolean\b` fields, …) that give +downstream agents authoritative numbers the (possibly sampled) AI prose cannot reliably produce. See `AiFieldGenerationSelector` (selection + validation), `AiCondition`/ `AiConditionEvaluator` (the tree), and `AiIndexPlan` (the rendered plan). This is how one run can index different file kinds/sizes with different models *and* prompts. diff --git a/README.md b/README.md index d2cd3bf..f3e4bab 100644 --- a/README.md +++ b/README.md @@ -230,12 +230,14 @@ The plugin is configured from three building blocks, declared on the plugin insi Quality from a real model is best within Granite's validated 128K (~500 KB) and degrades gradually beyond it — a whole-file (or chunked) summary still beats a trimmed one. - **Exact counts with `` (optional, any strategy).** A sampled/chunked AI summary can't reliably - *count* things (no single call sees the whole file — it will guess "25 rows"). Add a `` list to - the rule and each `{label, pattern}` reports its regex match count over the **whole** source, prepended - to the body as an exact facts line. It's fully generic — the meaning is in the regex, so the same - mechanism counts SQL `INSERT` rows or Java `\bboolean\b` fields; multi-line matching is opt-in via the - inline `(?m)` flag. Example output: `**Facts (exact, whole file):** INSERT rows: 36738; tables: 122; views: 4`. + **Exact counts with `` (optional, every file — not just oversize).** A sampled/chunked AI + summary can't reliably *count* things (no single call sees the whole file — it will guess "25 rows"). + Add a `` list to a rule and each `{label, pattern}` reports its regex match count over the + **whole** source, prepended to the body of every file the rule matches as an exact facts line — so + downstream agents get authoritative structural counts in every summary. It's fully generic — the + meaning is in the regex, so the same mechanism counts SQL `INSERT` rows or Java `\bboolean\b` fields; + multi-line matching is opt-in via the inline `(?m)` flag. Keep patterns robust (a fact that miscounts + is worse than none). Example: `**Facts (exact, whole file):** INSERT rows: 36738; tables: 122; views: 4`. ```xml diff --git a/pom.xml b/pom.xml index 20f6d2f..60e281f 100644 --- a/pom.xml +++ b/pom.xml @@ -2099,6 +2099,13 @@ message: the path on the first line, then a blank line, then the summaries to sy data: sample (head only, one call) or deterministic (no model, instant). --> mapReduce 6 + + + (?m)^\s*(?:public\s+|final\s+|abstract\s+|sealed\s+|non-sealed\s+|static\s+)*(?:class|interface|enum|record)\s+\w + \bpublic\b + \b(?:TODO|FIXME)\b + @Override\b + @@ -2126,9 +2133,11 @@ message: the path on the first line, then a blank line, then the summaries to sy so the same mechanism counts INSERT rows here or `\bboolean\b` fields in Java. Multi-line matching is opt-in via the inline (?m) flag. --> - (?m)^\s*INSERT\b - (?m)^\s*CREATE TABLE\b - (?m)^\s*CREATE VIEW\b + (?im)^\s*INSERT\s+INTO\b + (?im)^\s*CREATE\s+TABLE\b + (?im)^\s*CREATE\s+(?:OR\s+REPLACE\s+)?VIEW\b + (?im)^\s*CREATE\s+(?:UNIQUE\s+)?INDEX\b + (?im)^\s*CREATE\s+(?:OR\s+REPLACE\s+)?(?:FUNCTION|PROCEDURE)\b @@ -2187,6 +2196,17 @@ message: the path on the first line, then a blank line, then the summaries to sy java file-body-java ${ai.model} + + + (?m)^\s*(?:public\s+|final\s+|abstract\s+|sealed\s+|non-sealed\s+|static\s+)*(?:class|interface|enum|record)\s+\w + \bpublic\b + \b(?:TODO|FIXME)\b + @Override\b + .java @@ -2195,6 +2215,13 @@ message: the path on the first line, then a blank line, then the summaries to sy sql file-body-sql ${ai.model} + + (?im)^\s*INSERT\s+INTO\b + (?im)^\s*CREATE\s+TABLE\b + (?im)^\s*CREATE\s+(?:OR\s+REPLACE\s+)?VIEW\b + (?im)^\s*CREATE\s+(?:UNIQUE\s+)?INDEX\b + (?im)^\s*CREATE\s+(?:OR\s+REPLACE\s+)?(?:FUNCTION|PROCEDURE)\b + .sql diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfig.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfig.java index 27cead6..006d69e 100644 --- a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfig.java +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFieldGenerationConfig.java @@ -92,11 +92,12 @@ public AiFieldGenerationConfig() { private int maxChunks; /** - * Optional deterministic "fact" counters ({@code }). When set and the file takes an oversize - * path (sample/mapReduce/deterministic), each counter's {@code label: } is prepended to the generated body — exact, language-agnostic counts (e.g. SQL {@code INSERT} - * rows, Java {@code boolean} fields) the sampled AI summary cannot reliably produce. {@code null}/empty - * = no facts block. See {@link AiFactExtractor}. + * Optional deterministic "fact" counters ({@code }). When set, each counter's + * {@code label: } is prepended to the generated body of every + * file this rule matches (oversize or not) — exact, language-agnostic structural counts (e.g. SQL + * {@code INSERT} rows / tables / views, Java types / {@code boolean} fields) that give downstream + * agents authoritative numbers a sampled AI summary cannot reliably produce. {@code null}/empty = no + * facts block. See {@link AiFactExtractor}. */ private @Nullable List facts; diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupport.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupport.java index 0047856..a686ff1 100644 --- a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupport.java +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupport.java @@ -248,18 +248,18 @@ public AiGenerationResult processFieldGenerations( final AiPreparedPrompt preparedPrompt = promptPreparationSupport.preparePrompt(request, effectiveMaxInputChars); - // Deterministic, whole-file "facts" (exact counts the sampled AI body cannot produce) are - // prepended to the body on the oversize path; computed once here over the FULL source. - String factsPrefix = ""; + // Deterministic, whole-file "facts" (exact regex counts) are prepended to the body of EVERY + // file that configures - not just oversize ones - so downstream agents get exact + // structural counts in the summary. Computed once here over the FULL source ("" when unset). + final String factsPrefix = AiFactExtractor.factsBlock(fieldGeneration.getFacts(), sourceText); // An oversized file (the source did not fit the window) is handled per the rule's onOversize // strategy. DETERMINISTIC/MAP_REDUCE replace the single generation entirely; FAIL/SAMPLE fall // through to the single-shot path below (the source was already trimmed to the window head). if (preparedPrompt.trimmed()) { - final String factsBlock = AiFactExtractor.factsBlock(fieldGeneration.getFacts(), sourceText); final AiOversizeStrategy oversize = fieldGeneration.getOversizeStrategy(); if (oversize == AiOversizeStrategy.DETERMINISTIC) { - body = factsBlock + body = factsPrefix + AiDeterministicSummary.body( sourceText, contextName(contextFile), DETERMINISTIC_SAMPLE_LINES); log.info("Oversize " + contextType + " '" + contextFile + "' -> deterministic body (no model), " @@ -267,12 +267,11 @@ public AiGenerationResult processFieldGenerations( continue; } if (oversize == AiOversizeStrategy.MAP_REDUCE) { - body = factsBlock + body = factsPrefix + mapReduceSummarize( fieldGeneration, contextFile, sourceText, baseHeader, effectiveMaxInputChars); continue; } - factsPrefix = factsBlock; } if (preparedPrompt.trimmed() && generationConfig.isWarnOnTrim()) { diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupportTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupportTest.java index 28563fb..08325ac 100644 --- a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupportTest.java +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupportTest.java @@ -318,9 +318,25 @@ private static AiModelDefinitionSupport tinyWindowModel(final String key) { } private AiFieldGenerationSupport supportWith(final AiGenerationProvider provider, final String modelKey) { + return supportWithModels(provider, tinyWindowModel(modelKey)); + } + + private AiFieldGenerationSupport supportWithModels( + final AiGenerationProvider provider, final AiModelDefinitionSupport models) { final AiPromptSupport promptSupport = new AiPromptSupport(CommonTestFixtures.createFilePromptDefinitions()); return new AiFieldGenerationSupport( - capturingLog, provider, new AiPromptPreparationSupport(promptSupport), tinyWindowModel(modelKey)); + capturingLog, provider, new AiPromptPreparationSupport(promptSupport), models); + } + + /** A model with a normal window, so a small source fits and is NOT treated as over-window. */ + private static AiModelDefinitionSupport normalWindowModel(final String key) { + final AiModelDefinition def = new AiModelDefinition(); + def.setKey(key); + def.setModelPath("unused.gguf"); + def.setContextSize(2048); + def.setCharsPerToken(4); + def.setMaxOutputTokens(64); + return new AiModelDefinitionSupport(Arrays.asList(def)); } private static AiFieldGenerationConfig oversizeRule( @@ -406,6 +422,30 @@ public void onOversize_mapReduce_withFacts_prependsExactWholeFileCountsToBody() assertThat(result.body(), containsString("PARTIAL")); } + @Test + public void facts_prependedOnNonOversizeFile() throws Exception { + // A small source that fits the window (NOT oversize) still gets the exact facts block prepended, + // so downstream agents get authoritative counts in every summary, not just huge-file ones. + final Path contextFile = Files.createTempFile("Small", ".java"); + final AiGenerationProvider provider = request -> "AI SUMMARY"; + final AiFieldGenerationSupport support = supportWithModels(provider, normalWindowModel("m")); + + final AiFieldGenerationConfig rule = new AiFieldGenerationConfig(); + rule.setPromptKey(CommonTestFixtures.PROMPT_KEY_FILE_BODY); + rule.setAiDefinitionKey("m"); + final AiFactCounter counter = new AiFactCounter(); + counter.setLabel("boolean fields"); + counter.setPattern("\\bboolean\\b"); + rule.setFacts(Collections.singletonList(counter)); + + final AiGenerationResult result = support.processFieldGenerations( + Collections.singletonList(rule), contextFile, "file", "boolean a; boolean b; int c;", anyHeader()); + + assertThat(result.body(), startsWith(AiFactExtractor.FACTS_HEADER)); + assertThat(result.body(), containsString("boolean fields: 2")); + assertThat(result.body(), containsString("AI SUMMARY")); + } + @Test public void onOversize_mapReduce_unbounded_reducesHierarchicallyInMultipleRounds() throws Exception { final Path contextFile = Files.createTempFile("Huge", ".java"); From 799c8fec299044061edd8727cab9966c5f925044 Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Wed, 1 Jul 2026 20:54:38 +0200 Subject: [PATCH 07/18] feat: shared registry + retry-on-overflow for the general path Implements the two follow-ups surfaced by the facts/mapReduce work. #3 Shared fact registry (removes the per-rule duplication). - config/AiFactDefinition {key, facts}: a named, reusable fact-counter group. - config/AiFactDefinitionSupport: key-indexed lookup; resolveFactsKeys(rules) copies a referenced group's counters onto each rule's facts (factsKey wins over inline), so the rest of the pipeline reads getFacts() uniformly. - AiFieldGenerationConfig gains ; GenerateMojo adds and resolves keys (as a MojoExecutionException on a bad key) before validation. - Self-test POM: java-facts / sql-facts defined once, referenced by java+huge-java and sql+huge-sql via instead of four inline blocks. #5 Retry-on-overflow (the cpt-estimate safety net beyond map-reduce). - generateWithOverflowRetry: if the provider rejects a prompt as exceeding the context (char-based trim under-counted tokens on dense content), re-trim to a smaller window and retry (bounded, message-matched so the indexer stays decoupled from the JNI binding); any non-overflow error propagates immediately. Now covers the single-shot path (sample / dense-but-fitting file), not just map-reduce, which already reserves a token headroom. Quality gates: mvn verify green, SpotBugs clean (scoped suppressions for the config-bean/service patterns + the deliberate NPE->MojoExecutionException bridge and the re-throw in the retry loop), PIT 531/531 (100%) incl the two new classes. Docs (CLAUDE.md) updated. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd --- CLAUDE.md | 11 +- pom.xml | 68 +++++---- spotbugs-exclude.xml | 41 +++++ .../aiindex/config/AiFactDefinition.java | 76 ++++++++++ .../config/AiFactDefinitionSupport.java | 103 +++++++++++++ .../config/AiFieldGenerationConfig.java | 27 ++++ .../indexer/AiFieldGenerationSupport.java | 92 ++++++++++- .../llamacpp/aiindex/mojo/GenerateMojo.java | 31 ++++ .../config/AiFactDefinitionSupportTest.java | 143 ++++++++++++++++++ .../aiindex/config/AiFactDefinitionTest.java | 41 +++++ .../config/AiFieldGenerationConfigTest.java | 8 + .../indexer/AiFieldGenerationSupportTest.java | 64 ++++++++ 12 files changed, 665 insertions(+), 40 deletions(-) create mode 100644 src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFactDefinition.java create mode 100644 src/main/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFactDefinitionSupport.java create mode 100644 src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFactDefinitionSupportTest.java create mode 100644 src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiFactDefinitionTest.java diff --git a/CLAUDE.md b/CLAUDE.md index ed0802e..cc1be7a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -154,7 +154,12 @@ entries abort. A rule may also carry an orthogonal, language-agnostic **` + `AiFactExtractor`): each `{label, pattern}` reports its regex match count over the **whole** source, prepended to the body of **every** file the rule matches (oversize or not) — exact structural counts (SQL `INSERT` rows / tables / views, Java type declarations / `\bboolean\b` fields, …) that give -downstream agents authoritative numbers the (possibly sampled) AI prose cannot reliably produce. See `AiFieldGenerationSelector` (selection + validation), `AiCondition`/ +downstream agents authoritative numbers the (possibly sampled) AI prose cannot reliably produce. A fact +set can be defined once in a shared `` group and referenced from many rules by +`` (`AiFactDefinition` + `AiFactDefinitionSupport`, resolved onto each rule before indexing), +instead of repeating the `` block inline. Generation is also guarded by a **retry-on-overflow**: +if the provider rejects a prompt as exceeding the context (the char-based trim under-counted tokens on +dense content), the call is re-trimmed to a smaller window and retried rather than failing the build. See `AiFieldGenerationSelector` (selection + validation), `AiCondition`/ `AiConditionEvaluator` (the tree), and `AiIndexPlan` (the rendered plan). This is how one run can index different file kinds/sizes with different models *and* prompts. @@ -202,7 +207,9 @@ the top of `execute()`. Covered by `MojoPhaseSkipTest`. | `AiOversizeStrategy` | Enum of the per-rule `` strategies (`fail`/`sample`/`mapReduce`/`deterministic`) + case-insensitive parser; default `fail` | | `AiSourceChunker` | Pure line-boundary chunker for `mapReduce` (overlap + `maxChunks` representative sampling) | | `AiFactCounter` | Maven `@Parameter` POJO for one `` — a `{label, pattern}` deterministic counter | -| `AiFactExtractor` | Pure: counts each `` regex over the whole source → the exact "facts" block prepended to the oversize body (+ fail-fast pattern validation) | +| `AiFactExtractor` | Pure: counts each `` regex over the whole source → the exact "facts" block prepended to the body (+ fail-fast pattern validation) | +| `AiFactDefinition` | Maven `@Parameter` POJO for a named, reusable `` group `{key, facts}` | +| `AiFactDefinitionSupport` | Key-indexed lookup: resolves each rule's `` to its shared group's counters (copies onto the rule) | | `AiDeterministicSummary` | Pure model-free body builder for `deterministic` (size, line count, head/tail sample) | | `AiProgressBar` | Pure ASCII progress-bar renderer (`[##### ] 42%`); `GenerateMojo` logs it after each file, advancing by the running sum of per-file plan estimates over the grand total (with estimated time left + actual elapsed) | | `PackageIndexer` | Creates `package.ai.md` files with contents listings, calls AI to fill the document body | diff --git a/pom.xml b/pom.xml index 60e281f..17ee700 100644 --- a/pom.xml +++ b/pom.xml @@ -741,6 +741,8 @@ SPDX-License-Identifier: Apache-2.0 net.ladenthin.maven.llamacpp.aiindex.provider.MockAiGenerationProvider net.ladenthin.maven.llamacpp.aiindex.config.AiOversizeStrategy net.ladenthin.maven.llamacpp.aiindex.config.AiFactCounter + net.ladenthin.maven.llamacpp.aiindex.config.AiFactDefinition + net.ladenthin.maven.llamacpp.aiindex.config.AiFactDefinitionSupport net.ladenthin.maven.llamacpp.aiindex.config.AiFactExtractor net.ladenthin.maven.llamacpp.aiindex.support.AiChecksumSupport net.ladenthin.maven.llamacpp.aiindex.support.AiDeterministicSummary @@ -2066,6 +2068,36 @@ message: the path on the first line, then a blank line, then the summaries to sy .java .sql + + + + java-facts + + (?m)^\s*(?:public\s+|final\s+|abstract\s+|sealed\s+|non-sealed\s+|static\s+)*(?:class|interface|enum|record)\s+\w + \bpublic\b + \b(?:TODO|FIXME)\b + @Override\b + + + + sql-facts + + (?im)^\s*INSERT\s+INTO\b + (?im)^\s*CREATE\s+TABLE\b + (?im)^\s*CREATE\s+(?:OR\s+REPLACE\s+)?VIEW\b + (?im)^\s*CREATE\s+(?:UNIQUE\s+)?INDEX\b + (?im)^\s*CREATE\s+(?:OR\s+REPLACE\s+)?(?:FUNCTION|PROCEDURE)\b + + + mapReduce 6 - - - (?m)^\s*(?:public\s+|final\s+|abstract\s+|sealed\s+|non-sealed\s+|static\s+)*(?:class|interface|enum|record)\s+\w - \bpublic\b - \b(?:TODO|FIXME)\b - @Override\b - + java-facts @@ -2132,13 +2158,7 @@ message: the path on the first line, then a blank line, then the summaries to sy source. Fully generic/language-agnostic - the meaning is in the regex, so the same mechanism counts INSERT rows here or `\bboolean\b` fields in Java. Multi-line matching is opt-in via the inline (?m) flag. --> - - (?im)^\s*INSERT\s+INTO\b - (?im)^\s*CREATE\s+TABLE\b - (?im)^\s*CREATE\s+(?:OR\s+REPLACE\s+)?VIEW\b - (?im)^\s*CREATE\s+(?:UNIQUE\s+)?INDEX\b - (?im)^\s*CREATE\s+(?:OR\s+REPLACE\s+)?(?:FUNCTION|PROCEDURE)\b - + sql-facts @@ -2198,15 +2218,9 @@ message: the path on the first line, then a blank line, then the summaries to sy ${ai.model} - - (?m)^\s*(?:public\s+|final\s+|abstract\s+|sealed\s+|non-sealed\s+|static\s+)*(?:class|interface|enum|record)\s+\w - \bpublic\b - \b(?:TODO|FIXME)\b - @Override\b - + not restate. Shared, defined once in above and referenced by + key here (and by huge-java) instead of repeating the block inline. --> + java-facts .java @@ -2215,13 +2229,7 @@ message: the path on the first line, then a blank line, then the summaries to sy sql file-body-sql ${ai.model} - - (?im)^\s*INSERT\s+INTO\b - (?im)^\s*CREATE\s+TABLE\b - (?im)^\s*CREATE\s+(?:OR\s+REPLACE\s+)?VIEW\b - (?im)^\s*CREATE\s+(?:UNIQUE\s+)?INDEX\b - (?im)^\s*CREATE\s+(?:OR\s+REPLACE\s+)?(?:FUNCTION|PROCEDURE)\b - + sql-facts .sql diff --git a/spotbugs-exclude.xml b/spotbugs-exclude.xml index a76d8d6..cfd423d 100644 --- a/spotbugs-exclude.xml +++ b/spotbugs-exclude.xml @@ -313,6 +313,45 @@ SPDX-License-Identifier: Apache-2.0 + + + + + + + + + + + + + + + + + + + + + net.ladenthin.maven.llamacpp.aiindex.config.AiCalibration net.ladenthin.maven.llamacpp.aiindex.config.AiCondition net.ladenthin.maven.llamacpp.aiindex.config.AiConditionEvaluator net.ladenthin.maven.llamacpp.aiindex.config.AiFieldGenerationConfig @@ -723,6 +724,7 @@ SPDX-License-Identifier: Apache-2.0 net.ladenthin.maven.llamacpp.aiindex.config.AiRangeCondition net.ladenthin.maven.llamacpp.aiindex.config.AiModelDefinition net.ladenthin.maven.llamacpp.aiindex.config.AiModelDefinitionSupport + net.ladenthin.maven.llamacpp.aiindex.indexer.AiCalibrationMeasurement net.ladenthin.maven.llamacpp.aiindex.indexer.AiInputWindowCalculator net.ladenthin.maven.llamacpp.aiindex.document.AiGenerationRequest net.ladenthin.maven.llamacpp.aiindex.document.AiGenerationResult @@ -736,6 +738,7 @@ SPDX-License-Identifier: Apache-2.0 net.ladenthin.maven.llamacpp.aiindex.prompt.AiPromptPreparationSupport net.ladenthin.maven.llamacpp.aiindex.prompt.AiPromptSupport net.ladenthin.maven.llamacpp.aiindex.provider.AiCompletionParser + net.ladenthin.maven.llamacpp.aiindex.provider.AiGenerationTimings net.ladenthin.maven.llamacpp.aiindex.provider.AiGenerationProvider net.ladenthin.maven.llamacpp.aiindex.provider.AiGenerationProviderFactory net.ladenthin.maven.llamacpp.aiindex.provider.MockAiGenerationProvider diff --git a/spotbugs-exclude.xml b/spotbugs-exclude.xml index cfd423d..7e8bc77 100644 --- a/spotbugs-exclude.xml +++ b/spotbugs-exclude.xml @@ -84,6 +84,7 @@ SPDX-License-Identifier: Apache-2.0 + @@ -116,6 +117,7 @@ SPDX-License-Identifier: Apache-2.0 + @@ -352,6 +354,29 @@ SPDX-License-Identifier: Apache-2.0 + + + + + + + + + + + + + + + + + \n"); + out.append("\n"); + out.append(String.format( + Locale.ROOT, + " %.1f%n", + m.prefillTokensPerSecond())); + out.append(String.format( + Locale.ROOT, " %.1f%n", m.decodeTokensPerSecond())); + out.append(String.format(Locale.ROOT, " %.2f%n", m.charsPerToken())); + out.append("\n"); + } +} diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/provider/AiGenerationProvider.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/provider/AiGenerationProvider.java index 262de49..3699410 100644 --- a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/provider/AiGenerationProvider.java +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/provider/AiGenerationProvider.java @@ -21,6 +21,19 @@ public interface AiGenerationProvider extends AutoCloseable { */ String generate(AiGenerationRequest request) throws IOException; + /** + * Generates text and returns it together with the model's measured timing, for the + * {@code ai-index:calibrate} goal. The default implementation delegates to {@link #generate} and + * reports no timings (rates {@code 0}); providers that expose timings override this. + * + * @param request the generation request + * @return the generated text plus timing (rates {@code 0} when unavailable) + * @throws IOException if the underlying provider fails + */ + default AiGenerationTimings generateWithTimings(final AiGenerationRequest request) throws IOException { + return new AiGenerationTimings(generate(request), 0, 0.0d, 0, 0.0d); + } + @Override default void close() throws IOException {} } diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/provider/AiGenerationTimings.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/provider/AiGenerationTimings.java new file mode 100644 index 0000000..f493442 --- /dev/null +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/provider/AiGenerationTimings.java @@ -0,0 +1,96 @@ +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: Apache-2.0 +package net.ladenthin.maven.llamacpp.aiindex.provider; + +import lombok.EqualsAndHashCode; +import lombok.ToString; +import net.ladenthin.maven.llamacpp.aiindex.support.ConvertToRecord; + +/** + * A generated text plus the model's measured timing for that generation, used by the + * {@code ai-index:calibrate} goal to derive per-machine throughput. Prefill = prompt processing; decode = + * answer generation. Rates of {@code 0} mean the provider did not report timings (e.g. the mock provider + * or a binding that omits them). + * + *

Record-shaped value type marked {@link ConvertToRecord} for the future Java 17+ migration; the + * accessors follow record style ({@code text()}, not {@code getText()}).

+ */ +@ConvertToRecord +@ToString +@EqualsAndHashCode +public final class AiGenerationTimings { + + private final String text; + private final int promptTokens; + private final double prefillTokensPerSecond; + private final int predictedTokens; + private final double decodeTokensPerSecond; + + /** + * Creates a new {@link AiGenerationTimings}. + * + * @param text the generated (parsed) text + * @param promptTokens number of prompt tokens the model processed (prefill) + * @param prefillTokensPerSecond measured prefill throughput (tokens/second) + * @param predictedTokens number of tokens the model generated (decode) + * @param decodeTokensPerSecond measured decode throughput (tokens/second) + */ + public AiGenerationTimings( + final String text, + final int promptTokens, + final double prefillTokensPerSecond, + final int predictedTokens, + final double decodeTokensPerSecond) { + this.text = text; + this.promptTokens = promptTokens; + this.prefillTokensPerSecond = prefillTokensPerSecond; + this.predictedTokens = predictedTokens; + this.decodeTokensPerSecond = decodeTokensPerSecond; + } + + /** + * Returns the generated (parsed) text. + * + * @return the text + */ + public String text() { + return text; + } + + /** + * Returns the number of prompt tokens the model processed. + * + * @return the prompt token count + */ + public int promptTokens() { + return promptTokens; + } + + /** + * Returns the measured prefill throughput (tokens/second). + * + * @return the prefill tokens per second + */ + public double prefillTokensPerSecond() { + return prefillTokensPerSecond; + } + + /** + * Returns the number of tokens the model generated. + * + * @return the predicted token count + */ + public int predictedTokens() { + return predictedTokens; + } + + /** + * Returns the measured decode throughput (tokens/second). + * + * @return the decode tokens per second + */ + public double decodeTokensPerSecond() { + return decodeTokensPerSecond; + } +} diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/provider/LlamaCppJniAiGenerationProvider.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/provider/LlamaCppJniAiGenerationProvider.java index 8df8580..02942e0 100644 --- a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/provider/LlamaCppJniAiGenerationProvider.java +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/provider/LlamaCppJniAiGenerationProvider.java @@ -128,6 +128,22 @@ private LlamaModel model() { @Override public String generate(final AiGenerationRequest request) throws IOException { + return completionParser.parseCompletion(model().chatCompleteText(buildInferenceParameters(request))); + } + + // generateWithTimings is intentionally NOT overridden: the binding's stats-returning completion path + // does not do prompt-size-proportional work here (near-window runs match mid-window ones), so it + // yields no usable prefill/decode rates. The interface default returns the real generated text with + // zero rates; ai-index:calibrate then derives throughput from the wall-clock differential of two + // sized generations (see AiCalibrationRunner), which uses this same real chatCompleteText path. + + /** + * Builds the immutable {@link InferenceParameters} for the given request from the resolved config. + * + * @param request the generation request + * @return the inference parameters + */ + private InferenceParameters buildInferenceParameters(final AiGenerationRequest request) { // Static instructions go in the SYSTEM message (byte-identical across files, so its KV // prefix is reused by cache_prompt); the variable file name + source go in the USER message. final String systemPrompt = promptSupport.systemPrompt(request.promptKey()); @@ -166,13 +182,10 @@ public String generate(final AiGenerationRequest request) throws IOException { // Only override the DRY sequence breakers when explicitly configured; an empty list keeps // the binding/model default set instead of clearing it. - final InferenceParameters inferenceParameters = - config.drySequenceBreakers().isEmpty() - ? baseParameters - : baseParameters.withDrySequenceBreakers( - config.drySequenceBreakers().toArray(new String[0])); - - return completionParser.parseCompletion(model().chatCompleteText(inferenceParameters)); + return config.drySequenceBreakers().isEmpty() + ? baseParameters + : baseParameters.withDrySequenceBreakers( + config.drySequenceBreakers().toArray(new String[0])); } @Override diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/provider/MockAiGenerationProvider.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/provider/MockAiGenerationProvider.java index d0207c8..feadf42 100644 --- a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/provider/MockAiGenerationProvider.java +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/provider/MockAiGenerationProvider.java @@ -12,6 +12,18 @@ @ToString public class MockAiGenerationProvider implements AiGenerationProvider { + /** Synthetic prefill throughput reported by the mock, so calibrate runs deterministically without a model. */ + private static final double MOCK_PREFILL_TOKENS_PER_SECOND = 1000.0d; + + /** Synthetic decode throughput reported by the mock. */ + private static final double MOCK_DECODE_TOKENS_PER_SECOND = 100.0d; + + /** Synthetic characters per token used to derive the mock prompt-token count from the source length. */ + private static final int MOCK_CHARS_PER_TOKEN = 4; + + /** Synthetic decode token count reported by the mock. */ + private static final int MOCK_PREDICTED_TOKENS = 64; + /** Creates a new {@link MockAiGenerationProvider}. */ public MockAiGenerationProvider() { // no-op @@ -24,4 +36,15 @@ public String generate(final AiGenerationRequest request) throws IOException { final String fileName = fileNamePath != null ? fileNamePath.toString() : file.toString(); return "Mock summary for " + fileName; } + + @Override + public AiGenerationTimings generateWithTimings(final AiGenerationRequest request) throws IOException { + final int promptTokens = request.sourceText().length() / MOCK_CHARS_PER_TOKEN; + return new AiGenerationTimings( + generate(request), + promptTokens, + MOCK_PREFILL_TOKENS_PER_SECOND, + MOCK_PREDICTED_TOKENS, + MOCK_DECODE_TOKENS_PER_SECOND); + } } diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/support/AiGenerationTimeEstimator.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/support/AiGenerationTimeEstimator.java index 437bee0..96fc6bc 100644 --- a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/support/AiGenerationTimeEstimator.java +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/support/AiGenerationTimeEstimator.java @@ -165,6 +165,60 @@ public long estimateSeconds(final int sourceChars, final int expectedOutputToken return Math.round(millis / MILLIS_PER_SECOND * ETA_SAFETY_MARGIN); } + /** + * Calibrated estimate assuming {@link #DEFAULT_EXPECTED_OUTPUT_TOKENS} of output. Delegates to + * {@link #estimateSecondsCalibrated(int, int, double, double, double)}. + * + * @param sourceChars number of source characters that will be sent to the model + * @param prefillTokensPerSecond measured prefill throughput, or {@code <= 0} to fall back + * @param decodeTokensPerSecond measured decode throughput, or {@code <= 0} to fall back + * @param calibratedCharsPerToken measured characters per token, or {@code <= 0} to use the default + * @return estimated total time in seconds, rounded to the nearest second + */ + public long estimateSecondsCalibrated( + final int sourceChars, + final double prefillTokensPerSecond, + final double decodeTokensPerSecond, + final double calibratedCharsPerToken) { + return estimateSecondsCalibrated( + sourceChars, + DEFAULT_EXPECTED_OUTPUT_TOKENS, + prefillTokensPerSecond, + decodeTokensPerSecond, + calibratedCharsPerToken); + } + + /** + * Estimates total generation time (seconds) using measured per-machine calibration when available. + * + *

When both {@code prefillTokensPerSecond} and {@code decodeTokensPerSecond} are {@code > 0}, uses + * the linear model {@code promptTokens / prefillTps + outputTokens / decodeTps} (with + * {@code calibratedCharsPerToken} for the token estimate when {@code > 0}); otherwise falls back to the + * built-in reference-CPU model in {@link #estimateSeconds(int, int)}.

+ * + * @param sourceChars number of source characters that will be sent to the model + * @param expectedOutputTokens number of tokens the model is expected to generate + * @param prefillTokensPerSecond measured prefill throughput, or {@code <= 0} to fall back + * @param decodeTokensPerSecond measured decode throughput, or {@code <= 0} to fall back + * @param calibratedCharsPerToken measured characters per token, or {@code <= 0} to use the default + * @return estimated total time in seconds, rounded to the nearest second + */ + public long estimateSecondsCalibrated( + final int sourceChars, + final int expectedOutputTokens, + final double prefillTokensPerSecond, + final double decodeTokensPerSecond, + final double calibratedCharsPerToken) { + if (prefillTokensPerSecond <= 0.0d || decodeTokensPerSecond <= 0.0d) { + return estimateSeconds(sourceChars, expectedOutputTokens); + } + final double charsPerToken = + calibratedCharsPerToken > 0.0d ? calibratedCharsPerToken : ESTIMATION_CHARS_PER_TOKEN; + final int promptTokens = (int) Math.round(sourceChars / charsPerToken) + PROMPT_TEMPLATE_TOKEN_OVERHEAD; + final double seconds = promptTokens / prefillTokensPerSecond + expectedOutputTokens / decodeTokensPerSecond; + return Math.round(seconds * ETA_SAFETY_MARGIN); + } + /** * Formats a second count as a short human-readable string: "{@code ~N s}" below * {@link #MINUTE_FORMAT_THRESHOLD_SECONDS}, otherwise "{@code ~N min}". diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiCalibrationTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiCalibrationTest.java new file mode 100644 index 0000000..2b292c1 --- /dev/null +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiCalibrationTest.java @@ -0,0 +1,44 @@ +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: Apache-2.0 +package net.ladenthin.maven.llamacpp.aiindex.config; + +import static org.hamcrest.CoreMatchers.containsString; +import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.MatcherAssert.assertThat; + +import org.junit.jupiter.api.Test; + +public class AiCalibrationTest { + + @Test + public void prefill_defaultsZeroAndRoundTrips() { + final AiCalibration c = new AiCalibration(); + assertThat(c.getPrefillTokensPerSecond(), is(0.0d)); + c.setPrefillTokensPerSecond(1234.5d); + assertThat(c.getPrefillTokensPerSecond(), is(1234.5d)); + } + + @Test + public void decode_defaultsZeroAndRoundTrips() { + final AiCalibration c = new AiCalibration(); + assertThat(c.getDecodeTokensPerSecond(), is(0.0d)); + c.setDecodeTokensPerSecond(45.6d); + assertThat(c.getDecodeTokensPerSecond(), is(45.6d)); + } + + @Test + public void charsPerToken_defaultsZeroAndRoundTrips() { + final AiCalibration c = new AiCalibration(); + assertThat(c.getCharsPerToken(), is(0.0d)); + c.setCharsPerToken(3.7d); + assertThat(c.getCharsPerToken(), is(3.7d)); + } + + @Test + public void toString_includesTheRates() { + final AiCalibration c = new AiCalibration(); + c.setPrefillTokensPerSecond(1000.0d); + assertThat(c.toString(), containsString("1000.0")); + } +} diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiModelDefinitionSupportTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiModelDefinitionSupportTest.java index 329c34e..656a5aa 100644 --- a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiModelDefinitionSupportTest.java +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiModelDefinitionSupportTest.java @@ -7,6 +7,7 @@ import static org.hamcrest.CoreMatchers.equalTo; import static org.hamcrest.CoreMatchers.is; import static org.hamcrest.CoreMatchers.notNullValue; +import static org.hamcrest.CoreMatchers.sameInstance; import static org.hamcrest.MatcherAssert.assertThat; import static org.junit.jupiter.api.Assertions.assertThrows; @@ -67,6 +68,11 @@ public void getConfig_knownKey_propagatesAllFields() { definition.setTopNSigma(1.3f); definition.setRepeatPenalty(1.15f); definition.setStopStrings(Arrays.asList("", "STOP")); + final AiCalibration calibration = new AiCalibration(); + calibration.setPrefillTokensPerSecond(900.0d); + calibration.setDecodeTokensPerSecond(45.0d); + calibration.setCharsPerToken(4.2d); + definition.setCalibration(calibration); final AiModelDefinitionSupport support = new AiModelDefinitionSupport(Arrays.asList(definition)); // act @@ -89,6 +95,9 @@ public void getConfig_knownKey_propagatesAllFields() { assertThat(config.getTopNSigma(), is(equalTo(1.3f))); assertThat(config.getRepeatPenalty(), is(equalTo(1.15f))); assertThat(config.getStopStrings(), is(equalTo(Arrays.asList("", "STOP")))); + // The calibration is propagated — kills the void-call mutant that would drop the + // setCalibration(...) copy from toConfig(). + assertThat(config.getCalibration(), is(sameInstance(calibration))); // Non-default cachePrompt propagates — kills the void-call mutant that would drop the // setCachePrompt(...) copy from toConfig(). assertThat(config.isCachePrompt(), is(false)); diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiModelDefinitionTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiModelDefinitionTest.java index 7d327d0..d618635 100644 --- a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiModelDefinitionTest.java +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/AiModelDefinitionTest.java @@ -6,6 +6,7 @@ import static org.hamcrest.CoreMatchers.hasItem; import static org.hamcrest.CoreMatchers.is; import static org.hamcrest.CoreMatchers.nullValue; +import static org.hamcrest.CoreMatchers.sameInstance; import static org.hamcrest.MatcherAssert.assertThat; import java.util.Arrays; @@ -30,6 +31,18 @@ public void stopStringsNullByDefault() { assertThat(new AiModelDefinition().getStopStrings(), is(nullValue())); } + @Test + public void calibrationNullByDefaultAndRoundTrips() { + final AiModelDefinition d = new AiModelDefinition(); + assertThat(d.getCalibration(), is(nullValue())); + final AiCalibration calibration = new AiCalibration(); + calibration.setPrefillTokensPerSecond(1000.0d); + d.setCalibration(calibration); + // Asserting the same instance kills the empty-return mutant on the getter and the + // assignment-removal mutant on the setter. + assertThat(d.getCalibration(), is(sameInstance(calibration))); + } + @Test public void stopStringsRoundTrip() { AiModelDefinition d = new AiModelDefinition(); diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiCalibrationMeasurementTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiCalibrationMeasurementTest.java new file mode 100644 index 0000000..b3c5a00 --- /dev/null +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiCalibrationMeasurementTest.java @@ -0,0 +1,22 @@ +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: Apache-2.0 +package net.ladenthin.maven.llamacpp.aiindex.indexer; + +import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.MatcherAssert.assertThat; + +import org.junit.jupiter.api.Test; + +public class AiCalibrationMeasurementTest { + + @Test + public void accessorsReturnConstructorValues() { + final AiCalibrationMeasurement m = new AiCalibrationMeasurement(3.5d, 900.0d, 45.0d, 4.2d, 1200.0d); + assertThat(m.loadSeconds(), is(3.5d)); + assertThat(m.prefillTokensPerSecond(), is(900.0d)); + assertThat(m.decodeTokensPerSecond(), is(45.0d)); + assertThat(m.charsPerToken(), is(4.2d)); + assertThat(m.midPrefillTokensPerSecond(), is(1200.0d)); + } +} diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiCalibrationRunnerTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiCalibrationRunnerTest.java new file mode 100644 index 0000000..01c0dab --- /dev/null +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiCalibrationRunnerTest.java @@ -0,0 +1,75 @@ +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: Apache-2.0 +package net.ladenthin.maven.llamacpp.aiindex.indexer; + +import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.MatcherAssert.assertThat; + +import net.ladenthin.maven.llamacpp.aiindex.CommonTestFixtures; +import net.ladenthin.maven.llamacpp.aiindex.config.AiGenerationConfig; +import net.ladenthin.maven.llamacpp.aiindex.document.AiGenerationRequest; +import net.ladenthin.maven.llamacpp.aiindex.prompt.AiPromptPreparationSupport; +import net.ladenthin.maven.llamacpp.aiindex.prompt.AiPromptSupport; +import net.ladenthin.maven.llamacpp.aiindex.provider.AiGenerationProvider; +import net.ladenthin.maven.llamacpp.aiindex.provider.AiGenerationTimings; +import net.ladenthin.maven.llamacpp.aiindex.provider.MockAiGenerationProvider; +import org.junit.jupiter.api.Test; + +public class AiCalibrationRunnerTest { + + private static AiPromptPreparationSupport prep() { + return new AiPromptPreparationSupport(new AiPromptSupport(CommonTestFixtures.createFilePromptDefinitions())); + } + + private static AiGenerationConfig config() { + final AiGenerationConfig config = new AiGenerationConfig(); + config.setContextSize(2048); + config.setCharsPerToken(4); + config.setMaxOutputTokens(64); + return config; + } + + @Test + public void measure_withMockProvider_reportsTheProvidersSyntheticThroughput() throws Exception { + final AiCalibrationRunner runner = new AiCalibrationRunner(); + final AiCalibrationMeasurement m = runner.measure( + new MockAiGenerationProvider(), config(), CommonTestFixtures.PROMPT_KEY_FILE_BODY, prep()); + + // The mock reports 1000 prefill / 100 decode tok/s and ~4 chars/token (the synthetic source rounds + // up to whole lines, so it is a hair under 4); the runner surfaces them. + assertThat(m.prefillTokensPerSecond(), is(1000.0d)); + assertThat(m.decodeTokensPerSecond(), is(100.0d)); + assertThat(m.charsPerToken() > 3.9d && m.charsPerToken() <= 4.0d, is(true)); + assertThat(m.loadSeconds() >= 0.0d, is(true)); + } + + @Test + public void measure_zeroRateProvider_takesWallClockFallback() throws Exception { + // A provider that reports zero rates (like the real JNI path) forces the wall-clock fallback; the + // measured charsPerToken then comes from the config (4), proving the fallback branch ran. + final AiGenerationProvider zeroRateProvider = new AiGenerationProvider() { + @Override + public String generate(final AiGenerationRequest request) { + return "t"; + } + + @Override + public AiGenerationTimings generateWithTimings(final AiGenerationRequest request) { + return new AiGenerationTimings("t", 0, 0.0d, 0, 0.0d); + } + }; + final AiCalibrationMeasurement m = new AiCalibrationRunner() + .measure(zeroRateProvider, config(), CommonTestFixtures.PROMPT_KEY_FILE_BODY, prep()); + assertThat(m.charsPerToken(), is(4.0d)); + assertThat(m.prefillTokensPerSecond() >= 0.0d, is(true)); + assertThat(m.decodeTokensPerSecond() >= 0.0d, is(true)); + } + + @Test + public void windowChars_isPositiveForANormalWindow() { + final long window = + new AiCalibrationRunner().windowChars(config(), CommonTestFixtures.PROMPT_KEY_FILE_BODY, prep()); + assertThat(window > 0, is(true)); + } +} diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/provider/AiGenerationProviderDefaultTimingsTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/provider/AiGenerationProviderDefaultTimingsTest.java new file mode 100644 index 0000000..b63cf2d --- /dev/null +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/provider/AiGenerationProviderDefaultTimingsTest.java @@ -0,0 +1,33 @@ +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: Apache-2.0 +package net.ladenthin.maven.llamacpp.aiindex.provider; + +import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.MatcherAssert.assertThat; + +import java.nio.file.Paths; +import net.ladenthin.maven.llamacpp.aiindex.document.AiGenerationRequest; +import net.ladenthin.maven.llamacpp.aiindex.document.AiMdHeader; +import org.junit.jupiter.api.Test; + +public class AiGenerationProviderDefaultTimingsTest { + + private static final AiMdHeader HEADER = new AiMdHeader( + "Foo.java", "1.0", "C", "2026-01-01T00:00:00Z", "2026-01-01T00:00:10Z", "0.1.0", "1.0.0", "file"); + + @Test + public void defaultGenerateWithTimings_delegatesToGenerateWithZeroRates() throws Exception { + // A provider that implements only generate() inherits the interface default, which must return the + // generated text with zero timings (no measurement available). + final AiGenerationProvider provider = request -> "TEXT"; + final AiGenerationTimings timings = + provider.generateWithTimings(new AiGenerationRequest("summary", Paths.get("Foo.java"), "src", HEADER)); + + assertThat(timings.text(), is("TEXT")); + assertThat(timings.promptTokens(), is(0)); + assertThat(timings.prefillTokensPerSecond(), is(0.0d)); + assertThat(timings.predictedTokens(), is(0)); + assertThat(timings.decodeTokensPerSecond(), is(0.0d)); + } +} diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/provider/AiGenerationTimingsTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/provider/AiGenerationTimingsTest.java new file mode 100644 index 0000000..9df4ec4 --- /dev/null +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/provider/AiGenerationTimingsTest.java @@ -0,0 +1,22 @@ +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: Apache-2.0 +package net.ladenthin.maven.llamacpp.aiindex.provider; + +import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.MatcherAssert.assertThat; + +import org.junit.jupiter.api.Test; + +public class AiGenerationTimingsTest { + + @Test + public void accessorsReturnConstructorValues() { + final AiGenerationTimings t = new AiGenerationTimings("summary", 1500, 900.0d, 64, 45.0d); + assertThat(t.text(), is("summary")); + assertThat(t.promptTokens(), is(1500)); + assertThat(t.prefillTokensPerSecond(), is(900.0d)); + assertThat(t.predictedTokens(), is(64)); + assertThat(t.decodeTokensPerSecond(), is(45.0d)); + } +} diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/provider/MockAiGenerationProviderTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/provider/MockAiGenerationProviderTest.java index 5d1843b..602f851 100644 --- a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/provider/MockAiGenerationProviderTest.java +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/provider/MockAiGenerationProviderTest.java @@ -38,4 +38,17 @@ public void generateFallsBackToFullPathWhenFileNameNull() throws IOException { AiGenerationRequest request = new AiGenerationRequest("summary", root, "src", HEADER); assertThat(provider.generate(request), is("Mock summary for " + root)); } + + @Test + public void generateWithTimings_reportsDeterministicSyntheticTimings() throws IOException { + // 16 chars of source / 4 chars-per-token -> 4 mock prompt tokens; positive synthetic rates so a + // calibrate run works without a real model. + final AiGenerationRequest request = + new AiGenerationRequest("summary", Paths.get("Foo.java"), "0123456789012345", HEADER); + final AiGenerationTimings timings = provider.generateWithTimings(request); + assertThat(timings.text(), is("Mock summary for Foo.java")); + assertThat(timings.promptTokens(), is(4)); + assertThat(timings.prefillTokensPerSecond() > 0.0d, is(true)); + assertThat(timings.decodeTokensPerSecond() > 0.0d, is(true)); + } } diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/support/AiGenerationTimeEstimatorTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/support/AiGenerationTimeEstimatorTest.java index 26d3b27..e05a926 100644 --- a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/support/AiGenerationTimeEstimatorTest.java +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/support/AiGenerationTimeEstimatorTest.java @@ -55,6 +55,44 @@ public void estimateSeconds_defaultOverload_usesDefaultExpectedOutputTokens() { } // + // + @Test + public void calibrated_usesLinearMeasuredThroughput() { + // sourceChars 4800 / cpt 4.8 = 1000 + 700 template = 1700 prompt tokens. + // (1700/1000 prefill + 800/100 decode) = 9.7 s; ×1.15 margin -> 11 s. + assertThat(estimator.estimateSecondsCalibrated(4800, 800, 1000.0d, 100.0d, 4.8d), is(11L)); + } + + @Test + public void calibrated_zeroPrefillRate_fallsBackToBuiltInModel() { + assertThat( + estimator.estimateSecondsCalibrated(4800, 800, 0.0d, 100.0d, 4.8d), + is(estimator.estimateSeconds(4800, 800))); + } + + @Test + public void calibrated_zeroDecodeRate_fallsBackToBuiltInModel() { + assertThat( + estimator.estimateSecondsCalibrated(4800, 800, 1000.0d, 0.0d, 4.8d), + is(estimator.estimateSeconds(4800, 800))); + } + + @Test + public void calibrated_zeroCharsPerToken_usesDefaultEstimationRate() { + assertThat( + estimator.estimateSecondsCalibrated(4800, 800, 1000.0d, 100.0d, 0.0d), + is(estimator.estimateSecondsCalibrated(4800, 800, 1000.0d, 100.0d, 4.8d))); + } + + @Test + public void calibrated_defaultOverload_usesDefaultExpectedOutputTokens() { + assertThat( + estimator.estimateSecondsCalibrated(4800, 1000.0d, 100.0d, 4.8d), + is(estimator.estimateSecondsCalibrated( + 4800, AiGenerationTimeEstimator.DEFAULT_EXPECTED_OUTPUT_TOKENS, 1000.0d, 100.0d, 4.8d))); + } + // + // @Test public void formatDuration_belowThreshold_rendersSeconds() { From a9acc38453f052f9c79cd33707e3fb1c095a3cce Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Wed, 1 Jul 2026 23:47:49 +0200 Subject: [PATCH 09/18] fix: use for the live per-file ETA too, not just the plan The calibrate work wired per-machine calibration into the plan (SourceFileIndexer .classify) but left the runtime per-file "estimated ~X" log using the built-in reference-CPU model, so the live ETA disagreed with the (calibrated) plan. AiFieldGenerationSupport now applies the same calibration-aware estimate. mvn verify-affected green, SpotBugs clean. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd --- .../indexer/AiFieldGenerationSupport.java | 26 ++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-) diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupport.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupport.java index 67e54fc..73c3be6 100644 --- a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupport.java +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupport.java @@ -10,6 +10,7 @@ import java.util.List; import java.util.Map; import lombok.ToString; +import net.ladenthin.maven.llamacpp.aiindex.config.AiCalibration; import net.ladenthin.maven.llamacpp.aiindex.config.AiFactExtractor; import net.ladenthin.maven.llamacpp.aiindex.config.AiFieldGenerationConfig; import net.ladenthin.maven.llamacpp.aiindex.config.AiGenerationConfig; @@ -306,7 +307,9 @@ public AiGenerationResult processFieldGenerations( final String sourceSizeKb = sourceText.length() < KIBIBYTES_DIVISOR ? SIZE_BELOW_ONE_KIB_LABEL : Integer.toString(sourceText.length() / KIBIBYTES_DIVISOR); - final long estimatedSeconds = timeEstimator.estimateSeconds(processedSourceChars); + // Use the model's measured when present, so the live per-file ETA matches the + // (already calibrated) plan ETA instead of the built-in reference-CPU model. + final long estimatedSeconds = estimateSeconds(processedSourceChars, generationConfig); log.info(PROCESSING_LOG_PREFIX + contextType + " '" + contextFile + "'" + PROCESSING_LOG_SIZE_INFIX + sourceSizeKb + PROCESSING_LOG_TOKENS_INFIX + timeEstimator.estimatePromptTokens(processedSourceChars) @@ -543,6 +546,27 @@ private static boolean isContextOverflow(final Throwable error) { return false; } + /** + * Estimates one generation's seconds for the given char count, using the model's {@code } + * (measured per-machine throughput) when present, otherwise the estimator's built-in model. Mirrors + * {@code SourceFileIndexer}'s plan-time estimate so the live per-file ETA agrees with the plan. + * + * @param sourceChars the processed source character count + * @param config the model's generation config (may carry a calibration) + * @return the estimated seconds + */ + private long estimateSeconds(final int sourceChars, final AiGenerationConfig config) { + final AiCalibration calibration = config.getCalibration(); + if (calibration == null) { + return timeEstimator.estimateSeconds(sourceChars); + } + return timeEstimator.estimateSecondsCalibrated( + sourceChars, + calibration.getPrefillTokensPerSecond(), + calibration.getDecodeTokensPerSecond(), + calibration.getCharsPerToken()); + } + /** * Returns a short display name for a context file (its file name, or the full path as a fallback). * From 78ad879c8392e82916aff785ee68e9db10a8fc94 Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Wed, 1 Jul 2026 23:51:45 +0200 Subject: [PATCH 10/18] test: end-to-end real-model mapReduce+facts smoke test (SmolLM2, guarded) Adds AiFieldGenerationSupportRealModelTest: drives onOversize=mapReduce (chunk -> hierarchical reduce) plus the exact block through the REAL llama.cpp JNI provider and the bundled 135M test model, asserting the body leads with the facts line and carries a map-reduced summary. Skipped unless -DrunNativeLlamaTests=true and the model is present (same guard as LlamaCppJniAiGenerationProviderTest). Verified locally: SmolLM2 loaded, 7090 chars -> 2 chunks -> reduce 2->1, body "**Facts (exact, whole file):** lines: 400 ...", 3.7s; skips cleanly by default. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd --- ...AiFieldGenerationSupportRealModelTest.java | 139 ++++++++++++++++++ 1 file changed, 139 insertions(+) create mode 100644 src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupportRealModelTest.java diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupportRealModelTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupportRealModelTest.java new file mode 100644 index 0000000..16faeba --- /dev/null +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiFieldGenerationSupportRealModelTest.java @@ -0,0 +1,139 @@ +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: Apache-2.0 +package net.ladenthin.maven.llamacpp.aiindex.indexer; + +import static org.hamcrest.CoreMatchers.containsString; +import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.CoreMatchers.startsWith; +import static org.hamcrest.MatcherAssert.assertThat; + +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.Collections; +import net.ladenthin.maven.llamacpp.aiindex.CommonTestFixtures; +import net.ladenthin.maven.llamacpp.aiindex.config.AiFactCounter; +import net.ladenthin.maven.llamacpp.aiindex.config.AiFactExtractor; +import net.ladenthin.maven.llamacpp.aiindex.config.AiFieldGenerationConfig; +import net.ladenthin.maven.llamacpp.aiindex.config.AiGenerationConfig; +import net.ladenthin.maven.llamacpp.aiindex.config.AiModelDefinition; +import net.ladenthin.maven.llamacpp.aiindex.config.AiModelDefinitionSupport; +import net.ladenthin.maven.llamacpp.aiindex.document.AiGenerationResult; +import net.ladenthin.maven.llamacpp.aiindex.document.AiMdHeader; +import net.ladenthin.maven.llamacpp.aiindex.prompt.AiPromptPreparationSupport; +import net.ladenthin.maven.llamacpp.aiindex.prompt.AiPromptSupport; +import net.ladenthin.maven.llamacpp.aiindex.provider.LlamaCppJniAiGenerationProvider; +import net.ladenthin.maven.llamacpp.aiindex.provider.LlamaCppJniConfig; +import org.apache.maven.plugin.logging.SystemStreamLog; +import org.junit.jupiter.api.Assumptions; +import org.junit.jupiter.api.Test; + +/** + * End-to-end smoke test of the {@code onOversize=mapReduce} pipeline (chunk → hierarchical reduce) + * plus the exact {@code } block, driven by the real llama.cpp JNI provider and the small + * bundled test model. Skipped unless the native lib is available and {@code -DrunNativeLlamaTests=true}. + * Unit tests cover the orchestration with the mock provider; this proves the same path works against a + * real model (real generation, real prompt-cache reuse, real trimming). + */ +public class AiFieldGenerationSupportRealModelTest { + + private static final String MODEL_PATH = Paths.get("src", "test", "resources", "SmolLM2-135M-Instruct-Q3_K_M.gguf") + .toAbsolutePath() + .toString(); + + /** Small context so a modest synthetic source is over-window and triggers map-reduce with the tiny model. */ + private static final int SMALL_CONTEXT = 512; + + private static final String MODEL_KEY = "smol"; + + private static AiMdHeader header() { + return new AiMdHeader( + "Data.java", "1.0", "0", "2026-01-01T00:00:00Z", "2026-01-01T00:00:00Z", "0", "0", "file"); + } + + private static String largeSource(final int lines) { + final StringBuilder sb = new StringBuilder(); + for (int i = 0; i < lines; i++) { + sb.append("line ").append(i).append(" = value;\n"); + } + return sb.toString(); + } + + @Test + public void mapReduceWithFacts_realModel_producesFactsPlusSummary() throws Exception { + Assumptions.assumeTrue( + Boolean.getBoolean("runNativeLlamaTests"), + "Native llama test disabled. Enable with -DrunNativeLlamaTests=true"); + Assumptions.assumeTrue(Files.exists(Paths.get(MODEL_PATH)), "Model file missing: " + MODEL_PATH); + + final AiModelDefinition def = new AiModelDefinition(); + def.setKey(MODEL_KEY); + def.setModelPath(MODEL_PATH); + def.setContextSize(SMALL_CONTEXT); + def.setMaxOutputTokens(48); + def.setCharsPerToken(3); + final AiModelDefinitionSupport models = new AiModelDefinitionSupport(Collections.singletonList(def)); + + final AiPromptSupport promptSupport = new AiPromptSupport(CommonTestFixtures.createFilePromptDefinitions()); + final AiPromptPreparationSupport prep = new AiPromptPreparationSupport(promptSupport); + final LlamaCppJniConfig jniConfig = new LlamaCppJniConfig( + null, + MODEL_PATH, + SMALL_CONTEXT, + 48, + 0.15f, + 8, + AiGenerationConfig.DEFAULT_TOP_P, + AiGenerationConfig.DEFAULT_TOP_K, + AiGenerationConfig.DEFAULT_MIN_P, + AiGenerationConfig.DEFAULT_TOP_N_SIGMA, + AiGenerationConfig.DEFAULT_REPEAT_PENALTY, + AiGenerationConfig.DEFAULT_CHAT_TEMPLATE_ENABLE_THINKING, + AiGenerationConfig.DEFAULT_CACHE_PROMPT, + AiGenerationConfig.DEFAULT_SWA_FULL, + AiGenerationConfig.DEFAULT_CACHE_REUSE, + AiGenerationConfig.DEFAULT_GPU_LAYERS, + AiGenerationConfig.DEFAULT_MAIN_GPU, + AiGenerationConfig.DEFAULT_DEVICES, + AiGenerationConfig.DEFAULT_REASONING_EFFORT, + AiGenerationConfig.DEFAULT_REASONING_BUDGET_TOKENS, + AiGenerationConfig.DEFAULT_DRY_MULTIPLIER, + AiGenerationConfig.DEFAULT_DRY_BASE, + AiGenerationConfig.DEFAULT_DRY_ALLOWED_LENGTH, + AiGenerationConfig.DEFAULT_DRY_PENALTY_LAST_N, + Collections.emptyList(), + Collections.emptyList()); + + final AiFieldGenerationConfig rule = new AiFieldGenerationConfig(); + rule.setPromptKey(CommonTestFixtures.PROMPT_KEY_FILE_BODY); + rule.setAiDefinitionKey(MODEL_KEY); + rule.setOnOversize("mapReduce"); + rule.setMaxChunks(2); + final AiFactCounter counter = new AiFactCounter(); + counter.setLabel("lines"); + counter.setPattern("(?m)^line "); + rule.setFacts(Collections.singletonList(counter)); + + final int lineCount = 400; + final String source = largeSource(lineCount); + final Path contextFile = Files.createTempFile("Data", ".java"); + + try (LlamaCppJniAiGenerationProvider provider = new LlamaCppJniAiGenerationProvider(jniConfig, promptSupport)) { + final AiFieldGenerationSupport support = + new AiFieldGenerationSupport(new SystemStreamLog(), provider, prep, models); + final AiGenerationResult result = support.processFieldGenerations( + Collections.singletonList(rule), contextFile, "file", source, header()); + + // The exact facts (counted over the whole source) lead the body; the map-reduced AI summary + // follows. This proves the real-model chunk -> hierarchical-reduce path completes end-to-end. + assertThat(result.body(), startsWith(AiFactExtractor.FACTS_HEADER)); + assertThat(result.body(), containsString("lines: " + lineCount)); + assertThat( + result.body().length() + > AiFactExtractor.factsBlock(rule.getFacts(), source) + .length(), + is(true)); + } + } +} From 0c3db34cc22e6a5309a27b13c77ea0ca141bc824 Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Thu, 2 Jul 2026 09:28:48 +0200 Subject: [PATCH 11/18] feat(facts): add honestly-labelled Java structural counts (methods/ctors/fields) Adds three regex-based structural facts to the shipped java-facts group, from a tested design: "methods (approx)", "constructors", and "field declarations (w/ modifier)". Labels are deliberately honest about the regex heuristic limits: - methods (approx): ~3% on real code, and includes constructors (subtract the constructors fact for the true method count). - field declarations (w/ modifier): only counts fields with a STRONG modifier (public/private/protected/static/transient/volatile) - final-only and package-private fields are NOT counted, on purpose, so that `final` LOCAL variables in method bodies are not miscounted as fields (that was a +104% over-count in testing; excluding final-only swings it to a safe under-count). Validated in the real java.util.regex engine (JavaStructureFactPatternsTest, 14 cases covering multi-line methods, interface methods, control structures, calls, lambdas, multi-declarator fields, final locals) and cross-checked against real OpenJDK source. Patterns are XML-escaped in the POM (< -> <); an effective-pom round-trip confirms they unescape to the tested patterns. For EXACT structure a real Java parser is still the only reliable route. mvn verify green, SpotBugs clean. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd --- pom.xml | 9 ++ .../config/JavaStructureFactPatternsTest.java | 140 ++++++++++++++++++ 2 files changed, 149 insertions(+) create mode 100644 src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/JavaStructureFactPatternsTest.java diff --git a/pom.xml b/pom.xml index 786c75a..e26de79 100644 --- a/pom.xml +++ b/pom.xml @@ -2088,6 +2088,15 @@ message: the path on the first line, then a blank line, then the summaries to sy \bpublic\b \b(?:TODO|FIXME)\b @Override\b + + (?m)^[ \t]*(?:(?:public|private|protected|static|final|abstract|default|synchronized|native|strictfp)[ \t]+)*(?:<[^>]+>[ \t]*)?(?!(?:if|for|while|switch|catch|return|new|else|do|try|synchronized|assert|throw)\b)[A-Za-z_$][\w$.]*(?:<[^;{}=]*>)?(?:\[\])*[ \t]+([A-Za-z_$]\w*)[ \t]*\([^;{]*\)(?:[ \t]*throws[ \t][\w$., \t]+)?[ \t]*[{;] + (?m)^[ \t]*(?:(?:public|private|protected)[ \t]+)?([A-Z][A-Za-z0-9_$]*)[ \t]*\([^;{]*\)(?:[ \t]*throws[ \t][\w$., \t]+)?[ \t]*\{ + (?m)^[ \t]*(?:(?:public|private|protected|static|transient|volatile|final)[ \t]+)*(?:public|private|protected|static|transient|volatile)[ \t]+(?:(?:public|private|protected|static|transient|volatile|final)[ \t]+)*(?!class\b|interface\b|enum\b|record\b|void\b|new\b)[A-Za-z_$][\w$.]*(?:<[^;{}=]*>)?(?:\[\])*[ \t]+([A-Za-z_$]\w*(?:[ \t]*,[ \t]*[A-Za-z_$]\w*)*)[ \t]*(?:=[^;]*)?; diff --git a/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/JavaStructureFactPatternsTest.java b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/JavaStructureFactPatternsTest.java new file mode 100644 index 0000000..8bec48d --- /dev/null +++ b/src/test/java/net/ladenthin/maven/llamacpp/aiindex/config/JavaStructureFactPatternsTest.java @@ -0,0 +1,140 @@ +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: Apache-2.0 +package net.ladenthin.maven.llamacpp.aiindex.config; + +import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.MatcherAssert.assertThat; + +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import org.junit.jupiter.api.Test; + +/** + * Validates the Java structural {@code } regexes shipped in the POM's {@code java-facts} group, + * in the real {@link java.util.regex} engine (the same engine {@code AiFactExtractor} uses), against the + * tricky-case self-tests. These are deliberately approximate ("methods (approx)", "field declarations") + * — the tests pin the accepted false-positive/negative boundary so the patterns cannot silently drift. + * + *

Keep these three patterns in sync with the {@code java-facts} {@code } in + * {@code pom.xml} (there they are XML-escaped: {@code <} becomes {@code <}).

+ */ +public class JavaStructureFactPatternsTest { + + /** Methods: modifiers, type params, generic/array return, multi-line params, throws, {@code {} or ;}. */ + private static final String METHOD_PATTERN = + "(?m)^[ \\t]*(?:(?:public|private|protected|static|final|abstract|default|synchronized|native|strictfp)" + + "[ \\t]+)*" + + "(?:<[^>]+>[ \\t]*)?" + + "(?!(?:if|for|while|switch|catch|return|new|else|do|try|synchronized|assert|throw)\\b)" + + "[A-Za-z_$][\\w$.]*(?:<[^;{}=]*>)?(?:\\[\\])*[ \\t]+" + + "([A-Za-z_$]\\w*)[ \\t]*" + + "\\([^;{]*\\)" + + "(?:[ \\t]*throws[ \\t][\\w$., \\t]+)?" + + "[ \\t]*[{;]"; + + /** Field declarations that carry a STRONG modifier (final-only excluded, to skip final locals). */ + private static final String FIELD_PATTERN = "(?m)^[ \\t]*" + + "(?:(?:public|private|protected|static|transient|volatile|final)[ \\t]+)*" + + "(?:public|private|protected|static|transient|volatile)[ \\t]+" + + "(?:(?:public|private|protected|static|transient|volatile|final)[ \\t]+)*" + + "(?!class\\b|interface\\b|enum\\b|record\\b|void\\b|new\\b)" + + "[A-Za-z_$][\\w$.]*(?:<[^;{}=]*>)?(?:\\[\\])*[ \\t]+" + + "([A-Za-z_$]\\w*(?:[ \\t]*,[ \\t]*[A-Za-z_$]\\w*)*)" + + "[ \\t]*(?:=[^;]*)?;"; + + /** Constructors: capitalized name directly followed by {@code (}, ending in {@code {}. */ + private static final String CTOR_PATTERN = "(?m)^[ \\t]*(?:(?:public|private|protected)[ \\t]+)?" + + "([A-Z][A-Za-z0-9_$]*)[ \\t]*\\([^;{]*\\)" + + "(?:[ \\t]*throws[ \\t][\\w$., \\t]+)?[ \\t]*\\{"; + + private static int count(final String pattern, final String source) { + final Matcher matcher = Pattern.compile(pattern).matcher(source); + int count = 0; + while (matcher.find()) { + count++; + } + return count; + } + + // + @Test + public void method_annotatedOnOwnLine_counts() { + assertThat(count(METHOD_PATTERN, "@Override\npublic int foo() {\n}"), is(1)); + } + + @Test + public void method_genericMultilineThrows_counts() { + assertThat(count(METHOD_PATTERN, "public T bar(List x,\n int y) throws IOException {\n}"), is(1)); + } + + @Test + public void method_arrayReturn_counts() { + assertThat(count(METHOD_PATTERN, "int[] baz() {\n}"), is(1)); + } + + @Test + public void method_abstractSemicolon_counts() { + assertThat(count(METHOD_PATTERN, "abstract void run();"), is(1)); + } + + @Test + public void method_doesNotCountControlStructures() { + assertThat(count(METHOD_PATTERN, "if (foo()) { bar(); }"), is(0)); + assertThat(count(METHOD_PATTERN, " for (int i = 0; i < n; i++) {\n}"), is(0)); + } + + @Test + public void method_doesNotCountCalls() { + assertThat(count(METHOD_PATTERN, " obj.doThing(x);\n foo(y);"), is(0)); + assertThat(count(METHOD_PATTERN, " return compute(x);"), is(0)); + } + + @Test + public void method_doesNotCountLambda() { + assertThat(count(METHOD_PATTERN, " Runnable r = () -> doWork();"), is(0)); + } + // + + // + @Test + public void field_multipleDeclarators_countAsOneStatement() { + // AiFactExtractor counts matches (statements), so "int a, b, c;" is ONE field declaration. + assertThat(count(FIELD_PATTERN, " private int a, b, c;"), is(1)); + } + + @Test + public void field_staticFinalWithInit_counts() { + assertThat(count(FIELD_PATTERN, " public static final String NAME = \"x\";"), is(1)); + } + + @Test + public void field_genericType_counts() { + assertThat(count(FIELD_PATTERN, " private Map> m;"), is(1)); + } + + @Test + public void field_transientArray_counts() { + assertThat(count(FIELD_PATTERN, " transient Object[] elementData;"), is(1)); + } + + @Test + public void field_doesNotCountLocalsOrFinalLocals() { + assertThat(count(FIELD_PATTERN, " int local = 5;"), is(0)); + assertThat(count(FIELD_PATTERN, " final int s;"), is(0)); + assertThat(count(FIELD_PATTERN, " final Object[] es = elementData;"), is(0)); + } + // + + // + @Test + public void ctor_counts() { + assertThat(count(CTOR_PATTERN, " public Foo(int x) {\n}"), is(1)); + } + + @Test + public void ctor_doesNotCountLowercaseMethod() { + assertThat(count(CTOR_PATTERN, " public void foo(int x) {\n}"), is(0)); + } + // +} From 1726857643946ddcc32f97df0a12ebe0aaf0a6a8 Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Thu, 2 Jul 2026 09:40:46 +0200 Subject: [PATCH 12/18] docs(todo): retire the done oversize/smoke-test item; note calibrate decode limit The "files beyond the big-window preset" item is fully delivered (onOversize with fail/sample/mapReduce/deterministic, hierarchical reduce, facts, and the real-model end-to-end smoke test), so it is removed. Replaced with the one genuine open limitation: ai-index:calibrate's decode rate is a wall-clock approximation (prefill is solid). Exact-Java-facts-via-parser is intentionally NOT tracked (a decided non-goal; the shipped regex facts are honestly labelled). Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd --- TODO.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/TODO.md b/TODO.md index d58953b..5d74440 100644 --- a/TODO.md +++ b/TODO.md @@ -15,7 +15,7 @@ recorded in git history and `crossrepostatus.md`, not here. - **Expand PIT mutation scope (optional).** `pom.xml` wires `100` over an explicit `` list (config / document / prompt / provider / support value+logic classes, plus `indexer.AiInputWindowCalculator` and `support.AiProgressBar`), all killed at 100%. Still out (optional, need careful fixtures): `document.AiMdDocumentCodec` / `AiMdHeaderCodec`, `prompt.AiPromptPreparationSupport`, and the newer `indexer.AiIndexPlan` / `config.AiConditionGroup`. The orchestration layers (`indexer.*` walk, `mojo.*`) and the JNI provider stay out of PIT — they need a Maven/native context rather than pure-unit mutation (see crossrepostatus "Deliberate non-parity"). -- **Files beyond the big-window preset — handled via ``.** The `granite-4.0-h-tiny-bigwindow` preset (384K context) covers source files up to ~1 MB. Larger files (or any file over its routed model's window) are handled per rule by ``: `fail` *(default — hard abort)*, `sample` (head only), `mapReduce` (chunk at line boundaries + combine, ``-capped), or `deterministic` (model-free metadata body). See `AiOversizeStrategy` / `AiSourceChunker` / `AiDeterministicSummary` and the README context-window note. Optional follow-up: a real end-to-end Granite `mapReduce` smoke test (orchestration is unit-tested + planOnly-verified today), and a still-larger 512K preset is wired in the POM only as an unused example. +- **Calibrate: decode rate is approximate (known limitation).** `ai-index:calibrate` derives per-machine throughput; the binding does not report prompt-proportional timings, so the runner uses a wall-clock differential. **Prefill** is solid (mid→near difference). **Decode** is a rough hint: it assumes the model emitted the full `maxOutputTokens`, so if generation stops early the decode tok/s is over-reported. Fine for the plan ETA (prefill dominates large files). Optional improvement: a dedicated decode-measurement pass (small prompt, large output) instead of the near-run residual (`AiCalibrationRunner`). - **Cross-repo code-quality TODOs** — see [`../workspace/policies/code-quality-todos.md`](../workspace/policies/code-quality-todos.md) for the canonical `@VisibleForTesting` design-fit review, package hierarchy review, and class/method naming review. This repo has no `@VisibleForTesting` usages today; the package and naming reviews are still open here. From a05f141d8abb5a128c1d53bbf6a786c7fb6fe4a0 Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Thu, 2 Jul 2026 10:11:35 +0200 Subject: [PATCH 13/18] improve(calibrate): decode from a probe run's ACTUAL output; honest tok/s note #10 follow-up. The wall-clock decode previously assumed the model emitted the full maxOutputTokens (over-reported). AiCalibrationRunner now measures decode from a dedicated tiny-prompt probe (its prefill KV is reused from the warmup, so its wall-clock is ~pure decode) and divides by the ACTUAL generated-text length, not an assumed budget. Validated against real GPU server timings and documented the honest cap: because the binding exposes no per-call token counts, calibrate estimates tokens via charsPerToken, so the DISPLAYED tok/s can differ from the engine's (measured ~4590 prefill / ~164 decode vs displayed ~6588 / ~893 on granite H-1B, since the filler is ~4.4 chars/token not 3). This does NOT hurt the plan ETA: the charsPerToken factor cancels in the prefill term (the plan estimates a file's tokens with the same cpt), so the estimate stays self-consistent; decode is a minor contributor for large files. The calibrate output and TODO.md now say the numbers are hardware-relative plan estimates, not exact engine throughput. mvn verify green (412 tests), SpotBugs clean. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd --- TODO.md | 2 +- .../aiindex/indexer/AiCalibrationRunner.java | 50 +++++++++++++------ .../llamacpp/aiindex/mojo/CalibrateMojo.java | 5 +- 3 files changed, 39 insertions(+), 18 deletions(-) diff --git a/TODO.md b/TODO.md index 5d74440..8ccd89b 100644 --- a/TODO.md +++ b/TODO.md @@ -15,7 +15,7 @@ recorded in git history and `crossrepostatus.md`, not here. - **Expand PIT mutation scope (optional).** `pom.xml` wires `100` over an explicit `` list (config / document / prompt / provider / support value+logic classes, plus `indexer.AiInputWindowCalculator` and `support.AiProgressBar`), all killed at 100%. Still out (optional, need careful fixtures): `document.AiMdDocumentCodec` / `AiMdHeaderCodec`, `prompt.AiPromptPreparationSupport`, and the newer `indexer.AiIndexPlan` / `config.AiConditionGroup`. The orchestration layers (`indexer.*` walk, `mojo.*`) and the JNI provider stay out of PIT — they need a Maven/native context rather than pure-unit mutation (see crossrepostatus "Deliberate non-parity"). -- **Calibrate: decode rate is approximate (known limitation).** `ai-index:calibrate` derives per-machine throughput; the binding does not report prompt-proportional timings, so the runner uses a wall-clock differential. **Prefill** is solid (mid→near difference). **Decode** is a rough hint: it assumes the model emitted the full `maxOutputTokens`, so if generation stops early the decode tok/s is over-reported. Fine for the plan ETA (prefill dominates large files). Optional improvement: a dedicated decode-measurement pass (small prompt, large output) instead of the near-run residual (`AiCalibrationRunner`). +- **Calibrate: displayed tok/s are estimates, not exact engine throughput (binding limitation).** `ai-index:calibrate` derives per-machine throughput by wall-clock (the binding does not expose per-call token counts). Prefill from the mid→near differential; decode from a tiny probe run using the ACTUAL generated-text length (not an assumed output budget). Because token counts are estimated via `charsPerToken`, the displayed tok/s can differ from the engine's real numbers (e.g. measured prefill ~4590 / decode ~164 tok/s vs displayed ~6588 / ~893 on granite H-1B, because the filler tokenizes at ~4.4 chars/token not 3). This does NOT hurt the plan: the `charsPerToken` error CANCELS in the prefill ETA (the plan estimates a file's tokens with the same cpt), so the estimate stays self-consistent; decode is a minor contributor for large files. Exact tok/s would require the `net.ladenthin:llama` binding to return per-call token counts (separate repo) — until then this is as accurate as feasible. - **Cross-repo code-quality TODOs** — see [`../workspace/policies/code-quality-todos.md`](../workspace/policies/code-quality-todos.md) for the canonical `@VisibleForTesting` design-fit review, package hierarchy review, and class/method naming review. This repo has no `@VisibleForTesting` usages today; the package and naming reviews are still open here. diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiCalibrationRunner.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiCalibrationRunner.java index cd73ebc..4c1d768 100644 --- a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiCalibrationRunner.java +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/indexer/AiCalibrationRunner.java @@ -76,9 +76,12 @@ public AiCalibrationMeasurement measure( final int midChars = clampSourceChars((long) (availableSourceChars * MID_WINDOW_FRACTION)); final int nearChars = clampSourceChars((long) (availableSourceChars * NEAR_WINDOW_FRACTION)); final AiMdHeader header = new AiMdHeader(CALIBRATION_CONTEXT_NAME, "1.0", "", "", "", "", "", "file"); + // The warmup and the decode probe use the SAME tiny source, so the probe reuses the warmup's + // prefill KV (prompt cache) and its wall-clock is close to pure decode. + final String tinySource = syntheticSource(WARMUP_SOURCE_CHARS); final long loadStartNanos = System.nanoTime(); - provider.generateWithTimings(request(promptKey, contextFile, syntheticSource(WARMUP_SOURCE_CHARS), header)); + provider.generateWithTimings(request(promptKey, contextFile, tinySource, header)); final double loadSeconds = (System.nanoTime() - loadStartNanos) / NANOS_PER_SECOND; final long midStartNanos = System.nanoTime(); @@ -91,6 +94,13 @@ public AiCalibrationMeasurement measure( provider.generateWithTimings(request(promptKey, contextFile, syntheticSource(nearChars), header)); final double nearWallSeconds = (System.nanoTime() - nearStartNanos) / NANOS_PER_SECOND; + // Decode probe: a TINY prompt (negligible prefill) so its wall-clock is almost pure decode. Its + // generated text gives the ACTUAL output size, so decode is not derived from an assumed output. + final long probeStartNanos = System.nanoTime(); + final AiGenerationTimings probe = + provider.generateWithTimings(request(promptKey, contextFile, tinySource, header)); + final double probeWallSeconds = (System.nanoTime() - probeStartNanos) / NANOS_PER_SECOND; + // Prefer the model's own reported throughput; fall back to a wall-clock differential when the // binding does not populate timings (rates come back 0). if (near.prefillTokensPerSecond() > 0.0d) { @@ -102,21 +112,26 @@ public AiCalibrationMeasurement measure( charsPerToken, mid.prefillTokensPerSecond()); } - return wallClockFallback(loadSeconds, midChars, nearChars, midWallSeconds, nearWallSeconds, near, config); + return wallClockFallback( + loadSeconds, midChars, nearChars, midWallSeconds, nearWallSeconds, probe, probeWallSeconds, config); } /** - * Derives throughput from wall-clock timing when the binding reports none. Prefill comes from the - * mid→near difference (decode cancels because both runs use the same output budget); decode comes - * from the near run's residual after subtracting the derived prefill. + * Derives throughput from wall-clock timing when the binding reports none. + * + *

Prefill comes from the mid→near difference (decode cancels because both runs use the + * same output budget). Decode comes from the tiny probe run: its wall-clock minus the tiny + * prompt's (negligible) prefill, divided by the actual number of generated tokens (from the + * probe's returned text length ÷ chars-per-token) — not an assumed output budget.

* - * @param loadSeconds the measured load + first-generation seconds - * @param midChars the mid-window source length - * @param nearChars the near-window source length - * @param midWallSeconds wall-clock seconds for the mid-window generation + * @param loadSeconds the measured load + first-generation seconds + * @param midChars the mid-window source length + * @param nearChars the near-window source length + * @param midWallSeconds wall-clock seconds for the mid-window generation * @param nearWallSeconds wall-clock seconds for the near-window generation - * @param near the near-window timings (for a possible predicted-token count) - * @param config the model config (chars/token and output budget) + * @param probe the tiny decode-probe result (its text is the actual generated output) + * @param probeWallSeconds wall-clock seconds for the decode probe + * @param config the model config (chars/token) * @return the measurement derived from wall-clock timing */ private static AiCalibrationMeasurement wallClockFallback( @@ -125,7 +140,8 @@ private static AiCalibrationMeasurement wallClockFallback( final int nearChars, final double midWallSeconds, final double nearWallSeconds, - final AiGenerationTimings near, + final AiGenerationTimings probe, + final double probeWallSeconds, final AiGenerationConfig config) { final double charsPerToken = config.getCharsPerToken() > 0 ? config.getCharsPerToken() : FALLBACK_CHARS_PER_TOKEN; @@ -135,10 +151,12 @@ private static AiCalibrationMeasurement wallClockFallback( final double deltaWall = nearWallSeconds - midWallSeconds; final double prefillTps = deltaTokens > 0.0d && deltaWall > 0.0d ? deltaTokens / deltaWall : 0.0d; - final double prefillNearSeconds = prefillTps > 0.0d ? nearTokens / prefillTps : nearWallSeconds; - final double decodeSeconds = Math.max(MIN_DECODE_SECONDS, nearWallSeconds - prefillNearSeconds); - final int outputTokens = near.predictedTokens() > 0 ? near.predictedTokens() : config.getMaxOutputTokens(); - final double decodeTps = outputTokens > 0 ? outputTokens / decodeSeconds : 0.0d; + // Decode from the tiny probe: subtract its (tiny) prefill, divide by the ACTUAL generated tokens. + final double probePromptTokens = WARMUP_SOURCE_CHARS / charsPerToken; + final double probePrefillSeconds = prefillTps > 0.0d ? probePromptTokens / prefillTps : 0.0d; + final double decodeSeconds = Math.max(MIN_DECODE_SECONDS, probeWallSeconds - probePrefillSeconds); + final double outputTokens = probe.text().length() / charsPerToken; + final double decodeTps = outputTokens > 0.0d ? outputTokens / decodeSeconds : 0.0d; return new AiCalibrationMeasurement(loadSeconds, prefillTps, decodeTps, charsPerToken, prefillTps); } diff --git a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/mojo/CalibrateMojo.java b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/mojo/CalibrateMojo.java index dc5ae3c..04fc4f0 100644 --- a/src/main/java/net/ladenthin/maven/llamacpp/aiindex/mojo/CalibrateMojo.java +++ b/src/main/java/net/ladenthin/maven/llamacpp/aiindex/mojo/CalibrateMojo.java @@ -108,7 +108,10 @@ public void execute() throws MojoExecutionException { } getLog().info(""); - getLog().info("Paste each onto its matching (numbers are per machine):"); + getLog().info("Paste each onto its matching (numbers are per machine)."); + getLog().info("Note: these are hardware-relative estimates for the PLAN ETA, not exact engine tok/s " + + "(the binding does not expose per-call token counts, so tokens are estimated via charsPerToken; " + + "that estimate cancels out in the prefill ETA, so the plan stays self-consistent)."); for (final String line : pasteBlocks.toString().split("\n", -1)) { getLog().info(line); } From e085e8b7aeb2b19f00f476678a6745abb6ced10b Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Thu, 2 Jul 2026 11:40:38 +0200 Subject: [PATCH 14/18] feat(calibrate): read the engine's EXACT per-call timings (resolves #10) The binding already returns per-call timings on the chat path; chatCompleteText just discarded them. LlamaCppJniAiGenerationProvider.generateWithTimings now calls chatComplete(...) and parses the OAI JSON via the binding's ChatResponseParser, surfacing the model's own prompt_n / predicted_n / prompt_per_second / predicted_per_second (no charsPerToken estimate, no binding change, no upstream change). AiCalibrationRunner now runs the decode probe ONLY on the fallback path (the reported path needs no probe). Verified end-to-end on the real model: calibrate reports 417 prefill / 12 decode tok/s and chars/token 4.41, matching the server's own 416.66 / 12.48 print_timing lines exactly (previously wall-clock-estimated ~6588 / ~893). - New real-model test asserts generateWithTimings returns non-zero prefill/decode and a prompt-token count that scales with prompt size (proves the real generation path). - Wall-clock + charsPerToken remains only as the mock/no-timings fallback. - Removed the "hardware-relative estimate, not exact" notes (CalibrateMojo, TODO.md); #10 is resolved. Narrowed the override's bogus throws IOException (BED); scoped RCN suppression for the defensive getTimings() null-guard. mvn verify green (413 tests, 3 native skipped), SpotBugs clean. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd --- TODO.md | 2 +- spotbugs-exclude.xml | 13 +++ .../aiindex/indexer/AiCalibrationRunner.java | 18 ++-- .../llamacpp/aiindex/mojo/CalibrateMojo.java | 5 +- .../LlamaCppJniAiGenerationProvider.java | 34 ++++++-- .../LlamaCppJniAiGenerationProviderTest.java | 85 +++++++++++++------ 6 files changed, 114 insertions(+), 43 deletions(-) diff --git a/TODO.md b/TODO.md index 8ccd89b..66f53a2 100644 --- a/TODO.md +++ b/TODO.md @@ -15,7 +15,7 @@ recorded in git history and `crossrepostatus.md`, not here. - **Expand PIT mutation scope (optional).** `pom.xml` wires `100` over an explicit `` list (config / document / prompt / provider / support value+logic classes, plus `indexer.AiInputWindowCalculator` and `support.AiProgressBar`), all killed at 100%. Still out (optional, need careful fixtures): `document.AiMdDocumentCodec` / `AiMdHeaderCodec`, `prompt.AiPromptPreparationSupport`, and the newer `indexer.AiIndexPlan` / `config.AiConditionGroup`. The orchestration layers (`indexer.*` walk, `mojo.*`) and the JNI provider stay out of PIT — they need a Maven/native context rather than pure-unit mutation (see crossrepostatus "Deliberate non-parity"). -- **Calibrate: displayed tok/s are estimates, not exact engine throughput (binding limitation).** `ai-index:calibrate` derives per-machine throughput by wall-clock (the binding does not expose per-call token counts). Prefill from the mid→near differential; decode from a tiny probe run using the ACTUAL generated-text length (not an assumed output budget). Because token counts are estimated via `charsPerToken`, the displayed tok/s can differ from the engine's real numbers (e.g. measured prefill ~4590 / decode ~164 tok/s vs displayed ~6588 / ~893 on granite H-1B, because the filler tokenizes at ~4.4 chars/token not 3). This does NOT hurt the plan: the `charsPerToken` error CANCELS in the prefill ETA (the plan estimates a file's tokens with the same cpt), so the estimate stays self-consistent; decode is a minor contributor for large files. Exact tok/s would require the `net.ladenthin:llama` binding to return per-call token counts (separate repo) — until then this is as accurate as feasible. +- **Calibrate uses the engine's exact per-call timings (no open limitation).** `ai-index:calibrate` reads the model's own `prompt_per_second` / `predicted_per_second` (and `prompt_n` / `predicted_n`) from the chat response: `LlamaCppJniAiGenerationProvider.generateWithTimings` calls `chatComplete(...)` and parses the OAI JSON via the binding's `ChatResponseParser` (the plain `chatCompleteText` had discarded the `timings`). The wall-clock + `charsPerToken` estimate remains only as a fallback for the mock provider / a build that returns no timings. - **Cross-repo code-quality TODOs** — see [`../workspace/policies/code-quality-todos.md`](../workspace/policies/code-quality-todos.md) for the canonical `@VisibleForTesting` design-fit review, package hierarchy review, and class/method naming review. This repo has no `@VisibleForTesting` usages today; the package and naming reviews are still open here. diff --git a/spotbugs-exclude.xml b/spotbugs-exclude.xml index 7e8bc77..986f75c 100644 --- a/spotbugs-exclude.xml +++ b/spotbugs-exclude.xml @@ -330,6 +330,19 @@ SPDX-License-Identifier: Apache-2.0 + + + + + + + org.pitest pitest-maven @@ -863,7 +863,7 @@ SPDX-License-Identifier: Apache-2.0 @@ -1120,7 +1120,7 @@ SPDX-License-Identifier: Apache-2.0 @@ -1142,7 +1142,7 @@ SPDX-License-Identifier: Apache-2.0 @@ -1166,7 +1166,7 @@ SPDX-License-Identifier: Apache-2.0 Same model as Q4_K_M above; Q6_K_P is a custom lossless quantization by HauhauCS that preserves quality where it matters most (effectively 1-2 quant levels higher at only 5-15 % larger file size). - Temperature 1.0 is critical for Gemma 4 — lowering degrades quality. + Temperature 1.0 is critical for Gemma 4 - lowering degrades quality. 32k context is a practical sweet spot for code summarization on local hardware. https://huggingface.co/HauhauCS/Gemma-4-E4B-Uncensored-HauhauCS-Aggressive --> @@ -1186,7 +1186,7 @@ SPDX-License-Identifier: Apache-2.0 <end_of_turn> - @@ -1323,10 +1323,10 @@ SPDX-License-Identifier: Apache-2.0 1.05 gpt-oss-20B-c16k @@ -1644,7 +1644,7 @@ SPDX-License-Identifier: Apache-2.0 1.05 @@ -1713,7 +1713,7 @@ SPDX-License-Identifier: Apache-2.0 You are a Java code indexer. Produce a dense, structured markdown summary of ONE Java source file for a searchable code index. Prioritize core logic, types, and data flow. Avoid filler words. Write in English. ## Output: start with the lead, then the sections below. -Begin with a single blockquote line — one sentence capturing what this file does in DOMAIN terms: the business/functional purpose a developer would search for (what it accomplishes), not how it is named or structured. Prefer the words a human would search ("Calculates VAT for invoices") over class mechanics ("Service class with three methods"): +Begin with a single blockquote line - one sentence capturing what this file does in DOMAIN terms: the business/functional purpose a developer would search for (what it accomplishes), not how it is named or structured. Prefer the words a human would search ("Calculates VAT for invoices") over class mechanics ("Service class with three methods"): > Then output these sections as markdown. Omit any section that does not apply. Use '####' for section labels. Never use '###' or higher. @@ -1765,7 +1765,7 @@ line, then a blank line, then the source. Index only what is literally present t You are a SQL schema indexer. Produce a dense, structured markdown summary of ONE SQL source file (DDL/DML: tables, views, procedures, functions, triggers) for a searchable code index. Prioritize the schema shape, the tables read vs written, columns, and relationships. Avoid filler words. Write in English. ## Output: start with the lead, then the sections below. -Begin with a single blockquote line — one sentence capturing what this file does in DOMAIN terms: the business/functional purpose a developer would search for (what data it models or what operation it performs), not how it is named: +Begin with a single blockquote line - one sentence capturing what this file does in DOMAIN terms: the business/functional purpose a developer would search for (what data it models or what operation it performs), not how it is named: > Then output these sections as markdown. Omit any section that does not apply. Use '####' for section labels. Never use '###' or higher. @@ -1813,7 +1813,7 @@ line, then a blank line, then the source. Index only what is literally present t You are a multi-language code indexer. Produce a dense, structured markdown summary of ONE source file (Java, C/C++, OpenCL, SQL, or similar) for a searchable code index. Prioritize core logic and data flow. Avoid filler words. Write in English. ## Output: start with the lead, then the sections below. -Begin with a single blockquote line — one sentence capturing what this file does in DOMAIN terms: the business/functional purpose a developer would search for (what it accomplishes), not how it is named or structured. Prefer the words a human would search ("Calculates VAT for invoices") over class mechanics ("Service class with three methods"): +Begin with a single blockquote line - one sentence capturing what this file does in DOMAIN terms: the business/functional purpose a developer would search for (what it accomplishes), not how it is named or structured. Prefer the words a human would search ("Calculates VAT for invoices") over class mechanics ("Service class with three methods"): > Then output these sections as markdown. Omit any section that does not apply. Use '####' for section labels. Never use '###' or higher. @@ -1858,10 +1858,10 @@ line, then a blank line, then the source. Index only what is literally present t package-body