From c3d06b477944bc5ce6a3f17eecbea63c16d4b558 Mon Sep 17 00:00:00 2001 From: eric-sabe Date: Tue, 7 Jul 2026 19:12:02 -0400 Subject: [PATCH 1/4] feat(verdict): provenance tier + configurable severity floor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The baseline pins specific reviewed findings; this is the broad dial for the daily marketplace noise. Two settings, applied after suppression: • Provenance: a finding whose location matches HONEY_TRUSTED_PATTERNS (default claude-plugins-official) is first-party; the report tags it [1st-party]. • Severity floor: a finding escalates OVERALL only at/above the floor for its provenance (HONEY_VERDICT_FLOOR / HONEY_VERDICT_FLOOR_TRUSTED). Below-floor findings move to a non-blocking "review" tier — still printed and counted (`OVERALL: … (65 review)`), but they don't flip the verdict. Every classified finding now carries _provenance and _blocking; report.sh / daily-cycle.sh (and the PowerShell mirrors) share the same classification, so they agree. A review-only run is CLEAN. Safe by default: floors default to `none` (everything blocks, as before) — a security tool must not silently hide findings out of the box. Overrides that the floor can never mute: bumblebee always blocks (known-compromised catalog), and a MUTATED pin always blocks (rug-pull tripwire); incomplete/scan_error are never reclassified. Multi-OS parity: lib/verdict.sh + win/lib/Verdict.psm1; report/daily-cycle updated on both sides. shellcheck clean; PSScriptAnalyzer clean; default verdict unchanged when no floor is set. Docs: docs/VERDICT.plan.md, README (section + 3 config vars), routine-prompt.md + triage-guide.md (the review tier). Stacked on feat/suppression-baseline. Co-Authored-By: Claude Opus 4.8 (1M context) --- README.md | 29 ++++++++++++ docs/VERDICT.plan.md | 104 ++++++++++++++++++++++++++++++++++++++++++ lib/baseline.sh | 48 +++++++++++++------ lib/verdict.sh | 59 ++++++++++++++++++++++++ report.sh | 95 +++++++++++++++++++++++--------------- routine-prompt.md | 10 +++- triage-guide.md | 18 +++++--- win/daily-cycle.ps1 | 2 +- win/lib/Baseline.psm1 | 23 ++++++++-- win/lib/Verdict.psm1 | 58 +++++++++++++++++++++++ win/report.ps1 | 70 ++++++++++++++++++---------- 11 files changed, 428 insertions(+), 88 deletions(-) create mode 100644 docs/VERDICT.plan.md create mode 100755 lib/verdict.sh create mode 100644 win/lib/Verdict.psm1 diff --git a/README.md b/README.md index c2b6b1e..510288e 100644 --- a/README.md +++ b/README.md @@ -217,6 +217,32 @@ first place — sleeper/trigger-activated code, instructions fetched from a remo URL at runtime, or content hidden in bundled images. The baseline can only suppress what a lens surfaced; widening the scanned surface is separate work. +## Verdict policy (provenance + severity floor) + +Where the baseline pins *specific* findings, the verdict policy is the *broad* +dial for the daily marketplace noise. Two settings, applied after suppression +([`docs/VERDICT.plan.md`](docs/VERDICT.plan.md)): + +- **Provenance** — a finding whose location matches `HONEY_TRUSTED_PATTERNS` + (default `claude-plugins-official`) is tagged **first-party**; the report shows + a `[1st-party]` marker. +- **Severity floor** — a finding escalates `OVERALL` only at/above the floor for + its provenance (`HONEY_VERDICT_FLOOR` for third-party, `HONEY_VERDICT_FLOOR_TRUSTED` + for first-party). Below-floor findings move to a **review tier**: still + printed and counted (`OVERALL: … (65 review)`), but non-blocking. + +```bash +# Demote first-party low/medium noise to the review tier (high/critical still block): +export HONEY_VERDICT_FLOOR_TRUSTED=high +# Or, for a marketplace you fully trust, demote all of its non-critical findings: +export HONEY_VERDICT_FLOOR_TRUSTED=critical +``` + +**Safe by default:** floors default to `none` — every finding blocks, exactly as +before. Provenance labeling is on (informational). **bumblebee always blocks** +(known-compromised catalog), and a **MUTATED** pin always blocks (rug-pull +tripwire) — the floor is a lens-noise dial, never a mute for genuine danger. + ## Optional: daily Slack triage via a Claude Local routine If you use [Claude Code](https://claude.com/claude-code) and have the Slack @@ -348,6 +374,9 @@ one-off run. You can also hand-edit `honey.conf` — one `export VAR=…` per li | `HONEY_OSV_NO_DEDUPE` | `0` | osv-scanner: `1` keeps one finding per path; default collapses identical `package@version`+advisory across paths into one (`+N more`) | | `HONEY_SKILLSPECTOR_LLM` | `0` | skillspector: enable its own LLM stage (default static `--no-llm`) | | `HONEY_BASELINE` | `/honey.baseline.json` | path to the suppression baseline (pin-and-diff); see [Suppression baseline](#suppression-baseline-pin-and-diff) | +| `HONEY_TRUSTED_PATTERNS` | `claude-plugins-official` | ERE; a finding whose location matches is **first-party** (else third-party). Empty ⇒ nothing trusted. See [Verdict policy](#verdict-policy-provenance--severity-floor) | +| `HONEY_VERDICT_FLOOR` | `none` | min severity (`none\|low\|medium\|high\|critical`) at which a **third-party** finding escalates OVERALL; below it → non-blocking "review" tier (still reported) | +| `HONEY_VERDICT_FLOOR_TRUSTED` | = `HONEY_VERDICT_FLOOR` | same, for **first-party** findings. Set `high` (or `critical`) to demote trusted-marketplace low/medium noise to review | ## Layout diff --git a/docs/VERDICT.plan.md b/docs/VERDICT.plan.md new file mode 100644 index 0000000..8281a83 --- /dev/null +++ b/docs/VERDICT.plan.md @@ -0,0 +1,104 @@ +# Verdict policy — provenance tier + severity floor — design spec + +Status: **implemented** (this branch). Companion to +[`BASELINE.plan.md`](BASELINE.plan.md). Where the baseline pins *specific* +reviewed findings, the verdict policy is the *broad* dial: it decides which +findings escalate `OVERALL` at all. + +## 1. Problem + +The suppression baseline is surgical — you pin individual reviewed findings. But +the daily noise from a trusted, first-party skill marketplace is *broad*: dozens +of low/medium false positives you don't want to pin one by one, yet don't want +flipping the verdict every day either. And even after the baseline, `OVERALL` is +still a flat worst-wins: a single low finding reads the same as a critical. + +## 2. Two dials, applied after suppression + +Both operate at the report/verdict layer, on the already-classified findings, so +`report.sh` and `daily-cycle.sh` (and their PowerShell mirrors) agree. + +### Provenance +Each finding's location is matched against `HONEY_TRUSTED_PATTERNS` (an ERE, +default `claude-plugins-official` — Anthropic's official marketplace, the usual +source of the doc-scan false positives). A match → **first-party**; otherwise +**third-party**. Provenance is shown in the report (`[1st-party]` tag) and feeds +the floor. Set `HONEY_TRUSTED_PATTERNS=` (empty) to treat everything as +third-party. + +### Severity floor +A finding **blocks** (escalates `OVERALL`) only if its severity is at or above +the floor **for its provenance**: + +- `HONEY_VERDICT_FLOOR` (default `none`) — floor for third-party/unknown. +- `HONEY_VERDICT_FLOOR_TRUSTED` (default = `HONEY_VERDICT_FLOOR`) — floor for + first-party. + +Values: `none | low | medium | high | critical`. A finding **below** its floor +is **not** dropped — it moves to a **review tier**: still printed (dimmed, marked +`[non-blocking]`), still counted on the verdict line, but it does not flip +`OVERALL`. + +## 3. Classification + +Every classified finding now also carries `_provenance` and `_blocking`. Three +outcomes for an active (non-suppressed) finding: + +| | escalates OVERALL? | shown? | +|---|---|---| +| **blocking** (`_blocking=yes`) | yes | yes, prominent | +| **review** (`_blocking=no`) | no | yes, dimmed `[non-blocking]` | +| suppressed (from the baseline) | no | summarized only | + +## 4. Non-negotiable overrides + +The floor is a *lens-noise* dial, never a way to mute genuinely dangerous +findings. Regardless of floor: + +- **bumblebee always blocks** — it is the known-compromised-package catalog + scanner; a match is never "review". +- **MUTATED pins always block** — the rug-pull tripwire fires no matter the floor. +- `incomplete` / `scan_error` are untouched (the policy only reclassifies + `exposed` findings, never partial coverage or a crash). + +## 5. Safe default + +Floors default to `none`, i.e. **every finding blocks, exactly as before** — a +security tool must not silently hide findings out of the box. Provenance +*labeling* is on by default (informational, safe). To actually quiet first-party +low/medium noise, opt in, e.g. in `honey.conf`: + +``` +export HONEY_VERDICT_FLOOR_TRUSTED=high # first-party low/medium -> review; high/critical still block +# or, to demote ALL first-party noise (it's a doc-scan marketplace you trust): +export HONEY_VERDICT_FLOOR_TRUSTED=critical +``` + +## 6. Verdict line + +The tallies are always on the line so a held run can't read as a bare all-clear: + +- `OVERALL: CLEAN (123 review)` — nothing blocking; 123 findings held below the + floor (still listed). +- `OVERALL: EXPOSED (12 suppressed, 65 review)` — real blocking findings drive + it; 65 first-party lows/mediums demoted, 12 baseline-pinned. + +`daily-cycle` exits `0` only when nothing blocks (review-only ⇒ clean). + +## 7. How it composes with the baseline + +- **Baseline** = "I reviewed *this exact finding* and it's benign" (content-hashed, per-finding). +- **Verdict policy** = "findings from *this source* below *this severity* are background noise, not alarms" (broad, per-provenance). + +Together: the marketplace's structural false positives drop to the review tier +via one `HONEY_VERDICT_FLOOR_TRUSTED` setting, and the handful you want to fully +vanish get pinned. A real HIGH in a first-party skill still blocks; a MUTATED pin +still screams. + +## 8. Files + +- **bash:** `lib/verdict.sh` (provenance + floor helpers), sourced by + `lib/baseline.sh`; `report.sh` renders blocking vs review; `daily-cycle.sh` + inherits the review count via `baseline_effective_overall`. +- **win:** `win/lib/Verdict.psm1`, imported by `Baseline.psm1`; `report.ps1`, + `daily-cycle.ps1` updated. diff --git a/lib/baseline.sh b/lib/baseline.sh index acdcaef..366f207 100755 --- a/lib/baseline.sh +++ b/lib/baseline.sh @@ -14,6 +14,11 @@ # Portability: pure bash 3.2 + jq. No associative arrays, no ${x^^}. Occurrence # indices and path tildify are done in jq (deterministic), not in shell. +# Verdict policy (provenance tier + severity floor). Sourced here so every +# classified finding also carries _provenance and _blocking. +# shellcheck source=lib/verdict.sh +. "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/verdict.sh" + # --- config ----------------------------------------------------------------- # Absolute path to the baseline file. Override with HONEY_BASELINE. @@ -123,17 +128,25 @@ _honey_classify() { ) | .out[]' 2>/dev/null \ | while IFS= read -r finding; do [ -n "$finding" ] || continue - local rule loc idx locfile absfile curhash entry expires csource cmp class reason + local rule loc idx sev prov blk locfile absfile curhash entry expires csource cmp class reason rule="$(printf '%s' "$finding" | jq -r '.rule // ""')" loc="$(printf '%s' "$finding" | jq -r '._loc // ""')" idx="$(printf '%s' "$finding" | jq -r '._index // 0')" + # Verdict policy: provenance (from location) + blocking (severity vs floor). + sev="$(printf '%s' "$finding" | jq -r '.severity // "unknown"')" + prov="$(honey_provenance "$loc")" + blk="$(honey_is_blocking "$sev" "$prov")" + # bumblebee is the known-compromised catalog scanner — its matches always + # block; the severity floor is a lens-noise dial, not a way to mute a + # known-bad package. + [ "$scanner" = "bumblebee" ] && blk="yes" entry="$(printf '%s' "$entries" | jq -c --arg s "$scanner" --arg r "$rule" \ --arg l "$loc" --argjson i "$idx" \ 'map(select(.scanner==$s and .rule==$r and .location==$l and ((.index // 0)==$i))) | .[0] // empty' 2>/dev/null)" if [ -z "$entry" ]; then - printf '%s' "$finding" | jq -c '. + {_class:"active",_reason:""}' + printf '%s' "$finding" | jq -c --arg pv "$prov" --arg bk "$blk" '. + {_class:"active",_reason:"",_provenance:$pv,_blocking:$bk}' continue fi @@ -162,7 +175,10 @@ _honey_classify() { # Content changed since review (or file gone): the tripwire. Resurface. class="mutated" fi - printf '%s' "$finding" | jq -c --arg c "$class" --arg r "$reason" '. + {_class:$c,_reason:$r}' + # A MUTATED pin is the rug-pull tripwire — it always blocks, floor be damned. + [ "$class" = "mutated" ] && blk="yes" + printf '%s' "$finding" | jq -c --arg c "$class" --arg r "$reason" --arg pv "$prov" --arg bk "$blk" \ + '. + {_class:$c,_reason:$r,_provenance:$pv,_blocking:$bk}' done } @@ -211,21 +227,24 @@ honey_classify_run() { baseline_effective_overall() { local run_dir="$1" local manifest="$run_dir/manifest.json" - local sup=0 mut=0 exp=0 overall="clean" + local sup=0 mut=0 exp=0 rev=0 overall="clean" _rank() { case "$1" in scan_error) echo 4;; exposed) echo 3;; incomplete) echo 2;; clean|skipped) echo 1;; *) echo 0;; esac; } _worse() { [ "$(_rank "$2")" -gt "$(_rank "$1")" ] && printf '%s' "$2" || printf '%s' "$1"; } _cnt() { printf '%s\n' "$1" | jq -r "select(._class==\"$2\")|._class" 2>/dev/null | grep -c . ; } + # Active + blocking findings escalate OVERALL; active + non-blocking are the + # "review" tier (reported, not blocking). suppressed never counts either way. + _blk() { printf '%s\n' "$1" | jq -r 'select(._class!="suppressed" and ._blocking=="yes")|1' 2>/dev/null | grep -c . ; } + _rev() { printf '%s\n' "$1" | jq -r 'select(._class!="suppressed" and ._blocking=="no")|1' 2>/dev/null | grep -c . ; } # bumblebee if [ -f "$manifest" ]; then local bst; bst="$(jq -r '.status // "unknown"' "$manifest" 2>/dev/null)" if [ "$bst" = "exposed" ] && [ -f "$run_dir/findings.ndjson" ]; then - local cls s m e a + local cls cls="$(honey_classify_bumblebee "$run_dir/findings.ndjson")" - s="$(_cnt "$cls" suppressed)"; m="$(_cnt "$cls" mutated)"; e="$(_cnt "$cls" expired)" - a="$(printf '%s\n' "$cls" | jq -r 'select(._class!="suppressed")|._class' 2>/dev/null | grep -c .)" - sup=$((sup+s)); mut=$((mut+m)); exp=$((exp+e)) - [ "$a" -gt 0 ] && overall="$(_worse "$overall" exposed)" + sup=$((sup+$(_cnt "$cls" suppressed))); mut=$((mut+$(_cnt "$cls" mutated))); exp=$((exp+$(_cnt "$cls" expired))) + rev=$((rev+$(_rev "$cls"))) + [ "$(_blk "$cls")" -gt 0 ] && overall="$(_worse "$overall" exposed)" else overall="$(_worse "$overall" "$bst")" fi @@ -239,16 +258,15 @@ baseline_effective_overall() { name="$(jq -r '.lens // ""' "$lj" 2>/dev/null)" lst="$(jq -r '.status // "unknown"' "$lj" 2>/dev/null)" if [ "$lst" = "exposed" ]; then - local cls s m e a + local cls cls="$(honey_classify_lens "$name" "$lj")" - s="$(_cnt "$cls" suppressed)"; m="$(_cnt "$cls" mutated)"; e="$(_cnt "$cls" expired)" - a="$(printf '%s\n' "$cls" | jq -r 'select(._class!="suppressed")|._class' 2>/dev/null | grep -c .)" - sup=$((sup+s)); mut=$((mut+m)); exp=$((exp+e)) - [ "$a" -gt 0 ] && overall="$(_worse "$overall" exposed)" + sup=$((sup+$(_cnt "$cls" suppressed))); mut=$((mut+$(_cnt "$cls" mutated))); exp=$((exp+$(_cnt "$cls" expired))) + rev=$((rev+$(_rev "$cls"))) + [ "$(_blk "$cls")" -gt 0 ] && overall="$(_worse "$overall" exposed)" else overall="$(_worse "$overall" "$lst")" fi done - printf '%s suppressed=%s mutated=%s expired=%s' "$overall" "$sup" "$mut" "$exp" + printf '%s suppressed=%s mutated=%s expired=%s review=%s' "$overall" "$sup" "$mut" "$exp" "$rev" } diff --git a/lib/verdict.sh b/lib/verdict.sh new file mode 100755 index 0000000..eb7d578 --- /dev/null +++ b/lib/verdict.sh @@ -0,0 +1,59 @@ +#!/usr/bin/env bash +# lib/verdict.sh — honey's verdict policy: provenance tier + severity floor. +# +# Sourced by lib/baseline.sh (so every classified finding carries _provenance and +# _blocking) and thus by report.sh / daily-cycle.sh / honey-baseline.sh. See +# docs/VERDICT.plan.md. +# +# Two dials, both applied at the report/verdict layer AFTER suppression: +# • Provenance: a finding whose location matches HONEY_TRUSTED_PATTERNS is +# "first-party" (e.g. Anthropic's claude-plugins-official marketplace); +# everything else is "third-party". +# • Severity floor: a finding escalates OVERALL only if its severity is at or +# above the floor for its provenance. Below-floor findings are still +# REPORTED (a "review" tier) but do not flip the verdict — so a first-party +# doc-scanner false positive stops screaming without going invisible. +# +# SAFE DEFAULT: floors default to `none` (every finding blocks, exactly as +# before) — a security tool must not silently hide findings by default. Opt in +# with HONEY_VERDICT_FLOOR_TRUSTED=high to quiet first-party low/medium noise. + +# ERE matched against a finding's location to mark it first-party. Empty ⇒ +# nothing is first-party (everything third-party). Default trusts Anthropic's +# official marketplace, the usual source of the doc-scan false positives. +honey_trusted_patterns() { printf '%s' "${HONEY_TRUSTED_PATTERNS-claude-plugins-official}"; } + +# location -> first-party | third-party +honey_provenance() { + local loc="$1" pat; pat="$(honey_trusted_patterns)" + if [ -n "$pat" ] && printf '%s' "$loc" | grep -qE "$pat"; then + printf 'first-party' + else + printf 'third-party' + fi +} + +# severity -> numeric rank (higher = worse). none = 0 so a `none` floor blocks all. +honey_sev_rank() { + case "$1" in + critical) echo 5;; high) echo 4;; medium) echo 3;; low) echo 2;; unknown) echo 1;; none) echo 0;; *) echo 1;; + esac +} + +# provenance -> the floor severity that applies to it. +# third-party/unknown → HONEY_VERDICT_FLOOR (default none) +# first-party → HONEY_VERDICT_FLOOR_TRUSTED (default = HONEY_VERDICT_FLOOR) +honey_floor_for() { + local base="${HONEY_VERDICT_FLOOR:-none}" + case "$1" in + first-party) printf '%s' "${HONEY_VERDICT_FLOOR_TRUSTED:-$base}" ;; + *) printf '%s' "$base" ;; + esac +} + +# severity provenance -> yes | no (does this finding escalate OVERALL?) +honey_is_blocking() { + local sev="$1" prov="$2" floor; floor="$(honey_floor_for "$prov")" + [ "$floor" = "none" ] && { printf 'yes'; return; } + if [ "$(honey_sev_rank "$sev")" -ge "$(honey_sev_rank "$floor")" ]; then printf 'yes'; else printf 'no'; fi +} diff --git a/report.sh b/report.sh index d4dc1af..f92d0c4 100755 --- a/report.sh +++ b/report.sh @@ -37,8 +37,9 @@ COMPLETED="$(j '.scan_completed')"; CATDIR="$(j '.catalog_dir')" FILES="$(jq -r '.files_considered // "?"' "$RUN_DIR/summary.json" 2>/dev/null)" [ -z "$FILES" ] && FILES="?" # summary.json may be empty/absent on scan_error -# Suppression tallies, accumulated across bumblebee + every lens as we render. -SUP=0; MUT=0; EXP=0 +# Suppression + verdict-policy tallies, accumulated across bumblebee + every +# lens as we render. REV = active findings held below the severity floor. +SUP=0; MUT=0; EXP=0; REV=0 # Remediation guidance per ecosystem. Generic but correct starting points; # always "run manually" — honey never changes state. @@ -92,40 +93,59 @@ render_lenses() { [ "$(rank incomplete)" -gt "$(rank "$OVERALL")" ] && OVERALL="incomplete"; continue ;; esac - # exposed → classify against the baseline, then render only the findings - # that still contribute (active + mutated + expired). Suppressed ones are - # counted and summarized, not listed. - local cls s m e a + # exposed → classify against the baseline + verdict policy. Findings split + # three ways: suppressed (hidden), blocking (escalate OVERALL), and review + # (active but below the severity floor for their provenance — reported, not + # blocking). Suppressed ones are counted and summarized, not listed. + local cls s blk rev cls="$(honey_classify_lens "$name" "$lj")" - s="$(printf '%s\n' "$cls" | jq -r 'select(._class=="suppressed")|1' 2>/dev/null | grep -c .)" - m="$(printf '%s\n' "$cls" | jq -r 'select(._class=="mutated")|1' 2>/dev/null | grep -c .)" - e="$(printf '%s\n' "$cls" | jq -r 'select(._class=="expired")|1' 2>/dev/null | grep -c .)" - a="$(printf '%s\n' "$cls" | jq -r 'select(._class!="suppressed")|1' 2>/dev/null | grep -c .)" - SUP=$((SUP+s)); MUT=$((MUT+m)); EXP=$((EXP+e)) - - if [ "$a" -eq 0 ]; then - # Every finding suppressed: this lens no longer contributes to the verdict. + s="$( printf '%s\n' "$cls" | jq -r 'select(._class=="suppressed")|1' 2>/dev/null | grep -c .)" + blk="$(printf '%s\n' "$cls" | jq -r 'select(._class!="suppressed" and ._blocking=="yes")|1' 2>/dev/null | grep -c .)" + rev="$(printf '%s\n' "$cls" | jq -r 'select(._class!="suppressed" and ._blocking=="no")|1' 2>/dev/null | grep -c .)" + SUP=$((SUP+s)) + MUT=$((MUT+$(printf '%s\n' "$cls" | jq -r 'select(._class=="mutated")|1' 2>/dev/null | grep -c .))) + EXP=$((EXP+$(printf '%s\n' "$cls" | jq -r 'select(._class=="expired")|1' 2>/dev/null | grep -c .))) + REV=$((REV+rev)) + + if [ $((blk+rev)) -eq 0 ]; then echo "${G}✓ lens ${name}: clean${O} — all ${ltotal} finding(s) suppressed by baseline. ${D}(${lnote})${O}" continue fi - [ "$(rank exposed)" -gt "$(rank "$OVERALL")" ] && OVERALL="exposed" - local by supnote="" - by="$(printf '%s\n' "$cls" | jq -rs 'map(select(._class!="suppressed")) | group_by(.severity) - | map("\(length) \(.[0].severity)") | join(", ")' 2>/dev/null)" - [ "$s" -gt 0 ] && supnote=" ${D}(+${s} suppressed)${O}" - echo "${R}🚨 lens ${name}: ${a} finding(s)${O} [$by]${supnote} ${D}(${lnote})${O}" + local extras="" + [ "$s" -gt 0 ] && extras="${extras} ${D}(+${s} suppressed)${O}" + [ "$blk" -gt 0 ] && [ "$rev" -gt 0 ] && extras="${extras} ${D}(+${rev} review)${O}" + if [ "$blk" -gt 0 ]; then + [ "$(rank exposed)" -gt "$(rank "$OVERALL")" ] && OVERALL="exposed" + local by + by="$(printf '%s\n' "$cls" | jq -rs 'map(select(._class!="suppressed" and ._blocking=="yes")) | group_by(.severity) + | map("\(length) \(.[0].severity)") | join(", ")' 2>/dev/null)" + echo "${R}🚨 lens ${name}: ${blk} finding(s)${O} [$by]${extras} ${D}(${lnote})${O}" + else + # No blocking findings — only review-tier (first-party / below floor). + echo "${Y}● lens ${name}: ${rev} review finding(s)${O} — ${D}non-blocking (first-party or below the severity floor)${O}${extras} ${D}(${lnote})${O}" + fi + + # List blocking findings first (prominent), then review (dim), each tagged + # with provenance and a [review] marker when non-blocking. printf '%s\n' "$cls" | jq -rs --argjson ord "$ORDER" ' - map(select(._class!="suppressed")) | sort_by($ord[.severity] // 9)[] - | [.severity,.title,.location,.detail,.ref,._class] | @tsv' 2>/dev/null \ - | while IFS=$'\t' read -r SEV TITLE LOC DET REF CLASS; do - case "$SEV" in critical|high) C="$R";; medium) C="$Y";; *) C="$D";; esac + map(select(._class!="suppressed")) + | sort_by([ (if ._blocking=="yes" then 0 else 1 end), ($ord[.severity] // 9) ])[] + | [.severity,.title,.location,.detail,.ref,._class,._blocking,._provenance] | @tsv' 2>/dev/null \ + | while IFS=$'\t' read -r SEV TITLE LOC DET REF CLASS BLK PROV; do SEV_UC="$(printf '%s' "$SEV" | tr '[:lower:]' '[:upper:]')" REFSUFFIX=""; [ -n "$REF" ] && REFSUFFIX=" ${D}(${REF})${O}" - echo " $(class_prefix "$CLASS")${C}● ${SEV_UC}${O} ${B}${TITLE}${O}${REFSUFFIX}" - echo " where : $LOC" - [ -n "$DET" ] && echo " detail: $DET" - [ "$CLASS" = "mutated" ] && echo " ${R}note : content changed since this was pinned reviewed-benign — re-review before re-pinning.${O}" + PTAG=""; [ "$PROV" = "first-party" ] && PTAG=" ${D}[1st-party]${O}" + if [ "$BLK" = "no" ]; then + echo " ${D}○ review ${SEV_UC} ${TITLE}${REFSUFFIX} [non-blocking]${O}" + echo " ${D}where : $LOC${O}" + else + case "$SEV" in critical|high) C="$R";; medium) C="$Y";; *) C="$D";; esac + echo " $(class_prefix "$CLASS")${C}● ${SEV_UC}${O} ${B}${TITLE}${O}${REFSUFFIX}${PTAG}" + echo " where : $LOC" + [ -n "$DET" ] && echo " detail: $DET" + [ "$CLASS" = "mutated" ] && echo " ${R}note : content changed since this was pinned reviewed-benign — re-review before re-pinning.${O}" + fi done done } @@ -187,21 +207,24 @@ esac # --- additional lenses ------------------------------------------------------ render_lenses -# --- suppression summary ---------------------------------------------------- -if [ $((SUP+MUT+EXP)) -gt 0 ]; then +# --- suppression / policy summary ------------------------------------------- +if [ $((SUP+MUT+EXP+REV)) -gt 0 ]; then echo echo "${D}baseline: ${SUP} suppressed, ${MUT} mutated, ${EXP} expired — honey.baseline.json. Review: ./honey-baseline.sh status${O}" + [ "$REV" -gt 0 ] && echo "${D}policy: ${REV} finding(s) held below the severity floor (review tier, non-blocking). Floor: ${HONEY_VERDICT_FLOOR:-none} / trusted ${HONEY_VERDICT_FLOOR_TRUSTED:-${HONEY_VERDICT_FLOOR:-none}}.${O}" [ "$MUT" -gt 0 ] && echo "${R}⚠ ${MUT} MUTATED: a file pinned as reviewed-benign has CHANGED. Treat as a possible rug pull and re-review.${O}" fi # --- overall verdict -------------------------------------------------------- -# Suffix tallies so a suppressed run can never read as a bare all-clear. +# Suffix tallies so a suppressed/held run can never read as a bare all-clear. SUFFIX="" -if [ $((SUP+MUT+EXP)) -gt 0 ]; then - SUFFIX=" (${SUP} suppressed" - [ "$MUT" -gt 0 ] && SUFFIX="${SUFFIX}, ${MUT} mutated" - [ "$EXP" -gt 0 ] && SUFFIX="${SUFFIX}, ${EXP} expired" - SUFFIX="${SUFFIX})" +if [ $((SUP+MUT+EXP+REV)) -gt 0 ]; then + parts="" + [ "$SUP" -gt 0 ] && parts="${parts}, ${SUP} suppressed" + [ "$REV" -gt 0 ] && parts="${parts}, ${REV} review" + [ "$MUT" -gt 0 ] && parts="${parts}, ${MUT} mutated" + [ "$EXP" -gt 0 ] && parts="${parts}, ${EXP} expired" + SUFFIX=" (${parts#, })" fi echo diff --git a/routine-prompt.md b/routine-prompt.md index efe09f5..3d3e274 100644 --- a/routine-prompt.md +++ b/routine-prompt.md @@ -62,8 +62,14 @@ SUPPRESSION BASELINE — the OVERALL line may carry a tally, e.g. fold it into "all clear". A run with any `mutated` is never CLEAN. • `expired` = a pin passed its expiry and resurfaced; treat as a normal active finding due for re-review. - • `CLEAN (N suppressed)` is still all-clear — but say "all clear (N previously - reviewed findings suppressed)", not a bare "all clear". + • `review` = active findings held BELOW the severity floor (verdict policy) — + typically first-party / trusted-marketplace low/medium noise. They are + reported but non-blocking. Summarize them briefly ("N low-severity first-party + findings, non-blocking"); do not treat them as urgent. A run whose only + findings are `review` is CLEAN. + • `CLEAN (N suppressed)` / `CLEAN (N review)` is still all-clear — but name the + counts ("all clear; N previously reviewed / N below-floor findings"), not a + bare "all clear". Treat report.sh's output as the factual baseline; do not contradict it or invent findings beyond it. For extra detail read HONEY_DIR/latest/lens-*.json diff --git a/triage-guide.md b/triage-guide.md index 8b9e850..d81b3c7 100644 --- a/triage-guide.md +++ b/triage-guide.md @@ -21,13 +21,17 @@ exactly matches a known-compromised entry in a threat-intelligence catalog. agent skills); the OVERALL verdict is the worst across all of them. Surface every scanner's findings, not just bumblebee's. - The OVERALL line may carry a suppression tally, e.g. `CLEAN (12 suppressed)` - or `EXPOSED — … (12 suppressed, 2 mutated)`. `suppressed` = findings the user - pinned as reviewed-benign in `honey.baseline.json` (dropped from the verdict — - don't re-alarm). `mutated` (🔁 in the report) = a pinned file CHANGED since - review — treat as high signal (possible rug pull), surface it, and never call - such a run clean. Manage pins with `./honey-baseline.sh` (see - [`docs/BASELINE.plan.md`](docs/BASELINE.plan.md)). + The OVERALL line may carry a tally, e.g. `CLEAN (12 suppressed)` or + `EXPOSED — … (12 suppressed, 65 review, 2 mutated)`. `suppressed` = findings + the user pinned as reviewed-benign in `honey.baseline.json` (dropped from the + verdict — don't re-alarm). `review` = active findings held below the severity + floor (verdict policy — usually first-party low/medium noise): reported but + non-blocking, so a `review`-only run is clean; summarize, don't alarm. + `mutated` (🔁 in the report) = a pinned file CHANGED since review — treat as + high signal (possible rug pull), surface it, and never call such a run clean. + Manage pins with `./honey-baseline.sh`; tune the floor with + `HONEY_VERDICT_FLOOR_TRUSTED` (see [`docs/BASELINE.plan.md`](docs/BASELINE.plan.md) + and [`docs/VERDICT.plan.md`](docs/VERDICT.plan.md)). 2. For detail beyond the report, read the run dir: - `manifest.json` — bumblebee verdict, counts, metadata. diff --git a/win/daily-cycle.ps1 b/win/daily-cycle.ps1 index 2bf5835..91b1a8a 100644 --- a/win/daily-cycle.ps1 +++ b/win/daily-cycle.ps1 @@ -82,5 +82,5 @@ if (Test-Path $lensDir) { $eff = Get-HoneyEffectiveOverall -RunDir $runDir $overall = Resolve-WorseStatus $eff.status $crash -Log "=== cycle end (bumblebee=$bbStatus overall=$overall suppressed=$($eff.suppressed) mutated=$($eff.mutated) expired=$($eff.expired)) ===" +Log "=== cycle end (bumblebee=$bbStatus overall=$overall suppressed=$($eff.suppressed) mutated=$($eff.mutated) expired=$($eff.expired) review=$($eff.review)) ===" if ($overall -eq 'clean') { exit 0 } else { exit 1 } diff --git a/win/lib/Baseline.psm1 b/win/lib/Baseline.psm1 index 48dbecc..305a26b 100644 --- a/win/lib/Baseline.psm1 +++ b/win/lib/Baseline.psm1 @@ -22,6 +22,7 @@ $ErrorActionPreference = 'Stop' # Import Honey WITHOUT -Force: a forced re-import here would evict Honey's # commands from a caller that already imported it (report.ps1/daily-cycle.ps1). Import-Module (Join-Path $PSScriptRoot 'Honey.psm1') +Import-Module (Join-Path $PSScriptRoot 'Verdict.psm1') # --- config ------------------------------------------------------------------ @@ -129,6 +130,14 @@ function Get-HoneyClassified { $f | Add-Member -NotePropertyName '_loc' -NotePropertyValue $loc -Force $f | Add-Member -NotePropertyName '_index' -NotePropertyValue $idx -Force + # Verdict policy: provenance (from location) + blocking (severity vs floor). + # bumblebee (known-compromised catalog) always blocks; the floor is a + # lens-noise dial, not a way to mute a known-bad package. + $sev = if ($f.PSObject.Properties.Name -contains 'severity' -and $f.severity) { [string]$f.severity } else { 'unknown' } + $prov = Get-HoneyProvenance $loc + $blk = if ($Scanner -eq 'bumblebee') { $true } else { (Test-HoneyBlocking $sev $prov) } + $f | Add-Member -NotePropertyName '_provenance' -NotePropertyValue $prov -Force + $entry = $entries | Where-Object { $_.scanner -eq $Scanner -and $_.rule -eq $rule -and $_.location -eq $loc -and (([int](Get-HoneyEntryIndex $_)) -eq $idx) @@ -137,6 +146,7 @@ function Get-HoneyClassified { if (-not $entry) { $f | Add-Member -NotePropertyName '_class' -NotePropertyValue 'active' -Force $f | Add-Member -NotePropertyName '_reason' -NotePropertyValue '' -Force + $f | Add-Member -NotePropertyName '_blocking' -NotePropertyValue $(if ($blk) { 'yes' } else { 'no' }) -Force [void]$out.Add($f); continue } @@ -155,8 +165,11 @@ function Get-HoneyClassified { if ($curhash -and $curhash -eq $cmp) { if ($expires -ne 'never' -and $expires -lt $today) { $class = 'expired' } else { $class = 'suppressed' } } + # A MUTATED pin is the rug-pull tripwire — it always blocks, floor be damned. + if ($class -eq 'mutated') { $blk = $true } $f | Add-Member -NotePropertyName '_class' -NotePropertyValue $class -Force $f | Add-Member -NotePropertyName '_reason' -NotePropertyValue $reason -Force + $f | Add-Member -NotePropertyName '_blocking' -NotePropertyValue $(if ($blk) { 'yes' } else { 'no' }) -Force [void]$out.Add($f) } return $out.ToArray() @@ -233,7 +246,7 @@ function Get-HoneyClassifiedRun { # through untouched. function Get-HoneyEffectiveOverall { param([string]$RunDir) - $sup = 0; $mut = 0; $exp = 0; $overall = 'clean' + $sup = 0; $mut = 0; $exp = 0; $rev = 0; $overall = 'clean' $manifestPath = Join-Path $RunDir 'manifest.json' if (Test-Path -LiteralPath $manifestPath) { @@ -246,7 +259,8 @@ function Get-HoneyEffectiveOverall { $sup += @($cls | Where-Object { $_._class -eq 'suppressed' }).Count $mut += @($cls | Where-Object { $_._class -eq 'mutated' }).Count $exp += @($cls | Where-Object { $_._class -eq 'expired' }).Count - if (@($cls | Where-Object { $_._class -ne 'suppressed' }).Count -gt 0) { $overall = Resolve-WorseStatus $overall 'exposed' } + $rev += @($cls | Where-Object { $_._class -ne 'suppressed' -and $_._blocking -eq 'no' }).Count + if (@($cls | Where-Object { $_._class -ne 'suppressed' -and $_._blocking -eq 'yes' }).Count -gt 0) { $overall = Resolve-WorseStatus $overall 'exposed' } } else { $overall = Resolve-WorseStatus $overall $bst } @@ -261,13 +275,14 @@ function Get-HoneyEffectiveOverall { $sup += @($cls | Where-Object { $_._class -eq 'suppressed' }).Count $mut += @($cls | Where-Object { $_._class -eq 'mutated' }).Count $exp += @($cls | Where-Object { $_._class -eq 'expired' }).Count - if (@($cls | Where-Object { $_._class -ne 'suppressed' }).Count -gt 0) { $overall = Resolve-WorseStatus $overall 'exposed' } + $rev += @($cls | Where-Object { $_._class -ne 'suppressed' -and $_._blocking -eq 'no' }).Count + if (@($cls | Where-Object { $_._class -ne 'suppressed' -and $_._blocking -eq 'yes' }).Count -gt 0) { $overall = Resolve-WorseStatus $overall 'exposed' } } else { $overall = Resolve-WorseStatus $overall $lst } } - return @{ status = $overall; suppressed = $sup; mutated = $mut; expired = $exp } + return @{ status = $overall; suppressed = $sup; mutated = $mut; expired = $exp; review = $rev } } Export-ModuleMember -Function Get-HoneyBaselineFile, Get-HoneyBaselineEntry, diff --git a/win/lib/Verdict.psm1 b/win/lib/Verdict.psm1 new file mode 100644 index 0000000..f878f00 --- /dev/null +++ b/win/lib/Verdict.psm1 @@ -0,0 +1,58 @@ +<# +.SYNOPSIS + honey (Windows): verdict policy (provenance tier + severity floor) — PowerShell + mirror of lib/verdict.sh. Imported by Baseline.psm1 so every classified finding + carries _provenance and _blocking. See docs/VERDICT.plan.md. + +.NOTES + Provenance: a finding whose location matches HONEY_TRUSTED_PATTERNS is + "first-party"; everything else "third-party". Severity floor: a finding + escalates OVERALL only if its severity is at/above the floor for its + provenance. Below-floor findings are still reported (a "review" tier) but do + not flip the verdict. SAFE DEFAULT: floors default to `none` (everything + blocks, as before). +#> + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' +Import-Module (Join-Path $PSScriptRoot 'Honey.psm1') + +# ERE matched against a location to mark it first-party. Unset -> the Anthropic +# marketplace default; empty -> nothing is first-party. (mirrors ${VAR-default}) +function Get-HoneyTrustedPattern { + Get-HoneySettingRaw 'HONEY_TRUSTED_PATTERNS' 'claude-plugins-official' +} + +function Get-HoneyProvenance { + param([string]$Location) + $pat = Get-HoneyTrustedPattern + if ($pat -and ($Location -match $pat)) { return 'first-party' } + return 'third-party' +} + +# severity -> numeric rank (higher = worse); none = 0 so a `none` floor blocks all. +function Get-HoneySeverityRank { + param([string]$Severity) + switch ($Severity) { + 'critical' { 5 } 'high' { 4 } 'medium' { 3 } 'low' { 2 } 'unknown' { 1 } 'none' { 0 } default { 1 } + } +} + +# provenance -> the floor severity that applies to it. +function Get-HoneyFloorFor { + param([string]$Provenance) + $base = Get-HoneySetting 'HONEY_VERDICT_FLOOR' 'none' + if ($Provenance -eq 'first-party') { return (Get-HoneySetting 'HONEY_VERDICT_FLOOR_TRUSTED' $base) } + return $base +} + +# severity, provenance -> $true if this finding escalates OVERALL. +function Test-HoneyBlocking { + param([string]$Severity, [string]$Provenance) + $floor = Get-HoneyFloorFor $Provenance + if ($floor -eq 'none') { return $true } + return ((Get-HoneySeverityRank $Severity) -ge (Get-HoneySeverityRank $floor)) +} + +Export-ModuleMember -Function Get-HoneyTrustedPattern, Get-HoneyProvenance, + Get-HoneySeverityRank, Get-HoneyFloorFor, Test-HoneyBlocking diff --git a/win/report.ps1 b/win/report.ps1 index 3767928..821bb0c 100644 --- a/win/report.ps1 +++ b/win/report.ps1 @@ -52,8 +52,9 @@ if (Test-Path $sumPath) { $L = New-Object System.Collections.ArrayList function Emit { param($s) [void]$L.Add($s) } -# Suppression tallies, accumulated across bumblebee + every lens as we render. -$script:sup = 0; $script:mut = 0; $script:exp = 0 +# Suppression + verdict-policy tallies, accumulated across bumblebee + every +# lens. rev = active findings held below the severity floor (review tier). +$script:sup = 0; $script:mut = 0; $script:exp = 0; $script:rev = 0 # A class marker printed before a finding line (mutated/expired resurface loudly). function Get-ClassMarker { param($c) switch ($c) { 'mutated' { '[MUTATED] ' } 'expired' { '[EXPIRED-PIN] ' } default { '' } } } @@ -119,45 +120,68 @@ Get-ChildItem -Path $RunDir -Filter 'lens-*.json' -ErrorAction SilentlyContinue 'exposed' { $cls = Get-HoneyClassifiedLens -Scanner $name -LensObj $lj $s = @($cls | Where-Object { $_._class -eq 'suppressed' }).Count - $active = @($cls | Where-Object { $_._class -ne 'suppressed' }) + $blocking = @($cls | Where-Object { $_._class -ne 'suppressed' -and $_._blocking -eq 'yes' }) + $review = @($cls | Where-Object { $_._class -ne 'suppressed' -and $_._blocking -eq 'no' }) $script:sup += $s $script:mut += @($cls | Where-Object { $_._class -eq 'mutated' }).Count $script:exp += @($cls | Where-Object { $_._class -eq 'expired' }).Count - if ($active.Count -eq 0) { + $script:rev += $review.Count + if (($blocking.Count + $review.Count) -eq 0) { Emit "[OK] lens ${name}: clean - all $tot finding(s) suppressed by baseline. ($note)" } else { - $overall = Resolve-WorseStatus $overall 'exposed' - $bySev = @($active) | Group-Object severity | ForEach-Object { "$($_.Count) $($_.Name)" } - $supStr = if ($s -gt 0) { " (+$s suppressed)" } else { "" } - Emit "[!!] lens ${name}: $($active.Count) finding(s) [$(( $bySev ) -join ', ')]$supStr ($note)" - $rank = @{ critical=0; high=1; medium=2; low=3; unknown=4 } - @($active) | Sort-Object @{ Expression = { $r = $rank[[string](Get-Prop $_ 'severity' 'unknown')]; if ($null -eq $r) { 9 } else { $r } } } | ForEach-Object { - Emit (" - {0}{1} {2}" -f (Get-ClassMarker $_._class), (Get-Prop $_ 'severity' '?').ToUpper(), (Get-Prop $_ 'title' '?')) - $loc = Get-Prop $_ 'location' '' - if ($loc) { Emit " where : $loc" } - $det = Get-Prop $_ 'detail' '' - if ($det) { Emit " detail: $det" } - if ($_._class -eq 'mutated') { Emit " note : content changed since this was pinned reviewed-benign - re-review before re-pinning." } + $extras = "" + if ($s -gt 0) { $extras += " (+$s suppressed)" } + if ($blocking.Count -gt 0 -and $review.Count -gt 0) { $extras += " (+$($review.Count) review)" } + if ($blocking.Count -gt 0) { + $overall = Resolve-WorseStatus $overall 'exposed' + $bySev = @($blocking) | Group-Object severity | ForEach-Object { "$($_.Count) $($_.Name)" } + Emit "[!!] lens ${name}: $($blocking.Count) finding(s) [$(( $bySev ) -join ', ')]$extras ($note)" + } else { + Emit "[~] lens ${name}: $($review.Count) review finding(s) - non-blocking (first-party or below the severity floor)$extras ($note)" } + $rank = @{ critical=0; high=1; medium=2; low=3; unknown=4 } + @($cls | Where-Object { $_._class -ne 'suppressed' }) | + Sort-Object @{ Expression = { if ($_._blocking -eq 'yes') { 0 } else { 1 } } }, @{ Expression = { $r = $rank[[string](Get-Prop $_ 'severity' 'unknown')]; if ($null -eq $r) { 9 } else { $r } } } | + ForEach-Object { + $ptag = if ($_._provenance -eq 'first-party') { ' [1st-party]' } else { '' } + if ($_._blocking -eq 'no') { + Emit (" o review {0} {1} [non-blocking]" -f (Get-Prop $_ 'severity' '?').ToUpper(), (Get-Prop $_ 'title' '?')) + Emit " where : $(Get-Prop $_ 'location' '')" + } else { + Emit (" - {0}{1} {2}{3}" -f (Get-ClassMarker $_._class), (Get-Prop $_ 'severity' '?').ToUpper(), (Get-Prop $_ 'title' '?'), $ptag) + $loc = Get-Prop $_ 'location' '' + if ($loc) { Emit " where : $loc" } + $det = Get-Prop $_ 'detail' '' + if ($det) { Emit " detail: $det" } + if ($_._class -eq 'mutated') { Emit " note : content changed since this was pinned reviewed-benign - re-review before re-pinning." } + } + } } } } } -# --- suppression summary ---------------------------------------------------- -if (($script:sup + $script:mut + $script:exp) -gt 0) { +# --- suppression / policy summary ------------------------------------------- +if (($script:sup + $script:mut + $script:exp + $script:rev) -gt 0) { Emit "" Emit "baseline: $($script:sup) suppressed, $($script:mut) mutated, $($script:exp) expired - honey.baseline.json. Review: honey-baseline.ps1 status" + if ($script:rev -gt 0) { + $ft = Get-HoneySetting 'HONEY_VERDICT_FLOOR' 'none' + $ftt = Get-HoneySetting 'HONEY_VERDICT_FLOOR_TRUSTED' $ft + Emit "policy: $($script:rev) finding(s) held below the severity floor (review tier, non-blocking). Floor: $ft / trusted $ftt." + } if ($script:mut -gt 0) { Emit "[!!] $($script:mut) MUTATED: a file pinned as reviewed-benign has CHANGED. Treat as a possible rug pull and re-review." } } # --- overall verdict -------------------------------------------------------- $suffix = "" -if (($script:sup + $script:mut + $script:exp) -gt 0) { - $suffix = " ($($script:sup) suppressed" - if ($script:mut -gt 0) { $suffix += ", $($script:mut) mutated" } - if ($script:exp -gt 0) { $suffix += ", $($script:exp) expired" } - $suffix += ")" +if (($script:sup + $script:mut + $script:exp + $script:rev) -gt 0) { + $parts = @() + if ($script:sup -gt 0) { $parts += "$($script:sup) suppressed" } + if ($script:rev -gt 0) { $parts += "$($script:rev) review" } + if ($script:mut -gt 0) { $parts += "$($script:mut) mutated" } + if ($script:exp -gt 0) { $parts += "$($script:exp) expired" } + $suffix = " (" + ($parts -join ', ') + ")" } Emit "" From 61fab25fffa30ca4f925f381f801c8f4794fbecd Mon Sep 17 00:00:00 2001 From: eric-sabe Date: Tue, 7 Jul 2026 19:17:33 -0400 Subject: [PATCH 2/4] =?UTF-8?q?feat(lens):=20smuggle=20=E2=80=94=20invisib?= =?UTF-8?q?le-Unicode/bidi=20+=20remote-include=20detection?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A honey-native lens targeting the scanner-evasion classes static pattern scanners (incl. skillspector's frozen patterns) miss, over agent-skill and instruction files: • invisible Unicode — tag chars U+E0000-E007F (ASCII smuggling): no legit text use, carry hidden instructions past humans and byte scanners. • bidirectional overrides — U+202A-202E / U+2066-2069 (Trojan Source): reorder how text renders vs. is interpreted. • zero-width — U+200B/200C/2060/180E (excludes the emoji ZWJ U+200D to avoid false positives). • remote includes — instructions telling the agent to fetch/read a remote URL at runtime (content the on-disk scan never sees). Contract-compliant lens: bash uses perl (self-skips with a "skipped" verdict if perl is absent — honey's core path unaffected); the Windows mirror uses native .NET Rune enumeration (no perl). Verified behavior-identical on fixtures (tag/bidi/zero-width/remote all caught; a clean file with an emoji ZWJ and a plain URL correctly NOT flagged). Fast: ~0.17s over real skill roots. Its findings ride the existing baseline + verdict policy (first-party remote-include docs land in the review tier / can be pinned). doctor.sh + doctor.ps1 report the lens; README lens table + config updated. shellcheck clean; PSScriptAnalyzer clean. Stacked on feat/verdict-policy. Co-Authored-By: Claude Opus 4.8 (1M context) --- README.md | 12 +++-- doctor.sh | 8 +++ lenses/smuggle.sh | 107 +++++++++++++++++++++++++++++++++++++++++ win/doctor.ps1 | 2 + win/lenses/smuggle.ps1 | 82 +++++++++++++++++++++++++++++++ 5 files changed, 207 insertions(+), 4 deletions(-) create mode 100755 lenses/smuggle.sh create mode 100644 win/lenses/smuggle.ps1 diff --git a/README.md b/README.md index 510288e..16372a3 100644 --- a/README.md +++ b/README.md @@ -110,14 +110,17 @@ the next run. `./doctor.sh` shows which lenses are active. | Lens | Tool | Covers | Install | |---|---|---|---| | `skillspector` | [NVIDIA SkillSpector](https://github.com/NVIDIA/skillspector) | AI agent skills (`SKILL.md` + scripts): prompt injection, data exfil, excessive agency, tool poisoning | per its README (Python 3.12+) | +| `smuggle` | honey-native | **Scanner-evasion** in agent-skill/instruction files: invisible-Unicode (tag chars), bidi overrides (Trojan Source), zero-width chars, and remote-include instructions — the tricks static pattern scanners miss | none (uses `perl`; PowerShell uses native .NET) | | `osv-scanner` | [Google/OSV osv-scanner](https://github.com/google/osv-scanner) | Known vulns in lockfiles/manifests across **all** ecosystems (npm, pypi, cargo, go, …) | `go install github.com/google/osv-scanner/cmd/osv-scanner@latest` | | `govulncheck` | [Go vuln team govulncheck](https://golang.org/x/vuln) | Go vulns that your code **actually calls** (reachability-aware → far less noise) | `go install golang.org/x/vuln/cmd/govulncheck@latest` | **What each answers.** bumblebee: *"do I have a known-compromised package?"* (exact catalog match). skillspector: *"is this agent skill behaving -maliciously?"* (content analysis). osv-scanner: *"do my dependencies have known -CVEs?"* (broad). govulncheck: *"do I actually reach a vulnerable Go function?"* -(deep, low-noise). Together they cover reactive, proactive, breadth, and depth. +maliciously?"* (content analysis). smuggle: *"is something hidden from the +scanners — invisible Unicode, a bidi trick, a remote include?"* (evasion). +osv-scanner: *"do my dependencies have known CVEs?"* (broad). govulncheck: *"do I +actually reach a vulnerable Go function?"* (deep, low-noise). Together they cover +reactive, proactive, breadth, depth, and anti-evasion. **Where the vuln lenses scan.** osv-scanner and govulncheck scan your *project* directories, not all of `$HOME`. Set `HONEY_PROJECT_ROOTS` (colon-separated; @@ -366,7 +369,8 @@ one-off run. You can also hand-edit `honey.conf` — one `export VAR=…` per li | `BUMBLEBEE_SCAN_ROOT` | `$HOME` | what bumblebee scans | | `BUMBLEBEE_MAX_DURATION` | `30m` | bumblebee scan time cap | | `HONEY_PROJECT_ROOTS` | `~/git:~/code:~/Developer:~/src` | where the vuln lenses (osv-scanner, govulncheck) look for projects | -| `HONEY_SKILL_ROOTS` | skill dirs under `~/.claude` | where the skillspector lens looks for agent skills | +| `HONEY_SKILL_ROOTS` | skill dirs under `~/.claude` | where the skillspector and smuggle lenses look for agent skills | +| `HONEY_SMUGGLE_EXCLUDE` | `node_modules`/`.git`/`vendor`/`.pnpm` | smuggle lens: skip files whose path matches (ERE) | | `HONEY_UPDATE_LENSES` | `1` | `go install @latest` the Go lens binaries (osv-scanner, govulncheck) before scanning; `0` to skip | | `HONEY_OSV_OFFLINE` | `0` | osv-scanner: use local vuln DBs instead of OSV.dev | | `HONEY_OSV_INCLUDE_GO_STDLIB` | `0` | osv-scanner: keep Go stdlib advisories (default defers them to govulncheck) | diff --git a/doctor.sh b/doctor.sh index 4c3a025..41b0028 100755 --- a/doctor.sh +++ b/doctor.sh @@ -53,6 +53,14 @@ if [ -d "$HONEY/lenses" ] && ls "$HONEY"/lenses/*.sh >/dev/null 2>&1; then bad "lens $lname inactive — govulncheck not installed (optional)" hint "Go reachability-aware vuln scanning; install: go install golang.org/x/vuln/cmd/govulncheck@latest" fi ;; + smuggle) + # honey-native lens (no external tool); needs perl for Unicode scanning. + if command -v perl >/dev/null 2>&1; then + ok "lens $lname active (native; perl $(perl -e 'print substr($^V,1)' 2>/dev/null))" + else + bad "lens $lname inactive — perl not found (optional)" + hint "invisible-Unicode/bidi + remote-include detection; install perl to enable (ships on macOS + most Linux)" + fi ;; *) if command -v "$lname" >/dev/null 2>&1; then ok "lens $lname active"; else bad "lens $lname inactive (optional tool '$lname' not installed)"; fi ;; esac diff --git a/lenses/smuggle.sh b/lenses/smuggle.sh new file mode 100755 index 0000000..393b12b --- /dev/null +++ b/lenses/smuggle.sh @@ -0,0 +1,107 @@ +#!/usr/bin/env bash +# +# honey lens: smuggle — detect scanner-evasion smuggling in agent-skill and +# instruction files that static pattern scanners (incl. skillspector) miss: +# +# • invisible Unicode — tag characters U+E0000–E007F carry ASCII instructions +# imperceptibly ("ASCII smuggling"); no legit text use. +# • bidirectional overrides — U+202A–202E / U+2066–2069, the "Trojan Source" +# trick that reorders how source/text renders vs. runs. +# • zero-width — U+200B/200C/2060 hidden joiners/spaces (excludes the +# emoji ZWJ U+200D to avoid false positives). +# • remote includes — instructions telling the agent to fetch/read a remote +# URL at runtime (content the on-disk scan never sees). +# +# Honey lens contract: +# - invoked as: lenses/smuggle.sh +# - writes /lens-smuggle.json in honey's normalized shape +# { lens, tool_version, status, findings_total, findings_by_severity, +# findings: [ {severity,title,location,detail,ref} ], note } +# status ∈ clean | exposed | scan_error | skipped +# - self-skips (exit 0 + "skipped") if perl isn't available — honey's core path +# is unaffected for anyone without it (perl ships on macOS + most Linux). + +set -uo pipefail +RUN_DIR="${1:?usage: smuggle.sh }" +OUT="$RUN_DIR/lens-smuggle.json" +HONEY="${HONEY:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}" +# shellcheck source=lib/load-config.sh +[ -f "$HONEY/lib/load-config.sh" ] && . "$HONEY/lib/load-config.sh" + +emit() { # emit STATUS TOTAL BY_SEV_JSON FINDINGS_JSON NOTE + jq -n --arg lens "smuggle" --arg tv "${SMUGGLE_VERSION:-1}" \ + --arg status "$1" --argjson total "$2" --argjson by_sev "$3" \ + --argjson findings "$4" --arg note "$5" \ + '{lens:$lens, tool_version:$tv, status:$status, findings_total:$total, + findings_by_severity:$by_sev, findings:$findings, note:$note}' >"$OUT" +} + +if ! command -v perl >/dev/null 2>&1; then + emit "skipped" 0 '{}' '[]' "perl not found — smuggle lens skipped (install perl to enable Unicode-smuggling detection)." + echo "lens smuggle: perl not installed, skipped" + exit 0 +fi + +# Same surface as skillspector: agent-skill roots (override via HONEY_SKILL_ROOTS) +# plus any MCP manifests. Text-ish files only; skip the run/vendor noise. +DEFAULT_ROOTS="$HOME/.claude/skills:$HOME/.claude/plugins:$HOME/.claude/scheduled-tasks:$HOME/.config/claude/skills" +ROOTS="${HONEY_SKILL_ROOTS:-$DEFAULT_ROOTS}" +EXCLUDE_RE="${HONEY_SMUGGLE_EXCLUDE:-/node_modules/|/\.git/|/vendor/|/\.pnpm/}" + +# Collect candidate files (portable; NUL-delimited to survive odd names). +files_nul="$RUN_DIR/.smuggle-files" +: >"$files_nul" +IFS=':' read -r -a roots <<<"$ROOTS" +for root in "${roots[@]}"; do + [ -d "$root" ] || continue + find "$root" -type f \( -name '*.md' -o -name '*.markdown' -o -name '*.txt' \ + -o -name '*.json' -o -name '*.mcp.json' -o -name '*.yaml' -o -name '*.yml' \ + -o -name '*.sh' -o -name '*.py' -o -name '*.js' -o -name '*.ts' \ + -o -name '*.html' -o -name 'SKILL.md' \) -print0 2>/dev/null +done | grep -zvE "$EXCLUDE_RE" >>"$files_nul" 2>/dev/null || true + +if [ ! -s "$files_nul" ]; then + emit "clean" 0 '{}' '[]' "no agent-skill/instruction files found under: $ROOTS" + echo "lens smuggle: no files to scan" + rm -f "$files_nul" + exit 0 +fi + +# --- Scan (perl): emit TSV severitytitlefilelinedetail ---- +# -CSD: treat args/stdin/stdout as UTF-8. `close ARGV if eof` resets $. per file. +# shellcheck disable=SC2016 # this is a perl program; $vars/$. are perl, not shell +tsv="$(xargs -0 perl -CSD -ne ' + BEGIN { $re_tag = qr/[\x{E0000}-\x{E007F}]/; + $re_bidi = qr/[\x{202A}-\x{202E}\x{2066}-\x{2069}]/; + $re_zw = qr/[\x{200B}\x{200C}\x{2060}\x{180E}]/; + $re_url = qr/(?:curl|wget|fetch|download|retrieve|read|load|import|include|open)\b[^\n]{0,60}https?:\/\//i; } + if (/($re_tag)/) { printf "high\tInvisible Unicode (tag chars)\t%s\t%d\tASCII-smuggling codepoint U+%04X — carries hidden instructions invisible to humans and byte scanners\n", $ARGV, $., ord($1); } + if (/($re_bidi)/) { printf "high\tBidirectional override (Trojan Source)\t%s\t%d\tbidi control U+%04X can reorder how text renders vs. is interpreted\n", $ARGV, $., ord($1); } + if (/($re_zw)/) { printf "medium\tZero-width character\t%s\t%d\thidden zero-width codepoint U+%04X (possible token smuggling)\n", $ARGV, $., ord($1); } + if (/$re_url/) { printf "medium\tRemote include instruction\t%s\t%d\tinstructs the agent to fetch/read a remote URL — content the on-disk scan never sees\n", $ARGV, $.; } + close ARGV if eof; + ' <"$files_nul")" + +rm -f "$files_nul" + +# Build normalized findings JSON from the TSV. +findings='[]' +if [ -n "$tsv" ]; then + findings="$(printf '%s\n' "$tsv" | jq -Rc 'select(length>0) | split("\t") + | {severity:.[0], title:.[1], location:(.[2] + ":" + .[3]), detail:.[4], ref:"scanner-evasion"}' \ + | jq -sc '.')" +fi +[ -n "$findings" ] || findings='[]' + +TOTAL="$(printf '%s' "$findings" | jq 'length')" +BY_SEV="$(printf '%s' "$findings" | jq 'group_by(.severity) | map({key:(.[0].severity), value:length}) | from_entries')" +[ -z "$BY_SEV" ] && BY_SEV='{}' +NOTE="scanned agent-skill/instruction files under $ROOTS for invisible/bidi/zero-width Unicode + remote-include instructions" + +if [ "$TOTAL" -gt 0 ]; then + emit "exposed" "$TOTAL" "$BY_SEV" "$findings" "$NOTE" +else + emit "clean" 0 '{}' '[]' "$NOTE" +fi +echo "lens smuggle: status written to $OUT ($TOTAL finding(s))" +exit 0 diff --git a/win/doctor.ps1 b/win/doctor.ps1 index b443979..d048b0a 100644 --- a/win/doctor.ps1 +++ b/win/doctor.ps1 @@ -75,6 +75,8 @@ foreach ($t in $lensTools.Keys) { if (Get-Command $t -ErrorAction SilentlyContinue) { Ok "lens $t active" } else { Write-HoneyConsole " [ ] lens $t inactive (optional)"; Hint $lensTools[$t] } } +# honey-native lens: no external tool, uses .NET Unicode enumeration — always on. +Ok "lens smuggle active (native; invisible-Unicode/bidi + remote-include detection)" # Suppression baseline (informational; never affects pass/fail). Write-HoneyConsole "" diff --git a/win/lenses/smuggle.ps1 b/win/lenses/smuggle.ps1 new file mode 100644 index 0000000..05ee78a --- /dev/null +++ b/win/lenses/smuggle.ps1 @@ -0,0 +1,82 @@ +<# +.SYNOPSIS + honey (Windows) lens: smuggle — detect scanner-evasion smuggling in agent-skill + and instruction files. PowerShell mirror of lenses/smuggle.sh (uses native .NET + Unicode enumeration, so it needs no perl). + +.DESCRIPTION + Flags what static pattern scanners miss: + • invisible Unicode — tag chars U+E0000–E007F (ASCII smuggling) + • bidirectional overrides — U+202A–202E / U+2066–2069 (Trojan Source) + • zero-width — U+200B/200C/2060/180E (excludes emoji ZWJ U+200D) + • remote includes — instructions to fetch/read a remote URL at runtime + +.PARAMETER RunDir + Run directory to write lens-smuggle.json into. +#> +param([string]$RunDir) + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' +Import-Module (Join-Path $PSScriptRoot '..' 'lib' 'Honey.psm1') -Force +Import-HoneyConfig + +if (-not $RunDir) { Write-Error 'usage: smuggle.ps1 -RunDir '; exit 2 } + +$home_ = Get-HoneyHome +$defaultRoots = @( + (Join-Path $home_ '.claude\skills'), (Join-Path $home_ '.claude\plugins'), + (Join-Path $home_ '.claude\scheduled-tasks'), (Join-Path $home_ '.config\claude\skills') +) -join ';' +$roots = (Get-HoneySetting 'HONEY_SKILL_ROOTS' $defaultRoots) -split ';' | Where-Object { $_ } +$excludeRe = Get-HoneySettingRaw 'HONEY_SMUGGLE_EXCLUDE' '[\\/](node_modules|\.git|vendor|\.pnpm)[\\/]' +$exts = '.md','.markdown','.txt','.json','.yaml','.yml','.sh','.py','.js','.ts','.html' +$urlRe = [regex]'(?i)(curl|wget|fetch|download|retrieve|read|load|import|include|open)\b[^\n]{0,60}https?://' + +$findings = New-Object System.Collections.ArrayList +function Add-Finding { param($Sev,$Title,$File,$Line,$Detail) + [void]$findings.Add([pscustomobject]@{ severity=$Sev; title=$Title; location="${File}:${Line}"; detail=$Detail; ref='scanner-evasion' }) +} + +$scanned = 0 +foreach ($root in $roots) { + if (-not (Test-Path -LiteralPath $root)) { continue } + Get-ChildItem -LiteralPath $root -Recurse -File -ErrorAction SilentlyContinue | + Where-Object { $exts -contains $_.Extension.ToLower() -and $_.FullName -notmatch $excludeRe } | + ForEach-Object { + $file = $_.FullName + $n = 0 + foreach ($line in [System.IO.File]::ReadLines($file)) { + $n++ + foreach ($rune in $line.EnumerateRunes()) { + $cp = $rune.Value + if ($cp -ge 0xE0000 -and $cp -le 0xE007F) { + Add-Finding 'high' 'Invisible Unicode (tag chars)' $file $n ("ASCII-smuggling codepoint U+{0:X4} — carries hidden instructions invisible to humans and byte scanners" -f $cp); break + } + } + foreach ($rune in $line.EnumerateRunes()) { + $cp = $rune.Value + if (($cp -ge 0x202A -and $cp -le 0x202E) -or ($cp -ge 0x2066 -and $cp -le 0x2069)) { + Add-Finding 'high' 'Bidirectional override (Trojan Source)' $file $n ("bidi control U+{0:X4} can reorder how text renders vs. is interpreted" -f $cp); break + } + } + foreach ($rune in $line.EnumerateRunes()) { + $cp = $rune.Value + if ($cp -eq 0x200B -or $cp -eq 0x200C -or $cp -eq 0x2060 -or $cp -eq 0x180E) { + Add-Finding 'medium' 'Zero-width character' $file $n ("hidden zero-width codepoint U+{0:X4} (possible token smuggling)" -f $cp); break + } + } + if ($urlRe.IsMatch($line)) { + Add-Finding 'medium' 'Remote include instruction' $file $n 'instructs the agent to fetch/read a remote URL - content the on-disk scan never sees' + } + } + $scanned++ + } +} + +$note = "scanned $scanned agent-skill/instruction file(s) under $($roots -join ';') for invisible/bidi/zero-width Unicode + remote-include instructions" +if ($scanned -eq 0) { $note = "no agent-skill/instruction files found under: $($roots -join ';')" } +$status = if (@($findings).Count -gt 0) { 'exposed' } else { 'clean' } +Write-LensResult -RunDir $RunDir -Lens 'smuggle' -ToolVersion '1' -Status $status -Findings @($findings) -Note $note | Out-Null +Write-HoneyConsole "lens smuggle: status written ($(@($findings).Count) finding(s))" +exit 0 From 429dcbafa1de5d8393594a4b4df73ef0fe9d51e3 Mon Sep 17 00:00:00 2001 From: eric-sabe Date: Tue, 7 Jul 2026 19:23:44 -0400 Subject: [PATCH 3/4] =?UTF-8?q?feat(lens):=20mcp=20=E2=80=94=20MCP=20manif?= =?UTF-8?q?est=20inventory=20+=20rug-pull=20(drift)=20detection?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes honey's biggest coverage gap: an MCP server defined by a .mcp.json or host config with no SKILL.md is invisible to skillspector. This native lens inventories every MCP server (host configs: Claude/Cursor/VS Code/Windsurf, plus every .mcp.json under the project roots) and implements the canonical rug-pull defense — "hash the manifest on first sight, diff every subsequent run": • MCP-DRIFT (high) — a known server's definition CHANGED since last run (the manifest-alteration rug-pull vector). • MCP-NEW (low) — a server appeared that wasn't there before. • MCP-RISKY (medium) — launch command fetches-and-executes remote code (curl|wget | sh/bash, bash -c, eval). The first run SEEDS the baseline manifest silently (no drift/new spam on a fresh box); detection starts the next run. Hashes are stored in a gitignored .mcp-state.json (HONEY_MCP_STATE). Content-based RISKY fires every run. On a real machine this inventories 23 MCP servers across host configs that honey could not see before. Findings ride the baseline + verdict policy like any lens. Offline: bash uses jq (deep-sorted canonical via `jq walk`); the Windows mirror uses native .NET (recursive key sort). Verified behavior-identical: seed → unchanged → drift + new + risky. shellcheck clean; PSScriptAnalyzer clean. doctor/doctor.ps1 report the lens; README lens table + config + "what each answers" updated; .gitignore excludes the state file. Stacked on feat/lens-hardening. Co-Authored-By: Claude Opus 4.8 (1M context) --- .gitignore | 4 ++ README.md | 12 ++-- doctor.sh | 3 + lenses/mcp.sh | 140 ++++++++++++++++++++++++++++++++++++++++++++ win/doctor.ps1 | 3 +- win/lenses/mcp.ps1 | 141 +++++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 298 insertions(+), 5 deletions(-) create mode 100755 lenses/mcp.sh create mode 100644 win/lenses/mcp.ps1 diff --git a/.gitignore b/.gitignore index 718db26..89d9f4e 100644 --- a/.gitignore +++ b/.gitignore @@ -14,6 +14,10 @@ honey.conf.ps1 # committed and reviewed in git — that auditability is an anti-gaming control # (see docs/BASELINE.plan.md). Do not add it here. +# MCP-manifest lens state — auto-maintained hashes of MCP server definitions for +# rug-pull (drift) detection. Machine-specific; contains local config hashes. +.mcp-state.json + # Local secrets, if you ever add any (e.g. Slack webhook/token). secrets.env *.env diff --git a/README.md b/README.md index 16372a3..488f9d9 100644 --- a/README.md +++ b/README.md @@ -111,16 +111,18 @@ the next run. `./doctor.sh` shows which lenses are active. |---|---|---|---| | `skillspector` | [NVIDIA SkillSpector](https://github.com/NVIDIA/skillspector) | AI agent skills (`SKILL.md` + scripts): prompt injection, data exfil, excessive agency, tool poisoning | per its README (Python 3.12+) | | `smuggle` | honey-native | **Scanner-evasion** in agent-skill/instruction files: invisible-Unicode (tag chars), bidi overrides (Trojan Source), zero-width chars, and remote-include instructions — the tricks static pattern scanners miss | none (uses `perl`; PowerShell uses native .NET) | +| `mcp` | honey-native | **MCP servers** (`.mcp.json` + host configs — invisible to skillspector): inventories every server and **hash-diffs its definition across runs to catch rug pulls** (a server that changes after you approve it); flags fetch-and-exec launch commands | none (jq / native .NET) | | `osv-scanner` | [Google/OSV osv-scanner](https://github.com/google/osv-scanner) | Known vulns in lockfiles/manifests across **all** ecosystems (npm, pypi, cargo, go, …) | `go install github.com/google/osv-scanner/cmd/osv-scanner@latest` | | `govulncheck` | [Go vuln team govulncheck](https://golang.org/x/vuln) | Go vulns that your code **actually calls** (reachability-aware → far less noise) | `go install golang.org/x/vuln/cmd/govulncheck@latest` | **What each answers.** bumblebee: *"do I have a known-compromised package?"* (exact catalog match). skillspector: *"is this agent skill behaving maliciously?"* (content analysis). smuggle: *"is something hidden from the -scanners — invisible Unicode, a bidi trick, a remote include?"* (evasion). -osv-scanner: *"do my dependencies have known CVEs?"* (broad). govulncheck: *"do I -actually reach a vulnerable Go function?"* (deep, low-noise). Together they cover -reactive, proactive, breadth, depth, and anti-evasion. +scanners — invisible Unicode, a bidi trick, a remote include?"* (evasion). mcp: +*"did an MCP server's definition change since I approved it?"* (rug-pull +diffing). osv-scanner: *"do my dependencies have known CVEs?"* (broad). +govulncheck: *"do I actually reach a vulnerable Go function?"* (deep, low-noise). +Together they cover reactive, proactive, breadth, depth, anti-evasion, and drift. **Where the vuln lenses scan.** osv-scanner and govulncheck scan your *project* directories, not all of `$HOME`. Set `HONEY_PROJECT_ROOTS` (colon-separated; @@ -371,6 +373,8 @@ one-off run. You can also hand-edit `honey.conf` — one `export VAR=…` per li | `HONEY_PROJECT_ROOTS` | `~/git:~/code:~/Developer:~/src` | where the vuln lenses (osv-scanner, govulncheck) look for projects | | `HONEY_SKILL_ROOTS` | skill dirs under `~/.claude` | where the skillspector and smuggle lenses look for agent skills | | `HONEY_SMUGGLE_EXCLUDE` | `node_modules`/`.git`/`vendor`/`.pnpm` | smuggle lens: skip files whose path matches (ERE) | +| `HONEY_MCP_CONFIGS` | Claude/Cursor/VS Code/Windsurf host configs | mcp lens: colon-separated MCP config files to inventory (plus every `.mcp.json` under the project roots) | +| `HONEY_MCP_STATE` | `/.mcp-state.json` | mcp lens: where the per-run manifest hashes are stored for rug-pull diffing (gitignored) | | `HONEY_UPDATE_LENSES` | `1` | `go install @latest` the Go lens binaries (osv-scanner, govulncheck) before scanning; `0` to skip | | `HONEY_OSV_OFFLINE` | `0` | osv-scanner: use local vuln DBs instead of OSV.dev | | `HONEY_OSV_INCLUDE_GO_STDLIB` | `0` | osv-scanner: keep Go stdlib advisories (default defers them to govulncheck) | diff --git a/doctor.sh b/doctor.sh index 41b0028..952f811 100755 --- a/doctor.sh +++ b/doctor.sh @@ -61,6 +61,9 @@ if [ -d "$HONEY/lenses" ] && ls "$HONEY"/lenses/*.sh >/dev/null 2>&1; then bad "lens $lname inactive — perl not found (optional)" hint "invisible-Unicode/bidi + remote-include detection; install perl to enable (ships on macOS + most Linux)" fi ;; + mcp) + # honey-native lens (jq only) — MCP manifest inventory + rug-pull diffing. + ok "lens $lname active (native; MCP manifest hash-and-diff, jq only)" ;; *) if command -v "$lname" >/dev/null 2>&1; then ok "lens $lname active"; else bad "lens $lname inactive (optional tool '$lname' not installed)"; fi ;; esac diff --git a/lenses/mcp.sh b/lenses/mcp.sh new file mode 100755 index 0000000..a864f1e --- /dev/null +++ b/lenses/mcp.sh @@ -0,0 +1,140 @@ +#!/usr/bin/env bash +# +# honey lens: mcp — inventory MCP server manifests and detect RUG PULLS by +# hashing each server's definition and diffing it across runs. +# +# This closes honey's biggest blind spot: an MCP server defined by a `.mcp.json` +# / host config with no SKILL.md is invisible to the skillspector lens. The +# canonical defense against a manifest-alteration rug pull (a server that earns +# approval with a benign definition, then silently changes) is exactly this: +# "hash the tool manifest on first sight and diff every subsequent run." +# (Invariant Labs / policylayer.) +# +# Findings: +# • MCP-DRIFT (high) — a known server's definition CHANGED since last run. +# • MCP-NEW (low) — a server appeared that wasn't here before. +# • MCP-RISKY (medium) — the launch command fetches-and-executes remote code +# (curl|wget … | sh/bash, `bash -c`, `eval`). +# +# The first run SEEDS the state silently (everything is "new" on a fresh box, so +# NEW/DRIFT are only reported once a baseline manifest exists). RISKY is +# content-based and reported every run. +# +# Honey lens contract: writes /lens-mcp.json (normalized shape); +# self-skips only if there are no MCP configs to look at. Offline; jq only. + +set -uo pipefail +RUN_DIR="${1:?usage: mcp.sh }" +OUT="$RUN_DIR/lens-mcp.json" +HONEY="${HONEY:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}" +# shellcheck source=lib/load-config.sh +[ -f "$HONEY/lib/load-config.sh" ] && . "$HONEY/lib/load-config.sh" + +STATE="${HONEY_MCP_STATE:-$HONEY/.mcp-state.json}" + +emit() { # STATUS TOTAL BY_SEV FINDINGS NOTE + jq -n --arg lens "mcp" --arg tv "1" --arg status "$1" --argjson total "$2" \ + --argjson by_sev "$3" --argjson findings "$4" --arg note "$5" \ + '{lens:$lens, tool_version:$tv, status:$status, findings_total:$total, + findings_by_severity:$by_sev, findings:$findings, note:$note}' >"$OUT" +} + +hash_stdin() { + if command -v sha256sum >/dev/null 2>&1; then sha256sum | awk '{print $1}' + elif command -v shasum >/dev/null 2>&1; then shasum -a 256 | awk '{print $1}' + else openssl dgst -sha256 2>/dev/null | awk '{print $NF}'; fi +} + +# --- Discover MCP config files --------------------------------------------- +# Host configs (override via HONEY_MCP_CONFIGS, colon-separated) + every +# .mcp.json / mcp.json under the project roots. +DEFAULT_CONFIGS="$HOME/.claude.json:$HOME/Library/Application Support/Claude/claude_desktop_config.json:$HOME/.config/claude/claude_desktop_config.json:$HOME/.cursor/mcp.json:$HOME/.vscode/mcp.json:$HOME/.codeium/windsurf/mcp_config.json" +CONFIGS="${HONEY_MCP_CONFIGS:-$DEFAULT_CONFIGS}" +PROJECT_ROOTS="${HONEY_PROJECT_ROOTS:-$HOME/git:$HOME/code:$HOME/Developer:$HOME/src}" + +configs_file="$RUN_DIR/.mcp-configs"; : >"$configs_file" +IFS=':' read -r -a carr <<<"$CONFIGS" +for c in "${carr[@]}"; do [ -f "$c" ] && printf '%s\n' "$c" >>"$configs_file"; done +IFS=':' read -r -a proots <<<"$PROJECT_ROOTS" +for root in "${proots[@]}"; do + [ -d "$root" ] || continue + find "$root" -maxdepth 6 \( -name '.mcp.json' -o -name 'mcp.json' \) -type f 2>/dev/null \ + | grep -vE '/node_modules/|/\.git/|/\.claude/worktrees/' >>"$configs_file" 2>/dev/null || true +done +sort -u "$configs_file" -o "$configs_file" + +if [ ! -s "$configs_file" ]; then + emit "skipped" 0 '{}' '[]' "no MCP configs found (looked at host configs + .mcp.json under $PROJECT_ROOTS)" + echo "lens mcp: no MCP configs, skipped"; rm -f "$configs_file"; exit 0 +fi + +# --- Extract servers: cfg name base64(canonical def) ------------ +extract="$RUN_DIR/.mcp-servers"; : >"$extract" +while IFS= read -r cfg; do + [ -n "$cfg" ] || continue + jq -r --arg cfg "$cfg" ' + def norm: walk(if type=="object" then to_entries|sort_by(.key)|from_entries else . end); + ((.mcpServers // {}) + (.servers // {}) + (((.projects // {}) | [.[]?.mcpServers // {}]) | add // {})) + | to_entries[] | [$cfg, .key, (.value | norm | tojson | @base64)] | @tsv + ' "$cfg" 2>/dev/null >>"$extract" || true +done <"$configs_file" +rm -f "$configs_file" + +SEEDING=0 +[ -f "$STATE" ] || SEEDING=1 +[ -f "$STATE" ] || echo '{"servers":{}}' >"$STATE" + +# --- Diff against state + scan for risky commands -------------------------- +findings='[]' +newstate='{"servers":{}}' +today="$(date -u +%Y-%m-%d)" +add_finding() { # severity title location detail + findings="$(printf '%s' "$findings" | jq -c --arg s "$1" --arg t "$2" --arg l "$3" --arg d "$4" \ + '. + [{severity:$s,title:$t,location:$l,detail:$d,ref:"mcp-manifest"}]')" +} + +while IFS=$'\t' read -r cfg name defb64; do + [ -n "$name" ] || continue + canonical="$(printf '%s' "$defb64" | base64 -d 2>/dev/null)" + h="sha256:$(printf '%s' "$canonical" | hash_stdin)" + key="$cfg::$name" + prev="$(jq -r --arg k "$key" '.servers[$k].hash // ""' "$STATE" 2>/dev/null)" + first="$(jq -r --arg k "$key" '.servers[$k].first_seen // ""' "$STATE" 2>/dev/null)" + [ -n "$first" ] || first="$today" + + if [ "$SEEDING" -eq 0 ]; then + if [ -z "$prev" ]; then + add_finding "low" "New MCP server: $name" "$cfg" "server '$name' appeared since the last run — confirm you added it. Definition: $canonical" + elif [ "$prev" != "$h" ]; then + add_finding "high" "MCP manifest changed (possible rug pull): $name" "$cfg" "server '$name' definition CHANGED since last run (was ${prev%%:*}:${prev:7:12}…, now ${h:7:12}…). A server that alters its manifest after approval is the canonical rug-pull vector — re-review. Now: $canonical" + fi + fi + + # Content check (every run): launch command fetches-and-executes remote code. + if printf '%s' "$canonical" | grep -qiE '(curl|wget)[^|]*\|[[:space:]]*(sh|bash)|bash[[:space:]]+-c|(^|[^a-z])sh[[:space:]]+-c|[^a-z]eval[^a-z]'; then + add_finding "medium" "MCP server runs a fetch-and-exec command: $name" "$cfg" "launch command pipes remote content into a shell or uses bash -c/eval — a code-execution vector. Definition: $canonical" + fi + + newstate="$(printf '%s' "$newstate" | jq -c --arg k "$key" --arg h "$h" --arg f "$first" --arg l "$today" \ + '.servers[$k] = {hash:$h, first_seen:$f, last_seen:$l}')" +done <"$extract" +srvcount="$(wc -l <"$extract" | tr -d ' ')" +rm -f "$extract" + +# Persist the new manifest state (current servers only — a removed server just +# drops out; it is not a finding). +printf '%s' "$newstate" | jq '.' >"$STATE" 2>/dev/null || true + +TOTAL="$(printf '%s' "$findings" | jq 'length')" +BY_SEV="$(printf '%s' "$findings" | jq 'group_by(.severity) | map({key:(.[0].severity), value:length}) | from_entries')" +[ -z "$BY_SEV" ] && BY_SEV='{}' +NOTE="inventoried $srvcount MCP server(s); manifest hash-and-diff for rug pulls (state: $STATE)" +[ "$SEEDING" -eq 1 ] && NOTE="$NOTE; FIRST RUN seeded the baseline manifest (drift/new detection starts next run)" + +if [ "$TOTAL" -gt 0 ]; then + emit "exposed" "$TOTAL" "$BY_SEV" "$findings" "$NOTE" +else + emit "clean" 0 '{}' '[]' "$NOTE" +fi +echo "lens mcp: status written to $OUT ($TOTAL finding(s), $srvcount server(s), seeding=$SEEDING)" +exit 0 diff --git a/win/doctor.ps1 b/win/doctor.ps1 index d048b0a..4c06d43 100644 --- a/win/doctor.ps1 +++ b/win/doctor.ps1 @@ -75,8 +75,9 @@ foreach ($t in $lensTools.Keys) { if (Get-Command $t -ErrorAction SilentlyContinue) { Ok "lens $t active" } else { Write-HoneyConsole " [ ] lens $t inactive (optional)"; Hint $lensTools[$t] } } -# honey-native lens: no external tool, uses .NET Unicode enumeration — always on. +# honey-native lenses: no external tool — always on. Ok "lens smuggle active (native; invisible-Unicode/bidi + remote-include detection)" +Ok "lens mcp active (native; MCP manifest hash-and-diff for rug pulls)" # Suppression baseline (informational; never affects pass/fail). Write-HoneyConsole "" diff --git a/win/lenses/mcp.ps1 b/win/lenses/mcp.ps1 new file mode 100644 index 0000000..5beee7b --- /dev/null +++ b/win/lenses/mcp.ps1 @@ -0,0 +1,141 @@ +<# +.SYNOPSIS + honey (Windows) lens: mcp — inventory MCP server manifests and detect RUG PULLS + by hashing each server's definition and diffing across runs. PowerShell mirror + of lenses/mcp.sh (offline; no external tool). + +.DESCRIPTION + Closes the MCP blind spot (servers with no SKILL.md). Findings: + • MCP-DRIFT (high) — a known server's definition CHANGED since last run. + • MCP-NEW (low) — a server appeared that wasn't here before. + • MCP-RISKY (medium) — launch command fetches-and-executes remote code. + First run seeds the state silently; drift/new detection starts next run. + +.PARAMETER RunDir + Run directory to write lens-mcp.json into. +#> +param([string]$RunDir) + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' +Import-Module (Join-Path $PSScriptRoot '..' 'lib' 'Honey.psm1') -Force +Import-HoneyConfig + +if (-not $RunDir) { Write-Error 'usage: mcp.ps1 -RunDir '; exit 2 } + +$home_ = Get-HoneyHome +$honeyRoot = Get-HoneyRoot +$state = Get-HoneySetting 'HONEY_MCP_STATE' (Join-Path $honeyRoot '.mcp-state.json') +$today = (Get-Date).ToUniversalTime().ToString('yyyy-MM-dd') +$riskyRe = [regex]'(?i)(curl|wget)[^|]*\|\s*(sh|bash)|bash\s+-c|(^|[^a-z])sh\s+-c|[^a-z]eval[^a-z]' + +function Get-Prop { param($o,$n) if ($o -and $o.PSObject.Properties.Name -contains $n) { $o.$n } else { $null } } + +# Deep-sort object keys so the canonical form (and its hash) is stable across a +# benign key reorder — mirrors the bash `jq walk` normalization. +function ConvertTo-SortedNode { + param($o) + if ($null -eq $o) { return $o } + if ($o -is [System.Management.Automation.PSCustomObject]) { + $ord = [ordered]@{} + $o.PSObject.Properties | Sort-Object Name | ForEach-Object { $ord[$_.Name] = (ConvertTo-SortedNode $_.Value) } + return [pscustomobject]$ord + } + if ($o -is [System.Collections.IEnumerable] -and $o -isnot [string]) { + return @($o | ForEach-Object { ConvertTo-SortedNode $_ }) + } + return $o +} +function Get-CanonHash { param($def) + $canon = (ConvertTo-SortedNode $def | ConvertTo-Json -Depth 20 -Compress) + $sha = [System.Security.Cryptography.SHA256]::Create() + try { return @{ canon = $canon; hash = 'sha256:' + (($sha.ComputeHash([Text.Encoding]::UTF8.GetBytes($canon)) | ForEach-Object { $_.ToString('x2') }) -join '') } } + finally { $sha.Dispose() } +} + +# --- discover configs ------------------------------------------------------- +$defaultConfigs = @( + (Join-Path $home_ '.claude.json'), + (Join-Path $home_ 'AppData\Roaming\Claude\claude_desktop_config.json'), + (Join-Path $home_ 'Library/Application Support/Claude/claude_desktop_config.json'), + (Join-Path $home_ '.cursor\mcp.json'), (Join-Path $home_ '.vscode\mcp.json'), + (Join-Path $home_ '.codeium\windsurf\mcp_config.json') +) -join ';' +$configs = New-Object System.Collections.ArrayList +foreach ($c in ((Get-HoneySetting 'HONEY_MCP_CONFIGS' $defaultConfigs) -split ';' | Where-Object { $_ })) { + if (Test-Path -LiteralPath $c -PathType Leaf) { [void]$configs.Add($c) } +} +$projectRoots = (Get-HoneySetting 'HONEY_PROJECT_ROOTS' ((@( + (Join-Path $home_ 'source\repos'), (Join-Path $home_ 'git'), (Join-Path $home_ 'code')) -join ';'))) -split ';' | Where-Object { $_ } +foreach ($root in $projectRoots) { + if (-not (Test-Path -LiteralPath $root)) { continue } + Get-ChildItem -LiteralPath $root -Recurse -Depth 6 -File -ErrorAction SilentlyContinue | + Where-Object { ($_.Name -eq '.mcp.json' -or $_.Name -eq 'mcp.json') -and $_.FullName -notmatch '[\\/](node_modules|\.git)[\\/]' } | + ForEach-Object { [void]$configs.Add($_.FullName) } +} +$configs = @($configs | Sort-Object -Unique) + +if ($configs.Count -eq 0) { + Write-LensResult -RunDir $RunDir -Lens 'mcp' -Status 'skipped' -Findings @() -Note "no MCP configs found (host configs + .mcp.json under project roots)" | Out-Null + Write-HoneyConsole "lens mcp: no MCP configs, skipped"; exit 0 +} + +# --- load state ------------------------------------------------------------- +$seeding = -not (Test-Path -LiteralPath $state) +$prev = @{} +if (-not $seeding) { + try { + $sj = Get-Content -LiteralPath $state -Raw | ConvertFrom-Json + $sv = Get-Prop $sj 'servers' + if ($sv) { $sv.PSObject.Properties | ForEach-Object { $prev[$_.Name] = $_.Value } } + } catch { $prev = @{} } +} + +# --- extract servers, diff, scan ------------------------------------------- +$findings = New-Object System.Collections.ArrayList +function Add-Finding { param($Sev,$Title,$Loc,$Detail) + [void]$findings.Add([pscustomobject]@{ severity=$Sev; title=$Title; location=$Loc; detail=$Detail; ref='mcp-manifest' }) +} +$newState = @{} +$srvCount = 0 + +foreach ($cfg in $configs) { + try { $doc = Get-Content -LiteralPath $cfg -Raw | ConvertFrom-Json } catch { continue } + $serverMaps = @() + $m = Get-Prop $doc 'mcpServers'; if ($m) { $serverMaps += $m } + $s = Get-Prop $doc 'servers'; if ($s) { $serverMaps += $s } + $projects = Get-Prop $doc 'projects' + if ($projects) { $projects.PSObject.Properties | ForEach-Object { $pm = Get-Prop $_.Value 'mcpServers'; if ($pm) { $serverMaps += $pm } } } + + foreach ($map in $serverMaps) { + foreach ($p in $map.PSObject.Properties) { + $name = $p.Name; $def = $p.Value + $srvCount++ + $ch = Get-CanonHash $def + $key = "$cfg::$name" + $firstSeen = if ($prev.ContainsKey($key) -and (Get-Prop $prev[$key] 'first_seen')) { [string]$prev[$key].first_seen } else { $today } + + if (-not $seeding) { + if (-not $prev.ContainsKey($key)) { + Add-Finding 'low' "New MCP server: $name" $cfg "server '$name' appeared since the last run - confirm you added it. Definition: $($ch.canon)" + } elseif ([string]$prev[$key].hash -ne $ch.hash) { + Add-Finding 'high' "MCP manifest changed (possible rug pull): $name" $cfg "server '$name' definition CHANGED since last run - a server that alters its manifest after approval is the canonical rug-pull vector. Re-review. Now: $($ch.canon)" + } + } + if ($riskyRe.IsMatch($ch.canon)) { + Add-Finding 'medium' "MCP server runs a fetch-and-exec command: $name" $cfg "launch command pipes remote content into a shell or uses bash -c/eval - a code-execution vector. Definition: $($ch.canon)" + } + $newState[$key] = [ordered]@{ hash = $ch.hash; first_seen = $firstSeen; last_seen = $today } + } + } +} + +# --- persist state ---------------------------------------------------------- +([ordered]@{ servers = $newState } | ConvertTo-Json -Depth 6) | Set-Content -LiteralPath $state -Encoding utf8 + +$note = "inventoried $srvCount MCP server(s); manifest hash-and-diff for rug pulls (state: $state)" +if ($seeding) { $note += "; FIRST RUN seeded the baseline manifest (drift/new detection starts next run)" } +$status = if (@($findings).Count -gt 0) { 'exposed' } else { 'clean' } +Write-LensResult -RunDir $RunDir -Lens 'mcp' -ToolVersion '1' -Status $status -Findings @($findings) -Note $note | Out-Null +Write-HoneyConsole "lens mcp: status written ($(@($findings).Count) finding(s), $srvCount server(s), seeding=$seeding)" +exit 0 From 37a3c4b40f5218f721e1203221620a021e09a0b7 Mon Sep 17 00:00:00 2001 From: eric-sabe Date: Tue, 7 Jul 2026 19:33:55 -0400 Subject: [PATCH 4/4] feat(lens): ocr (offline multimodal) + opt-in mcp-scan & garak wrappers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three more lenses from the research, per the "opt-in, offline by default" choice: • ocr (offline, native) — the multimodal/SkillCamo defense. OCRs images bundled in agent skills (tesseract) and flags hidden instruction/exfil text — the blind spot text scanners can't see. Self-skips if tesseract is absent. Feeds images via stdin (some leptonica builds mis-read absolute paths). Verified on fixtures: two images with hidden "ignore previous instructions" / "curl … | bash" caught; a plain diagram not flagged. • mcp-scan (opt-in, external) — wraps Invariant/Snyk Agent Scan (hybrid rules+model analysis). It phones home, so it stays inert unless HONEY_ENABLE_MCP_SCAN=1 and never runs in the default cycle. honey's native offline `mcp` lens covers the no-network path. Defensive JSON normalization. • garak (opt-in, external) — wraps NVIDIA garak (live-model probing). Doubly gated (HONEY_ENABLE_GARAK=1 + HONEY_GARAK_TARGET); it's a live-model red-team, not a file scan, so it's never part of the offline cycle. All three are contract-compliant lenses (baseline + verdict policy apply). The two external ones default OFF and were verified on their skip paths (tools not installed here). shellcheck clean; PSScriptAnalyzer clean. doctor/doctor.ps1 report each lens's tool/enable state; README lens table + config + an offline-vs-external note added. Stacked on feat/mcp-lens. Co-Authored-By: Claude Opus 4.8 (1M context) --- README.md | 15 +++++++ doctor.sh | 28 +++++++++++++ lenses/garak.sh | 83 +++++++++++++++++++++++++++++++++++++++ lenses/mcp-scan.sh | 83 +++++++++++++++++++++++++++++++++++++++ lenses/ocr.sh | 85 ++++++++++++++++++++++++++++++++++++++++ win/doctor.ps1 | 10 +++++ win/lenses/garak.ps1 | 73 ++++++++++++++++++++++++++++++++++ win/lenses/mcp-scan.ps1 | 74 +++++++++++++++++++++++++++++++++++ win/lenses/ocr.ps1 | 87 +++++++++++++++++++++++++++++++++++++++++ 9 files changed, 538 insertions(+) create mode 100755 lenses/garak.sh create mode 100755 lenses/mcp-scan.sh create mode 100755 lenses/ocr.sh create mode 100644 win/lenses/garak.ps1 create mode 100644 win/lenses/mcp-scan.ps1 create mode 100644 win/lenses/ocr.ps1 diff --git a/README.md b/README.md index 488f9d9..a5cb894 100644 --- a/README.md +++ b/README.md @@ -107,11 +107,23 @@ Every lens is **opt-in**: honey never installs a lens's tool for you, and an uninstalled lens is fully inert. Install the tool, and the lens activates on the next run. `./doctor.sh` shows which lenses are active. +**Offline vs. external.** The first six lenses run **offline** — nothing leaves +your machine (`bumblebee`, `smuggle`, `mcp`, and `ocr` are honey-native or use a +local tool; `osv-scanner`/`govulncheck` do static DB lookups). The last two are +**external and off by default**: `mcp-scan` calls a cloud API, and `garak` +probes a live model — so both stay inert unless you *explicitly* enable them +(`HONEY_ENABLE_MCP_SCAN=1` / `HONEY_ENABLE_GARAK=1`) and they never run in the +default cycle. honey's native `mcp` lens covers the MCP surface without the +cloud call. + | Lens | Tool | Covers | Install | |---|---|---|---| | `skillspector` | [NVIDIA SkillSpector](https://github.com/NVIDIA/skillspector) | AI agent skills (`SKILL.md` + scripts): prompt injection, data exfil, excessive agency, tool poisoning | per its README (Python 3.12+) | | `smuggle` | honey-native | **Scanner-evasion** in agent-skill/instruction files: invisible-Unicode (tag chars), bidi overrides (Trojan Source), zero-width chars, and remote-include instructions — the tricks static pattern scanners miss | none (uses `perl`; PowerShell uses native .NET) | | `mcp` | honey-native | **MCP servers** (`.mcp.json` + host configs — invisible to skillspector): inventories every server and **hash-diffs its definition across runs to catch rug pulls** (a server that changes after you approve it); flags fetch-and-exec launch commands | none (jq / native .NET) | +| `ocr` | honey-native | **Image-payload / SkillCamo**: OCRs images bundled in skills and flags hidden instructions — the multimodal blind spot text scanners can't see | [tesseract](https://github.com/tesseract-ocr/tesseract) (`brew install tesseract`) | +| `mcp-scan` | [Invariant / Snyk Agent Scan](https://github.com/snyk/agent-scan) | Hybrid rules+model analysis of MCP servers & skills (tool poisoning/shadowing, toxic flows) — **opt-in; cloud** | `pip install mcp-scan`; enable with `HONEY_ENABLE_MCP_SCAN=1` | +| `garak` | [NVIDIA garak](https://github.com/NVIDIA/garak) | Probes a **live model** for jailbreaks/injection/leakage — **opt-in; not offline** | `pip install garak`; enable with `HONEY_ENABLE_GARAK=1` + a target | | `osv-scanner` | [Google/OSV osv-scanner](https://github.com/google/osv-scanner) | Known vulns in lockfiles/manifests across **all** ecosystems (npm, pypi, cargo, go, …) | `go install github.com/google/osv-scanner/cmd/osv-scanner@latest` | | `govulncheck` | [Go vuln team govulncheck](https://golang.org/x/vuln) | Go vulns that your code **actually calls** (reachability-aware → far less noise) | `go install golang.org/x/vuln/cmd/govulncheck@latest` | @@ -375,6 +387,9 @@ one-off run. You can also hand-edit `honey.conf` — one `export VAR=…` per li | `HONEY_SMUGGLE_EXCLUDE` | `node_modules`/`.git`/`vendor`/`.pnpm` | smuggle lens: skip files whose path matches (ERE) | | `HONEY_MCP_CONFIGS` | Claude/Cursor/VS Code/Windsurf host configs | mcp lens: colon-separated MCP config files to inventory (plus every `.mcp.json` under the project roots) | | `HONEY_MCP_STATE` | `/.mcp-state.json` | mcp lens: where the per-run manifest hashes are stored for rug-pull diffing (gitignored) | +| `HONEY_OCR_MAX` | `300` | ocr lens: cap on images OCR-scanned per run | +| `HONEY_ENABLE_MCP_SCAN` | `0` | set `1` to enable the opt-in `mcp-scan` lens (**cloud/phones home**; also needs `SNYK_TOKEN`/`OPENAI_API_KEY`) | +| `HONEY_ENABLE_GARAK` | `0` | set `1` to enable the opt-in `garak` lens (**probes a live model**; also needs `HONEY_GARAK_TARGET`) | | `HONEY_UPDATE_LENSES` | `1` | `go install @latest` the Go lens binaries (osv-scanner, govulncheck) before scanning; `0` to skip | | `HONEY_OSV_OFFLINE` | `0` | osv-scanner: use local vuln DBs instead of OSV.dev | | `HONEY_OSV_INCLUDE_GO_STDLIB` | `0` | osv-scanner: keep Go stdlib advisories (default defers them to govulncheck) | diff --git a/doctor.sh b/doctor.sh index 952f811..3401c38 100755 --- a/doctor.sh +++ b/doctor.sh @@ -64,6 +64,34 @@ if [ -d "$HONEY/lenses" ] && ls "$HONEY"/lenses/*.sh >/dev/null 2>&1; then mcp) # honey-native lens (jq only) — MCP manifest inventory + rug-pull diffing. ok "lens $lname active (native; MCP manifest hash-and-diff, jq only)" ;; + ocr) + # honey-native multimodal lens; needs tesseract for image OCR. + if command -v tesseract >/dev/null 2>&1; then + ok "lens $lname active (native; image-payload OCR via tesseract $(tesseract --version 2>&1 | head -1 | awk '{print $2}'))" + else + bad "lens $lname inactive — tesseract not installed (optional)" + hint "image-payload (SkillCamo) detection; macOS: brew install tesseract · Debian/Ubuntu: apt-get install tesseract-ocr" + fi ;; + mcp-scan) + # opt-in external lens (cloud/phones home). + if [ "${HONEY_ENABLE_MCP_SCAN:-0}" != "1" ]; then + ok "lens $lname opt-in (disabled) — set HONEY_ENABLE_MCP_SCAN=1 to enable (note: cloud/phones home)" + elif command -v mcp-scan >/dev/null 2>&1; then + ok "lens $lname active (enabled; mcp-scan present)" + else + bad "lens $lname enabled but mcp-scan not installed" + hint "Invariant/Snyk Agent Scan: pip install mcp-scan (https://github.com/snyk/agent-scan)" + fi ;; + garak) + # opt-in external lens (probes a live model). + if [ "${HONEY_ENABLE_GARAK:-0}" != "1" ]; then + ok "lens $lname opt-in (disabled) — set HONEY_ENABLE_GARAK=1 + HONEY_GARAK_TARGET (note: probes a live model)" + elif command -v garak >/dev/null 2>&1; then + ok "lens $lname active (enabled; garak present)" + else + bad "lens $lname enabled but garak not installed" + hint "NVIDIA garak: pip install garak (https://github.com/NVIDIA/garak)" + fi ;; *) if command -v "$lname" >/dev/null 2>&1; then ok "lens $lname active"; else bad "lens $lname inactive (optional tool '$lname' not installed)"; fi ;; esac diff --git a/lenses/garak.sh b/lenses/garak.sh new file mode 100755 index 0000000..1256dd7 --- /dev/null +++ b/lenses/garak.sh @@ -0,0 +1,83 @@ +#!/usr/bin/env bash +# +# honey lens: garak — wrap NVIDIA garak, an LLM vulnerability scanner that sends +# dynamic probes to a LIVE model endpoint (jailbreaks, prompt injection, data +# leakage, toxicity). Unlike honey's file/manifest lenses, garak red-teams a +# running model — it is NOT a scan of on-disk content, it makes network calls, +# and a full run takes minutes. +# +# Doubly OPT-IN: it needs both an explicit enable flag AND a target model, so it +# never runs in the default cycle: +# export HONEY_ENABLE_GARAK=1 +# export HONEY_GARAK_TARGET='--model_type openai --model_name gpt-4o-mini' +# export HONEY_GARAK_PROBES='promptinject,dan,leakreplay' # optional subset +# +# Contract: writes /lens-garak.json (normalized shape). + +set -uo pipefail +RUN_DIR="${1:?usage: garak.sh }" +OUT="$RUN_DIR/lens-garak.json" + +emit() { # STATUS TOTAL BY_SEV FINDINGS NOTE + jq -n --arg lens "garak" --arg tv "${GARAK_VERSION:-unknown}" --arg status "$1" \ + --argjson total "$2" --argjson by_sev "$3" --argjson findings "$4" --arg note "$5" \ + '{lens:$lens, tool_version:$tv, status:$status, findings_total:$total, + findings_by_severity:$by_sev, findings:$findings, note:$note}' >"$OUT" +} + +if [ "${HONEY_ENABLE_GARAK:-0}" != "1" ]; then + emit "skipped" 0 '{}' '[]' "opt-in lens (probes a LIVE model, makes network calls, slow) — set HONEY_ENABLE_GARAK=1 and HONEY_GARAK_TARGET to enable." + echo "lens garak: not enabled (opt-in), skipped"; exit 0 +fi +if ! command -v garak >/dev/null 2>&1; then + emit "skipped" 0 '{}' '[]' "HONEY_ENABLE_GARAK=1 but garak not installed — pip install garak (https://github.com/NVIDIA/garak)." + echo "lens garak: enabled but not installed, skipped"; exit 0 +fi +if [ -z "${HONEY_GARAK_TARGET:-}" ]; then + emit "skipped" 0 '{}' '[]' "garak enabled but no target — set HONEY_GARAK_TARGET (e.g. '--model_type openai --model_name gpt-4o-mini')." + echo "lens garak: no target, skipped"; exit 0 +fi + +GARAK_VERSION="$(garak --version 2>/dev/null | head -1)" +prefix="$RUN_DIR/.garak" +probes_arg="" +[ -n "${HONEY_GARAK_PROBES:-}" ] && probes_arg="--probes ${HONEY_GARAK_PROBES}" + +# shellcheck disable=SC2086 # intentional word-splitting of target/probe args +garak $HONEY_GARAK_TARGET $probes_arg --report_prefix "$prefix" >"$RUN_DIR/.garak.log" 2>&1 || true + +report="$(find "$RUN_DIR" -maxdepth 1 -name '.garak*.report.jsonl' 2>/dev/null | head -1)" +if [ -z "$report" ] || [ ! -s "$report" ]; then + emit "scan_error" 0 '{}' '[]' "garak produced no report — see $RUN_DIR/.garak.log (model/auth/target config?)." + echo "lens garak: no report"; exit 0 +fi + +# Normalize: garak emits eval records with a passrate per probe/detector; a +# probe that the model fails (passrate < threshold) is a finding. Threshold via +# HONEY_GARAK_MIN_PASS (default 0.9). +minpass="${HONEY_GARAK_MIN_PASS:-0.9}" +findings="$(jq -sc --argjson mp "$minpass" ' + [ .[] | select(.entry_type? == "eval") ] + | map(select(((.passed // 0) / (if (.total // 0) == 0 then 1 else .total end)) < $mp) + | { rate: ((.passed // 0) / (if (.total // 0) == 0 then 1 else .total end)), + probe: (.probe // "?"), detector: (.detector // "?") }) + | map({ severity: (if .rate < 0.5 then "high" elif .rate < 0.8 then "medium" else "low" end), + title: ("Model failed probe: " + .probe), + location: (.detector), + detail: ("pass rate " + ((.rate*100)|floor|tostring) + "% on probe \(.probe) / detector \(.detector)"), + ref: "garak" }) + ' "$report" 2>/dev/null)" +[ -n "$findings" ] || findings='[]' + +TOTAL="$(printf '%s' "$findings" | jq 'length')" +BY_SEV="$(printf '%s' "$findings" | jq 'group_by(.severity) | map({key:(.[0].severity), value:length}) | from_entries')" +[ -z "$BY_SEV" ] && BY_SEV='{}' +NOTE="garak (${GARAK_VERSION:-?}) probed ${HONEY_GARAK_TARGET}; findings are model weaknesses below ${minpass} pass rate (report: $report)" + +if [ "$TOTAL" -gt 0 ]; then + emit "exposed" "$TOTAL" "$BY_SEV" "$findings" "$NOTE" +else + emit "clean" 0 '{}' '[]' "$NOTE" +fi +echo "lens garak: status written to $OUT ($TOTAL finding(s))" +exit 0 diff --git a/lenses/mcp-scan.sh b/lenses/mcp-scan.sh new file mode 100755 index 0000000..7dae767 --- /dev/null +++ b/lenses/mcp-scan.sh @@ -0,0 +1,83 @@ +#!/usr/bin/env bash +# +# honey lens: mcp-scan — wrap Invariant Labs' mcp-scan / Snyk Agent Scan for a +# hybrid (deterministic-rules + calibrated-model) read of MCP servers and agent +# skills: tool poisoning, tool shadowing, toxic flows, prompt injection, etc. +# +# OPT-IN and NOT offline. Unlike honey's other lenses, mcp-scan's default mode +# invokes a CLOUD API (Snyk Agent Scan / Invariant Guardrails) and shares tool +# names + descriptions with the vendor; even --local-only needs an OPENAI key. +# So this lens SELF-SKIPS unless you explicitly opt in with: +# export HONEY_ENABLE_MCP_SCAN=1 +# and it never runs in the default daily cycle otherwise. honey's native `mcp` +# lens (offline manifest diffing) covers the no-phone-home path. +# +# Contract: writes /lens-mcp-scan.json (normalized shape). + +set -uo pipefail +RUN_DIR="${1:?usage: mcp-scan.sh }" +OUT="$RUN_DIR/lens-mcp-scan.json" +HONEY="${HONEY:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}" +# shellcheck source=lib/load-config.sh +[ -f "$HONEY/lib/load-config.sh" ] && . "$HONEY/lib/load-config.sh" + +emit() { # STATUS TOTAL BY_SEV FINDINGS NOTE + jq -n --arg lens "mcp-scan" --arg tv "${MCP_SCAN_VERSION:-unknown}" --arg status "$1" \ + --argjson total "$2" --argjson by_sev "$3" --argjson findings "$4" --arg note "$5" \ + '{lens:$lens, tool_version:$tv, status:$status, findings_total:$total, + findings_by_severity:$by_sev, findings:$findings, note:$note}' >"$OUT" +} + +# Opt-in gate: this lens phones home, so it stays inert unless explicitly enabled. +if [ "${HONEY_ENABLE_MCP_SCAN:-0}" != "1" ]; then + emit "skipped" 0 '{}' '[]' "opt-in lens (cloud/phones home) — set HONEY_ENABLE_MCP_SCAN=1 to enable; honey's native offline 'mcp' lens covers the no-network path." + echo "lens mcp-scan: not enabled (opt-in), skipped" + exit 0 +fi +if ! command -v mcp-scan >/dev/null 2>&1; then + emit "skipped" 0 '{}' '[]' "HONEY_ENABLE_MCP_SCAN=1 but mcp-scan not installed — see https://github.com/snyk/agent-scan (pip install mcp-scan / uvx mcp-scan)." + echo "lens mcp-scan: enabled but not installed, skipped" + exit 0 +fi + +MCP_SCAN_VERSION="$(mcp-scan --version 2>/dev/null | head -1)" +raw="$RUN_DIR/.mcp-scan-raw.json" +# --local-only keeps the heaviest cloud call off (still needs an OPENAI key per +# the tool's docs); pass extra args via HONEY_MCP_SCAN_ARGS. +# shellcheck disable=SC2086 # intentional word-splitting of the args string +if ! mcp-scan scan --json ${HONEY_MCP_SCAN_ARGS:---local-only} >"$raw" 2>"$RUN_DIR/.mcp-scan.log"; then + # mcp-scan exits nonzero when it finds issues; only treat non-JSON as an error. + if ! jq -e . "$raw" >/dev/null 2>&1; then + emit "scan_error" 0 '{}' '[]' "mcp-scan produced no valid JSON — see $RUN_DIR/.mcp-scan.log (auth? SNYK_TOKEN/OPENAI_API_KEY needed for non-local scans)." + echo "lens mcp-scan: no valid JSON"; exit 0 + fi +fi + +# Normalize defensively: the schema varies across mcp-scan/agent-scan versions, +# so map the common field names and fall back gracefully. Verify on first real +# run and tune via jq if the shape differs. +findings="$(jq -c ' + [ (.. | objects | select((.severity? // .risk? // .level?) and (.title? // .name? // .label? // .issue? // .description?))) ] + | map({ + severity: ((.severity // .risk // .level // "unknown") | ascii_downcase), + title: (.title // .name // .label // .issue // "mcp-scan finding"), + location: (.location // .path // .server // .tool // .file // ""), + detail: (.description // .detail // .message // ""), + ref: (.code // .rule // "mcp-scan") + }) + | unique' "$raw" 2>/dev/null)" +[ -n "$findings" ] || findings='[]' +rm -f "$raw" + +TOTAL="$(printf '%s' "$findings" | jq 'length')" +BY_SEV="$(printf '%s' "$findings" | jq 'group_by(.severity) | map({key:(.[0].severity), value:length}) | from_entries')" +[ -z "$BY_SEV" ] && BY_SEV='{}' +NOTE="mcp-scan / Snyk Agent Scan (${MCP_SCAN_VERSION:-?}); hybrid rules+model analysis. Args: ${HONEY_MCP_SCAN_ARGS:---local-only}" + +if [ "$TOTAL" -gt 0 ]; then + emit "exposed" "$TOTAL" "$BY_SEV" "$findings" "$NOTE" +else + emit "clean" 0 '{}' '[]' "$NOTE" +fi +echo "lens mcp-scan: status written to $OUT ($TOTAL finding(s))" +exit 0 diff --git a/lenses/ocr.sh b/lenses/ocr.sh new file mode 100755 index 0000000..b910f99 --- /dev/null +++ b/lenses/ocr.sh @@ -0,0 +1,85 @@ +#!/usr/bin/env bash +# +# honey lens: ocr — extract text from images bundled in agent skills and flag +# hidden INSTRUCTIONS, the multimodal blind spot ("SkillCamo"): current skill +# scanners read text/manifests but not images, so malicious operational +# instructions rendered into an image evade scanning yet are recovered by a +# multimodal agent at runtime. This lens OCRs the images and scans the text. +# +# Offline: uses tesseract (Apache-2.0). Self-skips (exit 0 + "skipped") if +# tesseract isn't installed — honey's core path is unaffected. +# +# Finding: OCR-INJECT (high) — an image contains instruction/exfil/injection +# text it has no business carrying (agent skills use images for diagrams/logos, +# not instructions). + +set -uo pipefail +RUN_DIR="${1:?usage: ocr.sh }" +OUT="$RUN_DIR/lens-ocr.json" +HONEY="${HONEY:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}" +# shellcheck source=lib/load-config.sh +[ -f "$HONEY/lib/load-config.sh" ] && . "$HONEY/lib/load-config.sh" + +emit() { # STATUS TOTAL BY_SEV FINDINGS NOTE + jq -n --arg lens "ocr" --arg tv "${OCR_VERSION:-1}" --arg status "$1" --argjson total "$2" \ + --argjson by_sev "$3" --argjson findings "$4" --arg note "$5" \ + '{lens:$lens, tool_version:$tv, status:$status, findings_total:$total, + findings_by_severity:$by_sev, findings:$findings, note:$note}' >"$OUT" +} + +if ! command -v tesseract >/dev/null 2>&1; then + emit "skipped" 0 '{}' '[]' "tesseract not found — ocr lens skipped (brew install tesseract / apt-get install tesseract-ocr to enable image-payload detection)." + echo "lens ocr: tesseract not installed, skipped"; exit 0 +fi + +DEFAULT_ROOTS="$HOME/.claude/skills:$HOME/.claude/plugins:$HOME/.claude/scheduled-tasks:$HOME/.config/claude/skills" +ROOTS="${HONEY_SKILL_ROOTS:-$DEFAULT_ROOTS}" +MAX="${HONEY_OCR_MAX:-300}" # cap images scanned (OCR is the slow part) +# Instruction/exfil/injection markers that have no place in a skill image. +INJECT_RE="${HONEY_OCR_PATTERNS:-ignore[[:space:]]+(the[[:space:]]+)?(previous|prior|above|earlier|all)|disregard[[:space:]]+(the|all|any)|system[[:space:]]+prompt|exfiltrat|curl[[:space:]]+[^[:space:]]*http|wget[[:space:]]+[^[:space:]]*http|base64[[:space:]]+-d|~/\.(ssh|aws|config)|do[[:space:]]+not[[:space:]]+(tell|reveal|mention|inform)|override[[:space:]].*instruction|send[[:space:]]+.*(token|secret|credential|key)}" + +# Collect images. +imgs="$RUN_DIR/.ocr-imgs"; : >"$imgs" +IFS=':' read -r -a roots <<<"$ROOTS" +for root in "${roots[@]}"; do + [ -d "$root" ] || continue + find "$root" -type f \( -iname '*.png' -o -iname '*.jpg' -o -iname '*.jpeg' \ + -o -iname '*.tif' -o -iname '*.tiff' -o -iname '*.bmp' -o -iname '*.gif' -o -iname '*.webp' \) \ + -print 2>/dev/null +done | head -n "$MAX" >>"$imgs" + +if [ ! -s "$imgs" ]; then + emit "clean" 0 '{}' '[]' "no images found under agent-skill roots: $ROOTS" + echo "lens ocr: no images to scan"; rm -f "$imgs"; exit 0 +fi + +findings='[]'; scanned=0 +while IFS= read -r img; do + [ -n "$img" ] || continue + scanned=$((scanned+1)) + # Feed the image via stdin, not by path: some leptonica builds mis-read certain + # absolute paths ("Image file cannot be read"); stdin sidesteps path resolution. + text="$(tesseract stdin stdout <"$img" 2>/dev/null)" || continue + [ -n "$text" ] || continue + hit="$(printf '%s' "$text" | grep -ioE "$INJECT_RE" | head -1)" + if [ -n "$hit" ]; then + snippet="$(printf '%s' "$text" | tr '\n' ' ' | grep -ioE ".{0,30}${hit}.{0,30}" | head -1)" + [ -n "$snippet" ] || snippet="$hit" + findings="$(printf '%s' "$findings" | jq -c --arg l "$img" --arg d "OCR recovered instruction-like text from this image (possible SkillCamo hidden payload): …${snippet}…" \ + '. + [{severity:"high", title:"Instructions hidden in image (possible SkillCamo)", location:$l, detail:$d, ref:"multimodal-evasion"}]')" + fi +done <"$imgs" +rm -f "$imgs" + +TOTAL="$(printf '%s' "$findings" | jq 'length')" +BY_SEV="$(printf '%s' "$findings" | jq 'group_by(.severity) | map({key:(.[0].severity), value:length}) | from_entries')" +[ -z "$BY_SEV" ] && BY_SEV='{}' +NOTE="OCR-scanned $scanned image(s) under $ROOTS for hidden instructions (tesseract $(tesseract --version 2>&1 | head -1 | awk '{print $2}'))" + +if [ "$TOTAL" -gt 0 ]; then + emit "exposed" "$TOTAL" "$BY_SEV" "$findings" "$NOTE" +else + emit "clean" 0 '{}' '[]' "$NOTE" +fi +echo "lens ocr: status written to $OUT ($TOTAL finding(s), $scanned image(s))" +exit 0 diff --git a/win/doctor.ps1 b/win/doctor.ps1 index 4c06d43..11d7b48 100644 --- a/win/doctor.ps1 +++ b/win/doctor.ps1 @@ -78,6 +78,16 @@ foreach ($t in $lensTools.Keys) { # honey-native lenses: no external tool — always on. Ok "lens smuggle active (native; invisible-Unicode/bidi + remote-include detection)" Ok "lens mcp active (native; MCP manifest hash-and-diff for rug pulls)" +# ocr: native multimodal lens, needs tesseract. +if (Get-Command tesseract -ErrorAction SilentlyContinue) { Ok "lens ocr active (native; image-payload OCR via tesseract)" } +else { Write-HoneyConsole " [ ] lens ocr inactive - tesseract not installed (optional)"; Hint "image-payload (SkillCamo) detection; winget install UB-Mannheim.TesseractOCR" } +# opt-in external lenses. +if ((Get-HoneySetting 'HONEY_ENABLE_MCP_SCAN' '0') -ne '1') { Ok "lens mcp-scan opt-in (disabled) - set HONEY_ENABLE_MCP_SCAN=1 (note: cloud/phones home)" } +elseif (Get-Command mcp-scan -ErrorAction SilentlyContinue) { Ok "lens mcp-scan active (enabled; mcp-scan present)" } +else { Bad "lens mcp-scan enabled but mcp-scan not installed"; Hint "pip install mcp-scan (https://github.com/snyk/agent-scan)" } +if ((Get-HoneySetting 'HONEY_ENABLE_GARAK' '0') -ne '1') { Ok "lens garak opt-in (disabled) - set HONEY_ENABLE_GARAK=1 + HONEY_GARAK_TARGET (note: probes a live model)" } +elseif (Get-Command garak -ErrorAction SilentlyContinue) { Ok "lens garak active (enabled; garak present)" } +else { Bad "lens garak enabled but garak not installed"; Hint "pip install garak (https://github.com/NVIDIA/garak)" } # Suppression baseline (informational; never affects pass/fail). Write-HoneyConsole "" diff --git a/win/lenses/garak.ps1 b/win/lenses/garak.ps1 new file mode 100644 index 0000000..590b80e --- /dev/null +++ b/win/lenses/garak.ps1 @@ -0,0 +1,73 @@ +<# +.SYNOPSIS + honey (Windows) lens: garak — wrap NVIDIA garak (LLM vulnerability scanner that + probes a LIVE model). PowerShell mirror of lenses/garak.sh. + +.DESCRIPTION + Doubly OPT-IN: probes a live model, makes network calls, slow. Self-skips + unless HONEY_ENABLE_GARAK=1 AND HONEY_GARAK_TARGET is set. Never part of the + default cycle. + +.PARAMETER RunDir + Run directory to write lens-garak.json into. +#> +param([string]$RunDir) + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' +Import-Module (Join-Path $PSScriptRoot '..' 'lib' 'Honey.psm1') -Force +Import-HoneyConfig + +if (-not $RunDir) { Write-Error 'usage: garak.ps1 -RunDir '; exit 2 } +function Skip($note) { Write-LensResult -RunDir $RunDir -Lens 'garak' -Status 'skipped' -Findings @() -Note $note | Out-Null; Write-HoneyConsole "lens garak: skipped"; exit 0 } + +if ((Get-HoneySetting 'HONEY_ENABLE_GARAK' '0') -ne '1') { + Skip "opt-in lens (probes a LIVE model, makes network calls, slow) - set HONEY_ENABLE_GARAK=1 and HONEY_GARAK_TARGET to enable." +} +if (-not (Get-Command garak -ErrorAction SilentlyContinue)) { + Skip "HONEY_ENABLE_GARAK=1 but garak not installed - pip install garak (https://github.com/NVIDIA/garak)." +} +$target = Get-HoneySetting 'HONEY_GARAK_TARGET' '' +if (-not $target) { + Skip "garak enabled but no target - set HONEY_GARAK_TARGET (e.g. '--model_type openai --model_name gpt-4o-mini')." +} + +$ver = (& garak --version 2>$null | Select-Object -First 1) +$prefix = Join-Path $RunDir '.garak' +$gargs = @($target -split ' ') +$probes = Get-HoneySetting 'HONEY_GARAK_PROBES' '' +if ($probes) { $gargs += @('--probes', $probes) } +$gargs += @('--report_prefix', $prefix) +& garak @gargs *> (Join-Path $RunDir '.garak.log') + +$report = Get-ChildItem -LiteralPath $RunDir -Filter '.garak*.report.jsonl' -ErrorAction SilentlyContinue | Select-Object -First 1 +if (-not $report) { + Write-LensResult -RunDir $RunDir -Lens 'garak' -Status 'scan_error' -Findings @() -Note "garak produced no report - see $RunDir\.garak.log (model/auth/target config?)." | Out-Null + Write-HoneyConsole "lens garak: no report"; exit 0 +} + +$minpass = [double](Get-HoneySetting 'HONEY_GARAK_MIN_PASS' '0.9') +$findings = New-Object System.Collections.ArrayList +foreach ($line in [System.IO.File]::ReadLines($report.FullName)) { + if (-not $line.Trim()) { continue } + try { $e = $line | ConvertFrom-Json } catch { continue } + if (($e.PSObject.Properties.Name -contains 'entry_type') -and $e.entry_type -ne 'eval') { continue } + $total = if ($e.PSObject.Properties.Name -contains 'total' -and $e.total) { [double]$e.total } else { 1 } + $passed = if ($e.PSObject.Properties.Name -contains 'passed') { [double]$e.passed } else { 0 } + $rate = if ($total -eq 0) { 0 } else { $passed / $total } + if ($rate -lt $minpass) { + $probe = if ($e.PSObject.Properties.Name -contains 'probe') { [string]$e.probe } else { '?' } + $detector = if ($e.PSObject.Properties.Name -contains 'detector') { [string]$e.detector } else { '?' } + $sev = if ($rate -lt 0.5) { 'high' } elseif ($rate -lt 0.8) { 'medium' } else { 'low' } + [void]$findings.Add([pscustomobject]@{ + severity = $sev; title = "Model failed probe: $probe"; location = $detector + detail = "pass rate $([math]::Floor($rate*100))% on probe $probe / detector $detector"; ref = 'garak' + }) + } +} + +$note = "garak ($ver) probed $target; findings are model weaknesses below $minpass pass rate (report: $($report.FullName))" +$status = if (@($findings).Count -gt 0) { 'exposed' } else { 'clean' } +Write-LensResult -RunDir $RunDir -Lens 'garak' -ToolVersion ([string]$ver) -Status $status -Findings @($findings) -Note $note | Out-Null +Write-HoneyConsole "lens garak: status written ($(@($findings).Count) finding(s))" +exit 0 diff --git a/win/lenses/mcp-scan.ps1 b/win/lenses/mcp-scan.ps1 new file mode 100644 index 0000000..cf8867b --- /dev/null +++ b/win/lenses/mcp-scan.ps1 @@ -0,0 +1,74 @@ +<# +.SYNOPSIS + honey (Windows) lens: mcp-scan — wrap Invariant Labs mcp-scan / Snyk Agent Scan + (hybrid rules+model analysis of MCP servers & skills). PowerShell mirror of + lenses/mcp-scan.sh. + +.DESCRIPTION + OPT-IN and NOT offline: mcp-scan's default mode calls a CLOUD API and shares + tool names/descriptions with the vendor. Self-skips unless + HONEY_ENABLE_MCP_SCAN=1. honey's native offline `mcp` lens covers the + no-phone-home path. + +.PARAMETER RunDir + Run directory to write lens-mcp-scan.json into. +#> +param([string]$RunDir) + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' +Import-Module (Join-Path $PSScriptRoot '..' 'lib' 'Honey.psm1') -Force +Import-HoneyConfig + +if (-not $RunDir) { Write-Error 'usage: mcp-scan.ps1 -RunDir '; exit 2 } +function Skip($note) { Write-LensResult -RunDir $RunDir -Lens 'mcp-scan' -Status 'skipped' -Findings @() -Note $note | Out-Null; Write-HoneyConsole "lens mcp-scan: skipped"; exit 0 } + +if ((Get-HoneySetting 'HONEY_ENABLE_MCP_SCAN' '0') -ne '1') { + Skip "opt-in lens (cloud/phones home) - set HONEY_ENABLE_MCP_SCAN=1 to enable; honey's native offline 'mcp' lens covers the no-network path." +} +if (-not (Get-Command mcp-scan -ErrorAction SilentlyContinue)) { + Skip "HONEY_ENABLE_MCP_SCAN=1 but mcp-scan not installed - see https://github.com/snyk/agent-scan (pip install mcp-scan / uvx mcp-scan)." +} + +$ver = (& mcp-scan --version 2>$null | Select-Object -First 1) +$argStr = Get-HoneySetting 'HONEY_MCP_SCAN_ARGS' '--local-only' +$raw = Join-Path $RunDir '.mcp-scan-raw.json' +& mcp-scan scan --json @($argStr -split ' ') 2>(Join-Path $RunDir '.mcp-scan.log') | Set-Content -LiteralPath $raw -Encoding utf8 + +$findings = New-Object System.Collections.ArrayList +try { + $doc = Get-Content -LiteralPath $raw -Raw | ConvertFrom-Json +} catch { + Write-LensResult -RunDir $RunDir -Lens 'mcp-scan' -Status 'scan_error' -Findings @() -Note "mcp-scan produced no valid JSON - see $RunDir\.mcp-scan.log (auth? SNYK_TOKEN/OPENAI_API_KEY needed for non-local scans)." | Out-Null + Write-HoneyConsole "lens mcp-scan: no valid JSON"; exit 0 +} + +# Defensive recursive walk: collect objects that look like findings. +function Get-Field($o, [string[]]$names) { foreach ($n in $names) { if ($o.PSObject.Properties.Name -contains $n -and $o.$n) { return [string]$o.$n } } return '' } +function Walk($o) { + if ($null -eq $o) { return } + if ($o -is [System.Management.Automation.PSCustomObject]) { + $sev = Get-Field $o @('severity','risk','level') + $title = Get-Field $o @('title','name','label','issue','description') + if ($sev -and $title) { + [void]$findings.Add([pscustomobject]@{ + severity = $sev.ToLower(); title = $title + location = (Get-Field $o @('location','path','server','tool','file')) + detail = (Get-Field $o @('description','detail','message')) + ref = (Get-Field $o @('code','rule')); _k = "$sev|$title" + }) + } + foreach ($p in $o.PSObject.Properties) { Walk $p.Value } + } elseif ($o -is [System.Collections.IEnumerable] -and $o -isnot [string]) { + foreach ($i in $o) { Walk $i } + } +} +Walk $doc +$uniq = @($findings | Sort-Object _k -Unique | ForEach-Object { $_.PSObject.Properties.Remove('_k'); $_ }) +Remove-Item -LiteralPath $raw -ErrorAction SilentlyContinue + +$note = "mcp-scan / Snyk Agent Scan ($ver); hybrid rules+model analysis. Args: $argStr" +$status = if (@($uniq).Count -gt 0) { 'exposed' } else { 'clean' } +Write-LensResult -RunDir $RunDir -Lens 'mcp-scan' -ToolVersion ([string]$ver) -Status $status -Findings @($uniq) -Note $note | Out-Null +Write-HoneyConsole "lens mcp-scan: status written ($(@($uniq).Count) finding(s))" +exit 0 diff --git a/win/lenses/ocr.ps1 b/win/lenses/ocr.ps1 new file mode 100644 index 0000000..1786e8a --- /dev/null +++ b/win/lenses/ocr.ps1 @@ -0,0 +1,87 @@ +<# +.SYNOPSIS + honey (Windows) lens: ocr — OCR images bundled in agent skills and flag hidden + INSTRUCTIONS (the multimodal "SkillCamo" blind spot). PowerShell mirror of + lenses/ocr.sh. Offline; uses tesseract. Self-skips if tesseract isn't installed. + +.PARAMETER RunDir + Run directory to write lens-ocr.json into. +#> +param([string]$RunDir) + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' +Import-Module (Join-Path $PSScriptRoot '..' 'lib' 'Honey.psm1') -Force +Import-HoneyConfig + +if (-not $RunDir) { Write-Error 'usage: ocr.ps1 -RunDir '; exit 2 } + +if (-not (Get-Command tesseract -ErrorAction SilentlyContinue)) { + Write-LensResult -RunDir $RunDir -Lens 'ocr' -Status 'skipped' -Findings @() -Note "tesseract not found - ocr lens skipped (install tesseract to enable image-payload detection)." | Out-Null + Write-HoneyConsole "lens ocr: tesseract not installed, skipped"; exit 0 +} + +$home_ = Get-HoneyHome +$defaultRoots = @( + (Join-Path $home_ '.claude\skills'), (Join-Path $home_ '.claude\plugins'), + (Join-Path $home_ '.claude\scheduled-tasks'), (Join-Path $home_ '.config\claude\skills') +) -join ';' +$roots = (Get-HoneySetting 'HONEY_SKILL_ROOTS' $defaultRoots) -split ';' | Where-Object { $_ } +$max = [int](Get-HoneySetting 'HONEY_OCR_MAX' '300') +$injectRe = [regex](Get-HoneySettingRaw 'HONEY_OCR_PATTERNS' '(?i)ignore\s+(the\s+)?(previous|prior|above|earlier|all)|disregard\s+(the|all|any)|system\s+prompt|exfiltrat|curl\s+\S*http|wget\s+\S*http|base64\s+-d|~/\.(ssh|aws|config)|do\s+not\s+(tell|reveal|mention|inform)|override\s+.*instruction|send\s+.*(token|secret|credential|key)') +$exts = '.png','.jpg','.jpeg','.tif','.tiff','.bmp','.gif','.webp' + +# Feed image bytes to tesseract via stdin (avoids leptonica path-read quirks). +function Invoke-Tesseract { + param([string]$Image) + $psi = [System.Diagnostics.ProcessStartInfo]::new() + $psi.FileName = 'tesseract'; $psi.Arguments = 'stdin stdout' + $psi.RedirectStandardInput = $true; $psi.RedirectStandardOutput = $true + $psi.RedirectStandardError = $true; $psi.UseShellExecute = $false + $p = [System.Diagnostics.Process]::Start($psi) + try { + $bytes = [System.IO.File]::ReadAllBytes($Image) + $p.StandardInput.BaseStream.Write($bytes, 0, $bytes.Length) + $p.StandardInput.Close() + $out = $p.StandardOutput.ReadToEnd() + $p.WaitForExit() + return $out + } catch { return '' } finally { $p.Dispose() } +} + +$findings = New-Object System.Collections.ArrayList +$scanned = 0 +$images = foreach ($root in $roots) { + if (Test-Path -LiteralPath $root) { + Get-ChildItem -LiteralPath $root -Recurse -File -ErrorAction SilentlyContinue | + Where-Object { $exts -contains $_.Extension.ToLower() } + } +} +$images = @($images) | Select-Object -First $max + +foreach ($img in $images) { + $scanned++ + $text = Invoke-Tesseract $img.FullName + if (-not $text) { continue } + $m = $injectRe.Match($text) + if ($m.Success) { + $flat = ($text -replace '\s+', ' ') + $idx = $flat.IndexOf($m.Value) + $start = [Math]::Max(0, $idx - 30); $len = [Math]::Min($flat.Length - $start, $m.Value.Length + 60) + $snippet = $flat.Substring($start, $len) + [void]$findings.Add([pscustomobject]@{ + severity = 'high'; title = 'Instructions hidden in image (possible SkillCamo)' + location = $img.FullName + detail = "OCR recovered instruction-like text from this image (possible SkillCamo hidden payload): ...$snippet..." + ref = 'multimodal-evasion' + }) + } +} + +$tv = ((& tesseract --version 2>&1 | Select-Object -First 1) -split '\s+')[1] +$note = "OCR-scanned $scanned image(s) under $($roots -join ';') for hidden instructions (tesseract $tv)" +if ($scanned -eq 0) { $note = "no images found under agent-skill roots: $($roots -join ';')" } +$status = if (@($findings).Count -gt 0) { 'exposed' } else { 'clean' } +Write-LensResult -RunDir $RunDir -Lens 'ocr' -ToolVersion ([string]$tv) -Status $status -Findings @($findings) -Note $note | Out-Null +Write-HoneyConsole "lens ocr: status written ($(@($findings).Count) finding(s), $scanned image(s))" +exit 0