Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 37 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,14 +110,17 @@ the next run. `./doctor.sh` shows which lenses are active.
| Lens | Tool | Covers | Install |
|---|---|---|---|
| `skillspector` | [NVIDIA SkillSpector](https://github.com/NVIDIA/skillspector) | AI agent skills (`SKILL.md` + scripts): prompt injection, data exfil, excessive agency, tool poisoning | per its README (Python 3.12+) |
| `smuggle` | honey-native | **Scanner-evasion** in agent-skill/instruction files: invisible-Unicode (tag chars), bidi overrides (Trojan Source), zero-width chars, and remote-include instructions — the tricks static pattern scanners miss | none (uses `perl`; PowerShell uses native .NET) |
| `osv-scanner` | [Google/OSV osv-scanner](https://github.com/google/osv-scanner) | Known vulns in lockfiles/manifests across **all** ecosystems (npm, pypi, cargo, go, …) | `go install github.com/google/osv-scanner/cmd/osv-scanner@latest` |
| `govulncheck` | [Go vuln team govulncheck](https://golang.org/x/vuln) | Go vulns that your code **actually calls** (reachability-aware → far less noise) | `go install golang.org/x/vuln/cmd/govulncheck@latest` |

**What each answers.** bumblebee: *"do I have a known-compromised package?"*
(exact catalog match). skillspector: *"is this agent skill behaving
maliciously?"* (content analysis). osv-scanner: *"do my dependencies have known
CVEs?"* (broad). govulncheck: *"do I actually reach a vulnerable Go function?"*
(deep, low-noise). Together they cover reactive, proactive, breadth, and depth.
maliciously?"* (content analysis). smuggle: *"is something hidden from the
scanners — invisible Unicode, a bidi trick, a remote include?"* (evasion).
osv-scanner: *"do my dependencies have known CVEs?"* (broad). govulncheck: *"do I
actually reach a vulnerable Go function?"* (deep, low-noise). Together they cover
reactive, proactive, breadth, depth, and anti-evasion.

**Where the vuln lenses scan.** osv-scanner and govulncheck scan your *project*
directories, not all of `$HOME`. Set `HONEY_PROJECT_ROOTS` (colon-separated;
Expand Down Expand Up @@ -217,6 +220,32 @@ first place — sleeper/trigger-activated code, instructions fetched from a remo
URL at runtime, or content hidden in bundled images. The baseline can only
suppress what a lens surfaced; widening the scanned surface is separate work.

## Verdict policy (provenance + severity floor)

Where the baseline pins *specific* findings, the verdict policy is the *broad*
dial for the daily marketplace noise. Two settings, applied after suppression
([`docs/VERDICT.plan.md`](docs/VERDICT.plan.md)):

- **Provenance** — a finding whose location matches `HONEY_TRUSTED_PATTERNS`
(default `claude-plugins-official`) is tagged **first-party**; the report shows
a `[1st-party]` marker.
- **Severity floor** — a finding escalates `OVERALL` only at/above the floor for
its provenance (`HONEY_VERDICT_FLOOR` for third-party, `HONEY_VERDICT_FLOOR_TRUSTED`
for first-party). Below-floor findings move to a **review tier**: still
printed and counted (`OVERALL: … (65 review)`), but non-blocking.

```bash
# Demote first-party low/medium noise to the review tier (high/critical still block):
export HONEY_VERDICT_FLOOR_TRUSTED=high
# Or, for a marketplace you fully trust, demote all of its non-critical findings:
export HONEY_VERDICT_FLOOR_TRUSTED=critical
```

**Safe by default:** floors default to `none` — every finding blocks, exactly as
before. Provenance labeling is on (informational). **bumblebee always blocks**
(known-compromised catalog), and a **MUTATED** pin always blocks (rug-pull
tripwire) — the floor is a lens-noise dial, never a mute for genuine danger.

## Optional: daily Slack triage via a Claude Local routine

If you use [Claude Code](https://claude.com/claude-code) and have the Slack
Expand Down Expand Up @@ -340,14 +369,18 @@ one-off run. You can also hand-edit `honey.conf` — one `export VAR=…` per li
| `BUMBLEBEE_SCAN_ROOT` | `$HOME` | what bumblebee scans |
| `BUMBLEBEE_MAX_DURATION` | `30m` | bumblebee scan time cap |
| `HONEY_PROJECT_ROOTS` | `~/git:~/code:~/Developer:~/src` | where the vuln lenses (osv-scanner, govulncheck) look for projects |
| `HONEY_SKILL_ROOTS` | skill dirs under `~/.claude` | where the skillspector lens looks for agent skills |
| `HONEY_SKILL_ROOTS` | skill dirs under `~/.claude` | where the skillspector and smuggle lenses look for agent skills |
| `HONEY_SMUGGLE_EXCLUDE` | `node_modules`/`.git`/`vendor`/`.pnpm` | smuggle lens: skip files whose path matches (ERE) |
| `HONEY_UPDATE_LENSES` | `1` | `go install @latest` the Go lens binaries (osv-scanner, govulncheck) before scanning; `0` to skip |
| `HONEY_OSV_OFFLINE` | `0` | osv-scanner: use local vuln DBs instead of OSV.dev |
| `HONEY_OSV_INCLUDE_GO_STDLIB` | `0` | osv-scanner: keep Go stdlib advisories (default defers them to govulncheck) |
| `HONEY_OSV_EXCLUDE_PATHS` | `node_modules`/`vendor`/`.pnpm`/`testdata`/`fixtures`/`.claude/worktrees` (ERE) | osv-scanner: skip findings whose source path matches — keeps the scan on *declared* manifests, not installed/vendored copies. Set empty to scan everything. |
| `HONEY_OSV_NO_DEDUPE` | `0` | osv-scanner: `1` keeps one finding per path; default collapses identical `package@version`+advisory across paths into one (`+N more`) |
| `HONEY_SKILLSPECTOR_LLM` | `0` | skillspector: enable its own LLM stage (default static `--no-llm`) |
| `HONEY_BASELINE` | `<repo>/honey.baseline.json` | path to the suppression baseline (pin-and-diff); see [Suppression baseline](#suppression-baseline-pin-and-diff) |
| `HONEY_TRUSTED_PATTERNS` | `claude-plugins-official` | ERE; a finding whose location matches is **first-party** (else third-party). Empty ⇒ nothing trusted. See [Verdict policy](#verdict-policy-provenance--severity-floor) |
| `HONEY_VERDICT_FLOOR` | `none` | min severity (`none\|low\|medium\|high\|critical`) at which a **third-party** finding escalates OVERALL; below it → non-blocking "review" tier (still reported) |
| `HONEY_VERDICT_FLOOR_TRUSTED` | = `HONEY_VERDICT_FLOOR` | same, for **first-party** findings. Set `high` (or `critical`) to demote trusted-marketplace low/medium noise to review |

## Layout

Expand Down
104 changes: 104 additions & 0 deletions docs/VERDICT.plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# Verdict policy — provenance tier + severity floor — design spec

Status: **implemented** (this branch). Companion to
[`BASELINE.plan.md`](BASELINE.plan.md). Where the baseline pins *specific*
reviewed findings, the verdict policy is the *broad* dial: it decides which
findings escalate `OVERALL` at all.

## 1. Problem

The suppression baseline is surgical — you pin individual reviewed findings. But
the daily noise from a trusted, first-party skill marketplace is *broad*: dozens
of low/medium false positives you don't want to pin one by one, yet don't want
flipping the verdict every day either. And even after the baseline, `OVERALL` is
still a flat worst-wins: a single low finding reads the same as a critical.

## 2. Two dials, applied after suppression

Both operate at the report/verdict layer, on the already-classified findings, so
`report.sh` and `daily-cycle.sh` (and their PowerShell mirrors) agree.

### Provenance
Each finding's location is matched against `HONEY_TRUSTED_PATTERNS` (an ERE,
default `claude-plugins-official` — Anthropic's official marketplace, the usual
source of the doc-scan false positives). A match → **first-party**; otherwise
**third-party**. Provenance is shown in the report (`[1st-party]` tag) and feeds
the floor. Set `HONEY_TRUSTED_PATTERNS=` (empty) to treat everything as
third-party.

### Severity floor
A finding **blocks** (escalates `OVERALL`) only if its severity is at or above
the floor **for its provenance**:

- `HONEY_VERDICT_FLOOR` (default `none`) — floor for third-party/unknown.
- `HONEY_VERDICT_FLOOR_TRUSTED` (default = `HONEY_VERDICT_FLOOR`) — floor for
first-party.

Values: `none | low | medium | high | critical`. A finding **below** its floor
is **not** dropped — it moves to a **review tier**: still printed (dimmed, marked
`[non-blocking]`), still counted on the verdict line, but it does not flip
`OVERALL`.

## 3. Classification

Every classified finding now also carries `_provenance` and `_blocking`. Three
outcomes for an active (non-suppressed) finding:

| | escalates OVERALL? | shown? |
|---|---|---|
| **blocking** (`_blocking=yes`) | yes | yes, prominent |
| **review** (`_blocking=no`) | no | yes, dimmed `[non-blocking]` |
| suppressed (from the baseline) | no | summarized only |

## 4. Non-negotiable overrides

The floor is a *lens-noise* dial, never a way to mute genuinely dangerous
findings. Regardless of floor:

- **bumblebee always blocks** — it is the known-compromised-package catalog
scanner; a match is never "review".
- **MUTATED pins always block** — the rug-pull tripwire fires no matter the floor.
- `incomplete` / `scan_error` are untouched (the policy only reclassifies
`exposed` findings, never partial coverage or a crash).

## 5. Safe default

Floors default to `none`, i.e. **every finding blocks, exactly as before** — a
security tool must not silently hide findings out of the box. Provenance
*labeling* is on by default (informational, safe). To actually quiet first-party
low/medium noise, opt in, e.g. in `honey.conf`:

```
export HONEY_VERDICT_FLOOR_TRUSTED=high # first-party low/medium -> review; high/critical still block
# or, to demote ALL first-party noise (it's a doc-scan marketplace you trust):
export HONEY_VERDICT_FLOOR_TRUSTED=critical
```

## 6. Verdict line

The tallies are always on the line so a held run can't read as a bare all-clear:

- `OVERALL: CLEAN (123 review)` — nothing blocking; 123 findings held below the
floor (still listed).
- `OVERALL: EXPOSED (12 suppressed, 65 review)` — real blocking findings drive
it; 65 first-party lows/mediums demoted, 12 baseline-pinned.

`daily-cycle` exits `0` only when nothing blocks (review-only ⇒ clean).

## 7. How it composes with the baseline

- **Baseline** = "I reviewed *this exact finding* and it's benign" (content-hashed, per-finding).
- **Verdict policy** = "findings from *this source* below *this severity* are background noise, not alarms" (broad, per-provenance).

Together: the marketplace's structural false positives drop to the review tier
via one `HONEY_VERDICT_FLOOR_TRUSTED` setting, and the handful you want to fully
vanish get pinned. A real HIGH in a first-party skill still blocks; a MUTATED pin
still screams.

## 8. Files

- **bash:** `lib/verdict.sh` (provenance + floor helpers), sourced by
`lib/baseline.sh`; `report.sh` renders blocking vs review; `daily-cycle.sh`
inherits the review count via `baseline_effective_overall`.
- **win:** `win/lib/Verdict.psm1`, imported by `Baseline.psm1`; `report.ps1`,
`daily-cycle.ps1` updated.
8 changes: 8 additions & 0 deletions doctor.sh
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,14 @@ if [ -d "$HONEY/lenses" ] && ls "$HONEY"/lenses/*.sh >/dev/null 2>&1; then
bad "lens $lname inactive — govulncheck not installed (optional)"
hint "Go reachability-aware vuln scanning; install: go install golang.org/x/vuln/cmd/govulncheck@latest"
fi ;;
smuggle)
# honey-native lens (no external tool); needs perl for Unicode scanning.
if command -v perl >/dev/null 2>&1; then
ok "lens $lname active (native; perl $(perl -e 'print substr($^V,1)' 2>/dev/null))"
else
bad "lens $lname inactive — perl not found (optional)"
hint "invisible-Unicode/bidi + remote-include detection; install perl to enable (ships on macOS + most Linux)"
fi ;;
*)
if command -v "$lname" >/dev/null 2>&1; then ok "lens $lname active"; else bad "lens $lname inactive (optional tool '$lname' not installed)"; fi ;;
esac
Expand Down
107 changes: 107 additions & 0 deletions lenses/smuggle.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
#!/usr/bin/env bash
#
# honey lens: smuggle — detect scanner-evasion smuggling in agent-skill and
# instruction files that static pattern scanners (incl. skillspector) miss:
#
# • invisible Unicode — tag characters U+E0000–E007F carry ASCII instructions
# imperceptibly ("ASCII smuggling"); no legit text use.
# • bidirectional overrides — U+202A–202E / U+2066–2069, the "Trojan Source"
# trick that reorders how source/text renders vs. runs.
# • zero-width — U+200B/200C/2060 hidden joiners/spaces (excludes the
# emoji ZWJ U+200D to avoid false positives).
# • remote includes — instructions telling the agent to fetch/read a remote
# URL at runtime (content the on-disk scan never sees).
#
# Honey lens contract:
# - invoked as: lenses/smuggle.sh <RUN_DIR>
# - writes <RUN_DIR>/lens-smuggle.json in honey's normalized shape
# { lens, tool_version, status, findings_total, findings_by_severity,
# findings: [ {severity,title,location,detail,ref} ], note }
# status ∈ clean | exposed | scan_error | skipped
# - self-skips (exit 0 + "skipped") if perl isn't available — honey's core path
# is unaffected for anyone without it (perl ships on macOS + most Linux).

set -uo pipefail
RUN_DIR="${1:?usage: smuggle.sh <RUN_DIR>}"
OUT="$RUN_DIR/lens-smuggle.json"
HONEY="${HONEY:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}"
# shellcheck source=lib/load-config.sh
[ -f "$HONEY/lib/load-config.sh" ] && . "$HONEY/lib/load-config.sh"

emit() { # emit STATUS TOTAL BY_SEV_JSON FINDINGS_JSON NOTE
jq -n --arg lens "smuggle" --arg tv "${SMUGGLE_VERSION:-1}" \
--arg status "$1" --argjson total "$2" --argjson by_sev "$3" \
--argjson findings "$4" --arg note "$5" \
'{lens:$lens, tool_version:$tv, status:$status, findings_total:$total,
findings_by_severity:$by_sev, findings:$findings, note:$note}' >"$OUT"
}

if ! command -v perl >/dev/null 2>&1; then
emit "skipped" 0 '{}' '[]' "perl not found — smuggle lens skipped (install perl to enable Unicode-smuggling detection)."
echo "lens smuggle: perl not installed, skipped"
exit 0
fi

# Same surface as skillspector: agent-skill roots (override via HONEY_SKILL_ROOTS)
# plus any MCP manifests. Text-ish files only; skip the run/vendor noise.
DEFAULT_ROOTS="$HOME/.claude/skills:$HOME/.claude/plugins:$HOME/.claude/scheduled-tasks:$HOME/.config/claude/skills"
ROOTS="${HONEY_SKILL_ROOTS:-$DEFAULT_ROOTS}"
EXCLUDE_RE="${HONEY_SMUGGLE_EXCLUDE:-/node_modules/|/\.git/|/vendor/|/\.pnpm/}"

# Collect candidate files (portable; NUL-delimited to survive odd names).
files_nul="$RUN_DIR/.smuggle-files"
: >"$files_nul"
IFS=':' read -r -a roots <<<"$ROOTS"
for root in "${roots[@]}"; do
[ -d "$root" ] || continue
find "$root" -type f \( -name '*.md' -o -name '*.markdown' -o -name '*.txt' \
-o -name '*.json' -o -name '*.mcp.json' -o -name '*.yaml' -o -name '*.yml' \
-o -name '*.sh' -o -name '*.py' -o -name '*.js' -o -name '*.ts' \
-o -name '*.html' -o -name 'SKILL.md' \) -print0 2>/dev/null
done | grep -zvE "$EXCLUDE_RE" >>"$files_nul" 2>/dev/null || true

if [ ! -s "$files_nul" ]; then
emit "clean" 0 '{}' '[]' "no agent-skill/instruction files found under: $ROOTS"
echo "lens smuggle: no files to scan"
rm -f "$files_nul"
exit 0
fi

# --- Scan (perl): emit TSV severity<TAB>title<TAB>file<TAB>line<TAB>detail ----
# -CSD: treat args/stdin/stdout as UTF-8. `close ARGV if eof` resets $. per file.
# shellcheck disable=SC2016 # this is a perl program; $vars/$. are perl, not shell
tsv="$(xargs -0 perl -CSD -ne '
BEGIN { $re_tag = qr/[\x{E0000}-\x{E007F}]/;
$re_bidi = qr/[\x{202A}-\x{202E}\x{2066}-\x{2069}]/;
$re_zw = qr/[\x{200B}\x{200C}\x{2060}\x{180E}]/;
$re_url = qr/(?:curl|wget|fetch|download|retrieve|read|load|import|include|open)\b[^\n]{0,60}https?:\/\//i; }
if (/($re_tag)/) { printf "high\tInvisible Unicode (tag chars)\t%s\t%d\tASCII-smuggling codepoint U+%04X — carries hidden instructions invisible to humans and byte scanners\n", $ARGV, $., ord($1); }
if (/($re_bidi)/) { printf "high\tBidirectional override (Trojan Source)\t%s\t%d\tbidi control U+%04X can reorder how text renders vs. is interpreted\n", $ARGV, $., ord($1); }
if (/($re_zw)/) { printf "medium\tZero-width character\t%s\t%d\thidden zero-width codepoint U+%04X (possible token smuggling)\n", $ARGV, $., ord($1); }
if (/$re_url/) { printf "medium\tRemote include instruction\t%s\t%d\tinstructs the agent to fetch/read a remote URL — content the on-disk scan never sees\n", $ARGV, $.; }
close ARGV if eof;
' <"$files_nul")"

rm -f "$files_nul"

# Build normalized findings JSON from the TSV.
findings='[]'
if [ -n "$tsv" ]; then
findings="$(printf '%s\n' "$tsv" | jq -Rc 'select(length>0) | split("\t")
| {severity:.[0], title:.[1], location:(.[2] + ":" + .[3]), detail:.[4], ref:"scanner-evasion"}' \
| jq -sc '.')"
fi
[ -n "$findings" ] || findings='[]'

TOTAL="$(printf '%s' "$findings" | jq 'length')"
BY_SEV="$(printf '%s' "$findings" | jq 'group_by(.severity) | map({key:(.[0].severity), value:length}) | from_entries')"
[ -z "$BY_SEV" ] && BY_SEV='{}'
NOTE="scanned agent-skill/instruction files under $ROOTS for invisible/bidi/zero-width Unicode + remote-include instructions"

if [ "$TOTAL" -gt 0 ]; then
emit "exposed" "$TOTAL" "$BY_SEV" "$findings" "$NOTE"
else
emit "clean" 0 '{}' '[]' "$NOTE"
fi
echo "lens smuggle: status written to $OUT ($TOTAL finding(s))"
exit 0
Loading
Loading