Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@ honey.conf.ps1
# committed and reviewed in git — that auditability is an anti-gaming control
# (see docs/BASELINE.plan.md). Do not add it here.

# MCP-manifest lens state — auto-maintained hashes of MCP server definitions for
# rug-pull (drift) detection. Machine-specific; contains local config hashes.
.mcp-state.json

# Local secrets, if you ever add any (e.g. Slack webhook/token).
secrets.env
*.env
Expand Down
45 changes: 41 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,14 +110,19 @@ the next run. `./doctor.sh` shows which lenses are active.
| Lens | Tool | Covers | Install |
|---|---|---|---|
| `skillspector` | [NVIDIA SkillSpector](https://github.com/NVIDIA/skillspector) | AI agent skills (`SKILL.md` + scripts): prompt injection, data exfil, excessive agency, tool poisoning | per its README (Python 3.12+) |
| `smuggle` | honey-native | **Scanner-evasion** in agent-skill/instruction files: invisible-Unicode (tag chars), bidi overrides (Trojan Source), zero-width chars, and remote-include instructions — the tricks static pattern scanners miss | none (uses `perl`; PowerShell uses native .NET) |
| `mcp` | honey-native | **MCP servers** (`.mcp.json` + host configs — invisible to skillspector): inventories every server and **hash-diffs its definition across runs to catch rug pulls** (a server that changes after you approve it); flags fetch-and-exec launch commands | none (jq / native .NET) |
| `osv-scanner` | [Google/OSV osv-scanner](https://github.com/google/osv-scanner) | Known vulns in lockfiles/manifests across **all** ecosystems (npm, pypi, cargo, go, …) | `go install github.com/google/osv-scanner/cmd/osv-scanner@latest` |
| `govulncheck` | [Go vuln team govulncheck](https://golang.org/x/vuln) | Go vulns that your code **actually calls** (reachability-aware → far less noise) | `go install golang.org/x/vuln/cmd/govulncheck@latest` |

**What each answers.** bumblebee: *"do I have a known-compromised package?"*
(exact catalog match). skillspector: *"is this agent skill behaving
maliciously?"* (content analysis). osv-scanner: *"do my dependencies have known
CVEs?"* (broad). govulncheck: *"do I actually reach a vulnerable Go function?"*
(deep, low-noise). Together they cover reactive, proactive, breadth, and depth.
maliciously?"* (content analysis). smuggle: *"is something hidden from the
scanners — invisible Unicode, a bidi trick, a remote include?"* (evasion). mcp:
*"did an MCP server's definition change since I approved it?"* (rug-pull
diffing). osv-scanner: *"do my dependencies have known CVEs?"* (broad).
govulncheck: *"do I actually reach a vulnerable Go function?"* (deep, low-noise).
Together they cover reactive, proactive, breadth, depth, anti-evasion, and drift.

**Where the vuln lenses scan.** osv-scanner and govulncheck scan your *project*
directories, not all of `$HOME`. Set `HONEY_PROJECT_ROOTS` (colon-separated;
Expand Down Expand Up @@ -217,6 +222,32 @@ first place — sleeper/trigger-activated code, instructions fetched from a remo
URL at runtime, or content hidden in bundled images. The baseline can only
suppress what a lens surfaced; widening the scanned surface is separate work.

## Verdict policy (provenance + severity floor)

Where the baseline pins *specific* findings, the verdict policy is the *broad*
dial for the daily marketplace noise. Two settings, applied after suppression
([`docs/VERDICT.plan.md`](docs/VERDICT.plan.md)):

- **Provenance** — a finding whose location matches `HONEY_TRUSTED_PATTERNS`
(default `claude-plugins-official`) is tagged **first-party**; the report shows
a `[1st-party]` marker.
- **Severity floor** — a finding escalates `OVERALL` only at/above the floor for
its provenance (`HONEY_VERDICT_FLOOR` for third-party, `HONEY_VERDICT_FLOOR_TRUSTED`
for first-party). Below-floor findings move to a **review tier**: still
printed and counted (`OVERALL: … (65 review)`), but non-blocking.

```bash
# Demote first-party low/medium noise to the review tier (high/critical still block):
export HONEY_VERDICT_FLOOR_TRUSTED=high
# Or, for a marketplace you fully trust, demote all of its non-critical findings:
export HONEY_VERDICT_FLOOR_TRUSTED=critical
```

**Safe by default:** floors default to `none` — every finding blocks, exactly as
before. Provenance labeling is on (informational). **bumblebee always blocks**
(known-compromised catalog), and a **MUTATED** pin always blocks (rug-pull
tripwire) — the floor is a lens-noise dial, never a mute for genuine danger.

## Optional: daily Slack triage via a Claude Local routine

If you use [Claude Code](https://claude.com/claude-code) and have the Slack
Expand Down Expand Up @@ -340,14 +371,20 @@ one-off run. You can also hand-edit `honey.conf` — one `export VAR=…` per li
| `BUMBLEBEE_SCAN_ROOT` | `$HOME` | what bumblebee scans |
| `BUMBLEBEE_MAX_DURATION` | `30m` | bumblebee scan time cap |
| `HONEY_PROJECT_ROOTS` | `~/git:~/code:~/Developer:~/src` | where the vuln lenses (osv-scanner, govulncheck) look for projects |
| `HONEY_SKILL_ROOTS` | skill dirs under `~/.claude` | where the skillspector lens looks for agent skills |
| `HONEY_SKILL_ROOTS` | skill dirs under `~/.claude` | where the skillspector and smuggle lenses look for agent skills |
| `HONEY_SMUGGLE_EXCLUDE` | `node_modules`/`.git`/`vendor`/`.pnpm` | smuggle lens: skip files whose path matches (ERE) |
| `HONEY_MCP_CONFIGS` | Claude/Cursor/VS Code/Windsurf host configs | mcp lens: colon-separated MCP config files to inventory (plus every `.mcp.json` under the project roots) |
| `HONEY_MCP_STATE` | `<repo>/.mcp-state.json` | mcp lens: where the per-run manifest hashes are stored for rug-pull diffing (gitignored) |
| `HONEY_UPDATE_LENSES` | `1` | `go install @latest` the Go lens binaries (osv-scanner, govulncheck) before scanning; `0` to skip |
| `HONEY_OSV_OFFLINE` | `0` | osv-scanner: use local vuln DBs instead of OSV.dev |
| `HONEY_OSV_INCLUDE_GO_STDLIB` | `0` | osv-scanner: keep Go stdlib advisories (default defers them to govulncheck) |
| `HONEY_OSV_EXCLUDE_PATHS` | `node_modules`/`vendor`/`.pnpm`/`testdata`/`fixtures`/`.claude/worktrees` (ERE) | osv-scanner: skip findings whose source path matches — keeps the scan on *declared* manifests, not installed/vendored copies. Set empty to scan everything. |
| `HONEY_OSV_NO_DEDUPE` | `0` | osv-scanner: `1` keeps one finding per path; default collapses identical `package@version`+advisory across paths into one (`+N more`) |
| `HONEY_SKILLSPECTOR_LLM` | `0` | skillspector: enable its own LLM stage (default static `--no-llm`) |
| `HONEY_BASELINE` | `<repo>/honey.baseline.json` | path to the suppression baseline (pin-and-diff); see [Suppression baseline](#suppression-baseline-pin-and-diff) |
| `HONEY_TRUSTED_PATTERNS` | `claude-plugins-official` | ERE; a finding whose location matches is **first-party** (else third-party). Empty ⇒ nothing trusted. See [Verdict policy](#verdict-policy-provenance--severity-floor) |
| `HONEY_VERDICT_FLOOR` | `none` | min severity (`none\|low\|medium\|high\|critical`) at which a **third-party** finding escalates OVERALL; below it → non-blocking "review" tier (still reported) |
| `HONEY_VERDICT_FLOOR_TRUSTED` | = `HONEY_VERDICT_FLOOR` | same, for **first-party** findings. Set `high` (or `critical`) to demote trusted-marketplace low/medium noise to review |

## Layout

Expand Down
104 changes: 104 additions & 0 deletions docs/VERDICT.plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# Verdict policy — provenance tier + severity floor — design spec

Status: **implemented** (this branch). Companion to
[`BASELINE.plan.md`](BASELINE.plan.md). Where the baseline pins *specific*
reviewed findings, the verdict policy is the *broad* dial: it decides which
findings escalate `OVERALL` at all.

## 1. Problem

The suppression baseline is surgical — you pin individual reviewed findings. But
the daily noise from a trusted, first-party skill marketplace is *broad*: dozens
of low/medium false positives you don't want to pin one by one, yet don't want
flipping the verdict every day either. And even after the baseline, `OVERALL` is
still a flat worst-wins: a single low finding reads the same as a critical.

## 2. Two dials, applied after suppression

Both operate at the report/verdict layer, on the already-classified findings, so
`report.sh` and `daily-cycle.sh` (and their PowerShell mirrors) agree.

### Provenance
Each finding's location is matched against `HONEY_TRUSTED_PATTERNS` (an ERE,
default `claude-plugins-official` — Anthropic's official marketplace, the usual
source of the doc-scan false positives). A match → **first-party**; otherwise
**third-party**. Provenance is shown in the report (`[1st-party]` tag) and feeds
the floor. Set `HONEY_TRUSTED_PATTERNS=` (empty) to treat everything as
third-party.

### Severity floor
A finding **blocks** (escalates `OVERALL`) only if its severity is at or above
the floor **for its provenance**:

- `HONEY_VERDICT_FLOOR` (default `none`) — floor for third-party/unknown.
- `HONEY_VERDICT_FLOOR_TRUSTED` (default = `HONEY_VERDICT_FLOOR`) — floor for
first-party.

Values: `none | low | medium | high | critical`. A finding **below** its floor
is **not** dropped — it moves to a **review tier**: still printed (dimmed, marked
`[non-blocking]`), still counted on the verdict line, but it does not flip
`OVERALL`.

## 3. Classification

Every classified finding now also carries `_provenance` and `_blocking`. Three
outcomes for an active (non-suppressed) finding:

| | escalates OVERALL? | shown? |
|---|---|---|
| **blocking** (`_blocking=yes`) | yes | yes, prominent |
| **review** (`_blocking=no`) | no | yes, dimmed `[non-blocking]` |
| suppressed (from the baseline) | no | summarized only |

## 4. Non-negotiable overrides

The floor is a *lens-noise* dial, never a way to mute genuinely dangerous
findings. Regardless of floor:

- **bumblebee always blocks** — it is the known-compromised-package catalog
scanner; a match is never "review".
- **MUTATED pins always block** — the rug-pull tripwire fires no matter the floor.
- `incomplete` / `scan_error` are untouched (the policy only reclassifies
`exposed` findings, never partial coverage or a crash).

## 5. Safe default

Floors default to `none`, i.e. **every finding blocks, exactly as before** — a
security tool must not silently hide findings out of the box. Provenance
*labeling* is on by default (informational, safe). To actually quiet first-party
low/medium noise, opt in, e.g. in `honey.conf`:

```
export HONEY_VERDICT_FLOOR_TRUSTED=high # first-party low/medium -> review; high/critical still block
# or, to demote ALL first-party noise (it's a doc-scan marketplace you trust):
export HONEY_VERDICT_FLOOR_TRUSTED=critical
```

## 6. Verdict line

The tallies are always on the line so a held run can't read as a bare all-clear:

- `OVERALL: CLEAN (123 review)` — nothing blocking; 123 findings held below the
floor (still listed).
- `OVERALL: EXPOSED (12 suppressed, 65 review)` — real blocking findings drive
it; 65 first-party lows/mediums demoted, 12 baseline-pinned.

`daily-cycle` exits `0` only when nothing blocks (review-only ⇒ clean).

## 7. How it composes with the baseline

- **Baseline** = "I reviewed *this exact finding* and it's benign" (content-hashed, per-finding).
- **Verdict policy** = "findings from *this source* below *this severity* are background noise, not alarms" (broad, per-provenance).

Together: the marketplace's structural false positives drop to the review tier
via one `HONEY_VERDICT_FLOOR_TRUSTED` setting, and the handful you want to fully
vanish get pinned. A real HIGH in a first-party skill still blocks; a MUTATED pin
still screams.

## 8. Files

- **bash:** `lib/verdict.sh` (provenance + floor helpers), sourced by
`lib/baseline.sh`; `report.sh` renders blocking vs review; `daily-cycle.sh`
inherits the review count via `baseline_effective_overall`.
- **win:** `win/lib/Verdict.psm1`, imported by `Baseline.psm1`; `report.ps1`,
`daily-cycle.ps1` updated.
11 changes: 11 additions & 0 deletions doctor.sh
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,17 @@ if [ -d "$HONEY/lenses" ] && ls "$HONEY"/lenses/*.sh >/dev/null 2>&1; then
bad "lens $lname inactive — govulncheck not installed (optional)"
hint "Go reachability-aware vuln scanning; install: go install golang.org/x/vuln/cmd/govulncheck@latest"
fi ;;
smuggle)
# honey-native lens (no external tool); needs perl for Unicode scanning.
if command -v perl >/dev/null 2>&1; then
ok "lens $lname active (native; perl $(perl -e 'print substr($^V,1)' 2>/dev/null))"
else
bad "lens $lname inactive — perl not found (optional)"
hint "invisible-Unicode/bidi + remote-include detection; install perl to enable (ships on macOS + most Linux)"
fi ;;
mcp)
# honey-native lens (jq only) — MCP manifest inventory + rug-pull diffing.
ok "lens $lname active (native; MCP manifest hash-and-diff, jq only)" ;;
*)
if command -v "$lname" >/dev/null 2>&1; then ok "lens $lname active"; else bad "lens $lname inactive (optional tool '$lname' not installed)"; fi ;;
esac
Expand Down
140 changes: 140 additions & 0 deletions lenses/mcp.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
#!/usr/bin/env bash
#
# honey lens: mcp — inventory MCP server manifests and detect RUG PULLS by
# hashing each server's definition and diffing it across runs.
#
# This closes honey's biggest blind spot: an MCP server defined by a `.mcp.json`
# / host config with no SKILL.md is invisible to the skillspector lens. The
# canonical defense against a manifest-alteration rug pull (a server that earns
# approval with a benign definition, then silently changes) is exactly this:
# "hash the tool manifest on first sight and diff every subsequent run."
# (Invariant Labs / policylayer.)
#
# Findings:
# • MCP-DRIFT (high) — a known server's definition CHANGED since last run.
# • MCP-NEW (low) — a server appeared that wasn't here before.
# • MCP-RISKY (medium) — the launch command fetches-and-executes remote code
# (curl|wget … | sh/bash, `bash -c`, `eval`).
#
# The first run SEEDS the state silently (everything is "new" on a fresh box, so
# NEW/DRIFT are only reported once a baseline manifest exists). RISKY is
# content-based and reported every run.
#
# Honey lens contract: writes <RUN_DIR>/lens-mcp.json (normalized shape);
# self-skips only if there are no MCP configs to look at. Offline; jq only.

set -uo pipefail
RUN_DIR="${1:?usage: mcp.sh <RUN_DIR>}"
OUT="$RUN_DIR/lens-mcp.json"
HONEY="${HONEY:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}"
# shellcheck source=lib/load-config.sh
[ -f "$HONEY/lib/load-config.sh" ] && . "$HONEY/lib/load-config.sh"

STATE="${HONEY_MCP_STATE:-$HONEY/.mcp-state.json}"

emit() { # STATUS TOTAL BY_SEV FINDINGS NOTE
jq -n --arg lens "mcp" --arg tv "1" --arg status "$1" --argjson total "$2" \
--argjson by_sev "$3" --argjson findings "$4" --arg note "$5" \
'{lens:$lens, tool_version:$tv, status:$status, findings_total:$total,
findings_by_severity:$by_sev, findings:$findings, note:$note}' >"$OUT"
}

hash_stdin() {
if command -v sha256sum >/dev/null 2>&1; then sha256sum | awk '{print $1}'
elif command -v shasum >/dev/null 2>&1; then shasum -a 256 | awk '{print $1}'
else openssl dgst -sha256 2>/dev/null | awk '{print $NF}'; fi
}

# --- Discover MCP config files ---------------------------------------------
# Host configs (override via HONEY_MCP_CONFIGS, colon-separated) + every
# .mcp.json / mcp.json under the project roots.
DEFAULT_CONFIGS="$HOME/.claude.json:$HOME/Library/Application Support/Claude/claude_desktop_config.json:$HOME/.config/claude/claude_desktop_config.json:$HOME/.cursor/mcp.json:$HOME/.vscode/mcp.json:$HOME/.codeium/windsurf/mcp_config.json"
CONFIGS="${HONEY_MCP_CONFIGS:-$DEFAULT_CONFIGS}"
PROJECT_ROOTS="${HONEY_PROJECT_ROOTS:-$HOME/git:$HOME/code:$HOME/Developer:$HOME/src}"

configs_file="$RUN_DIR/.mcp-configs"; : >"$configs_file"
IFS=':' read -r -a carr <<<"$CONFIGS"
for c in "${carr[@]}"; do [ -f "$c" ] && printf '%s\n' "$c" >>"$configs_file"; done
IFS=':' read -r -a proots <<<"$PROJECT_ROOTS"
for root in "${proots[@]}"; do
[ -d "$root" ] || continue
find "$root" -maxdepth 6 \( -name '.mcp.json' -o -name 'mcp.json' \) -type f 2>/dev/null \
| grep -vE '/node_modules/|/\.git/|/\.claude/worktrees/' >>"$configs_file" 2>/dev/null || true
done
sort -u "$configs_file" -o "$configs_file"

if [ ! -s "$configs_file" ]; then
emit "skipped" 0 '{}' '[]' "no MCP configs found (looked at host configs + .mcp.json under $PROJECT_ROOTS)"
echo "lens mcp: no MCP configs, skipped"; rm -f "$configs_file"; exit 0
fi

# --- Extract servers: cfg <TAB> name <TAB> base64(canonical def) ------------
extract="$RUN_DIR/.mcp-servers"; : >"$extract"
while IFS= read -r cfg; do
[ -n "$cfg" ] || continue
jq -r --arg cfg "$cfg" '
def norm: walk(if type=="object" then to_entries|sort_by(.key)|from_entries else . end);
((.mcpServers // {}) + (.servers // {}) + (((.projects // {}) | [.[]?.mcpServers // {}]) | add // {}))
| to_entries[] | [$cfg, .key, (.value | norm | tojson | @base64)] | @tsv
' "$cfg" 2>/dev/null >>"$extract" || true
done <"$configs_file"
rm -f "$configs_file"

SEEDING=0
[ -f "$STATE" ] || SEEDING=1
[ -f "$STATE" ] || echo '{"servers":{}}' >"$STATE"

# --- Diff against state + scan for risky commands --------------------------
findings='[]'
newstate='{"servers":{}}'
today="$(date -u +%Y-%m-%d)"
add_finding() { # severity title location detail
findings="$(printf '%s' "$findings" | jq -c --arg s "$1" --arg t "$2" --arg l "$3" --arg d "$4" \
'. + [{severity:$s,title:$t,location:$l,detail:$d,ref:"mcp-manifest"}]')"
}

while IFS=$'\t' read -r cfg name defb64; do
[ -n "$name" ] || continue
canonical="$(printf '%s' "$defb64" | base64 -d 2>/dev/null)"
h="sha256:$(printf '%s' "$canonical" | hash_stdin)"
key="$cfg::$name"
prev="$(jq -r --arg k "$key" '.servers[$k].hash // ""' "$STATE" 2>/dev/null)"
first="$(jq -r --arg k "$key" '.servers[$k].first_seen // ""' "$STATE" 2>/dev/null)"
[ -n "$first" ] || first="$today"

if [ "$SEEDING" -eq 0 ]; then
if [ -z "$prev" ]; then
add_finding "low" "New MCP server: $name" "$cfg" "server '$name' appeared since the last run — confirm you added it. Definition: $canonical"
elif [ "$prev" != "$h" ]; then
add_finding "high" "MCP manifest changed (possible rug pull): $name" "$cfg" "server '$name' definition CHANGED since last run (was ${prev%%:*}:${prev:7:12}…, now ${h:7:12}…). A server that alters its manifest after approval is the canonical rug-pull vector — re-review. Now: $canonical"
fi
fi

# Content check (every run): launch command fetches-and-executes remote code.
if printf '%s' "$canonical" | grep -qiE '(curl|wget)[^|]*\|[[:space:]]*(sh|bash)|bash[[:space:]]+-c|(^|[^a-z])sh[[:space:]]+-c|[^a-z]eval[^a-z]'; then
add_finding "medium" "MCP server runs a fetch-and-exec command: $name" "$cfg" "launch command pipes remote content into a shell or uses bash -c/eval — a code-execution vector. Definition: $canonical"
fi

newstate="$(printf '%s' "$newstate" | jq -c --arg k "$key" --arg h "$h" --arg f "$first" --arg l "$today" \
'.servers[$k] = {hash:$h, first_seen:$f, last_seen:$l}')"
done <"$extract"
srvcount="$(wc -l <"$extract" | tr -d ' ')"
rm -f "$extract"

# Persist the new manifest state (current servers only — a removed server just
# drops out; it is not a finding).
printf '%s' "$newstate" | jq '.' >"$STATE" 2>/dev/null || true

TOTAL="$(printf '%s' "$findings" | jq 'length')"
BY_SEV="$(printf '%s' "$findings" | jq 'group_by(.severity) | map({key:(.[0].severity), value:length}) | from_entries')"
[ -z "$BY_SEV" ] && BY_SEV='{}'
NOTE="inventoried $srvcount MCP server(s); manifest hash-and-diff for rug pulls (state: $STATE)"
[ "$SEEDING" -eq 1 ] && NOTE="$NOTE; FIRST RUN seeded the baseline manifest (drift/new detection starts next run)"

if [ "$TOTAL" -gt 0 ]; then
emit "exposed" "$TOTAL" "$BY_SEV" "$findings" "$NOTE"
else
emit "clean" 0 '{}' '[]' "$NOTE"
fi
echo "lens mcp: status written to $OUT ($TOTAL finding(s), $srvcount server(s), seeding=$SEEDING)"
exit 0
Loading
Loading