Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ cycle.log
honey.conf
honey.conf.ps1

# NOTE: honey.baseline.json is intentionally NOT ignored. Suppression pins are
# committed and reviewed in git — that auditability is an anti-gaming control
# (see docs/BASELINE.plan.md). Do not add it here.

# Local secrets, if you ever add any (e.g. Slack webhook/token).
secrets.env
*.env
Expand Down
64 changes: 63 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,60 @@ and honey treats them differently:
honey never mutates Python environments. Re-install it periodically per its
README to get new patterns.

## Suppression baseline (pin-and-diff)

Static scanners are noisy. A daily run over trusted, first-party agent skills
can emit dozens of findings that are false positives against reference or
documentation material — enough to drag `OVERALL` to `EXPOSED` every day and
bury a *real* finding. honey lets you acknowledge a reviewed-benign finding
**once** so it stops contributing to the daily verdict — without going blind to
the thing honey exists to catch.

The mechanism is **pin, not mute**. A suppression entry doesn't say "ignore this
skill" (trust-by-location is blindness-by-location — a first-party plugin that
ships a malicious update keeps its path). It says *"I reviewed this exact content
and judged this finding benign — tell me the instant the content changes."* Each
entry records a **sha256 of the referenced file**, so:

- pinned finding, file **unchanged** → **suppressed** (dropped from the verdict, still counted);
- pinned finding, file **changed** → **MUTATED** — resurfaces loudly (the rug-pull tripwire);
- pin past its **expiry** (default 90 days) → resurfaces as normal.

Every suppressed finding is therefore also a change-detector on the content it
vouches for. Suppression is a **report-layer re-rank, never a deletion** — raw
run records are untouched — and a suppressed run **never** reads as a bare
all-clear: the verdict always carries the tallies, e.g.
`OVERALL: CLEAN (12 suppressed)` or `OVERALL: EXPOSED — … (12 suppressed, 2 mutated)`.
Design details and the threat model are in [`docs/BASELINE.plan.md`](docs/BASELINE.plan.md).

**Manage it** with [`honey-baseline.sh`](honey-baseline.sh) (Windows:
[`win/honey-baseline.ps1`](win/honey-baseline.ps1)) — it only edits
`honey.baseline.json`, never your system:

```bash
./honey-baseline.sh status # dry-run: suppressed/mutated/expired/active tally
./honey-baseline.sh add --scanner skillspector \
--location references/ \
--reason "first-party marketplace docs; describes patterns, not executable" --expires 90
./honey-baseline.sh list [--expired|--active] # review pins (flags expired)
./honey-baseline.sh remove --scanner skillspector --location references/
./honey-baseline.sh prune # drop expired pins
```

`add` filters are combinable (AND): `--all` (excludes bumblebee), `--scanner`,
`--severity`, `--rule`, `--location`. Pinning a **bumblebee** (known-compromised)
match is opt-in and loud — `--all` excludes it; you must name `--scanner
bumblebee` explicitly. `honey.baseline.json` is **committed and reviewable** on
purpose: baseline edits show up in `git diff` / PR review, which is itself an
anti-gaming control.

**What it does and does not defend.** The content hash is the right defense
against silent post-review mutation (manifest-alteration rug pulls, source
swaps). It is structurally blind to payloads that were never flagged in the
first place — sleeper/trigger-activated code, instructions fetched from a remote
URL at runtime, or content hidden in bundled images. The baseline can only
suppress what a lens surfaced; widening the scanned surface is separate work.

## Optional: daily Slack triage via a Claude Local routine

If you use [Claude Code](https://claude.com/claude-code) and have the Slack
Expand Down Expand Up @@ -293,6 +347,7 @@ one-off run. You can also hand-edit `honey.conf` — one `export VAR=…` per li
| `HONEY_OSV_EXCLUDE_PATHS` | `node_modules`/`vendor`/`.pnpm`/`testdata`/`fixtures`/`.claude/worktrees` (ERE) | osv-scanner: skip findings whose source path matches — keeps the scan on *declared* manifests, not installed/vendored copies. Set empty to scan everything. |
| `HONEY_OSV_NO_DEDUPE` | `0` | osv-scanner: `1` keeps one finding per path; default collapses identical `package@version`+advisory across paths into one (`+N more`) |
| `HONEY_SKILLSPECTOR_LLM` | `0` | skillspector: enable its own LLM stage (default static `--no-llm`) |
| `HONEY_BASELINE` | `<repo>/honey.baseline.json` | path to the suppression baseline (pin-and-diff); see [Suppression baseline](#suppression-baseline-pin-and-diff) |

## Layout

Expand All @@ -305,13 +360,18 @@ honey/
├── daily-cycle.sh # one cycle: run-scan, exit 0=clean / 1=needs attention
├── notify-cycle.sh # scan + native desktop notification (no-Claude path)
├── report.sh # deterministic triage report (bumblebee + all lenses; no AI)
├── honey-baseline.sh # manage the pin-and-diff suppression baseline (add/list/remove/prune/status)
├── honey.baseline.json # the suppression baseline (committed & reviewable; NOT gitignored)
├── install-schedule.sh # schedule notify-cycle via launchd (macOS) / cron (Linux)
├── lib/baseline.sh # shared suppression classifier (sourced by report/daily-cycle/CLI)
├── lenses/ # optional additional scanners, each self-skips if its tool is absent
│ ├── skillspector.sh # AI agent skills (NVIDIA SkillSpector)
│ ├── osv-scanner.sh # multi-ecosystem lockfile vulns (Google/OSV)
│ └── govulncheck.sh # Go reachability-aware vulns (Go vuln team)
├── docs/BASELINE.plan.md # suppression baseline design + threat model
├── routine-prompt.md # prompt for the Claude Local routine (scheduled path)
├── triage-guide.md # guide for triaging a run by hand in a chat
├── win/ # native Windows (PowerShell 7) port — mirrors every script above
├── runs/<TS>/ # one timestamped run per scan (gitignored — host inventory)
│ ├── manifest.json # bumblebee verdict + metadata
│ ├── findings.ndjson # bumblebee finding records
Expand All @@ -320,7 +380,9 @@ honey/
```

`runs/`, `latest`, and `*.log` are **gitignored** — they contain an inventory
of your machine and should never be published.
of your machine and should never be published. `honey.baseline.json` is
deliberately **not** gitignored: suppression pins are meant to be reviewed in
version control.

## Troubleshooting

Expand Down
24 changes: 19 additions & 5 deletions daily-cycle.sh
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@ CYCLE_LOG="$HONEY/cycle.log"
# so a bare-env Local routine or cron job picks it up. env var > honey.conf > default.
# shellcheck source=lib/load-config.sh
. "$HONEY/lib/load-config.sh"
# Suppression baseline classifier, so the cycle's exit code honors pinned
# findings exactly as report.sh does (one source of truth for the verdict).
# shellcheck source=lib/baseline.sh
. "$HONEY/lib/baseline.sh"

# A scheduler / Local routine may hand us a bare PATH. Ensure the dirs that
# hold the scanners are findable BEFORE we invoke run-scan and the lenses —
Expand Down Expand Up @@ -75,7 +79,9 @@ clog "manifest status=$STATUS findings=$TOTAL run_dir=$RUN_DIR"
overall_rank() { # map a status to a severity rank for "worst wins"
case "$1" in scan_error) echo 4;; exposed) echo 3;; incomplete) echo 2;; clean) echo 1;; *) echo 0;; esac
}
OVERALL="$STATUS"
# CRASH tracks only lenses that RAN but wrote no verdict — a genuine crash must
# always escalate, and the baseline (which reads written JSON) can't see it.
CRASH="clean"
if [ -d "$HONEY/lenses" ]; then
for lens in "$HONEY/lenses"/*.sh; do
[ -f "$lens" ] || continue # no lenses installed → loop body never runs
Expand All @@ -89,17 +95,25 @@ if [ -d "$HONEY/lenses" ]; then
# "skipped" JSON, so this only triggers on a genuine crash.)
if [ ! -f "$LJSON" ] || ! jq -e '.status' "$LJSON" >/dev/null 2>&1; then
clog "lens $lname: CRASHED (rc=$LRC, no valid verdict) — escalating to scan_error"
[ "$(overall_rank scan_error)" -gt "$(overall_rank "$OVERALL")" ] && OVERALL="scan_error"
CRASH="scan_error"
continue
fi
LSTATUS="$(jq -r '.status' "$LJSON" 2>/dev/null)"
LTOTAL="$(jq -r '.findings_total' "$LJSON" 2>/dev/null)"
clog "lens $lname: status=$LSTATUS findings=$LTOTAL"
[ "$LSTATUS" = "skipped" ] && continue
[ "$(overall_rank "$LSTATUS")" -gt "$(overall_rank "$OVERALL")" ] && OVERALL="$LSTATUS"
done
fi

clog "=== cycle end (bumblebee=$STATUS overall=$OVERALL) ==="
# Effective verdict AFTER the suppression baseline: report.sh and this cycle
# must agree, so both go through the same classifier. baseline_effective_overall
# recomputes worst-wins over bumblebee + every written lens JSON, dropping
# suppressed findings (but never downgrading incomplete/scan_error, and always
# surfacing MUTATED pins). A crashed lens (no JSON) is then worst-wins'd in.
EFF="$(baseline_effective_overall "$RUN_DIR")"
OVERALL="$(printf '%s' "$EFF" | awk '{print $1}')"
SUPPCOUNTS="$(printf '%s' "$EFF" | cut -d' ' -f2-)"
[ "$(overall_rank "$CRASH")" -gt "$(overall_rank "$OVERALL")" ] && OVERALL="$CRASH"

clog "=== cycle end (bumblebee=$STATUS overall=$OVERALL $SUPPCOUNTS) ==="
# 0 when overall clean; 1 when anything (bumblebee or a lens) needs attention.
[ "$OVERALL" = "clean" ] && exit 0 || exit 1
Loading
Loading