Ci/image hardening

.claude/agents/review-correctness.md

@@ -0,0 +1,48 @@
+---
+name: review-correctness
+description: Correctness, accuracy and efficiency reviewer for booktree changes. Use after every code step, before committing, on the diff of the working tree or a commit range.
+tools: Read, Grep, Glob, Bash
+---
+
+You review changes to booktree, a Python tool that matches audiobook files against Audible and MyAnonamouse (MAM)
+metadata and hardlinks them into a media library. You are the correctness/accuracy/efficiency reviewer; a separate
+agent covers security. Report findings only; do not edit files.
+
+## What you check, in priority order
+
+1. **Behavioural correctness of the change.** Trace every changed code path with a concrete input. For matching
+   changes, reason about the actual data shapes: id3 tags can be empty or junk (`AudioTrack 01`, `unknown artist`),
+   Audible per-ASIN lookups can return a skeleton `{asin, asset_details, is_vvab}` with no title, MAM `author_info`
+   / `series_info` are JSON strings, series parts can be decimals (`14.5`) or empty.
+2. **Output contract (byte-compatibility).** The run-log CSV must keep the 46 columns from
+   `myx_utilities.getLogHeaders()` in order, `isMatched`/`isHardLinked` written as `True`/`False`, file name
+   `booktree_log_YYYYMMDDHHMMSS.csv`. The stdout phrases `Processing: …`, `Searching Audible for` + indented
+   `asin:`/`title:`/`authors:`/`narrators:`/`keywords:`, `Searching MAM for` + `TitleFilename:`,
+   `Checking cache: <kind>/<hash>`, `Caching <hash> in File:`, `Found N Audible match(es)`,
+   `Creating Hardlinks for N matched books`, `Hardlinking files for <title>` + `from`/`to` must be unchanged unless the
+   change is explicitly about them. Flag any print() edit that touches these.
+3. **Hard operational rules.** Nothing may write, move, rename or re-tag anything under the source path
+   (`source_path`, seeded torrents). Only hardlink/copy into `media_path`. `fixid3` must default to off. Flag any
+   `os.rename`, `shutil.move`, tag writing, or file creation whose target could resolve under the source path.
+4. **External-call budget.** MAM calls must stay bounded per run and ≥ 6 s apart; Audible calls should not multiply
+   per book without a cap. Cache keys must remain deterministic (sha256 of the query) so consumers can map them.
+5. **Efficiency.** Repeated ffprobe/network/disk work inside loops, O(n²) over the book list, re-reading the same
+   cache file, regex compiled per row, unnecessary copies of large JSON. Only flag what matters at ~2k files per run.
+6. **Tests.** Every fix needs a unit test with the failing input from the bug report (e.g. `&` in publisher,
+   `Author - Title.m4b`, skeleton ASIN response). Check the test actually exercises the changed code and would have
+   failed before the change. Check the replay baseline comparison was run when matching/output code changed.
+7. **Python correctness.** Mutable default args, mutated dataclass fields shared across instances (upstream `Book`
+   has `metadata={}` and `matchRate=0` as class attributes), exception swallowing that hides failures, `bool("False")`
+   truthiness bugs (upstream has one in `buildTreeFromLog`), encoding of non-ASCII titles.
+
+## How to work
+
+* Start with `git diff` (or the range you were given) and read the full functions around each hunk, not just the hunk.
+* Run the unit tests (`python3 -m unittest discover -s tests -p 'test_*.py'`) and, if `tests/replay/out/baseline`
+  exists and the change touches matching/output code, confirm a replay comparison was produced; report its numbers.
+* Verify each finding by reading the code; do not speculate. If you cannot confirm, say so and mark it PLAUSIBLE.
+
+## Output
+
+Findings ranked by severity, each with: file:line, one-sentence defect, a concrete failing input → wrong output, and
+the minimal fix. Then a short list of what you verified and found correct. No praise, no restating the diff.

.claude/agents/review-security.md

@@ -0,0 +1,55 @@
+---
+name: review-security
+description: Security reviewer for booktree changes. Use after every code step, before committing, on the diff of the working tree or a commit range; also for reviewing third-party PRs before merging.
+tools: Read, Grep, Glob, Bash
+---
+
+You review changes to booktree for security. booktree runs unattended (docker, non-root) against a download folder of
+untrusted files, holds a private-tracker session cookie (MAM `mam_id`), calls the Audible and MAM HTTP APIs, and
+writes into a media library that Audiobookshelf then parses. Report findings only; do not edit files.
+
+## Threat model
+
+* **Untrusted input:** file names and folder names under `source_path`; id3 tags read via ffprobe; every field of
+  Audible and MAM API responses; the contents of the cache directory and `cookies.pkl` (world-readable dirs on the
+  host); CSV log rows fed back in log mode (`fix.csv`); `--hints` JSON; mousehole `state.json`.
+* **Assets:** the MAM session cookie (account ban / hit-and-run exposure if leaked or over-used), the seeded torrent
+  files (any write = data loss for the swarm and a tracker violation), the media library, the host filesystem
+  outside the mounted volumes.
+
+## What you check
+
+1. **Secrets.** The MAM session must never reach stdout, the CSV, the cache, exception messages, or `--json-log`.
+   Env/secret-file loading must not fall back to printing the value. Check that redaction covers `Cookie:` headers
+   in any debug output and that the config example files contain placeholders only.
+2. **Path safety.** Every path built from metadata or file names (`getConfigTargetPath`, hardlink/copy targets,
+   OPF/cover destinations, cache files keyed by hash, calibre ingest path) must resolve under the intended root.
+   Check `sanitize_filename` is applied to each component, that `..`, leading `/`, and empty components cannot
+   escape, and that nothing is created under `source_path`.
+3. **Deserialization and injection.** `pickle.load` of `cookies.pkl` from a shared directory is arbitrary code
+   execution if the file is replaced; flag any new pickle use and recommend a plain-text/JSON cookie store. XML/OPF:
+   all text nodes escaped or CDATA-safe (`]]>` inside CDATA). Shell: subprocess must use argument lists, never
+   `shell=True` with metadata. Regex: patterns built from metadata must be `re.escape`d (upstream `getAltTitle` and
+   `getCleanTitle` interpolate author/series names into regexes). JSON from MAM (`author_info` etc.) parsed with
+   `json.loads` only, sizes bounded.
+4. **Network hygiene.** TLS verification on; timeouts on every request; MAM calls throttled (≥ 6 s) and bounded per
+   run; no retries that could hammer MAM on 429/403; errors never cached; `User-Agent` set. No new hosts contacted.
+5. **Cache poisoning / TOCTOU.** Cache files are trusted on read: check that a malformed or attacker-written cache
+   entry cannot crash the run into a bad state or redirect a hardlink. Atomic writes for cache/state files.
+6. **Supply chain.** New dependencies pinned in `requirements.txt`; GitHub Actions pinned to a major tag or SHA;
+   Dockerfile base pinned; nothing downloaded at runtime.
+7. **Third-party PRs.** When reviewing an external PR (e.g. the mousehole integration), additionally look for
+   behaviour unrelated to the stated purpose, changed defaults, new network calls, and anything that reads files
+   outside the configured paths.
+
+## How to work
+
+* `git diff` (or the given range), then read the surrounding functions. Grep for every new `open(`, `os.`,
+  `shutil.`, `subprocess`, `pickle`, `requests`, `httpx`, `re.compile`, `print(` in the change.
+* Run `bandit -q -r . -x ./tests -ll` and `pip-audit -r requirements.txt` if available and include real results.
+* Confirm each finding by reading the code and constructing the malicious input; mark anything unconfirmed PLAUSIBLE.
+
+## Output
+
+Findings ranked by severity (critical / high / medium / low), each with: file:line, the vulnerability class, a
+concrete attack input and consequence, and the minimal fix. Then a short list of checks performed that passed.

.github/dependabot.yml

@@ -0,0 +1,11 @@
+version: 2
+updates:
+  - package-ecosystem: pip
+    directory: /
+    schedule: { interval: weekly }
+  - package-ecosystem: github-actions
+    directory: /
+    schedule: { interval: weekly }
+  - package-ecosystem: docker
+    directory: /
+    schedule: { interval: weekly }

.github/workflows/ci.yml

@@ -0,0 +1,95 @@
+name: CI
+
+on:
+  push:
+    branches: [ main ]
+    tags: [ 'v*' ]
+  pull_request:
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  checks:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v5
+        with: { fetch-depth: 0 }
+      - uses: actions/setup-python@v6
+        with: { python-version: '3.12' }
+      - name: Install
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements.txt ruff bandit pip-audit
+      - name: Lint
+        run: ruff check .
+      - name: Unit tests
+        run: python -m unittest discover -s tests -p 'test_*.py' -v
+      - name: Static security scan (bandit)
+        # high severity gates the build now; the two inherited medium findings (pickle load of cookies.pkl in
+        # myx_mam.py) are removed with the secrets/cookie-store work, after which this tightens to -ll
+        run: bandit -q -r . -x ./tests -lll
+      - name: Dependency audit
+        run: pip-audit -r requirements.txt
+      - name: Secret scan (gitleaks)
+        uses: gitleaks/gitleaks-action@v3
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+  image:
+    needs: checks
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+    steps:
+      - uses: actions/checkout@v5
+      - uses: docker/setup-qemu-action@v4
+      - uses: docker/setup-buildx-action@v4
+      - name: Login to GHCR
+        if: github.event_name != 'pull_request'
+        uses: docker/login-action@v4
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - name: Image metadata
+        id: meta
+        uses: docker/metadata-action@v6
+        with:
+          images: ghcr.io/mancolt/booktree
+          tags: |
+            type=raw,value=latest,enable={{is_default_branch}}
+            type=sha,format=long
+            type=semver,pattern={{version}}
+      - name: Build (and push on main/tags)
+        uses: docker/build-push-action@v7
+        with:
+          context: .
+          platforms: linux/amd64,linux/arm64
+          # pull requests build both platforms into the cache only (multi-arch images cannot be loaded locally)
+          push: ${{ github.event_name != 'pull_request' }}
+          provenance: true
+          sbom: true
+          build-args: |
+            UID=1000
+            GID=1000
+            USER=booktree
+            GROUPNAME=booktree
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          cache-from: type=gha
+          cache-to: type=gha,mode=max
+      - name: Scan image (Trivy)
+        if: github.event_name != 'pull_request'
+        uses: aquasecurity/trivy-action@v0.36.0
+        env:
+          # the package may be private until visibility is flipped; pull with the workflow token
+          TRIVY_USERNAME: ${{ github.actor }}
+          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}
+        with:
+          image-ref: ghcr.io/mancolt/booktree:sha-${{ github.sha }}
+          severity: HIGH,CRITICAL
+          exit-code: '1'
+          ignore-unfixed: true

.gitignore

@@ -0,0 +1,12 @@
+__pycache__/
+*.py[cod]
+.venv/
+# replay output is large (stdout/OPF transcripts per book); the summary is committed separately
+tests/replay/out/
+# never commit live configs, logs, caches or sessions
+config/
+logs/
+__cache__/
+*.pkl
+# site-specific operational notes and handoffs
+docs/site/

CONTRIBUTING.md

@@ -0,0 +1,20 @@
+# Contributing
+
+Thanks for helping. A few rules keep this fork safe to run unattended on other people's libraries.
+
+**Compatibility.** The run-log CSV columns and order, the `True`/`False` strings, the log file name pattern and the
+stdout phrases listed in `docs/FORK.md` are a public interface. Add columns or flags; do not rename or reorder.
+
+**Never write under the source path.** Downloads are often live torrents. No moves, renames, tag rewrites, or files
+created next to the sources. Hardlink/copy into the media path only. `fixid3` stays off by default.
+
+**External services.** MAM calls are bounded per run and spaced ≥ 6 s; Audible/MAM results are cached with a TTL for
+empty results; errors are never cached. Any new call site must go through the existing throttled helpers.
+
+**Secrets.** Never commit configs, logs, caches or cookie files (`.gitignore` covers them). Tests use synthetic
+fixtures only. CI runs gitleaks and push protection is on.
+
+**Tests.** `python3 -m unittest discover -s tests -p 'test_*.py'` must pass. Behaviour changes to matching or output
+need a replay comparison (`tests/replay/`) and the numbers in the PR description.
+
+**Style.** `ruff check .` clean. Small, single-purpose commits; explain *why* in the message.

Dockerfile

@@ -1,31 +1,29 @@
-FROM alpine:latest
+FROM alpine:3.21
 
-ARG UID
-ARG GID
-ARG USER
-ARG GROUPNAME
-ENV USER=${USER}
-ENV GROUPNAME=${GROUPNAME}
-ENV UID=${UID}
-ENV GID=${GID}
+ARG UID=1000
+ARG GID=1000
+ARG USER=booktree
+ARG GROUPNAME=booktree
+ENV USER=${USER} GROUPNAME=${GROUPNAME} UID=${UID} GID=${GID}
 
 COPY . /booktree/
-WORKDIR /booktree 
+WORKDIR /booktree
 RUN echo "**** installing system packages ****" \
     && apk update \
-    && apk add --update --no-cache python3 py3-pip \
-    && apk add py3-pip \
-    && apk add ffmpeg \
-    && ln -sf python3 /usr/bin/python \   
-    && mkdir -p /venv \
-    && python -m venv /venv \
-    && source /venv/bin/activate \ 
-    && pip install --no-cache-dir --requirement requirements.txt \ 
-    && pip install --upgrade pip \
+    && apk upgrade --no-cache \
+    # python3 ships ensurepip, so the venv gets its own pip; no system pip/setuptools are installed
+    && apk add --no-cache python3 ffmpeg \
+    && ln -sf python3 /usr/bin/python \
+    && python3 -m venv /venv \
+    && /venv/bin/pip install --no-cache-dir --upgrade pip \
+    && /venv/bin/pip install --no-cache-dir --requirement requirements.txt \
+    # the runtime image needs no installer; removing it also removes the scanner surface
+    && /venv/bin/pip uninstall -y pip \
+    && rm -rf /root/.cache \
     && if getent passwd ${UID} >/dev/null; then deluser $(getent passwd ${UID} | cut -d: -f1); fi \
     && if getent group ${GID} >/dev/null; then delgroup $(getent group ${GID} | cut -d: -f1); fi \
-    && addgroup --system --gid ${GID} ${GROUPNAME} \
-    && adduser --system --uid ${UID} --disabled-password --gecos "" --ingroup ${GROUPNAME} --no-create-home ${USER} \
+    && addgroup -S -g ${GID} ${GROUPNAME} \
+    && adduser -S -u ${UID} -G ${GROUPNAME} -H -D ${USER} \
     && chown -R ${UID}:${GID} /booktree
 
 USER ${USER}

README.md

@@ -1,4 +1,10 @@
-# booktree
+# booktree (maintained fork)
+
+> This is a maintained fork of [myxdvz/booktree](https://github.com/myxdvz/booktree). It keeps booktree's file layout and
+> run-log format so existing setups keep working, and fixes the matching, caching and OPF defects listed in
+> [docs/FORK.md](docs/FORK.md). Image: `ghcr.io/mancolt/booktree:latest`. Issues and pull requests are welcome; see
+> [CONTRIBUTING.md](CONTRIBUTING.md) and [SECURITY.md](SECURITY.md).
+
 Reorganize your audiobooks using ID3 or Audible metadata into a tree structure recommended and supported by media servers like Audibookshelf. The originals are untouched and will be hardlinked to their destination
 
 It does the following:

SECURITY.md

@@ -0,0 +1,12 @@
+# Security policy
+
+booktree runs unattended against a download folder, holds a private-tracker session cookie, and writes into a media
+library. Reports about any of the following are in scope: writes outside the configured media path, anything that
+touches files under the source path, credential leakage (config, logs, cache, stdout), unsafe deserialization,
+injection through file names or metadata (paths, OPF/XML, shell), and dependency vulnerabilities.
+
+Please report privately through GitHub's "Report a vulnerability" (Security tab) rather than a public issue. You will
+get an acknowledgement within a week. Fixes ship as a new image tag; the advisory is published once a fixed image exists.
+
+Hardening in this fork: the container runs as a non-root user; secrets can be supplied by environment variable or
+secret file instead of the config file; CI runs gitleaks, bandit, pip-audit and a Trivy scan of the built image.