drneox · Hiteek · May 2, 2026 · May 2, 2026 · May 3, 2026 · May 3, 2026
diff --git a/.gitignore b/.gitignore
@@ -24,6 +24,13 @@ logs/
 .semgrep/
 frida_scripts/
 config.yaml
+test.yaml
+config*.yaml
+!config.yaml.example
+nutcracker.db
+nutcracker.db-wal
+nutcracker.db-shm
+.nutcracker_tmp/
 
 # AI / IDE tool files
 .code-review-graph/

diff --git a/README.md b/README.md
diff --git a/ROADMAP.md b/ROADMAP.md
@@ -9,16 +9,25 @@
 ## Platforms
 - [ ] **iOS / IPA support** — download with `ipatool`, static Mach-O analysis with `jtool2`/`class-dump`, secrets in `.plist`/`.strings`, `Info.plist` analysis (permissions, ATS), jailbreak check detection. MVP static only; dynamic requires jailbreak.
 
+## aipwn
+- [ ] **Multimodal message format incompatible with z.ai/GLM (and possibly other OpenAI-compatible providers)** — found live 2026-07-27 while validating the dashboard chat wiring: after the first `take_screenshot()` tool call, the LLM call fails with `messages.content.type is invalid, allowed values: ['text']`. `frida_agent.py` builds the image message in a format OpenAI itself accepts but this provider's OpenAI-compatible endpoint rejects — needs investigating the exact expected content-array shape for z.ai, or falling back to text-only description when the provider doesn't support images.
+
 ## Reporting
-- [ ] **Differentiate runtime bypass vs DEX extraction in reports** — The current verdict only distinguishes `PROTECTED` from `PROTECTION BROKEN`, where `PROTECTION BROKEN` requires a successful in-memory DEX dump (`dex_count > 0`). This is misleading: the FridaAgent can bypass all anti-root protections and run the app on a rooted device without ever dumping DEX (e.g. apps with aggressive native anti-root that kill the process before FART hooks run). Proposed fix — two independent verdicts:
+- [~] **Differentiate runtime bypass vs DEX extraction in reports** — The current verdict only distinguishes `PROTECTED` from `PROTECTION BROKEN`, where `PROTECTION BROKEN` requires a successful in-memory DEX dump (`dex_count > 0`). This is misleading: the FridaAgent can bypass all anti-root protections and run the app on a rooted device without ever dumping DEX (e.g. apps with aggressive native anti-root that kill the process before FART hooks run). Proposed fix — two independent verdicts:
   - **Static protection** (`PROTECTED` / `UNPROTECTED`): driven by the existing static detectors.
-  - **Dynamic analysis** (`BYPASS CONFIRMED` / `DEX EXTRACTED` / `NOT ATTEMPTED`): fed by the FridaAgent result (`report_success`) and the FART/dexdump pipeline respectively. 
+  - **Dynamic analysis** (`BYPASS CONFIRMED` / `DEX EXTRACTED` / `NOT ATTEMPTED`): fed by the FridaAgent result (`report_success`) and the FART/dexdump pipeline respectively.
   Implementation touch-points:
-  1. Add `aipwn_bypass_confirmed: bool` field to `AnalysisResult` (set by `aipwn.py` after `report_success`).
-  2. Change `build_masvs_report()` in `masvs.py`: replace `bypass_confirmed = analysis.protection_broken` with `bypass_confirmed = analysis.protection_broken or analysis.aipwn_bypass_confirmed`. This makes `MASVS-RESILIENCE-*` controls flip to `bypass` status and apply the `_BYPASS_PENALTY` even when DEX extraction failed.
-  3. Update `reporter.py` to show a separate dynamic analysis banner alongside the static verdict.
-  4. Update the PDF report with a new dynamic analysis section.
+  1. [x] Add `aipwn_bypass_confirmed: bool` field to `AnalysisResult` (set by `aipwn.py` after `report_success`).
+  2. [x] Change `build_masvs_report()` in `masvs.py`: `bypass_confirmed = analysis.protection_broken or analysis.aipwn_bypass_confirmed`. `MASVS-RESILIENCE-*` controls now flip to `bypass` status and apply the `_BYPASS_PENALTY` even when DEX extraction failed. Done 2026-07-24, see `plan.md` Fase 2.4.
+  3. [ ] Update `reporter.py` to show a separate dynamic analysis banner alongside the static verdict.
+  4. [ ] Update the PDF report with a new dynamic analysis section.
 
 ## Architecture
-- [ ] **Split `vuln_scanner.py`** — 1400+ lines with three responsibilities. Split into `scan_types.py` (dataclasses), `vuln_scanner.py` (regex + semgrep) and `leak_scanner.py` (apkleaks, gitleaks). Do this before adding new leak sources or porting to Go.
-- [ ] **Port secret scanner to Go** — `string_extractor.py` and the HC* rules are the slowest steps. Go binary `nutcracker-strings` that takes a directory and patterns and returns JSON. Python invokes it as a subprocess, the same way it does semgrep. Prerequisite: split modules first.
+- [x] **Split `vuln_scanner.py`** — 1400+ lines with three responsibilities. Split into `scan_types.py` (dataclasses), `vuln_scanner.py` (regex + semgrep) and `leak_scanner.py` (apkleaks, gitleaks). Done 2026-07 (`plan.md` Fase 0.3) — `vuln_scanner.py` is now 456 lines.
+- [ ] **Port secret scanner to Go** — `string_extractor.py` and the HC* rules are the slowest steps. Go binary `nutcracker-strings` that takes a directory and patterns and returns JSON. Python invokes it as a subprocess, the same way it does semgrep. Prerequisite: split modules first (done, see above).
+- [x] **`pyproject.toml` packaging** — `pip install -e .` gives a `nutcracker` console command (entry point `nutcracker_core.cli:cli`); `.[dashboard]` extra installs FastAPI/uvicorn. Done 2026-07-26.
+- [x] ~~Live device video (scrcpy) in the dashboard~~ — **removed 2026-07-27** at the user's explicit request, superseded by WebUSB (next item) + the existing screenshot-polling fallback. Was: the dashboard drove the user's own real `scrcpy` binary headlessly (`--no-window --record=<tmp>.mkv`, re-read via `container.seek()` with PyAV). Had an architectural fps ceiling (~1.3fps even after optimization) and started automatically just from opening the Device tab (an earlier fix in the same session made that explicit-only, right before the feature was removed entirely). See `plan.md` for the full history.
+- [x] **WebUSB + WebCodecs device video (Tango/yume-chan)** — implemented 2026-07-27, `nutcracker_core/plugins/dashboard/webusb/` (the project's first npm/TypeScript/Vite subproject). Replicates app.webadb.com: the browser talks ADB/scrcpy directly over WebUSB (no server-side `adb`/`scrcpy` process at all for the video path), decoding raw H.264 natively via WebCodecs — resolves the ~1.3fps ceiling at its root. Typechecked against the real Tango `.d.ts` files (not guessed), build verified (scrcpy-server v3.3.1 binary confirmed byte-for-byte embedded in the bundle), browser support confirmed live via Playwright (`navigator.usb` present, button appears, module imports cleanly, `isSupported()` true, zero console errors). **Not verified**: an actual end-to-end USB connection against a physical device — WebUSB's permission dialog requires real human interaction by design and can't be automated; needs the user to click through it themselves (ideally from Chrome/Edge on the same machine the phone's USB is attached to). Real trade-offs (Chromium-only, phone must be on USB on the browser's own machine, needs a secure context, USB-exclusivity-with-system-adb unconfirmed) are documented in `plan.md` Fase 4 and `webusb/README.md`.
+- [x] **`aipwn` agent reasoning + chat wired into the dashboard** — `aipwn` is now a queue job kind (`queue add --aipwn`), reusing the existing live-logs WebSocket for its reasoning/tool-call stream; the chat mailbox (`chat_mailbox.py`) is polled by `FridaAgent._check_operator_chat()` once per ReAct iteration and injects pending operator messages as real conversation turns. Done 2026-07-27, validated live with a real LLM call against `com.example.tapjacking` — confirmed an injected chat message reaching the agent's actual conversation. (Found in passing, not fixed — out of scope: the configured provider, z.ai/GLM, rejects the multimodal screenshot message format `frida_agent.py` sends, a pre-existing bug unrelated to this wiring.)
+- [x] **Batch static+aipwn from a .txt file, with device-vs-store APK sourcing** — `queue add <list_file> --then-aipwn [--source device] --serial <s> --run` queues a static job for every package id in the file and, once each finishes OK, chains an `aipwn` job for it (both always run, in order, per the confirmed design — not per-line mixed control). `--source device` is a single global flag applying to the whole file (not per line): it makes `scan` pull the .apk already installed on the device via `adb shell pm path` + `adb pull` (`DeviceInstalledDownloader` in `downloader.py`, handles App Bundle splits) instead of downloading from a store — useful for internal/unpublished test apps or when store credentials aren't configured. Threaded through `cli/scan.py` (`--source`/`--serial`), `orchestrator.build_job_cmd(source=...)`, `Job.source` (in-memory only, like `is_local_apk`), and `QueueEngine.submit(source=...)`/`_run_job`. Done 2026-07-27, 16 new tests added (181→197 passing), covers the downloader, command building, the local-APK-reuse-heuristic override (an explicit `--source device` must not be silently skipped in favor of a stale downloaded APK), and the CLI chaining/failure/global-flag behavior.
+- [x] **Same batch (static+aipwn from a .txt, device-vs-store sourcing) wired into the dashboard** — new `POST /api/queue/batch` (`plugins/dashboard/api.py`) accepts a package-id list plus `source`/`serial`/`then_aipwn`, submits every static job, and runs the drain-then-chain-aipwn logic in a background thread (same pattern as the existing `_drain_in_background`, guarded by the same `engine._dashboard_draining` flag so it never races a concurrent single-job `queue add`). Frontend: a "Batch desde archivo" block in the queue card (`static/index.html`) reads a `.txt` client-side via `File.text()` — no multipart upload needed, just a parsed JSON array — with a source dropdown (store/device) reusing the already-selected device serial from the Device tab; progress shows up in the existing queue table (already polled every 5s), no new UI needed for that. Done 2026-07-27: 5 new `TestClient` tests (202/202 passing) plus a live end-to-end check against a real running server on an isolated scratchpad DB (both the store-download and `--source device` failure paths confirmed through the actual HTTP endpoint, not just mocks).
diff --git a/config.yaml.example b/config.yaml.example
@@ -1,3 +1,6 @@
+# Values can reference environment variables with "${VAR_NAME}", e.g.:
+#   aas_token: '${GOOGLE_PLAY_AAS_TOKEN}'
+# Useful to keep real secrets out of config.yaml (which is still gitignored).
 google_play:
   email: ''
   aas_token: ''
@@ -141,3 +144,58 @@ auto:
 batch:
   list_file: ''           # path to the list file (optional if passed as CLI argument)
   stop_on_error: false    # stop batch processing if an app fails
+
+# ── Persistence (SQLite) ──────────────────────────────────────────────────────
+# Every analysis is also recorded in a local SQLite DB (runs/findings/artifacts),
+# in addition to the existing reports/<pkg>/*.json and PDF (non-destructive).
+store:
+  enabled: true          # set false to skip SQLite persistence entirely
+  db_path: ''            # empty = ./nutcracker.db at the project root
+
+# ── Queue (mass execution — plan.md Fase 1) ────────────────────────────────────
+# `nutcracker queue add`/`nutcracker serve`: static jobs (decompile/scan/OSINT)
+# run in parallel; dynamic jobs (Frida/ADB on a physical device) are always
+# serialized per device serial, regardless of dynamic_workers.
+queue:
+  static_workers: 1      # parallel static analyses at once
+  dynamic_workers: 1      # concurrent *devices* for dynamic jobs (never same serial)
+
+# ── Scheduler (periodic reviews — plan.md Fase 1.2) ────────────────────────────
+# `nutcracker serve` re-queues every app whose next_due_at has passed, and
+# reschedules it after each run using its own `schedule` entry (or this default).
+scheduler:
+  enabled: true
+  poll_interval_minutes: 60      # how often the daemon checks for due apps
+  default_interval_days: 30      # ≥1 review/month per app unless overridden
+
+# ── Web dashboard (plugin — plan.md Fase 3) ────────────────────────────────────
+# `nutcracker dashboard`: starts its own queue + scheduler + a local web UI
+# (apps, live job logs, device screenshot, MASVS trend, schedule editor).
+dashboard:
+  bind: '127.0.0.1'
+  port: 8765
+  # Video en vivo del dispositivo: pestaña "Dispositivo" -> "🔌 USB directo
+  # (fluido)" (WebUSB + WebCodecs, ver nutcracker_core/plugins/dashboard/webusb/).
+  # Es la única vía de video: no hay fallback por screenshots.
+  #
+  # El navegador reclama el cable USB en exclusiva para ese video, así que
+  # nutcracker debe hablarle al teléfono por el serial de red
+  # (strategies.default_device_id = "<ip>:5555", tras `adb tcpip 5555`). Ese
+  # transporte TCP vive en el daemon adb y se cae solo (reinicio del daemon,
+  # doze/corte de red del teléfono); cada cuántos segundos revisarlo y
+  # reconectarlo en segundo plano. Mínimo efectivo: 10.
+  adb_keepalive_seconds: 60
+# ── Toolbox estático en Docker (opt-in) ────────────────────────────────────
+# Sandboxea jadx/apktool (y a futuro otras herramientas estáticas) dentro de
+# un contenedor Docker en vez de invocar binarios instalados en el host --
+# útil para no depender de tener cada herramienta instalada localmente (este
+# mismo entorno, por ejemplo, no tiene jadx) y para aislar el análisis de
+# APKs de terceros (potencialmente maliciosos) del resto del sistema.
+#
+# Herramientas dinámicas (adb, frida) NUNCA pasan por acá -- necesitan hablar
+# directo con el dispositivo físico conectado al host. Ver
+# nutcracker_core/toolbox/ para el diseño completo.
+toolbox:
+  enabled: false   # true = decompilar vía Docker en vez de binarios locales
+  image: 'nutcracker-toolbox-static:latest'
+