scripts: make the LINE extension capture work on macOS, Linux and Windows

.agents/commands/line-capture.md

@@ -2,9 +2,21 @@ Launch Chrome with network capture enabled for LINE Chrome Extension analysis.
 
 ## What to do
 
-1. **Close any running Chrome instances first** (Chrome only supports one instance per profile with remote debugging). Warn the user if Chrome is running.
+1. **Check dependencies.** Run `bash scripts/line-capture-preflight.sh` (on
+   Windows, from Git Bash). It prints the detected OS and an `OK` or `MISSING`
+   line per dependency. It does not say how to install anything: choose the
+   install method that fits the detected OS and the package managers that are
+   actually available (for example Homebrew on macOS; apt, dnf or pacman on
+   Linux; winget, Chocolatey or Scoop on Windows; `<python> -m pip install
+   --user` for Python packages, using the Python the preflight reported).
+   Show the user the missing items with the exact commands you plan to run,
+   ask them to confirm, then install and re-run the preflight until it exits 0.
+   Never install without that confirmation. The launcher refuses to start while
+   dependencies are missing.
 
-2. Run the capture script:
+2. **Close any running Chrome instances first** (Chrome only supports one instance per profile with remote debugging). Warn the user if Chrome is running.
+
+3. Run the capture script:
    ```
    bash scripts/line-chrome-capture.sh /tmp/line-chrome-extension-capture
    ```
@@ -28,11 +40,16 @@ Launch Chrome with network capture enabled for LINE Chrome Extension analysis.
     for `ophjlpahpchlmihnnnihgmmeilfjmjjc`, verify LINE / LY Corporation, click
     Add to Chrome, and accept the standard "Add extension" confirmation
 
-3. Tell the user:
-   - Open the LINE Chrome Extension (browser toolbar icon)
+4. Tell the user:
+   - Chrome opens on `about:blank`. Open LINE from the extension's toolbar icon
+     (puzzle-piece menu if it is not pinned), not by opening the extension's
+     `index.html` as a tab: that behaves differently from the real popup.
+     `LINE_CAPTURE_START_URL` overrides the start URL.
    - Log in manually if the capture profile is not already logged in
    - Perform whatever flow they want to analyze
-   - When done, quit Chrome completely (Cmd+Q)
+   - Do not open DevTools on the extension: LINE has `debugger;` traps that
+     pause it, and the recorder does not need DevTools
+   - When done, quit Chrome completely (all windows, not only the tab)
 
-4. Once Chrome is closed, the net-log file is ready. Continue with the
+5. Once Chrome is closed, the net-log file is ready. Continue with the
    line-analyze workflow.

.agents/commands/line-implement.md

@@ -38,11 +38,16 @@ Support Matrix group avatar changes by matching LINE Chrome Extension traffic.
    - Use a fresh `/tmp/line-chrome-extension-capture/<timestamp>-<slug>` directory with `umask 077`.
    - Do not commit raw capture files, Chrome profiles, screenshots, QR codes, or parsed JSON.
    - Do not request, store, or automate the LINE password.
+   - Do not open DevTools on the LINE extension during a capture (LINE has `debugger;` traps); the recorder does not need it. The launcher opens `about:blank`; the user opens LINE from the extension's toolbar icon.
    - The user performs LINE login manually, including email/password, QR, PIN, 2FA, and mobile approval.
    - Do not run the bridge and LINE Chrome Extension simultaneously unless the user explicitly accepts session invalidation.
    - Prefer the persistent capture Chrome profile from `scripts/line-chrome-capture.sh`. Use the user's normal Chrome profile only after explicit approval.
 
 5. **Run the capture.**
+   - Run `bash scripts/line-capture-preflight.sh` (Git Bash on Windows). If
+     anything is `MISSING`, pick install commands that fit the reported OS and
+     the package managers present, show them to the user, install only after
+     they confirm, and re-run until it passes. See `line-capture.md` step 1.
    - Check for running Chrome and `matrix-line` processes; warn before closing or stopping anything.
    - Start capture:
      ```

.agents/skills/line-implement/SKILL.md

@@ -19,6 +19,10 @@ Chrome Extension behavior.
    `pkg/line/client.go` `ExtensionVersion` and hardcoded LINE application
    header versions against the installed extension `manifest.json`.
 5. First inspect code and `docs/protocol`; capture only unknown behavior.
+   Before capturing, run `scripts/line-capture-preflight.sh` (macOS, Linux, or
+   Git Bash on Windows). For anything `MISSING`, choose install commands for
+   the detected OS and available package managers, and install only after the
+   user confirms.
 6. Treat all raw captures as secrets and work from redacted outputs whenever possible.
 
 ## Required Loop
@@ -67,6 +71,18 @@ ID `ophjlpahpchlmihnnnihgmmeilfjmjjc` (LINE, offered by LY Corporation), click
 Add to Chrome, and accept the standard "Add extension" confirmation. Do not
 accept prompts for any other extension or publisher without a new user request.
 
+During a capture, never open DevTools on the LINE extension (popup, service
+worker or `popup.html` windows): LINE has `debugger;` traps that pause it and
+show "Debugger paused in another tab". The CDP recorder attaches by itself and
+does not need DevTools. The launcher starts Chrome on `about:blank`; the user
+opens LINE from the extension's toolbar icon, because opening `index.html` as
+a tab behaves differently from the real popup.
+
+The recorder stores request bodies (binary uploads included, as size + sha256
+plus base64 in the raw capture only) and the real on-the-wire request headers.
+Redact before quoting: `scripts/redact-line-capture.py` removes tokens, MIDs,
+`picturePath` and the raw body bytes.
+
 Do not request, store, or automate LINE credentials. If the extension is not
 logged in, pause and ask the user to complete login manually in the capture
 browser, including email/password, QR, PIN, 2FA, and mobile approval.

scripts/line-capture-preflight.sh

@@ -0,0 +1,112 @@
+#!/bin/bash
+# Check that everything the LINE Chrome Extension capture needs is installed.
+# Works on macOS, Linux and Windows (Git Bash / MSYS2).
+#
+# Usage:
+#   ./scripts/line-capture-preflight.sh             # human/agent-readable report
+#   eval "$(./scripts/line-capture-preflight.sh --print-env)"   # sets CHROME and the PYTHON array
+#
+# The report prints the detected OS, one OK line per dependency found and one
+# MISSING line per dependency that is not. It deliberately does not say how to
+# install anything: the agent running it picks the install method that fits
+# the OS and the package managers available, and asks the user first.
+# Exits 1 when anything required is missing.
+#
+# Override detection with CHROME=/path/to/chrome or PYTHON=/path/to/python
+# (one executable; spaces in the path are fine).
+
+set -uo pipefail
+
+case "$(uname -s)" in
+    Darwin) OS=macos ;;
+    Linux)
+        if grep -qi microsoft /proc/version 2>/dev/null; then OS=wsl; else OS=linux; fi ;;
+    MINGW* | MSYS* | CYGWIN*) OS=windows ;;
+    *) OS=unknown ;;
+esac
+
+find_chrome() {
+    if [ -n "${CHROME:-}" ]; then
+        [ -x "$CHROME" ] && echo "$CHROME"
+        return
+    fi
+    local candidates=()
+    case "$OS" in
+        macos)
+            candidates=("/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
+                "$HOME/Applications/Google Chrome.app/Contents/MacOS/Google Chrome") ;;
+        windows)
+            candidates=("${PROGRAMFILES:-/c/Program Files}/Google/Chrome/Application/chrome.exe"
+                "${PROGRAMFILES_X86:-/c/Program Files (x86)}/Google/Chrome/Application/chrome.exe"
+                "${LOCALAPPDATA:-$HOME/AppData/Local}/Google/Chrome/Application/chrome.exe") ;;
+        *)
+            local name
+            for name in google-chrome google-chrome-stable chromium chromium-browser; do
+                command -v "$name" >/dev/null 2>&1 && { command -v "$name"; return; }
+            done ;;
+    esac
+    local path
+    for path in "${candidates[@]}"; do
+        [ -x "$path" ] && { echo "$path"; return; }
+    done
+}
+
+# Sets the PYTHON_FOUND array: the executable first, then optional arguments
+# ("py -3"). An array keeps a PYTHON override with spaces in its path in one piece.
+find_python() {
+    PYTHON_FOUND=()
+    local candidate
+    if [ -n "${PYTHON:-}" ]; then
+        "$PYTHON" -c "import sys; sys.exit(sys.version_info < (3, 8))" 2>/dev/null && PYTHON_FOUND=("$PYTHON")
+        return
+    fi
+    # "python3" on Windows can be the Microsoft Store stub, so the version check
+    # also filters out commands that exist but don't run.
+    for candidate in "python3" "python" "py -3"; do
+        # shellcheck disable=SC2206  # intentional split of "py -3"
+        local cmd=($candidate)
+        if "${cmd[@]}" -c "import sys; sys.exit(sys.version_info < (3, 8))" 2>/dev/null; then
+            PYTHON_FOUND=("${cmd[@]}")
+            return
+        fi
+    done
+}
+
+CHROME_FOUND="$(find_chrome)"
+find_python
+
+if [ "${1:-}" = "--print-env" ]; then
+    printf 'CHROME=%q\nPYTHON=(' "$CHROME_FOUND"
+    [ ${#PYTHON_FOUND[@]} -gt 0 ] && printf '%q ' "${PYTHON_FOUND[@]}"
+    printf ')\n'
+    exit 0
+fi
+
+missing=0
+ok() { echo "OK       $1"; }
+need() { echo "MISSING  $1"; missing=1; }
+
+echo "OS       $OS ($(uname -sm))"
+if [ "$OS" = wsl ]; then
+    echo "NOTE     WSL detected: use a Linux Chrome or Chromium inside WSL. Windows chrome.exe cannot use WSL paths or reach the recorder, so run from Git Bash on Windows instead."
+fi
+
+if [ -n "$CHROME_FOUND" ]; then
+    ok "Google Chrome: $CHROME_FOUND"
+else
+    need "Google Chrome (or Chromium on Linux); set CHROME=/path/to/chrome if it is installed somewhere else"
+fi
+
+if [ ${#PYTHON_FOUND[@]} -gt 0 ]; then
+    ok "Python 3.8+: ${PYTHON_FOUND[*]} ($("${PYTHON_FOUND[@]}" --version 2>&1))"
+    # line-cdp-capture.py records request bodies and SSE events through CDP.
+    if "${PYTHON_FOUND[@]}" -c "import websocket" >/dev/null 2>&1; then
+        ok "Python package websocket-client"
+    else
+        need "Python package websocket-client (pip name: websocket-client), for ${PYTHON_FOUND[*]}"
+    fi
+else
+    need "Python 3.8+"
+fi
+
+exit $missing

scripts/line-cdp-capture.py

@@ -4,7 +4,10 @@
 Connects to Chrome's remote debugging port and captures:
 - SSE EventSource messages (the critical piece net-log misses)
 - Thrift RPC request/response bodies
-- All LINE-domain request headers
+- All LINE-domain request headers, including the ones Chrome adds itself
+  (Network.requestWillBeSentExtraInfo)
+- Request bodies of every LINE request, binary ones (image uploads) included
+  (stored as size + sha256, and the bytes in base64 in this raw capture)
 
 Usage:
     # First, launch Chrome with --remote-debugging-port=9222
@@ -21,6 +24,7 @@
 import signal
 import time
 import base64
+import hashlib
 import threading
 from collections import defaultdict
 from datetime import datetime, timezone
@@ -69,6 +73,8 @@ def __init__(self, output_path: str):
         self.thrift_responses = {}  # requestId -> response body
         self.request_bodies = {}  # requestId -> request body
         self.pending_body_requests = {}  # msg_id -> (requestId, session_id)
+        self.pending_post_requests = {}  # msg_id -> requestId
+        self.extra_headers = {}  # requestId -> headers from requestWillBeSentExtraInfo
         self.sessions = {}  # sessionId -> targetInfo
 
         self.running = True
@@ -162,6 +168,12 @@ def handle_event(self, msg: dict):
                 "maxTotalBufferSize": 100 * 1024 * 1024,
                 "maxResourceBufferSize": 10 * 1024 * 1024,
             })
+            # Windows opened by the extension (e.g. the per-group popup.html)
+            # can arrive held at "waiting for debugger" even with
+            # waitForDebuggerOnStart off. Left alone they stay blank and grey
+            # out the extension with "Debugger paused in another tab".
+            if params.get("waitingForDebugger"):
+                self.send_to_session(sid, "Runtime.runIfWaitingForDebugger")
             # Track session -> target mapping
             self.sessions[sid] = target_info
             return
@@ -174,6 +186,8 @@ def handle_event(self, msg: dict):
 
         if method == "Network.requestWillBeSent":
             self._on_request(params, session_id)
+        elif method == "Network.requestWillBeSentExtraInfo":
+            self._on_request_extra_info(params)
         elif method == "Network.responseReceived":
             self._on_response(params, session_id)
         elif method == "Network.loadingFinished":
@@ -186,6 +200,27 @@ def handle_event(self, msg: dict):
 
     def handle_response(self, msg: dict):
         mid = msg.get("id")
+        if mid in self.pending_post_requests:
+            request_id = self.pending_post_requests.pop(mid)
+            result = msg.get("result", {})
+            post_data = result.get("postData")
+            if post_data is not None:
+                # Newer Chrome flags binary bodies with base64Encoded; older
+                # ones return text only, which is lossy for binary data.
+                if result.get("base64Encoded"):
+                    raw = base64.b64decode(post_data)
+                else:
+                    raw = post_data.encode("utf-8", errors="surrogateescape")
+                self.request_bodies[request_id] = {
+                    "size": len(raw),
+                    "sha256": hashlib.sha256(raw).hexdigest(),
+                    "base64": base64.b64encode(raw).decode("ascii"),
+                    "text": raw.decode("utf-8") if self._is_text(raw) else None,
+                }
+            else:
+                error = msg.get("error", {}).get("message", "unavailable")
+                self.request_bodies[request_id] = {"error": error}
+            return
         if mid in self.pending_body_requests:
             request_id, session_id = self.pending_body_requests.pop(mid)
             result = msg.get("result", {})
@@ -216,16 +251,33 @@ def _on_request(self, params: dict, session_id: str = ""):
             "session_id": session_id,
         }
 
-        # Capture POST body for Thrift calls
-        if request.get("hasPostData") and "/thrift/" in url:
-            post_data = request.get("postData", "")
-            if post_data:
-                self.request_bodies[request_id] = post_data
+        # Request bodies: the inline postData is missing for large or binary
+        # bodies (e.g. obs/uploadProfile), so always ask for it explicitly.
+        if request.get("hasPostData"):
+            target = session_id
+            if target:
+                mid = self.send_to_session(target, "Network.getRequestPostData", {"requestId": request_id})
+            else:
+                mid = self.send("Network.getRequestPostData", {"requestId": request_id})
+            self.pending_post_requests[mid] = request_id
 
         count = len(self.requests)
         if count % 10 == 1:
             print(f"  [{count} LINE requests captured]", file=sys.stderr)
 
+    @staticmethod
+    def _is_text(raw: bytes) -> bool:
+        try:
+            raw.decode("utf-8")
+        except UnicodeDecodeError:
+            return False
+        return b"\x00" not in raw
+
+    def _on_request_extra_info(self, params: dict):
+        """Headers as sent on the wire (Cookie, Origin, Sec-*, X-Line-* ...)."""
+        request_id = params.get("requestId")
+        self.extra_headers.setdefault(request_id, {}).update(params.get("headers", {}))
+
     def _on_response(self, params: dict, session_id: str = ""):
         request_id = params.get("requestId")
         if request_id not in self.requests:
@@ -295,6 +347,8 @@ def save(self):
             entry.pop("session_id", None)  # internal, don't export
             if rid in self.thrift_responses:
                 entry["response_body"] = self.thrift_responses[rid]
+            if rid in self.extra_headers:
+                entry["request_extra_headers"] = self.extra_headers[rid]
             if rid in self.request_bodies:
                 entry["request_body"] = self.request_bodies[rid]
             requests_list.append(entry)