diff --git a/.claude/hooks/ruff-check.sh b/.claude/hooks/ruff-check.sh deleted file mode 100755 index 75ecf0e..0000000 --- a/.claude/hooks/ruff-check.sh +++ /dev/null @@ -1,16 +0,0 @@ -#!/bin/bash -# Auto-run ruff after Claude edits a Python file - -INPUT=$(cat) -FILE_PATH=$(echo "$INPUT" | jq -r '.tool_input.file_path // empty') - -# Only lint Python files -if [[ "$FILE_PATH" != *.py ]]; then - exit 0 -fi - -RUFF="${HOME}/.local/bin/ruff" - -# Run ruff check (lint) with auto-fix, then format -"$RUFF" check --fix "$FILE_PATH" 2>&1 -"$RUFF" format "$FILE_PATH" 2>&1 diff --git a/.claude/plans/419-followups.md b/.claude/plans/419-followups.md deleted file mode 100644 index f040988..0000000 --- a/.claude/plans/419-followups.md +++ /dev/null @@ -1,403 +0,0 @@ -# Plan: Issue #419 — #416 PR2 /review follow-ups - -## Status - -| # | Decision | Choice | -|---|----------|--------| -| D1 | Drop obsolete items M6 (placeholder bleed-through guard) + P5 (cache inline Jinja template) | **Dropped** — PR3 already replaced the placeholder; route now uses `render_template("diagnostics.html.j2", ...)` which is cached by Flask's named-template bytecode cache | -| D2 | PR strategy | **3 PRs**: PR1 (tests + refactor bundled) → PR2 (perf, slimmed) → PR3 (docs/API contract) | -| D3 | Re-export contract for `routes/diagnostics/__init__.py` | **Enumerate explicitly** in plan with `__all__` | -| D4 | PR2 cache shape | **SUPERSEDED by D9** — original answer was inadequate; codex caught deeper issues | -| D5 | Shared `_network.read_ssid` return type | **`str \| None`** — adapt status.py callers to `ssid or "—"` at render boundary; keep `_wifi_ssid` as a thin alias per F5 | -| D6 | Time-pin in anomaly tests | **stdlib monkeypatch.setattr** — no new freezegun dep | -| D7 | Outside voice | **Ran codex** — caught 12 issues, all verified | -| D8 | Monkeypatch + re-export trap (codex F1+F2+F4) | **Update tests to patch actual binding sites** — drop the "zero modifications" promise | -| D9 | PR2 cache fate (codex F7+F8+F9, supersedes D4) | **Drop PR2 P2 entirely** — subprocess cache already does the heavy lifting; tuple cache had 3 correctness traps for unmeasured perf win | -| D10 | get_logs asc+limit semantics (codex F11) | **`order` is post-filter on newest-N** — `get_logs(limit=N, order='asc')` = take N newest, reorder ascending | - -**Plus mechanical fixes from codex absorbed directly (no fork):** - -- F3: Files moving into `routes/diagnostics/` need import depth +1 (`from ..log_buffer` → `from ...log_buffer`) -- F5: Retain `_wifi_ssid` as a thin alias in `routes/status.py` for `tests/test_control_server.py:2525` compat -- F6: `_network.read_ssid(ttl: float = STATUS_SUBPROC_TTL_S)` — status passes default, diagnostics passes `DIAG_SUBPROC_TTL_S` -- F10: `atexit.register(_shutdown_pool)` where `_shutdown_pool` dereferences the current `_JOURNAL_POOL` global (not bound-method capture) - -21 items, 3 PRs. - -## Scope - -21 informational items deferred from PR2 (#418) /review. All hygiene/perf/test/docs — no correctness bugs (yet — the plan itself had to avoid creating some). Categories: - -- Maintainability: 5 items (was 6 — M6 dropped per D1) -- Performance: 3 items (was 5 — P5 dropped per D1, P2 dropped per D9) -- Testing: 7 items -- API contract: 4 items -- Plus 1 "envelope flattening" decision item - -## PR strategy (locked) - -``` -PR1: tests + refactor (bundled) -├── New tests against new package layout -├── Test files updated to patch actual binding sites (D8) -├── routes/diagnostics.py → routes/diagnostics/ package -├── Relative-import depth +1 in all moved files (F3) -├── New helpers: control_server/_format.py, control_server/_network.py -├── _wifi_ssid alias retained in status.py (F5) -└── Docstring refresh (M5) - -PR2: performance (slimmer post-D9) -├── Module-level ThreadPoolExecutor + reset_for_tests (F10-aware atexit) -├── handler.snapshot() single-lock method -└── get_logs(order='asc') as post-filter on limit (D10) - -PR3: docs + API contract -├── SSE error-frame shape docstring -├── section_order ↔ anomalies invariant docstring -├── capacity-exceeded SSE wire event docstring -└── Envelope shape decision (document, do not flatten) -``` - ---- - -## PR1: tests + refactor - -**Branch:** `refactor/419-pr1-tests-and-package` - -### Re-export contract (per D3, expanded for codex F1) - -`routes/diagnostics/__init__.py` MUST re-export everything existing tests reach into. Verified via grep across all `tests/test_*.py`. Note: re-export alone does NOT make `monkeypatch.setattr(diagnostics, "X", ...)` redirect bindings inside `_sse.py` (D8) — that requires patching the actual binding sites. Re-export is for plain imports like `from control_server.routes.diagnostics import collect_diagnostics`. - -```python -# routes/diagnostics/__init__.py -from ._collectors import ( - # subprocess + cache - cached_subprocess, - _lazy_cache, _lazy_cache_lock, - # public API - collect_diagnostics, schema_keys, - # per-row readers - _read_iface, _read_ssid, _read_lan_ip, _read_gateway, _read_signal_dbm, - _read_timezone, _read_kernel_release, _read_cpu_temp_c, - _read_recent_log_entries, - _batched_is_active, _batched_journal_tails, - # constants - DIAG_UNITS, DIAG_JOURNAL_LINES_PER_UNIT, DIAG_SUBPROC_TTL_S, - SECTION_IDS, PRIVACY_POLICY, -) -from ._anomalies import _compute_anomalies, _recent_logs_contain_error -from ._copy_payload import build_copy_payload -from ._sse import ( - bp, - _sse_registry, _register_sse, _unregister_sse, - _sse_format, _generate_sse, -) - -__all__ = [ - # Public route surface - "bp", - "collect_diagnostics", "build_copy_payload", "schema_keys", - # Subprocess + cache (tests reach in) - "cached_subprocess", "_lazy_cache", "_lazy_cache_lock", - # Anomaly engine - "_compute_anomalies", "_recent_logs_contain_error", - # Per-row readers - "_read_iface", "_read_ssid", "_read_lan_ip", "_read_gateway", - "_read_signal_dbm", "_read_timezone", "_read_kernel_release", - "_read_cpu_temp_c", "_read_recent_log_entries", - "_batched_is_active", "_batched_journal_tails", - # SSE machinery (tests reach in) - "_sse_registry", "_register_sse", "_unregister_sse", - "_sse_format", "_generate_sse", - # Constants - "DIAG_UNITS", "DIAG_JOURNAL_LINES_PER_UNIT", "DIAG_SUBPROC_TTL_S", - "SECTION_IDS", "PRIVACY_POLICY", -] -``` - -PR1 adds `tests/test_control_server_diagnostics_reexport.py` — single test that does every import and asserts `__all__` matches. - -### Test-modification policy (per D8, supersedes earlier acceptance) - -PR1 EXPLICITLY updates 5-10 existing test files to patch the new binding sites. The Python rule: "patch where the name is LOOKED UP, not where it is DEFINED." - -Affected files + patches: - -| File | Today | Update to | -|------|-------|-----------| -| `tests/test_control_server_diagnostics.py:438` | `setattr(diag_mod, "collect_diagnostics", ...)` | `setattr("control_server.routes.diagnostics._sse.collect_diagnostics", ...)` (route's binding) | -| `tests/test_control_server_diagnostics.py:468` | same pattern | same fix | -| `tests/test_control_server_diagnostics.py:551` | `setattr(diagnostics, "_batched_journal_tails", ...)` | `setattr("control_server.routes.diagnostics._collectors._batched_journal_tails", ...)` | -| `tests/test_control_server_logs_routes.py:*` | `diagnostics._sse_registry` direct access | unchanged — direct attribute READS work fine on the re-exported name | - -Audit pattern: any `monkeypatch.setattr(diag_mod_or_diagnostics, "", ...)` where `` is called from inside a submodule needs the actual-binding-site path. - -### Relative-import depth (per F3) - -Every file moved from `routes/` to `routes/diagnostics/` needs `..` → `...` for ancestor-package imports: - -| Old | New | -|-----|-----| -| `from ..log_buffer import ...` | `from ...log_buffer import ...` | -| `from .._env import ...` | `from ..._env import ...` | -| `from .._subprocess import ...` | `from ..._subprocess import ...` | -| `from .._diagnostics_privacy import ...` | `from ..._diagnostics_privacy import ...` | -| `from .._redaction import ...` | `from ..._redaction import ...` | -| `from .status import _resolve_last_update` | `from ..status import _resolve_last_update` | - -PR1 acceptance includes `python3 -c "from control_server.routes.diagnostics import *"` succeeds without ImportError. - -### Refactor tasks - -1. **Split `routes/diagnostics.py` → `routes/diagnostics/` package** (M1) - - `routes/diagnostics/__init__.py` — re-exports per contract above + bp registration - - `routes/diagnostics/_collectors.py` — per-row readers + `collect_diagnostics` + subprocess cache + schema - - `routes/diagnostics/_anomalies.py` — anomaly thresholds + `_compute_anomalies` + `_recent_logs_contain_error` - - `routes/diagnostics/_copy_payload.py` — `build_copy_payload` - - `routes/diagnostics/_sse.py` — bp + SSE registry + generator + ALL routes (`/api/diagnostics`, `/diagnostics`, `/api/logs`, `/api/logs/stream`) - - DIAG_UNITS / DIAG_JOURNAL_LINES_PER_UNIT / DIAG_SUBPROC_TTL_S live in `_collectors.py` - - Imports updated per F3 depth fix - -2. **Extract `control_server/_format.py`** (M2) - - Move `_format_uptime` (byte-identical between status + diagnostics) - - Both call sites import from `_format` - -3. **Extract `control_server/_network.py`** (M3, per D5+F6) - ```python - # control_server/_network.py - STATUS_SUBPROC_TTL_S = 5.0 # status default - - def read_ssid(ttl: float = STATUS_SUBPROC_TTL_S) -> str | None: ... - def read_lan_ip(ttl: float = STATUS_SUBPROC_TTL_S) -> str | None: ... - def read_gateway(ttl: float = STATUS_SUBPROC_TTL_S) -> str | None: ... - def read_signal_dbm(ttl: float = STATUS_SUBPROC_TTL_S) -> int | None: ... - ``` - - status.py: keeps `_wifi_ssid` as a thin alias (`def _wifi_ssid() -> str: return read_ssid() or ""`) — preserves test_control_server.py:2525 monkeypatch surface (F5) - - `_collectors.py`: calls `read_ssid(ttl=DIAG_SUBPROC_TTL_S)` etc. - -4. **Document `os.environ` ↔ `current_app.config` precedence** (M4) - - Module docstring in `routes/diagnostics/__init__.py` - - Inline comment in `_collectors.py` at the precedence site - -5. **Refresh stale docstring** (M5) - - Kill "minimal HTML placeholder" line - - Kill "PR3 lands /api/logs*" references - - Full module docstring audit - -### Test tasks - -1. **Per-row reader tests with monkeypatched `cached_subprocess`** (T1) - - New file `tests/test_control_server_diagnostics_readers.py` - - Patches `control_server.routes.diagnostics._collectors.cached_subprocess` (actual binding) — and a parallel test imports via the re-exported path to confirm both work - - Cover: `_read_iface`, `_read_ssid`, `_read_signal_dbm`, `_read_timezone`, `_read_kernel_release`, `_batched_is_active` — 3 cases each (18 total) - -2. **Pin clock for time-based anomaly tests via monkeypatch** (T2, per D6) - - `monkeypatch.setattr("control_server.routes.diagnostics._anomalies.datetime", FakeDatetime(...))` - - `monkeypatch.setattr("control_server.routes.diagnostics._collectors.time", FakeTimeModule(...))` - - Threshold-boundary tests (at, +1ms, −1ms) - -3. **Per-row failure-path tests for malformed sources** (T3) — same as before - -4. **`build_copy_payload` edge-case tests** (T4) — same as before - -5. **Sid-shape parametrize extension** (T5) — same as before - -6. **Replace `cutoff = 0` hardcode in `test_backfill_then_hello`** (T6) — same as before - -7. **`/api/diagnostics` 500-envelope test** (T7) — same as before - -8. **Tighten deny-list positive control via env.sh** (T8) — same as before - -**Acceptance for PR1:** -- Lint clean -- All NEW tests pass -- All EXISTING tests pass (some test files have been updated per D8 to patch the new binding sites) -- `from control_server.routes.diagnostics import *` succeeds -- Re-export test asserts `__all__` matches - ---- - -## PR2: performance (slimmer post-D9) - -**Branch:** `perf/419-pr2-diagnostics` - -1. **Module-level `ThreadPoolExecutor`** (P1, F10-aware) - ```python - # routes/diagnostics/_collectors.py - _JOURNAL_POOL: ThreadPoolExecutor | None = None - - def _get_journal_pool() -> ThreadPoolExecutor: - global _JOURNAL_POOL - if _JOURNAL_POOL is None: - _JOURNAL_POOL = ThreadPoolExecutor( - max_workers=4, thread_name_prefix="diag-journal" - ) - return _JOURNAL_POOL - - def _shutdown_pool() -> None: - global _JOURNAL_POOL - if _JOURNAL_POOL is not None: - _JOURNAL_POOL.shutdown() - _JOURNAL_POOL = None - - atexit.register(_shutdown_pool) - - def reset_for_tests() -> None: - _shutdown_pool() # mirrors log_buffer.reset_for_tests - ``` - - F10 fix: `atexit` registers a function that dereferences the current global, NOT a bound method that captured the original executor - -2. **`handler.snapshot()` single-lock method** (P3) - ```python - def snapshot(self, level: str | None = None) -> tuple[list[LogEntry], int, int]: - """Atomic snapshot under one lock acquire.""" - with self._lock: - return ( - self._collect_entries_unlocked(level), - self._total_count_unlocked(level), - self._latest_seq_unlocked(), - ) - ``` - - `GET /api/logs` switches to `snapshot()` - - Tests: empty buffer, populated buffer, concurrent emit (thread test) - -3. **`order='asc'` param on `get_logs()`** (P4, per D10) - - **Semantics:** `get_logs(limit=N, order='asc')` returns the **N newest** entries, sorted oldest-first (in chronological order). Limit ALWAYS means "newest N"; order is a post-filter on that selection. - - Default stays `'desc'` (newest-first) for back-compat. - - `_generate_sse` backfill: `entries = handler.get_logs(limit=BACKFILL_N, order='asc')` — no `reversed()` needed. - - Tests: - - `get_logs(limit=4, order='asc')` on 10-entry buffer → entries[6..9] in chrono order - - default desc unchanged - - asc + level filter combo - - explicit docstring contract test - -**Acceptance:** all PR1 tests pass + new perf tests. ~5 new test cases. - ---- - -## PR3: docs + API contract - -**Branch:** `docs/419-pr3-api-contract` - -1. **Document SSE error-frame shape** (A1) -2. **Document `section_order` ↔ `anomalies` invariant** (A2) -3. **Document `capacity-exceeded` SSE wire event** (A3) — verify PR3 client backoff -4. **Keep `/api/diagnostics` envelope wrapped** (A4) — document why in `errors.py` - -**Acceptance:** docstring-only. No behavior change. - ---- - -## Cross-cutting risks (post-decisions) - -- **Monkeypatch binding-site updates (D8):** every test that monkeypatched `diag_mod.X` must now patch `diag_mod._collectors.X` or `diag_mod._sse.X`. PR1 explicitly audits + updates. -- **Relative-import depth (F3):** every moved file needs `..` → `...`. PR1 acceptance includes `import *` smoke test. -- **`_wifi_ssid` alias (F5):** preserved as a 1-line shim so existing test stays valid. -- **Cache TTL parameterized (F6):** `read_ssid(ttl=...)` so status + diagnostics keep their independent cache freshness. -- **PR2 perf is now small (D9):** the heavy cache work is dropped; PR2 has only the safe wins (pool hoist, snapshot, asc-order). - -## NOT in scope - -- Pre-#337 settings IA work (already shipped) -- Any change to `_diagnostics_privacy.py` redaction chain (locked in PR1 of EPIC #416) -- Diagnostics drawer client JS (PR3 of #416 shipped; #419 is server-side cleanup only) -- Flattening the `/api/diagnostics` envelope (would break the just-shipped PR3 client) -- PR2 P2 (the tuple cache for `(values, anomalies, copy_payload)`) — dropped per D9. If a real CPU bottleneck is later measured on a Pi Zero 2W, a targeted cache can be added with measurement. -- Distribution: no new artifact; refactoring only - -## What already exists - -- `routes/status.py:360` — `_format_uptime` (extracted to `_format.py`) -- `routes/status.py:125` — `_wifi_ssid` (kept as alias for `read_ssid()`) -- `src/control_server/log_buffer.py:390` — `reset_for_tests()` pattern (mirrored in `_collectors.py`) -- `src/control_server/_subprocess.py` — `cached_subprocess` (still the single subprocess-dedup point) -- `src/control_server/_diagnostics_privacy.py` — PRIVACY_POLICY (untouched) -- All existing tests under `tests/test_*.py` — some get monkeypatch-path updates per D8; most untouched - -## Validation per PR - -- Lint: `ruff check src/ image-gen/ tests/` -- Python tests: `python3 -m pytest tests/ --ignore=tests/test_eink_display.py -q` -- JS tests: `npm run test:js` (PR2 may touch log_buffer.py; verify SSE client JS still works) -- `/review` before each merge - -## Failure modes - -| Path | Failure mode | Test? | Error handling? | User-visible? | -|------|--------------|-------|-----------------|---------------| -| `read_ssid()` returns None (nmcli missing) | None propagates | ✅ PR1 T1 | status alias maps to ""; diagnostics keeps None | "—" via render | -| `_JOURNAL_POOL` shutdown after atexit fired | RuntimeError on submit | ⚠ Add catch+empty-list fallback in `_batched_journal_tails` | yes | Silent empty section (acceptable) | -| Monkeypatched test miss after D8 audit | False-positive test pass | ✅ PR1 re-export test + audit | n/a | Test infra only | -| `get_logs(asc, limit=N)` on empty buffer | Returns [] | ✅ explicit test | n/a | n/a | -| `handler.snapshot()` while emit holds lock | Brief contention (microseconds) | ✅ thread test | n/a | None | -| Relative-import depth wrong | ImportError at startup | ✅ `import *` smoke test in PR1 acceptance | yes — service won't start | Catastrophic (caught at deploy) | - -**No critical gaps.** (Restated honestly now — the codex pass caught the gaps the inside review missed; this revised plan addresses them.) - -## Worktree parallelization strategy - -Sequential implementation, no parallelization opportunity. PR2 depends on PR1's package layout; PR3 docs touch PR1+PR2 surfaces. - -## Implementation Tasks - -``` -PR1 — tests + refactor (largest): -- [ ] T1 (P1, human: ~3.5h / CC: ~30min) — routes/diagnostics/ package — Split + relative-import depth fix per F3 + re-export contract per D3 -- [ ] T2 (P1, human: ~30min / CC: ~5min) — control_server/_format.py — Extract _format_uptime -- [ ] T3 (P1, human: ~45min / CC: ~10min) — control_server/_network.py — Extract read_* with TTL param per F6; _wifi_ssid alias per F5 -- [ ] T4 (P1, human: ~1h / CC: ~10min) — tests/test_control_server_*.py — Audit + update monkeypatch sites per D8 -- [ ] T5 (P2, human: ~20min / CC: ~5min) — tests/test_control_server_diagnostics_reexport.py — Re-export contract test -- [ ] T6 (P1, human: ~2h / CC: ~15min) — tests/test_control_server_diagnostics_readers.py — 18 reader tests -- [ ] T7 (P1, human: ~1h / CC: ~10min) — tests/test_control_server_diagnostics_anomalies.py — Threshold tests with monkeypatched time -- [ ] T8 (P2, human: ~1h / CC: ~10min) — tests/test_control_server_diagnostics_failures.py — Malformed-source paths (5) -- [ ] T9 (P2, human: ~45min / CC: ~10min) — tests/test_control_server_diagnostics.py — Extend build_copy_payload + sid-shape -- [ ] T10 (P2, human: ~15min / CC: ~3min) — tests/test_control_server_logs_routes.py — Replace cutoff=0 hardcode -- [ ] T11 (P2, human: ~15min / CC: ~3min) — tests/test_control_server_diagnostics.py — /api/diagnostics 500-envelope test -- [ ] T12 (P2, human: ~30min / CC: ~5min) — tests/test_diagnostics_no_secrets.py — Tighten deny-list positive control -- [ ] T13 (P3, human: ~20min / CC: ~5min) — All touched modules — Docstring refresh (M5, M4) - -PR2 — perf (slimmer): -- [ ] T14 (P1, human: ~45min / CC: ~10min) — routes/diagnostics/_collectors.py — Module-level _JOURNAL_POOL + F10-aware atexit + reset_for_tests -- [ ] T15 (P1, human: ~1h / CC: ~15min) — log_buffer.py — handler.snapshot() + get_logs(order='asc') as post-filter on newest-N (D10) -- [ ] T16 (P1, human: ~1h / CC: ~15min) — tests/test_control_server_perf.py — ~5 new tests (pool, snapshot, asc-order semantics) - -PR3 — docs + API: -- [ ] T17 (P3, human: ~1h / CC: ~10min) — routes/diagnostics/_sse.py + __init__.py + errors.py — A1-A4 docstrings -``` - -Total: 17 build tasks. Down from 21 items (some merged, some absorbed mechanically). - -## TODOS.md updates - -None — this issue IS the TODO bucket. Two new follow-ups created by D9: -- **Future: measure `/api/diagnostics` CPU on Pi Zero 2W; if >50ms, add a targeted cache with real numbers.** (Filed as a comment on #419 itself, not a new issue.) - -## Completion summary - -- Step 0: Scope reduced per D1+D2 (24 items → 21 items, 4 PRs → 3 PRs) -- Architecture Review: 2 issues found, both resolved (D3, D4→superseded by D9) -- Code Quality Review: 1 issue found, resolved (D5) -- Test Review: coverage diagram produced, ~25 PLANNED branches + 0 GAP -- Performance Review: 0 new issues (P1+P3+P4 in plan; P2 dropped per D9) -- Outside voice (codex): 12 findings, all verified; 3 forks resolved (D8, D9, D10), 4 mechanical fixes absorbed (F3, F5, F6, F10), 2 already covered (F4, F1 via D8/D3 expanded) -- NOT in scope: written -- What already exists: written -- Failure modes: 0 critical gaps (the codex pass de-risked the plan) -- Parallelization: sequential, no opportunity -- Lake Score: 10/10 recommendations chose complete option (D1+D2+D3+D4+D5+D6+D7+D8+D9+D10) - -## GSTACK REVIEW REPORT - -| Review | Trigger | Why | Runs | Status | Findings | -|--------|---------|-----|------|--------|----------| -| CEO Review | `/plan-ceo-review` | Scope & strategy | 0 | — | — | -| Codex Review | `/codex review` | Independent 2nd opinion | 1 | issues_found→resolved | 12 findings, all verified, 3 absorbed as decisions (D8/D9/D10) + 4 mechanical fixes + 2 already covered | -| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 1 | CLEAR | 15 issues found (3 inside + 12 codex), 0 critical gaps remaining, 10 decisions locked | -| Design Review | `/plan-design-review` | UI/UX gaps | 0 | — | n/a — no UI scope | -| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | — | — | - -- **CODEX:** caught 5+ critical issues the inside review missed (monkeypatch-trap, anomaly-cache-staleness, asc+limit semantic trap, atexit-bound-method bug, incomplete `__all__`). All resolved or absorbed mechanically. -- **CROSS-MODEL:** strong agreement on the 3 absorbed decisions (D8/D9/D10). No unresolved tensions. -- **UNRESOLVED:** 0 -- **VERDICT:** ENG CLEARED — ready to implement. Start with PR1 on branch `refactor/419-pr1-tests-and-package`. No design review needed (server-side cleanup only). - diff --git a/.claude/settings.json b/.claude/settings.json deleted file mode 100644 index 56e3d2d..0000000 --- a/.claude/settings.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "hooks": { - "PostToolUse": [ - { - "matcher": "Edit|Write", - "hooks": [ - { - "type": "command", - "command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/ruff-check.sh", - "timeout": 30 - } - ] - } - ] - } -} diff --git a/.gitattributes b/.gitattributes index 66ec2af..77ac1c1 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,4 +1,4 @@ -# Rendered manual booklets (#404) are generated artifacts (build.sh). Chromium -# print-to-pdf embeds a creation timestamp, so bytes change every rebuild — -# mark binary so git doesn't attempt (noisy, useless) textual diffs. -docs/manual/*.pdf -diff -text +* text=auto +*.go text eol=lf +*.md text eol=lf +*.csv text eol=lf diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md deleted file mode 100644 index aaf26c9..0000000 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -name: Bug report -about: Report a problem with the clock -labels: bug ---- - -## Describe the bug - -A clear description of what's going wrong. - -## Steps to reproduce - -1. ... -2. ... -3. ... - -## Expected behavior - -What should happen instead. - -## Hardware - -- **Pi model**: (e.g., Pi Zero 2 W) -- **Display**: (e.g., Waveshare 7.5" V2) -- **OS**: (output of `cat /etc/os-release | head -2`) -- **Installation method**: (DIY install / pre-configured SD card) - -## Logs / diagnostics - -**Easiest — no shell needed:** open the Control PWA (`http://litclock.local`, or the -IP shown on the e-ink) → **Diagnostics** tab → tap **Copy support payload** (bottom of -the tab) and paste it here (or attach a screenshot). It includes version, render status, -WiFi/weather, error flags, and recent log tails, with secrets redacted. - -**If you already have a shell** (SSH ships off — enable it from the console first; -see the [Recovery guide](https://github.com/kapoorankush/litclock/blob/master/docs/recovery.md)): - -``` -journalctl -u litclock.service --since today -journalctl -u litclock-firstboot.service --since today -``` diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md deleted file mode 100644 index 84f6f05..0000000 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -name: Feature request -about: Suggest an idea for the clock -labels: enhancement ---- - -## Use case - -What problem does this solve, or what would it improve? - -## Proposed solution - -Describe how you'd like it to work. - -## Alternatives considered - -Any other approaches you've thought about. diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md deleted file mode 100644 index a2f24f6..0000000 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ /dev/null @@ -1,27 +0,0 @@ -## Summary - - - -## Changes - - - -- - -## Related Issues - - - -## Test Plan - - - -- [ ] Tested locally -- [ ] Tested on Raspberry Pi hardware -- [ ] No testing needed (documentation/config only) - -## Checklist - -- [ ] Commit messages follow [conventional commits](https://www.conventionalcommits.org/) format -- [ ] Documentation updated (if applicable) -- [ ] No secrets or credentials included diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 752afb2..01fffd3 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -1,23 +1,10 @@ version: 2 - updates: - # GitHub Actions — surface security updates and action bumps. - # Reviewed manually; rebuilds only happen on demand, not on a schedule. - - package-ecosystem: "github-actions" - directory: "/" + - package-ecosystem: gomod + directory: / schedule: - interval: "monthly" - open-pull-requests-limit: 5 - labels: - - "dependencies" - - "ci" - - # Python runtime dependencies baked into the Pi image. - - package-ecosystem: "pip" - directory: "/" + interval: weekly + - package-ecosystem: github-actions + directory: / schedule: - interval: "monthly" - open-pull-requests-limit: 5 - labels: - - "dependencies" - - "python" + interval: weekly diff --git a/.github/workflows/build-image.yml b/.github/workflows/build-image.yml deleted file mode 100644 index e23cae3..0000000 --- a/.github/workflows/build-image.yml +++ /dev/null @@ -1,284 +0,0 @@ -name: Build Image - -on: - push: - tags: ["v*"] - workflow_dispatch: - inputs: - litclock_ref: - description: "Git ref to bake into the image (tag, branch, or SHA)" - required: false - default: "master" - -jobs: - build: - name: Build Pi OS image - runs-on: ubuntu-24.04 - timeout-minutes: 180 - permissions: - contents: write # needed for creating releases - id-token: write # OIDC token for Sigstore signing (SLSA attestation) - attestations: write # post attestation to GitHub's attestation store - - steps: - - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 - with: - ref: ${{ github.event.inputs.litclock_ref || github.ref }} - submodules: true - - # Free disk space — pi-gen needs ~10 GB - - name: Free disk space - run: | - sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc \ - /usr/local/share/boost /usr/share/swift /opt/hostedtoolcache - sudo apt-get clean - df -h / - - - name: Determine version - id: version - env: - REF_TYPE: ${{ github.ref_type }} - REF_NAME: ${{ github.ref_name }} - DISPATCH_REF: ${{ github.event.inputs.litclock_ref || 'master' }} - run: | - if [[ "${REF_TYPE}" == "tag" ]]; then - VERSION="${REF_NAME}" - VERSION="${VERSION#v}" # strip leading v - REF="${REF_NAME}" - else - VERSION="dev-$(date +%Y%m%d)-${GITHUB_SHA::7}" - REF="${DISPATCH_REF}" - fi - echo "version=${VERSION}" >> "$GITHUB_OUTPUT" - echo "ref=${REF}" >> "$GITHUB_OUTPUT" - echo "sha=${GITHUB_SHA::7}" >> "$GITHUB_OUTPUT" - echo "Version: ${VERSION}, Ref: ${REF}" - - # pi-gen builds arm64 images via QEMU emulation on x86 runners - - name: Set up QEMU - run: sudo apt-get update && sudo apt-get install -y qemu-user-static - - - name: Clone pi-gen - # Pinned to the 2025-05-06 bookworm-arm64 tag. Tags are immutable in - # practice but CAN be force-updated — we verify the commit SHA to detect - # any upstream tampering. - env: - PIGEN_TAG: 2025-05-06-raspios-bookworm-arm64 - PIGEN_SHA: 49e1078c1b07ff4948d304e0149d3bc698e0006a - run: | - git clone --depth 1 --branch "$PIGEN_TAG" \ - https://github.com/RPi-Distro/pi-gen.git /tmp/pi-gen - ACTUAL=$(git -C /tmp/pi-gen rev-parse HEAD) - if [ "$ACTUAL" != "$PIGEN_SHA" ]; then - echo "ERROR: pi-gen HEAD $ACTUAL != expected $PIGEN_SHA" >&2 - echo "The upstream tag may have been force-updated. Review before bumping." >&2 - exit 1 - fi - - - name: Configure pi-gen - run: | - # Copy base config - cp pi-gen/config /tmp/pi-gen/config - - # Append build-time variables - echo "LITCLOCK_REF=${{ steps.version.outputs.ref }}" >> /tmp/pi-gen/config - echo "LITCLOCK_VERSION=${{ steps.version.outputs.version }}" >> /tmp/pi-gen/config - echo "LITCLOCK_SHA=${{ steps.version.outputs.sha }}" >> /tmp/pi-gen/config - - # Replace pi-gen's default stage3 with our custom stage - rm -rf /tmp/pi-gen/stage3 - cp -r pi-gen/stage3 /tmp/pi-gen/stage3 - - # Make scripts executable - find /tmp/pi-gen/stage3 -name "*.sh" -exec chmod +x {} + - - # Only export image from our stage - touch /tmp/pi-gen/stage2/SKIP_IMAGES - - # pi-gen's export-image unconditionally copies .bmap but bmap-tools - # is not in its Dockerfile — make the copy conditional - sed -i 's|cp "$BMAP_FILE" "$DEPLOY_DIR/"|[ -f "$BMAP_FILE" ] \&\& cp "$BMAP_FILE" "$DEPLOY_DIR/"|' \ - /tmp/pi-gen/export-image/05-finalise/01-run.sh - - - name: Stage repo for image - run: | - # Copy checkout into pi-gen so it's baked into the Docker image - # (chroot has no network/credentials to git clone). - # actions/checkout already uses depth 1, so .git is minimal. - cp -a . /tmp/pi-gen/litclock-src - # Remove pi-gen build artifacts to avoid nesting - rm -rf /tmp/pi-gen/litclock-src/pi-gen - - - name: Download quote images - # Quote images live in a GitHub Release (litclock-images-vN), pinned - # by .images-version in the repo root. Fetch them here so they get - # baked into the OS image alongside the code. The image build has no - # separate "first boot network" step, so if we don't stage them now - # the device ships with an empty images/ and every quote slot falls - # back to time-only until a power user runs update.sh. - env: - # Populate the token so download_images.sh can fetch while the - # repo is private. Becomes a no-op once the repo is public. - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - run: | - set -euo pipefail - - STAGED=/tmp/pi-gen/litclock-src - PINNED=$(tr -d '[:space:]' < "$STAGED/.images-version") - TAG="litclock-images-${PINNED}" - - # Pre-flight: confirm BOTH the tarball AND its .sha256 sidecar exist - # on the release. Missing either means the release was published - # incompletely (e.g. scripts/release_images.sh was interrupted); - # fail fast before kicking off a 30-60 minute pi-gen run. - ASSET_NAMES=$(gh release view "$TAG" --json assets --jq '.assets[].name' || true) - for NEEDED in "litclock-images.tar.gz" "litclock-images.tar.gz.sha256"; do - if ! echo "$ASSET_NAMES" | grep -qx "$NEEDED"; then - echo "::error::Release $TAG is missing required asset: $NEEDED" - echo "Publish a complete release with: scripts/release_images.sh $PINNED" - exit 1 - fi - done - - # Run the same download script the Pi uses, so we test the real - # code path. Point it at the staged repo so it writes images/ in - # the right place for pi-gen stage3 to pick up. - chmod +x "$STAGED/scripts/download_images.sh" - "$STAGED/scripts/download_images.sh" --repo-root "$STAGED" - - # Marker check: the download must have written the installed-version - # marker. Missing marker = graceful-exit-0 fired somewhere (network, - # 404, SHA mismatch). In CI we want a hard failure. - if [ ! -f "$STAGED/images/.installed-version" ]; then - echo "::error::download_images.sh ran but images/.installed-version was not created" - exit 1 - fi - - # Count floor: guard against a malformed-but-sha-matching tarball - # that only contains a few files. Expected corpus is 9500+ files. - COUNT=$(find "$STAGED/images" -type f -name '*.png' | wc -l) - MIN_COUNT=8000 - if [ "$COUNT" -lt "$MIN_COUNT" ]; then - echo "::error::Only $COUNT PNGs staged (expected >= $MIN_COUNT) — release tarball is likely corrupt" - exit 1 - fi - echo "Staged $COUNT quote images from $TAG into the pi-gen input directory" - - - name: Build image - run: | - cd /tmp/pi-gen - ./build-docker.sh - - # Smoke testing happens IN-CHROOT during pi-gen build at - # pi-gen/stage3/05-smoke-test/00-run.sh. If the venv is broken, a - # Python dep fails to install, or a systemd unit has invalid syntax, - # stage3 fails and no .img is ever produced. Post-export mount-based - # smoke testing was tried and removed — it produced false alarms from - # GH Actions loop-device quirks. See PR #202 and issue #114 for - # context and the community discussions linked there. - - - name: Compress image - id: compress - run: | - DEPLOY=/tmp/pi-gen/deploy - - # pi-gen outputs a zip containing the .img — extract it - ZIP=$(ls "${DEPLOY}"/image_*.zip 2>/dev/null | head -1) - if [ -z "$ZIP" ]; then - echo "ERROR: No image zip found in ${DEPLOY}/" - ls -la "${DEPLOY}/" 2>/dev/null || echo "deploy/ directory does not exist" - exit 1 - fi - unzip -o "$ZIP" -d /tmp - IMG=$(ls /tmp/*.img 2>/dev/null | head -1) - if [ -z "$IMG" ]; then - echo "ERROR: No .img file found after extracting $ZIP" - exit 1 - fi - - IMG_NAME="litclock-${{ steps.version.outputs.version }}.img" - - # Rename to versioned name - mv "${IMG}" "/tmp/${IMG_NAME}" - - # Compress with xz (parallel) - xz -9 -T0 "/tmp/${IMG_NAME}" - - # Generate checksums - cd /tmp - sha256sum "${IMG_NAME}.xz" > "${IMG_NAME}.xz.sha256" - - echo "img=/tmp/${IMG_NAME}.xz" >> "$GITHUB_OUTPUT" - echo "sha=/tmp/${IMG_NAME}.xz.sha256" >> "$GITHUB_OUTPUT" - echo "img_name=${IMG_NAME}.xz" >> "$GITHUB_OUTPUT" - - echo "Compressed image:" - ls -lh "/tmp/${IMG_NAME}.xz" - - # SLSA build provenance: sign the compressed image so downstream users - # can verify it was built by this workflow from a specific commit. - # Verify with: gh attestation verify litclock-.img.xz --owner kapoorankush - # Skipped on private repos — GitHub's attestation API rejects user-owned - # private repos. Will start working automatically when the repo goes public. - - name: Attest build provenance - if: ${{ !github.event.repository.private }} - uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373 # v4.1.1 - with: - subject-path: ${{ steps.compress.outputs.img }} - - # On manual dispatch: upload as a dev pre-release - # (GitHub Releases assets don't count toward Actions storage quota, - # unlike artifacts which accrue GB-hours against the 500 MB free tier) - - name: Upload dev build - if: github.ref_type != 'tag' - env: - GH_TOKEN: ${{ github.token }} - run: | - TAG="${{ steps.version.outputs.version }}" - # Delete existing release with this tag (idempotent re-runs) - gh release delete "${TAG}" --yes --cleanup-tag 2>/dev/null || true - # Clean up dev releases older than 7 days - gh release list --json tagName,createdAt \ - --jq '.[] | select(.tagName | startswith("dev-")) | select((.createdAt | fromdateiso8601) < (now - 604800)) | .tagName' \ - | xargs -I{} gh release delete {} --yes --cleanup-tag 2>/dev/null || true - gh release create "${TAG}" \ - --prerelease \ - --title "Dev build ${{ steps.version.outputs.version }}" \ - --notes "Dev build from \`${{ steps.version.outputs.ref }}\` ($(date -u +%Y-%m-%dT%H:%M:%SZ)). Auto-generated, not for production use." \ - --target "${{ github.sha }}" \ - "${{ steps.compress.outputs.img }}" \ - "${{ steps.compress.outputs.sha }}" - - # On tag push: create GitHub Release - # - # --target is REQUIRED: without it, GitHub stores the default branch - # name (e.g. "master") in the Release's target_commitish field. The - # #209 auto-update resolver reads tag_name and resolves it via - # git rev-list, but downstream tooling (and any future resolver that - # trusts target_commitish) needs honest metadata. Always stamp the - # actual commit SHA so the Release points where everyone thinks it does. - - name: Create release - if: github.ref_type == 'tag' - env: - GH_TOKEN: ${{ github.token }} - REF_NAME: ${{ github.ref_name }} - IMG: ${{ steps.compress.outputs.img }} - SHA: ${{ steps.compress.outputs.sha }} - # Idempotent: if a release with this tag already exists (someone - # pre-cut it manually with curated notes, or a prior workflow run - # got this far and partial-failed), upload the assets to it with - # --clobber instead of trying to recreate it. `gh release create` - # exits 1 on collision, which used to discard a freshly-built - # 1.4GB image alongside a 71-min build. See #242 for the postmortem. - run: | - if gh release view "${REF_NAME}" --json id >/dev/null 2>&1; then - echo "Release ${REF_NAME} already exists — uploading assets with --clobber" - gh release upload "${REF_NAME}" "${IMG}" "${SHA}" --clobber - else - gh release create "${REF_NAME}" \ - --title "LitClock ${{ steps.version.outputs.version }}" \ - --target "${{ github.sha }}" \ - --generate-notes \ - "${IMG}" \ - "${SHA}" - fi diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..743a09a --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,26 @@ +name: CI + +on: + push: + pull_request: + +permissions: + contents: read + +jobs: + test: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + - uses: actions/setup-go@v7 + with: + go-version-file: go.mod + cache: true + - name: Check formatting + run: test -z "$(gofmt -l .)" + - name: Test + run: go test -race ./... + - name: Vet + run: go vet ./... + - name: Build + run: go build ./cmd/litclock diff --git a/.github/workflows/corpus-integrity.yml b/.github/workflows/corpus-integrity.yml deleted file mode 100644 index 6589458..0000000 --- a/.github/workflows/corpus-integrity.yml +++ /dev/null @@ -1,132 +0,0 @@ -name: Corpus integrity - -# #299 enforcement layer. -# -# Goal: prevent CSV edits from landing without a matching image release. The -# bug we're guarding against is "edit the CSV, forget to re-run corpus_edit -# ship, merge anyway, ship desynced PNGs to every Pi" — the exact path that -# produced the 940 at-risk buckets observed on test Pi 192.168.2.132 in May -# 2026. Local pre-commit hooks were considered and rejected (codex-flagged -# bypassable via web edits, squash merges, --no-verify, missing-install). -# Required CI is the only enforceable gate. -# -# Trigger: PRs touching image-gen/litclock_annotated.csv only. Other PRs -# skip this workflow entirely (paths filter). -# -# Checks (in order; first failure aborts): -# 1. .images-version was bumped vs base (otherwise no new release was cut). -# 2. The expected litclock-images-vN release exists on GitHub. -# 3. The release publishes manifest.json as a top-level asset. -# 4. manifest.corpus_hash == sha1(PR's image-gen/litclock_annotated.csv). -# -# Auth: GITHUB_TOKEN (ambient, no secrets needed for own-repo release reads). - -on: - pull_request: - paths: - - 'image-gen/litclock_annotated.csv' - branches: [master] - -permissions: - contents: read - -jobs: - verify: - name: Verify CSV matches images release manifest - runs-on: ubuntu-latest - steps: - - name: Check out PR head - uses: actions/checkout@v7 - with: - fetch-depth: 0 - # Explicit repository pin makes fork-PR runs work: with `pull_request` - # (not pull_request_target) the runner clones the base by default, - # but the fork's head SHA only exists on the fork. Pin both. - repository: ${{ github.event.pull_request.head.repo.full_name }} - ref: ${{ github.event.pull_request.head.sha }} - - - name: Resolve PR and base .images-version - id: versions - run: | - set -euo pipefail - if [ ! -f .images-version ]; then - echo "::error file=.images-version::.images-version missing on PR head" >&2 - exit 1 - fi - PR_VERSION=$(tr -d '[:space:]' < .images-version) - BASE_SHA="${{ github.event.pull_request.base.sha }}" - BASE_VERSION=$(git show "$BASE_SHA:.images-version" 2>/dev/null | tr -d '[:space:]' || true) - echo "PR_VERSION=$PR_VERSION" >> "$GITHUB_OUTPUT" - echo "BASE_VERSION=$BASE_VERSION" >> "$GITHUB_OUTPUT" - echo "Base .images-version: ${BASE_VERSION:-}" - echo "PR .images-version: ${PR_VERSION}" - if [ -z "$PR_VERSION" ]; then - echo "::error::PR .images-version is empty" >&2 - exit 1 - fi - # Mirror release_images.sh's format check so a malformed pin fails - # with a clear message instead of a confusing "release not found". - if ! [[ "$PR_VERSION" =~ ^v[0-9]+$ ]]; then - echo "::error::PR .images-version $PR_VERSION is not in vN format (must match ^v[0-9]+$)." >&2 - exit 1 - fi - if [ "$PR_VERSION" = "$BASE_VERSION" ]; then - echo "::error::CSV changed but .images-version was not bumped (still $PR_VERSION)." >&2 - echo "::error::Re-ship via: python3 image-gen/corpus_edit.py ship \"\"" >&2 - exit 1 - fi - - - name: Locate release and download manifest.json - id: fetch - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - GH_REPO: ${{ github.repository }} - PR_VERSION: ${{ steps.versions.outputs.PR_VERSION }} - run: | - set -euo pipefail - RELEASE_TAG="litclock-images-${PR_VERSION}" - echo "Looking up release: $RELEASE_TAG (repo: $GH_REPO)" - - # --repo forces base-repo lookups even when running in a fork checkout. - if ! gh release view "$RELEASE_TAG" --repo "$GH_REPO" --json tagName >/dev/null 2>&1; then - echo "::error::Release $RELEASE_TAG not found in $GH_REPO." >&2 - echo "::error::CSV change must be paired with a matching litclock-images-${PR_VERSION} release." >&2 - echo "::error::Cut the release via scripts/release_images.sh ${PR_VERSION} (or rerun corpus_edit.py ship)." >&2 - exit 1 - fi - - if ! gh release download "$RELEASE_TAG" --repo "$GH_REPO" --pattern manifest.json --dir "$RUNNER_TEMP"; then - echo "::error::Release $RELEASE_TAG has no manifest.json asset." >&2 - echo "::error::Re-cut the release with the post-#299 release_images.sh." >&2 - exit 1 - fi - echo "MANIFEST_PATH=$RUNNER_TEMP/manifest.json" >> "$GITHUB_OUTPUT" - - - name: Compare manifest.corpus_hash to PR CSV - env: - MANIFEST_PATH: ${{ steps.fetch.outputs.MANIFEST_PATH }} - run: | - set -euo pipefail - MANIFEST_HASH=$(python3 -c ' - import json, sys, os - with open(os.environ["MANIFEST_PATH"]) as f: - m = json.load(f) - h = m.get("corpus_hash") - if not h: - sys.stderr.write("manifest.json has no corpus_hash field\n") - sys.exit(1) - print(h) - ') - ACTUAL_HASH=$(python3 -c ' - import hashlib, sys - with open("image-gen/litclock_annotated.csv", "rb") as f: - print(hashlib.sha1(f.read()).hexdigest()) - ') - echo "Manifest corpus_hash: $MANIFEST_HASH" - echo "PR CSV corpus_hash: $ACTUAL_HASH" - if [ "$MANIFEST_HASH" != "$ACTUAL_HASH" ]; then - echo "::error::Corpus hash mismatch — release was built from a different CSV than this PR." >&2 - echo "::error::Re-ship via: python3 image-gen/corpus_edit.py ship \"\"" >&2 - exit 1 - fi - echo "OK — release manifest matches PR CSV." diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml deleted file mode 100644 index 5f1df4a..0000000 --- a/.github/workflows/lint.yml +++ /dev/null @@ -1,80 +0,0 @@ -name: Lint - -on: - push: - branches: [master] - pull_request: - branches: [master] - -jobs: - ruff: - name: Python lint (ruff) - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v7 - - uses: astral-sh/ruff-action@v3 - with: - args: check src/ image-gen/ scripts/ - - shellcheck: - name: Shell lint (shellcheck) - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v7 - - name: Run shellcheck - # `find ... -print0 | xargs -0` is robust to tree shape changes — - # the prior bash glob `pi-gen/stage3/**/**/*.sh` silently broke when - # the only 3-level shell file (pi-gen/stage3/02-configure-system/ - # files/wifi-watchdog.sh) was promoted to scripts/ during M5. - # Without `nullglob` set, the unmatched glob was passed literally - # to shellcheck which failed with "openBinaryFile: does not exist". - run: | - find scripts pi-gen docs/manual -name '*.sh' -type f -print0 \ - | xargs -0 shellcheck --severity=warning - - test: - name: Unit tests (pytest) - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v7 - - uses: actions/setup-python@v6 - with: - python-version: "3.11" - cache: pip - - name: Install dependencies - run: | - pip install -r requirements.txt -r requirements-dev.txt - - name: Run tests - run: pytest - - pip-audit: - name: Dependency audit (pip-audit) - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v7 - - uses: actions/setup-python@v6 - with: - python-version: "3.11" - - name: Install pip-audit - run: pip install pip-audit - - name: Audit dependencies - run: pip-audit -r requirements.txt - - js-test: - # JS unit tests for src/control_server/static/js/* via vitest + jsdom - # (#338). Dev/CI only — never installed on the Pi. Runs in parallel with - # ruff/shellcheck/pytest/pip-audit and shows as a separate required-check - # on PRs. Branch protection is a separate manual repo setting; this job - # being red won't server-side block merge until that toggle is flipped. - name: Unit tests (vitest) - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v7 - - uses: actions/setup-node@v6 - with: - node-version: "20" - cache: npm - - name: Install JS dev dependencies - run: npm ci - - name: Run JS tests - run: npm run test:js diff --git a/.gitignore b/.gitignore index fd47462..2e66753 100644 --- a/.gitignore +++ b/.gitignore @@ -1,80 +1,12 @@ -__pycache__ -*.__pycache__ -*.pyc -venv/ -screen-output* -env.sh -env.sh.bak -*.log -.vscode -*.pickle -credentials* -*.json -# #338: package.json + lockfile MUST be tracked despite the *.json catch-all -# above. Keep these `!` unignores adjacent to the rule that would otherwise -# swallow them so the relationship stays obvious to future readers. -!package.json -!package-lock.json -*.bin - -# #338: vitest + jsdom dev dependencies. Dev/CI only — never on the Pi. -node_modules/ -.vitest-cache/ - -# Secrets / API keys — never commit +# Go build and test output +/litclock +*.test +*.out +coverage.txt + +# Editors and local environment +.DS_Store +.idea/ +.vscode/ .env .env.* -*.env -*.pem -*.key -*_api_key* -anthropic_key* -test.svg -.certs/ -.pip-packages-hash - -# Claude Code - track project config and hooks, but not local/personal settings -!.claude/settings.json -!.claude/hooks/ -.claude/settings.local.json - -# image-gen backup/intermediate files -image-gen/*.backup* -image-gen/*.pre* -image-gen/coverage_report.txt -image-gen/literaryclock_*.csv -image-gen/nsfw_flagged_for_review.csv -image-gen/parser_failures.txt -image-gen/downloads/ - -# #192 audit run artifacts — keep the tooling + gold set, drop per-run outputs -image-gen/audit_fails.csv -image-gen/audit_all.csv -image-gen/audit_fails.meta.json -image-gen/audit_fails.progress.jsonl -image-gen/audit_fails.batch_id -image-gen/gold_results.csv -image-gen/gold_results.progress.jsonl -image-gen/pilot*.csv -image-gen/pilot*.progress.jsonl - -# pi-gen build artifacts -pi-gen/work/ - -# Internal agent-planning scaffolding, browse logs, deploy reports — never commit -.gstack/ - -# Quote images live in GitHub Releases (issue #82). The .installed-version marker -# inside images/ tracks which release is extracted on disk — keep that tracked. -/images/ -!images/.installed-version - -# Scratch dirs and locks used by scripts/download_images.sh (transient). -.litclock-images-staging.* -.litclock-images.lock -.claude/scheduled_tasks.lock - -# Internal planning docs — maintainer-local, excluded from the public repo (#82 PII decision 2026-07-12) -DESIGN.md -PRD-LitClock-Control-PWA.md -PLAN-LitClock-Control-PWA.md diff --git a/.gitleaks.toml b/.gitleaks.toml deleted file mode 100644 index 7f647af..0000000 --- a/.gitleaks.toml +++ /dev/null @@ -1,11 +0,0 @@ -# Allowlist for gitleaks: these test suites contain DELIBERATE fake secrets -# (dummy ghp_/ghs_/sk_live_ tokens, a dummy OpenSSH key, PSK=hunter2foobar) -# used as fixtures to prove the diagnostics redaction strips secret shapes. -# Scoped by path only — a real secret anywhere else still fails the scan. -[allowlist] -description = "Deliberate fake secrets in redaction/no-secrets test fixtures" -paths = [ - '''tests/test_redaction\.py''', - '''tests/test_diagnostics_no_secrets\.py''', - '''tests/test_control_server_diagnostics\.py''', -] diff --git a/.gitmodules b/.gitmodules deleted file mode 100644 index 8f916ac..0000000 --- a/.gitmodules +++ /dev/null @@ -1,4 +0,0 @@ -[submodule "lib/e-Paper"] - path = lib/e-Paper - url = https://github.com/waveshare/e-Paper.git - ignore = dirty diff --git a/.images-version b/.images-version deleted file mode 100644 index 02a819f..0000000 --- a/.images-version +++ /dev/null @@ -1 +0,0 @@ -v7 diff --git a/3d-models/README.md b/3d-models/README.md deleted file mode 100644 index 943da6c..0000000 --- a/3d-models/README.md +++ /dev/null @@ -1,23 +0,0 @@ -# LitClock case — 3D print files - -Print-ready STLs for the LitClock case. The design is Arthur Gassner's -[Time Teller](https://timeteller.arthurgassner.com) case (v3), lightly modified -for LitClock. Licensed **CC BY** — see [NOTICE.md](../NOTICE.md) for full -attribution. - -| File | vs. Time Teller v3 | -|------|--------------------| -| `top-front.stl` | unmodified (included so you can print everything from here) | -| `top-back-with-notch.stl` | modified — adds a notch | -| `bottom-with-notch.stl` | modified — adds a notch | - -Print all three in PLA. For threaded inserts, the USB-C adapter mount, assembly -steps, and the full parts list, see -[Hardware Assembly](../docs/hardware-assembly.md). - -The original, unmodified design (STL + editable SolveSpace source) is published -by the author on -[GitHub](https://github.com/arthurgassner/timeteller/tree/main/3d-models), -[Printables](https://www.printables.com/model/1398618-timeteller-a-literature-clock), -[Thingiverse](https://www.thingiverse.com/thing:7130877), and -[MakerWorld](https://makerworld.com/en/models/1744549-timeteller-telling-the-time-through-quotes). diff --git a/3d-models/bottom-with-notch.stl b/3d-models/bottom-with-notch.stl deleted file mode 100644 index 6a82f69..0000000 Binary files a/3d-models/bottom-with-notch.stl and /dev/null differ diff --git a/3d-models/top-back-with-notch.stl b/3d-models/top-back-with-notch.stl deleted file mode 100644 index ce0915e..0000000 Binary files a/3d-models/top-back-with-notch.stl and /dev/null differ diff --git a/3d-models/top-front.stl b/3d-models/top-front.stl deleted file mode 100644 index 7d31e58..0000000 Binary files a/3d-models/top-front.stl and /dev/null differ diff --git a/CHANGELOG.md b/CHANGELOG.md deleted file mode 100644 index d322047..0000000 --- a/CHANGELOG.md +++ /dev/null @@ -1,389 +0,0 @@ -# Changelog - -All notable changes to LitClock are documented here. Format loosely follows [Keep a Changelog](https://keepachangelog.com/) — dates are ISO 8601. - -## [v0.220.0] - 2026-07-15 - -### Fixed -- **The Control PWA can no longer crash-loop itself into a brick over the port-80 floor** (#15, #16). control_server binds port 80 as the non-root `pi` user, which needs `net.ipv4.ip_unprivileged_port_start<=80`. A field incident showed the persistent `/etc/sysctl.d` drop-in can silently fail to apply during an OTA (the install error was swallowed, and the update's only warning checked the live sysctl value, not the persisted file) — so the update reported success, the next reboot reverted the floor to 1024, and the service crash-looped on `EACCES` with no recovery path for a keyboard-less owner. Now `litclock-control.service` self-heals the floor with an `ExecStartPre` that re-asserts it as root on every start (idempotent, ~1ms), and `update.sh` verifies the drop-in by content and warns loudly (with the exact re-install command) if it is missing or stale. Reviewed by two independent adversarial passes. -- **Personalized gift welcome messages no longer get their tail cut off** (#17). The "Prepare for Gifting" welcome splash rendered the message at a fixed 48pt capped at two lines and ellipsis-truncated anything longer — "May it always be a good ti…" on a real gift. The splash now auto-fits: it shrinks the font down a `(size, lines)` ladder (48→44→38→32→28pt) and picks the largest size that shows the whole message with no ellipsis, only truncating a message far past the 280-character input cap. Short greetings still render at 48pt exactly as before. -- **DIY installs no longer abort on a never-copied systemd unit** (#14). `scripts/install.sh` enabled `litclock-reresolve-location.service` (and expected `litclock-prepare-for-gift.service`) without ever copying them to `/etc/systemd/system`; under `set -e` the `systemctl enable` aborted the whole install. Both units are now copied, with a drift guard requiring every unit in `systemd/` to be wired into the installer. Flashed images were unaffected. - -### Added -- **Captive-portal probe logging now records the response side** (#11). Each `CAPTIVE-PROBE` line logs which branch answered (CNA bridge / redirect), the HTTP status, byte count, and the full user-agent, so a "the setup page didn't auto-open" report can be diagnosed from the journal instead of guessing. Diagnostic groundwork for the iOS captive-sheet investigation. - -### Changed -- **README rewritten around the builder's journey** (#12, #13): a real product photo as the hero, a numbered Build → Living-with-it → Give-one-away → Under-the-hood structure, phone screenshots of the control app, and the first-boot walkthrough — so someone deciding to build one can follow the order they'd actually do it in. - -## [v0.219.0] - 2026-07-13 - -### Security -- **pillow 12.2.0 → 12.3.0** — picks up the fixes for PYSEC-2026-2253 through PYSEC-2026-2257 (five advisories published 2026-07-13 against 12.2.0; caught by the pip-audit CI gate). pillow renders every frame on the device, so this rides the next release. - -### Added -- **"Factory reset" button in the Control PWA's System tab** (#510, #511). Erases everything — all settings and saved WiFi — and reboots into the first-time setup experience, distinct from "Reset WiFi" (which keeps settings). Guarded by the same confirm-sheet + CSRF flow as the other System actions; the env wipe is fail-closed (`--strict-env-wipe`), so a partial wipe aborts rather than leaving a half-reset clock. -- **Print-ready case STLs in `3d-models/`** (#521). The maintainer's lightly modified Time Teller v3 case parts (notches added to the top-back and bottom pieces; top-front unmodified) — Arthur Gassner's design, CC BY per his Printables listing, attributed in NOTICE.md. Builders no longer need to fetch the case from a separate site. -- **Owner recovery guide** (`docs/recovery.md`, #509). The SSH-off posture's companion: console access with default credentials, enabling SSH from the SD card's boot partition, reset-to-first-boot, and the read-only Diagnostics tab. - -### Changed -- **The appliance image no longer auto-updates OS packages** (#508). A fielded/gift LitClock must never apt-upgrade behind the owner's back — a surprise kernel or driver bump could break the e-ink stack with nobody at the keyboard. The pi-gen image zeroes the apt periodic knobs and masks `apt-daily{,-upgrade}.timer`. LitClock's own weekly self-updater (`litclock-update.timer`) is unaffected. -- **README rewritten to describe the current product** (#516–#520): the real EPIC-#383 setup flow (WiFi-only hotspot form, IP-geo auto-config, handoff splash), a Control App section with phone screenshots, a first-boot visual walkthrough, the quick-start booklet links, the exact Waveshare panel purchase link, and an auto-playing one-minute tour GIF (`docs/media/litclock-intro.gif`) rendered from the real device code paths. -- **PII sweep ahead of the public release** (#82). The gift-message placeholder in the System tab no longer carries the maintainer's real first name ("Love, Alexis" is the new example — same 32-char length, so the e-ink wrap docs/tests it anchors are unchanged). Sample weather coordinates in `README.md` / `env.sh.sample` are rounded to city-block precision (`30.27, -97.74`), and the full-precision latitude that appeared in redaction test fixtures, a code comment, and an earlier CHANGELOG entry is replaced with an arbitrary example value — the repo's own diagnostics-redaction standard (2-decimal rounding) now applies to the repo itself. Also: `docs/building-image.md` examples updated from stale CalVer tags to the current SemVer scheme, and `pi-gen/config` now documents the intentional default-creds + SSH-off posture (#387, closed). -- **Example city decoupled from the maintainer's location** (#82 PII sweep, part 2). Every "Frisco, TX" / zip-75033 reference — README and `env.sh.sample` samples, QA docs, code comments, and ~100 test fixtures including the `is_daytime` solar regression suite — swapped to Austin, TX (30.27, -97.74; same timezone and similar latitude, so the pinned day/night assertions hold unchanged). A test comment identifying "the founder's Pi's location" reworded. The internal planning docs (`DESIGN.md`, `PRD-`/`PLAN-LitClock-Control-PWA.md`) are now gitignored ahead of their exclusion from the public repo. - -## [v0.218.0] - 2026-07-11 - -### Changed -- **BREAKING: the Control PWA is now at `http://litclock.local` / `http://` — no port to type** (#343). control_server moved from port 8443 to **port 80**, so the URL a recipient scans from the e-ink QR, taps from the mDNS bookmark, or types by hand no longer carries a `:port` — a smaller, less technical-looking address for non-technical gift users. (This supersedes #343's original 8080 proposal: 8080 would still have shown a port; only 80 removes it.) The port is bound by the non-root `pi` service account via a one-line sysctl drop-in (`/etc/sysctl.d/30-litclock-unprivileged-ports.conf` → `net.ipv4.ip_unprivileged_port_start=80`), deliberately chosen over `AmbientCapabilities=CAP_NET_BIND_SERVICE`: the `litclock-control` unit runs `NoNewPrivileges=no` + setuid `sudo` for the reboot/poweroff actions, and adding a capability directive to a `User=pi` unit has repeatedly flipped the kernel `NoNewPrivs` bit and broken setuid sudo on this hardware — a sysctl never touches that wiring (and stays safe once #82 drops the `010` blanket sudo, when setuid `020` sudo becomes the only reboot path). `setup_server.py` stays HTTPS on 8443 (the iOS captive trust dance needs TLS) — the two are now different ports for different phases, no longer sharing 8443. The port + URL are a single source of truth (`src/control_url.py::control_base_url`, which omits `:80`), shared by control_server (bind) and the e-ink clock (QR); the mDNS probe in `status.js` derives the port from the live origin — so the QR target, the bookmark switch, and the actual listen port can never drift. Installed on fresh flash (pi-gen), manual install (`install.sh`), and OTA (`update.sh` applies the sysctl live before restarting control_server). - - **Migration (one-time, recoverable):** on OTA update the e-ink QR refreshes to the port-less URL within one paint cycle. A PWA already pinned to a phone home screen, or a saved `:8443` bookmark, will break (the origin includes the port) — **re-scan the QR (it's right there on the clock) and re-pin / re-bookmark.** No data is lost. - -## [v0.217.0] - 2026-07-10 - -### Added -- **Diagnostics: a "Download full logs" export for shell-less support** (#416 follow-up). The Control PWA's diagnostics "Copy support payload" already hands a helper the system state + a **3-line** journal preview per unit — deliberately shallow so it never blocks first paint. That's often too thin to actually debug a failing unit. New `GET /api/diagnostics/support-logs` assembles a single downloadable `text/plain` bundle: the same redacted system payload plus a **~50-line** journal tail per unit across the `DIAG_UNITS` allowlist, so a non-technical owner can hand over one file (or paste) that carries enough context — no SSH. Off the page/poll critical path; deep reads use a **distinct, line-count-scoped cache key** so they never serve or poison the 3-line page-preview cache; a wall-clock budget bounds the serial `journalctl` loop and appends an explicit *named-units* truncation note rather than silently dropping units; `no-store` + redacted (secrets in journal lines are stripped). The existing per-unit `GET /api/diagnostics/journal` also gained a capped `?lines=N` param (default 3, max 200) for pulling a deeper tail for a single unit. Surfaced as a plain `` link beside the Copy button, so it works with no JavaScript. - -- **LKG auto-revert: the clock now self-heals a bad over-the-air update without SSH or an SD reflash** (#209 follow-up). #209 shipped the *writer* (`litclock-lkg.service` records the last SHA that actually painted to `/var/lib/litclock/lkg-sha`, heartbeat-gated); this ships the *consumer*, `litclock-bootcheck.service`. It answers one question per boot — "did the clock paint a frame since this boot?" — via the tmpfs render heartbeat (network-independent, so a merely-offline clock is never touched). On a persistent failure it self-heals: fail 1 and 2 auto-reboot to retry (a single failed render may be transient), and on fail 3 it pins the last-known-good SHA and routes recovery **back through `update.sh`** in a new rollback mode, which does the *complete* install (git + submodules + venv hash-gate + units→`/etc` + sudoers + dispatcher + smoke) rather than a partial `git reset` — so a dependency- or unit-level bad update is actually recoverable, not just a code-only one. If the recovered code also fails it gives up (a persistent `bootcheck-gave-up` marker + best-effort "please re-flash" splash) instead of looping, bounding the whole sequence to ~4 reboots. Two supporting fixes make the recovery target reliable: `update.sh` **no longer clears `lkg-sha`** at the start of an update (the heartbeat-gated writer replaces it only once new code paints, so `lkg-sha` always points at the last code that actually rendered — a dead-on-arrival update can never blank the recovery target, closing the DOA gap), and a `blocked-sha` suppression stops the weekly timer from re-installing the exact release bootcheck just reverted from until a newer release supersedes it. Runs as `pi` using only the `systemctl reboot` + `systemctl start --no-block litclock-update.service` grants already in `sudoers/020_litclock-control` — no new privilege, works after #387 drops `010`. Hard prerequisite for #82 (public release: SSH is off, so a recipient's bricked clock would otherwise need a physical reflash). Design reshaped in `/plan-eng-review` + Codex outside-voice (plan: `docs/plans/lkg-bootcheck-plan.md`). - -### Security -- **Closed the pi→root escalation paths so dropping the blanket `010_pi-nopasswd` sudo is safe** (#387, prerequisite for #82). Each was harmless while `010` grants `pi` NOPASSWD:ALL, but becomes a live hole the moment the public image ships with only the scoped `020_litclock-control` allowlist. **(tz-wrapper)** `geocoding.set_system_timezone` (the IP-geo + A18 browser-tz fallback) ran `sudo timedatectl set-timezone `, which `020` could only authorize via a `set-timezone *` glob — a hole letting `pi` set the clock to any string reaching the resolver. It now calls a new root-owned wrapper `/usr/local/lib/litclock/litclock-set-timezone` that re-validates the timezone against the kernel zoneinfo list *in root-owned code* (a metachar pre-filter + `grep -Fxq`, and `timedatectl` resolved via PATH — never an env-var binary override that could cross `sudo`); `020` authorizes just that fixed path. **(C1)** the root NetworkManager dispatcher invoked the **pi-writable** `/home/pi/litclock/scripts/litclock-mark-collected.sh` — pi→root on the next DHCP event once `010` drops. It now runs a root-owned copy in `/usr/local/lib/litclock/` (no fallback to the repo copy). **(prepare-for-gift)** `litclock-prepare-for-gift.service` (root, startable by `pi` via `020`) exec'd the **pi-writable** `reset-setup.sh` — pi could edit it and get arbitrary root code. It now execs a root-owned copy (with its sourced `lib/state.sh` shipped root-owned alongside), and the gift-message step runs the root-owned system `/usr/bin/python3` instead of the pi-writable venv interpreter. **(NTP grant)** `first-boot.sh`/`install.sh` run `sudo timedatectl set-ntp true`, which `020` didn't authorize — a fresh clock would silently never sync time once `010` drops; added as a fixed-argv grant. **(C2)** the dispatcher + mark-collected writer run as root but write into the `0755 pi pi` `/run/litclock` and `/var/lib/litclock` dirs; both now refuse to follow a symlink at the target and stage via `mktemp` (O_EXCL) instead of a guessable `$MARKER.tmp.$$` path. (A residual sub-millisecond check-then-open race remains — closing it fully needs an `O_NOFOLLOW` open POSIX `sh` can't express — accepted as an explicit risk: it requires an already-compromised `pi` shell winning a race for a limited-content root write.) All root-owned helpers install root:root across pi-gen, `install.sh`, and `update.sh`; the gift-mode `timedatectl set-timezone UTC` fixed-argv path is unchanged. - -- **Diagnostics redaction now rounds compound-key and odd-shaped coordinates in the support-logs / journal export** (#497, #498). The diagnostics redaction rounds coordinates to ~city-block precision so the copy / support-logs / journal surfaces stay safe to paste into a public GitHub issue — but `_COORD_KEYED_RE` led with `\b`, which cannot match inside a compound key (the `_` before `LATITUDE` is a word char, so there is no boundary). systemd logs `env.sh` verbatim when a line uses `export` (`Ignoring invalid environment assignment 'export WEATHER_LATITUDE=33.1234'`), so the device's **full-precision, street-level home coordinates** rode un-rounded into the exported bundle while the structured field correctly showed the rounded `33.12`. The lead-in is now a zero-width negative lookbehind (`(?` endpoint (validated against the `DIAG_UNITS` allowlist, redacted, `no-store`) **after** first paint, so the page is interactive immediately and one slow unit can't stall another's tail (the multi-failure case that previously blew the client's 10 s budget). The tail cache TTL is decoupled to 45 s (> the 30 s poll) so a stuck-failed unit doesn't re-fork `journalctl` every poll; the copy support payload appends the client-hydrated logs **inside** its fenced block; a no-JS browser sees status chips + a `