Skip to content

Add opt-in hot-CPU profiler and heap diagnostics for the main process - #37

Open
huntharo wants to merge 9 commits into
mainfrom
feat/diagnostics-profiler
Open

huntharo wants to merge 9 commits into
mainfrom
feat/diagnostics-profiler

Conversation

@huntharo

@huntharo huntharo commented Sep 26, 2026 •

Copy link
Copy Markdown
Owner

What this adds

Opt-in diagnostics for DiskHound's main process, for bug reports about a hot or bloated main thread. Everything lands under <userData>/diagnostics/, one folder per session: session.json, samples.ndjson, events.ndjson and the captures.

  • Hot-CPU profiler. This ports PwrAgnt's hot-cpu-profiler.ts:
    • V8's sampling CPU profiler records the main thread continuously in 30 s windows. It keeps the last finished window plus the live one, and rotates only after a CPU sample has been evaluated.
    • Two 2 s samples at or above 50% of one core trigger a capture. It records 15 s more and joins both windows with joinCpuProfiles, so a .cpuprofile reaches 30–60 s before the trigger.
    • An armed recording that never triggers is discarded.
    • Captures are 60 s apart and at most 5 per launch.
    • The trigger reads process.threadCpuUsage(), the main thread only. Busy scan and folder-tree workers don't fire it, and heap snapshots are discounted.
  • Heap diagnostics.
    • V8's sampling heap profiler runs on the main process (32 KB interval). When main's used heap passes the gate (1,200 MB by default, 128–3,072 in Settings), it saves the allocation profile as a .heapprofile.
    • That happens at most once per app version per day, across restarts, via heap-gate-state.json.
    • With "Full heap snapshots" on, it also writes snapshot A at the gate and snapshot B 20 s later, without a heap-size or estimated-headroom veto. Each file's main-thread pause is logged.
    • Worker heaps come from worker.getHeapStatistics() through a small registry every worker runtime now uses. They are shown in status and used by the near-limit diagnostic breadcrumb.
  • Near-heap-limit breadcrumb (always on). When main plus workers pass 85% of the cage, one crash.log line is written synchronously (crash.log is otherwise buffered); if sampling is running, its profile is also saved synchronously. V8's OOM abort skips every JS handler, so this is the only record such a crash leaves. It re-arms below 68%.
  • Settings → Diagnostics.
    • Toggles, the threshold and the gate.
    • Live status: CPU profiler state, and main plus worker heaps against the cage.
    • Captures with Reveal folder, Take heap snapshot and Delete all.
    • Sticky notifications for saved CPU profiles, allocation profiles and full heap snapshots. Copy details includes the capture path, summary, file size, runtime versions and opening instructions. Notifications stay until dismissed during the current renderer session; new messages do not evict them.
    • Copyable handoff text listing every capture's path, with how to open it.
    • Every capture also logs its path to crash.log, and the periodic [memory] line gains a cage: summary.
  • A sync crash.log line before each heap snapshot. V8 blocks main while it writes, and a snapshot has killed Electron 40, so the path and heap size are on disk first.
  • Env overrides for one launch. DISKHOUND_HOT_CPU_PROFILING=1|0 (plus _THRESHOLD_PERCENT, _WINDOW_MS, …), DISKHOUND_HEAP_GATE_MB, DISKHOUND_HEAP_SNAPSHOTS=1 and others (see diagnosticsConfig.ts). Active overrides show in Settings, in the handoff text and in crash.log.
  • Retention. At most 12 sessions and 6 GB, nothing older than 30 days, and empty sessions are removed. It runs after each capture and at start, and never deletes the live sessions.

Snapshot behavior and prior measurements

Manual and enabled automatic captures call V8 without a fixed heap ceiling or estimated free-heap admission check, matching PwrAgnt/PwrGit. The daily capture limit and cancellation when snapshots are disabled remain in place. Full snapshots pause main and can consume substantial memory and disk space.

The earlier measurements below used Electron 40.6's Node (24.13.1) via ELECTRON_RUN_AS_NODE=1 --expose-gc, with a reported 4096 MB heap limit. This was a synthetic Map of path strings to {size, files, children, mtime} records, not the normal app launch configuration; its failures do not establish a universal Electron heap-size limit.

Used heap Snapshot file Main thread paused
186 MB 406 MB 6.0 s
266 MB 611 MB 9.4 s
403 MB 920 MB 15.0 s
556 MB 1,327 MB 27.1 s
602 MB process killed, 0-byte file —
656 MB process killed, 0-byte file —
742 MB process killed, 0-byte file —
  • How it dies. The crash reports show EXC_BREAKPOINT inside v8::HeapProfiler::TakeHeapSnapshot, with no crash.log line and exit code 133.
  • What doesn't reproduce it.
    • Plain Node 24.21 at 761 MB.
    • 9M small objects, and 4.5M unique strings at 628 MB.
  • DiskHound's own heap. Snapshots were smaller and faster: 1.55× the heap at 17 ms/MB (102 MB for 68 MB, 1.2 s).

Sampling runs from launch, not from a watch mark

The heap profiler was going to start at 75% of the gate. In the real app, a scan took the main heap from 30 to 83 MB within one 5 s check, and the gate's profile came out empty. A folder-tree load grows the heap the same way.

  • Cost. It is within run-to-run noise. On an allocation-heavy 470 MB workload the timings stayed inside the ±10% variation between runs, and the profile held ~580 KB.
  • Default. Sampling now runs from the first check, which happens at start, whenever heap diagnostics is on. DISKHOUND_HEAP_WATCH_MB can still set a watch mark.
  • Guard. The gate never saves an empty profile when sampling began in the same check.

Disk writes

All defaults are off. Budgets are in src/test/io-budgets.json, recorded by hotCpuProfiler.ioBudget.test.ts and heapMonitor.ioBudget.test.ts. Reads and samples stay in memory; nothing is written per tick.

Setting Writes Bytes
Everything off (default) 0/day; 1 crash.log line per approach to 85% of the heap limit —
Hot-CPU profiles on, idle (any threshold, including the 5% minimum) 0/day 0
Hot-CPU capture 6 write calls for the first capture of a launch, 5 after 0.6 MB measured (73 s profile), ~1.7 MB budgeted
Hot-CPU cap ≤ 5 captures per launch: ≤ 28 write calls, including the armed and capped log lines ≤ ~8.5 MB per launch
Heap diagnostics on, below the gate 0 writes, 1 read of heap-gate-state.json per launch 0
Heap gate fires (any gate, including 128 MB) 7 write calls, ≤ 1 per app version per day 0.16 MB measured, ~1.7 MB budgeted
+ snapshots 2 more files, same daily cap Workload-dependent; using the earlier 1.5–2.3× ratio, ~3.5–5.4 GiB per pair at 1200 MB. There is no snapshot-size ceiling.

The CPU profiler's idle cost is main-thread CPU of 0.92% with it on and 0.07% with it off. That is about 0.85% of a core, in the app with nothing happening.

Verification

  • Diagnostics unit tests: 80 passing, including 15 I/O budget scenarios.
    • The window join, ported from PwrAgnt.
    • Rotation, bounded history, and discard without a trigger.
    • Consecutive samples, cooldown and the per-launch cap across Settings restarts.
    • Main-thread-only CPU.
    • Automatic snapshot attempts at 1200, 1800 and 3072 MB thresholds, plus manual capture above half the reported heap limit (injected heap readings and snapshot writer).
    • Snapshot B is cancelled when snapshots are disabled, including during directory preparation.
    • The daily cap across restarts and app versions.
    • Near-limit breadcrumbs.
    • Retention.
    • Config and env.
    • The worker heap registry, including a worker that never answers.
    • That a capture shows in Settings as soon as it's committed.
  • I/O budgets. 15 scenarios, including bounded shutdown: an idle hour, a capped hour, each capture, the snapshot pair, near-limit with the gate off and on, and the manual snapshot.
  • E2E, e2e/diagnostics.spec.ts. It passes locally on macOS; CI runs it on Linux, Windows and macOS.
    • A forced hot loop in main yields a .cpuprofile that contains it, with more than 3 s of lookback.
    • Growing main's heap past a 128 MB gate saves a .heapprofile that attributes more than 50 MB to the allocating function.
    • Settings accepts an 1800 MB snapshot threshold, allows changing it to 2000 MB, and Take heap snapshot writes a valid snapshot; Delete all removes it.
    • The fixture gained launch({ env }) and a diagnostics settings seed.
    • All four diagnostics E2E tests pass locally on macOS, including capture notifications, clipboard success/retry, sticky lifetime, explicit dismissal and toast updates cancelling earlier dismissal timers.
  • In the app, on an isolated profile, macOS arm64.
    • CPU. A burst started 53 s into a recording gave main-hot-0001.cpuprofile, 638 KB, "73 s, 58 s before a 88% main-thread trigger". The 58,645 samples have no gaps at the join.
    • Heap gate. DISKHOUND_HEAP_GATE_MB=100 with snapshots, during a 624k-file scan of ~/github.
      • The .heapprofile (162 KB, "sampled for 20 s from 15 MB") is led by parseFolderTreeSidecarLine, readTripleArray and loadFolderTreeSidecar.
      • Snapshots A and B came 21 s apart: 102 MB and 118 MB, with pauses of 1.2 s and 1.5 s.
      • All three load in DevTools' Memory panel. Both snapshots are internally consistent (node and edge counts match their metadata), and the profile opens in Heavy (Bottom Up).
    • IPC. The manual snapshot works (32 MB, 0.3 s). Reveal refuses ../../etc and unknown names. Delete all keeps heap-gate-state.json, so the daily cap survives a clear.
  • bun run typecheck && bun run test && bun run build passes, and all 19 E2E tests pass locally.

Screenshots

Paths in the screenshots are shown as the default macOS userData folder; they were taken on a scratch profile.

Settings → Diagnostics, defaults:

Diagnostics settings, all off

Everything on, after the captures above, at the default 1.2 GB gate:

Diagnostics settings, all on with captures

The gate's snapshot A in DevTools → Memory:

Heap snapshot in DevTools

tzarebczan added a commit that referenced this pull request Sep 26, 2026
Walk Unix folders with dua-core instead of jwalk: 3–6× faster scans with less CPU
@huntharo
huntharo force-pushed the feat/diagnostics-profiler branch 4 times, most recently from a162963 to 520c10c Compare October 1, 2026 18:53
@huntharo
huntharo force-pushed the feat/diagnostics-profiler branch from 520c10c to da2e991 Compare October 14, 2026 19:05
crash.log is buffered now. The near-limit breadcrumb, and a new line
before each heap snapshot, append before returning, since the process
can die before the next flush. The I/O budgets log through the real
buffered crash.log.

DiagnosticsManager.stop() waits for a heap capture that already
started, and a tick that was reading the gate's day state when stop()
ran no longer fires the gate.
vi.waitUntil advances fake time on every check while it waits, so a
slow CI disk moved the second capture relative to the cooldown, or
timed the wait out. The tests now wait for onProfileWritten and move
fake time only where they say so. With writes delayed 400 ms, the old
tests fail as macOS CI did and the new ones pass.
@huntharo
huntharo force-pushed the feat/diagnostics-profiler branch from da2e991 to b87408e Compare October 22, 2026 01:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant