Repository navigation
LLM analysis: token-budgeted digests, an MCP server, and verdicts painted back onto the map - #36
Merged
Merged
Conversation
A 3 M-file tree cannot go into a chat context, and every obvious truncation lies about the disk: a depth cap buries the interesting folder, a top-N cap drops the context that made it interesting. TreeDigest spends a hard line budget greedily, largest-subtree-first — the same ranking a human applies by hand, with lines instead of pixels as the scarce resource. - TreeDigest: greedy max-heap expansion under an exact line cap, single-child chain collapsing (Users/rducom prints as one line), own-files block ranked against sub-folders, remainder rollup, cache:<id> / sparse / ~.ext annotations. Pure over FSNode + scalars, so the GUI and the future headless MCP server render byte-identical payloads. - Leaves omit their own-files line: the folder's size line already states those bytes, and leaves are the most numerous node kind. - Folder names are sanitised and framed as data — an unpacked archive can carry a directory named to read as an instruction. - ⌘⇧C copies the briefing for the current zoom root; scan-wide counters and the file-type table are withheld under a zoom root rather than shown against the wrong denominator. - --briefing <path> [max-nodes] prints the same text, so the digest is observable on a real disk without driving the GUI. 35,5 GiB / 198 k files at maxNodes 200 → 199 tree lines, ~3 k tokens. 17 new tests, full suite 141 green, menu path live-verified.
Size alone cannot say "delete this". "Regenerable AND untouched for a year" can, and that needs a timestamp the scanner never asked for. - ATTR_CMN_MODTIME rides the bulk buffer the walk already fills: same attribute list, same syscall. Parsed between OBJTYPE and FLAGS as a whole timespec — the position is the exposure, since a wrong offset misaligns every attribute after it (ScannerGoldenTests is the net). - FSNode.newestMTime is max-propagated by CAS on the same ancestor walk that sum-propagates sizes. Directory mtimes are excluded on purpose: they move on any entry add or remove, which would make nearly every folder read as warm. - 0 means unknown, never 1970. Streamed VM/SSH scans carry no timestamps, and reporting "56 years" for a missing attribute is worse than saying nothing. - cold:8mo / cold:3y in the digest, inherited rather than repeated: a child is always at least as cold as its parent, so re-stating the bucket costs a token and says nothing. An older child still earns its own mark. - TreeQuery.approximateTypes: the exact per-subtree rollup the spec planned cannot exist — no per-directory extension table survives a scan, because collapsing files into their parent IS the memory design. Reconstructed from dominant types instead, labelled an estimate everywhere it is shown. Cost: none measurable (base 1,38-1,43 s vs 1,23-1,47 s on 198 k files). Accuracy cross-checked against find/stat on three real subtrees: 180/313/529 days → cold:6mo/10mo/17mo, exact. Estimate validated against the exact table: top-10 overlap 10/10. 12 new tests, full suite 153 green.
claude -p is API-token-only, so the app cannot spawn a session. The flow inverts: the user starts the session and it calls the app. TCC was measured before a line of protocol was written, as the spec demanded, and it did not go as assumed. Full Disk Access turns out not to be granted to SpaceMatters at all here — the bundle is denied on ~/Library/Safari even when LaunchServices launches it — so whether a grant would follow a spawned binary stays open. Moot for shipping: the user already runs the GUI without it. What it mandates is reporting the denial, and the app owns a better detector than the ratio heuristic the spec proposed: FullDiskAccess.isGranted. - JSONRPC: newline-delimited JSON-RPC 2.0 on stdio, no dependency. stdout is the wire, so diagnostics go to stderr and a malformed line is skipped rather than ending the loop. - MCPServer: initialize/tools/list/tools/call/ping, eight read-only tools, lazy scan held for the process lifetime. Tool failures return isError results, not protocol errors, so the model can correct itself. - No deletion tool exists, and a test asserts no tool name reads as a mutation. The read-only boundary is why this is installable without a security conversation. - The access caveat is two-tier: the full paragraph on overview, one line elsewhere. Seventy words on twenty calls is the waste this design avoids. - TreeQuery gains the fence (a path outside the scan fails, never wanders onto the real filesystem), find (nested matches counted once), aged (outermost cold folder only), top, and size/duration parsing. - --mcp defaults to $HOME, not /: that is where the actionable bytes are. Real session on $HOME: 190 GiB / 1.9 M files scanned in 13.6 s once, then five tool calls totalling ~2 100 tokens against the spec's 15 k budget. find produced the finding it exists for: 19 node_modules, 6,67 GiB, several cold. 22 new tests, full suite 172 green.
The payoff phase: the scanner measures, the model classifies, and the map is where they meet. A verdict recolours the treemap and the sunburst, so a conclusion lands where the user already is instead of only in a transcript they have to read next to it. - MCPSocketServer listens on ~/Library/Application Support/SpaceMatters/ mcp.sock (0600), opened only once a scan has finished so an attaching session always finds a tree. MCPRelay makes --mcp try that socket first and fall back to scanning standalone. One config line works either way, which is the only way an MCP entry in a config file can be reliable. - The tools were written once: MCPScanSource abstracts "a scan" and respond(to:) returns the response object, so the two transports differ only in where that object goes. annotate/focus are advertised only when an app is attached — offering them against a standalone scan would earn the model an error per call. - The socket thread reads the tree directly; only the handle is taken on the main actor. FSNode's atomics and gTreeLock exist precisely so a reader outside the UI is safe. Mutations hop back to main. - Verdicts are stored by path, not by node: SPEC-02 invalidate rebuilds subtrees as fresh objects, and a verdict silently migrating to the wrong node is worse than losing it. Re-resolved on every tree bump, inherited by descendants so marking a folder paints its region. - The tint blends after the palette LUT in both renderers, so a verdict never poisons the cached type colours, and rides the existing highlight repack — a recolour, not a relayout. - reason is mandatory. A colour on the map is not an argument for deleting something, so the sentence behind it travels with it and shows on hover. - Fixed a false alarm the smoke test exposed: the access caveat fired on scans where nothing had been denied, because it was gated on the FDA permission rather than on what happened. Crying wolf there teaches a model to ignore the warning on the scan where it matters. Live-verified: session attached, ten tools listed, three annotates accepted, a bad path refused, focus selected the folder. Proof by A/B rather than by eye — mean map colour (107,158,115) with verdicts, a +51 green bias, against (117,120,116) after clearing, +3. Socket removed on quit. 9 new tests, full suite 181 green.
The MCP server is the data; the skill is the method. Packaging/skills/space carries the procedure (overview first, find before drilling, aged on every candidate, explain before proposing) and — in references/directories.md — what a model cannot infer from bytes and dates: which caches regenerate, which only look like caches and hold state you cannot get back (IndexedDB, Local Storage, browser profiles), what is never safe (photo libraries, .git, Time Machine snapshots), and which sizes are sparse and therefore lie. Setup lives in the analysis, not in a dialog. The first cut was an NSAlert and it was too intrusive: handing a scan to an assistant is an option, not a step in using the app. It is now a sparkles icon in the results toolbar beside the reconciliation "?", opening a small popover — copy-briefing on top, then the Claude Code state, with the exact lines it will write and run shown before either button is pressed. sparkles rather than Claude's own mark: naming the tool is fine, shipping a third party's logo is not. - Registration goes through `claude mcp add -s user`, never by editing ~/.claude.json — that file is the CLI's business and a GUI app rewriting it is how someone loses their MCP setup. The CLI defaults to `local` scope, which is per-directory and wrong for a disk analyser. - The CLI is looked for at its install paths, not through the shell: a GUI app inherits a minimal PATH. Not found → the skill still installs and the command is shown to copy. - Failures report what the CLI actually said; "already exists" and a broken install need different responses. Syntax verified against the real CLI with a throwaway entry, then removed. Live-verified in the bundled app: the icon is in the toolbar, the popover resolves the skill out of Contents/Resources and finds the CLI, and dismissing it wrote nothing. 3 new tests, full suite 184 green.
Two problems found by using it. The toolbar wraps its whole row in .lineLimit(1) so the bar stays one line high (#12). A popover is a child of that view, so it inherited the clamp and every explanation in the panel truncated to "…" — including the three-step instructions, which lost steps 2 and 3 entirely. Undone locally, plus the fixedSize the multi-line step text needed to stop being compressed, and an explicit panel background: without one the treemap showed through the text. And "Claude Code is connected" is a status, not an action — there was nothing telling anyone what to do with it. The connected state now spells out the three steps and offers the starter prompt to copy, phrased to name the server so the session reaches for it instead of shelling out to du, and to ask for the map marks so conclusions land where the user can see them.
health-checking the network to answer a local question Reported as "c'est un échec": a session found no spacematters tools even though the server was correctly registered and connected. It was: a session builds its tool roster at startup, so one already open when the server was added will never see it, and reports that in ways that read like a broken install. The panel now says so and points at /mcp, which is the difference between the feature working and the feature looking broken. Also: the registration probe ran `claude mcp list`, which health-checks every server the user has — Drive, Gmail, Calendar, datadog, linear — over the network, spending seconds of spinner to answer a question about one local entry. It now reads ~/.claude.json for the key, off the main actor since that file carries session history and can run to megabytes. Registration still goes through the CLI: writing is the half that must survive a format change. And the guidance is one string literal, not a `+` concatenation — SwiftUI only parses markdown out of literals, so the built-up version would have rendered its asterisks and backticks verbatim.
…ting it A real session reported "the SpaceMatters app isn't running" while the app was open on screen, and re-scanned 190 GiB instead of reading the tree in front of it. Three separate defects, all in the attach path. 1. The accept loop served each client inline, so one session held the listener for its whole lifetime. A second session queued behind it and, once the backlog filled, was refused outright — silently falling back to a full re-scan. Each connection now gets its own thread; verified with two concurrent relays, both attaching with all ten tools. 2. A socket *file* outlives the listener that owned it: a second app instance unlinks and re-binds the shared path, and an instance killed without applicationWillTerminate leaves the path pointing at a dead inode. fileExists() says yes and connect() says ECONNREFUSED. The bridge now probes reachability on every scan completion and re-binds when the answer is no — one connect() per scan to rule out the failure that looks fine from the inside. 3. The relay treated "refused" as "not running". A model then sees no annotate tool, concludes the app is closed, and tells the user so — sending them to look in entirely the wrong place. The two cases are now distinguished, and overview carries the reason, including an explicit instruction not to claim the app is closed when its socket merely went stale. Also: accept() errors no longer spin at 100% CPU on a listener that will never recover, and the backlog went 4 → 16. 5 new tests, including the file-without-a-listener case that started this. Full suite 189 green.
…ine too Reported from real use: "worth reviewing" was rendering green, and nothing in the left outline said which folders had been marked. The colour bug was in the blend. Mixing 60% toward the tint left 40% of the type hue contributing, so the same verdict rendered a different colour depending on what it sat on — review orange over a green subtree came out olive, and read as neither warning nor safe. A verdict has to look identical everywhere or it means nothing, so the hue is now replaced outright and only the brightness of the original survives, which is what carried relative size. review also moved from amber to a saturated orange: this app's palette is full of golds and an amber verdict was indistinguishable from an unmarked tile. keep moved from blue to slate for the same class of reason — the outline draws every unmarked folder in the accent, which is blue. In the outline, a verdict now recolours the folder icon (scanned vertically, a tinted icon reads where a trailing chip does not) and the reason is the row's tooltip. The verdict is passed in as a row input rather than read from the controller inside the view: the row's other inputs do not change when a mark lands, so SwiftUI diffed the view as identical and skipped the redraw — the treemap recoloured and the outline did not. And a filter, since a mark eight levels down is invisible in a 335 k-folder tree: one toolbar toggle prunes the outline to the marked folders plus the ancestors needed to place them, and nothing below a mark — its contents are not the finding, it is. The button only exists once something is marked, and the filter turns itself off when the last verdict is cleared: a filter that can only produce an empty list is worse than no filter. 3 new tests. Full suite 192 green.
A session marks as it goes, and one folder highlighted eight levels down a 335 k-folder tree is a finding nobody sees. The first annotate of a run now switches the outline to marked-only, so the analysis is visible while it happens and each new mark appears in a short list instead of vanishing into the tree. Once only, and latched: a user who turns the filter off mid-analysis will not have it turned back on by the session's next mark. Clearing the verdicts re-arms it for the next run. Also fixes a neighbouring lie: verdictCount counted stored paths rather than paths that resolve against the current tree, so after scanning a different disk the filter button stayed visible while having nothing to show. It now counts what resolved. The paths themselves stay — durable, so the marks come back if the user returns to that disk. 3 new tests. Full suite 195 green.
github-actions
Bot
added
the
documentation
…s ends Verdicts survived everything, so re-picking a disk from the splash brought back colours from a previous session's conclusions — asserting about a fresh scan things nobody had said about it. They belong to the analysis that produced them: - A rescan of the same root keeps them. The user is refreshing what they are working on, often right after deleting something, and the marks are the work. - Going Home ends the analysis. Re-picking the same disk starts a new one. - Scanning a different root drops them too — switching folders without going through the splash is still a different analysis. The discard path touches no nodes, so it is safe to call from startBackend while the tree is being torn down and replaced. 3 new tests. Full suite 198 green.
CI caught this as a timing failure in IncrementalRefreshTests; the cause was worse than the symptom. ScanController started the MCP socket server on every scan completion — including in the test process. So `swift test` unlinked a running app's socket, bound its own, and exited, leaving the app holding a listener nobody could reach. That is precisely the stale-socket failure the relay learned to detect two commits ago, and it explains where the one that broke a real session came from: I ran the suite dozens of times with the app open. It also explains the CI failure. Every scan completion bound a socket, started a thread, and (since the reachability probe) made a blocking connect() — enough added latency in the refresh path to turn a coalesced burst into two sub-scans on a slower machine. The socket path is a machine-wide rendezvous, so ownership is now explicit: the app calls MCPBridge.enable() at launch and nothing else does. A headless scan, a --briefing run and the test suite all leave it alone. This also undoes a layering violation — the controller had no business starting a server. Verified: the app's socket keeps its creation timestamp across a full `swift test` and stays reachable throughout. Locked by a test asserting the bridge never runs in the test process. Plus the two vertical-whitespace lint violations my edits left behind. Full suite 199 green.
…rest "L'icône IA n'est pas très compréhensible" — a lone glyph cannot answer "what does this do", and the bar already speaks in words (Home, Rescan). It now says "Ask Claude" and sits before the search field, where the eye lands, instead of among the trailing glyphs where preferences live. It is a feature to discover, not a setting to adjust. The word Claude, not the mark: naming the tool an app integrates with is ordinary; shipping someone else's logo in an unaffiliated app implies an endorsement that does not exist. The name is also simply clearer than any pictogram would be. And tooltips for the controls that had none — Home, Stop, Rescan, the search field and its clear button, the disk-changed action, and both Full Disk Access buttons. Each says what the control does and what it costs, not what it is called: Home mentions that it discards the analysis, Rescan that it keeps the assistant's marks, Relaunch why a permission granted mid-session needs one.
NSToolTipManager waits over a second by default. That is right for a hint you already know the answer to, and wrong for a toolbar where the tooltip *is* the documentation — the delay is long enough that most people move on before it appears, which makes the labels I just wrote nearly pointless. 300 ms, and put in the registration domain — the lowest-priority one — so a user who has set NSInitialToolTipDelay themselves, per-app or globally, still wins. Measured rather than assumed: synthetic hover over the toolbar, screenshot at 450 ms, tooltip present.
rducom
changed the title
CI failed twice in a row on different timing-sensitive tests. The second was mine, and reproducible locally at roughly one run in twenty — but only inside the full parallel suite, never alone (12/12 clean) and never in isolation (2500/2500 correct, including under heavy fd churn). Diagnostics caught it: distinct descriptors, the path still a socket, and connect() returning 0 anyway. So the test was asserting a kernel guarantee about the instant after close() — not a behaviour of my code, and not a state production ever observes. There the owner process is either alive, or gone with its entire descriptor table. So the test now builds the real thing: `nc -lU` binds the path, the connection is proved to succeed, the process is SIGKILLed and *reaped* with waitUntilExit(), and only then is the connection proved to be refused. No window to race, and it covers both directions — live owner accepts, dead owner refused — which is exactly what the relay has to tell apart. The narrower live-listener test folded into it. 15 consecutive full-suite runs, no failures. 198 tests.
rducom
added
enhancement
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
claude -pis API-token-only, so the app cannot spawn a session on a subscription. The flow inverts: the user starts the session, and the session calls the app.The guiding constraint throughout: the scanner measures, the LLM classifies. SpaceMatters knows how many bytes are in a folder and when it was last written. It does not know that
workspaceStorageholds state for repositories deleted two years ago, or that 38 GiB of.rawis a photo library that belongs on an external drive. Everything here exists to hand a model the smallest payload that still supports a correct verdict — and to bring that verdict back to the map, where the user already is.Full design and measurements: SPEC-14.
What ships
TreeDigest— a 3 M-file tree cannot go into a chat context, and every obvious truncation lies about the disk. A depth cap buries the interesting folder; a top-N cap drops the context that made it interesting. Instead a hard line budget is spent greedily, largest-subtree-first, with single-child chains collapsed and small siblings rolled up.~/Library/Application Support(35,5 GiB, 198 k files) atmaxNodes: 200→ 199 lines, ~2,5–3 k tokens.⌘⇧C "Copy briefing" — the same digest on the clipboard, for any assistant, no install.
ATTR_CMN_MODTIME— size alone cannot say "delete this"; regenerable and untouched for a year can. It rides the bulk buffer the walk already fills: same attribute list, same syscall, no measurable cost (base 1,38–1,43 s vs 1,23–1,47 s on 198 k files). Max-propagated by CAS on the same ancestor walk that sum-propagates sizes. Directory mtimes are excluded on purpose — they move on any entry add/remove.0means unknown, never 1970.A read-only MCP server (
--mcp) —overview,tree,top,types,find,aged,explain,cleanup_targets. No tool can delete anything, and a test fails if a tool name starts to look like a mutation.findis the one with no cheap shell equivalent: 19node_modules, 6,67 GiB between them, nested matches counted once. A real session on$HOME— 190 GiB / 1,9 M files scanned once in 13,6 s — spent ~2 100 tokens across five calls.Connected mode + verdicts — with the app open the session attaches over a Unix socket: no re-scan, and it sees the tree on screen.
annotatepaintssafe/review/keeponto the treemap, the sunburst and the outline;focusmoves the app to what is being discussed. A reason is mandatory — a colour is not an argument for deleting something — and it shows on hover. The outline switches itself to marked-only on the session's first mark, and one toolbar toggle controls it after that.A skill (
Packaging/skills/space) carrying the procedure and, inreferences/directories.md, what a model cannot infer from bytes and dates: which caches regenerate, which only look like caches and hold state you cannot get back, what is never safe. Setup is a ✦ button in the toolbar showing exactly what it will write, first.Decisions worth reviewing
revieworange over a green subtree came out olive. Only brightness survives, which is what carried relative size.invalidaterebuilds subtrees as fresh objects, and a verdict silently migrating to the wrong node is worse than losing it.claude mcp add -s user, never by editing~/.claude.json.errors > 0, not on the FDA permission: crying wolf on a clean scoped scan teaches a model to ignore the warning on the scan where it matters.Corrected against reality
Three things the spec assumed turned out to be false, and are corrected in the spec rather than worked around:
Fixed from real use
fileExists()said yes,connect()said ECONNREFUSED, and the relay reported "the app isn't running" — so a session told the user their open app was closed. The two cases are now distinguished and the bridge re-binds an unreachable socket on every scan completion..lineLimit(1)was inherited by the assistant popover and truncated every explanation to "…".Verification
195 tests green. Beyond unit coverage, each phase was live-verified: menu path driven through System Events; mtime cross-checked against
find -exec staton three real subtrees (180 / 313 / 529 days →cold:6mo/10mo/17mo, exact); the verdict painting proved by A/B rather than by eye — mean map colour (107, 158, 115) with verdicts, a +51 green bias, against (117, 120, 116) after clearing.Not done
Phase 3 attaches to whatever scope the app happens to show, and the socket only opens when a scan finishes — a session started during a long scan goes detached for its whole life. Both are noted in the spec.