Skip to content

Improve file browser performance and SSH workflows - #43

Closed
matchaboar wants to merge 2 commits into
nonatofabio:mainfrom
matchaboar:feat/performance-and-file-browser-improvements
Closed

matchaboar wants to merge 2 commits into
nonatofabio:mainfrom
matchaboar:feat/performance-and-file-browser-improvements

Conversation

@matchaboar

@matchaboar matchaboar commented Aug 2, 2026

Copy link
Copy Markdown

Performance

  • Move file-tree scanning off the main actor.
    • Cause: the previous DocumentStore.refreshFileTree() recursively called contentsOfDirectory and read resource values synchronously on @MainActor, freezing window interaction for the full scan.
    • Before → after: on a generated 5,000-document fixture (5,221 visible rows), median main-thread work dropped from 33.062 ms to 0.008 ms to dispatch the detached worker. The worker completes the scan in 65.085 ms without blocking the UI.
  • Render only visible file rows.
    • Cause: the previous recursive VStack eagerly created every expanded directory and file row, even when thousands were outside the viewport.
    • Before → after: in a 320×600 sidebar with 5,221 rows, median initial render dropped from 468.944 ms to 81.518 ms (82.6% faster), while initially realized rows dropped from 5,221 to 28 (99.5% fewer) by flattening the tree and using LazyVStack.
  • Batch Git metadata instead of spawning Git per row.
    • Cause: additions/deletions and last-edited timestamps become process-bound if each visible file invokes git diff and git log independently.
    • Before → after: for 200 Markdown files, a per-file baseline used 400 Git processes / 4,624.782 ms; the batched collector uses 3 Git processes / 44.397 ms (~104× faster). Collection also runs on a detached utility task.
  • Discard stale background work.
    • Tree and metadata refreshes are cancellable and generation-guarded, so an older scan cannot overwrite a newer folder/profile selection.
  • Add profiling fixtures and signposts.
    • Includes deterministic large-folder generation, a Time Profiler/sample workflow, and os_signpost points for tree build, flatten, publication, metadata, and local file reads.

Benchmark environment: Apple Silicon, macOS 26.5.2, optimized Swift (swiftc -O). Tree results are medians of 9 runs; render results are medians of 5 runs; Git results use 5 batched and 3 per-file runs.

Features

  • Open a local directory directly with ⌘⌥O and keep that directory as the window’s file-browser scope.
  • Browse .md, .markdown, .mdown, .mkd, .txt, and .pdf files.
  • Show Git working-tree additions/deletions, untracked status, and compact last-commit age badges.
  • Configure file-browser metadata and active-file highlighting in Settings.
  • Load SSH profiles from ~/Library/Application Support/Mindle/ssh-profiles.yaml, including a favorite profile opened from the toolbar.
  • List remote documents without copying the full workspace; fetch only the selected document and referenced relative Markdown images.
  • Fall back to the remote user’s $HOME when the configured SSH directory does not exist. Existing empty directories remain empty, and unreadable/failed searches remain errors.
  • Preserve remote directory structure in the local cache and migrate legacy cached documents/annotation sidecars.

UI

  • Add a scoped sidebar header showing the local folder or SSH profile name, with refresh control.
  • Add explicit loading, empty-directory, and connection/error states.
  • Add stronger active-file highlighting plus compact Git change and last-edited badges.
  • Add File Browser and SSH Profiles sections to Settings, including YAML location, reload, and open controls.
  • Add toolbar access for the favorite SSH profile and remote-file refresh.
  • Restore standard macOS title-bar double-click behavior, honoring the user’s AppleActionOnDoubleClick preference for zoom, minimize, or no action.
  • Add light, dark, empty, and error screenshot regression baselines.

Bug fixes

  • Force remote PDFs to recreate their PDFDocument after refresh instead of keeping stale bytes behind an unchanged cache URL.
  • Use unique fetch temporary files and an atomic replacement lock so concurrent SSH opens cannot race on one .fetch path.
  • Validate SSH hostnames during profile parsing instead of force-unwrapping an invalid target.
  • Preserve existing remote annotations when moving from the legacy cache layout to the mirrored directory layout.
  • Correctly handle percent-encoded # and ? characters in local and remote image filenames.
  • Mark image scheme responses no-store so refreshed remote assets are not hidden by WebKit caching.
  • Surface partial remote-image fetch failures without discarding the successfully opened document.
  • Reject malformed SSH listings that do not report an effective root, and scope fallback results to the reported $HOME.

Validation

  • ./run-tests.shall checks pass, with 89.73% affected pure-logic line coverage.
  • ./run-screenshot-tests.sh4 screenshot baselines pass, with 89.77% FileBrowserView line coverage.
  • ./build.sh — builds Mindle.app successfully.

@nonatofabio nonatofabio left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First off — thank you for this. I did a deep read of all 41 files and a lot of it is genuinely good engineering: the process-runner rewrite fixes a real deadlock class (draining pipes concurrently instead of after termination), the hostname validation in SSHTarget is a security improvement with proper injection tests, the generation-counted cancellation in FileBrowserState is exactly how off-main tree building should look, and the test coverage is substantial and honest. I want most of this in Mindle.

That said, I can't take it as-is. Blocking items first:

1. encodeImagePath is out of scope — breaks rendering for any doc with a local image. In Resources/web/reader.js, encodeImagePath is declared inside decodeImagePath (after its returns), but called from resolveImageSrc. Any file://, absolute, or relative image src throws ReferenceError, and since rewriteImages() runs inside applyAll() with no try/catch, the render pipeline dies mid-flight — annotations, search marks, and mermaid never get installed for that document. Move the function up a scope and please give the branch a manual pass with an image-bearing markdown file.

2. Remote asset fetching needs tighter scoping. Auto-fetching referenced images is a nice touch, but as written every ![](...) in a remote doc gets scp'd with ../ traversal allowed and no extension filter — so a hostile remote document can make Mindle copy anything the SSH user can read (e.g. ../../../.ssh/id_rsa) into the local cache. Nothing leaves the machine, but it stretches Mindle's "SSH only when you ask" promise further than I'm comfortable with. I'd take this with: an image-extension allowlist, resolved paths confined to the profile root, and a cap on fetch count per document.

3. CI: keep test execution out of the signing job. The new run-tests.sh && run-screenshot-tests.sh step runs inside the job that carries SIGN_IDENTITY and contents: write. Fork-PR protections cover the realistic path, but I'd rather the test step live in a separate job with permissions: contents: read (or checkout with persist-credentials: false).

Smaller things I'd like resolved or discussed:

  • LazyVStack in FileBrowserView — the non-lazy VStack on main was a deliberate fix for #36 (scroll nudge when row state flips). Your flattened equatable rows may well avoid the original problem, but I need to re-verify #36 manually before this ships. If you've tested that scenario, say so in the thread.
  • Missing remote root falls back to find "$HOME" — a silent whole-home scan is surprising; I'd rather fail loudly with a clear error.
  • Default profile ships with hostname: test — anyone with a real Host test alias in their SSH config gets a connection they didn't configure on first click. Ship the default commented out.
  • decodeImagePath truncating at ?/# changes behavior for local filenames containing those characters (goes with item 1).
  • The README edit drops the file browser's "Never escapes upward" line — the code still doesn't escape upward, so let's keep the promise in the docs.
  • The AGENTS.md edit is benign but unrelated to this PR — happy to take it, just flagging the churn.

If you'd rather split it: the file-browser performance core (FileTree, FileBrowserState, FileBrowserView, GitFileMetadata, PerformanceTrace + their tests) is clean and could land on its own quickly, with the SSH workflow changes as a follow-up PR once the scoping questions above are settled. Either way works for me.

@matchaboar

Copy link
Copy Markdown
Author

Rebased the work onto current main and split it into three focused PRs:

  1. Scale file browser performance for large folders #45Performance improvements

    • Off-main tree scanning, stable flattened rows, LazyVStack, batched Git metadata, generation guards, benchmarks, and red/green coverage.
    • The issue Sidebar file tree scrolls/shifts unexpectedly when closing tabs above the current selection #36 interactive checklist and fallback adapter options are documented. I could launch the fixture, but Accessibility automation was denied (-1743), so scroll stability is not claimed as manually verified and should be checked before merge.
  2. Polish file browser appearance and add screenshot tests #44UI appearance and screenshot tests

    • Sidebar appearance/states/settings, title-bar behavior, four screenshot baselines, and repeated screenshot runs.
    • Tests now run in a separate contents: read job with persisted checkout credentials disabled; the signing/release job is test-free.
  3. Add secure SSH profiles and remote asset fetching #46SSH, remote, and local asset fetching

    • Fixes encodeImagePath scope and local ?/# path behavior; isolates bad images from the render pipeline.
    • Adds extension allowlisting, canonical profile-root confinement, a 32-asset cap, loud missing-root failures, and a disabled default profile template.
    • The local WKWebView image fixture was manually rendered; no real SSH host was available, so remote behavior is covered by automated tests.

All three branches were created from and rebased onto main, committed and opened as GitHub user matchaboar. Closing this superseded combined PR.

@matchaboar matchaboar closed this Aug 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants