Skip to content

[HN-104] Knowledge graph panel downloads the entire vault with unbounded concurrency #467

Description

@ixjosemi

Problem

The knowledge-graph panel fetches every markdown file in the vault at once, with no concurrency limit:

const entries = await Promise.all(
  markdownPaths.map(async (path) => {
    const result = await readFile(path)
    return [path, result?.content ?? ''] as const
  })
)

Promise.all over markdownPaths means one in-flight request per file, all fired simultaneously. The effect re-runs whenever markdownPaths, readFile or reloadKey change.

Location

apps/web/src/components/workspace/knowledge-graph-panel.tsx:309-331

Impact

A 500-note vault issues 500 concurrent requests on panel open, and holds the full text of the entire vault in browser memory. Browsers cap per-host connections (~6), so the rest queue at the network layer — the burst does not go faster than a pool would, it just removes any ability to prioritise, cancel or show progress. On a large vault the panel stalls with no feedback.

Suggested fix

Short term: a concurrency pool (6-8 in flight) plus an AbortController so a panel close or a reloadKey change cancels the outstanding requests instead of racing the next batch.

Better: a server endpoint that returns the already-built graph, so the client never downloads the vault at all. That also fixes the companion issue about the O(N²) graph build blocking the main thread — both have the same good fix.

Source: independent verification pass (Claude Opus 5). Not part of the HN-001..HN-063 batch.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions