Skip to content

fix(ingest): say when a file is dropped for the 1 MB cap (#370) - #399

Open
Frankie-Xu wants to merge 1 commit into
trailhq:mainfrom
Frankie-Xu:fix/370-oversized-file-notice
Open

Frankie-Xu wants to merge 1 commit into
trailhq:mainfrom
Frankie-Xu:fix/370-oversized-file-notice

Conversation

@Frankie-Xu

Copy link
Copy Markdown
Contributor

Summary

  • Files over the 1 MB ingest cap were dropped with no message. graft build then reported parsed: 1 of 1 files, and ask / grep / skeleton told the agent the symbol or file did not exist.
  • The walk records those files. Build prints ⚠ skipped <path>: N MB > 1 MB cap and parsed: … (N skipped: size). The skipped set is stored on wiring.json meta.skipped so later queries can name it.

Not in this PR

  • Does not add --max-file-bytes.
  • Does not change the 1 MB cap.
  • Does not index the oversized file.
  • Does not add notices for silent symlink skips (Symlinked files are skipped #143).

Closes #370

Test plan

  • Git walk and filesystem fallback: ok.ts indexed, big.ts at 1 MB + 1 byte recorded in skipped
  • buildGraph result and skeleton big.ts name the skip instead of "no definitions indexed"
  • Zero-hit ask / grep notes include skipped big.ts: … > 1 MB cap
  • node --import tsx --test --test-name-pattern "#370" test/ingest-fs.test.ts test/search-grep.test.ts test/ask.test.ts

@github-actions

github-actions Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

🌱 graft blast radius

5 areas changed → 7 areas can be affected. 33 dependent symbols, depth 2.
Tests: no test reaches Command Line Interface; 4 areas updated their tests.
Tag: @anirudhkumar-nanonets — 4 of 12 areas · @shhdwi — 8 of 12 areas · @tpoignonec — 4 of 12 areas

flowchart TB
  A0(("Workspace Graph<br/>15 symbols"))
  A1(("Review Process<br/>8 symbols"))
  A2(("Tool Invocation<br/>3 symbols"))
  A3(("Context Management<br/>3 symbols"))
  A4(("Graph Engine<br/>2 symbols"))
  AX(("2 smaller areas<br/>2 symbols"))
  classDef reached fill:#D9EDF3,stroke:#3AA7C9,stroke-width:1.5px,color:#0E313C;
  class A0,A1,A2,A3,A4 reached;
  classDef tail fill:#EEF2F3,stroke:#9AA4A9,stroke-width:1px,color:#3A4247;
  class AX tail;
Loading
Can be affected Symbols Nearest hop Reached from
Workspace Graph 15 src/graph/refresh.ts:L150-L227 ensureFreshGraph — calls, depth 1 Ask Functionality, Graph Building +2
Review Process 8 src/app/brain-build.ts:L251-L358 readRepository — calls, depth 1 Graph Building, Filesystem Ingestion
Tool Invocation 3 src/mcp/tools.ts:L247-L329 callSingleTool — calls, depth 1 Ask Functionality, Graph Building +1
Context Management 3 src/context/build.ts:L109-L122 listContextFiles — calls, depth 1 Filesystem Ingestion
Graph Engine 2 src/engine.ts:L108-L117 ask — calls, depth 1 Ask Functionality, Graph Building +1
build-viewer.mjs 1 scripts/build-viewer.mjs:L1-L45 build-viewer.mjs — calls, depth 2 Graph Building
1 smaller area 1 Sync Execution see below
Who knows this code — 5 people across 12 areas
Area Who knows it
Graph Building · changed @tpoignonec — 1 commit, last 19d ago · @shhdwi — 1 commit, last 19d ago
Filesystem Ingestion · changed @tpoignonec — 1 commit, last 19d ago · @shhdwi — 1 commit, last 19d ago
Ask Functionality · changed @anirudhkumar-nanonets — 2 commits, last 5d ago · @shhdwi — 1 commit, last 19d ago
Grep Functionality · changed @shhdwi — 1 commit, last 19d ago
Command Line Interface · changed @anirudhkumar-nanonets — 5 commits, last 5d ago
Workspace Graph · affected Buseong Kim — 1 commit, last 15d ago · @tpoignonec — 1 commit, last 19d ago
Review Process · affected @anirudhkumar-nanonets — 11 commits, last 5d ago
Tool Invocation · affected @anirudhkumar-nanonets — 1 commit, last 13d ago · @afeddersen — 1 commit, last 15d ago
…4 further areas

Ownership is git history over each area's own files, weighted towards recent work (120-day half-life). Merge commits and bots are dropped, and you are dropped from your own PR. A name with no @ has no GitHub handle in its commit email — tag them by hand, or add a .mailmap entry. A suggestion from history, not a CODEOWNERS rule.

All 33 dependent symbols, grouped by area

Workspace Graph — 15 symbols in 6 files

  • src/graph/refresh.ts:L150-L227 — ensureFreshGraph (calls, depth 1)
  • src/graph/scopes.ts:L329-L341 — discoverWorkspaceChildren (calls, depth 1)
  • src/graph/scopes.ts:L133-L272 — discoverScopes (calls, depth 1)
  • src/graph/source-files.ts:L64-L85 — listSourceFiles (calls, depth 1)
  • src/graph/workspace-cli.ts:L88-L97 — runWorkspaceAsk (calls, depth 1)
  • src/graph/workspace-cli.ts:L99-L116 — runWorkspaceGrep (calls, depth 1)
  • src/graph/workspace.ts:L240-L558 — federateAsk (calls, depth 1)
  • src/graph/workspace.ts:L563-L603 — federateGrep (calls, depth 1)
  • src/graph/check.ts:L58-L164 — checkGraph (calls, depth 2)
  • src/graph/refresh.ts:L235-L261 — ensureFreshChildren (calls, depth 2)
  • src/graph/source-files.ts:L103-L120 — listSourceStats (calls, depth 2)
  • src/graph/workspace-cli.ts:L49-L70 — buildChild (calls, depth 2)
  • src/graph/workspace.ts:L97-L101 — isWorkspaceBuildRoot (calls, depth 2)
  • src/graph/workspace.ts:L145-L156 — loadWorkspaceGraphs (calls, depth 2)
  • src/graph/workspace.ts:L709-L722 — splitWorkspace (calls, depth 2)

Review Process — 8 symbols in 6 files

  • src/app/brain-build.ts:L251-L358 — readRepository (calls, depth 1)
  • src/app/review.ts:L45-L99 — reviewPullRequest (calls, depth 1)
  • src/app/brain-build-worker.ts:L1-L83 — brain-build-worker.ts (calls, depth 2)
  • src/app/brain-build-worker.ts:L29-L32 — DoneMessage (references, depth 2)
  • src/app/brain-build.ts:L237-L239 — buildRepoIntoBrain (calls, depth 2)
  • src/app/review-process.ts:L179-L183 — childReviewer (references, depth 2)
  • src/app/review-worker.ts:L67-L87 — run (calls, depth 2)
  • src/app/server.ts:L34-L46 — AppSeams (references, depth 2)

Tool Invocation — 3 symbols in 1 file

  • src/mcp/tools.ts:L247-L329 — callSingleTool (calls, depth 1)
  • src/mcp/tools.ts:L153-L199 — callWorkspaceTool (calls, depth 1)
  • src/mcp/tools.ts:L216-L244 — callTool (calls, depth 2)

Context Management — 3 symbols in 2 files

  • src/context/build.ts:L109-L122 — listContextFiles (calls, depth 1)
  • src/context/build.ts:L146-L374 — buildContext (calls, depth 2)
  • src/context/check.ts:L38-L111 — checkContext (calls, depth 2)

Graph Engine — 2 symbols in 1 file

  • src/engine.ts:L108-L117 — ask (calls, depth 1)
  • src/engine.ts:L91-L101 — graph (calls, depth 1)

build-viewer.mjs — 1 symbol in 1 file

  • scripts/build-viewer.mjs:L1-L45 — build-viewer.mjs (calls, depth 2)

Sync Execution — 1 symbol in 1 file

  • src/claude/sync-run.ts:L19-L33 — runSync (calls, depth 2)
Test signal per changed area — 4 ✓ · 1 ✗

Reached = a node under a test path has a resolved edge into the changed symbol. It undercounts anything called indirectly — through a CLI, a spawned process or a dynamic import — so read a low ratio as “look here”, never as a coverage gate.

  • Graph Building — 1 of 8 reached · 1 test file changed here: test/ask.test.ts
    • not reached: applyMinSubstanceGuard, readGoModules, formatBytes, formatSizeSkipLine, matchSkippedFile, skippedFromGraph, skippedQueryNote
  • Filesystem Ingestion — 2 of 11 reached · 1 test file changed here: test/ingest-fs.test.ts
    • not reached: canonicalWalkRoot, escapedCanonical, gitVisibleFiles, gitVisibleFilesShallow, noteOversized, remapSkipPaths, remapWalkPaths, skippedPath, …1 more
  • Ask Functionality — 4 of 44 reached · 1 test file changed here: test/ask.test.ts
    • not reached: askSavingsLine, attachBrainRules, baselineFor, bm25, computeIdf, computeIdfFromIndex, escalationNudge, fallthroughNoteFor, …32 more
  • Grep Functionality — 2 of 14 reached · 1 test file changed here: test/search-grep.test.ts
    • not reached: formatGrepHeader, formatGrepResult, formatGroup, groupHeader, runGrepCommand, truncationNote, computeInDegree, enclosingSymbol, …4 more
  • Command Line Interface — 0 of 11 reached · no test file reaches it
    • not reached: cliConfig, engineFrom, fmt, formatRetractions, noteHit, noteQuery, parseTabs, queryRoot, …3 more
33 test suites also reference this code

43 symbols, kept out of the diagram and the table so they cannot crowd out the areas a reviewer has to look at.

  • test/ask-fusion.test.ts
  • test/ask-index.test.ts
  • test/container-extract.test.ts
  • test/context-only-dir.test.ts
  • test/context.test.ts
  • test/covers.test.ts
  • test/generic-extract.test.ts
  • test/graph-go.test.ts
  • test/graph-incremental.test.ts
  • test/graph-invariants.test.ts
  • test/graph-java.test.ts
  • test/graph-languages.test.ts
  • test/graph-php.test.ts
  • test/graph-posix-paths.test.ts
  • test/graph-python.test.ts
  • test/graph-r-classes.test.ts
  • test/graph-r-phase3.test.ts
  • test/graph-r-phase4.test.ts
  • test/graph-r-phase5.test.ts
  • test/graph-r.test.ts
  • …13 more

graft blast · refs/graft/base...HEAD · depth 2 · 11 changed files

Open the interactive graph → — click an area to see the code that changed, and the line that reaches it.

github-actions Bot added a commit that referenced this pull request Sep 15, 2026
@Frankie-Xu
Frankie-Xu marked this pull request as ready for review September 15, 2026 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ingest: files over the 1 MB cap are dropped silently, and ask/grep/skeleton then deny they exist

1 participant