Skip to content

Per-directory token counts in tree; drop 10MB hard size limit - #45

Open
malakhov-dmitrii wants to merge 1 commit into
glebkudr:mainfrom
malakhov-dmitrii:feat/token-counts-and-non-blocking-size
Open

Per-directory token counts in tree; drop 10MB hard size limit#45
malakhov-dmitrii wants to merge 1 commit into
glebkudr:mainfrom
malakhov-dmitrii:feat/token-counts-and-non-blocking-size

Conversation

@malakhov-dmitrii

@malakhov-dmitrii malakhov-dmitrii commented Apr 14, 2026

Copy link
Copy Markdown

Why

Two small pain points when preparing context on a large repo:

  1. The 10MB byte gate fired well before Gemini's real token limit (user report: real Gemini tokenizer showed ~450k tokens, but the app errored with ErrContextTooLong and couldn't proceed). The byte-count and token-count don't line up, so the cap blocks valid selections.
  2. There was no way to see which directory was dominating the context size short of toggling folders off one at a time.
CleanShot 2026-04-14 at 18 39 37@2x

What changed

Per-directory token counts in the sidebar tree. FileNode gains size (int64, bytes) and tokenEstimate (int64, bytes / 4) fields. During buildTreeRecursive the size/estimate is set per-file from entry.Info() and aggregated for directories, skipping gitignored/customignored children so the displayed count matches what will actually be emitted to the LLM. FileTree.vue renders a small ~Xk / ~X.XM badge next to each node.

Non-blocking generation. Removed maxOutputSizeBytes and ErrContextTooLong and the three size-check branches in generateShotgunOutputWithProgress. The existing frontend counter (Step1PrepareContext.vue + LargeTextViewer.vue) already shows a color-coded total that turns yellow at 500k tokens and red at 1.5M. That's a clearer UX than a hard error: the user can see the budget and decide, instead of the app refusing after the fact.

Token estimate math. UI divisor switched from chars/3chars/4 in both Step1PrepareContext and the reusable LargeTextViewer, and the color thresholds now operate on estimated tokens (green < 500k, yellow < 1.5M, red ≥ 1.5M) instead of raw chars. bytes/4 is a rough match for Gemini's code/prose ratio — not perfect, but closer than /3 and enough for a visual gauge.

Docs. design/architecture.md section 6.2 rewritten to describe the non-blocking design and the per-node token fields.

Scope

6 files touched, additive API changes only (two new JSON fields on FileNode; clients that ignore them are unaffected). No test changes — the project doesn't have Go or Vue tests for these paths today.

Test plan

  • go build ./... passes
  • wails dev runs; the tree shows per-dir token badges
  • Badges aggregate only non-ignored children (root for a repo with a large .git / node_modules matches the generated context token count instead of summing everything)
  • Selecting a >10MB project no longer errors; context generates and the color-coded counter reflects the size
  • Reviewer sanity-check on UI: badge formatting on very small dirs, on very large files

Notes for reviewer

  • bytes/4 is a deliberate approximation; happy to make it configurable or swap for a more accurate heuristic if you'd prefer.
  • Dropped the hard cap entirely rather than raising it, on the assumption the user is best placed to decide "fits in my target LLM". If you'd rather keep a sanity cap (e.g. 100MB to guard against accidental node_modules selection), easy to add back.
  • Fields are added to FileNode with JSON tags size / tokenEstimate; wailsjs/go/models.ts regenerates cleanly on wails generate.

FileNode now carries size + tokenEstimate (bytes/4), aggregated for
dirs (skipping gitignored/customignored children so counts match what
actually ships to the LLM). FileTree renders a ~Xk/M badge next to each
node so users can see which dirs dominate the context without
guess-and-check.

Size gate removed from generateShotgunOutputWithProgress: ErrContextTooLong
returns at the three check points and the maxOutputSizeBytes constant are
gone. The UI already has a color-coded token counter (green < 500k,
yellow < 1.5M, red >= 1.5M) that tells the user when they're near an
LLM's window - better to trust the user with a visible signal than to
hard-stop at a byte count that doesn't map to Gemini's actual tokenizer.

Frontend token estimate switched from chars/3 to chars/4 in
Step1PrepareContext + LargeTextViewer so the counter matches real
Gemini-ish ratios more closely.

Docs (design/architecture.md) updated to reflect the new behavior.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant