[WRONG BRANCH] fix(kiro): avoid mixed-script estimator allocations - #442
[WRONG BRANCH] fix(kiro): avoid mixed-script estimator allocations#442luvs01 wants to merge 1 commit into
Conversation
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Team Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
✅ Deterministic PR hygiene checks passed. |
⏳ DRAFT
What to do
Its title has been prefixed with |
Motivation
String.repeat) when any CJK character appeared in a mostly-Latin payload, amplifying transient memory and enabling DoS from accepted loopback requests. The goal is to remove those allocations while preserving existing token-ratio behavior.Description
estimateTokensFromCharacterCountstosrc/lib/token-estimate.tsand delegatingestimateTokensto it.src/adapters/kiro.tsto callestimateTokensFromCharacterCountsinstead of building"x".repeat(...)/"한".repeat(...), and apply the existing Latin wire expansion only to the computed Latin portion.tests/lib/token-estimate.test.tsthat asserts equivalence between the pre-counted arithmetic and the original mixed-script estimate (including a large numeric bucket check) without materializing huge strings.Testing
bun test tests/lib/token-estimate.test.tsand the modified token-estimate tests passed.bun test tests/providers/kiro/kiro-stream.test.tswhich mostly passed but surfaced one unrelated existing assertion failure: thenormalized images contribute conservative context tokenstest failed.bun run typecheck(bun x tsc --noEmit) which succeeded.bun run privacy:scanwhich succeeded.bun run test) and observed unrelated current-HEAD test failures in other subsystems; the changes in this PR are isolated to the Kiro estimator and token-estimate sidecar.Codex Task