fix: unhang index builds on pathological tokens, and name the file when one hangs

[AkashGoenka]

What

Reported by a user whose index build hung indefinitely. Root-caused, fixed, and then made the same class of failure diagnosable.

The hang

The shape gate in content-token extraction used a catastrophic-backtracking regex — a quantified group inside a trailing +:

/^[a-z][a-z0-9]*(?:[A-Z][a-zA-Z0-9]*)+$/

One long mixed-case run that ultimately fails the match sends it exponential. Measured growth is 2× per 2 characters:

token length	time
46	41ms
66	~42s
~90	longer than anyone waits

The trigger is narrower than it appears, which is why it showed up in real fixtures: the token scanner accepts _ but the camelCase pattern does not, so only a mixed-case run containing an underscore can fail this way — base64url blobs and JWT signatures exactly. A pure alphanumeric camelCase run matches fast and is harmless.

Extraction runs on every file in every language, so the repo's language was never the point.

Symptoms, all reproduced: keeper at 100% CPU holding its lock, log frozen at Parsing files..., init blocked for its full 180s, and killing the process the only recovery — after which the next build hit the same file and hung identically.

The four patterns are rewritten without nested quantifiers, plus a token-length cap as defence in depth. Verified equivalent to the old patterns across 400,018 randomly generated tokens plus the existing corpus: zero disagreements.

The diagnosability gap

The bug was one regex; the hours went to not being able to see it. The log named no file, and status actively misled.

That failure blocks the event loop, which rules out every usual approach — watchdog timer, progress interval, signal handler, async write are all structurally unable to run once the spin starts. So the indexer now writes what it is about to do before doing it, synchronously, and clears it on clean completion. A record that outlives its own process is therefore the diagnosis.

working on: parse app/models/poison.rb (17s ago)
LAST SEEN: PID 55760 died during parse app/models/poison.rb — 18s ago, never finished

Repeated as a warning at the next keeper start and appended to repair.jsonl as died-in-progress, since startup is the last moment that evidence exists before the next build overwrites it.

Also fixed

coldstart status rendered the inProgress stamp without checking the recorded pid was alive. KeeperWork's own contract says a stamp outliving its keeper is harmless because readers check first — waitForCacheHead did, this renderer did not. Since killing the keeper was the only escape from the hang, the one command you would reach for while diagnosing it was guaranteed to lie.

Notable

The first version of the in-flight record was wrong, and only an end-to-end test caught it. Marking at the call site reported an innocent file while a different one span at 100% CPU: parseFile awaits a grammar load and a file read, and the caller runs a batch of 100 through Promise.all, so every file gets past the call site before any reaches the CPU-bound section. The mark now lives inside parseFile, immediately before the synchronous work with no await between. Pinned by a test and by a comment at the signature, because a diagnostic that confidently blames the wrong file is worse than none.

Testing

• 859 tests pass (was 848); typecheck clean.
• The status fix was verified to fail without the change, so it isn't a no-op assertion.
• End-to-end against a real hang: reproduced with a deliberately re-broken build, confirmed the record names the correct file, the warning fires on restart, and both status branches render.
• Fix confirmed on the same pathological repo that hung: 0.3s vs. never completing.

Notes

Notebook notes ride in a separate commit (kb commit), per repo convention.

[AkashGoenka]

Arun Francis M - thanks for reporting this high severity issue. This has been fixed in v2.3.2