Repository navigation
Bun: indexed queries silently truncate at the first non-ASCII key (ordered-binary readString ReferenceError) #2750
Description
On the Bun runtime, ordered-binary@1.6.1 (the version every shipped 5.2.x / 5.3.x tag pins) throws
ReferenceError: finishUtf8 is not defined while decoding any key that contains a byte >= 0x80.
Harper surfaces that as a truncated result set with HTTP 200 and nothing in hdb.log.
Node is unaffected. This is a runtime-specific defect in a dependency, not in resources/search.ts.
Impact
A search_by_conditions range/starts_with query on an @indexed string attribute returns 200 with
the decode failure embedded as an element of the result array, in place of every row after the first
non-ASCII key:
[{"id":"ascii","name":"PFX-alpha","nameScan":"PFX-alpha"},
{"error":"ReferenceError: finishUtf8 is not defined"}]Index iteration stops at the first non-ASCII key. A direct getRange over the index store (e.g. from a
custom Resource) does raise, producing a 500 — that is the only path where anything is logged.
So on Bun, any table whose indexed string values are not pure ASCII — accented Latin, CJK, emoji,
anything above U+007F — silently returns partial query results. Non-ASCII string primary keys go
through the same readKey decoder and fail the same way.
Root cause
ordered-binary@1.6.1, index.js:366-384:
const readString =
typeof process !== 'undefined' && process.isBun ?
(function (reading) { /* setPosition / getPosition wrapper */ })(
(new Function('fromCharCode', 'let position; let readString = ' + makeStringBuilder() + ';return {…}'))(fromCharCode)) :
eval(makeStringBuilder());makeStringBuilder() emits if (v >= 0x80) v = finishUtf8(v, source). Node takes the direct eval
branch, which closes over module scope and resolves finishUtf8 (index.js:336). Bun takes the
new Function branch, whose body compiles in global scope — finishUtf8 is unreachable there, so
every multibyte character throws.
finishUtf8 handles all non-ASCII, not just astral, which is why a plain BMP value like PFX-élan
fails identically to an emoji one.
Reproduction (no Harper)
import { toBufferKey, fromBufferKey } from 'ordered-binary';
for (const s of ['PFX-alpha', 'PFX-élan', 'PFX-日本', 'PFX-\u{1F600}tail'])
try { console.log(s, '->', fromBufferKey(Buffer.from(toBufferKey(s)))); }
catch (e) { console.log(s, '-> THREW', e.message); }| input | node v26.2.0 | bun 1.3.14 |
|---|---|---|
PFX-alpha |
PFX-alpha |
PFX-alpha |
PFX-élan |
PFX-élan |
THREW ReferenceError: finishUtf8 is not defined |
PFX-日本 |
PFX-日本 |
THREW |
PFX-😀tail |
PFX-😀tail |
THREW |
Fix
Already committed upstream and unreleased:
kriszyp/ordered-binary@aa2f3a4
(2026-06-21) embeds finishUtf8 and pendingSurrogate into the generated function so they bind to its
local, set/getPosition-synced position. v1.6.1 is tagged at the commit before it, so npm's latest
does not carry it.
Harper needs ordered-binary published at >= that commit and the pin in package.json bumped.
Verification
Reproduced and fixed locally against a fresh dist build, bun 1.3.14, with
integrationTests/database/qa648-astral-startswith.test.ts from #2723:
ordered-binary |
result |
|---|---|
| 1.6.1 as published | 5 of 6 tests fail — matches Integration Tests 5/6 (Bun) on #2723 exactly |
1.6.1 + aa2f3a4 applied |
6 of 6 pass, exit 0, every astral case included |
Coverage note
No integration test put a non-ASCII value in an @indexed attribute before #2723, which is why a defect
this broad survived on the Bun leg.
Refs #2723
Metadata
Metadata
Assignees
Labels
Type
Fields
Priority
Activity
deps: bump ordered-binary to 1.6.2 for the Bun key decoder
deps: bump ordered-binary to 1.6.2 for the Bun key decoder
test: drop the stale finishUtf8 Bun skip in northwind.test.mjs
Promote four regression anchors for shipped fixes (#1887, #1906, #1970,…