Skip to content

Bun: indexed queries silently truncate at the first non-ASCII key (ordered-binary readString ReferenceError) #2750

Closed

Description

@kriszyp

On the Bun runtime, ordered-binary@1.6.1 (the version every shipped 5.2.x / 5.3.x tag pins) throws
ReferenceError: finishUtf8 is not defined while decoding any key that contains a byte >= 0x80.
Harper surfaces that as a truncated result set with HTTP 200 and nothing in hdb.log.

Node is unaffected. This is a runtime-specific defect in a dependency, not in resources/search.ts.

Impact

A search_by_conditions range/starts_with query on an @indexed string attribute returns 200 with
the decode failure embedded as an element of the result array, in place of every row after the first
non-ASCII key:

[{"id":"ascii","name":"PFX-alpha","nameScan":"PFX-alpha"},
 {"error":"ReferenceError: finishUtf8 is not defined"}]

Index iteration stops at the first non-ASCII key. A direct getRange over the index store (e.g. from a
custom Resource) does raise, producing a 500 — that is the only path where anything is logged.

So on Bun, any table whose indexed string values are not pure ASCII — accented Latin, CJK, emoji,
anything above U+007F — silently returns partial query results. Non-ASCII string primary keys go
through the same readKey decoder and fail the same way.

Root cause

ordered-binary@1.6.1, index.js:366-384:

const readString =
  typeof process !== 'undefined' && process.isBun ?
    (function (reading) { /* setPosition / getPosition wrapper */ })(
      (new Function('fromCharCode', 'let position; let readString = ' + makeStringBuilder() + ';return {…}'))(fromCharCode)) :
    eval(makeStringBuilder());

makeStringBuilder() emits if (v >= 0x80) v = finishUtf8(v, source). Node takes the direct eval
branch, which closes over module scope and resolves finishUtf8 (index.js:336). Bun takes the
new Function branch, whose body compiles in global scope — finishUtf8 is unreachable there, so
every multibyte character throws.

finishUtf8 handles all non-ASCII, not just astral, which is why a plain BMP value like PFX-élan
fails identically to an emoji one.

Reproduction (no Harper)

import { toBufferKey, fromBufferKey } from 'ordered-binary';
for (const s of ['PFX-alpha', 'PFX-élan', 'PFX-日本', 'PFX-\u{1F600}tail'])
	try { console.log(s, '->', fromBufferKey(Buffer.from(toBufferKey(s)))); }
	catch (e) { console.log(s, '-> THREW', e.message); }
input node v26.2.0 bun 1.3.14
PFX-alpha PFX-alpha PFX-alpha
PFX-élan PFX-élan THREW ReferenceError: finishUtf8 is not defined
PFX-日本 PFX-日本 THREW
PFX-😀tail PFX-😀tail THREW

Fix

Already committed upstream and unreleased:
kriszyp/ordered-binary@aa2f3a4
(2026-06-21) embeds finishUtf8 and pendingSurrogate into the generated function so they bind to its
local, set/getPosition-synced position. v1.6.1 is tagged at the commit before it, so npm's latest
does not carry it.

Harper needs ordered-binary published at >= that commit and the pin in package.json bumped.

Verification

Reproduced and fixed locally against a fresh dist build, bun 1.3.14, with
integrationTests/database/qa648-astral-startswith.test.ts from #2723:

ordered-binary result
1.6.1 as published 5 of 6 tests fail — matches Integration Tests 5/6 (Bun) on #2723 exactly
1.6.1 + aa2f3a4 applied 6 of 6 pass, exit 0, every astral case included

Coverage note

No integration test put a non-ASCII value in an @indexed attribute before #2723, which is why a defect
this broad survived on the Bun leg.

Refs #2723

Activity

  1. added this to the v5.3 milestone on Sep 22, 2026
  2. added theissue type on Sep 22, 2026
  3. added 3 commits that reference this issue on Sep 22, 2026
    c4a78c1
    122f6d1
    f31a615
  4. added a commit that references this issue on Sep 28, 2026
    ba4978c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    Priority

    P1

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions