Skip to content

fix(hash): keep the folded salt-key constant an ArrayVector - #205

Merged
nooga merged 1 commit into
mainfrom
fix/folded-salt-vector-type
Sep 21, 2026
Merged

nooga merged 1 commit into
mainfrom
fix/folded-salt-vector-type

Conversation

@nooga

@nooga nooga commented Sep 21, 2026

Copy link
Copy Markdown
Owner

with-folded-salt's fully-literal branch crashes at runtime for any salt-key encoding to 33 bytes or more. Fixes #204.

What was happening

The branch encodes the whole salt-key at expansion time and splices the byte buffer into the emitted form as a compile-time constant. encode-salt builds that buffer with conj, and let-go's conj promotes ArrayVector to PersistentVector above 32 elements.

let-go's compileForm type switch has a case for ArrayVectorType but none for PersistentVectorType, and no default: — so the form emits no bytecode at all. The expansion compiled clean and then panicked on a short stack when called:

(defn f [seed] (with-folded-salt seed [:combat :hit 12 34]))
(f 42)   ;=> error: runtime error: index out of range [-1]

33 bytes is not an exotic size. The macro's own docstring example, [:combat :hit attacker target], reaches it the moment its variable terms are literals.

The fix

One vec, which rebuilds the buffer as an ArrayVector.

into would not do: it goes through conj and promotes straight back. The macro's other three splice sites already go through vec, which is why this was the only branch affected — and why the partial-literal path, which people actually hit in the inner loop, kept working and hid this.

Filed upstream as nooga/let-go#943 with a core-only 8-line repro. That issue also covers the missing default:, since SortedMapType and SortedSetType are absent from the same switch and fail identically — it is a class of silent miscompile, not one missing case. This workaround comes out when the upstream fix ships; the comment says so and names the test that will catch it going early.

The test

Straddles the boundary with one character of keyword name:

salt-key bytes before
[:combat :hi 1 2] 32 ok
[:combat :hit 1 2] 33 panic

Also covers a 48-byte key mixing strings with keywords and ints, and multi-byte UTF-8 in a folded string term.

Worth knowing: this guard cannot fail softly. Drop the vec and the panic takes the whole suite down rather than reporting an assertion failure. For a silent miscompile that is the point. Verified both directions — lg xsofy/test/run.lg exits 1 without the fix, 0 with it, so CI does catch it.

Why nothing caught it before

with-folded-salt-equals-combine-fully-literal uses [:combat :hit] — 15 bytes, under the threshold. The other three with-folded-salt tests exercise the partial-literal, no-literal-prefix and non-vector paths, which build their vectors at runtime where a PersistentVector is harmless. So the fully-literal path had coverage that could not fail.

Verification

  • 331 tests, 2870 assertions, 0 failures (was 330/2864)
  • make smoke-lg passes
  • Hashes are unchanged: vec produces an equal vector, and the existing golden-value tests pin that

Branched off main; touches only with-folded-salt, so it is independent of #202.

🤖 Generated with Claude Code

`with-folded-salt`'s fully-literal branch encodes the whole salt-key at
expansion time and splices the byte buffer into the emitted form as a
compile-time constant. `encode-salt` builds that buffer with `conj`, and
let-go's `conj` promotes `ArrayVector` to `PersistentVector` above 32
elements.

`compileForm`'s type switch has a case for `ArrayVectorType` but none for
`PersistentVectorType`, and no `default:`, so a `PersistentVector` form
emits no bytecode at all. The macro expansion compiled clean and then
panicked `index out of range [-1]` on a short stack when called.

So every fully-literal salt-key encoding to 33 bytes or more was a
runtime crash. That is not an exotic size — the macro's own docstring
example, [:combat :hit attacker target], reaches it the moment its
variable terms are literals:

    (defn f [seed] (with-folded-salt seed [:combat :hit 12 34]))
    (f 42)   ;=> index out of range [-1]

`vec` rebuilds the buffer as an `ArrayVector`, which the compiler does
handle. `into` would not work here: it goes through `conj` and promotes
straight back. The macro's other three splice sites already go through
`vec`, which is why only this branch broke.

Filed upstream as nooga/let-go#943, which also covers the missing
`default:` — the sorted map and set types are absent from the same switch
and fail identically, so this is a class of silent miscompile rather than
one missing case. This workaround comes out when that ships.

The regression test straddles the boundary with one character of keyword
name: [:combat :hi 1 2] is 32 bytes and always worked, [:combat :hit 1 2]
is 33 and panicked. It also covers a 48-byte key mixing strings with
keywords and ints, and multi-byte UTF-8 in a folded string term. Note it
cannot fail softly — drop the `vec` and the panic takes the whole suite
down rather than reporting an assertion failure, which for a silent
miscompile is the point. Verified: `lg xsofy/test/run.lg` exits 1 without
the fix and 0 with it.

The existing coverage could not have caught this.
with-folded-salt-equals-combine-fully-literal uses [:combat :hit] at 15
bytes, and the other three with-folded-salt tests exercise paths that
build their vectors at runtime, where a PersistentVector is harmless.

331 tests, 2870 assertions, 0 failures. Layer-1 smoke passes.

Refs #204
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@nooga
nooga requested a review from mparrett September 21, 2026 09:33

@mparrett mparrett left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No findings.

Verified locally:

  • Reproduced the pre-fix index out of range [-1] panic at the 33-byte boundary on the base commit.
  • Confirmed the patched expansion forces the folded constant back to ArrayVector and returns the same hash as combine.
  • lg xsofy/test/run.lg: 331 tests, 2,870 assertions, 0 failures, 0 errors.
  • make smoke-lg: passes.

The workaround is narrowly scoped, and the regression test covers both sides of the 32/33-byte boundary plus larger and multibyte inputs.

@nooga
nooga merged commit d0aa017 into main Sep 21, 2026
4 checks passed
@nooga
nooga deleted the fix/folded-salt-vector-type branch September 21, 2026 11:46
@github-actions

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-09-21 11:46 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

with-folded-salt: fully-literal path crashes at runtime for salt-keys encoding to 33+ bytes

2 participants