Skip to content

Reduce jp.Parse allocations#222

Open
trevorprater wants to merge 1 commit into
ohler55:developfrom
trevorprater:codex/jp-parse-token-allocation
Open

Reduce jp.Parse allocations#222
trevorprater wants to merge 1 commit into
ohler55:developfrom
trevorprater:codex/jp-parse-token-allocation

Conversation

@trevorprater

Copy link
Copy Markdown

This reduces allocations in jp.Parse by avoiding temporary token buffers while scanning dot and recursive-descent child fragments.

The parser previously built a temporary []byte with repeated append calls, then converted it to Child. This patch records the token start offset, advances p.pos until the fragment boundary, and converts the original input slice directly to Child. It also keeps the parser value on the stack in MustParse and preallocates the expression slice with a small capacity for the common short JSONPath case.

Validation:

go test -count=1 ./...

I also ran a temporary differential oracle against the previous parser over 10,058 inputs covering fixed examples from the parser tests, bracket notation, quoted children, unions, slices, filters, invalid syntax, Unicode, escaped strings, and deterministic randomized expressions/byte strings. The candidate output matched the previous parser exactly for String(), BracketString(), and error text.

Benchmark command:

go test -run '^$' -bench '^BenchmarkParse$' -benchmem -count=20 ./jp

benchstat:

Parse-10  175.7n ± 0%  ->  149.4n ± 0%  -14.97% (p=0.000 n=20)
Parse-10  374.0 B/op   ->  326.0 B/op   -12.83% (p=0.000 n=20)
Parse-10  12.00 allocs ->  10.00 allocs -16.67% (p=0.000 n=20)

I found the candidate while experimenting with Sleepy, then reduced the generated patch to the smaller token-scanning change above and validated it manually.

Sleepy proof page - https://sleepy.run/proof

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant