Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,17 @@ for tags and release notes while still in `0.x`.

### Performance

- Reduced the temporary recovery memo tier from 262,144 entries to 131,072
entries. The active table now uses 3 MiB on 64-bit systems. The retained
standard table uses 384 KiB. Total memo storage is 3.375 MiB before slice
overhead. The 20-seed C# issue #454 benchmark reduces bytes per operation
by `20.79%`, with `p<0.001`. Time is `155.9` ms for base and `158.0` ms for
the changed build, with `p=0.529`. Allocations are `998.5` and `998.0`, with
`p=0.074`. Both results are inconclusive. Two execution orders reduce
maximum resident set size by `11,520` KiB and `4,160` KiB. The primary trio
is order-sensitive and inconclusive. Focused C# correctness, locked-C
parity, and memory-budget contract tests pass.

- Recorded P25aw at merged main commit
`af9ded2b77b7828b12b1d2da7c9fff8dd5ca053b`. Four authenticated Swift corpus
edits used old trees and one final-newline edit. Every profile reported
Expand Down
69 changes: 67 additions & 2 deletions docs/perf-attribution.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,73 @@ every function that more than one component could otherwise claim. It
records the noise floor of the local measurement host, and the first
published receipt.

This is measurement infrastructure only. It changes no parser code, no
routing, and no shipped behavior.
This document records measurement evidence for parser changes and unchanged
controls.

## 2026-08-24 Recovery memo tier reduction

The benchmark base is commit
`8e8f18b82a9e122499e90b364e7e779f03af8679`. This commit adds only
`BenchmarkCSharpRecoveryMemoTier32KiB`. The benchmark function SHA-256 is
`bfb21482aa8f2f268293357e02ab605f7a8b1b87b0c367dafc93872b62ab004b`.

The parser change reduces the temporary recovery memo from 262,144 entries to
131,072 entries. The packed entry uses 24 bytes on 64-bit systems. The change
therefore lowers the active table from 6,291,456 bytes to 3,145,728 bytes.
The retained standard table uses 393,216 bytes. Total memo storage is
3,538,944 bytes before slice overhead. These values equal 3 MiB, 384 KiB, and
3.375 MiB.

The 32 KiB C# issue #454 benchmark used one CPU, 20 shuffle seeds, 750
milliseconds, and benchmark memory reporting. The command was:

```text
GOMAXPROCS=1 scripts/run_randomized_benchmarks.sh --output <report> --runs 20 --seed-start 1 --benchtime 750ms --bench-regex '^BenchmarkCSharpRecoveryMemoTier32KiB$' --package ./grammars --tags gts_parsercorephase0
```

Benchstat reports these changes:

- Time: base `155.9` ms and changed build `158.0` ms, with `p=0.529`.
The result is inconclusive.
- Bytes: `-20.79%`, with `p<0.001`.
- Allocations: base `998.5` and changed build `998.0`, with `p=0.074`.
The result is inconclusive.

The resident-memory runs used this command inside a one-CPU Docker container:

```text
/usr/bin/time -v env GOMAXPROCS=1 go test ./grammars -run '^$' -bench '^BenchmarkCSharpRecoveryMemoTier32KiB$' -benchtime=750ms -count=10 -benchmem -timeout 20m -v
```

The base-first order reports a maximum resident set size (RSS) of `253,920`
KiB for base and `242,400` KiB for the changed build. The changed-build-first
order reports `254,400` KiB for the changed build and `258,560` KiB for base.
The two orders reduce RSS by `11,520` KiB and `4,160` KiB.

The full-tier attribution recorded 210,781 unique subtree pointers. Peak live
entries ranged from 161,967 to 164,432. The cache recorded 103,634 to 104,405
two-way collisions and 37,085 to 38,057 evictions. It crossed one standard
resize and one temporary resize. The set lookup hit rate was about `71.5%`.

The 131,072-entry attribution recorded the same 210,781 unique subtree
pointers. Peak live entries ranged from 116,975 to 117,147. The cache recorded
151,908 to 152,328 collisions and 88,643 to 88,849 evictions. It crossed one
standard resize and one temporary resize. The set lookup hit rate was about
`71.1%`.

The memo remains an evictable cache. Every miss recomputes the exact subtree
cost and visible-node count. Focused C# correctness, locked-C parity, and
cache-size memory-budget contract tests passed.

The primary trio used this command:

```text
GOMAXPROCS=1 scripts/run_randomized_benchmarks.sh --output <report> --runs 20 --seed-start 1 --benchtime 750ms --bench-regex '^(BenchmarkGoParseFullDFA|BenchmarkGoParseIncrementalSingleByteEditDFA|BenchmarkGoParseIncrementalNoEditDFA)$' --package .
```

The primary trio is order-sensitive and inconclusive. Base-first runs change
the geometric mean by `+5.13%`. Changed-build-first runs change it by `-8.50%`.
The allocation counts are identical.

## 2026-08-24 P25bb recovery election-summary contract

Expand Down
22 changes: 22 additions & 0 deletions grammars/csharp_issue454_regression_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,28 @@ func BenchmarkIssue454CSharpRecoveredFullParse(b *testing.B) {
}
}

func BenchmarkCSharpRecoveryMemoTier32KiB(b *testing.B) {
lang := grammars.CSharpLanguage()
source := issue454CSharpSource(32 * 1024)
site := bytes.Index(source, []byte("x0"))
if site < 0 {
b.Fatal("C# edit marker is absent")
}
source = append(append([]byte(nil), source[:site]...), source[site+1:]...)
parser := gotreesitter.NewParser(lang)
parser.SetAdmissionCandidateRoute(false)
b.ReportAllocs()
b.SetBytes(int64(len(source)))
b.ResetTimer()
for i := 0; i < b.N; i++ {
tree, err := parser.Parse(source)
if err != nil {
b.Fatal(err)
}
tree.Release()
}
}

func issue454CSharpSource(targetBytes int) []byte {
var source strings.Builder
source.Grow(targetBytes + 256)
Expand Down
6 changes: 4 additions & 2 deletions parser_recover_c.go
Original file line number Diff line number Diff line change
Expand Up @@ -1603,8 +1603,10 @@ const (
cNodeMemoCacheSize = 16384
// cNodeMemoRecoveryCacheSize is a temporary second tier for recovery
// parses that continue to collide after the standard cache grows. The
// packed 24-byte entry makes this tier 6 MiB on 64-bit systems.
cNodeMemoRecoveryCacheSize = 262144
// packed 24-byte entry makes this active tier 3 MiB on 64-bit systems.
// The retained 384 KiB standard tier makes total storage 3.375 MiB before
// slice overhead.
cNodeMemoRecoveryCacheSize = 131072
// cNodeMemoRecoveryThrashGrowThreshold requires one collision per standard
// cache entry before the parser allocates the temporary recovery tier.
cNodeMemoRecoveryThrashGrowThreshold = cNodeMemoCacheSize
Expand Down
25 changes: 25 additions & 0 deletions parser_recover_c_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2590,6 +2590,31 @@ func TestRecoveryNodeMemoTierMetrics(t *testing.T) {
}
}

func TestCNodeMemoRecoveryTierFitsMemoryBudget(t *testing.T) {
if cNodeMemoRecoveryCacheSize&(cNodeMemoRecoveryCacheSize-1) != 0 {
t.Fatalf("temporary memo entries = %d, want a power of two", cNodeMemoRecoveryCacheSize)
}
if cNodeMemoRecoveryCacheSize <= cNodeMemoCacheSize {
t.Fatalf("temporary memo entries = %d, want more than standard entries %d", cNodeMemoRecoveryCacheSize, cNodeMemoCacheSize)
}
const (
recoveryMemoActiveBudgetBytes = 3 << 20
recoveryMemoRetainedBudgetBytes = 384 << 10
recoveryMemoCombinedBudgetBytes = recoveryMemoActiveBudgetBytes + recoveryMemoRetainedBudgetBytes
)
activeBytes := cNodeMemoCacheBytesForEntries(cNodeMemoRecoveryCacheSize)
if activeBytes > recoveryMemoActiveBudgetBytes {
t.Fatalf("active temporary memo bytes = %d, exceeds budget %d", activeBytes, recoveryMemoActiveBudgetBytes)
}
retainedBytes := cNodeMemoCacheBytesForEntries(cNodeMemoCacheSize)
if retainedBytes > recoveryMemoRetainedBudgetBytes {
t.Fatalf("retained standard memo bytes = %d, exceeds budget %d", retainedBytes, recoveryMemoRetainedBudgetBytes)
}
if combinedBytes := activeBytes + retainedBytes; combinedBytes > recoveryMemoCombinedBudgetBytes {
t.Fatalf("combined memo bytes = %d, exceeds budget %d", combinedBytes, recoveryMemoCombinedBudgetBytes)
}
}

// TestCNodeMemoSlotAdaptiveGrowFiresExactlyAtThrashThreshold is a mechanism-
// level (no real parse, no wall clock) proof of the W2 adaptive growth
// trigger (issue #380/#388): cNodeMemoSlot must grow the cache from
Expand Down
Loading