Skip to content

add(diagnostics, tests): Add parsing fork instrumentation and parity probes - #911

Closed
odvcencio wants to merge 2 commits into
mainfrom
codex/archive-sweep-wins-20260823
Closed

add(diagnostics, tests): Add parsing fork instrumentation and parity probes#911
odvcencio wants to merge 2 commits into
mainfrom
codex/archive-sweep-wins-20260823

Conversation

@odvcencio

Copy link
Copy Markdown
Owner

Summary

This branch adds diagnostic token source forking and parity probes for several grammars. The new code captures parser state and GLR stack data during forks. Retry-paired benchmarks report wall times, allocations, and digests. Probes validate Doxygen, Templ, Wolfram, and OCaml parses against the locked C oracle. Exact-language tests isolate normalization paths by disabling admission routing. The changelog removes the archived WGSL blocker note. These changes support attribution work without changing public APIs.

Changes

  • Fork token sources for diagnostics while keeping all mutable state independent from the live parser.
  • Record retry attempts, merge stacks, and node counts inside retry-paired incremental benchmarks.
  • Validate raw, production, compact, forest, and incremental routes against the locked C oracle for Doxygen, Templ, Wolfram, and OCaml.
  • Isolate normalization paths by running exact-language route tests with admission candidate routing disabled.
  • Remove the archived WGSL blocker record from the changelog and update retirement documentation.
  • Check materialization cycle regressions and verify C token source fork isolation.

Testing

  • Run go test ./cgo_harness -run 'TestTemplNextLiveArmLockedCRoutes|TestWolframDispatchRetirementLockedCParity|TestDoxygenRegisteredWitnessesLockedCDeepParity' -tags=cgo,treesitter_c_parity
  • Run go test ./cgo_harness -run TestP25vRetryPaired -tags=cgo,treesitter_c_parity
  • Run go test . -run TestDiagnosticIncludedRangeForkIsIndependent -tags=gts_workcount

@odvcencio odvcencio closed this Aug 23, 2026
@odvcencio
odvcencio force-pushed the codex/archive-sweep-wins-20260823 branch from a42c5b8 to af9ded2 Compare August 23, 2026 21:29
- Add locked C oracle parity checks for Doxygen, Templ, Wolfram, WGSL, and Objective-C inputs.
- Introduce benchmark cases for incremental recovery deletion and early newline edits.
- Provide diagnostic token source forking to capture parser state and range digests during retries.
- Guard against cyclic node references in trees produced by repeated concurrent parses.
- Record Doxygen normalization blocker receipts and expected dispatch rewrite counts.

Buckley-Change-Hash: sha256:9f86426c9a11d6035e0f27923664d3c8dea2ae32d0e89f688c26220058a61e19
Buckley-Change-Stats: files=20 insertions=4265 deletions=0 binaries=0
- Add external scanner checkpoint lifecycle to manage payload ownership across parse epochs and GLR branches.
- Verify checkpoint deserialization before accepting scan results. Roll back state on failed restores.
- Track version forks and merge exact duplicates. Discard mismatched branches after failed scans.
- Add unit tests for lifecycle opt-in, failed restore cleanup, exact versus mismatched fork handling, and payload isolation.
- Extend CGO parity harness with locked-C verification for BitBake and Corn normalization fixtures.
- Add Swift probe tests for associated type conformance witnesses and masking shift operators.
- Validate Rust dispatch routes against the locked C deep digest.
- Expand Issue 454 diagnostics across five file sizes and four target languages.

Buckley-Change-Hash: sha256:31565164b2d60bd430f5c14f7d419d021adbe92d98b486f18ce23a9d673e5bb9
Buckley-Change-Stats: files=12 insertions=2026 deletions=0 binaries=0
@odvcencio

Copy link
Copy Markdown
Owner Author

Archived working state belongs in Hyphae, not the merge train. This PR will not proceed.

@odvcencio odvcencio closed this Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant