Skip to content

feat: Epoch nursery allocator with @compact primitive#43

Open
gavlooth wants to merge 1 commit intoHigherOrderCO:mainfrom
gavlooth:epoch-allocator
Open

feat: Epoch nursery allocator with @compact primitive#43
gavlooth wants to merge 1 commit intoHigherOrderCO:mainfrom
gavlooth:epoch-allocator

Conversation

@gavlooth
Copy link

Summary

  • Epoch nursery allocator: Splits HEAP into stable region + per-thread nursery. At epoch boundary, nursery resets in O(1) pointer rewinds. Leverages IC's cycle-freedom: after SNF, all nursery data except the result tree is dead — no tracing needed.
  • @compact primitive: @compact(term) normalizes to SNF, deep-copies live result to stable region, resets nursery. Enables iterative algorithms (Bellman-Ford, etc.) that would otherwise exhaust the heap.
  • 9 safety hardening fixes from production audit:
    1. CRITICAL: Nested @compact support via EPOCH_DEPTH counter — inner compacts don't corrupt outer epoch
    2. HIGH: Null sentinel guard (loc==0) in compact_deep_copy — prevents infinite spin-wait on HEAP[0]
    3. HIGH: COMPACT_MAX_DEPTH (64K) recursion limit — prevents stack overflow on deep SNF trees
    4. MEDIUM: epoch_init validates stable_fraction >= 2
    5. MEDIUM: epoch_reset_zero warns about spin-wait amplification
    6. MEDIUM: F_EQL_R ext-field stale pointer documented
    7. MEDIUM: epoch_bench calls thread_set_count(1) before running
    8. LOW: epoch_reset_stable() helper replaces direct STABLE_NEXT manipulation
    9. LOW: Removed dead EPOCH_NUM variable

New files

File Lines Purpose
clang/heap/epoch.c 354 Epoch infrastructure: init, begin/reset, stable alloc, stats, benchmarks
clang/heap/free.c 8 Stub for heap_free_reset() backward compatibility
clang/prim/fn/compact.c 122 @compact primitive with epoch-aware deep copy
clang/docs/EPOCH_ALLOCATOR.md 120 Architecture documentation

CLI

./main <file.hvm4> --epoch        # Enable epoch mode (stable = 1/4 HEAP)
./main <file.hvm4> --epoch=8      # Stable = 1/8 HEAP (larger nursery)
./main --epoch-bench              # Run allocation microbenchmarks

Test results

  • Single-threaded: 190 PASS, 0 FAIL, 6 SKIP (4 intentional parse errors + 2 collapse-mode)
  • Multi-threaded (-T4): 187 PASS, 0 FAIL, 6 SKIP, 2 flaky (pre-existing MT non-determinism)
  • Zero regressions: All tests produce identical results with and without --epoch

Benchmark (--epoch-bench)

Nursery alloc: 192 M allocs/s (zero overhead vs current allocator)
Epoch reset:   1 ns/reset (O(threads) pointer rewind)
Stable alloc:  195 M allocs/s
Multi-epoch:   0.16 ms/epoch (100K nursery allocs + 1K stable promotions)

🤖 Generated with Claude Code

Epoch-based memory reclamation for iterative HVM4 programs.
Leverages IC's cycle-freedom guarantee: after normalization to SNF,
all nursery data except the result tree is dead, making O(1) bulk
reclaim provably safe without tracing.

New files:
- heap/epoch.c: Epoch nursery infrastructure (stable/nursery split,
  begin/reset lifecycle, nesting support, benchmarks)
- heap/free.c: Stub for heap_free_reset() backward compatibility
- prim/fn/compact.c: @compact(term) primitive — normalize, deep-copy
  to stable region, reset nursery
- docs/EPOCH_ALLOCATOR.md: Architecture documentation

Safety hardening (9 fixes from production audit):
1. CRITICAL: Nested @compact nesting support via EPOCH_DEPTH counter
2. HIGH: Null sentinel guard (loc==0) in compact_deep_copy DP/VAR loop
3. HIGH: COMPACT_MAX_DEPTH (64K) recursion limit with graceful fallback
4. MEDIUM: epoch_init validates stable_fraction >= 2
5. MEDIUM: epoch_reset_zero warns about spin-wait amplification
6. MEDIUM: F_EQL_R ext-field stale pointer documented
7. MEDIUM: epoch_bench calls thread_set_count(1) before running
8. LOW: epoch_reset_stable() helper replaces direct STABLE_NEXT access
9. LOW: Removed dead EPOCH_NUM variable

CLI: --epoch, --epoch=N, --epoch-bench
Test results: 190 ST PASS, 187 MT PASS, zero regressions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants