Skip to content

runtime: migrate local state to locality directives - #2210

Merged
xushiwei merged 14 commits into
xgo-dev:mainfrom
cpunion:codex/runtime-locality-directives
Jul 30, 2026
Merged

runtime: migrate local state to locality directives#2210
xushiwei merged 14 commits into
xgo-dev:mainfrom
cpunion:codex/runtime-locality-directives

Conversation

@cpunion

@cpunion cpunion commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Depends on #2209. Follow-up to #2079.

Summary

  • use a pointer-free //llgo:tls slot for the current-G locator
  • use //llgo:gls for caller shadow-stack state and FIPS bypass depth on threaded targets
  • retain the existing dynamic clite/tls handle for sync.Pool, which cannot be represented by a fixed static locality declaration
  • retain pthread.Key in _demo/c/thread/thd.go because the program intentionally demonstrates pthread key isolation
  • remove unused SSA pthread-key lowering helpers
  • update generated-entry goldens now that runtime GLS requires LocalContext installation

Lifecycle choices

  • current G: TLS. It locates the G running on the current M and must be available before the goroutine LocalContext is installed.
  • caller/FIPS state: GLS. It belongs to the logical goroutine and should follow that goroutine if a future scheduler permits migration.
  • sync.Pool cache: per-P/execution-resource state, not goroutine state. The current 1:1 P/M/thread backend therefore uses dynamic TLS. A future movable-P scheduler needs a dynamic P-local facility, not GLS.

TLS and GLS currently share one physical owner in the 1:1 backend, but the directives record the intended future lifetime.

Remaining dynamic TLS and pthread-key boundaries

getg no longer calls pthread_getspecific. Its pthread key remains only as a destructor sidecar for lazy main/foreign-thread runtime contexts; runtime-owned M threads use direct TLS and mexit.

sync.Pool remains the sole clite/tls consumer. Each dynamically created Pool needs its own per-execution-resource slot plus thread-exit victim handoff. Replacing it with one static TLS map adds a never-pruned *Pool key set, a map lookup on every Get/Put, and drops the existing victim behavior. Fully replacing this helper requires a dynamic locality/P-local API and GC-cycle pool cleanup.

_demo/c/thread/thd.go intentionally retains pthread.Key: it is a pthread API demonstration that verifies key isolation between the main and child threads, not runtime-owned locality state.

Bare-metal limitation

The current GLS package-block accessor uses a native TLS address cache. LLVM backends without native TLS relocation support therefore cannot lower even logically GLS state. Bare-metal runtime builds currently have one logical context, so caller/FIPS use ordinary-global fallback files there. This is an implementation limitation of locality lowering, separate from the stdlib-overlay ownership and activation fixes in #2209.

Tests

  • go test ./ssa -count=1
  • focused ssa/cl/internal-build locality and alternate-package tests
  • pure-C external-PCLN integration remains runtime-free
  • runtime module host compilation
  • ./dev/llgo.sh test ./test/llgoext
  • ./dev/llgo.sh test ./test/syncpool
  • ./dev/llgo.sh run _demo/c/thread/thd.go
  • ae-rp2040, arduino-mkr1000, arduino-leonardo, esp32-coreboard-v2, and nintendoswitch target builds

@fennoai fennoai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: runtime locality directive migration

Reviewed by four passes (quality, performance, security, docs) against the full PR diff. The locality-directive migration for getg/setg, caller.go, and the FIPS bypass flag is sound — the currentG uintptr slot is safe because the runtimeContext is AllocRoot/uncollectable, and the FIPS //llgo:gls flag preserves per-entry isolation (zeroed on LocalContext creation). No memory-safety defect was confirmed in those areas.

The findings concentrate in the sync.Pool rewrite and one SSA locality owner-filtering gap. Inline comments below.

Summary of concerns

  • sync.Pool unbounded retention (high): poolLocals is a TLS/root-reachable map keyed by *Pool that is never pruned. Every *Pool ever pinned on a thread — plus its cached objects — is retained for the thread's lifetime, so a discarded Pool can never be collected. Long-running processes that create many short-lived Pools grow without bound.
  • sync.Pool reuse regression (high): getSlow now unconditionally returns nil; the victim cache and cross-P popTail stealing are gone. Objects Put on one thread can never be retrieved by a Get on another, and objects are dropped rather than surviving a GC cycle — the core allocation-amortization benefit of sync.Pool is reduced. If intended, please justify in the PR description and doc/defer-tls-gc.md.
  • sync.Pool hot-path cost (medium): pin() now does a Go map lookup (hash *Pool, probe buckets) on every Get/Put in place of a single atomic.LoadPointer. This is a per-op regression on the type's hot path.
  • SSA owner filtering not applied at lowering (medium): the new PackageLocalitiesFor owner tracking is used in PrepareLocalVariables, but planLocalPackage (the codegen lowering path via localPackageFor) and validateLocalInitializers still use the unfiltered PackageLocalities(path) — see inline.

Non-blocking review (event: COMMENT).

Findings without inline locations

  • runtime/internal/lib/sync/pool.go:124: Stale doc comment: pin() no longer disables preemption, and there is no runtime_procUnpin() anywhere in the package — callers (Get/Put) never unpin. Please update this comment to describe the new TLS-map lookup and drop the unpin obligation.
  • runtime/internal/lib/sync/pool.go:63: After this rewrite, local, localSize, victim, victimSize, and once are dead — grep finds zero reads/writes of p.local/p.victim/p.once/localSize/victimSize in the package. Consider removing them (keeping noCopy/New) or adding a note explaining why they are retained; the field comments ("local from previous cycle", etc.) now describe unused state.

Comment thread runtime/internal/lib/sync/pool.go Outdated
Comment thread runtime/internal/lib/sync/pool.go
Comment thread runtime/internal/runtime/g_tls.go
Comment thread cl/locality.go Outdated
Comment thread ssa/locality.go Outdated
@cpunion
cpunion force-pushed the codex/runtime-locality-directives branch from 3bd7bf7 to 0c3a746 Compare July 29, 2026 05:10
@cpunion
cpunion force-pushed the codex/runtime-locality-directives branch from a4475dc to 05a84d1 Compare July 29, 2026 05:37
@codecov

codecov Bot commented Jul 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@cpunion
cpunion force-pushed the codex/runtime-locality-directives branch 2 times, most recently from 5c1c5f5 to 8e672ef Compare July 29, 2026 06:27
@cpunion
cpunion force-pushed the codex/runtime-locality-directives branch from 8e672ef to 13452a3 Compare July 29, 2026 06:36
@cpunion

cpunion commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator Author

已复核最新 review:当前 5 个 inline threads 均已解决,sync.Pool 动态 TLS/victim 语义、setg lazy auto-G 生命周期以及 owner-aware locality lowering 都已在现有提交中处理。本次另在 583488c 同步 #2209 的 review 清理,删除未使用的 canonical-path helper,并补充 ownerless/legacy API 边界说明;聚焦测试通过。PR 保持 open,未合并。

@cpunion

cpunion commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator Author

补充处理:_demo/c/thread/thd.go 是 pthread key 功能演示,不属于 runtime locality 迁移范围。已在 150c757 恢复 pthread.Key,运行 demo 验证子线程与主线程分别得到 thread value / main value;PR 描述也已明确将其列为有意保留的 pthread-key 边界。

@cpunion

cpunion commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator Author

@fennoai 最新 head 150c757ab 的 CI 已全部通过(38 pass、1 expected skip),Go 双平台 coverage 与 Codecov 汇总均为 SUCCESS,review threads 已全部解决。补充调整中保留了 _demo/c/thread/thd.gopthread.Key,因为它用于演示 pthread key 的线程隔离语义;runtime locality 迁移边界及 sync.Pool 动态 TLS 语义保持不变。麻烦基于最新提交再 review 一次,谢谢。

@fennoai

fennoai Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Re-review complete — LGTM ✅

Re-reviewed the full 24-file diff (+903/-229) against base 2f75ed28 on the latest head 150c757ab. The runtime locality-directive migration is coherent, behavior-preserving, and well-tested. No blocking issues found.

I submitted the detailed findings as a PR review.

Highlights

  • getg/setg lifecycle (g_tls.go) is faithful to the removed g_pthread.go. Runtime-owned M threads use the pointer-free //llgo:tls currentG slot and never touch the pthread key; mexit frees the root exactly once (no double-free — the destructor-transfer branch is unreachable on runtime-owned threads). Lazy main/foreign-thread contexts keep the pthread destructor sidecar.
  • Build tags for the new *_global.go / *_gls.go splits are mutually exclusive and exhaustive; fipsBypassDepth has exactly one definition per configuration and one use site. TestBaremetalRuntimeAvoidsLocalityDirectives directly guards the stated bare-metal limitation.
  • Owner-aware locality metadata (ssa/locality.go, cl/locality.go) cleanly isolates standard vs. alternate packages sharing canonical symbol names. NeedsLocalContext correctly gates on activePackages so inactive alternate packages no longer force a LocalContext. New cl/ssa/build tests cover the ownership edge cases.
  • Cleanup — removed SSA pthread-key helpers have no dangling references; caller.go no longer imports clite/tls.
  • Boundary rationale for retaining sync.Pool dynamic clite/tls and the _demo/c/thread/thd.go pthread.Key demo is well-argued; doc/defer-tls-gc.md now accurately maps the TLS / GLS / dynamic-TLS split. Confirmed unchanged, as you noted.

Non-blocking note

  • fipsbypass_llgo.go carries only //go:build go1.26 (no llgo), so its !llgo variant makes fipsBypassDepth a process-wide global. Fine for the host-compilation overlay path and single-context bare-metal; the _llgo filename slightly undersells that it also compiles for non-llgo hosts.

Verification note

I reviewed correctness by reading. I could not run the Go test suite locally — the sandbox has no LLVM C headers, so the ssa / cl / internal/build cgo packages don't compile here. Build/test verification relies on the green CI you reported at 150c757ab (38 pass, 1 expected skip; Go dual-platform coverage + Codecov SUCCESS).

View job run

@fennoai fennoai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review at 150c757ab — LGTM

Reviewed the full 24-file diff against base 2f75ed28 on the latest head. The locality-directive migration is coherent, behavior-preserving, and well-tested. No blocking issues found. I could not run the Go test suite locally (the sandbox has no LLVM C headers, so the ssa/cl/internal/build cgo packages don't compile here); I relied on the green CI you reported at 150c757ab for build/test verification and reviewed correctness by reading.

What I verified

getg/setg lifecycle (runtime/internal/runtime/g_tls.go) — faithful to the removed g_pthread.go:

  • Runtime-owned M threads (mstartsetg(gp)) never touch the pthread key: currentGHasLifecycle stays false on a fresh thread, so setg only writes the //llgo:tls currentG slot. mexit calls setg(nil) (lifecycle-false path → currentG = 0) and frees the root exactly once — no double free, since the destructor-transfer branch is unreachable on runtime-owned threads.
  • Lazy main/foreign-thread contexts still install the pthread destructor via setAutoG, matching the old getg path plus the new TLS fast read.
  • The setg ownership-transfer branch (old != gpdestroyG(old)) is defensive for a future thread-reuse case; harmless today and correctly avoids freeing a transferred same-G root.

Build-tag partitioning — the new split files are mutually exclusive and exhaustive:

  • caller: (!llgo || baremetal) vs (llgo && !baremetal).
  • fipsbypass: both gated on go1.26, same partition; fipsBypassDepth has exactly one definition per configuration and one use site (fipsbypass_llgo.go, tag go1.26), so no missing/duplicate symbol.
  • TestBaremetalRuntimeAvoidsLocalityDirectives guards the stated bare-metal limitation directly.

Owner-aware locality metadata (ssa/locality.go, cl/locality.go) — the *For(pkg,...) API cleanly isolates standard vs. alternate (llgo/_pkg) packages that share canonical symbol names. NeedsLocalContext correctly gates declaration entries on activePackages and skips legacy entries that are shadowed by ownerless/declaration maps, so inactive alternate packages no longer force a LocalContext. PrepareInactiveLocalVariables + the build driver's active-root walk (internal/build/build.go) implement this end-to-end. Ownership edge cases are covered by the new cl/ssa/build tests.

Cleanup — removed SSA pthread-key helpers (pthreadKeyCreate/Getspecific/Setspecific + their signature caches) have no remaining references; caller.go no longer imports clite/tls.

Non-blocking observations

  • runtime/internal/lib/runtime/fipsbypass_llgo.go carries only //go:build go1.26 (no llgo), yet the !llgo global-state variant makes fipsBypassDepth a process-wide global rather than per-thread. This is fine for the overlay's host-compilation path and for single-context bare-metal, but the file's _llgo name slightly undersells that it also compiles for non-llgo hosts — worth a one-line note if it ever gains real host runtime semantics.
  • The retention rationale for sync.Pool (dynamic clite/tls) and the _demo/c/thread/thd.go pthread.Key demo are well-argued in the PR body and doc/defer-tls-gc.md; agreed these are the correct boundaries under the current 1:1 P/M/thread backend. The doc now reads as an accurate map of the TLS/GLS/dynamic-TLS split.

Confirming your notes: the migration boundaries and sync.Pool dynamic-TLS semantics are unchanged, and the thd.go pthread-key demo is intentionally preserved. Nice work on the directive-lifetime documentation — the "record intended future lifetime even though TLS/GLS share one owner today" framing is clear.

@xushiwei
xushiwei merged commit ab2fe9c into xgo-dev:main Jul 30, 2026
39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants