coro: activate deferred syscall replacements from requests

.github/workflows/coroutine.yml

@@ -108,6 +108,7 @@ jobs:
             ./internal/runtime/coro_execution_quota_default.go \
             ./internal/runtime/coro_ready_distribution_default.go \
             ./internal/runtime/coro_native_fleet.go \
+            ./internal/runtime/coro_native_deferred_replacement_default.go \
             ./internal/runtime/coro_native_fleet_reactor.go \
             ./internal/runtime/coro_native_atomic_host.go \
             ./internal/runtime/coro_native_fleet_test.go \

doc/coro-performance-baseline.md

@@ -1805,3 +1805,46 @@ runtime package, 20-repeat native standby, and focused compiler syscall/worker
 tests also pass. The next file-performance target is the remaining coroutine
 frame chain through `io.ReadFull`, `os.File` and `internal/poll`, not another
 file-specific runtime path.
+
+### Request-driven deferred replacement checkpoint
+
+The follow-up on 2026-08-14 closes the direct-only cross-route liveness gap
+without restoring eager compensation. A blocking native syscall now reserves
+the exact logical handoff and a replacement directory slot, releases its
+managed-execution permit, and arms one pointer-free 32-bit dispatch gate. It
+does not consume a physical M unless a durable executor request wins the gate.
+Every native source uses one common request tail: preserve the existing
+doorbell transport first, then activate an armed replacement for an accepted
+request. A quick syscall return withdraws the arm with one CAS; a request winner
+uses the existing generation-bound cancel, return, and strong-recycle path.
+
+The new linked gate constructs the dependency directly. Route A's sole current
+M blocks in a real compiler-certified `llgo.syscall`; route B completes a direct
+channel rendezvous for a waiter on A; the accepted request starts A's deferred
+replacement; the resumed waiter releases the C call. There is no timer, poll,
+worker, or additional runtime waiter in the cycle. The scenario and the nearby
+same-route/timer/poll/nested/retirement replacement set pass five repeated runs.
+
+The exact comparison parent is merge `a9bb968259c791c3eaaefd8a0db9a327647449f0`.
+The parent and candidate use the same LLVM 22 compiler/workload artifacts and
+31-process medians. A background Docker build was active on the machine, so
+these measurements are a regression guard rather than a new Go comparison:
+
+| 500-operation workload | parent | deferred candidate | delta |
+| --- | ---: | ---: | ---: |
+| direct `syscall.Seek/Write/Seek/Read` | 2.853167 ms | 3.043458 ms | +6.7% |
+| standard `os.File`/`io.ReadFull` chain | 7.441375 ms | 7.461333 ms | effectively neutral |
+
+The direct boundary pays roughly 95 ns per syscall in this fixture for logical
+handoff/slot prepare plus the uncontended Arm/Withdraw race; no physical thread
+is started. The standard-library path is unchanged within process noise. The
+stripped artifact grows from 7,050,672 to 7,129,760 bytes (+79,088, about 1.1%).
+This is accepted as the temporary correctness gate; follow-up profiling should
+reduce the prepare/rollback cost without weakening the direct-only liveness
+proof.
+
+One pre-existing `LockedOrdinarySuspend` stress case remains flaky: the merge
+parent failed 1 of 20 isolated runs with the same bare `abort trap`, while the
+candidate reproduced a similar rate. It does not exercise the deferred syscall
+path, so this checkpoint does not claim to fix or regress that independent
+native-fleet race.

doc/llvm-coro-runtime-design.md

@@ -3178,13 +3178,38 @@ event source 只要复用 common park/detach 协议，就自动进入这项判
 
 未匹配 direct-channel waiter 不能独立完成，因此不能等同 timer/poll，否则一个
 常驻 runtime waiter 会使每个短 syscall 都启动 M。producer 完成匹配后必须先发布
-typed completion，再发布 exact executor request。完整的跨 route 闭环还要求：若
-目标 route 正处于“已释放 lease、原 M 阻塞、尚无 replacement”的窗口，这个 request
-必须能按需激活预留的 replacement。该 follow-up 应复用
-`ExecutionDomainHandoff` 和 active-M directory；不得重新把所有 channel waiter
-加入 eager gate，也不得引入第二套 channel 调度器。在此闭环验收前，timer/poll
-驱动的 blocking compensation 已验证，但“remote producer 唤醒目标 G，而该 G
-反过来解除原 syscall”的 direct-only 环不能标为完成。
+typed completion，再发布 exact executor request。该 request 现在通过公共物理 tail
+按需激活预留的 replacement，不把未匹配 waiter 加入 eager gate，也不
+引入第二套 channel 调度器。
+
+这个闭环复用 `ExecutionDomainHandoff` 和 active-M directory，只在稳定的
+parent M owner 中增加一个 pointer-free 32-bit `DeferredExecutorHandoff`：
+
+1. syscall owner 预先建立 exact logical handoff 并保留 replacement directory slot，
+   但不请求物理 M；
+2. 释放 managed-execution lease 后发布 `Armed`，再重查一次已存在
+   demand，关闭 request-before-arm 窗口；
+3. 每个 source 在事实与 executor request 都持久化后，先完成原有
+   doorbell transport，再调用公共 activation tail；
+4. accepted request 以单次 CAS 把 `Armed -> Starting`，随后发布
+   `Queued` 或 `Started`；syscall 快速返回则以同一个 CAS 尝试
+   `Armed -> Idle`；
+5. request 胜出时继续复用 generation-bound return/cancel/recycle，完成 strong
+   recycle 后才清回 `Idle`。
+
+`Starting` 只是一个有界的发布窗口，不是新的 scheduler wait state。slot 只是
+路由 hint；延迟到达的 accepted request 可与同一稳定 parent 下的下一次
+armed call 合并，而真正防止 stale physical owner 的 authority 仍是
+`ExecutionDomainHandoff` generation 和 replacement baton。该 gate 不保存 stack
+boundary、G/P、LLVM coroutine handle 或函数地址，因此 request 从其他 route
+到达时不需要反向解析编译器对象。
+
+已接入该公共 tail 的生产者包括 Manual/Worker/Poll/TaskControl ingress、
+keyed post、direct channel completion、controlled timer、ready distribution 和普通
+executor request。linked E2E 已验证：目标 route 的 sole M 在真实
+`llgo.syscall` C 调用中阻塞，另一 route 的 direct-channel producer 发布
+completion/request，request-driven replacement 恢复目标 waiter，waiter 又解除
+原 syscall。该测试不借助 timer、poll、worker 或额外 runtime waiter。
 
 物理 M 使用 generation 化的异步 standby request/cancel：request 只把 clean M 从
 standby 转入 `DISPATCHING`，不等待 C->Go；原 syscall 快速返回时只能取消仍处于
@@ -3199,5 +3224,5 @@ owner 的硬门槛。
 - sole-M blocking pipe + timer 必须继续前进；
 - TCP nonblocking poll 路径结构和性能不变；
 - queued-cancel、dispatch-wins、handoff return、shutdown 各竞态重复通过；
-- direct-only 跨 route 依赖在 request-driven replacement 完成前保持显式未完成，
-  不得用常驻 waiter eager compensation 掩盖。
+- direct-only 跨 route 依赖必须由 request-driven replacement 闭环，不得用
+  常驻 waiter eager compensation 掩盖。

internal/build/coro_native_fleet_e2e_test.go

@@ -953,6 +953,125 @@ func Check() int32 {
 }
 `
 
+// coroNativeFleetDeferredDirectChannelReplacementE2ESource isolates the one
+// liveness edge which cannot be inferred before a syscall starts: a source-free
+// one-case channel waiter on the blocked route is completed later by a peer
+// route. The raw llgo.syscall call must stay on the current M, while that
+// durable direct-channel request starts the pre-armed replacement which runs
+// the waiter and releases the C block.
+const coroNativeFleetDeferredDirectChannelReplacementE2ESource = `package main
+
+import _ "unsafe"
+
+var Failed uint32
+var Ready chan uint32
+var Signal chan uint32
+var MainThread uintptr
+var WaiterBefore uintptr
+var WaiterAfter uintptr
+var SenderThread uintptr
+var Got uint32
+
+//llgo:link funcPCABI0 llgo.funcPCABI0
+func funcPCABI0(fn any) uintptr
+
+//llgo:link raw llgo.syscall
+func raw(fn uintptr) (uintptr, uintptr, uintptr)
+
+//llgo:coro contract foreign.v1 scope=declaration progress=may-block affinity=any-thread reentry=none memory=by-value abi=word-call.v1/0
+func libc___llgo_coro_native_fleet_e2e_block_v1_trampoline()
+
+//go:linkname schedulerYield llgo.coroYield
+func schedulerYield()
+
+//llgo:coro noblock
+//go:linkname threadID C.__llgo_coro_native_fleet_e2e_thread_id_v1
+func threadID() uintptr
+
+//llgo:coro noblock
+//go:linkname resetState C.__llgo_coro_native_fleet_e2e_block_reset_v1
+func resetState()
+
+//llgo:coro noblock
+//go:linkname isWaiting C.__llgo_coro_native_fleet_e2e_blocked_v1
+func isWaiting() uintptr
+
+//llgo:coro noblock
+//go:linkname unblock C.__llgo_coro_native_fleet_e2e_release_v1
+func unblock()
+
+func directBlock() {
+	raw(funcPCABI0(libc___llgo_coro_native_fleet_e2e_block_v1_trampoline))
+}
+
+func waiter() {
+	thread := threadID()
+	if thread != MainThread {
+		go waiter()
+		return
+	}
+	WaiterBefore = thread
+	Ready <- 1
+	Got = <-Signal
+	WaiterAfter = threadID()
+	unblock()
+}
+
+func sender() {
+	thread := threadID()
+	if thread == MainThread {
+		go sender()
+		return
+	}
+	SenderThread = thread
+	for isWaiting() == 0 {
+	}
+	Signal <- 0xd1ec7
+}
+
+func Setup() {
+	Failed = 0
+	Ready = make(chan uint32, 1)
+	Signal = make(chan uint32)
+	MainThread = threadID()
+	WaiterBefore = 0
+	WaiterAfter = 0
+	SenderThread = 0
+	Got = 0
+}
+
+func main() {
+	resetState()
+	go waiter()
+	<-Ready
+	// The buffered Ready send lets waiter continue directly into its one-case
+	// receive. Yield is an explicit stable boundary which proves that receive
+	// has parked before this route enters C.
+	schedulerYield()
+	go sender()
+	before := threadID()
+	directBlock()
+	after := threadID()
+	if isWaiting() == 0 {
+		Failed = 151
+		return
+	}
+	if before != MainThread || after != before {
+		Failed = 152
+		return
+	}
+	if WaiterBefore != MainThread || WaiterAfter == 0 ||
+		WaiterAfter == MainThread || SenderThread == 0 ||
+		SenderThread == MainThread || Got != 0xd1ec7 {
+		Failed = 153
+	}
+}
+
+func Check() int32 {
+	return int32(Failed)
+}
+`
+
 const coroNativeFleetLockedOrdinarySuspendE2ESource = `package main
 
 import _ "unsafe"
@@ -1836,6 +1955,16 @@ func TestCoroNativeFleetSameRouteReplacementE2E(t *testing.T) {
 	runCoroNativeFleetE2E(t, coroNativeFleetSameRouteReplacementE2ESource, "same-route-replacement", true, 1)
 }
 
+func TestCoroNativeFleetDeferredDirectChannelReplacementE2E(t *testing.T) {
+	runCoroNativeFleetE2E(
+		t,
+		coroNativeFleetDeferredDirectChannelReplacementE2ESource,
+		"deferred-direct-channel-replacement",
+		true,
+		2,
+	)
+}
+
 func TestCoroNativeFleetLockedOrdinarySuspendE2E(t *testing.T) {
 	runCoroNativeFleetE2E(t, coroNativeFleetLockedOrdinarySuspendE2ESource, "locked-ordinary-suspend", true, 2)
 }
@@ -2006,6 +2135,7 @@ func buildCoroNativeFleetE2ERuntimeIsland(t *testing.T, temp string) []string {
 		filepath.Join("..", "..", "runtime", "internal", "runtime", "coro_panic_trace_release.go"),
 		filepath.Join("..", "..", "runtime", "internal", "runtime", "coro_spawn.go"),
 		filepath.Join("..", "..", "runtime", "internal", "runtime", "coro_native_atomic_llgo.go"),
+		filepath.Join("..", "..", "runtime", "internal", "runtime", "coro_native_deferred_replacement_llgo.go"),
 		filepath.Join("..", "..", "runtime", "internal", "runtime", "coro_native_fleet.go"),
 		filepath.Join("..", "..", "runtime", "internal", "runtime", "coro_native_fleet_owner_llgo.go"),
 		filepath.Join("..", "..", "runtime", "internal", "runtime", "coro_native_fleet_program_llgo.go"),

internal/build/coro_spawn_native_e2e_test.go

@@ -490,6 +490,16 @@ func buildCoroSpawnNativeE2EUserSource(
 			semantics, intrinsic, err := coroIntrinsicCallSiteSemanticsForTest(universe, call)
 			return intrinsic && semantics.ElidesManagedCall(), err
 		},
+		ClassifyElidedCallCertificate: func(_ *ssa.Function, call ssa.CallInstruction) (string, error) {
+			certificate, certified, err := universe.CoroWorkerSyscallCertificate(call)
+			if err != nil || !certified {
+				return "", err
+			}
+			return certificate.ID, nil
+		},
+		ClassifyStaticCodeAddressCallArgument: func(_ *ssa.Function, call ssa.CallInstruction, argument int) (bool, error) {
+			return universe.CoroStaticCodeAddressCallArgument(call, argument)
+		},
 	})
 	if err != nil {
 		t.Fatal(err)

internal/build/coro_stdlib_sync_acceptance_test.go

@@ -256,6 +256,7 @@ func assertCoroStdlibSyncRuntimeSelection(t *testing.T, fixture coroStdlibSyncFi
 	t.Helper()
 	const runtimePackage = "github.com/goplus/llgo/runtime/internal/runtime"
 	required := map[string]bool{
+		"coro_native_deferred_replacement_llgo.go":     false,
 		"coro_execution_quota_native_llgo.go":          false,
 		"coro_executor_driver_timer_llgo.go":           false,
 		"coro_keyed_registry_atomic_llgo.go":           false,

runtime/coro_runnable_distribution_source_test.go

@@ -189,8 +189,9 @@ func TestCoroNativeFleetUsesFixedTopologyLogicalQuotaAndScalarPeerABI(t *testing
 	for _, required := range []string{
 		"coroNativeMDirectoryCapacityV1 uint32 = 10_000",
 		"coroNativeMPageCapacityV1      uint32 = 64",
-		"handoff coro.ExecutionDomainHandoff",
-		"resume  coro.ExecutorResumeHandoff",
+		"handoff  coro.ExecutionDomainHandoff",
+		"deferred coro.DeferredExecutorHandoff",
+		"resume   coro.ExecutorResumeHandoff",
 		"token  uint32",
 		"owners [coroNativeFleetDomainCapacityV1]coroNativeMOwnerV1",
 		"pages  [coroNativeMPageCountV1]unsafe.Pointer",
@@ -292,7 +293,7 @@ func TestCoroNativeFleetUsesFixedTopologyLogicalQuotaAndScalarPeerABI(t *testing
 	for _, required := range []string{
 		"coroNativeFleetActiveDomainForRouteV1(id.Route())",
 		"coroNativeFleetV1State.fleet.PostManualAndRequest(id)",
-		"coroNativeFleetRequestNeedsRingV1(domain, result.Executor)",
+		"coroNativeFleetFinishExecutorRequestV1(domain, result.Executor)",
 	} {
 		if !strings.Contains(keyed, required) {
 			t.Errorf("managed keyed completion lacks owner-joined post marker %q", required)

runtime/coro_target_selection_test.go

@@ -129,6 +129,7 @@ func TestCoroNativeFleetTargetBuildSelection(t *testing.T) {
 	}
 	for _, required := range []string{
 		"coro_target_native_fleet_llgo.go",
+		"coro_native_deferred_replacement_llgo.go",
 		"coro_physical_thread_capacity_native_llgo.go",
 		"coro_native_fleet_owner_llgo.go",
 		"coro_native_fleet_program_llgo.go",
@@ -142,6 +143,7 @@ func TestCoroNativeFleetTargetBuildSelection(t *testing.T) {
 		}
 	}
 	for _, forbidden := range []string{
+		"coro_native_deferred_replacement_default.go",
 		"coro_target_native_llgo.go",
 		"coro_ready_distribution_default.go",
 		"coro_target_executor_retired_default.go",