feat(shm): fork-shared arena, IPC primitives, result slots, mutable shared mode (closes #16, closes #18, closes #17)

[lisachenko]

Implements E1, E2 and E3 of EPIC #15: the fork-shared memory foundation, the opt-in mutable shared mode on top of it, and the IPC primitives that live inside it.

Closes #16. Closes #18. Closes #17. Part of #15. Consumes the allocator seam from lisachenko/z-engine#223.

---

E1 — the arena

Shm\Arena — one mmap(NULL, size, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) region created before any fork, so every worker inherits it at the same virtual address and an address handed to a sibling means the same thing there.

  0        header words   magic, layout version, size, bump cursor, creator pid,
                          measured mutex size, roots capacity
  128      mutex bank     64 slots x 64 bytes, PTHREAD_PROCESS_SHARED + PTHREAD_MUTEX_ROBUST
                          slot 0 = allocator, slot 1 = roots directory, 2.. = consumer stripes
  4224     roots dir      64 entries x 64 bytes: hash, address, name length, name[40]
  16384    payload        bump-allocated, cursor grows upwards, never shrinks

• fixed size from SHARED_DATA_ARENA_SIZE (64 MB default), typed ArenaException on exhaustion, leak-until-teardown: the region lives until the creating process exits and nothing unmaps it at request shutdown (see the teardown fix below); a child's destroy() is a deliberate no-op;
• allocate() moves the shared cursor under the allocator mutex, so any child can allocate concurrently;
• sizeof(pthread_mutex_t) is measured at runtime (paint a buffer, initialize a mutex into it, look at what moved: 40 on x86-64 glibc) and verified against the 64-byte slot stride, instead of being assumed;
• EOWNERDEAD is handled at every lock site — pthread_mutex_consistent() then proceed; no lock result is ever discarded;
• no public method returns FFI\CData. Callers see integers and strings; the pointer views are bound once per process at map time, which is also what keeps critical sections to plain word loads and stores.

Shm\ArenaAllocator implements z-engine's ZEngine\Memory\Allocator over the arena (zeroed blocks — a bump allocator never recycles; 16-aligned; ownsAllocations() === true, so z-engine never frees an individual block), plus createTable() which pairs a table struct with a pre-sized external arData block through the seam's install API.

Registry-in-arena — Registry::createInArena() / fromArena() put the root, entries, objects and every per-entry/per-object record table into the arena, published in the arena roots directory so a forked child finds them with nothing but the mapping. Bucket keys are arena-interned too: a malloc-backed key is a pointer no sibling can follow. Module globals shrink to [arena base, LAYOUT_VERSION], and children only ever read them (the page is COW — a child's write would silently desynchronize the family).

Growth is refused, never attempted. z-engine guards the tables it minted, but a registry recovered in a child rebuilds borrowed views that know nothing about their storage, so the guard is re-derived from nNumUsed/nTableSize, and recovery bounds-checks HT_GET_DATA_ADDR(ht) = arData - HT_HASH_SIZE(nTableMask) (mask read signed) against the arena. This is not belt-and-braces: a resize pereallocs shared buckets into one worker's private heap and writes that pointer into the shared struct before failing, so siblings read plausible garbage with no signal at all.

Arena-backed persist path — PersistentStore::bootShared($arena) is the opt-in entry point, anchored in its own persistent module so globals[0] always means one thing per module. The Persister threads the allocator through every minting call (clones, snapshots, strings, sealed arrays and their keys): one malloc-backed block inside a shared graph is a pointer a sibling cannot follow, so there is no half-way. addressOf() / attachObject() are the eight-byte exchange protocol between workers.

Registry::LAYOUT_VERSION 3 → 5 (4 = arena tables, 5 = the per-object role E2 adds), hard-fail preserved, version history in the class docblock.

Acceptance criteria (#16)

Criterion	Where
Parent persists pre-fork, two children read identical values by address	ArenaStoreForkTest::testTwoChildrenReadTheGraphPersistedBeforeTheFork
Child persists post-fork, parent + sibling attach via 8-byte address over a pipe	ArenaStoreForkTest::testGraphPersistedByAChildIsAttachedByTheParentAndASiblingThroughItsAddress
Registry tables never grow via the engine	ArenaRegistryTest (refusal, table named, data block verified unmoved, upsert still allowed)
Watermark exposed, exhaustion throws a typed exception	ArenaTest, ArenaStoreForkTest

---

E2 — opt-in mutable shared mode

Sharing the memory was E1. E2 is about the two things that are still per-process inside a shared object: its engine state, and its lifecycle.

Mutation contract

persist($key, $object, mutable: true) (arena mode only) keeps everything that makes a persistent clone safe — PIN_BASELINE refcount pin, GC_PERSISTENT|GC_NOT_COLLECTABLE, bare non-refcounted slot payloads, sealed immutable arrays — and gives up frozen semantics: detach() never memcpys a request-old snapshot over slots other workers are writing. The role is recorded in the registry, not in the persisting process, so every worker that attaches the address later applies the same lifecycle; one object belonging to a frozen and a mutable graph is refused (SharedMutationException::modeConflict), because the two lifecycles contradict each other.

PersistentStore::mutableHandle($object) returns the synchronized write path, guarded by Arena::stripeFor($address):

Slot	Rule
scalar	in-place overwrite, payload word then type word, under the stripe lock (writeScalar, writeScalars for several slots in one critical section)
string	new bytes interned into the arena before the lock, then an 8-byte pointer swap; the old block leaks until teardown, because a reader may be following it right now
object ref	pointer swap to another object of this arena only — anything else gets a typed exception naming persist()
plain array	forbidden, unchanged: a shared zend_array cannot grow, so it stays sealed (Ipc\SharedArray is the mutable collection)

Reads (readScalar(s), readString, readReference, read) take the same lock, because the type of a slot can change under them — correction #1. A single aligned 8-byte pointer read may skip it, and the fork test exercises exactly that: an unlocked reader following a string pointer swapped 100 times sees a complete string every time, never a mixture (correction #2). Declared property types are enforced by the write path, since the engine never sees the assignment.

No write_property hooks, deliberately: a persistent clone is rewired to std_object_handlers by construction (the only handlers block whose address survives a fork), so there is nothing to hook without giving that up. A direct $obj->prop = ... therefore still works and is unsynchronized — visible everywhere and racy for scalars; for a string/array/object it stores a request-heap pointer inside shared memory, and such a slot is restored from the persisted image at detach() instead of being left for a sibling to dereference (repairedSlotCount() reports it). Both halves are tested.

Per-process side table

SideTable, keyed by arena address, holds the three fields that describe the reader rather than the object:

• handle — attach registers through z-engine's ObjectEntry::register() and keeps the returned handle here; the shared field is then overwritten with SHARED_HANDLE_SENTINEL so no process can trust it (children inherit one object-store free list and are handed identical numbers). Detach puts the side-table handle back for the length of ObjectEntry::unregister(), which verifies the slot still holds this object — a stale-handle refusal is tolerated, that is the guard working. That restore is the one moment the shared field is not the sentinel, it is documented, and nothing in the package reads that field;
• ce — rebound per process at attach and recorded here; the shared field is advisory (pre-fork class loading is still required);
• properties — forced NULL at attach and never dereferenced in shared mode. The trigger list is documented on scrubProperties(): get_object_vars(), var_dump(), json_encode(), (array), serialize(), debug_zval_dump(), ReflectionObject. inspect($object, $reader) brackets such a call so the cache dies in the process that caused it.

Identity: spl_object_id() reads the sentinel in every process and is documented as meaningless; PersistentStore::sharedIdOf($obj) returns the arena address, which is the identity every process agrees on.

Role-aware lifecycle

• detach() never rolls back a mutable graph (frozen behaviour is byte-identical, and the frozen path still releases the dynamic-property table exactly as before; the shared path drops the pointer unread);
• drop()'s refcount === PIN_BASELINE alias predicate is disabled for shared graphs: a refcount in the arena is written by every worker that ever copied the value, so it saturates above the baseline and can neither prove nor disprove a live alias — and it protects nothing, since an arena block is never handed back;
• children never free: the refusal moved to the last line before the free (Reclaimer::destroyTable() / freeBlock()), armed by bootShared() for the arena it maps, so every free path reachable in a child refuses with a typed ArenaException naming the role of the process.

One crash fixed on the way (teardown ordering)

The arena used to be unmapped by a shutdown function armed in Arena::create(), and PHP destroys the symbol table, the object store and every remaining zval after its shutdown functions have run — so a variable still holding a shared object was released against unmapped memory. The suite printed OK (…) and the process then died with SIGSEGV (exit 139). No ordering inside the class can fix it (the mapping is necessarily created first), so the automatic unmap is gone: the kernel reclaims the region at process exit, which is the lifetime leak-until-teardown already assumes. destroy() stays for a caller who owns the moment. Regression test: a worker that keeps a shared object alive and simply exits must exit with status 0, not a signal.

Acceptance criteria (#17)

Criterion	Where
Two children mutate/read scalar + string + shared-ref properties under stripe locks, visibility over 10⁵ rounds	MutableSharedForkTest::testTwoChildrenWriteAndReadTheSameObjectUnderItsStripeLock, …AChildWritesAStringAndAReferenceThatEveryOtherProcessCanFollow
Concurrent attach: per-process handles, spl_object_id sanity, sentinel in the shared struct	MutableSharedForkTest::testConcurrentAttachKeepsEveryHandlePerProcessAndTheSharedFieldASentinel
No cross-heap dereference after a sibling's var_dump()/get_object_vars() (S14)	MutableSharedForkTest::testAChildInspectingASharedObjectLeavesNoHeapPointerForItsSibling
Frozen mode byte-identical: rollback + drop predicate intact, whole existing suite green	MutableSharedForkTest::testSharedMutableStateIsNeverRolledBackWhileAFrozenGraphStillIs, SharedMutationTest::testTheFrozenStoreStillRefusesToDropAGraphTheRequestCanReach
Shared-mode detach never rolls back; child free paths blocked	…testSharedMutableStateIsNeverRolledBack…, MutableSharedForkTest::testAChildIsRefusedWhenItTriesToFreeArenaMemory, SharedMutationTest::testArenaBlocksAreRefusedByEveryFreePathOfThisProcess
Unsynchronized direct write: documented behaviour	MutableSharedForkTest::testADirectWriteIsVisibleToSiblingsButItsHeapStringIsRepairedAtDetach, SharedMutationTest::testMutatingASealedArrayThroughPhpIsContainedRatherThanShared
Refusals (sealed array, foreign reference, frozen graph, unknown property, type mismatch)	SharedMutationTest

---

E3 — IPC primitives as shared objects

Everything below is a structure in the arena, found by address or by a name in the roots directory; nothing is inherited as PHP state except the notification descriptors, which cannot be anything else.

Value records — the whole type system of the shared area

uint8 tag | 7 pad | uint64 payload, sixteen bytes:

Tag	Payload
NIL / TRUE / FALSE	none
INT / FLOAT	the value inline (IEEE-754 bit pattern for floats)
STR	address of an arena-interned, immutable zend_string (structural memcpy at send; the receiver overlays a non-refcounted zval, zero copy)
OBJ	address of a shared zend_object the registry knows — zero copy, same address in every process
ARR	address of a SharedArray
CLOSE	channel control

A value with no address-shaped form is refused, never encoded: plain arrays (→ SharedArray), resources, non-shared objects (→ PersistentStore::persist()) and closures each get a NotShareableValueException naming the remedy. Closures are rejected on provenance, not shape — a stale post-fork address was observed holding a valid Closure of a different function, so inspection can never establish safety; the message points at Task objects / #20.

The primitives

Primitive	Notes
SharedChannel	ring of records + sender/receiver waiter tables under its own robust mutex (a channel locks on every op; the 62-slot bank stays for structures that share). Monotonic head/tail counters, so the fill level is a subtraction and a rendezvous handshake is one word comparison. Capacity 0 = true cross-process rendezvous. close() crosses processes: receivers drain, then [null, false]; senders get ClosedChannelException
SharedArray	fixed-capacity vector of records, ArrayAccess/Countable/IteratorAggregate, per-instance stripe by address hash. Growth is impossible by construction — that is the point
ResultSlotTable	futures: allocateSlot() / complete() / completePanic() / readSlot() / await(). Settles exactly once. The same table carries spawn arguments downwards
SharedError	what a panic actually sends: a persisted three-string object (class, message, trace) in the arena, referenced by address — a Throwable itself can never be shared
SharedMutex	robust pshared mutex, trylock-with-backoff so a caller never blocks the whole process; EOWNERDEAD recovered and reported through wasRecovered()
AtomicInt	plain aligned get()/set() (an 8-byte load never tears), stripe-locked add()/compareAndSet() — FFI has no CAS
SharedWaitGroup	counter + waiter table + pokes at zero; a negative counter throws instead of clamping
WakeRegistry	one inherited socket pair per process, dead owners' slots recycled

Notification plane: signalling, never payload

A sender that makes a ring non-empty or settles a slot writes ONE fixed record to each parked process:

  0  uint8  opcode    WAKE | RESULT | PANIC | CLOSE
  1  uint8  tag       ValueTag of what became available
  2  uint16 padding
  4  uint32 id        channel / slot id
  8  uint64 address   arena address — and ONLY when the tag is address-shaped (0 otherwise)

Descriptors are per-process, so the pairs are minted before the fork and inherited; the arena side is only a claim table (pid → slot). Waking is level-triggered: a waiter registers in the structure's waiter table and re-checks the state inside the same critical section, so a wakeup can be spurious but never lost, and every blocking loop re-polls on a bounded slice — a dropped write (full buffer) costs latency, never correctness. Blocking helpers are a spin loop over the descriptor; every primitive also exposes its non-blocking half plus notificationStream(), so a coroutine runtime parks a Fiber in its own loop instead. Parking a Fiber is deliberately not in this package.

Locking discipline throughout: values are encoded before the lock and decoded after it, waiters are notified once it is released, and inside every critical section there is nothing but aligned word access — a 16-byte record store is not atomic (correction #1), an 8-byte one is (correction #2).

Acceptance criteria (#18)

Criterion	Where
Producer child + consumer child, FIFO, blocking recv parks and wakes, close drains then [null, false]	SharedChannelForkTest::testProducerAndConsumerChildrenExchangeRecordsInFifoOrder, …BlockingReceiveWakes…, …CloseCrossesProcesses…
Capacity-0 rendezvous across processes (sender blocks until taken)	SharedChannelForkTest::testCapacityZeroChannelMakesTheSenderWaitForItsReceiver, …TrySendOnlySucceedsWhileAReceiverIsParked
Result slot round trip per tag incl. OBJ zero-copy (same address observed), waiter woken by event record, PANIC carries a shared error object	ResultSlotForkTest::testAChildCompletesEveryTagKind…, …ParentParksOnTheSocketAndWakesOnTheEventRecord, …PanicTravelsAsASharedErrorObject…
NotShareableValueException on arrays/closures/resources/non-shared objects, message names the remedy	ValueCodecTest (5 cases)
A guard proves serialize() is never called on any data path	NotificationPlaneForkTest::testAFullProducerConsumerRoundTripCallsNoEncodingFunctionAtAll
Sockets carry only event records	NotificationPlaneForkTest::testEveryByteThatCrossesASocketIsAFixedEventRecord
Soak: watermark plateaus, zero leftover children	SharedChannelForkTest::testTheArenaWatermarkPlateausWhileRecordsChurnThroughARing, soaks below

Two claims are tested rather than asserted. The registry's single socket-write point is wrapped and every byte that crossed is parsed back as a 16-byte event record, with the payload string searched for and absent. And the Never-Serialize Rule gets a guard with teeth: namespace-local shadows of serialize, unserialize, igbinary_*, json_* and var_export in the three namespaces the data path runs through, proven to intercept a real call before the round trip is measured at zero.

Also covered by real processes: 4 children summing into one AtomicInt (1000/1000), a compareAndSet token exactly one child wins, a worker SIGKILLed inside a SharedMutex critical section with the lock recovered afterwards, a wait group whose units finish in four other processes, and a SharedArray filled by four workers.

One more crash fixed on the way

phpinfo() walks every registered module, and an arena-backed module's globals[0] is the arena base, not a registry pointer — reading it as a hashtable was a SIGSEGV, not an exception. State is now reported through the module's live store, with the arena magic as the discriminator. Regression test: Shm\ArenaModuleInfoTest.

---

Documentation

The vendored validation sweep (spikes/c1/, review feedback) is removed: it was a throwaway harness from a run outside this repository, with paths bound to the machine it ran on. Its knowledge is distilled into docs/shared-memory-model.md — what works, why the solution is shaped this way, and how it is implemented per primitive: fork-only sharing, the atomicity/tearing contract, the three per-process fields and the side table, the arData law, robust-mutex rules, leak-until-teardown accounting, closure provenance, and the identity story. Raw verdicts stay on the spike-gate record on #15; the claims the code depends on are promoted to tests. The two surviving spikes are repo-native and bootstrap through Composer only.

Test evidence

PHP 8.4.19 (z-engine 8.4 line) and PHP 8.5.9 (master line), both -d ffi.enable=1 -d opcache.jit=off:

vendor/bin/phpunit                    OK (151 tests, 14358 assertions)   # 8.4 and 8.5, exit code 0
vendor/bin/phpunit --order-by=random  OK (151 tests, 14358 assertions)   # 8.4 (x10) and 8.5
MALLOC_CHECK_=3 MALLOC_PERTURB_=85    OK (151 tests, 14358 assertions)   # 8.4 and 8.5
tools/soak.php 5000                   SOAK OK       final delta -2488 bytes (allowed 65536)
tools/soak-drop.php 5000              SOAK-DROP OK  request +31616 bytes, 3.97 kB/cycle (budget 6)

Exit codes are checked explicitly now, not inferred from the summary line — that is how the teardown segfault above hid behind a green report.

Known limits of this iteration

• classes must be loaded before the fork — a shared clone carries one ce slot for the whole family;
• spl_object_id() is meaningless on a shared object (it reads the sentinel); identity is sharedIdOf(), the arena address;
• a direct $obj->prop = ... is unsynchronized: fine for scalars, and for anything else the slot is repaired from the persisted image at detach rather than shared. The synchronized path is mutableHandle();
• plain-array properties stay immutable; mutable collections are Ipc\SharedArray;
• rewriting a string property N times costs N arena blocks (leak-until-teardown), as does sending N distinct strings through a channel;
• the wake registry must be created before any fork, and one SharedError entry per store — a second panic replaces the first;
• blocking is a spin loop with a bounded slice; Fiber parking belongs to the consumer package;
• the shared handle field is not the sentinel for a few instructions while a process detaches (recycling a store slot is an engine call and engine calls cannot run under an arena mutex) — harmless, because nothing here reads that field.

Dependency state (the pin is gone)

lisachenko/z-engine is 8.4.x-dev || 8.5.x-dev, one dev line per supported minor, resolved through the vcs repository entry (no path repository). The 8.5 runs reported above were done against the master line resolved by Composer under PHP 8.5.

Not done here, deliberately

• PHPStan / php-cs-fixer are not configured in this repository (no phpstan/cs:check scripts) and could not be installed in this environment; the new code follows the existing PER-CS2.0 style by hand, with explicit int casts on FFI field reads.
• A follow-up for the seam owner: a borrowed PersistentHashTable view cannot re-adopt the external storage block it is sitting on, so the growth guard had to be re-derived in this package (TODO in Registry::assertRegistryRoom()).

🤖 Generated with Claude Code

https://claude.ai/code/session_01B9xaBchjdo1atarNZ6sZqe

---

Generated by Claude Code

[lisachenko]

If you need some knowledge extracted - collect and report this either as a doc here with limitations/problems or report ticket if needed to fix something somewhere