Skip to content

feat(fluids): one GPU particle pool, stepped by a WebGPU compute pass; WebGL2 refused by name until #759 (#420) - #776

Merged
pasquelin merged 16 commits into
developfrom
420-particle-pool
Sep 26, 2026
Merged

pasquelin merged 16 commits into
developfrom
420-particle-pool

Conversation

@pasquelin

Copy link
Copy Markdown
Owner

Closes #420

What changed

Scope after the CTO's split:

Changes:

  • packages/sdk-core/src/fluids/particles.ts: ParticlePool.
    • Capacity is fixed at creation. Emission is a ring, so past capacity it overwrites the oldest slots.
    • The frame's records go into one staging Float32Array made at creation, eight words each: position then age, velocity then lifetime.
    • flush() hands each image its step { first, count, dt } as the same object every time.
    • No CPU compaction, no allocation per frame, no GPU→CPU read.
    • dt is clamped to 1/15 s. An idle pool (nothing staged, nothing alive) takes no time, so it neither dispatches nor breaks the hold.
  • Emitter-relative positions (CTO's decision): a pool has an origin, the emitter's place in the world, fixed for its life. emit() takes world positions and subtracts the origin in double precision before rounding to 32 bits. A particle 10 km out still moves by 0.4 mm per step.
  • The origin is not in the WebGPU step's uniform. The step only integrates velocity and position, so the result is the same wherever the origin is. Drawing (Particles draw without a global sort: additive fire, sorted-per-emitter smoke, soft edges #755) adds the origin back, camera-relative.
  • packages/sdk-browser/src/particles/webgpuParticles.ts:
    • PARTICLES_WGSL runs one compute pass labelled Trillion3D particles. passesGpu shows it, and the stage profile files it under physics.
    • One dispatch per moving pool over one storage buffer, covering only the slots emitted into so far. A newborn slot takes its record; a live particle integrates; a dead one is skipped by age.
    • A pipeline that fails to compile is reported as particles-unavailable, and its pools are refused: they stop asking frames and count what is emitted into them as dropped.
    • Compiled asynchronously through createCheckedShaderModule. Bindings use the shared bounceLayout and bounceGroup helpers.
    • A pool the world lets go of gives its buffers back; removing it asks the image that does so.
    • Called from encodeBlend, the stage every WebGPU image path reaches, beside the water pass. particlesMoved breaks the hold, next to guidesMoved and effectsMoved.
  • How the pools reach the renderer: the world holds them in worldSwitches.held.particles. They flow through the session options into BackendContext.particles and the WebGL2 composer, the same path guides and effects take.
  • No public API (Fluids P2: GPU particles, fire, smoke flipbooks, heat distortion, wind API #423): the measurement entry exports attachParticles(world, pool) and ParticlePool. attachParticles advances the pool by the world's frame time, and keeps asking for frames while the pool moves, checked after every frame hook has emitted. A pool attached twice, or to a world never made, is refused as PARTICLES_ATTACH.
  • On WebGL2, a world with a pool is refused by name. The frame composer throws PARTICLES_UNSUPPORTED rather than drawing without the particles, until Particles update on WebGL2 and keep their precision anywhere in the world #759 lands (AGENTS.md rule 1, Streaming without holes: rules and objectives for geometry, memory and shadows #483). It marks the pools refused first, so they ask no frame of their own.
  • A pool counts one clamped step past its longest lifetime, so the GPU's 32-bit age always reaches the lifetime before the CPU calls the pool idle.
  • docs/ENGINE.md (physics timing): the physics stage's GPU column is the particle step.

Proof

  • node --test packages/sdk-core/src/fluids/particles.test.ts: 6 pass. They cover:
    • the record layout;
    • the ring wrapping onto the oldest slots, with the same staging and step objects every image;
    • a full staging refused and counted, and the clamp;
    • refusals by name;
    • the 10 km particle keeping its 0.4 mm step, which world floats round away;
    • an idle pool taking no time.
  • node --test packages/sdk-browser/src/particles/particles.test.ts: 3 pass.
    • The WebGPU words: the step uniform and the staged records, one timed pass over the emitted slots, buffers made once, a released pool's buffers destroyed.
    • A step that cannot compile is heard once, and its pool stops moving and refuses emission.
    • A still frame is held until one of the world's pools moves.
  • node --test packages/sdk-browser/src/world/core/worldSession.test.ts: attachParticles holds the pool, refuses a second attach and a world never made, asks for frames only while it moves, hands the world's frame time to the step, and is undone by the remover, which asks one image.
  • node --test packages/sdk-browser/src/world/render/compose.test.ts: a world with a pool is refused by name on WebGL2, never drawn without it, and its pool stops moving.
  • engineShaders.test.ts: PARTICLES_WGSL is on the engine's shader list and declares every name it uses.
  • Fast gates, all exit 0:
    • pnpm run build:native, after git submodule update --init;
    • pnpm run validate --group typescript;
    • pnpm run check:changed (1384 pass);
    • pnpm run test:changed;
    • pnpm run validate --group quick.
  • node scripts/check-pr-size.ts: 597 hand-written lines added (limit 600).
  • The measurer, after merge:
    • GPU proof of this pull request: node --experimental-strip-types tests/browser/probes/particles-step-gpu.ts, a new probe that pnpm run test:gpu also runs. It compiles PARTICLES_WGSL on a real WebGPU device and runs one step of a pool whose origin is 10 km out. The newborn particle must drift 0.4 mm and rise, and age by the step. The particle born dead must stay as staged, and the never-emitted slot untouched. Only the probe reads the state back.
    • Non-regression of the changes to encodeBlend, the hold check and the composer: pnpm run test:gpu.
    • No page steps a world's pool yet: cost and capacity per tier come with the bench switch on 420-particle-bench, which lands with Particles draw without a global sort: additive fire, sorted-per-emitter smoke, soft edges #755. The command then is node bench/runner/bench.ts --scene fluids --moteur webgpu --particles 16k|64k|256k [--particles-update-only].

Local review before push

  • Simplification pass: 2 review agents (reuse and simplification; efficiency and altitude). reviewer: 2 review agents (reuse and simplification; efficiency and altitude); fixed: a failed pipeline refuses its pools, which stop asking frames; one release for a let-go pool and dispose, which also frees a released pool while an idle one stays; dispatches only over the emitted slots; the pool's dead moving ? branch in flush; gravity from GRAVITY_PRESETS.earth; a one-caller test helper inlined and a duplicated layout assertion dropped; particlesMoved in one line; comments trimmed to the line budget; skipped: the probe's bindings through namedBufferEntries (the probe cannot be run here), the glue moved to webgpu/pages/render/ and the registry folded into Access (lines, world.ts at its 200-line cap), the WebGL2 refusal moved into attachParticles (only the composer knows the engine does not step), the frame's step settled on submit and the WGSL load reordered (lines).
    • Fixed:
      • bindings through bounceLayout and bounceGroup;
      • the one-implementation ParticleBackend interface removed (it returns with Particles update on WebGL2 and keep their precision anywhere in the world #759);
      • an idle pool neither dispatches nor breaks the hold (moving, particlesMoved beside guidesMoved and effectsMoved), and attachParticles asks for frames only while it moves;
      • setPipeline once per pass;
      • a released pool's buffers destroyed;
      • the remover guarded against a second call;
      • the step's uniform words shared with the GPU probe (createStepWords).
    • Skipped:
      • moving the step to where the frame's command buffer opens: an abandoned command buffer loses the records either way;
      • particles as a field of the registry's access record: world.ts is at its 200-line cap;
      • a second WebGL2 refusal in attachParticles: the composer's refusal also covers a session reopened on WebGL2.
  • Correctness review: 9 findings. reviewer: 8 findings, 5 fixed: a pool counts one clamped step past its lifetime, so a particle whose 32-bit GPU age lags is never left frozen alive; the frame request is checked in onFrame, after every hook's emission; WebGL2's refusal marks the pools refused, so it does not repeat at display rate; a pool attached twice or to a world never made is refused (PARTICLES_ATTACH); the remover asks the image that frees the buffers. Skipped: one pool attached to two worlds (3-4 lines past the budget, an internal entry until Fluids P2: GPU particles, fire, smoke flipbooks, heat distortion, wind API #423); an emission from outside a frame on an idle world wakes nothing (Fluids P2: GPU particles, fire, smoke flipbooks, heat distortion, wind API #423); the WebGL timer query left open by the composer's throw (same as the existing HOST_DRAW_UNSUPPORTED, belongs to draw.ts); a harmless extra release on a pool's first frame. Auditor list: Closes #420 on the scope the CTO's split comments leave; the lead's design note followed (WebGPU step beside the water, own timed pass, no allocation per frame, no GPU→CPU read in the engine, emitter-relative positions, WebGL2 refused by name); every changed behaviour has a fast test that fails on develop; ENGINE.md follows; no path without a pool changes (an empty pool list returns before any work, the hold check sees no pool moving).
    • Fixed:
      • an idle pool no longer banks time that later flings newborn particles;
      • a removed last pool's buffers are released;
      • Trillion3D particles has its row in the stage table (physics) instead of swelling geometry.
    • Skipped, reported to the lead:
      • a pool never stepped stays moving, so the world redraws without end. This happens if the pipeline fails to compile (fixed by the reviewer: the pools are refused).
      • an image whose command buffer is abandoned after the step loses that step's records.
      • two WebGPU engines in one comparison session share the pools, so the second steps nothing.
      • attaching the same pool twice, or to a world never registered, is not refused (fixed by the reviewer: PARTICLES_ATTACH).
      • on a lost WebGPU device, the composer's refusal names particles before the host-draw error.

Lead verification

Read by the lead on head 28598dd (054a73f reviewed OK, then a clean merge of develop) against #420 as split by the CTO (drawing → #755, TAA and flipbooks → #756, WebGL2 backend → #759), its design note and scope comments, and #417's rules 1–3, 8 and 10.

  • Search first (reuse any engine sprite or particle path): delivered, no particle path exists (sprites are page-geometry clusters); the step reuses createCheckedShaderModule, the bounce bind-group helpers and the water's place in encodeBlend; the open world's code is not read into it.
  • Fixed capacity per pool, buffers allocated once, ring emission with no CPU compaction: delivered in packages/sdk-core/src/fluids/particles.ts:34 (ParticlePool: ring cursor, fixed staging, zero allocation after creation, emitter-relative positions per the CTO), proved by the ring wraps past capacity onto the oldest slots, and allocates nothing after creation, a full staging refuses and counts; the step clamps its time and consumes it and a particle ten kilometres out keeps its sub-millimetre steps: positions are from the origin (sdk-core/src/fluids/particles.test.ts).
  • WebGPU compute step (WGSL): delivered in packages/sdk-browser/src/particles/webgpuParticles.ts:64 (one timed pass PARTICLES_PASS, dispatch over the emitted slots only, idle pool = no dispatch), run from webgpu/pages/render/encodeBlend.ts:36, proved by WebGPU: one timed pass writes the step words and the staged records, once and WebGPU: a still frame is held until one of the world's pools moves (sdk-browser/src/particles/particles.test.ts), and on a real device by the probe tests/browser/probes/particles-step-gpu.ts (measurer, after the merge per the CTO's rule).
  • The same emission API on both renderers: delivered as far as this issue goes — WebGL2 refuses a world with a pool by name (PARTICLES_UNSUPPORTED, world/render/compose.ts:39) until Particles update on WebGL2 and keep their precision anywhere in the world #759, proved by a world with a particle pool is refused by name on WebGL2, never drawn without it (compose.test.ts:123).
  • World wiring: delivered in packages/sdk-browser/src/world/core/worldSession.ts:28 (attachParticles, refused twice or without a world: PARTICLES_ATTACH), proved by attachParticles gives the pool to every session and the frames the world draws (#420) (worldSession.test.ts:19).
  • A shader that cannot compile is heard and stops asking frames (Fluids: water, fire and smoke beside Jolt #417 rule 8, nothing silent): delivered in webgpuParticles.ts:62, proved by WebGPU: a step that cannot compile is heard, and its pools stop asking frames.
  • Measure capacity and cost per tier: moved to Particles draw without a global sort: additive fire, sorted-per-emitter smoke, soft edges #755 with the bench switch (CTO); the step's pass is timed and shown in the stage table under physics (docs/ENGINE.md).
  • Whole promise: delivered, every To-do left to Fluids S2 (spike): one GPU particle system on WebGPU and WebGL2 #420 after the CTO's splits.
  • Tests that bite: delivered, all new tests fail on develop (modules absent or behaviour new); no fixed delay.
  • No image loss: delivered, nothing is drawn; a world without a pool takes no new work and its frame is held as before (hold.ts:158).
  • Reuse: delivered, ParticlePool, createWebgpuParticles, attachParticles searched in sdk-core and sdk-browser: no twin.
  • Docs follow the code: delivered, docs/ENGINE.md (the stage's GPU column); no public API (that is Fluids P2: GPU particles, fire, smoke flipbooks, heat distortion, wind API #423), attachParticles is a measurement entry.
  • Measured first: delivered, a spike: its numbers are the measurer's after the merge (probe, then Particles draw without a global sort: additive fire, sorted-per-emitter smoke, soft edges #755's bench).
  • Path: delivered, the CTO's splits and precision decision are on Fluids S2 (spike): one GPU particle system on WebGPU and WebGL2 #420/Particles update on WebGL2 and keep their precision anywhere in the world #759; in review removed and to measure set at hand-over.
  • Streaming without holes: rules and objectives for geometry, memory and shadows #483 and CONTRIBUTING.md §Streaming, memory and shadows: delivered, fixed budgets (capacity at creation), no GPU→CPU read in the engine, nothing rebuilt per frame.
  • rounds: 2 (size gate, then restructure to the split scope)

Not proven / left out

…s the hold; bindings reuse the compute helpers (#420)
… is attached once, dispatches its emitted slots only (#420)
…fter every emission, and WebGL2 refuses it once (#420)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

audited merged pull request re-read by the auditor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant