Problem: audio is a flat gain chain with no spatialization and procedural-only music. Verified on main: packages/shell/src/audio/audioEngine.ts:38 setListenerPose(position) takes position only; no PannerNode in packages/shell/src/audio/; packages/shell/src/audio/musicDirector.ts crossfadeTo fades all other layers to zero; packages/assets/src/manifest.ts AssetSourceKind has no audio kind.
Tasks, one PR each, in order. Audio has no screenshot; each PR includes a headless Web Audio graph test using a mocked AudioContext (see how audioEngine.test.ts mocks it, or add a small mock in packages/shell/src/audio/testAudioContext.ts excluded from the build).
-
Listener orientation and panners. Change setListenerPose to setListenerPose({ position, forward, up }) (keep accepting a bare Vec3 for one release, mapping to forward [0,0,-1]); set AudioListener.positionX/forwardX/upX params. Add spatial?: { panning: "hrtf" | "equalpower"; refDistance?: number; maxDistance?: number; rolloff?: number; coneInner?: number; coneOuter?: number; coneOuterGain?: number } to SoundDef in packages/core/src/audio/audioFalloff.ts; when set, route the source through a PannerNode instead of the scalar falloff gain. The shell's camera rig feeds forward/up each frame (packages/shell/src/audio/audioWire.ts). Test: a sound to the left of a listener facing -Z gets a panner with the expected position. CHANGELOG Migrate (signature) and Added.
-
Buses, sends, ducking. In packages/core/src/audio/audioFalloff.ts extend AudioBusDef with sends?: Record<string, number>, insert?: ("compressor" | "lowpass" | "reverb")[], duckedBy?: { bus: string; amount: number; attackMs: number; releaseMs: number }. New pure packages/core/src/audio/busGraph.ts: resolveBusGains(buses, activity: Record<busId, boolean>, previous, dtMs) -> Record<busId, number> implementing ducking envelopes; tests. Shell realizes sends as extra GainNode connections and inserts as DynamicsCompressorNode / BiquadFilterNode / ConvolverNode, and applies resolveBusGains with setTargetAtTime.
-
Zones and occlusion. New packages/core/src/audio/audioZones.ts: AudioZone { id; bounds: { min: Vec3; max: Vec3 }; reverb?: { irId: string; wet: number }; lowpassHz?: number }, resolveListenerZone(zones, position), occlusionFactor(from, to, raycast: (a, b) => boolean) -> 0 | 1; tests. Add an editor volume kind audioZone in packages/core/src/scene/builtinSceneKinds.ts and a runtime consumer that feeds the shell's zone state; the shell sets the master lowpass and the reverb wet from it. Impulse responses: generate a synthetic exponential-decay IR in code (no asset needed).
-
Sample banks and voices. Add SoundBankDef { id; sounds: Record<string, SoundDef>; preload: "eager" | "lazy" }, SoundDef.variants?: string[], pitchJitter?: number, maxVoices?: number, priority?: number. New pure packages/core/src/audio/voiceAllocator.ts: createVoiceAllocator({ maxTotal }) with request(soundId, priority) -> { ok: boolean; steal?: voiceId }, release(voiceId), snapshot/restore; tests. Shell preloads eager banks and uses the allocator before creating sources.
-
Streamed layered music. Add MusicTrackDef { id; url?: string; streaming?: boolean; layers?: { id; url; gain }[]; stingers?: { id; url; quantizeBeats?: number; bpm?: number }[] } to the audio config; playMusic accepts a track id as well as a theme id; streamed layers use createMediaElementSource on <audio> elements started in sync, layer gains driven by MusicStateConfig tiers, stingers as buffers scheduled on the next beat. Test: layer gains follow the state config. Use a tiny synthesized OGG committed under a test fixture (generate with a script, do not download).
-
Audio as an asset kind. Add "audio" to AssetSourceKind in packages/assets/src/manifest.ts, accept .ogg/.wav/.mp3 in the pull CLI (packages/assets/src/cli/pull.ts) with duration metadata, and index a small CC0 set from freesound.org (record licenses in CREDITS.md). bun run gen.
-
Editor Audio workspace. Flip supported: false in packages/editor/src/shell/WorkspaceRail.tsx; packages/editor/src/AudioPanel.tsx lists banks, zones, and a preview button per sound. Shots of the panel. Closes #1687.
Rules: wait for bun run agent:bootstrap --check; branch claude/<slug> off origin/main; claim comment first; one task per PR with Refs #1687; bun run gen after export changes (JSDoc on every export; create* factories need snapshot/restore); CHANGELOG bullet; check-types and test on core, shell, assets, editor as touched plus check-stateful-ratchet, check-doc-symbols, check-orphan-ratchet; never add Kenney audio; merge origin/main before pushing; squash auto-merge; fix CI on the same branch.
Problem: audio is a flat gain chain with no spatialization and procedural-only music. Verified on main:
packages/shell/src/audio/audioEngine.ts:38setListenerPose(position)takes position only; noPannerNodeinpackages/shell/src/audio/;packages/shell/src/audio/musicDirector.tscrossfadeTofades all other layers to zero;packages/assets/src/manifest.tsAssetSourceKindhas no audio kind.Tasks, one PR each, in order. Audio has no screenshot; each PR includes a headless Web Audio graph test using a mocked
AudioContext(see howaudioEngine.test.tsmocks it, or add a small mock inpackages/shell/src/audio/testAudioContext.tsexcluded from the build).Listener orientation and panners. Change
setListenerPosetosetListenerPose({ position, forward, up })(keep accepting a bareVec3for one release, mapping to forward[0,0,-1]); setAudioListener.positionX/forwardX/upXparams. Addspatial?: { panning: "hrtf" | "equalpower"; refDistance?: number; maxDistance?: number; rolloff?: number; coneInner?: number; coneOuter?: number; coneOuterGain?: number }toSoundDefinpackages/core/src/audio/audioFalloff.ts; when set, route the source through aPannerNodeinstead of the scalar falloff gain. The shell's camera rig feeds forward/up each frame (packages/shell/src/audio/audioWire.ts). Test: a sound to the left of a listener facing -Z gets a panner with the expected position. CHANGELOG Migrate (signature) and Added.Buses, sends, ducking. In
packages/core/src/audio/audioFalloff.tsextendAudioBusDefwithsends?: Record<string, number>,insert?: ("compressor" | "lowpass" | "reverb")[],duckedBy?: { bus: string; amount: number; attackMs: number; releaseMs: number }. New purepackages/core/src/audio/busGraph.ts:resolveBusGains(buses, activity: Record<busId, boolean>, previous, dtMs) -> Record<busId, number>implementing ducking envelopes; tests. Shell realizes sends as extraGainNodeconnections and inserts asDynamicsCompressorNode/BiquadFilterNode/ConvolverNode, and appliesresolveBusGainswithsetTargetAtTime.Zones and occlusion. New
packages/core/src/audio/audioZones.ts:AudioZone { id; bounds: { min: Vec3; max: Vec3 }; reverb?: { irId: string; wet: number }; lowpassHz?: number },resolveListenerZone(zones, position),occlusionFactor(from, to, raycast: (a, b) => boolean) -> 0 | 1; tests. Add an editor volume kindaudioZoneinpackages/core/src/scene/builtinSceneKinds.tsand a runtime consumer that feeds the shell's zone state; the shell sets the master lowpass and the reverb wet from it. Impulse responses: generate a synthetic exponential-decay IR in code (no asset needed).Sample banks and voices. Add
SoundBankDef { id; sounds: Record<string, SoundDef>; preload: "eager" | "lazy" },SoundDef.variants?: string[],pitchJitter?: number,maxVoices?: number,priority?: number. New purepackages/core/src/audio/voiceAllocator.ts:createVoiceAllocator({ maxTotal })withrequest(soundId, priority) -> { ok: boolean; steal?: voiceId },release(voiceId), snapshot/restore; tests. Shell preloads eager banks and uses the allocator before creating sources.Streamed layered music. Add
MusicTrackDef { id; url?: string; streaming?: boolean; layers?: { id; url; gain }[]; stingers?: { id; url; quantizeBeats?: number; bpm?: number }[] }to the audio config;playMusicaccepts a track id as well as a theme id; streamed layers usecreateMediaElementSourceon<audio>elements started in sync, layer gains driven byMusicStateConfigtiers, stingers as buffers scheduled on the next beat. Test: layer gains follow the state config. Use a tiny synthesized OGG committed under a test fixture (generate with a script, do not download).Audio as an asset kind. Add
"audio"toAssetSourceKindinpackages/assets/src/manifest.ts, accept.ogg/.wav/.mp3in the pull CLI (packages/assets/src/cli/pull.ts) with duration metadata, and index a small CC0 set from freesound.org (record licenses inCREDITS.md).bun run gen.Editor Audio workspace. Flip
supported: falseinpackages/editor/src/shell/WorkspaceRail.tsx;packages/editor/src/AudioPanel.tsxlists banks, zones, and a preview button per sound. Shots of the panel.Closes #1687.Rules: wait for
bun run agent:bootstrap --check; branchclaude/<slug>offorigin/main; claim comment first; one task per PR withRefs #1687;bun run genafter export changes (JSDoc on every export;create*factories need snapshot/restore); CHANGELOG bullet;check-typesandteston core, shell, assets, editor as touched pluscheck-stateful-ratchet,check-doc-symbols,check-orphan-ratchet; never add Kenney audio; mergeorigin/mainbefore pushing; squash auto-merge; fix CI on the same branch.