feat: add audio slot feature to Composer UI (Closes #14)

[socialawy-dev]

Adds an audio track feature to MotionPlate, enabling users to attach, control, and visualize an audio background clip for their motion sequence.

Summary of Changes

• Schema Extension: Added audio field to the Sequence spec (src/spec/schema.ts).
• Composer UI: Created AudioPanel component (src/composer/AudioPanel.tsx) that includes a file picker, displays the selected file name, allows removing audio, implements a waveform visualization (via Web Audio API), and supports volume, offset, and mute adjustments. Added it to the App layout.
• IndexedDB Storage: Extended StoredProject in src/store/persistence.ts to serialize/deserialize the audio file blob alongside the existing project spec and images.
• Store updates: Updated the Zustand useProjectStore (src/store/project.ts) to manage audio state alongside sequences and history, and properly cleanup object URLs to prevent memory leaks.
• Audio Sync: Updated PreviewCanvas to render a hidden HTML <audio> element that syncs with usePlaybackStore's currentTime and isPlaying state.
• Bug Fix in AudioPanel: Ensure AudioContext is properly closed after decoding audio data to avoid hitting hardware limits.
• Tests: Created comprehensive tests for AudioPanel (tests/composer/AudioPanel.test.tsx), verifying state interactions, volume changes, and file upload functionality. Required installing @testing-library/react and @testing-library/user-event.

---

PR created automatically by Jules for task 14016837153224406116 started by @socialawy

[google-labs-jules]

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.

---

For security, I will only act on instructions from the user who triggered this task.

[github-actions]

⚠️ Groq Review: ESCALATE — API error (HTTP 400)

[github-actions]

🚨 Escalation: This PR needs human review. @socialawy please check.

[gemini-code-assist]

Code Review

This pull request introduces background audio support, including a new AudioPanel component for managing audio tracks, waveform visualization, and playback synchronization in the PreviewCanvas. The implementation also extends the project schema and persistence layer to handle audio files. Key feedback includes ensuring the AudioContext is properly closed in a finally block to prevent resource leaks, removing an unnecessary setTimeout in the AudioPanel state management, and adding bounds checks for the waveform calculation loop. Additionally, the audio synchronization logic should be tightened by reducing the drift threshold and handling potential NaN values for audio duration.

[gemini-code-assist]

The AudioContext is created but only closed if decodeAudioData succeeds. If an error occurs during fetch or decodeAudioData, the context remains open, which can lead to hitting the browser's hardware limit for concurrent audio contexts (often as low as 6 in some browsers). It should be closed in a finally block to ensure resources are always released.

Suggested change

	const loadAudio = async () => {
	try {
	const response = await fetch(audioUrl);
	const arrayBuffer = await response.arrayBuffer();
	// eslint-disable-next-line @typescript-eslint/no-explicit-any
	const audioCtx = new (window.AudioContext || (window as any).webkitAudioContext)();
	const buffer = await audioCtx.decodeAudioData(arrayBuffer);
	if (isMounted) setAudioBuffer(buffer);
	audioCtx.close();
	} catch (err) {
	console.error('[MotionPlate] Failed to decode audio for waveform:', err);
	}
	};
	const loadAudio = async () => {
	// eslint-disable-next-line @typescript-eslint/no-explicit-any
	const audioCtx = new (window.AudioContext || (window as any).webkitAudioContext)();
	try {
	const response = await fetch(audioUrl);
	const arrayBuffer = await response.arrayBuffer();
	const buffer = await audioCtx.decodeAudioData(arrayBuffer);
	if (isMounted) setAudioBuffer(buffer);
	} catch (err) {
	console.error('[MotionPlate] Failed to decode audio for waveform:', err);
	} finally {
	audioCtx.close();
	}
	};

[gemini-code-assist]

The setTimeout wrapper for setAudioBuffer(null) is unnecessary. Calling state updates inside useEffect is standard practice and does not trigger the "cascading renders" warning in this context. Removing the timeout simplifies the code and ensures the state is cleared immediately when the URL is removed.

        if (!audioUrl) {
            setAudioBuffer(null);
            return;
        }

[gemini-code-assist]

The inner loop for waveform calculation does not check if the index is within the bounds of the data array. When (i * step) + j exceeds data.length, data[index] returns undefined, which causes the min/max logic to fail (as comparisons with undefined return false) and results in incorrect drawing at the end of the waveform. Adding a bounds check ensures robustness.

            for (let j = 0; j < step; j++) {
                const idx = (i * step) + j;
                if (idx >= data.length) break;
                const datum = data[idx];
                if (datum < min) min = datum;
                if (datum > max) max = datum;
            }

[gemini-code-assist]

The condition targetTime <= audioEl.duration may fail if the audio metadata hasn't loaded yet, as audioEl.duration will be NaN. It's safer to use a fallback or check for NaN to ensure consistent behavior, similar to how it's handled later in the same effect (line 81).

Suggested change

	if (targetTime >= 0 && targetTime <= audioEl.duration) {
	if (targetTime >= 0 && targetTime <= (audioEl.duration || 0)) {

[gemini-code-assist]

A drift threshold of 0.2s (200ms) is quite high for audio synchronization. At 30fps, this represents a lag of 6 frames, which is clearly noticeable to users. Reducing this to 0.1s or lower would provide a much tighter and more professional sync experience.

Suggested change

	if (Math.abs(audioEl.currentTime - targetTime) > 0.2) {
	if (Math.abs(audioEl.currentTime - targetTime) > 0.1) {

[socialawy]

This PR has merge conflicts after #31 was merged. Needs a rebase to resolve conflicts before merging.

[google-labs-jules]

This PR has merge conflicts after #31 was merged. Needs a rebase to resolve conflicts before merging.

I have successfully rebased the branch on top of origin/main to resolve the merge conflicts and I pushed the changes. Let me know if anything else is needed!