π― Objective & Overview
Enable text-to-sound-effects (SFX), ambient audio, and full musical song generation by creating ComfyUI workflow presets for Stable Audio Open 3.0 and YuE (δΉ).
π Files & Components to Touch
- Workflow Presets (
Workflows/Audio/):
Workflows/Audio/stable_audio_open_sfx.json (Generates up to 47s stereo SFX / music clips)
Workflows/Audio/yue_full_song.json (Dual-track lyrics-to-music generation)
- Endpoints (
Endpoints/WorkflowEndpoints.cs):
- Add
GET /api/audio/workflows listing available audio generation workflow templates.
- Add
POST /api/audio/generate queuing audio workflow execution on ComfyUI.
- Add
GET /api/audio/files listing generated .wav / .flac / .mp3 output files from output_audio/.
- UI Player (
LocalLLMServerManager.Shared/Views/Controls/EngineStudioTabControl.axaml):
- Add audio waveform visualizer or embedded HTML5
<audio controls> player for previewing generated tracks.
π API Contract
POST /api/audio/generate
{
"workflowId": "stable_audio_open_sfx",
"prompt": "Cyberpunk atmospheric ambient drone, heavy synthesizer, cinematic low end, 48kHz stereo",
"negativePrompt": "low quality, harsh distortion",
"durationSeconds": 30,
"seed": -1
}
Response: 200 OK
{
"promptId": "f9e8d7c6-b5a4-...",
"status": "queued",
"wsUrl": "ws://127.0.0.1:8188/ws"
}
π οΈ Step-by-Step Implementation Guide
- Create and validate ComfyUI workflow JSONs using
ComfyUI-Stable-Audio or native audio nodes.
- Implement substitution and queuing in
WorkflowEndpoints.cs.
- Save generated audio outputs into
wwwroot/output_audio/ and serve them statically.
- Add audio player controls to the Studio UI.
π§ͺ Verification
dotnet build
dotnet test --filter Category!=LiveExternal
npm run lint & npx tsc --noEmit
π― Objective & Overview
Enable text-to-sound-effects (SFX), ambient audio, and full musical song generation by creating ComfyUI workflow presets for Stable Audio Open 3.0 and YuE (δΉ).
π Files & Components to Touch
Workflows/Audio/):Workflows/Audio/stable_audio_open_sfx.json(Generates up to 47s stereo SFX / music clips)Workflows/Audio/yue_full_song.json(Dual-track lyrics-to-music generation)Endpoints/WorkflowEndpoints.cs):GET /api/audio/workflowslisting available audio generation workflow templates.POST /api/audio/generatequeuing audio workflow execution on ComfyUI.GET /api/audio/fileslisting generated.wav/.flac/.mp3output files fromoutput_audio/.LocalLLMServerManager.Shared/Views/Controls/EngineStudioTabControl.axaml):<audio controls>player for previewing generated tracks.π API Contract
POST /api/audio/generate{ "workflowId": "stable_audio_open_sfx", "prompt": "Cyberpunk atmospheric ambient drone, heavy synthesizer, cinematic low end, 48kHz stereo", "negativePrompt": "low quality, harsh distortion", "durationSeconds": 30, "seed": -1 }Response:
200 OK{ "promptId": "f9e8d7c6-b5a4-...", "status": "queued", "wsUrl": "ws://127.0.0.1:8188/ws" }π οΈ Step-by-Step Implementation Guide
ComfyUI-Stable-Audioor native audio nodes.WorkflowEndpoints.cs.wwwroot/output_audio/and serve them statically.π§ͺ Verification
dotnet builddotnet test --filter Category!=LiveExternalnpm run lint&npx tsc --noEmit