Implement Job Queue System for TTS Model

[huwenjie333]

This PR implemented the job queue system for our TTS model on Modal platform. It now has 2 endpoints:

• The /submit endpoint accepts a requests, and submits a new job to TTS model, and returns the job's ID to the client immediately.
• The /result/{call_id} endpoint allows the client to poll for the job's result using the job ID. If the job has expired, it returns a 404 status code to indicate that the job is not found. If the job hasn't completed yet, it returns a 202 status code to indicate that the job is still being processed, which also has a status flied that describe the details:
  • loading means the GPU container is starting or the model is loading.
  • processing means the model is running inference for the request.

The job queue system is running a separate CPU container that will be kept running, which costs ~$5 per month.
The new URL is: https://sb-modal-ws--spark-tts-salt-job-queue-fastapi-app.modal.run

Please see the usage and example in client.py:

(deploy) wenjiehu@Wenjies-MacBook-Pro spark-tts-salt % python client.py
Submitting task to https://sb-modal-ws--spark-tts-salt-fastapi-app-dev.modal.run...
✅ Job submitted successfully! Call ID: fc-01KJPMH75GBCJV5KAXNVXST2PG
Polling for result...
[0s] Job is currently: loading...
[5s] Job is currently: loading...
[10s] Job is currently: loading...
[15s] Job is currently: loading...
[20s] Job is currently: loading...
[25s] Job is currently: loading...
[30s] Job is currently: loading...
[35s] Job is currently: loading...
[40s] Job is currently: loading...
[45s] Job is currently: loading...
[50s] Job is currently: loading...
[55s] Job is currently: processing...

✅ Successfully generated audio!
Saved to output.wav (Size: 105644 bytes)