Skip to content

Commit ef60f9d

Browse files
techpro-aimlapigitbook-bot
authored andcommitted
GITBOOK-734: docs: add lip-sync examples for pixverse v5 models
1 parent 9e10f7e commit ef60f9d

4 files changed

Lines changed: 730 additions & 6 deletions

File tree

docs/api-references/video-models/pixverse/lip-sync.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ After sending a request for video generation, this task is added to the queue. T
5151
If the video generation task status is `completed`, the response will include the final result — with the generated video URL and additional metadata.
5252

5353
{% openapi-operation spec="universal-video-endpoint-fetch" path="/v2/video/generations" method="get" %}
54-
[OpenAPI universal-video-endpoint-fetch](https://raw.githubusercontent.com/aimlapi/api-docs/refs/heads/main/docs/api-references/video-models/ByteDance/omnihuman-pair.json)
54+
[OpenAPI universal-video-endpoint-fetch](https://raw.githubusercontent.com/aimlapi/api-docs/refs/heads/main/docs/api-references/video-models/universal-video-fetch.json)
5555
{% endopenapi-operation %}
5656

5757
## Full Example: Generating and Retrieving the Video From the Server
@@ -128,7 +128,7 @@ def main():
128128

129129
status = response_data.get("status")
130130

131-
if status in ["waiting", "queued", "generating"]:
131+
if status in ["queued", "generating"]:
132132
print(f"Status: {status}. Checking again in 15 seconds.")
133133
time.sleep(15)
134134
else:

docs/api-references/video-models/pixverse/v5-image-to-video.md

Lines changed: 242 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ After sending a request for video generation, this task is added to the queue. T
5656
If the video generation task status is `complete`, the response will include the final result — with the generated video URL and additional metadata.
5757

5858
{% openapi-operation spec="universal-video-endpoint-fetch" path="/v2/video/generations" method="get" %}
59-
[OpenAPI universal-video-endpoint-fetch](https://raw.githubusercontent.com/aimlapi/api-docs/refs/heads/main/docs/api-references/video-models/ByteDance/omnihuman-pair.json)
59+
[OpenAPI universal-video-endpoint-fetch](https://raw.githubusercontent.com/aimlapi/api-docs/refs/heads/main/docs/api-references/video-models/universal-video-fetch.json)
6060
{% endopenapi-operation %}
6161

6262
## Full Example: Generating and Retrieving the Video From the Server
@@ -138,7 +138,7 @@ def main():
138138
status = response_data.get("status")
139139
print("Status:", status)
140140

141-
if status == "waiting" or status == "active" or status == "queued" or status == "generating":
141+
if status == "queued" or status == "generating":
142142
print("Still waiting... Checking again in 10 seconds.")
143143
time.sleep(10)
144144
else:
@@ -309,3 +309,243 @@ Processing complete:/n {'id': '8ac142d3-7c9f-4071-bdc6-d0f2d3d9b327:pixverse/v5/
309309
**Low-res GIF preview**:
310310

311311
<div align="left"><figure><img src="../../../.gitbook/assets/pixverse-v5-image-to-video_preview.gif" alt=""><figcaption><p><code>"Mona Lisa puts on glasses with her hands."</code></p></figcaption></figure></div>
312+
313+
## Full Example #2: Lip-Sync
314+
315+
Now let’s test the parameters related to the lip-sync feature. We’ll generate a video with some character and give them a piece of text to speak. The text goes into the `lip_sync_tts_content` parameter, and the `lip_sync_tts_speaker` parameter selects one of the predefined voices.
316+
317+
The code below, just like in the first example, creates a video generation task and then automatically polls the server every 15 seconds until it finally receives the video URL.
318+
319+
{% tabs %}
320+
{% tab title="Python" %}
321+
{% code overflow="wrap" %}
322+
```python
323+
import requests
324+
import time
325+
326+
# Insert your AI/ML API key instead of <YOUR_AIMLAPI_KEY>:
327+
api_key = "<YOUR_AIMLAPI_KEY>"
328+
329+
# Creating and sending a video generation task to the server
330+
def generate_video():
331+
url = "https://api.aimlapi.com/v2/video/generations"
332+
headers = {
333+
"Authorization": f"Bearer {api_key}",
334+
}
335+
336+
data = {
337+
"model": "pixverse/v5/image-to-video",
338+
"image_url": "https://raw.githubusercontent.com/aimlapi/api-docs/main/reference-files/news-presenter.jpg",
339+
"prompt": "A young news presenter standing in the studio, facing the camera directly, eyes always on the camera, calm and professional, very still posture, minimal head movement, no sudden gestures, with a gentle friendly smile, confident stance, studio lighting, broadcast framing, realistic style, neutral background activity.",
340+
"lip_sync_tts_content": "Hello and welcome. This is our latest news update, and here are the headlines.",
341+
"lip_sync_tts_speaker": "Chloe"
342+
}
343+
344+
response = requests.post(url, json=data, headers=headers)
345+
if response.status_code >= 400:
346+
print(f"Error: {response.status_code} - {response.text}")
347+
else:
348+
response_data = response.json()
349+
# print(response_data)
350+
return response_data
351+
352+
353+
# Requesting the result of the task from the server using the generation_id
354+
def get_video(gen_id):
355+
url = "https://api.aimlapi.com/v2/video/generations"
356+
params = {
357+
"generation_id": gen_id,
358+
}
359+
360+
headers = {
361+
"Authorization": f"Bearer {api_key}",
362+
"Content-Type": "application/json"
363+
}
364+
365+
response = requests.get(url, params=params, headers=headers)
366+
return response.json()
367+
368+
369+
def main():
370+
# Running video generation and getting a task id
371+
gen_response = generate_video()
372+
print(gen_response)
373+
gen_id = gen_response.get("id")
374+
print("Generation ID: ", gen_id)
375+
376+
# Try to retrieve the video from the server every 15 sec
377+
if gen_id:
378+
start_time = time.time()
379+
380+
timeout = 1000
381+
while time.time() - start_time < timeout:
382+
response_data = get_video(gen_id)
383+
384+
if response_data is None:
385+
print("Error: No response from API")
386+
break
387+
388+
status = response_data.get("status")
389+
390+
if status in ["queued", "generating"]:
391+
print(f"Status: {status}. Checking again in 15 seconds.")
392+
time.sleep(15)
393+
else:
394+
print("Processing complete:\n", response_data)
395+
return response_data
396+
397+
print("Timeout reached. Stopping.")
398+
return None
399+
400+
401+
if __name__ == "__main__":
402+
main()
403+
```
404+
{% endcode %}
405+
{% endtab %}
406+
407+
{% tab title="JavaScript" %}
408+
{% code overflow="wrap" %}
409+
```javascript
410+
// Insert your AIML API Key instead of <YOUR_AIMLAPI_KEY>
411+
const apiKey = '<YOUR_AIMLAPI_KEY>';
412+
413+
// Creating and sending a video generation task to the server
414+
async function generateVideo() {
415+
const url = 'https://api.aimlapi.com/v2/video/generations';
416+
417+
const data = {
418+
model: 'pixverse/v5/image-to-video',
419+
image_url: 'https://raw.githubusercontent.com/aimlapi/api-docs/main/reference-files/news-presenter.jpg',
420+
prompt: 'A young news presenter standing in the studio, facing the camera directly, eyes always on the camera, calm and professional, very still posture, minimal head movement, no sudden gestures, with a gentle friendly smile, confident stance, studio lighting, broadcast framing, realistic style, neutral background activity.',
421+
lip_sync_tts_content: 'Hello and welcome. This is our latest news update, and here are the headlines.',
422+
lip_sync_tts_speaker: 'Chloe'
423+
};
424+
425+
try {
426+
const response = await fetch(url, {
427+
method: 'POST',
428+
headers: {
429+
Authorization: `Bearer ${apiKey}`,
430+
'Content-Type': 'application/json',
431+
},
432+
body: JSON.stringify(data),
433+
});
434+
435+
if (!response.ok) {
436+
const errorText = await response.text();
437+
console.error(`Error: ${response.status} - ${errorText}`);
438+
return null;
439+
}
440+
441+
const responseData = await response.json();
442+
console.log(responseData);
443+
return responseData;
444+
} catch (error) {
445+
console.error('Request failed:', error);
446+
return null;
447+
}
448+
}
449+
450+
// Requesting the result of the task from the server using the generation_id
451+
async function getVideo(genId) {
452+
const url = new URL('https://api.aimlapi.com/v2/video/generations');
453+
url.searchParams.append('generation_id', genId);
454+
455+
try {
456+
const response = await fetch(url, {
457+
method: 'GET',
458+
headers: {
459+
Authorization: `Bearer ${apiKey}`,
460+
'Content-Type': 'application/json',
461+
},
462+
});
463+
464+
return await response.json();
465+
} catch (error) {
466+
console.error('Error fetching video:', error);
467+
return null;
468+
}
469+
}
470+
471+
// Initiates video generation and checks the status every 15 seconds until completion or timeout
472+
async function main() {
473+
const genResponse = await generateVideo();
474+
475+
if (!genResponse || !genResponse.id) {
476+
console.error("No generation ID received.");
477+
return;
478+
}
479+
480+
const genId = genResponse.id;
481+
console.log("Generation ID:", genId);
482+
483+
const timeout = 1000 * 1000; // 1000 sec
484+
const interval = 15 * 1000; // 15 sec
485+
const startTime = Date.now();
486+
487+
const checkStatus = async () => {
488+
if (Date.now() - startTime >= timeout) {
489+
console.log("Timeout reached. Stopping.");
490+
return;
491+
}
492+
493+
const responseData = await getVideo(genId);
494+
495+
if (!responseData) {
496+
console.error("Error: No response from API");
497+
return;
498+
}
499+
500+
const status = responseData.status;
501+
502+
if (["waiting", "queued", "generating"].includes(status)) {
503+
console.log(`Status: ${status}. Checking again in 15 seconds.`);
504+
await new Promise(resolve => setTimeout(resolve, interval));
505+
return checkStatus();
506+
} else {
507+
console.log("Processing complete:\n", responseData);
508+
}
509+
};
510+
511+
await checkStatus();
512+
}
513+
514+
main();
515+
```
516+
{% endcode %}
517+
{% endtab %}
518+
{% endtabs %}
519+
520+
<details>
521+
522+
<summary>Statuses</summary>
523+
524+
<table><thead><tr><th width="169.99993896484375">Status</th><th>Description</th></tr></thead><tbody><tr><td><code>queued</code></td><td>Job is waiting in queue</td></tr><tr><td><code>generating</code></td><td>Video is being generated</td></tr><tr><td><code>completed</code></td><td>Generation successful, video available</td></tr><tr><td><code>error</code></td><td>Generation failed, check <code>error</code> field</td></tr></tbody></table>
525+
526+
</details>
527+
528+
<details>
529+
530+
<summary>Response</summary>
531+
532+
{% code overflow="wrap" %}
533+
```json5
534+
{'id': '3yFHGAkECD5RPnpL11mHe', 'status': 'queued', 'meta': {'usage': {'credits_used': 2000000}}}
535+
Generation ID: 3yFHGAkECD5RPnpL11mHe
536+
Status: queued. Checking again in 15 seconds.
537+
Status: generating. Checking again in 15 seconds.
538+
Status: generating. Checking again in 15 seconds.
539+
Status: generating. Checking again in 15 seconds.
540+
Processing complete:
541+
{'id': '3yFHGAkECD5RPnpL11mHe', 'status': 'succeeded', 'video': {'url': 'https://cdn.aimlapi.com/panda/pixverse%2Fmp4%2Fmedia%2Fweb%2Fori%2FJVT-OZSEbeCvZ2IKlQK6p_seed1592035041.mp4'}}
542+
```
543+
{% endcode %}
544+
545+
</details>
546+
547+
**Processing time**: \~1 min 2 sec.
548+
549+
**Generated video** (1280x720, with sound):
550+
551+
{% embed url="https://drive.google.com/file/d/1Ua5yIzQyILhoQ0mhyrykt_2xcEVQjB3s/view" %}

0 commit comments

Comments
 (0)