To add new audio modality, we do the following steps.
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "What's in this image?"},
{
"type": "audio_url",
"audio_url": {
"url": "https://st.com/audio.wav",
}
},
],
}
],
or
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "What's in this image?"},
{
"type": "audio_b64_json.",
"audio_b64_json.": {
"b64_json.": "Base64EncoedAudio",
}
},
],
}
],
To add new audio modality, we do the following steps.
or