An AI Agent reading a local database with a voice interface based on twillio

Setup is based on: https://www.twilio.com/en-us/blog/voice-ai-assistant-openai-realtime-api

Create .env file

First create a .env file in the root directory of your project. edit it with text editor and add the following line:

    OPENAI_API_KEY=your_openai_api_key
    

You can start the server

Use command

uvicorn main:app --host 0.0.0.0 --port 5050

if you want to change the port number you can add the following line to .env file

PORT=your_port_number

Set up a port tunneling service

In this example we use zrok. Download and register to zrok. Afterwards run following commands:

zrok enable YOUR_TOKEN 
zrok share public --headless http://localhost:5050

Configure twillio

Set up a twillio account and get a phone number. Configure the webhook to point to your server's URL (from the port tunneling service) followed by /incoming-call endpoint.

References

code based on: https://www.twilio.com/en-us/blog/voice-ai-assistant-openai-realtime-api-python