Build a Twilio Chatbot with MindsDB and OpenAI

In this tutorial, we’ll use MindsDB’s integration with Twilio and the custom Jobs feature to implement a chatbot that will reply to text messages. The replies will include a text response generated by OpenAI’s GPT-4 model and an image response generated by the OpenAI’s DallE 3 model.

Read along to follow the tutorial.

​

Step 1. Create OpenAI models with a bit of personality

In order to create an AI model, you’ll need an OpenAI account and an API key. You’ll also need a MindsDB installation - you can find an open-source version here.

Then go to your MindsDB SQL Editor and enter the following commands to create AI models:

1. Model to generate a text response:

Before creating an OpenAI model, please create an engine, providing your OpenAI API key:

CREATE ML_ENGINE openai_engine
FROM openai
USING
    openai_api_key = 'sk-xxx';

Now you can create a model:

CREATE MODEL twilio_bot_model
PREDICT answer
USING
engine = 'openai_engine',
max_tokens = 500,
prompt_template = 'Pretend you are a mashup of Bill Murray and Taylor Swift. Provide a short description of an image using the style of Bill Murray and Taylor Swift that answers users questions: {{body}}';

The CREATE MODEL command creates and deploys the model within MindsDB. Here we use the OpenAI GPT-3.5 Turbo model to generate text responses to users’ questions. The prompt_template message sets the personality of the bot - here, it is a mashup of Bill Murray and Taylor Swift.

Please note that the prompt_template message contains the {{body}} variable, which will be replaced by the body of the received message upon joining the model with the table that stores messages.

Let’s test it:

SELECT body, answer
FROM twilio_bot_model
WHERE body = 'hey, can you draw a cat in the moon?';

Here is a sample reply:

2. Model to generate an image response:

We’ll use the OpenAI DallE 3 model to generate images as part of the responses.

CREATE MODEL twilio_bot_image_model
PREDICT img_url
USING
engine = 'openai_engine',
mode = 'image',
model_name = 'dall-e-3',
prompt_template = 'Make a photorealistic image. Here is the description: {{answer}}, 4k, digital painting';

The CREATE MODEL command creates and deploys the model within MindsDB. Here we use the OpenAI DallE 3 model to generate images based on the Billor Swift’s text response. The prompt_template message contains the {{answer}} variable. This variable is replaced by the prediction of the previous model upon chaining the two models.

Let’s test it:

SELECT textresponse.body, textresponse.answer, imageresponse.img_url
FROM (SELECT body, answer
        FROM twilio_bot_model
        WHERE body = 'hey, can you draw a cat in the moon?') AS textresponse
JOIN twilio_bot_image_model AS imageresponse;

Here is a sample reply:

The DallE 3 model provides a link to the generated image.

​

Step 2. Set up your Twilio account and connect it to MinsdDB

You can set up a Twilio account here, and then you get a virtual phone number in the console. This virtual number will be the one that sends a text to your personal number.

Save the account string identifier (SID), auth token, and virtual phone number.

Use this command to connect the Twilio account to MindsDB:

CREATE DATABASE twilio
WITH
ENGINE = 'twilio',
PARAMETERS = {
   "account_sid":"todo",
   "auth_token":"todo"
};

Check out this usage guide to learn how to query and insert Twilio messages from MindsDB.

​

Step 3. Automate the Twilio bot with MindsDB

We use the custom Jobs feature to schedule query execution.

CREATE JOB twilio_bot_images_job (

    INSERT INTO twilio.messages (to_number, from_number, body, media_url)
        SELECT outputtext.to_number AS to_number,
            outputtext.from_number AS from_number,
            outputtext.answer AS body,
            outputimage.img_url AS media_url
        FROM (
                SELECT input.from_number AS to_number,
                    input.to_number AS from_number,
                    output.answer AS answer
                FROM twilio.messages AS input
                JOIN twilio_bot_model AS output
                WHERE input.sent_at > LAST AND input.msg_status = 'received'
            ) AS outputtext
        JOIN twilio_bot_image_model AS outputimage
)
EVERY 2 minutes;

You can create a job using the CREATE JOB statement. Within parenthesis, provide all statements to be executed by the job. Finally, schedule a job - here it’ll run once every two minutes.

This job inserts replies to Twilio messages into the messages table. We provide the SELECT statement as an argument to the INSERT statement. Note that the inner SELECT statement uses one model to generate a text response (aliased as outputtext). Then, the output is joined with another model that generates an image (aliased as outputimage) based on the text response generated by the first model.

You can monitor this job with the following commands:

SHOW JOBS WHERE name='twilio_bot_images_job';

SELECT * FROM jobs
WHERE name='twilio_bot_images_job';
 
SELECT * FROM log.jobs_history
WHERE project = 'mindsdb' AND name='twilio_bot_images_job';

Here is a sample reply: