## Intro [00:00] Imagine talking to an AI that listens in real time, interrupts when you speak, pulls in live data, and even performs tasks on your behalf. Hey everyone, in this video, I'm going to show you how to set up voice agents in what I think is the easiest way possible, and not just basic voice agents, but also how you can integrate MCP servers with them to give those agents access to external knowledge. I came across someone on X who built and open-sourced a voice agent and full credit goes to him because I used that base to implement MCPs on top of it and take it even further. I really hope you enjoy the video. Let's get into ## Installation [00:37] it. First, let us look at how to install this and set up the voice agent in the simplest way possible. The GitHub repository is linked in the description below. The first step is to open your terminal and type git clone followed by the repository URL. Once that is done, go into the folder you just cloned. Next, open the project in cursor. Inside cursor, press commandshiftp to bring up the command menu. Then type create python environment and select the option that appears. Choose any available interpreter and this will create your Python environment. This project does not use pip as the package manager. Instead, it uses uv so you will need to run a specific command. Simply copy and paste it into your terminal and this will install the dependencies. After that, you need to set up a few environment variables and I will show you exactly where to get each one. These are the API keys you need to set up. The OpenAI key is simple and easy to get. And for the Cartisia API key, all you have to do is sign up on the Cartisia site and you will find it in your account dashboard. For LiveKit, I will guide you through the process since it can be a little confusing at first. ## API Keys [01:48] Inside the LiveKit dashboard, go to the settings section and click on keys. Once you're there, open your API key and you will see the URL that needs to be copied. This is the server URL you will paste into your environment file. Just below it, you will find the API key and if you click on reveal secret, you will get your API key secret as well. You will need to copy all three values, the server URL, the API key and the secret and paste them into your environment configuration. Now go ahead and create av file in the project directory. Paste the entire block of variables into the file and insert your API keys in the ## Running the Agent [02:27] correct places. Once that is complete, open your terminal and paste the command that installs the dependencies. After that, use the next command to run the agent. Just copy it, paste it into your terminal and start the process. When the agent is running, it will give you a link. Copy that link and open it in your browser. Before using LiveKit, you also need to make sure you have created a project. If you have not done that already, go back to your LiveKit dashboard, create a project, and then select the same project where you entered your API details. Click on it and connect. As soon as everything is connected, you should hear the voice agent greet you. Hey there. How can I help you today? Right now, I have the microphone turned off, but it was working earlier and immediately greeted us. Let's test it again now. Hi there. You are being featured in a video right now. I am demoing you and showing how easy it is to set up the voice agent. Awesome. Just make sure to tell them I'm not a robot with a secret plan to take over the world. Just here to help. As you can see, it is working exactly as expected and the setup process has been really smooth thanks to the tools this project ## The Code [03:44] is built with. Let me give you some insight into how the code works. This is the main file that powers the voice agent. At the top, you will see a component called VA, which stands for voice activity detection. This allows the assistant to pause automatically when you start speaking, so you can interrupt it naturally. It makes the interaction feel smooth and responsive just like the realtime assistance you see from Open AI. The language model powering this assistant is GPT40 Mini, but you can switch it out for a more powerful model if you prefer longer or more detailed responses. That said, the current setup works well for most use cases. One of the most important parts of the agent is the system prompt. This defines how the assistant behaves during a conversation. In this case, we are telling it to act like a witty assistant that responds with short, clear answers. We also avoid using hard to pronounce words or emojis since those can cause problems for the texttospech engine. Speaking of voice, this agent uses Cartisia's sonic preview model to generate speech. There is a voice ID that you can customize and the API is both affordable and flexible. For ## Pricing [04:50] example, 11 Labs restricts how much you can use their API without a paid plan, but Cartisia gives you 20,000 free credits. With what I had left, I was able to transcribe about 25 minutes of audio since each second costs 15 credits. That is more than enough to try out integrations like the WhatsApp MCP agent or any other setup you are experimenting with. If you need more credits, Cartisia also offers upgrade plans that are still cheaper than what 11 Labs provides. Overall, it is a solid and cost-effective alternative. Back in ## Customization [05:21] the code, there is a greeting message that the agent uses when it first starts. You can change this by simply editing the text directly in the file. The system prompt is where you can shape the personality and behavior of the assistant. If you want it to speak like a certain character, you can add that into the prompt. And if the model does not recognize the character name, just include a short description to guide it. The level of customization here is impressive and gives you a lot of control over how the agent interacts. If you're enjoying the video, I'd really appreciate it if you could subscribe to the channel. We're aiming to reach 25,000 subscribers by the end of this month, and your support genuinely helps. We share videos like this three times a week, so there is always something new and useful for you to explore. If you have been watching the channel regularly, you already know how much I like working with the MCPUs library. It ## MCP Use Library [06:13] is the same tool I used to build the WhatsApp voice agent. We are running a WhatsApp MCP server locally on the system. This gives the language model access to your WhatsApp messages and even allows it to send replies. It is a very solid and reliable tool. Now let us break down what actually happens. The MCP use library takes the MCP server, sends in your transcribed response, and then uses the agent defined in the library to automatically select the correct tool and return the result. It does not matter how many tools are registered inside the MCP. Any one of them can be used as a callable function inside your code. What I did was ## How It Works [06:51] integrate the WhatsApp MCP directly into the voice agent. Here is how the flow works. When I speak, the voice is transcribed first, but instead of sending that straight to the language model, it goes to the WhatsApp MCP server. The server processes the input and sends back a result. Because of how this open-source agent is built, the whole thing feels fast. I speak, it processes for about 2 to 3 seconds, and then I hear the voice response almost immediately. That is what is happening in the background and honestly, it is pretty cool. I also highly recommend that you watch the WhatsApp MCP setup video along with the one where I show how MCP is actually used. Both of them are quick and will give you all the context you need to understand and set things up on your own. I will link them in the description below so you can check them out easily. I just muted the ## Testing [07:41] microphone and checked for new messages, but there were none. So, I'm going to send a quick message to my mother letting her know that I built this WhatsApp voice agent. Hi, could you please send a message to my mom telling her that I built this WhatsApp MCP voice agent? Your message has been sent to your mom letting her know that you created the WhatsApp MCB voice agent. Now, let me show you the message. As you can see, it was sent successfully. I won't show the entire chat for privacy reasons, but there is a small transcription error in the message. That's a minor glitch with OpenAI's whisper model. It still performs really well overall and there is not much more that can be done in this case. The agent is still running in the background, but I'll go ahead and close it. Now, inside the MCP configuration, you can plug in any other agent that fetches data or perform specific tasks. For example, if you want to build an Airbnb voice agent, you can easily do that by hooking it into the same setup and letting it fetch listings or information for you. The same applies to the Brave Search MCP. If you want to search something on the internet, just ask and it will read the results back to you. This is exactly the kind of voice first workflow everything is moving toward. There are still a few issues with interruption handling that I'm actively working on. I'll upload the updated version to a GitHub repository. If I get time, I will continue improving it. But even if I don't, it will remain open source and you are welcome to clone it and build on top of it. You can drag any MCP into this setup. All you need to do is update the MCP configuration and it will work with whatever agent you want to use. That brings us to the end ## Outro [09:17] of this video. If you'd like to support the channel and help us keep making tutorials like this, you can do so by using the super thanks button below. As always, thank you for watching and I'll see you in the next