ESP32 Text-to-Speech with wit.ai and OpenAI TTS

Introduction

Adding voice capabilities to microcontroller projects brings a whole new level of interactivity.  While you can display information or status alerts using displays and LEDs, voice capability provides a more “human-friendly” experience.  And with Text-to-Speech systems, vocal output becomes a reality.

Text-to-speech is exactly what it sounds like: you feed in some text, and speech comes out. It is a great way to add another dimension to your microcontroller projects, whether that is a smart home device announcing an alert, a robot with a personality, or an assistive device that reads information aloud.

There are two general approaches to text-to-speech on an ESP32. The first is a standalone method in which you store a collection of prerecorded words on the microcontroller and match them to incoming text. This works, but it limits you to whatever vocabulary you can fit into the available RAM and PSRAM, and the voice quality is usually poor. You end up with a robotic sound, a sort of Stephen Hawking effect, which can be fine for some projects but falls short for most.

The second approach, which we will use today, is an online method. The ESP32 sends your text over Wi-Fi to a cloud-based AI service through an API, and the service returns the speech as an audio stream. With this method, you can type in pretty much anything, in any language, and have it spoken back to you in a natural voice.

We will be working with two of these services. The first is wit.ai, a free service from Meta. The second is OpenAI TTS, a paid service that costs only a fraction of a cent per request. Both systems can be tested using the same hookup – I’m using a Seeeduino XIAO ESP32S3 and a MAX98357A I2S audio amplifier for thsese demonstrations.

So let’s see what it takes to make our ESP32 speak to us!

Text-To-Speech

Electronic text-to-speech systems have been around for quite some time. In 1939, Bell Telephone Laboratories introduced the Voder, the first electronic speech synthesizer. It did not read text automatically; instead, a trained operator used a keyboard to shape electronic tones into recognizable speech.

The 1960s saw several developments in formant text-to-speech systems, with much of the pioneering work taking place in Sweden and the United Kingdom. These systems generated speech mathematically rather than playing back recordings, laying the groundwork for the synthesizers that followed.

The first commercial text-to-speech systems arrived in the 1980s. The best known of these was DECtalk, which became famous as the voice of Stephen Hawking. In the 1990s, unit selection became popular. This technique drew on a large database of prerecorded sounds that could be joined together to form natural-sounding sentences.

From 2016 onward, neural network TTS has taken over. These are deep learning and transformer models, and they are the type of text-to-speech systems we will be working with today.

Types of Text-to-Speech Systems

Formant synthesis is one of the earlier types. It uses mathematical models to simulate the human vocal tract.

Frequencies called formants define the vowels and consonants in speech, and a rule engine converts phonemes, the basic units of speech, into those formants.

Formant synthesis has low storage and computational requirements, which makes it ideal for microcontrollers. It needs no network connection, and you can control many of its parameters.

But it also has a major downside – the sound is robotic and unnatural. You can think of this as the “classic computer voice”, one that you are familiar with in toys and early speech synthesizers. It also relies on complex linguistic rules and struggles with unfamiliar words.

Concatenative synthesis came next. It uses recorded utterances from voice actors, separated not into words but into smaller units called diphones and triphones. These produce a more natural voice output.

Databases for this type of voice synthesis can become very large, and they are often domain-specific, with specialized vocabularies. The output sounds very realistic, and you can distinctly recognize the speaker, but the storage requirements are large.

This is not a type of synthesis we can run on a microcontroller.

Neural network TTS is the type we will be using today, and it can be divided into three sections.

The first is the text front end, where the text is normalized into exactly what we want to say, and a grapheme-to-phoneme (G2P) converter turns the written letters into phoneme sequences.

In the second stage, those phoneme sequences are translated into a mel spectrogram, a detailed visual representation of the sound, built frame by frame using autoregressive prediction.

The third and final section is the vocoder, which uses a deep neural network to translate the mel spectrogram into actual audio waveforms. Those waveforms go to an audio amplifier and out to a speaker.

Neural network TTS produces a very natural output with an essentially unlimited number of speaker identities. The catch is the high computational requirement, which puts it well beyond what a microcontroller can handle on its own.

But by using API’s and cloud services, we can give our ESP32 (and other microcontroller) projects a voice.

How the ESP32 Fits In

Here is how the system works:

The ESP32 formats the text as a request and sends it over an encrypted HTTPS connection to an API at either wit.ai or OpenAI. The service processes the text using its own neural network and returns audio, which the ESP32 decodes and streams out to the speaker. The microcontroller handles text formatting and audio decoding, and the external service handles the rest.

Other Text-to-Speech Services for the ESP32

While we are only working with wit.ai and OpenAI TTS today, they are far from the only options. There are many other voice services that can be used by the ESP32. Here are a few worth investigating:

Google Cloud Text-to-Speech offers a wide range of languages and neural voices via a REST API, with a generous free tier. Authentication is a bit more involved than a simple API key.

Amazon Polly returns MP3, Ogg, or PCM audio and supports several speech engines. It requires an AWS account and signed requests.

Microsoft Azure AI Speech provides a REST API with many neural voices, SSML controls for fine-tuning pronunciation, and several output formats.

ElevenLabs produces some of the most realistic voices available and supports streaming playback, though larger projects will need a paid plan.

Offline options also exist if a network connection is out of the question. Libraries like Talkie run entirely on the microcontroller. The vocabulary and voice quality are limited, but there is no API cost and no Wi-Fi requirement.

wit.ai

The first service we will be using is wit.ai, a product of Meta, the company that owns Facebook. It is a cloud-based text-to-speech service with a free tier, which we will use today. You will not be charged anything, and you will not need to enter any credit card information. The service provides 23 different voices with multilingual support and works via a simple HTTP GET request.  So it’s pretty easy to use.

Another advantage of using wit.ai is its ESP32 library, which makes programming very straightforward.

Logging In and Creating an API Key

Since wit.ai is owned by Meta, you will need a Meta account to log in. This can be a Facebook or Instagram account. Here is the process:

1. Go to wit.ai and log in with your Meta account.
2. From the main screen, click the button to create a new app. Give it a name, select your language, and you can keep it as a private app if you wish. Hit the Create button, and your app is ready.
3. Open your new app, and under Management, go to Settings.
4. In Settings, you will see both a Server Access Token and a Client Access Token. You want the Server Access Token. Click on it, and it will be copied to your clipboard.
5. Paste the token somewhere safe, because you will need it in your code.

That is really all there is to do on the wit.ai side. Treat the token like a password: do not publish it or include it in code you share, and if it is ever exposed, you can replace it from the wit.ai console.

The WitAITTS Library

To make things easy, we will be using the WitAITTS library by Jobit Joseph, which was created specifically for using wit.ai with the ESP32. You can find it on GitHub, but the simplest way to install it is through the Arduino IDE. Open the Library Manager, search for “WitAITTS”, and install it along with any dependencies it requests.

There is one other setting you will need before compiling. Go into Tools, find the PSRAM entry, and change it from Disabled to OPI PSRAM. The library needs the extra memory for its audio buffers. You will want this enabled for both this experiment and the OpenAI experiment later on.

Hooking Up the Hardware

For both experiments today, we are using the following circuit. It is based on a Seeed Studio XIAO ESP32-S3, although another ESP32-S3 board would work as well. We will pair it with a MAX98357A I2S audio amplifier module and a 4- to 8-ohm speaker compatible with the module.

One more note, if you are using the XIAO ESP32-S3 with an external antenna: make sure the antenna is connected before using Wi-Fi. If your ESP32-S3 board has a built-in antenna, you do not need to worry about this.

The Library Example Sketch

The WitAITTS library includes several examples, and the ESP32_S3_Basic example is a good one to look through, as it documents nearly every option the library offers.

Near the top of the sketch, you will find places for your Wi-Fi SSID and password, which the ESP32 needs to reach wit.ai, along with a spot to paste the Server Access Token you copied earlier.

The example is written for the default I2S pins of a full-size ESP32-S3 development board, and the XIAO does not bring those pins out. So you will need to uncomment the custom pin definitions and change them to match our wiring:

Remember, these are the actual ESP32 GPIO numbers, not the D-labels printed on the XIAO. They are always one off: D8 is GPIO 7, D9 is GPIO 8, and D10 is GPIO 9. You will also need to comment out the line that creates the TTS object with default pins and uncomment the line that creates it with the custom pins.

The real value of this example is the documentation of the options. It lists all the available voices: a number of US English female and male voices, UK English voices (including a formal male and a Cockney voice), a pirate, fantasy characters, a non-binary voice, a cartoon baby, and even Canadian female and male voices (as a Canadian male, I approve of this LOL!). The default is wit$Rebecca, an American female, but you can change it to any of the others.

The example also shows the speaking styles: the default normal voice can be changed to soft, formal, fast, or projected. Speed and pitch both use 100 as the standard value, and you can adjust them up or down. There is a gain setting from 0.0 to 1.0, which is worth experimenting with because it is a good way to tame distortion. There are even sound effects you can layer on, such as an alien effect. But to get started, I would stick with normal voices; aliens can come later!

After printing its configuration to the Serial Monitor, the example waits for you to type some text. Press Enter, and that text gets spoken from the loop. We will not run this example here because our own sketch does exactly the same thing with far fewer lines, but it is the best reference for the options available.

The wit-ai-tts-esp32.ino Sketch

Here is a simplified version of that sketch, and it is the one we will actually run.

The sketch starts by including the WitAITTS library and defining our credentials: the Wi-Fi SSID and password, plus the Server API key from wit.ai.

Next come the connections to the I2S amplifier, and we use them to create a custom WitAITTS object. The constructor takes the pins in the order BCLK, LRC, DIN:

In setup(), we start the Serial Monitor and set the wit.ai debug level to DEBUG_INFO, which gives us useful connection messages without flooding the screen. Then a single call connects us to both the Wi-Fi and wit.ai:

If the connection succeeds, we print that we are ready and configure the voice. I set the voice to Colin, the Canadian male. I picked that one because I am a Canadian male, but of course, you can change it to any voice from the list in the example sketch. The style stays at default, speed and pitch stay at 100, and I set the gain to 0.3 because I found this setup tends to clip quite a bit:

The sketch then prompts you to type any text and press Enter. The else branch handles a failed initialization, printing a reminder to check your Wi-Fi credentials and wit.ai token.

The loop is simple. The first line is tts.loop(), which constantly checks for data and keeps the audio stream moving. This should probably be the first line you put in your loop. After that, we check whether anything has been typed into the Serial Monitor. If it has, we read the line, trim it, and send it off with tts.speak(text). The text goes up to wit.ai and comes back as speech. It is just as simple as that.

Testing wit.ai

With the sketch loaded onto the XIAO ESP32-S3, I typed a familiar phrase into the Serial Monitor: “Hello and welcome to the workshop.” And sure enough, Colin (the Canadian male) spoke it back to me!

The system works properly, and for a free service, it works pretty well. However, there is some clipping and distortion in the output. I adjusted the gain and pitch settings and tried different voices, but I have not been able to eliminate it entirely. Lowering the gain helps, and a stronger 5-volt supply for the amplifier is worth trying, but a little distortion seems to come with the territory.

Still, if you need to add speech to a project at no cost, this would work out quite nicely.

OpenAI TTS

The other service we are checking out today is OpenAI TTS. It works similarly to wit.ai: you get an API key, send text to an HTTPS endpoint, and receive audio back.

OpenAI TTS is a natural end-to-end speech generation system featuring fast inference and a high-quality voice. The output is versatile, supporting WAV, MP3, and raw audio formats.

One difference from wit.ai is that this service will cost you something. Mind you, not very much: a request costs a fraction of a fraction of a cent. But it does require you to enter credit card information and put a balance on your account. That balance can be as low as five dollars, and at the rate we are using it, that will last a very long time.

In return for that small cost, you get a more human-quality voice than wit.ai delivers, and, in my experiments, better fidelity as well.

Logging In and Creating a Secret Key

Head to the OpenAI developer platform and sign in with your OpenAI account. One point worth stressing: the API is billed separately from ChatGPT. A ChatGPT Plus subscription does not include API access; the developer platform has its own account and billing setup.

Once you are on the platform, go to the API keys section in the sidebar and click to create a new secret key. Give the key a name, assign it to your default project, and, if you wish, restrict its permissions. Then click Create secret key.

Now, here is the important part: copy the key immediately. This is the only time you will ever see the full key. If you come back later, you will only see a small portion of it, and there is no way to recover the rest. If you lose it, you will have to create a new one.

Paste it somewhere safe for use in your code, and as with the wit.ai token, never publish it or share it in a sketch.

Setting Up a Funding Source

Before the key will work, you need to fund the account. In your settings, open the Billing section, add a payment method, and purchase some credits. Five dollars will do the trick and probably last you for days and days of experimenting. You do need to apply some funding to the account, though, or your requests will be rejected. It is also a good idea to set a usage limit, so a runaway sketch cannot burn through credits faster than you expect.

The openai-tts-esp32.ino Sketch

Unlike the wit.ai example, this sketch does not use a service-specific library. We are simply sending data up to an API and getting audio back, and the ESP32-audioI2S library by schreibfaul1 handles all of it: the HTTPS request, the MP3 decoding, the buffering, and the I2S output. Make sure to install “ESP32-audioI2S” from the Library Manager, and keep OPI PSRAM enabled just as before. We are also using the same wiring as the wit.ai experiment, so you can use the same breadboard.

The sketch begins by including the required libraries, including Audio.h from ESP32-audioI2S. Then come the Wi-Fi credentials and your OpenAI API key, the very long key you just created. The I2S pin definitions match our wiring (GPIO 7 for BCLK, GPIO 8 for LRC, GPIO 9 for data), and we create an Audio object to represent the audio player.

Next, the sketch lists several different voices. We are using alloy, a neutral, versatile voice, but there are several others you can pick, such as echo, fable, onyx, nova, and shimmer, so feel free to experiment. You can also change the TTS model. I am using tts-1, the faster one, but there is a higher-quality tts-1-hd model. Keep in mind that the better quality model incurs a higher cost. The format we are requesting is MP3 (WAV and raw formats are also available), and the speed is set to 1, which is normal:

In setup(), we start the Serial Monitor and connect to the Wi-Fi. Then we initialize I2S with audio.setPinout() and set the playback level to 12 with audio.setVolume(12). If the output is not loud enough for you, this is the value to play with.

The sketch then builds a text string with a greeting and passes it to speakOpenAI(). This means the board speaks as soon as it powers up, which also serves as a handy self-test: if you hear the greeting, then the Wi-Fi, the API key, the billing, the decoding, and the wiring are all working.

In the loop, we continually call audio.loop(), which must run constantly to feed the decoder and the I2S output. Then we check the Serial Monitor, and if any text has been entered, we trim it and pass it to speakOpenAI(), which sends it to the API and reads it back to us.

Finally, speakOpenAI() is the function doing the actual work. It packages up our settings and the text into a single call to the library:

The empty string is a field for voice instructions, a style hint that some models can use. Behind the scenes, the library builds the HTTPS POST request to the OpenAI speech endpoint, adds the authorization header, streams the response, and starts the MP3 decoder.

The sketch also includes a couple of callbacks: audio_info() prints library information, and audio_eof_speech() reports when the speech stream has finished. Both are handy for debugging.

Testing OpenAI TTS

For the test, I used the same hardware setup as before, but I left the USB cable disconnected until I was ready because the board greets me as soon as it starts up. On connecting it, the XIAO introduced itself: “Hello from the XIAO ESP32-S3. This is the OpenAI text-to-speech engine.” And it does sound remarkably natural and clear, although a little low in amplitude, which you can fix with the volume setting.

I then gave it the same sentence I gave wit.ai: “Hello and welcome to the workshop.” The difference was immediately apparent. The demonstration shows that OpenAI TTS produces much more natural speech than wit.ai, with smoother timing, clearer pronunciation, and a more human rhythm. If you can handle a small cost, it is by far the superior system.

Conclusion

Text-to-speech is no longer a complex, crude system that sounds like a 1980’s toy. It can produce natural-sounding voices and handle many languages.

You can use this capability to create ESP32 projects that interact with you. And although we didn’t cover it here, the same two services can also convert speech to text.  Let me know if that interests you, and I’ll cover it in a future article.

However you use it, text-to-speech is a great capability to have in your ESP32 toolbox. And with one free service and one that costs pennies, there is no reason not to give your next project a voice.

Keep talking!