ESP32 AI Hardware Projects — Voice Assistants, AI Boxes & Smart Toys
The ESP32 became the default brain of small AI gadgets for a concrete reason: it's the cheapest board with enough RAM and Wi-Fi to stream audio to a cloud model, wake from deep sleep in milliseconds, and drive a display and speaker — all for a few dollars. These are the builds that prove it, ordered from easiest to most ambitious.
Quick answer: The five builds that define the genre: 1) an AI voice assistant (ESP32 + microphone + speaker, streaming audio to a cloud speech API — the 'xiao zhi' style assistant that went viral), 2) a desktop AI box (small screen + wake word + LLM answers read aloud), 3) a status/AI-token display (your LLM quota, server logs or stock ticker on a tiny screen), 4) a smart knob (encoder + haptic ring + LLM for intent, e.g. controlling lights by voice or rotation), 5) an AI toy (camera + vision API, or a plushie with a chat personality). Build the voice assistant first — it reuses everything.
Build 1: AI voice assistant
Parts: ESP32-S3 or C3 dev board, INMP441 I2S microphone, MAX98357A I2S amplifier + small speaker, optional PSRAM (needed for high-quality audio streaming on the S3). Wiring: I2S data/bclk/lrclk from the mic to three GPIOs, the amp's data pin to one GPIO, common ground throughout. Software: record a wake phrase, stream audio to a speech-to-text API, send the transcript to an LLM, synthesize speech and play it. The pipeline is the same no matter which assistant front-end you use — and the whole thing runs on ~10 mA average with deep sleep between interactions.
| Part | Typical part | Interface | GPIO class |
|---|---|---|---|
| Microphone | INMP441 (I2S) | I2S BCK/WS/DT | any output-capable pin |
| Amplifier | MAX98357A (I2S) | I2S data + SD mode | any output-capable pin |
| Speaker | 3 W / 4 Ω, 8 Ω | amp output | — |
| Display (optional) | ST7789 1.3–2.4" | SPI | SPI-capable pins |
| Button | any momentary | GPIO + pull-up | avoid strapping pins |
Build 2–4: desktop box, status display, smart knob
A desktop AI box is the voice assistant with a screen and a bolder shell: question text scrolls on a small SPI display, the answer is spoken, and the whole thing looks like a $200 product built for $30. The status/token display is the easiest real-world gadget: an S3 or classic board, a small display, and an MQTT or HTTP poll that shows your LLM API quota, GitHub stars, or any number — it teaches the 'server without a server' pattern. The smart knob (rotary encoder + optional haptic feedback) teaches input handling and acting on LLM intent: turn to adjust, press-and-hold to ask.
Build 5: AI toy
Two flavors: camera toys (ESP32-CAM or an S3 + OV2640, sending frames to a vision API for 'what do I see?') and chat toys (a plushie or badge that talks to an LLM with a personality). Both boil down to the voice-assistant pipeline with one modality swapped. Camera toys need more RAM — prefer an S3 with PSRAM and keep JPEG resolution small.
The software pattern under all five
Every build is the same loop: capture (audio/vision/encoder) → preprocess → call a cloud API → act (speak/display/move) → deep sleep. The two decisions that define your project are the wake condition (button, wake word, or schedule) and where the intelligence lives (cloud LLM vs local intent parsing). None of it needs a backend you operate — free-tier speech and chat APIs cover hobby traffic.
Frequently Asked Questions
What's the cheapest way to start with AI hardware?
Do I need PSRAM for AI projects?
How much does it cost to run per month?
Is the audio quality good enough?
Where do I start if this is my first ESP32 project?
More tools used in this guide
Interactive pinout for the ESP32 family: click any pin on a DevKitC V4, S3-DevKitC-1, C3-DevKitM-1 or C6-DevKitC-1 to see its GPIO number, every official function, and the boot-time traps — transcribed from Espressif's official header tables.
Battery runtime for microcontroller projects: enter capacity, active current, duty cycle and sleep current to get real-world battery life in hours and days — the way it actually behaves on an ESP32-class board.
Power supply sizing for LED strips: enter length and W/m to get total watts, total amps, the PSU wattage to buy with headroom, and how many power injection points the strip needs.
More guides
10 starter projects in the right order: from blink to Wi-Fi to your first cloud dashboard, with the hardware list and the wiring traps each one teaches.
How to actually get weeks-to-months battery life: deep sleep modes, wake sources, the dev-board overhead leak, and a measurement recipe to find your real sleep current.