Ito: Streaming Neural Speech on a $5 Chip
A 4.9 MB text-to-speech model for the ESP32-S3. It streams, runs without a cloud and is verified in emulation. Board timings to come.
Real production measurements, not marketing rounding: where the time goes in a turn, and how each of our models compares, with the methodology to reproduce it.
In real-world voice AI, TTS latency alone doesn't tell the story. We measure time to first audio: from the moment the user stops speaking to the first sound of the reply. The agent streams token by token, so the caller never waits for a whole sentence — only for the first short chunk.
Measured on production on 10 October 2026: 30 calls, one at a time, from a machine in the same AWS region as our servers (Ireland), so your own internet connection is not in the figure. Each call says one Spanish sentence after the agent's greeting; the stages come from our servers' own log of each of those turns. Hover a segment.
Ours The language-model provider, and delivery to the caller
The 596 ms is the median of the 30 calls’ own totals. Each stage is shown at its own median, and those add to 591 ms: the median of a set of totals is not the sum of the medians of its parts.
| Stage (median of 30 calls) | Latency | Runs on |
|---|---|---|
| Waiting to be sure the caller has finished | 180 ms | Our servers |
| Speech recognition (Conv 1.0) | 0 ms on the critical path — it completes inside that wait | Our GPU servers |
| Language model: the reply, started while the caller was still finishing | 225 ms | Provider |
| Speech synthesis, to first audio (the model in production on 10 October) | 64 ms — between 63 and 67 ms on every call | Our GPU servers |
| Delivery to the caller | 122 ms — the gap between our servers' clock and the caller's, the same on every call | — |
| Caller-observed time to first audio | 596 ms median — fastest 557 ms, 90th percentile 925 ms | — |
The language model starts on the caller's words before they have finished; on 26 of the 30 calls its reply was ready by the end of the wait, and on the other 4 the caller waited 925–1,137 ms. Published guidance treats under 800 ms as conversationally natural; an independent survey places production voice-AI platforms at a 1,400–1,700 ms median. These end-to-end figures were measured on 10 October 2026, before speech synthesis changed on 26 October; they will be re-measured on the current model. Speech synthesis on its own is measured in the next section.
About 5.2¢ a minute for a voice agent at a typical prompt: 4.2¢ for the platform, with speech recognition, the voice, turn-taking and orchestration inside it, plus the language model by the token. Prepaid and billed by the second; there are no plans.
| Line | Price | Billed |
|---|---|---|
| Voice agent, platform | $0.042 / min | Per second |
| Language model, input / output | $0.44 / $0.94 per million tokens | Per token |
| A typical minute, all in | 5.2¢ | Web or app |
| Phone line, call received in Spain | 3.5¢ / min | Per started minute, on top of the agent |
| Text-to-speech API | $4.10 per million characters | Per character |
| Speech-to-text API | $0.0014 / min | Per second |
Phone numbers $5 a month each; every line and country on the pricing page. Dedicated and private deployments are quoted separately.
Versa, our speech synthesis model, has served production since 26 October 2026 in its current form: an autoregressive model that generates speech one 70 ms frame at a time, for every stream in flight at once. It has 120.0 million deployable parameters, speaks nine languages in ten voices, and clones a new voice from a few seconds of reference audio.
On one production T4 GPU, measured by our backend: the first audio arrives in about 25 ms for a single request, and in a median of about 90 ms with 32 simultaneous streams, which is what one card carries. One card produces about 54 seconds of speech per second.
We synthesise held-out sentences and transcribe them with Whisper large-v3, the same recogniser for every language, then compare with the same recogniser on the real recordings of the same sentences (“real recordings”). Each held-out speaker is cloned zero-shot from a different recording of theirs; 100 to 200 sentences per language. Lower is better.
| Language | Versa | Real recordings |
|---|---|---|
| English | 2.77% | 2.19% |
| Spanish (Common Voice) | 2.11% | 3.64% |
| Spanish | 5.41% | 4.83% |
| Catalan | 3.93% | 4.59% |
| Galician | 15.57% | 16.18% |
| Basque | 26.57% | 37.90% |
| German | 7.67% | 6.84% |
| French | 11.54% | 7.60% |
| Portuguese | 8.66% | 8.36% |
| Italian | 11.99% | 12.08% |
| Mean of the ten | 9.62% | 10.42% |
Read the second column before the first: it is the recogniser, not the voice, that sets the floor. Whisper large-v3 makes more errors on real Galician and Basque speech than on any other language here, so those rows are high whatever voice is used. Versa scores lower than the real recordings on Spanish (Common Voice), Catalan, Galician, Basque and Italian; within a point above them in English, Portuguese, German and Spanish; and about four points above in French. These figures use a different recogniser setup from earlier versions of this page and are not comparable with them.
Speaker similarity, measured with ECAPA embeddings (higher is better): 0.475 when cloning 108 unseen speakers zero-shot (English, German, Spanish and French), 0.698 for our studio reference voices, against 0.87 between two real recordings of the same speaker, which is the ceiling.
Psst is our proprietary strictly-causal noise suppression model, running as low-latency middleware ahead of speech recognition. We score it on downstream word-error rate — the only metric that reaches the caller — not on perceptual audio quality, which turned out to point the wrong way entirely.
| System | Avg. noisy WER | PESQ | SI-SDR | RTF (1 core) |
|---|---|---|---|---|
| Psst (ours, in production) | 21.74% | 2.096 | 9.89dB | 0.053 (int8) |
| No enhancement | 25.32% | 1.970 | 8.44dB | — |
| Our previous suppressor | 28.91% | 2.338 | 14.68dB | 0.102 |
| DeepFilterNet3 (published SOTA) | 33.79% | 2.794 | 17.69dB | 0.058 |
300 LibriSpeech test-clean utterances mixed with held-out noise and interfering speakers; no evaluation audio appears anywhere in training. Psst leads on downstream word error by 36% over DeepFilterNet3, the strongest published real-time CPU denoiser, at comparable cost and inside a 32ms strictly-causal budget DeepFilterNet3 doesn't meet. Against competing speech — other people talking nearby, the most common real-world complaint — Psst recovers 14–28% of word errors. It runs 18 concurrent call legs per CPU core at 2.7MB, so noise suppression adds no meaningful capacity cost.
The word-error and perceptual rankings invert completely. DeepFilterNet3 leads PESQ by 0.70 and SI-SDR by 8dB and is the worst system here for transcription — worse than no denoising at all. Psst is the only system tested that improves on doing nothing. Optimizing for the industry-standard perceptual metric would have degraded the product.
Turno, our turn-taking and barge-in model, is 18,651 parameters. On a neutral real-world test set it beats Silero VAD on Average Precision (0.931 vs 0.876) and holds that precision flat from clean audio down to 0dB signal-to-noise, at roughly half Silero's per-inference cost (0.083ms per frame). TEN-VAD is ahead on voice activity alone (0.985); Turno also predicts the end of a turn and tells a real interruption from a backchannel, which voice-activity models do not. Retraining on real echo data cut the real-device echo false-confirm rate from 93.0% to 11.4%.
Conv 2.0, our own recogniser, has served production since 20 October 2026, on our own GPUs with no cloud fallback: one model for nine languages, about 466 million parameters, a fifth smaller than the NVIDIA Parakeet TDT 0.6B model it replaced (Conv 1.0). In production it transcribes a typical utterance in about 55 ms and carries around 36 utterances per second per card.
Its accuracy, in two configurations. With its language model, the configuration the benchmark is defined for: the recogniser streams and a per-language language model rescores what it hears. As served today: production calls use the GPU's direct decode alone, because on the speech card's processor, which synthesis shares, the language model does not yet run fast enough for a live call. The figures below are the model in production since the evening of 20 October 2026 (round 11).
English is scored like the Artificial Analysis streaming benchmark (AA-WER) with its published normalisation, on our own harness: the conversational set is our public stand-in for Artificial Analysis's held-out agent and customer set (the first 200 clips of it and of VoxPopuli, and six Earnings22 calls as one continuous stream), streaming in 2.56 s chunks with the language set. The eight other languages (Catalan, German, Spanish, Basque, French, Galician, Italian and Portuguese) are scored on our own test set of 100 utterances per language (60 for Basque), pooled, streaming in 5.12 s chunks with the language given. Per language with the language model: Spanish 5.13%, Italian 5.43%, Catalan 7.25%, Basque 7.57%, German 8.05%, Portuguese 8.27%, French 8.34%, Galician 10.85%.
| Benchmark | With its language model | As served today |
|---|---|---|
| Weighted conversational WER (AA-WER-style) | 5.82% | 6.05% |
| Conversational set (50% of the score) | 4.15% | 4.36% |
| VoxPopuli (25%) | 4.30% | 4.43% |
| Earnings22 long-form calls (25%, one continuous stream) | 10.66% | 11.04% |
| Eight other languages, pooled (100 utterances each, 5.12 s chunks) | 7.56% | 9.62% |
| In production: one utterance, median, on our T4 GPUs | — | about 55 ms |
State-of-the-art voice AI today means giant models on giant GPU fleets. That model breaks down the moment voice AI needs to live outside a data center.
A 4.9 MB text-to-speech model for the ESP32-S3. It streams, runs without a cloud and is verified in emulation. Board timings to come.
English and Spanish speech recognition on a $5 ESP32-S3 at 3.7% word error. Accurate, but about twice slower than real time. Open source.
The plans announced on 23 September 2026 at about 2¢ a minute, replaced on 9 October by a prepaid balance at about 5¢ a minute, model included.
Robots, wearables and connected devices can't carry a GPU cloud bill for every unit sold. Why voice AI is following compute back to the edge.
CPU-native voice AI runs recognition, understanding and speech synthesis on ordinary processors instead of GPUs. What it means and why it matters.
Want to run these benchmarks on your own data? Contact us for a technical consultation or access to our evaluation scripts.