Natural conversation has a rhythm. The average gap between one person finishing a turn and the next starting is roughly 200 milliseconds — faster than the human reaction time to visual stimuli. This implies we don’t just listen and respond; we predict when the other person is about to finish.
For voice AI, replicating this rhythm is the hardest problem. Systems that wait for silence create awkward gaps. Systems that barge-in prematurely clip the user’s speech. Both break the illusion of natural conversation.
Today we are introducing Vela — our purpose-built turn-taking model that solves this problem.
The Code
Vela is being developed in the open. You can follow our progress at dev/BSC/Vela. The repository includes:
- The full training pipeline for our turn-prediction model
- Evaluation datasets and metrics
- Pre-trained checkpoints for inference
- Integration examples with the Lokutor orchestrator
How It Works
Traditional voice agents use Voice Activity Detection (VAD) — they wait for a period of silence before assuming the user has finished speaking. This approach has fundamental limitations:
- Silence thresholds are arbitrary: Too short and you clip the user. Too long and you create awkward pauses.
- Silence doesn’t mean finished: People pause mid-sentence to think. A VAD-based system will interrupt them.
- Prosody is ignored: Humans use pitch, rhythm, and intonation to signal turn completion. VAD sees none of this.
Vela takes a different approach. It analyzes the acoustic and prosodic features of the incoming audio stream in real-time and predicts whether the speaker is approaching a Transition-Relevance Place (TRP) — the point in speech where a turn change is natural.
Architecture
Vela is a lightweight transformer-based model designed to run on CPU with minimal overhead:
Audio Frame → Feature Extractor → Temporal Encoder → TRP Predictor- Feature Extractor: Extracts pitch, energy, spectral features, and voice activity from 32ms audio frames
- Temporal Encoder: A small transformer that models context across ~2 seconds of audio
- TRP Predictor: Outputs a probability that the current moment is a natural turn transition
The model adds approximately 8ms of latency to the pipeline — negligible compared to the 200-500ms improvement in turn-taking quality.
Why It Matters
Turn-taking is the difference between a voice agent that feels like a tool and one that feels like a conversation partner. Vela enables:
- Natural response timing: The agent speaks exactly when a human would
- No awkward silences: Eliminates the “are you still there?” moments
- Smooth barge-in: The agent can be interrupted naturally without fighting for the floor
Next Steps
Vela is currently in active development. We are working on:
- Expanding training data to cover more languages and speaking styles
- Reducing the model size for edge deployment
- Publishing a research paper with full benchmarks
Check out the code at dev/BSC/Vela and let us know what you think.