Psst: Our Noise Cancellation Model for Real-World Voice AI

Voice AI is only as good as its ability to listen. In a quiet room, speech recognition is nearly perfect. But the real world is not a quiet room.

We are introducing Psst — our real-time noise cancellation model designed specifically for voice AI pipelines. It cleans the audio signal before it reaches the STT engine, improving accuracy by up to 25% in high-noise environments with under 5ms of added latency.

The Code

Psst is being developed in the open. You can follow our progress at dev/Psst. The repository includes:

• The full model architecture and training code
• Pre-trained weights for CPU inference
• Dataset generation tools and evaluation scripts
• Integration with the Lokutor orchestrator pipeline

The Problem

Background noise is the #1 cause of voice AI failure in production. It causes three problems:

1. STT hallucination: Noise is misinterpreted as speech, creating phantom transcriptions
2. False VAD triggers: The agent thinks the user is speaking when it’s just background chatter
3. Reduced confidence: The LLM receives garbled input, producing worse responses

Our Approach: Lightweight Spectral Gating

Psst uses a mask-based spectral gating approach optimized for real-time streaming:

Noisy Audio → STFT → Mask Estimation → Spectral Gating → iSTFT → Clean Audio

The key innovation is in the Mask Estimation network — a small recurrent architecture that predicts a time-frequency mask based on the spectral characteristics of human speech vs. environmental noise.

Unlike traditional noise cancellation that removes fixed frequency bands, Psst learns what speech sounds like and preserves it while removing everything else.

Key Design Decisions

• Single-channel: Works with standard mono microphones, no special hardware needed
• No lookahead: Processes each frame as it arrives, suitable for streaming
• CPU-first: Optimized for ARM and x86 without GPU acceleration
• 5ms budget: The entire model runs in under 5ms on modern CPU hardware

Benchmarks

Psst has been tested against standard objective metrics:

Metric	Score	Description
PESQ (WB)	1.82	Perceptual quality (higher is better)
STOI	78.1%	Speech intelligibility (higher is better)
Latency	<5ms	Added pipeline overhead
CPU usage	<3%	On modern x86 core (one thread)

Integration

Psst integrates as a drop-in preprocessing step in the Lokutor pipeline:

const client = new VoiceAgentClient({
  apiKey: '...',
  noiseCancellation: true,
  noiseCancellationModel: 'psst-v1',
});

Or as a standalone module in your own pipeline:

from lokutor.noise import PsstCanceller

canceller = PsstCanceller()
clean_audio = canceller.process(noisy_audio)

Next Steps

Psst is in active development. We are working on:

• Multi-channel support for array microphones
• Adaptive noise profiling for dynamic environments
• Music preservation mode for creative applications

Check out the code at dev/Psst and let us know what you build.