Bandwidth Labs built our own streaming Speech-to-Text model — LISTEN is now in beta

[u/LostMathGuy]

My team at Bandwidth Labs has been working on something for a while that I'm pretty excited to finally put in people's hands.
We set out on this journey after building voice agents and working with customers to deploy them. When we started I was pretty impressed with how good transcription models actually were as far as accuracy goes… But they left a lot to be desired when we introduced real world telephone calls. On top of that as we wrestled to claw back all the milliseconds we could, the way transcriptions were handled became an area of research for us.

We built our own Speech to Text model from scratch.

The goal was to build it from day one to be streaming native, and pay extra attention to things like:
• 8khz telephone audio and challenging acoustic conditions
• Low Latency
• Not having partial transcripts constantly changing underneath you
• Efficiency that would allow us to deploy it at our network edges for optimized latency

It supports μ-law, A-law, G.722, Linear16 and Opus, including 8khz phone audio.
You get per word timestamps, and we also have keyword boosting, PII redaction and normal “offline” style transcription endpoints as well. In our testing we measure sub 60ms to final transcript when using the streaming modes.

Our model only emits stable words, even while streaming. Once we emit a word, we don't go back and revise it. In our own testing against the OpenASR Leaderboard tests we are seeing an overall average WER of 4.225% (official test results coming soon). We also evaluate against some internal benchmarks around real world common telephony quality and conditions and feel it does really well.

In our own agent use cases this allows us to begin executing work early as soon as something of value shows up while a user is speaking. This can often dramatically reduce voice agent latency.
It's English only right now. And it’s experimental, especially the word boosting and PII redaction - so we would love for the community to try it out, and give us some feedback.
There's a browser demo at https://labs.bandwidth.com/experiments/speech-to-text that doesn't require an account if you just wanna try it real quick, and if you sign up for a labs account you can get free access to the model while it’s on labs via API.
One important disclosure: this is a research/beta service. We monitor usage and may retain and review audio, transcripts and related data submitted to the experiment so we can evaluate the model, investigate failures and improve it. Full details are in the Labs terms.

If you build voice agents or voice Apps, or tinker in this space at all - come try it out and let us know what you think.

[u/deelight_0909]

Stable words are the interesting bit. Measure false stability, not just average WER: how often does an emitted word trigger work and later disagree with the human transcript? Split it by 8 kHz mu-law, G.722, and clean Linear16. A transcript that never revises is lovely for latency. A confidently wrong stable token just launches the wrong tool faster.