8.5x Faster Speech-to-Text: From 429ms to 50ms on a Single GPU

My STT journey started with Whisper. One of Freya's STT models was a fine-tuned Whisper Large-V3. I optimized it with TensorRT, got it to 130ms, around 94x realtime. Solid, but we hit a wall. The architecture has a heavy encoder and a lightweight decoder, and TRT could only help with the encoder side. We needed something better. STT is the first link in our voice agent chain: STT → LLM → TTS. When…