Table of Contents
1. Benchmark Methodology
Selecting the optimal Whisper model size for real-time speech translation involves balancing transcription accuracy against inference latency and system RAM/VRAM consumption. To provide empirical guidance for Voxxwire users, we conducted rigorous benchmarks across four Whisper model variants (Tiny, Base, Small, Medium) executing under faster-whisper with CTranslate2 INT8 quantization.
Test Environment Hardware:
- CPU Testbed: Intel Core i7-13700K (16 Cores, 24 Threads) @ 5.4 GHz, 32 GB DDR5 RAM.
- GPU Testbed: NVIDIA GeForce RTX 3060 (12 GB VRAM, CUDA 12.1).
- Audio Corpus: 1,000 multi-speaker conversational audio samples (English, Japanese, Spanish, German) from the LibriSpeech and Common Voice datasets.
2. Full Benchmark Comparison Table
| Model Variant | Parameters | RAM / VRAM (INT8) | CPU Latency (5s chunk) | GPU Latency (5s chunk) | Avg WER |
|---|---|---|---|---|---|
| Whisper Tiny | 39 M | ~ 150 MB | 120 ms | 35 ms | 10.4% |
| Whisper Base | 74 M | ~ 290 MB | 210 ms | 55 ms | 7.8% |
| Whisper Small | 244 M | ~ 780 MB | 480 ms | 110 ms | 5.2% |
| Whisper Medium | 769 M | ~ 2.1 GB | 1,420 ms | 260 ms | 4.1% |
3. CPU (AVX-512) vs GPU (CUDA INT8) Performance
Our benchmarks demonstrate that INT8 quantization reduces memory consumption by approximately 60% compared to standard FP32 weights, enabling even Whisper Small to operate comfortably inside 1 GB of RAM.
On modern CPUs equipped with AVX-2 or AVX-512 instructions, Whisper Base achieves a Real-Time Factor (RTF) of 0.04 (meaning 5 seconds of spoken audio is transcribed in just 210 milliseconds). On NVIDIA GPUs via CUDA, latency drops to an astonishing 55ms.
4. Word Error Rate (WER) vs Real-Time Factor (RTF)
While Whisper Medium provides superior recognition accuracy for complex medical or technical vocabulary, its CPU latency (1.4 seconds) can create slight visual delay during live subtitles. Therefore, Voxxwire recommends Whisper Base or Whisper Small as the ideal sweet spot for real-time meeting translation on typical laptop hardware.
5. Hardware Recommendation Matrix
- Entry-Level Laptops (4 GB - 8 GB RAM, i3/i5 CPU): Use
Whisper TinyorBase. - Mid-Range Desktop Workstations (16 GB RAM, i7/Ryzen 7): Use
Whisper Small(CPU mode). - Gaming / AI Workstations (NVIDIA RTX GPU): Use
Whisper SmallorMedium(CUDA mode).