Speech-to-text Sourced statement

faster-whisper (large-v3, CTranslate2) on NVIDIA GPU, 8 GB VRAM

Benchmark pending. We haven't measured this pair yet. Below is what the sources say about this hardware, quoted with links.

Entry gaming GPU. Fine for most small/medium speech models; tight for 3B+ models in fp16.

EvidenceSourced statement

Commercial useYes

Licensemit

Stated memory~2.9 GB VRAM (large-v2, int8, beam 5, project benchmark)

Device memory8 GB VRAM

Results

Benchmark pending. No measured numbers from us yet for faster-whisper (large-v3, CTranslate2) on 8 GB GPU. Get notified or send us your numbers (with the method).

What the sources say about 8 GB GPU

Third-party docs faster-whisper README, large-v2 on an RTX 3070 Ti 8 GB (CUDA 12.4, beam 5): fp16 1m03s / 4525 MB VRAM; int8 59s / 2926 MB; fp16 batch 8: 17s / 6090 MB. openai/whisper fp16: 2m23s / 4708 MB; whisper.cpp (flash attention) 1m05s / 4127 MB. source
Caveat: Measured with large-v2, not large-v3 (same architecture and size)

Method

We haven't run this pair. Everything on this page is quoted from the linked sources, with the source type labelled (model authors vs third parties). Vendor numbers are often measured under favourable conditions.

Ways to run faster-whisper (large-v3, CTranslate2)

Runtime	Precision	Documented devices
openai-whisper (PyTorch)	fp16/fp32	CPU, CUDA	source
faster-whisper (CTranslate2)	int8, fp16, fp32	CPU, CUDA	source
whisper.cpp (ggml)	fp16, quantized (q5/q8)	CPU (x86/ARM), Apple Metal + Core ML, CUDA, Vulkan, OpenVINO	source
Hugging Face transformers	fp16/fp32	CPU, CUDA	source

Run it

pip install faster-whisper
python -c "from faster_whisper import WhisperModel; m=WhisperModel('large-v3', device='cuda', compute_type='float16'); segs,_=m.transcribe('audio.wav', beam_size=5); print(' '.join(s.text for s in segs))"

Copied from the linked README. We haven't run it on this device.

Caveats

Measured with large-v2, not large-v3 (same architecture and size)