You finish a sentence, take your hands off the keyboard, and wait for the words to appear. One second feels instant. Five seconds feels like the app froze. Fifteen seconds and you have already started typing it yourself.
That wait is the whole question behind local voice typing. Everyone agrees that keeping your audio on your own machine is nice. What nobody tells you is how long you will sit there for it.
We build Meander, a voice typing app, and we recently added local models next to our cloud mode. Before we did, we wanted real numbers instead of benchmark screenshots. So we took real dictation clips from our own daily use, ran them through local speech recognition and local AI cleanup on a normal desktop, and timed everything against our cloud.
TL;DR
- On a GPU, local is as fast as the cloud or faster: about 1 second for a 50-second dictation.
- On CPU only, expect to wait 10 to 20 seconds. That is about 8 times longer than the cloud.
- Whisper large-v3-turbo (the 574 MB quantized build) was the best speech model we tested, faster and more accurate than the full large-v3.
- Small cleanup LLMs are fast, but some of them stop editing and start chatting. Gemma 4 E2B was the one we would trust.
- Without a GPU, the cloud was the fastest option by a wide margin.
How we tested
We used real dictation, not read-aloud sample sentences. Six clips from 5 to 50 seconds, mostly Mandarin and Mandarin mixed with English technical terms, because that is how our team talks. We added the classic JFK sample for clean English. We wrote out and checked by hand what was said in each clip, then scored every model against that. One English dictation clip was timed but not scored, because the speaker's accent would have made it a test of the accent, not the model.
The machine: an Intel Core Ultra 7 265K desktop. For the CPU runs we capped it at 8 threads, so the models did not get the whole chip. A thin laptop will be slower still. For the GPU runs we used an RTX 5080 through Vulkan.
The engines: whisper.cpp for speech recognition and llama.cpp for cleanup. These are the same engines Meander uses for its local mode. The cleanup step used Meander's real cleanup prompt and sampling settings, so the models got the exact job they get in the app.
The cloud: the same clips sent through Meander's normal cloud mode, from Adelaide, Australia, the same way the app sends them.
Every number below is the median of 3 runs, with the models already loaded in memory.
Step 1: speech recognition
This is the part that turns your voice into raw text.
| Model | Download | 5 s clip (CPU) | 50 s clip (CPU) | 50 s clip (GPU) | Error rate |
|---|---|---|---|---|---|
| Whisper small | 488 MB | 1.4 s | 4.0 s | 0.4 s | 17.5% |
| Whisper large-v3-turbo (q5) | 574 MB | 5.2 s | 11.8 s | 0.4 s | 4.7% |
| Whisper large-v3 | 3.1 GB | 8.1 s | 21.4 s | 1.2 s | 10.8% |
| Breeze ASR 25 (q5) | 1.1 GB | 5.7 s | 15.0 s | 1.2 s | 12.1% |
Error rate is character-level for Chinese and word-level for English, averaged across the scored clips.
Three things surprised us.
Small is fast, and not usable. It misheard short technical phrases and dropped the first half of one sentence. At 17.5% errors, you would spend more time fixing the text than you saved by talking.
The biggest model was not the most accurate. When the speaker corrected themselves mid-sentence, large-v3 tended to skip the false start and write only the corrected version. Turbo wrote down what was said.
Breeze did not win. Breeze ASR 25 is MediaTek's fine-tune of Whisper for Taiwanese Mandarin and Chinese-English code-switching, which is exactly how we speak. We expected it to lead. On our clips it landed behind turbo, and it turned a short product term into two unrelated English words.
One more thing no local model got right: product names. A sentence like "post it on the Meander blog" came back as "tomorrow's block-up" and "your broker" and a few other creative guesses. If your work is full of names, you want a custom dictionary on top, and that is where the next step helps.
Step 2: AI cleanup
Raw transcripts are messy. The cleanup step removes filler words, fixes punctuation, and repairs obvious mishearings, without rewriting what you meant. This is a second model, an LLM, and it is where local gets heavy.
| Model | 22 s clip (CPU) | 50 s clip (CPU) | 50 s clip (GPU) | RAM | How it behaved |
|---|---|---|---|---|---|
| Qwen2.5 1.5B | 4.6 s | 6.3 s | 0.5 s | 1.9 GB | Answered the user, translated to English |
| Qwen2.5 3B | 7.9 s | 10.5 s | 0.5 s | 3.6 GB | Rewrote phrases it should have kept |
| Gemma 4 E2B | 6.5 s | 9.4 s | 0.7 s | 4.4 GB | Faithful, near word-for-word |
| Gemma 4 E4B | 11.7 s | 17.0 s | 1.1 s | 7.2 GB | Good, with one stray foreign character |
| Qwen3 8B | 18.0 s | 28.0 s | 1.6 s | 8.7 GB | Good, too slow on CPU |
The smallest model had the most interesting failure. We fed it a short instruction someone had dictated, along the lines of "switch that setting to 15." A cleanup tool should return that sentence with a period at the end. Qwen2.5 1.5B replied (translated from Mandarin):
"Sure, I understand. Could you tell me exactly what you want to change?"
It treated the dictation as a request addressed to it. On another clip it quietly translated the whole Mandarin paragraph into English. Neither is a small mistake. Both put words in your document that you never said.
Gemma 4 E2B was the sweet spot. It left the speaker's words alone, fixed the punctuation, and kept the English technical terms where they were. At the other end, Qwen3 8B did a decent job and took 28 seconds on a 50-second clip. That is the model most people try first because it is "the good one," and it is the one that makes local feel broken on a machine without a GPU.
Step 3: the number you feel
Here is the full wait, from the moment you stop talking to the moment the cleaned-up text lands, using the best local pair from above (turbo + Gemma 4 E2B):
| You talk for | Cloud | Local, GPU | Local, CPU (8 threads) |
|---|---|---|---|
| 5 seconds | 1.2 s | 0.3 s | 10.4 s |
| 22 seconds | 1.6 s | 0.6 s | 12.3 s |
| 50 seconds | 2.5 s | 1.1 s | 21.2 s |
Swap Gemma 4 E2B for Qwen3 8B on CPU and the 50-second dictation takes 40 seconds.
A GPU changes everything. With one, local beats the cloud on speed, and your audio never leaves the machine. Without one, the cloud is about 8 times faster.
So which should you use?
Use local if you have a decent GPU. It is faster than the cloud, the audio stays on your computer, and it keeps working when the internet does not. Budget roughly 1 GB of memory for turbo and 4 to 5 GB for Gemma 4 E2B.
Use local if privacy is a hard requirement, even on CPU. A 10 to 20 second wait is a fair price when the audio cannot leave the building. Stick with turbo and a 2B to 3B cleanup model, and skip the 8B ones.
Use local if you have one GPU box and several computers. Run the models on the machine with the graphics card and let the laptops use it over your home network.
Use the cloud if you are on a laptop without a GPU and you mostly want your words on screen fast. It is the quickest path there, and it runs cleanup models far larger than anything that fits on a desktop.
For the full list of local speech models worth knowing about, including SenseVoice, Parakeet and Cohere Transcribe, see our local model roundup.
Why Meander does both
We ran this test because we did not want to pick one side for our users. Meander now lets each dictation mode choose where its speech recognition and cleanup run: cloud, local, or your own OpenAI-compatible endpoint, like Ollama or LM Studio. You can mix them too. Local recognition with cloud cleanup is a nice middle ground: your audio stays on your machine, and only text goes out.
Local mode runs on whisper.cpp and llama.cpp with Vulkan GPU support, on Linux and Windows, and it is part of Meander Pro. The models in this article are all in Meander's built-in model library, turbo and Gemma 4 E2B included. Ultimate adds Share on LAN, which turns the machine with the GPU into a model server for the other computers in the house.
If you want to try it, download Meander. On Linux, here is how to set it up on Wayland.