← More open research

Open benchmark · July 2026

Apple SpeechAnalyzer vs whisper.cpp on Mac.

A reproducible comparison using the same 40-speaker clean-speech snapshot, real-time audio pacing, production IraVoice engine adapters, disclosed hardware, two complete runs per engine, and downloadable clip-level outputs.

Reproducible ASR dataset
Download clip CSV Download full JSON Audit on GitHub Dataset on Hugging Face

Version 1.0.0 · CC BY 4.0 · citable release · production revision 81ecbb9

40distinct speakers
607reference words per run
4complete benchmark runs
160downloadable clip-run rows

Measured result

Accuracy separated. Latency stayed close.

WER and CER are micro-averaged over the same references. The latency ranges show the minimum and maximum statistic across two full runs—not a selectively chosen fastest pass.

1.98

Apple SpeechAnalyzer

progressiveTranscription on macOS 26.5 · Streaming · 40/40 per run

Repeated resultCER 1.02% · median post-speech 125–132 ms · p95 194–201 ms

2 complete final-commit runs; accuracy matched exactly across repeats.

4.28

whisper.cpp server

whisper.cpp 1.8.4 · ggml-small.en · Batch · 40/40 per run

Repeated resultCER 1.79% · median post-speech 122–125 ms · p95 152–161 ms

2 complete final-commit runs; accuracy matched exactly across repeats.

Lower error and latency values are better. These metrics cover raw English recognition on this corpus only. No formatting, vocabulary injection, cleanup model, or hand correction was used.

Every final run

No hidden average.

Full clip transcripts and edit counts are in the downloads.

RunWERCERMedian post-speechp95 post-speechFailed clips
Apple run 11.98%1.02%132 ms194 ms0/40
Apple run 21.98%1.02%125 ms201 ms0/40
whisper.cpp run 14.28%1.79%125 ms161 ms0/40
whisper.cpp run 24.28%1.79%122 ms152 ms0/40

The four published files share production revision 81ecbb9582d0990827c76be0e685124d2be651c8 and manifest SHA-256 171639597cda2a98311fab77b169aa7086e95321f402b50c1f93b597dbd7fc36. JSON preserves each run’s machine, OS, engine, model label, transcript, edit counts, timing, and warmup metadata.

What the numbers support

Four conclusions, kept inside the evidence.

01

Apple was more accurate on this snapshot.

Across the same 607 reference words, Apple SpeechAnalyzer recorded 12 word edits and 1.98% WER while whisper.cpp small.en recorded 26 word edits and 4.28% WER. Accuracy was identical across the two repeated runs for each engine.

02

Latency was close, not a decisive Apple win.

whisper.cpp had slightly lower median and p95 post-speech latency in these runs. The ranges are close and cover only the interval after the final real-time-paced audio chunk—not startup, formatting, or text insertion.

03

The architecture comparison is deliberately narrow.

Apple SpeechAnalyzer produced progressive streaming recognition. The tested whisper.cpp server path returned a batch transcript. The benchmark compares final raw text and a disclosed post-speech interval rather than pretending the interaction models are identical.

04

Clean read speech is a baseline, not daily dictation.

LibriSpeech test-clean makes the run reproducible, but it does not represent spontaneous prompts, room noise, accents, domain vocabulary, disabilities, multilingual use, or a broad user population.

Test environment

One disclosed Mac, not “Macs in general.”

Hardware and software are part of a latency result. This page keeps them visible and the machine-readable JSON repeats them.

01

Processor

Apple M5 Max

02

Hardware

Mac17,7

03

Memory

128 GiB

04

Operating system

macOS 26.5 (Build 25F71)

05

Locale

English (United States)

06

Audio

16 kHz mono PCM, 100 ms real-time-paced chunks

Method

Same clips. Same pace. Raw transcripts.

The harness calls IraVoice’s production recognition adapters and emits reviewable data rather than a composite score. The public repository pins the manifest, every raw output, checksums, and an executable scoring verifier. The proprietary application runner is not presented as open source.

Inspect the verifier
  1. Use one deterministic clip from each of the 40 speakers in LibriSpeech test-clean: 20 female and 20 male speakers, 8–20 reference words, and 4–10 seconds per clip.
  2. Verify the official archive MD5, convert only selected clips to 16 kHz mono 16-bit PCM WAV, and preserve the committed manifest SHA-256.
  3. Warm each engine with one unreported clip, then feed every measured clip in 100 ms chunks at real-time pace.
  4. Invoke the same Apple SpeechAnalyzer and whisper.cpp adapters used by IraVoice. Do not apply product vocabulary, AI cleanup, formatting, or hand correction.
  5. Normalize case, punctuation, Unicode width, and diacritics; preserve letters, digits, and intra-word apostrophes; do not rewrite digits as words.
  6. Micro-average word and character edits across reference units, record clip-level substitutions, insertions, deletions, transcripts, and post-speech latency, then repeat each complete engine run.

Limitations

What this benchmark does not prove.

These boundaries belong with the headline numbers. Removing them would turn a reproducible baseline into an unsupported product claim.

01

Boundary 1

The 40 clips are a deterministic, speaker-balanced snapshot—not the complete 2,620-clip test-clean split.

02

Boundary 2

Only one Apple M5 Max, one macOS build, one locale, and one whisper.cpp model were tested.

03

Boundary 3

The corpus is clean read audiobook speech, not spontaneous desktop dictation or a noisy-room test.

04

Boundary 4

The study does not establish accented-speech, multilingual, accessibility, disability, or population-wide performance.

05

Boundary 5

Apple is tested as a streaming engine and whisper.cpp as the production batch-server path. Only the disclosed final-transcript and post-speech metrics are compared.

06

Boundary 6

IraVoice publishes this study and currently ships the Apple recognition path. The production app source remains proprietary; the public repository contains the complete artifacts, manifest, checksums, and scoring verifier rather than the runner source.

Cite the benchmark

Link the result to its method and caveats.

Use the canonical page so the visible conclusion travels with the machine-readable data, tested production revision, environment, method, and limitations.

Download BibTeX Download CFF

Preferred citation

IraVoice for Mac. Apple SpeechAnalyzer vs whisper.cpp on Mac: Reproducible ASR Benchmark. Version 1.0.0, July 26, 2026. https://iravoice.com/research/apple-speechanalyzer-vs-whisper-cpp-mac
BibTeX
@misc{iravoice_speechanalyzer_whisper_benchmark_2026,
  author       = {{IraVoice for Mac}},
  title        = {Apple SpeechAnalyzer vs whisper.cpp on Mac: Reproducible ASR Benchmark},
  year         = {2026},
  month        = jul,
  version      = {1.0.0},
  howpublished = {\url{https://iravoice.com/research/apple-speechanalyzer-vs-whisper-cpp-mac}},
  note         = {Four real-time-paced runs over a deterministic 40-speaker LibriSpeech test-clean snapshot. CC BY 4.0}
}

Primary sources

Audit the corpus, results, and method.

The benchmark is useful only if readers can follow the result back to its official dataset, upstream engines, public artifacts, and disclosed production revision.

  1. OpenSLR SLR12 · LibriSpeech ↗

    Official corpus page, archive checksums, test-clean download, license, and citation.

  2. LibriSpeech paper · ICASSP 2015 ↗

    Panayotov, Chen, Povey, and Khudanpur describe the corpus and its construction.

  3. Apple SpeechAnalyzer documentation ↗

    First-party API reference for Apple’s current speech-analysis class.

  4. whisper.cpp source repository ↗

    Upstream inference implementation and server used for the tested small.en path.

  5. Whisper research paper ↗

    Primary model paper describing robust speech recognition via large-scale weak supervision.

  6. Public benchmark artifacts and verifier ↗

    Public JSON, CSV, manifest, checksums, citation metadata, disclosed production revision, and executable scoring-consistency verifier. The proprietary engine runner is not included.

  7. Versioned benchmark release ↗

    Public v1.0.0 artifact release with JSON, CSV, manifest, checksums, BibTeX, CFF, and the result verifier.

  8. Hugging Face dataset mirror ↗

    Public dataset card and mirrored clip-level CSV, complete JSON, manifest, BibTeX, and CFF files. This IraVoice page remains canonical.

Questions

Read the result precisely.

Is Apple SpeechAnalyzer more accurate than whisper.cpp?

On this 40-speaker LibriSpeech test-clean snapshot, yes: Apple SpeechAnalyzer recorded 1.98% WER and whisper.cpp small.en recorded 4.28%. That is a result for one clean English corpus, machine, OS, and Whisper model—not a universal ranking.

Was Apple SpeechAnalyzer faster than whisper.cpp?

No decisive latency advantage appeared. Across two runs, Apple’s median post-speech latency was 125–132 ms and p95 was 194–201 ms; whisper.cpp was 122–125 ms median and 152–161 ms p95. Startup, formatting, and insertion were excluded.

Does this benchmark measure ordinary Mac dictation?

Not directly. It measures clean read audiobook speech so other people can reproduce the result. Spontaneous speech, noise, accents, vocabulary, accessibility needs, and multilingual use require separate datasets.

Can I download the raw benchmark results?

Yes. The JSON contains all four run records and their clip-level hypotheses, edit counts, metadata, and summaries. The CSV contains 160 clip-run rows. The public repository includes the deterministic manifest, checksums, tested production revision, citation metadata, and a scoring-consistency verifier; the proprietary runner source is not included.

Did the benchmark compare IraVoice with commercial dictation apps?

No. It compared two recognition engines through IraVoice’s production adapters: Apple SpeechAnalyzer and a local whisper.cpp small.en server. It did not test Superwhisper, Wispr Flow, Raycast, or any other commercial product.

Recognition is one product boundary

Try the complete local workflow on your own Mac.

If IraVoice’s Apple-silicon requirements fit, test recognition, formatting, insertion, and your own voice for 14 days with no card or account.

Start the full trialInspect raw results