whisper-large-v3-6bit-coreml

openai/whisper-large-v3 for Apple silicon: a transcribe model package of compiled Core ML models, mostly on the Neural Engine. The transcribe-model-darwin Rust crate loads it (AnyModel::load).

Package	whisper-large-v3: format 3, architecture whisper
Source weights	https://openaipublic.azureedge.net/main/whisper/models/e5b1a55b89c1367dacf97e3e19bfd829a01529dbfdeefa8caeb59b3f1b81dadb/large-v3.pt
License	MIT (LICENSE, NOTICE)
Languages	100: af, am, ar, as, az, ba, be, bg, bn, bo, br, bs, ca, cs, cy, da, de, el, en, es, et, eu, fa, fi, fo, fr, gl, gu, ha, haw, he, hi, hr, ht, hu, hy, id, is, it, ja, jw, ka, kk, km, kn, ko, la, lb, ln, lo, lt, lv, mg, mi, mk, ml, mn, mr, ms, mt, my, ne, nl, nn, no, oc, pa, pl, ps, pt, ro, ru, sa, sd, si, sk, sl, sn, so, sq, sr, su, sv, sw, ta, te, tg, th, tk, tl, tr, tt, uk, ur, uz, vi, yi, yo, yue, zh
Size	1,110 MiB in 17 files
Requires	macOS 15.0 on Apple silicon
Built with	transcribe-models 0.0.1 at commit ff6f17639ee4d20f3724f9dfa6d99c34755dc77b, coremltools 9.0

Models

Model	Compute units	Functions	Weights
frontend	CPU	audio_160000, audio_320000, audio_480000	not compressed
encoder	CPU and Neural Engine	audio_160000, audio_320000, audio_480000	6-bit k-means palettes per tensor
decoder	CPU and Neural Engine	one	6-bit k-means palettes per tensor

Usage

Download manifest.json and every file it lists from https://huggingface.co/yorganci/whisper-large-v3-6bit-coreml/resolve/<revision>/<path>, checking each file's size and SHA-256 against the manifest, then load the directory:

use transcribe_core::{Audio, TranscribeOptions, Transcriber};
use transcribe_model_darwin::{AnyModel, LoadOptions};

let mut model = AnyModel::load("whisper-large-v3", LoadOptions::default())?;
let transcript = model.transcribe(Audio::new(&samples, 16_000), &TranscribeOptions::default())?;

The first load on a machine compiles the models for the Neural Engine (seconds to a couple of minutes); later loads are fast.

Evaluation

Error rates of this build with transcribe-eval (Apple M4; characters for Japanese, words otherwise):

Set	Error rate	Substitutions	Deletions	Insertions	Reference units
LibriSpeech test-clean, 20 utterances	1.40 %	4	2	1	500
LibriSpeech test-clean, 62 s clip, long-form with timestamps	0.70 %	1	0	0	142
FLEURS de, 10 utterances	3.79 %	5	1	2	211
FLEURS fr, 10 utterances	11.11 %	6	3	19	252
FLEURS es, 10 utterances	4.30 %	3	3	5	256
FLEURS ja, 10 utterances	3.90 %	14	3	1	462

License

whisper-large-v3

OpenAI Whisper large-v3 (https://github.com/openai/whisper), Copyright (c) 2022 OpenAI, licensed under the MIT License.

Source: https://openaipublic.azureedge.net/main/whisper/models/e5b1a55b89c1367dacf97e3e19bfd829a01529dbfdeefa8caeb59b3f1b81dadb/large-v3.pt
License: MIT (LICENSE)

Changes: this package is a modified form of the source weights. transcribe-models
(https://github.com/atahanyorganci/transcribe) ported the model to PyTorch and converted
it to Core ML with coremltools 9.0, in float16 (the front end's constants in float32),
with these weight compressions:

- frontend: not compressed
- encoder: 6-bit k-means palettes per tensor
- decoder: 6-bit k-means palettes per tensor