Instructions to use Zyphra/ZAC-1-hf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Zyphra/ZAC-1-hf with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Zyphra/ZAC-1-hf", device_map="auto") - Notebooks
- Google Colab
- Kaggle
ZAC-1 (native π€ Transformers)
ZAC-1 (Zyphra Audio Codec 1) is the 44.1 kHz speech codec of ZONOS3-Mini: 15 Hz frames of 10 codebooks of 4096 entries. Codebook 0 is semantic (distilled from a frozen Wav2Vec2-BERT 2.0 teacher), codebooks 1-9 are an acoustic residual RVQ, and a causal multi-resolution iSTFT decoder synthesizes the waveform.
This repo holds the checkpoint for the native transformers port (zac): the encoder (acoustic encoder, semantic
teacher and branch, quantizers) and the decoder in one model, in float32. The main branch has the default
keep2_plain decoder; the full_mr branch has the full-size decoder.
Usage
from datasets import Audio, load_dataset
from transformers import AutoFeatureExtractor, AutoModel
model_id = "Zyphra/ZAC-1-hf"
model = AutoModel.from_pretrained(model_id, device_map="auto") # revision="full_mr" for the full-size decoder
feature_extractor = AutoFeatureExtractor.from_pretrained(model_id)
dataset = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
dataset = dataset.cast_column("audio", Audio(sampling_rate=feature_extractor.sampling_rate))
audio = dataset[0]["audio"]["array"]
inputs = feature_extractor(audio, sampling_rate=feature_extractor.sampling_rate, return_tensors="pt").to(model.device)
audio_codes = model.encode(**inputs).audio_codes # (batch_size, 10, num_frames)
audio_values = model.decode(audio_codes).audio_values # (batch_size, 1, num_frames * 2940)
# playback level, as ZAC-1's `output_gain()`
audio_values = audio_values * 10 ** (-3 / 20)
The encoder reproduces the original lane schedule: audio longer than 20 s is encoded in lanes of 300 frames with 150 frames of left context. In float32 the codes match the original encoder's exactly.
License
Apache-2.0. The semantic_model.* weights are the first 16 layers of
facebook/w2v-bert-2.0 (MIT).
Citation
@misc{zyphra2026zonos3mini,
title = {{ZONOS3-Mini Technical Report}},
author = {Gabriel Clark and Sofian Mejjoute and George Close and Beren Millidge},
year = {2026},
eprint = {2611.16525},
archivePrefix = {arXiv},
primaryClass = {cs.SD},
url = {https://arxiv.org/abs/2611.16525},
}
- Downloads last month
- 57