ZAC-1 (native πŸ€— Transformers)

ZAC-1 (Zyphra Audio Codec 1) is the 44.1 kHz speech codec of ZONOS3-Mini: 15 Hz frames of 10 codebooks of 4096 entries. Codebook 0 is semantic (distilled from a frozen Wav2Vec2-BERT 2.0 teacher), codebooks 1-9 are an acoustic residual RVQ, and a causal multi-resolution iSTFT decoder synthesizes the waveform.

This repo holds the checkpoint for the native transformers port (zac): the encoder (acoustic encoder, semantic teacher and branch, quantizers) and the decoder in one model, in float32. The main branch has the default keep2_plain decoder; the full_mr branch has the full-size decoder.

Usage

from datasets import Audio, load_dataset
from transformers import AutoFeatureExtractor, AutoModel

model_id = "Zyphra/ZAC-1-hf"
model = AutoModel.from_pretrained(model_id, device_map="auto")  # revision="full_mr" for the full-size decoder
feature_extractor = AutoFeatureExtractor.from_pretrained(model_id)

dataset = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
dataset = dataset.cast_column("audio", Audio(sampling_rate=feature_extractor.sampling_rate))
audio = dataset[0]["audio"]["array"]

inputs = feature_extractor(audio, sampling_rate=feature_extractor.sampling_rate, return_tensors="pt").to(model.device)
audio_codes = model.encode(**inputs).audio_codes  # (batch_size, 10, num_frames)
audio_values = model.decode(audio_codes).audio_values  # (batch_size, 1, num_frames * 2940)

# playback level, as ZAC-1's `output_gain()`
audio_values = audio_values * 10 ** (-3 / 20)

The encoder reproduces the original lane schedule: audio longer than 20 s is encoded in lanes of 300 frames with 150 frames of left context. In float32 the codes match the original encoder's exactly.

License

Apache-2.0. The semantic_model.* weights are the first 16 layers of facebook/w2v-bert-2.0 (MIT).

Citation

@misc{zyphra2026zonos3mini,
  title         = {{ZONOS3-Mini Technical Report}},
  author        = {Gabriel Clark and Sofian Mejjoute and George Close and Beren Millidge},
  year          = {2026},
  eprint        = {2611.16525},
  archivePrefix = {arXiv},
  primaryClass  = {cs.SD},
  url           = {https://arxiv.org/abs/2611.16525},
}
Downloads last month
57
Safetensors
Model size
2B params
Tensor type
F32
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Paper for Zyphra/ZAC-1-hf