NanoAvatar: talking avatars on older phones
Collection
Lifelike talking avatars on older Android phones. Fast responses, smooth video, no cloud GPU. Code, weights, demos and APK available. โข 2 items โข Updated
๐ฑ NanoAvatar
Changing how often avatar applications can be used and how many users they can serve
๐ฑ 2023 mobile chipset ยท 41 FPS ยท โก 103 ms first-frame latency
๐ฑ Android downloads (Full / Lite) ยท ๐ป Code and usage
๐ฌ Demos ยท ๐ Benchmarks ยท English ยท ็ฎไฝไธญๆ
Generate high-fidelity talking-avatar videos directly on your phone, without a cloud GPU.
๐ฌ Demos
Generalization demo: The examples below were not used to train the model, nor was it specifically fine-tuned for them.
| English demo | Chinese demo |
|---|---|
Try the on-device avatar generation model offline. The app includes the models and an avatar. Record your voice to animate it. Android guide.
๐ Performance
| Model | Device | Model FPS | First frame | Memory / VRAM estimate |
|---|---|---|---|---|
| ๐ฑ NanoAvatar | 2023 ยท Snapdragon 8 Gen 3 Android 14 |
39 FPS | 115 ms | 834 MiB |
| ๐ฑ NanoAvatar Lite | 2023 ยท Snapdragon 8 Gen 3 Android 14 |
41 FPS | 103 ms | 700 MiB |
| ๐ฑ NanoAvatar Lite | 2021 ยท Snapdragon 8 Gen 1 Android 15 |
18 FPS | 183 ms | 693 MiB |
| โก Quantized | RTX 4090 Windows CUDA |
333 FPS | 18 ms | 834 MiB |
| ๐ฅ๏ธ Full precision | RTX 4090 Windows CUDA |
224 FPS | 37 ms | 1119 MiB |
Android first-frame timing runs from audio feature extraction to the first generated image.
The model generates a 256 ร 256 face region, composited at the source video's original resolution.
๐ Streaming generation: start speaking as audio arrives. With a streaming LLM and streaming TTS, NanoAvatar starts speaking in about 0.3 seconds in our tests, without waiting for the complete audio.
๐ฆ Choose and download a model
| Model | Files | Contents |
|---|---|---|
| โก Quantized | quantized/ |
PyTorch/CUDA: HuBERT W8A16, mixed INT8 lip-sync model and CUDA DLL |
| ๐ฅ๏ธ Full precision | full-precision/๏ผcome soon๏ผ |
PyTorch: HuBERT FP16 and lip-sync model FP32 |
| ๐ฑ Android compiled | android-qnn/ |
CPU: original FP32 face/audio encoders. QNN: HuBERT W8A16 and HTP FP16 generator |
The quantized CUDA runtime targets Windows CUDA SM89 (RTX 4090). The Android package targets Snapdragon 8 Gen 3 / HTP v75.
Install the Hugging Face CLI, then download the package you need:
python -m pip install huggingface_hub
โก Quantized
hf download wpydcr/NanoAvatar --include "quantized/*" --local-dir models
๐ฅ๏ธ Full precision
hf download wpydcr/NanoAvatar --include "full-precision/*" --local-dir models
๐ฑ Android compiled
hf download wpydcr/NanoAvatar --include "android-qnn/*" --local-dir models
Keep the complete directory structure when downloading.
Running the models and integrating them: NanoAvatar on GitHub.
License
chinese-hubert-large and its quantized and compiled versions retain the upstream MIT license.
Our lip-sync model and its quantized and compiled versions use CC BY-NC 4.0. Academic research and other noncommercial uses are permitted, subject to attribution and the other license terms.
Commercial licensing: wupingyu@mail.ustc.edu.cn.
See LICENSE for the full terms.
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support