Release WorldSonus C3 no-H0 150k inference weights
Browse files- LICENSE +18 -0
- README.md +54 -0
- THIRD_PARTY_NOTICES.md +37 -0
- audio_codec.pt +3 -0
- worldsonus_150k.pt +3 -0
- z_stats.pt +3 -0
LICENSE
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Creative Commons Attribution-NonCommercial 4.0 International
|
| 2 |
+
SPDX-License-Identifier: CC-BY-NC-4.0
|
| 3 |
+
|
| 4 |
+
Copyright (c) 2026 WorldSonus contributors
|
| 5 |
+
|
| 6 |
+
Except for third-party material identified in THIRD_PARTY_NOTICES.md,
|
| 7 |
+
WorldSonus's original contributions are licensed under the Creative Commons
|
| 8 |
+
Attribution-NonCommercial 4.0 International Public License.
|
| 9 |
+
|
| 10 |
+
The full legal terms are incorporated by reference and are available at:
|
| 11 |
+
https://creativecommons.org/licenses/by-nc/4.0/legalcode.en
|
| 12 |
+
|
| 13 |
+
You may share and adapt the licensed material for non-commercial purposes,
|
| 14 |
+
provided you give appropriate attribution, link to the license, and indicate
|
| 15 |
+
changes. The legal terms linked above govern, not this short summary.
|
| 16 |
+
|
| 17 |
+
Third-party components and source media retain their respective licenses.
|
| 18 |
+
No warranties are provided. See the legal terms for the full disclaimer.
|
README.md
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-nc-4.0
|
| 3 |
+
library_name: pytorch
|
| 4 |
+
tags:
|
| 5 |
+
- worldsonus
|
| 6 |
+
- video-to-audio
|
| 7 |
+
- audio-generation
|
| 8 |
+
- stereo
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# WorldSonus: Bringing Sound to Worlds
|
| 12 |
+
|
| 13 |
+
Inference weights for the final **C3 no-H0, 150k EMA** model.
|
| 14 |
+
|
| 15 |
+
Code and usage: [NoizAI/WorldSonus](https://github.com/NoizAI/WorldSonus).
|
| 16 |
+
|
| 17 |
+
## Contents
|
| 18 |
+
|
| 19 |
+
- `worldsonus_150k.pt`: 638 inference tensors. No optimizer, training arguments,
|
| 20 |
+
inactive history adapters, or training-only alignment projector.
|
| 21 |
+
- `audio_codec.pt`: causal stereo 48 kHz decoder, 64 latent channels at 30 Hz.
|
| 22 |
+
- `z_stats.pt`: matching latent normalization statistics. Do not substitute
|
| 23 |
+
statistics from another model or recompute them for these weights.
|
| 24 |
+
|
| 25 |
+
All files load with `torch.load(..., weights_only=True)`. DINOv3 and T5Gemma 2
|
| 26 |
+
encoders are downloaded separately under their upstream licenses.
|
| 27 |
+
|
| 28 |
+
## Usage
|
| 29 |
+
|
| 30 |
+
Install the inference repository, then run:
|
| 31 |
+
|
| 32 |
+
```bash
|
| 33 |
+
python scripts/download_model.py --repo FF2416/WorldSonus --output assets
|
| 34 |
+
python scripts/download_encoders.py --output assets
|
| 35 |
+
```
|
| 36 |
+
|
| 37 |
+
See the code repository for feature extraction and audio generation commands.
|
| 38 |
+
Defaults: Euler15, AR visual/text CFG3/3, DINO APG8, eta0, momentum0.25,
|
| 39 |
+
radius0.12421, AR noise0.1, no recache. Three latents per chunk (100 ms).
|
| 40 |
+
|
| 41 |
+
## Limitations and responsible use
|
| 42 |
+
|
| 43 |
+
Generated audio may not match every visual event and may contain artifacts.
|
| 44 |
+
Do not present generated audio as authentic recorded evidence. Respect source
|
| 45 |
+
media rights and avoid recreating identifiable voices without consent.
|
| 46 |
+
|
| 47 |
+
This is a research inference release, not a fully qualified live-streaming
|
| 48 |
+
deployment. Training code and datasets are not included.
|
| 49 |
+
|
| 50 |
+
## License
|
| 51 |
+
|
| 52 |
+
WorldSonus contributions and these released model weights are provided under
|
| 53 |
+
CC BY-NC 4.0 for non-commercial use with attribution. Upstream third-party
|
| 54 |
+
components retain their own terms; see LICENSE and THIRD_PARTY_NOTICES.md.
|
THIRD_PARTY_NOTICES.md
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Third-party notices
|
| 2 |
+
|
| 3 |
+
WorldSonus's original contributions use CC BY-NC 4.0 (see LICENSE). This does
|
| 4 |
+
not replace upstream licenses or restrict rights independently granted by them.
|
| 5 |
+
The original implementation's MIT copyright is retained below:
|
| 6 |
+
|
| 7 |
+
Copyright (c) 2026 Koichi Saito
|
| 8 |
+
|
| 9 |
+
The causal audio decoder follows the Oobleck codec implementation in
|
| 10 |
+
[stable-audio-tools](https://github.com/Stability-AI/stable-audio-tools),
|
| 11 |
+
Copyright (c) 2023 Stability AI. SnakeBeta follows
|
| 12 |
+
[BigVGAN](https://github.com/NVIDIA/BigVGAN),
|
| 13 |
+
Copyright (c) 2024 NVIDIA CORPORATION. Both are MIT-licensed under the grant below.
|
| 14 |
+
|
| 15 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 16 |
+
of this software and associated documentation files (the "Software"), to deal
|
| 17 |
+
in the Software without restriction, including without limitation the rights
|
| 18 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 19 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 20 |
+
furnished to do so, subject to the following conditions:
|
| 21 |
+
|
| 22 |
+
The above copyright notice and this permission notice shall be included in all
|
| 23 |
+
copies or substantial portions of the Software.
|
| 24 |
+
|
| 25 |
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 26 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 27 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 28 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 29 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 30 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 31 |
+
SOFTWARE.
|
| 32 |
+
|
| 33 |
+
Model weights are distributed separately. The supplied demonstration video
|
| 34 |
+
does not grant rights to third-party source footage depicted within it.
|
| 35 |
+
No datasets or cached features are distributed here.
|
| 36 |
+
DINOv3 and T5Gemma 2 assets have their own model licenses and access conditions;
|
| 37 |
+
this code license does not grant access to or relicense those assets.
|
audio_codec.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b25deff42e6e1f0c060de1d84b31e031f362c90b801475ac0598773f172d4fdd
|
| 3 |
+
size 301015270
|
worldsonus_150k.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c6c197850cacda63d98eb47cdf4123fc23f7c900e2f4babc93a8c643142bd078
|
| 3 |
+
size 4123796958
|
z_stats.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0e25f63564240a84763a7926eb55bc64b24d0e866e57c791a2959bec9757d65e
|
| 3 |
+
size 2016
|