Commit 路
032f161
1
Parent(s): 1adaf2a
add vpp2 pretrained video model in stage1 and stage2 (#1)
Browse files- Add VPP2 Stage-1 49-frame video checkpoint (b5f0faf66df9fc53ab52a046d5414ff949ff5248)
- Add VPP2 Stage-2 17-frame video checkpoint (a9f66dd6bc79471e27d1eb66c7abaa6a4ab255f2)
Co-authored-by: Yanjiang Guo <yjguo@users.noreply.huggingface.co>
README.md
CHANGED
|
@@ -12,7 +12,7 @@ base_model: Wan-AI/Wan2.1-I2V-14B-480P
|
|
| 12 |
|
| 13 |
# Video Prediction Policy 2: Predict Better, Act Better
|
| 14 |
|
| 15 |
-
Official VPP2 weights for RoboDojo and LIBERO.
|
| 16 |
|
| 17 |
[Paper](https://arxiv.org/abs/2610.10270) 路 [Code](https://github.com/roboterax/video-prediction-policy-2) 路 [Project page](https://robert-gyj.github.io/video-prediction-policy-2/) 路 [ModelScope](https://modelscope.cn/models/haodong123/VPP2_preview)
|
| 18 |
|
|
@@ -20,6 +20,8 @@ Official VPP2 weights for RoboDojo and LIBERO.
|
|
| 20 |
|
| 21 |
| Checkpoint | Path | Use |
|
| 22 |
|---|---|---|
|
|
|
|
|
|
|
| 23 |
| RoboDojo history-conditioned Video-10k | `checkpoints/initialization/robodojo_his10k.pt` | Joint + Action2B training initializer |
|
| 24 |
| RoboDojo joint Video-100k | `checkpoints/joint2b_s100000/video.pt` | Paired RoboDojo evaluation |
|
| 25 |
| RoboDojo Action2B-100k | `checkpoints/joint2b_s100000/action.pt` | Paired RoboDojo evaluation |
|
|
@@ -32,12 +34,34 @@ normalization statistics. Keep each pair together.
|
|
| 32 |
|
| 33 |
## TODO
|
| 34 |
|
| 35 |
-
- [ ]
|
| 36 |
|
| 37 |
## Download
|
| 38 |
|
|
|
|
|
|
|
| 39 |
```bash
|
| 40 |
python -m pip install -U huggingface_hub
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
hf download Haodong082399/VPP2 --local-dir weights
|
| 42 |
```
|
| 43 |
|
|
@@ -48,7 +72,7 @@ hf download Haodong082399/VPP2 --local-dir weights \
|
|
| 48 |
--include 'checkpoints/joint2b_s100000/*' --include 'checkpoints/Wan2.1-I2V-14B-480P/*'
|
| 49 |
```
|
| 50 |
|
| 51 |
-
The directory structure matches the ModelScope release. Follow the official
|
| 52 |
[RoboDojo guide](https://github.com/roboterax/video-prediction-policy-2/blob/main/docs/robodojo.md)
|
| 53 |
or [LIBERO guide](https://github.com/roboterax/video-prediction-policy-2/blob/main/docs/libero.md)
|
| 54 |
for installation, training and evaluation.
|
|
|
|
| 12 |
|
| 13 |
# Video Prediction Policy 2: Predict Better, Act Better
|
| 14 |
|
| 15 |
+
Official VPP2 weights for zero-shot video prediction, RoboDojo and LIBERO.
|
| 16 |
|
| 17 |
[Paper](https://arxiv.org/abs/2610.10270) 路 [Code](https://github.com/roboterax/video-prediction-policy-2) 路 [Project page](https://robert-gyj.github.io/video-prediction-policy-2/) 路 [ModelScope](https://modelscope.cn/models/haodong123/VPP2_preview)
|
| 18 |
|
|
|
|
| 20 |
|
| 21 |
| Checkpoint | Path | Use |
|
| 22 |
|---|---|---|
|
| 23 |
+
| VPP2 Stage-1 Video (49 frames) | `checkpoints_video/vpp2-video-stage1-49f.pth` | Zero-shot image-to-video prediction |
|
| 24 |
+
| VPP2 Stage-2 Video (17 frames) | `checkpoints_video/vpp2-video-stage2-17f.pth` | Zero-shot image-to-video prediction |
|
| 25 |
| RoboDojo history-conditioned Video-10k | `checkpoints/initialization/robodojo_his10k.pt` | Joint + Action2B training initializer |
|
| 26 |
| RoboDojo joint Video-100k | `checkpoints/joint2b_s100000/video.pt` | Paired RoboDojo evaluation |
|
| 27 |
| RoboDojo Action2B-100k | `checkpoints/joint2b_s100000/action.pt` | Paired RoboDojo evaluation |
|
|
|
|
| 34 |
|
| 35 |
## TODO
|
| 36 |
|
| 37 |
+
- [ ] Merge the Stage-1 and Stage-2 video checkpoint release ([PR #1](https://huggingface.co/Haodong082399/VPP2/discussions/1)).
|
| 38 |
|
| 39 |
## Download
|
| 40 |
|
| 41 |
+
Install the Hugging Face CLI:
|
| 42 |
+
|
| 43 |
```bash
|
| 44 |
python -m pip install -U huggingface_hub
|
| 45 |
+
```
|
| 46 |
+
|
| 47 |
+
The Stage-1 and Stage-2 video checkpoints are available on PR #1 while the release awaits merging:
|
| 48 |
+
|
| 49 |
+
```bash
|
| 50 |
+
hf download Haodong082399/VPP2 \
|
| 51 |
+
checkpoints_video/vpp2-video-stage1-49f.pth \
|
| 52 |
+
checkpoints_video/vpp2-video-stage2-17f.pth \
|
| 53 |
+
--revision refs/pr/1 --local-dir weights
|
| 54 |
+
```
|
| 55 |
+
|
| 56 |
+
Use 49 frames for Stage-1 or 17 frames for Stage-2. Both checkpoints use regular
|
| 57 |
+
denoising with 30 steps, CFG=4, and sigma shift 3 for standalone video prediction.
|
| 58 |
+
The shared Wan encoders and tokenizer listed above are also required. Follow the
|
| 59 |
+
[video prediction guide](https://github.com/roboterax/video-prediction-policy-2/blob/main/docs/video_prediction.md)
|
| 60 |
+
for image preparation, prompt enhancement, and inference.
|
| 61 |
+
|
| 62 |
+
To download the checkpoints on the main branch:
|
| 63 |
+
|
| 64 |
+
```bash
|
| 65 |
hf download Haodong082399/VPP2 --local-dir weights
|
| 66 |
```
|
| 67 |
|
|
|
|
| 72 |
--include 'checkpoints/joint2b_s100000/*' --include 'checkpoints/Wan2.1-I2V-14B-480P/*'
|
| 73 |
```
|
| 74 |
|
| 75 |
+
The RoboDojo and LIBERO directory structure matches the ModelScope release. Follow the official
|
| 76 |
[RoboDojo guide](https://github.com/roboterax/video-prediction-policy-2/blob/main/docs/robodojo.md)
|
| 77 |
or [LIBERO guide](https://github.com/roboterax/video-prediction-policy-2/blob/main/docs/libero.md)
|
| 78 |
for installation, training and evaluation.
|
checkpoints_video/vpp2-video-stage1-49f.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7c39effb224318c24584b07c866f465d1d6e080cf79fbc812aba5f3388e08f9f
|
| 3 |
+
size 65580046353
|
checkpoints_video/vpp2-video-stage2-17f.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d7c3efbbc8c3ebe71bbede9794410411fa624c6d5fc812a36f3496e53f780bf8
|
| 3 |
+
size 65580046353
|