Haodong082399 yjguo commited on
Commit
032f161
路
1 Parent(s): 1adaf2a

add vpp2 pretrained video model in stage1 and stage2 (#1)

Browse files

- Add VPP2 Stage-1 49-frame video checkpoint (b5f0faf66df9fc53ab52a046d5414ff949ff5248)
- Add VPP2 Stage-2 17-frame video checkpoint (a9f66dd6bc79471e27d1eb66c7abaa6a4ab255f2)


Co-authored-by: Yanjiang Guo <yjguo@users.noreply.huggingface.co>

README.md CHANGED
@@ -12,7 +12,7 @@ base_model: Wan-AI/Wan2.1-I2V-14B-480P
12
 
13
  # Video Prediction Policy 2: Predict Better, Act Better
14
 
15
- Official VPP2 weights for RoboDojo and LIBERO.
16
 
17
  [Paper](https://arxiv.org/abs/2610.10270) 路 [Code](https://github.com/roboterax/video-prediction-policy-2) 路 [Project page](https://robert-gyj.github.io/video-prediction-policy-2/) 路 [ModelScope](https://modelscope.cn/models/haodong123/VPP2_preview)
18
 
@@ -20,6 +20,8 @@ Official VPP2 weights for RoboDojo and LIBERO.
20
 
21
  | Checkpoint | Path | Use |
22
  |---|---|---|
 
 
23
  | RoboDojo history-conditioned Video-10k | `checkpoints/initialization/robodojo_his10k.pt` | Joint + Action2B training initializer |
24
  | RoboDojo joint Video-100k | `checkpoints/joint2b_s100000/video.pt` | Paired RoboDojo evaluation |
25
  | RoboDojo Action2B-100k | `checkpoints/joint2b_s100000/action.pt` | Paired RoboDojo evaluation |
@@ -32,12 +34,34 @@ normalization statistics. Keep each pair together.
32
 
33
  ## TODO
34
 
35
- - [ ] Release the robot-video pretrained backbone checkpoint.
36
 
37
  ## Download
38
 
 
 
39
  ```bash
40
  python -m pip install -U huggingface_hub
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
41
  hf download Haodong082399/VPP2 --local-dir weights
42
  ```
43
 
@@ -48,7 +72,7 @@ hf download Haodong082399/VPP2 --local-dir weights \
48
  --include 'checkpoints/joint2b_s100000/*' --include 'checkpoints/Wan2.1-I2V-14B-480P/*'
49
  ```
50
 
51
- The directory structure matches the ModelScope release. Follow the official
52
  [RoboDojo guide](https://github.com/roboterax/video-prediction-policy-2/blob/main/docs/robodojo.md)
53
  or [LIBERO guide](https://github.com/roboterax/video-prediction-policy-2/blob/main/docs/libero.md)
54
  for installation, training and evaluation.
 
12
 
13
  # Video Prediction Policy 2: Predict Better, Act Better
14
 
15
+ Official VPP2 weights for zero-shot video prediction, RoboDojo and LIBERO.
16
 
17
  [Paper](https://arxiv.org/abs/2610.10270) 路 [Code](https://github.com/roboterax/video-prediction-policy-2) 路 [Project page](https://robert-gyj.github.io/video-prediction-policy-2/) 路 [ModelScope](https://modelscope.cn/models/haodong123/VPP2_preview)
18
 
 
20
 
21
  | Checkpoint | Path | Use |
22
  |---|---|---|
23
+ | VPP2 Stage-1 Video (49 frames) | `checkpoints_video/vpp2-video-stage1-49f.pth` | Zero-shot image-to-video prediction |
24
+ | VPP2 Stage-2 Video (17 frames) | `checkpoints_video/vpp2-video-stage2-17f.pth` | Zero-shot image-to-video prediction |
25
  | RoboDojo history-conditioned Video-10k | `checkpoints/initialization/robodojo_his10k.pt` | Joint + Action2B training initializer |
26
  | RoboDojo joint Video-100k | `checkpoints/joint2b_s100000/video.pt` | Paired RoboDojo evaluation |
27
  | RoboDojo Action2B-100k | `checkpoints/joint2b_s100000/action.pt` | Paired RoboDojo evaluation |
 
34
 
35
  ## TODO
36
 
37
+ - [ ] Merge the Stage-1 and Stage-2 video checkpoint release ([PR #1](https://huggingface.co/Haodong082399/VPP2/discussions/1)).
38
 
39
  ## Download
40
 
41
+ Install the Hugging Face CLI:
42
+
43
  ```bash
44
  python -m pip install -U huggingface_hub
45
+ ```
46
+
47
+ The Stage-1 and Stage-2 video checkpoints are available on PR #1 while the release awaits merging:
48
+
49
+ ```bash
50
+ hf download Haodong082399/VPP2 \
51
+ checkpoints_video/vpp2-video-stage1-49f.pth \
52
+ checkpoints_video/vpp2-video-stage2-17f.pth \
53
+ --revision refs/pr/1 --local-dir weights
54
+ ```
55
+
56
+ Use 49 frames for Stage-1 or 17 frames for Stage-2. Both checkpoints use regular
57
+ denoising with 30 steps, CFG=4, and sigma shift 3 for standalone video prediction.
58
+ The shared Wan encoders and tokenizer listed above are also required. Follow the
59
+ [video prediction guide](https://github.com/roboterax/video-prediction-policy-2/blob/main/docs/video_prediction.md)
60
+ for image preparation, prompt enhancement, and inference.
61
+
62
+ To download the checkpoints on the main branch:
63
+
64
+ ```bash
65
  hf download Haodong082399/VPP2 --local-dir weights
66
  ```
67
 
 
72
  --include 'checkpoints/joint2b_s100000/*' --include 'checkpoints/Wan2.1-I2V-14B-480P/*'
73
  ```
74
 
75
+ The RoboDojo and LIBERO directory structure matches the ModelScope release. Follow the official
76
  [RoboDojo guide](https://github.com/roboterax/video-prediction-policy-2/blob/main/docs/robodojo.md)
77
  or [LIBERO guide](https://github.com/roboterax/video-prediction-policy-2/blob/main/docs/libero.md)
78
  for installation, training and evaluation.
checkpoints_video/vpp2-video-stage1-49f.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7c39effb224318c24584b07c866f465d1d6e080cf79fbc812aba5f3388e08f9f
3
+ size 65580046353
checkpoints_video/vpp2-video-stage2-17f.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d7c3efbbc8c3ebe71bbede9794410411fa624c6d5fc812a36f3496e53f780bf8
3
+ size 65580046353