VBench/IBench Leaderboard

"Which Video/Image Generation Model is better?"
🏆 Welcome to the leaderboard of the VBench/IBench! 🎦 A Comprehensive Benchmark Suite for Video/Image Generative Models

Our evaluation of the model's video generation capabilities is based on VBench.

VBench:

While our evaluation of the model's image generation capabilities is based on ICE-Bench.

ICE-Bench:

Taking into account both model sampling costs and evaluation accuracy, we optimized the original process as follows:

Test Set Selection: While covering all test points, we streamlined the evaluation set. This streamlining is effective, reducing sampling costs while still allowing for effective evaluation of the model's generation capabilities (with minimal deviation from the official leaderboard).
Manual Review of Results: We manually reviewed each sampled evaluation result, improving the accuracy of the evaluation and reducing illusions during the model evaluation process.
Dimension Optimization: We optimized the evaluation logic for certain dimensions, improving the corresponding accuracy.

Sampled results: The sampled videos/images for the models in the leaderboard were obtained by calling the official API and are all real-world test data. What specific details does the model sample from the video/image? See HERE

Model Name	Resolution	Duration	FPS	Total Score	Selected Score	Creativity Score	Commonsense Score	Controllability Score	Human Fidelity Score	Physics Score	Text Rendering Score	Quality Score	Human Anatomy	Human Clothes	Human Identity	Composition	Mechanics	Material	Thermotics	Multi-View Consistency	Dynamic Spatial Relationship	Dynamic Attribute	Motion Order Understanding	Human Interaction	Complex Landscape	Complex Plot	Camera Motion	Motion Rationality	Instance Preservation	OCR Score	Text Attribute	Dynamic Text	Subject Consistency	Background Consistency	Motion Smoothness	Dynamic Degree	Aesthetic Quality	Imaging Quality	Temporal Flickering
veo-3.1-generate-preview	1920×1080	5s	24	74.15%	74.15%	63.70%	88.89%	73.65%	86.20%	64.44%	63.58%	80.21%	97.30%	100.00%	72.42%	63.70%	77.78%	100.00%	100.00%	35.53%	44.44%	100.00%	88.89%	100.00%	62.22%	53.33%	100.00%	100.00%	100.00%	61.10%	64.81%	64.81%	85.40%	93.22%	98.75%	83.01%	54.12%	61.97%	97.60%

Model Name	Resolution	Duration	FPS	Total Score	Selected Score	Creativity Score	Commonsense Score	Controllability Score	Human Fidelity Score	Physics Score	Text Rendering Score	Quality Score	Human Anatomy	Human Clothes	Human Identity	Composition	Mechanics	Material	Thermotics	Multi-View Consistency	Dynamic Spatial Relationship	Dynamic Attribute	Motion Order Understanding	Human Interaction	Complex Landscape	Complex Plot	Camera Motion	Motion Rationality	Instance Preservation	OCR Score	Text Attribute	Dynamic Text	Subject Consistency	Background Consistency	Motion Smoothness	Dynamic Degree	Aesthetic Quality	Imaging Quality	Temporal Flickering
seedance-2-0-260128	1280×720	5s	24	74.15%	74.15%	63.70%	88.89%	73.65%	86.20%	64.44%	63.58%	80.21%	97.30%	88.89%	72.42%	63.70%	77.78%	88.89%	55.56%	35.53%	44.44%	88.89%	88.89%	100.00%	62.22%	53.33%	77.78%	88.89%	88.89%	61.10%	64.81%	64.81%	85.40%	93.22%	98.75%	83.01%	54.12%	61.97%	97.60%
happyhorse-1.0/video	1920×1080	5s	24	71.69%	71.69%	77.41%	94.44%	71.43%	75.53%	72.81%	25.02%	80.85%	75.29%	100.00%	51.32%	77.41%	77.78%	88.89%	77.78%	46.78%	66.67%	88.89%	77.78%	88.89%	51.11%	37.78%	88.89%	100.00%	88.89%	30.62%	35.19%	09.26%	90.44%	95.02%	98.79%	79.08%	50.66%	63.33%	97.49%
wan2.6-t2v	1920×1080	5s	30	69.17%	69.17%	73.70%	88.89%	63.49%	73.22%	68.01%	57.03%	82.13%	88.29%	88.89%	42.47%	73.70%	66.67%	88.89%	66.67%	49.80%	44.44%	88.89%	66.67%	77.78%	66.67%	22.22%	77.78%	88.89%	88.89%	56.26%	61.11%	53.70%	89.37%	94.72%	98.63%	84.31%	53.34%	68.01%	97.70%
veo-3.1-generate-preview	1280x720	6s	24	67.64%	67.64%	76.30%	88.89%	64.13%	83.04%	61.93%	29.68%	81.74%	86.32%	100.00%	62.78%	76.30%	75.00%	87.50%	55.56%	29.68%	44.44%	77.78%	88.89%	88.89%	71.11%	33.33%	44.44%	88.89%	88.89%	08.49%	49.07%	31.48%	89.81%	95.06%	99.04%	75.50%	54.83%	66.70%	97.44%
sora-2-pro	1280x720	4s	30	67.28%	67.28%	77.41%	88.89%	58.41%	87.87%	60.56%	45.86%	79.32%	89.08%	100.00%	74.51%	77.41%	66.67%	77.78%	77.78%	20.00%	44.44%	55.56%	77.78%	77.78%	40.00%	68.89%	44.44%	100.00%	77.78%	39.42%	46.30%	51.85%	89.53%	94.65%	99.01%	66.67%	50.44%	57.90%	98.49%
MiniMax-Hailuo-2.3	1366×768	6s	24	67.20%	67.20%	59.63%	88.89%	63.49%	89.16%	64.24%	20.27%	80.20%	90.10%	100.00%	77.38%	59.63%	77.78%	88.89%	55.56%	34.74%	44.44%	100.00%	66.67%	88.89%	48.89%	17.78%	77.78%	88.89%	88.89%	18.21%	24.07%	18.52%	92.60%	96.41%	99.37%	55.56%	52.17%	59.81%	98.65%
kling-video-o1	1920×1080	5s	24	66.28%	66.28%	60.74%	88.89%	64.76%	73.00%	73.80%	08.38%	81.28%	83.32%	100.00%	35.68%	60.74%	66.67%	66.67%	100.00%	61.86%	44.44%	88.89%	88.89%	88.89%	51.11%	24.44%	66.67%	88.89%	88.89%	01.99%	23.15%	00.00%	90.26%	95.49%	98.70%	81.05%	52.16%	64.51%	97.14%
wan2.5-t2v-preview	1920×1080	5s	30	65.28%	65.28%	66.67%	72.22%	60.63%	83.07%	58.51%	53.43%	82.76%	86.79%	100.00%	62.42%	66.67%	33.33%	77.78%	88.89%	34.04%	22.22%	88.89%	66.67%	100.00%	60.00%	31.11%	55.56%	77.78%	66.67%	53.80%	45.37%	61.11%	91.53%	95.32%	98.94%	77.12%	55.96%	69.74%	97.30%
seedance-1-5-pro-251215	1280×720	5s	24	65.07%	65.07%	67.04%	88.89%	68.25%	68.56%	62.21%	12.34%	81.39%	67.95%	88.89%	48.83%	67.04%	44.44%	77.78%	66.67%	59.94%	44.44%	77.78%	88.89%	88.89%	64.44%	24.44%	88.89%	88.89%	88.89%	10.17%	19.44%	07.41%	87.63%	93.66%	98.77%	92.16%	53.35%	65.39%	96.53%
MiniMax-Hailuo-02	1366×768	6s	24	63.78%	63.78%	68.15%	83.33%	59.68%	80.50%	61.20%	21.56%	81.05%	87.03%	88.89%	65.58%	68.15%	44.44%	100.00%	66.67%	33.69%	22.22%	88.89%	88.89%	77.78%	48.89%	13.33%	77.78%	88.89%	77.78%	21.16%	25.00%	18.52%	93.21%	96.53%	99.45%	61.44%	52.74%	60.16%	98.85%
veo-3.0-generate-preview	1280×720	8s	24	63.63%	63.63%	79.26%	66.67%	60.63%	78.48%	62.90%	29.67%	80.70%	80.69%	100.00%	54.76%	79.26%	77.78%	77.78%	66.67%	29.38%	44.44%	77.78%	88.89%	88.89%	44.44%	35.56%	44.44%	66.67%	66.67%	15.85%	43.52%	29.63%	89.51%	95.02%	99.24%	72.55%	52.37%	61.69%	98.13%
kling-v2-5-turbo	1280×720	5s	24	63.35%	63.35%	81.85%	83.33%	62.54%	71.57%	61.33%	11.09%	81.39%	74.11%	100.00%	40.60%	81.85%	88.89%	44.44%	55.56%	56.43%	33.33%	88.89%	77.78%	88.89%	55.56%	37.78%	55.56%	77.78%	88.89%	03.64%	25.93%	03.70%	90.30%	94.98%	98.97%	77.12%	53.18%	65.26%	97.35%
viduq2	1920×1080	5s	24	62.95%	62.95%	79.26%	72.22%	66.15%	74.03%	54.87%	18.06%	81.07%	71.47%	88.89%	61.75%	79.26%	25.00%	66.67%	77.78%	50.03%	77.78%	77.78%	87.50%	88.89%	55.56%	20.00%	55.56%	55.56%	88.89%	23.63%	25.00%	05.56%	89.39%	94.61%	98.04%	80.79%	55.25%	67.99%	95.88%
pixverse-v5.5	1280×720	5s	24	61.88%	61.88%	70.00%	88.89%	54.60%	79.84%	54.89%	30.14%	82.86%	86.97%	100.00%	52.53%	70.00%	33.33%	77.78%	66.67%	41.77%	33.33%	77.78%	55.56%	100.00%	46.67%	13.33%	55.56%	100.00%	77.78%	54.32%	30.56%	05.56%	91.60%	95.27%	99.02%	81.05%	55.44%	68.72%	97.29%
kling-v2-6	1920×1080	5s	24	61.63%	61.63%	52.96%	83.33%	63.81%	76.11%	55.61%	09.08%	80.75%	89.85%	88.89%	49.59%	52.96%	55.56%	55.56%	44.44%	66.89%	33.33%	77.78%	66.67%	77.78%	57.78%	33.33%	100.00%	88.89%	77.78%	04.10%	19.44%	03.70%	89.53%	94.91%	98.75%	76.47%	52.10%	64.58%	97.23%
sora-2	1280x720	4s	30	59.76%	59.76%	84.81%	83.33%	42.54%	88.75%	48.74%	56.99%	78.83%	94.02%	88.89%	83.35%	84.81%	44.44%	77.78%	44.44%	28.28%	11.11%	77.78%	11.11%	77.78%	28.89%	46.67%	44.44%	88.89%	77.78%	49.68%	62.04%	59.26%	90.04%	95.16%	99.17%	54.90%	50.60%	57.67%	98.83%
pixverse-v5	1280×720	5s	24	59.72%	59.72%	70.37%	83.33%	49.21%	83.59%	53.49%	31.19%	83.30%	85.87%	100.00%	64.91%	70.37%	33.33%	55.56%	77.78%	47.31%	33.33%	66.67%	44.44%	100.00%	46.67%	08.89%	44.44%	77.78%	88.89%	46.35%	36.11%	11.11%	92.07%	95.46%	99.05%	84.97%	55.14%	68.76%	97.38%
seedance-1-0-pro-250528	1920×1088	5s	24	56.14%	56.14%	53.33%	88.89%	55.24%	66.47%	47.67%	11.40%	83.43%	66.73%	88.89%	43.79%	53.33%	44.44%	55.56%	44.44%	46.22%	44.44%	66.67%	33.33%	88.89%	44.44%	20.00%	88.89%	88.89%	88.89%	16.60%	17.59%	00.00%	93.90%	96.21%	98.95%	83.01%	52.34%	69.50%	97.77%
wan2.2-t2v-plus	1920×1080	5s	30	52.39%	52.39%	60.00%	88.89%	39.68%	80.84%	45.34%	09.83%	82.53%	74.63%	100.00%	67.87%	60.00%	44.44%	55.56%	44.44%	36.91%	11.11%	44.44%	33.33%	88.89%	40.00%	04.44%	55.56%	77.78%	100.00%	16.51%	12.96%	00.00%	94.23%	96.84%	99.20%	58.82%	57.11%	67.25%	98.38%
kling-v2-1-master	1920×1080	5s	24	49.85%	49.85%	63.70%	77.78%	43.49%	65.67%	44.75%	04.27%	82.01%	76.87%	77.78%	42.36%	63.70%	11.11%	66.67%	55.56%	45.66%	55.56%	55.56%	11.11%	88.89%	17.78%	08.89%	66.67%	66.67%	88.89%	03.56%	09.26%	00.00%	92.30%	96.09%	99.15%	70.59%	53.71%	66.16%	97.89%
viduq1	1280×720	5s	24	42.12%	42.12%	54.78%	38.89%	40.63%	66.48%	34.36%	04.10%	80.28%	67.66%	88.89%	42.91%	54.78%	33.33%	11.11%	22.22%	70.78%	00.00%	44.44%	55.56%	88.89%	33.33%	06.67%	55.56%	22.22%	55.56%	05.83%	06.48%	00.00%	89.52%	94.19%	98.35%	81.05%	51.81%	63.83%	96.52%

Model Name	Resolution	Duration	FPS	Total Score	I2V Score	Quality Score	Selected Score	Video-Image Subject Consistency	Video-Image Background Consistency	Subject Consistency	Background Consistency	Motion Smoothness	Dynamic Degree	Aesthetic Quality	Imaging Quality	Temporal Flickering
kling-v2-1-master	1920×1080	5s	24	91.41%	98.91%	83.92%	91.41%	98.51%	99.67%	94.29%	98.60%	98.84%	68.00%	61.79%	69.22%	97.09%
veo-3.1-generate-preview	1280x720	6s	24	91.28%	99.03%	83.52%	91.28%	98.75%	99.65%	94.97%	98.83%	99.08%	56.00%	60.99%	68.83%	98.09%
kling-v2-5-turbo	1280×720	5s	24	91.23%	99.11%	83.36%	91.23%	98.86%	99.67%	95.25%	98.88%	98.94%	53.06%	62.11%	68.64%	97.93%
kling-v2-6	1920×1080	5s	24	91.20%	99.06%	83.33%	91.20%	98.65%	99.79%	95.51%	98.96%	98.76%	54.00%	62.00%	68.68%	97.80%
kling-video-o1	1920×1080	5s	24	91.04%	98.94%	83.13%	91.04%	98.50%	99.74%	95.21%	99.00%	98.69%	53.06%	61.74%	68.73%	97.74%
veo-3.0-generate-preview	1280×720	8s	24	91.03%	98.85%	83.21%	91.03%	98.65%	99.47%	94.90%	98.57%	99.01%	54.17%	60.55%	69.58%	97.81%
wan2.2-i2v-plus	1920×1080	5s	30	91.01%	99.01%	83.02%	91.01%	98.72%	99.63%	94.50%	98.54%	99.01%	54.00%	61.07%	67.79%	98.04%
pixverse-v5	1280×720	5s	24	90.96%	98.19%	83.74%	90.96%	97.88%	99.15%	93.14%	97.93%	98.59%	76.00%	60.42%	69.25%	96.83%
seedance-1-0-pro-250528	1920×1088	5s	24	90.71%	98.10%	83.32%	90.71%	97.98%	98.94%	94.00%	97.56%	98.76%	68.00%	59.31%	68.94%	97.44%
viduq1	1280×720	5s	24	90.70%	98.98%	82.42%	90.70%	98.90%	99.45%	96.82%	98.75%	99.38%	34.00%	60.42%	68.68%	98.63%
seedance-1-5-pro-251215	1280×720	5s	24	90.68%	97.88%	83.48%	90.68%	97.90%	98.69%	92.94%	96.64%	98.70%	80.00%	59.40%	69.11%	96.49%
MiniMax-Hailuo-02	1366×768	6s	24	90.67%	98.17%	83.17%	90.67%	97.79%	99.21%	93.11%	98.23%	98.82%	60.00%	60.42%	69.05%	98.07%
MiniMax-Hailuo-2.3	1366×768	6s	24	90.58%	98.40%	82.76%	90.58%	98.01%	99.36%	93.18%	98.25%	98.89%	54.00%	60.38%	68.94%	98.13%
wan2.6-i2v	1920×1080	5s	30	90.31%	96.89%	83.72%	90.31%	97.17%	97.86%	94.54%	97.47%	98.24%	74.00%	59.49%	70.57%	97.09%
wan2.5-i2v-preview	1920×1080	5s	30	90.24%	97.80%	82.67%	90.24%	97.85%	98.61%	94.75%	97.61%	98.66%	60.00%	59.97%	69.69%	96.59%

Model Name	Resolution	Total Score	Selected Score	Aesthetic	Image Quality	Prompt Semantic Alignment
Nano Banana Pro	1024x1024	79.51%	79.51%	60.44%	71.11%	93.25%
gpt-image-1.5	auto	79.22%	79.22%	60.14%	71.06%	92.83%
doubao-seedream-4-5-251128	2048x2048	78.67%	78.67%	61.51%	67.51%	92.82%
Nano Banana	1024x1024	78.40%	78.40%	62.13%	70.31%	90.59%
wan2.6-t2i	1280x1280	78.20%	78.20%	61.96%	69.87%	90.48%
doubao-seedream-4-0-250828	2048x2048	78.07%	78.07%	61.80%	69.24%	90.63%
imagen-4	1024x1024	77.96%	77.96%	61.34%	71.82%	89.34%
gpt-4o	auto	76.84%	76.84%	58.23%	69.86%	89.63%
wan2.5-t2i	1280x1280	76.62%	76.62%	60.79%	69.83%	87.93%
doubao-seedream-3-0-t2i-250415	2048x2048	75.27%	75.27%	59.97%	68.96%	86.06%
midjourney	auto	69.35%	69.35%	62.13%	70.30%	72.49%
wan2.6-image	1280x1280	68.81%	68.81%	60.67%	69.22%	72.68%

Model Name	Total Score	Selected Score	Image Colorize	Image Deblur	Pose-guided Generation	Depth-guided Generation	Edge-guided Generation	Color Editing	Face Editing	Motion Editing	Texture Editing	Subject Addition	Subject Removal	Subject Change	Style Editing	Scene Editing	Text Render	Text Removal	Composite Editing	Inpainting	Outpainting	Local Subject Addition	Local Subject Removal	Local Text Render	Local Text Removal	Face Reference Generation	Subject Reference Generation	Style Reference Generation	Virtual Try On	Face Swap	Subject-guided Inpainting	Style Reference Editing
doubao-seedream-4-5-251128	71.67%	71.67%	67.89%	58.55%	55.08%	70.21%	50.71%	81.04%	77.76%	80.44%	80.06%	78.89%	77.67%	76.27%	74.89%	76.49%	76.30%	78.42%	76.03%	76.49%	73.44%	77.15%	72.91%	77.31%	79.67%	66.42%	66.24%	61.38%	69.37%	66.49%	65.60%	61.04%
Nano Banana Pro	71.34%	71.34%	67.31%	62.23%	52.81%	70.53%	51.39%	81.28%	76.40%	80.57%	79.50%	78.91%	77.26%	77.10%	73.48%	75.96%	75.32%	77.73%	75.69%	76.56%	73.82%	76.36%	71.99%	76.33%	78.73%	60.36%	66.12%	62.44%	69.02%	65.39%	64.73%	64.96%
gpt-image-1.5	70.99%	70.99%	68.03%	58.96%	51.20%	70.57%	50.59%	81.03%	76.97%	80.08%	79.49%	78.87%	76.91%	76.32%	72.40%	76.08%	76.79%	76.47%	75.13%	75.65%	73.11%	76.09%	71.63%	76.55%	78.48%	63.23%	65.80%	61.18%	68.78%	64.50%	66.18%	62.53%
wan2.6	70.69%	70.69%	68.07%	61.40%	51.81%	69.39%	50.57%	81.24%	77.92%	80.21%	78.97%	79.09%	77.52%	76.41%	72.34%	74.82%	75.88%	77.28%	75.62%	76.74%	71.68%	77.40%	71.07%	77.06%	79.51%	56.38%	65.68%	59.39%	69.21%	62.21%	64.45%	61.36%
Nano Banana	70.07%	70.07%	67.54%	46.20%	51.39%	69.24%	50.21%	81.30%	76.84%	80.20%	78.04%	79.26%	76.81%	75.92%	72.33%	76.53%	76.05%	77.85%	76.96%	75.81%	72.12%	76.44%	73.69%	76.80%	78.85%	61.43%	64.57%	61.68%	68.36%	56.75%	64.40%	58.53%
doubao-seedream-4-0-250828	69.99%	69.99%	64.88%	48.81%	51.40%	68.13%	47.86%	80.07%	76.36%	79.74%	78.61%	78.56%	76.14%	75.57%	73.38%	75.03%	76.85%	76.91%	74.49%	75.42%	72.91%	75.68%	72.20%	76.39%	78.20%	65.18%	66.94%	59.94%	69.50%	61.98%	63.65%	59.03%
wan2.5	69.48%	69.48%	67.21%	57.32%	50.51%	70.56%	49.89%	80.79%	75.45%	78.14%	78.03%	78.56%	77.33%	75.53%	70.65%	73.86%	75.98%	78.54%	74.15%	75.39%	70.56%	76.91%	72.62%	77.10%	79.20%	54.27%	64.97%	54.91%	69.13%	53.98%	64.86%	58.01%

VBench/IBench Leaderboard

About this leaderboard

Tab 1 — Text to Video (VBench 2.0)

Summary scores

Dimension definitions (shown as columns)

Tab 2 — Image to Video

Dimension definitions (shown as columns)

Notes on scores displayed in the table

Tab 3 — Text to Image

Dimension definitions (shown as columns)

Subject filtering

Tab 4 — Image to Image

Task categories (buttons)

Sub-dimension (task) definitions (shown as checkboxes)

Controllable Generation

Global Editing

Local Editing

Reference Generation

Reference Editing

Interpreting the table