VideoPhy-2 banner

VideoPhy-2

A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation

ICLR 2026 · Best Paper, ICML 2025 World Models Workshop

(*Equal contribution)
1University of California, Los Angeles
2Google Research

Abstract

Large-scale video generative models, capable of creating realistic videos of diverse visual concepts, are strong candidates for general-purpose physical world simulators. However, their adherence to physical commonsense across real-world actions remains unclear (e.g., playing tennis, backflip). Existing benchmarks suffer from limitations such as limited size, lack of human evaluation, sim-to-real gaps, and absence of fine-grained physical rule analysis. To address this, we introduce VideoPhy2, an action-centric dataset for evaluating physical commonsense in generated videos. We curate 200 diverse actions and detailed prompts for video synthesis from modern generative models. We perform human evaluation that assesses semantic adherence, physical commonsense, and grounding of physical rules in the generated videos. Our findings reveal major shortcomings, with even the best model achieving only 22% joint performance (i.e., high semantic and physical commonsense adherence) on the hard subset of VideoPhy2. We find that the models particularly struggle with conservation laws like mass and momentum. Finally, we also train VideoPhy2-eval, an automatic evaluator for fast, reliable assessment on our dataset. Overall, VideoPhy2 serves as a rigorous benchmark, exposing critical gaps in video generative models and guiding future research in physically-grounded video generation.

VideoPhy-2 data and annotation pipeline VideoPhy-2 pipeline overview

VideoPhy-2 pipeline. Top: We generate a text prompt from the seed action using an LLM, create a video with a text-to-video model, and caption it with a VLM to extract candidate physical rules. Bottom: Human annotators rate the video's physical likelihood, verify rule violations, suggest missing rules, and assess semantic adherence to the input prompt.

Leaderboard on Video Generation Models

Scores are joint: the percentage of test prompts whose video both follows the prompt (SA ≥ 4) and is physically plausible (PC ≥ 4). PA and OI are the physical-activity and object-interaction categories. Last updated November 2026.

Human Leaderboard

#ModelTypeAllHardPAOI
1Wan2.1-14BOpen32.621.931.536.2
2CogVideoX-5BOpen25.00.024.626.1
3Cosmos-Diff-7BOpen24.110.922.627.4
4Sora†Closed23.35.322.226.7
5Ray2†Closed20.38.321.018.5
6Hunyuan-13BOpen17.26.217.615.9
7VideoCrafter-2Open10.52.910.113.1

AutoEval Leaderboard

VideoPhy-2-AutoEval scores we computed on the videos from our paper (per-video scores). Videos were generated from the upsampled captions; semantic adherence is scored against the original caption, matching our human evaluation. AutoEval scores are not directly comparable with human scores.

#ModelTypeAllHard
1Sora†Closed21.75.3
2Hunyuan-13BOpen19.66.2
3CogVideoX-5BOpen19.55.1
4Ray2†Closed17.55.0
5Wan2.1-14BOpen16.92.2
6VideoCrafter-2Open16.43.6
7Cosmos-Diff-7BOpen14.73.4

† Sora was evaluated on a 60-prompt subset through its UI (no API); Ray2 on 394 prompts due to API budget.

Results Reported in Other Papers

VideoPhy-2-AutoEval joint scores reported by other papers, not verified by us. Their generation and scoring setups vary (for example, which caption semantic adherence is scored against), so these numbers are not directly comparable with the tables above.

#ModelTypeAllHardReported in
1PhyEvolveAgentic30.112.8PhyEvolve
2CogVideoX-5BOpen25.95.0PhyEvolve
3Wan2.2-TI2V-5BOpen24.97.2PhyEvolve
4Co-DirectorAgentic24.95.0PhyEvolve
5Minimax-H3Closed23.94.4PhyEvolve
6Veo 3.1Closed23.7*–Physis-Lang
7Wan2.1-T2V-14BOpen23.5*–Physis-Lang
8NEWTONAgentic23.26.7PhyEvolve
9CogVideoX1.5-5BOpen23.0*–Physis-Lang
10Cosmos3-NanoOpen22.8–VeriPhy
11Cosmos-Predict2.5Open22.77.8PhyEvolve
12HunyuanVideoOpen22.56.7PhyEvolve
13LTX-Video-2.5Open22.23.9PhyEvolve
14LTX-Video-2BOpen21.44.4PhyEvolve

* The source does not state the split or metric; we assume the full-test joint score and are confirming with the authors. – means not reported.


To add your model, run VideoPhy-2-AutoEval on the test set and open a pull request adding your self-reported scores with a link to your paper or code. Questions: cipeng@g.ucla.edu.

VideoPhy-2 Benchmark

Physical Laws Violation Analysis

Auto Evaluation of Physical Commonsense, Semantic Adherence, and Rule Applicability

We use VideoCon-Physics as the base model for robust semantic adherence evaluation. We prompt the fine-tuned model to rate (1-5) the semantic adherence, physical commonsense, and rule applicability of each generated video.

Effectiveness of Our Auto-Evaluator

AutoEval correlation with human scores

Auto-rater evaluation results (Pearson correlation ×100) between the predicted scores and ground-truth scores (1-5) on the unseen prompts and unseen video models.

Effectiveness of Our Auto-Evaluator

AutoEval joint score accuracy and F1

Auto-rater evaluation on joint score judgments. We present the joint accuracy and F1 score between the predicted scores and ground-truth scores (0-1) for our VideoPhy-2-AutoEval and VideoCon-Physics.

Effectiveness of Our Auto-Evaluator

AutoEval physical rule classification accuracy

Auto-rater evaluation on physical rule classification.We present the accuracy results for VideoPhy-2-AutoEval and other video-language models on the rule classification tasks.

Citation

@inproceedings{bansal2026videophy2,
  title     = {VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation},
  author    = {Bansal, Hritik and Peng, Clark and Bitton, Yonatan and Goldenberg, Roman and Grover, Aditya and Chang, Kai-Wei},
  booktitle = {International Conference on Learning Representations (ICLR)},
  year      = {2026},
  url       = {https://arxiv.org/abs/2503.06800}
}