CRONOS: Benchmarking Counterfactual Physical Consistency in Video Models
Abstract
CRONOS is a benchmark for evaluating counterfactual physical consistency in video prediction models through controlled interventions in viewpoint, scene, object category, and appearance while maintaining fixed physical event types.
Video prediction is increasingly viewed as a path toward generalizable world models, yet it remains unclear whether these systems learn underlying causal structure or merely exploit superficial visual correlations for future prediction. We introduce CRONOS, an intervention-based benchmark designed to evaluate counterfactual physical consistency: whether a model's predictions of physical events respond appropriately to controlled changes in the visual input, such as variations of scene context, viewpoint, object appearance, and object category. Built in a photorealistic Unreal Engine environment, CRONOS enables controlled, high-fidelity generation of videos across diverse scenes and dynamics. In contrast to previous benchmarks, CRONOS systematically intervenes on four key factors - viewpoint, scene, object category, and object appearance - while keeping the underlying physical event type, such as a collision, occlusion, or fall, fixed. Our evaluation of recent open-source video generators reveals substantial failures in counterfactual physical consistency: prediction quality for the same physical event type is affected by appearance, environment, and, particularly by viewpoint changes. CRONOS provides a controlled and reproducible testbed for diagnosing how the quality of generated videos changes for different interventions, establishing a concrete target for developing models that perform consistently across changes of multiple conditions. The dataset and code are available at our project page.
Community
An intervention-based benchmark designed to evaluate counterfactual physical consistency: whether a model’s predictions of physical events respond appropriately to controlled changes in the visual input, such as variations of scene context, viewpoint, object appearance, and object category.
the full-factorial intervention design in CRONOS is where the paper shines, isolating viewpoint, scene context, object appearance, and object category while keeping the core event fixed. but i worry the object-centric metrics like appearance stability or 3d-shape stability can be steered by textures or backgrounds, so a model might pass the test by rendering plausible visuals across views rather than truly modeling the physics. i'd like to see a stricter causal test, where you perturb motion cues or disable texture signals and check if the predicted outcomes still align with physical laws under those counterfactuals. the arxivlens breakdown helped me parse the method details, and it's nice how it distills section-by-section insights (https://arxivlens.com/PaperView/Details/cronos-benchmarking-counterfactual-physical-consistency-in-video-models-2068-b784eae8). a concrete next step could be a counterfactual that swaps viewpoint but fixes lighting and texture, to see if the model truly generalizes dynamics beyond appearance.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- PhyCo: Learning Controllable Physical Priors for Generative Motion (2026)
- CCTVBench: Contrastive Consistency Traffic VideoQA Benchmark for Multimodal LLMs (2026)
- PhyWorld: Physics-Faithful World Model for Video Generation (2026)
- Quantitative Video World Model Evaluation for Geometric-Consistency (2026)
- WorldReasonBench: Human-Aligned Stress Testing of Video Generators as Future World-State Predictors (2026)
- PhysInOne: Visual Physics Learning and Reasoning in One Suite (2026)
- SymphoMotion: Joint Control of Camera Motion and Object Dynamics for Coherent Video Generation (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2605.23699 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 1
Spaces citing this paper 0
No Space linking this paper