Menu

1 / 30

Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models

NVIDIA Technical Blog·Moritz Reuss·2 days ago
#MJQ7BOJG
#developer#action#model#robot#latent
Reading 0:00
15s threshold

World-Action Models (WAMs) are emerging as a second major recipe for robot foundation models alongside VLM-based Vision-Language-Action (VLA) models, using pretrained video backbones to predict future states and actions.

Three core WAM formulations exist: inverse dynamics models that generate future video then infer actions, joint prediction models that emit video and actions together, and representation-only models that skip video generation at inference for faster execution.

Action integration approaches include default action tokens, encoding actions as visual targets inside the video denoising interface, and compressing behavior into latent plans or actions learned from video.

Architecture choices span hierarchical pipelines, monolithic transformers, and Mixture-of-Transformers (MoT) designs where modality-specific experts share attention.

DreamZero, a joint-prediction WAM built on the Wan 2.1-I2V-14B backbone, reached 1750 Elo on the April 2026 RoboArena leaderboard, outperforming Pi-0.5 at…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.