Tag
14 items · newest first

world-models

Learned simulators and predictive models used for planning and control.

#
18 Sep

PreDE predicts offline whether quantization will degrade a world action model (paper · 15 minute read)

PreDE predicts whether post-training quantization will degrade a world action model's task performance from offline action deviations, calibrating two thresholds on a development set and accepting, rejecting or deferring new configurations. Across 28 held-out configurations from two policies it issued 21 decisions before observing closed-loop outcomes, all matching the observed labels. In 450 Franka Research 3 trials, W4A4 gave a 1.37x action-query speedup and roughly 44% lower peak memory.

— cs.RO updates on arXiv.orgworld-models
#
8 Sep

AgiBot says GE-Act 2.0 keeps improving as its training data scales 100× (6 minute read)

GE-Act 2.0 trains every parameter from random initialisation on embodied manipulation data, rather than adding action prediction to a pretrained video generator. AgiBot reports that scaling the corpus from 300 to 30,000 hours lifted zero-shot success on its G1-OP evaluation from 17.1% to 44.1%, and the number of tasks with any success from 39 to 76, with towel-folding and pen-recapping appearing only at the larger scales. It says performance had not saturated at 30,000 hours.

#
2 Sep

Atlas: A World Model for Spatial Intelligence (10 minute read)

World Labs has introduced Atlas, a multimodal autoregressive diffusion transformer pretrained from scratch on text, images, video and 3D. All inputs share one spatial context, from which the model generates what comes next while staying consistent in 3D with everything it has already seen. It covers world generation, reconstruction and simulation, including camera-controlled generation from one or more images, and World Labs says performance keeps improving as training compute grows.

— worldlabs.aiworld-modelssimulation
#
28 Aug

CLAP: Cross-Embodiment Video World Models are Zero-Shot Physical Simulators (paper · 15 minute read)

CLAP trains action-conditioned video generation across both human and robot footage, reconciling incompatible action spaces through end-effector poses, language and latents. It covers DROID, Bridge, bimanual YAM robots and G1 humanoids, and all code and models are open-sourced. A curriculum first learns physical priors from unlabelled video before specialising, since action representations differ sharply between platforms and are absent from human footage altogether.

#
19 Aug

Hydra-0: Action Flow for Generalist World Modeling and Control (paper · 15 minute read)

Hydra-0 represents robot actions as pixel motion, giving one visual interface for learning action consequences across embodiments, tasks and video backbones. The best configuration cuts robot-motion error by 90.4% against an action-conditioned baseline, and an emergent inverse mode predicts robot motion from object flow taken from a human demonstration. On the RoboLab benchmark it reaches a Pearson correlation of 0.96 between replayed and reference success rates, and supports zero-shot composition.

#
14 Aug

DreamX-Phi 1.0: Action-Conditioned Video World Model for Robotic Manipulation (paper · 15 minute read)

DreamX-Phi 1.0 predicts the frames that follow a prescribed sequence of end-effector poses and gripper states, with per-arm geometric encoding so each arm follows its commanded path, and a depth branch and object masks to keep the scene consistent. It took first place on Track 1 and second on Track 2 of the WorldArena 2.0 Challenge; the model and code are to be released.

#
12 Aug

A Functional Taxonomy of World Models – By Fei-Fei Li (8 minute read)

World Labs sets out which functional pieces actually compose a world model and what each one is for, arguing the term has become one of the most overloaded in AI. The premise is that language models learn the statistical structure of text while world models must learn the structure of space and time. Computer vision, robotics, reinforcement learning and generative AI each claim to be building world models, and each means something quite different by it.

— worldlabs.aiworld-models
#
11 Aug

Flex-π: A Multi-Stream World-Action Model with Compute Flexibility (paper · 15 minute read)

Flex-π, a 6B-parameter world-action model, is supervised on 3D pointmaps and DINO semantics as well as RGB, all encoded by the same frozen video VAE, and one checkpoint runs on any subset of those streams. The authors report gains of up to 2–7x over the strongest baselines on real bimanual tasks, running faster than π0.5.

#
4 Aug

Beyond VLAs: How World Action Models Reshape Robot Manipulation (5 minute read)

NVIDIA argues that a vision-language backbone learns to describe the world rather than predict how it evolves, which is exactly what a robot needs when a task depends on anticipating change. The post sets out world action models as the replacement for that backbone. NVIDIA researcher Jim Fan set out the shift in his Robotics' End Game talk, later condensed to the line that VLAs are dead.

— Robotics – NVIDIA Technical Blogvla-foundation-modelsworld-modelsmanipulation