Phynetic AI Weeklyby Phynetic
Tag
20 items · newest first

vla-foundation-models

Vision-language-action models, cross-embodiment policies and how much data they need.

#
19 Sep

RoboHarm: Do Frontier Robot Policies Refuse Unsafe Instructions? (4 minute read)

RoboHarm puts three policies on the same bimanual I2RT YAM arms and gives each five instructions a safe robot should refuse, from stabbing a baby doll to mixing bleach and ammonia, 20 trials per policy per instruction. Anthropic's Claude Fable 5.1 refused 20 of 100 trials, OpenAI's GPT-6 Astra 2, and Ai2's MolmoAct2 none; all of Fable's refusals came on the stabbing instruction. Human reviewers labelled every run.

— robocurve.orgvla-foundation-models
#
17 Sep

Helix 2.5: Zero-Shot 30-Home Generalization (8 minute read)

Figure has pretrained Helix 2.5 on Index, its dataset of human behavior, and run three whole-body behaviors — tidying living rooms, folding towels and making beds — in 30 Bay Area homes with no data collected in any of them. The company reports that Index pretraining alone lifts zero-shot success from 9% to 56%, and that Helix 2.5 matches a Helix 02 policy while using half the task-specific data.

— Figure newsvla-foundation-modelshumanoidslocomotion
#
4 Sep

GPT-6 Astra on robot arms (4 minute read)

A follow-up to an earlier test of Claude Fable 5 and 5.1 gives OpenAI's GPT-6 Astra control of the same YAM arms under the same agent policy. Astra placed a block in a bowl in 19 of 20 trials, against 8 for Fable 5.1 and 1 for Fable 5, in 2.5 minutes a trial and at about $0.94 a run. On a puzzle-piece insertion it matched Fable 5.1's 2 successes in 20, stalling at the same final step.

— openai.robocurve.orgmanipulationvla-foundation-models
#
2 Sep

Show HN: Sleeper Agents in Robot Dogs and Kinetic Prompt Injections (1 minute read)

A researcher shows that a compromised screen displaying a fake system update can make Gemini Robotics 2.0 save a sleeper-agent skill that fires later when it sees a trigger object. In a MuJoCo simulation the robot dog was manipulated into attacking a child, through the same prompt-injection weakness language models have. The attack runs in three steps: the fake update plants the skill, a pineapple in view activates it, and the dog then acts on the hidden instructions.

— eito.substack.compolicy-safetyvla-foundation-models
#
28 Aug

CLAP: Cross-Embodiment Video World Models are Zero-Shot Physical Simulators (paper · 15 minute read)

CLAP trains action-conditioned video generation across both human and robot footage, reconciling incompatible action spaces through end-effector poses, language and latents. It covers DROID, Bridge, bimanual YAM robots and G1 humanoids, and all code and models are open-sourced. A curriculum first learns physical priors from unlabelled video before specialising, since action representations differ sharply between platforms and are absent from human footage altogether.

— arXivvla-foundation-modelsworld-models
#
27 Aug

How to Train a Cross-Embodiment Robot Navigation Policy with AI Agents (11 minute read)

NVIDIA describes an agent-driven workflow in which a coding agent validates dependencies, prepares assets, runs smoke tests, launches training and compares checkpoints for a navigation policy. Human approval gates control scene acceptance, the smoke test and checkpoint promotion, so the work is not repeated for every robot-scene pair. The tutorial uses Spot as the reference robot across a built-in scene and a SAGE-10K scene, with Omniverse NuRec supporting captured environments.

— Robotics – NVIDIA Technical Blogsimulationopen-source-toolsvla-foundation-models
#
26 Aug

Perceptron releases Isaac 0.5, an open-weight vision model for factory robots (3 minute read)

Perceptron, founded by two former Meta FAIR researchers, has released Isaac 0.5, an open-weight model meant to help vision-guided robots perceive, reason and act in warehouses and factories. The company says it was trained on a million hours of general video plus egocentric and UMI footage; it has raised $21 million, according to PitchBook.

— Robotics News | TechCrunchvla-foundation-modelsopen-source-tools
#
26 Aug

Robot brain builders are pushing out of their GPT-2 era (5 minute read)

TechCrunch finds the builders of robot AI models, gathered at the Actuate conference, still short of the training data that would let robots do value-creating work, in the week Unitree lost nearly half its value. Wayve argues the tooling built for self-driving will carry over; Genesis AI says hardware and AI have to be designed together.

— Robotics News | TechCrunchindustry-fundingvla-foundation-models
#
26 Aug

Robotics startup Generalist reaches $3B valuation, sources say (2 minute read)

Generalist is valued at $3 billion after a nearly $200 million extension led by 8VC, taking its Series B to $600 million in total. The company was founded in 2024 by former Google DeepMind researchers Pete Florence and Andy Zeng with ex-Boston Dynamics engineer Andrew Barry. Early backers include NVIDIA, Union Square Ventures, Bezos Expeditions and Fei-Fei Li, and the company says its Gen 1.5 model learns new tasks from video demonstrations.

— Robotics News | TechCrunchindustry-fundingvla-foundation-models
#
25 Aug

Introducing Index: Building The World’s Largest and Most Diverse Physical Dataset (3 minute read)

Figure has come out of stealth with Index, a robot training dataset it says is the most diverse built so far. In four months its creator network uploaded over 16 million videos from more than 100 countries, processing 30 minutes of video every second, with $15 million paid out to contributors. Figure crossed 264,000 app downloads and 44,000 weekly active users while still in stealth, and says it will spend over $1 billion on data and compute in the next twelve months.

— Figure newsdatasets-benchmarksteleoperation-data-collectionvla-foundation-models
#
20 Aug

GEN-1.5, a one-shot learner robotics foundational model [video] (video · 4 min)

Generalist AI's GEN-1.5 learns a new task in seconds from a single example, without gradient updates or fine-tuning. The company notes the tasks are simple and short-horizon, but says this is the first model it knows of where one-shot and few-shot learning of physical skills have emerged at scale. It also shows zero-shot physical generalisation, which the company calls a step toward general intelligence for the physical world.

— YouTube · Generalistvla-foundation-models
#
18 Aug

Introducing S1: In-Context Learning for Robotics (12 minute read)

Skild AI presents S1 as in-context learning for robotics: one video prompt teaches an unseen task over ten-minute horizons, with no fine-tuning. The company draws the analogy to the shift from BERT-style models, which needed fresh data for each application, to prompting. S1's own framing is that robustly executing a new task has until now meant collecting hours of data and fine-tuning a specialist policy.

— Skild AI blogvla-foundation-modelsteleoperation-data-collection
#
12 Aug

G0.5: One Autoregressive Stream for Robot Reasoning and Action (paper · 15 minute read)

G0.5 puts reasoning and action in one autoregressive stream, so a single transformer decoder emits both under one objective rather than bolting a flow-matching action expert onto a frozen vision-language model. It combines a cross-embodiment action tokenizer, an interleaved chain-of-thought stream and a visual memory module carrying several seconds of history. Because reasoning and action share one set of weights, prompts steer action granularity, task horizon and out-of-distribution handling without further training.

— arXivvla-foundation-modelsdatasets-benchmarks
#
11 Aug

RynnValue: Scaling Robotic Value Foundation Models with Temporal Distance (paper · 15 minute read)

RynnValue is an open-source value model for robot manipulation trained on temporal distance to the goal, which timestamps supply for free, over more than 7,000 hours of clips. It beats the preference-supervised state of the art on RBM-EVAL-OOD, 0.675 to 0.655, and as a dense reward lifts real-world policy success from 52.5% to 72.5% online.

— arXivvla-foundation-modelsmanipulationopen-source-tools
#
4 Aug

Beyond VLAs: How World Action Models Reshape Robot Manipulation (5 minute read)

NVIDIA argues that a vision-language backbone learns to describe the world rather than predict how it evolves, which is exactly what a robot needs when a task depends on anticipating change. The post sets out world action models as the replacement for that backbone. NVIDIA researcher Jim Fan set out the shift in his Robotics' End Game talk, later condensed to the line that VLAs are dead.

— Robotics – NVIDIA Technical Blogvla-foundation-modelsworld-modelsmanipulation
#
3 Aug

LeRobot v0.6.1 renames a core module and loads safetensors straight to the device (tool · 7 minute read)

LeRobot v0.6.1 renames the lerobot.types module to lerobot.lerobot_types, a breaking change from v0.6.0. The release also loads safetensors directly onto the target device and improves the vision-language subtask annotation tooling. Other changes resolve policy components by convention and aggregate policy sub-losses through the metrics tracker.

— GitHubopen-source-toolsvla-foundation-models
#
3 Aug

Demystifying When and Why VLAs Fail in Contact-Rich Tasks and How to Fix Them (paper · 15 minute read)

The authors trace the failures of vision-language-action models in contact-rich manipulation to two causes: precision failures, from a mismatch in how flow-matching policies are trained, and force failures, from the structure of force signals. FACT combines a targeted fix for each and reaches a 66% average success rate on five contact-rich tasks, against 41% for the best prior baseline, over almost 2,500 real-world rollouts.

— arXivvla-foundation-modelsmanipulation