NVIDIA’s four open Cosmos 3 Nano models let robot and driving teams simulate worlds that react

Physical AI needs worlds that react. NVIDIA’s Cosmos platform added four open-weight Cosmos 3 Nano post-trained models for interactive simulation, hosted on Hugging Face. Transfer-Auto generates multi-view driving scenes with weather and lighting control. Sim-Auto streams camera and LiDAR. Sim-Transfer turns classical simulator controls into photoreal video. Sim-Bimanual produces action-conditioned video of two-armed robots for YAM and AgiBot-style embodiments.

Each has 16 billion trainable parameters and ships under OpenMDW 1.1 for commercial and non-commercial use. Published recipes cover bidirectional adaptation, teacher forcing, consistency distillation and self-forcing. This follows NVIDIA’s broader push to make Cosmos a full assembly line for physical AI (our coverage) and its earlier interactive simulation update (coverage).

The backbone is a Mixture-of-Transformers design linking text and visual generators. It predicts short chunks from prompts, controls and history, and is distilled down to a handful of denoising steps for streaming. Self-forcing matters because in a generated world, a slightly wrong object becomes the next moment’s context, so training has to expose the model to that drift. This is infrastructure for teams building driving and robot policies who can’t film every edge case on real roads and factory floors.

Closed simulation stacks keep progress inside a few labs. Open world models that researchers can fine-tune let the field compare methods. Cosmos 3’s approach is a set of complementary checkpoints: offline multi-sensor driving data, fast causal streaming for closed-loop rollouts, a bridge from physics simulators to camera imagery, and two-arm robot interaction. Google’s Genie 4 is pursuing interactive worlds from a different angle, with audio and real-time steering (our Genie 4 analysis). NVIDIA has also released open Earth-2 weather models on the same open-weights logic (coverage).

These tools should be used to stress-test autonomy and manipulation before real hardware moves. The danger is synthetic comfort outrunning validation: policies that look great in Cosmos rollouts and fail on the first odd wet night or unexpected gripper slip. NVIDIA is explicit that fidelity for policy learning remains an open measurement problem. Open weights help, but they don’t excuse skipping held-out real-world evaluation. Useful physical AI will combine these generators with stubborn measurement, as NVIDIA’s own SimReady guidance acknowledges (SimReady walkthrough). More in our robotics coverage, autonomous vehicles coverage and NVIDIA coverage.