Papers
arxiv:2611.04870

PrimeSim: A Generalist World Model Simulator from Large-Scale Cross-Embodiment Demonstrations

Published on Nov 3
· Submitted by
Travis Davies
on Nov 4
Authors:
,
,
,
,
,
,
,
,
,
,
,
,

Abstract

PrimeSim is an action-conditioned multiview world model that uses progressive training, unified action representations, view-aware attention, and hierarchical memory to enable consistent, long-horizon robot simulation and policy optimization.

A robot world model that serves as a simulator must do more than generate plausible videos: it must faithfully follow commanded actions, maintain consistency across camera views, and sustain coherent long-horizon rollouts. We present PrimeSim, an action-conditioned, multiview world model simulator that addresses these challenges through progressive midtraining and posttraining. Starting from a pretrained video generator, we midtrain on 30,110 hours of robot demonstrations spanning 22 platforms. To improve action following, we introduce a unified cross-embodiment action representation and frame-aligned dual-path conditioning to strengthen action-frame correspondence. To maintain multiview consistency, we develop view-aware interleaved attention with explicit camera identities, enabling cross-view information exchange while preserving view-specific representations. We further introduce a fast-video, slow-memory posttraining framework that transforms the bidirectional video generator into an efficient autoregressive simulator. By combining fast causal generation, self-rollout training, and hierarchical temporal memory, PrimeSim mitigates error accumulation from self-generated histories and supports coherent long-horizon simulation. PrimeSim runs at 10.1 FPS, remains stable over long-horizon rollouts on rigid and deformable manipulation tasks, and transfers to held-out platforms within 100 to 200 gradient steps where the pretrained backbone does not. Against five open-source action-conditioned world models it leads four of the five WorldSimProbe suites and posts the best overall EWMBench score, while fine-grained local calibration still trails the field; its rollouts are faithful enough to support GRPO-based policy optimization. We release the model weights, inference code, and training code.

Community

PrimeSim is an action-conditioned, multiview world model that runs as a robot
simulator. It starts from a pretrained video generator, midtrains on 30,110 hours
across 22 platforms, then posttrains into a causal few-step generator. Posttraining
combines self-rollout training, so the model learns from its own generated
histories rather than only real frames, with a selected KV-cache memory that
carries context across chunks.

Inference runs at over 10 FPS with all three camera views. PrimeSim scores first
overall on EWMBench and leads four of five WorldSimProbe suites against
state-of-the-art baselines. Rollouts stay coherent over long horizons and transfer
to held-out platforms within 100-200 gradient steps.

We also use the model as an environment for reinforcement learning: GRPO
post-training inside PrimeSim raises LIBERO success from 61.7% to 73.2%, and the
model's success predictions track the physics simulator closely enough to rank
policies. All experiments are in simulation (AgiBot G2, LIBERO, RoboTwin); no
physical robot is used, and closed-loop hardware deployment is future work.

Weights and code will be released on the 6th of November.

Project page: https://primebottech.github.io/PrimeSim-website/

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2611.04870
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.04870 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.04870 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.04870 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.