FRED-5B-DROID

Checkpoint of FRED (Future REpresentation learning via self-Distillation with temporal context) from Self-Distilling Future Video Representations for World Action Models.

Project page · Code · Paper (arXiv)

FRED trains World Action Models (WAMs) to produce better future video representations from a single video denoising step: the video model is conditioned on a short history of observations, and its intermediate representation of the future is trained by self-distillation from an EMA teacher that sees the noisy ground-truth future observations.


Video backbone	Wan2.2-TI2V-5B
Action decoder	16-layer DiT
Training data	DROID (GEAR-Dreams/DreamZero-DROID-Data)
Training	batch 128, 150K iterations
Files	`checkpoint-150000/` (~23 GB)

Zero-shot environment generalization on a Franka Research 3 robot (new environment, no environment-specific fine-tuning; from the paper):

In-domain easy	Long-horizon	Out-of-domain	Object grounding
47.2	46.7	37.5	37.5

Long-horizon is the average task progress (%); the others are average success rates (%).

Usage

Install the code, then download the checkpoint:

hf download FRED-AGI-LAB/fred-wan22-5b-droid --local-dir fred-wan22-5b-droid

The checkpoint directory is fred-wan22-5b-droid/checkpoint-150000; pass this local path to the scripts (not the repo ID).

Serve the policy (see docs/droid.md for the text cache and the robot-side client):

.venv/bin/python fred/eval/serve_policy.py --model-path fred-wan22-5b-droid/checkpoint-150000 --port 5555 \
    --text-embed-cache ~/.cache/wam/droid_wan22_umt5.pt

The same directory can warm-start training with BASE_MODEL_PATH=fred-wan22-5b-droid/checkpoint-150000.

Citation

@article{fred2026,
  title   = {Self-Distilling Future Video Representations for World Action Models},
  author  = {Jang, Huiwon and Jang, Suhyeok and Kang, Hyeonmo and Kim, Seungku and Shin, Jinwoo and Yu, Sihyun},
  journal = {arXiv preprint arXiv:2610.30320},
  year    = {2026}
}