README.md

---
license: apache-2.0
language:
- en
tags:
- sawman
- image-to-video
- action-conditioned
- six-view
- distillation
- safetensors
base_model:
- alibaba-pai/Wan2.1-Fun-V1.1-1.3B-InP
- Wan-AI/Wan2.2-TI2V-5B
---

# SAWMAN: Action-conditioned six-view world models

Selected **Stage3 EMA** checkpoints for Wan2.1 Fun-InP 1.3B and Wan2.2 TI2V 5B.
[Code and installation instructions](https://github.com/EmbodiedCity/SAWMAN.code).

| Directory | Backbone | Checkpoint | Training rollout | Inference |
| --- | --- | --- | --- | --- |
| `wan1p3b/` | Wan2.1 Fun-InP 1.3B | Stage3 iteration 100, EMA | 2 steps | 2 or 4 steps |
| `wan5b/` | Wan2.2 TI2V 5B | Stage3 iteration 100, EMA | 4 steps | 2 or 4 steps |

Two steps are the fast inference preset. The 5B two-step setting is an additional
sampling evaluation; its DMD training used four steps. Each selected checkpoint
represents 20 generator updates. Selection used the highest mean PSNR among the
evaluated Stage3 EMA checkpoints on six representative training examples (one
per action). This is not a claim of optimal performance on an independent test set.

## Download and use

From the code repository root, after installation:

```bash
hf download EmbodiedCity/SAWMAN --local-dir weights/sawman

CUDA_VISIBLE_DEVICES=0 bash scripts/infer_wan1p3b.sh \
  --checkpoint weights/sawman/wan1p3b/diffusion_pytorch_model_ema.safetensors \
  --model-root weights/wan1p3b/base --metadata data/metadata.jsonl \
  --output outputs/eval/wan1p3b_2step --steps 2 --limit 2

CUDA_VISIBLE_DEVICES=0 bash scripts/infer_wan5b.sh \
  --checkpoint weights/sawman/wan5b/diffusion_pytorch_model_ema.safetensors \
  --model-root weights/wan5b/base --metadata data/metadata.jsonl \
  --output outputs/eval/wan5b_2step --steps 2 --limit 2
```

For one model only, add `--include 'wan1p3b/*'` or `--include 'wan5b/*'` to the
download command. Supply the corresponding base model files and your own
21-frame six-view videos as described in the code README. The inference helper
uses metadata videos for offline evaluation. Future ground-truth frames are not
student conditioning. The public EMA serves directly as the inference DiT;
separate SAWMAN SFT or Stage2 weights are not required for this inference path.

## Format and input contract

Each directory contains a complete **BF16 DiT state dictionary** in Safetensors
format and a portable inference `config.json`. These are neither LoRA adapters
nor standalone Hugging Face Diffusers pipelines. Use the native code entries;
`DiffusionPipeline.from_pretrained` is not the loading interface. Text encoders,
tokenizers, VAEs, the 1.3B CLIP encoder and upstream base files are supplied
separately. No LightTAE weights or optimizer/resume states are included.
`manifest.json` records file sizes, SHA-256 checksums and tensor counts.

- 21 RGB frames at height 640 × width 960, including the initial observation.
- Fixed 2×3 canvas: `[left, front, right]` above `[up, back, down]`.
- Actions: `move up`, `move down`, `move left`, `move right`, `move forth`, `move back`.
- Native bidirectional whole-video prediction, not causal blockwise CFPP inference.
- 1.3B uses CLIP plus masked-video conditioning; 5B clamps the observed first latent.
- Student guidance scale 1; sigma shift 5. Playback FPS in config is not generation throughput.

## Scope and limitations

These checkpoints support research on six-view aerial prediction. They can show
blur, motion underestimation and geometric drift. Higher PSNR does not establish
better motion or navigation quality. The release does not include a closed-loop
VLM navigation agent or certify navigation benchmark scores for these checkpoints.
The checkpoint iteration is retained for provenance; training to a larger iteration
did not give the highest score in the available small selection evaluation.

The repository declares Apache-2.0 for these released weights. Refer also to the
respective upstream model licenses. Code licensing and third-party attribution
are documented in the code repository.