Blame for 1.3B/README.md

# Elastic Forcing — 1.3B

[Overview](../README.md) · [14B guide](../14B/README.md) · [Checkpoints](https://huggingface.co/Elastic-forcing/Elastic-forcing-checkpoint/tree/main/checkpoints/1.3B)

The 1.3B model uses Wan2.1-T2V-1.3B with DINOv3, V-JEPA 2, and VideoMAEv2 for distribution matching. It generates 81-frame videos at 832 × 480 resolution and 16 FPS, with four causal denoising steps per chunk.

## Environment

Use Linux, Python 3.10, and NVIDIA GPUs with BF16 support. Run these commands from `1.3B/`:

```bash
conda create -n elastic-forcing python=3.10 -y
conda activate elastic-forcing
python -m pip install torch==2.5.1 torchvision==0.20.1 \
  --index-url https://download.pytorch.org/whl/cu121
python -m pip install -r requirements.txt
python -m pip install flash-attn==2.7.4.post1 --no-build-isolation
export PYTHON_BIN="$(command -v python)"
```

## Models and assets

Inference requires the Wan2.1-T2V-1.3B base model, T5 text encoder, tokenizer, VAE, and the Elastic Forcing generator checkpoint. Place the base-model bundle under `$ASSET_ROOT/models/Wan2.1-T2V-1.3B/`. Training also requires the Self-Forcing ODE initialization, local DINOv3, V-JEPA 2, and VideoMAEv2 models, and precomputed reference features.

Download the generator checkpoint:

```python
from pathlib import Path
from huggingface_hub import snapshot_download

folder = Path(snapshot_download(
    "Elastic-forcing/Elastic-forcing-checkpoint",
    allow_patterns=["checkpoints/1.3B/*"],
))
print(folder / "checkpoints/1.3B/model.pt")
```

Set the local asset root and generate the configuration:

```bash
export ASSET_ROOT=/path/to/assets
python scripts/prepare_config.py --asset-root "$ASSET_ROOT"
```

## Inference

Use the downloaded checkpoint and a text file containing one prompt per line:

```bash
CHECKPOINT=/path/to/model.pt \
GPUS=0 \
PROMPTS=code/prompts/test_8.txt \
OUTPUT_DIR=outputs/demo \
bash scripts/infer.sh
```

The launcher produces one video per prompt with seed 0 and non-EMA generator weights. Prepare base-model assets before launching; model loading uses local files.

## Training

Training requires an ODE initialization, all three frozen encoders, a matched prompt/video manifest, a reference feature bank, and one Nyström summary per encoder. Prepare the model and reference assets at the locations specified in [configs/three_encoder.yaml](configs/three_encoder.yaml), generate the local configuration, and start training:

```bash
GPUS=0,1,2,3 LOGDIR=runs/train bash scripts/train.sh
```

| Setting | Value |
|---|---|
| Training updates | 150 |
| Distribution / differentiation / reference batch | 256 / 128 / 128 |
| Per-GPU generated / differentiated videos | 64 / 32 |
| Reference videos | 8,682 |
| Encoder weights | DINOv3 : V-JEPA 2 : VideoMAEv2 = 10 : 1 : 1 |
| Loss divisor | 3 |
| Nyström / Monte Carlo weights | 2 : 1 |
| Optimizer | AdamW, betas (0, 0.999), weight decay 0 |
| Learning rate | Cosine from 2.5e-6 to 5e-7 over 125 updates, then constant |
| Gradient clipping | Maximum norm 10 |
| Checkpoint interval | Every five updates |

The launcher uses FSDP and gradient checkpointing. Training settings are in [configs/three_encoder.yaml](configs/three_encoder.yaml). Use a distinct `LOGDIR` for each run; checkpoints include optimizer and random-state files for resuming training.

## Evaluation

| VBench Total ↑ | Quality ↑ | Semantic ↑ |
|---:|---:|---:|
| 84.25 | 85.06 | 80.99 |

Use [VBench](https://github.com/Vchitect/VBench) to evaluate generated videos. Per-dimension scores are in the [reference results](results/reference/three-encoder-summary.json). Run the CPU numerical tests from `1.3B/`:

```bash
python -m pip install pytest
python scripts/test_cpu.py
```