ReMind 5B — Teaching Video Generators to Remember

Official ReMind 5B checkpoints for Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolution (arXiv:2605.25333).

ReMind post-trains a causal video generator to preserve and evolve hidden world state through camera motion, occlusion, and illumination changes.

Release status

Model	Status
ReMind 5B	Available in this repository
ReMind 1.3B	Coming soon

This repository currently releases only the validated ReMind 5B weights. It does not contain ReMind 1.3B weights.

Release checklist:

• Release ReMind-5B checkpoints
• Document Qwen3-VL prompt alignment for custom inputs
• Release ReMind-1.3B checkpoints

Files

File	Size	Purpose
ReMind-5B.safetensors	10.37 GiB	Complete ReMind-5B generator
ReMind-5b-dmd-ema.safetensors	1.38 GiB	Rank-128 SF-DMD EMA student LoRA

The DMD EMA file is not a standalone model. Load the official Wan-AI/Wan2.2-TI2V-5B model, overlay the ReMind-5B generator, cast it to BF16, and then merge the ReMind DMD EMA LoRA.

Only inference weights are included. Optimizer state, gradients, critics, raw student adapters, schedulers, and training-state checkpoints are excluded.

Setup

Clone and install the public ReMind code:

git clone https://github.com/Applied-Intuition-Open-Source/ReMind.git
cd ReMind
python -m venv .venv
source .venv/bin/activate
pip install -U pip
pip install -r requirements.txt
pip install -e .

Download the official Wan 2.2 TI2V 5B model in its original repository layout:

hf download Wan-AI/Wan2.2-TI2V-5B \
  --local-dir checkpoints/Wan2.2-TI2V-5B

Place the two ReMind files together, for example under checkpoints/ReMind-5B/, and run one of the seven bundled presets:

python inference.py \
  --preset examples/presets/01_latte_occluder_recovery.yaml \
  --config configs/model_5b.yaml \
  --model-folder checkpoints/Wan2.2-TI2V-5B \
  --base-checkpoint checkpoints/ReMind-5B/ReMind-5B.safetensors \
  --ema-checkpoint checkpoints/ReMind-5B/ReMind-5b-dmd-ema.safetensors \
  --output outputs/latte_occluder_recovery.mp4

The public inference recipe uses 81 frames at 832×480 and 16 fps, seven three-latent-frame chunks, and the shifted four-step schedule [1000, 938, 833, 625]. Camera examples use the bundled pair-fixed GT camera trajectories.

Qwen3-VL prompt alignment for custom inputs

The optional adapter turns one initial image and an ordinary user request into an audited, executable ReMind preset using greedy Qwen/Qwen3-VL-8B-Instruct inference:

pip install -r requirements_prompt_engineering.txt

python tools/rewrite_remind_prompt.py \
  --input-image /path/to/first_frame.png \
  --user-prompt 'Keep pouring cereal into the plate so the pile gradually grows, without spilling outside.' \
  --name cereal_user \
  --output-dir outputs/prompt_alignment/cereal_user

python inference.py \
  --preset outputs/prompt_alignment/cereal_user/preset.yaml \
  --dry-run

python inference.py \
  --preset outputs/prompt_alignment/cereal_user/preset.yaml \
  --model-folder checkpoints/Wan2.2-TI2V-5B \
  --base-checkpoint checkpoints/ReMind-5B/ReMind-5B.safetensors \
  --ema-checkpoint checkpoints/ReMind-5B/ReMind-5b-dmd-ema.safetensors \
  --output outputs/cereal_user.mp4

The adapter keeps scene dynamics separate from camera/visibility controls, compiles seven chunk-local prompts, and only exports a preset after schema and heuristic VLM self-audit checks pass. It never invents camera geometry: camera requests must select one of the bundled pair-fixed trajectories.

See docs/prompt_alignment.md for camera, occluder, light, local-model, and failure-handling options. The self-audit is heuristic, so review unusual requests and keep fixed project-page captions unchanged.

Intended use

ReMind is a research artifact for studying dynamic memory, controlled camera motion, reversible visibility disturbances, and clean image-to-video generation. The bundled examples demonstrate representative inputs and controls; they are not a benchmark or a guarantee of behavior.

Limitations

• Long or complex motion can accumulate autoregressive errors.
• Camera control is approximate and may not follow every requested path.
• Recovery depends on scene content, event timing, seed, and model size.
• Outputs can contain physical, geometric, identity, lighting, and temporal artifacts.
• The model should not be used for high-stakes decisions or to misrepresent generated media as real.

License

The two ReMind weight files are licensed under Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0). See LICENSE for the full legal code.

The upstream Wan model is not included and remains governed by its own terms. The public ReMind code and bundled third-party components are governed by the licenses and notices in the code repository.

Citation

@article{xu2026teaching,
  title={Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolution},
  author={Xu, Tianshuo and Xie, Yichen and Meng, Depu and Peng, Chensheng and Herau, Quentin and Jiang, Bo and Hu, Yihan and Zhan, Wei},
  journal={arXiv preprint arXiv:2605.25333},
  year={2026}
}