| --- |
| license: mit |
| library_name: diffusers |
| pipeline_tag: image-to-video |
| tags: |
| - worldcrafter |
| - camera-control |
| - autoregressive |
| - context-memory |
| - arxiv:2609.24984 |
| --- |
| |
| # WorldCrafter-Context |
|
|
| **An autoregressive video baseline with standard historical-frame context memory |
| and UCPE camera control, without a memory encoder.** |
|
|
| WorldCrafter-Context is an intermediate training checkpoint released to help |
| researchers fine-tune the model and develop their own memory mechanisms, such as |
| point-cloud conditioning. It precedes the learned memory encoder used by |
| WorldCrafter-Base and WorldCrafter-Fast. |
|
|
| ## Components |
|
|
| | Component | Source | |
| | --- | --- | |
| | Transformer | `TencentARC/WorldCrafter-Base`, `transformer/` | |
| | UCPE camera adapter | This repository, `camera_adapter.bin` (FP32, approximately 2.10 GB) | |
| | Text encoder, tokenizer, VAE and scheduler | `TencentARC/WorldCrafter-Fast` | |
| | Memory encoder | None; history is provided as video latents | |
|
|
| Shared components are referenced at fixed revisions in `inference_config.json` |
| and `model_index.json`; their weights are not duplicated here. `manifest.json` |
| records file hashes and sizes. |
|
|
| ## Download |
|
|
| Install the Hugging Face CLI with `pip install huggingface_hub`, then run: |
|
|
| ```bash |
| hf download Drexubery/WorldCrafter-Context --local-dir weights/WorldCrafter-Context |
| hf download TencentARC/WorldCrafter-Base \ |
| --revision d6b4731d4cec53f90b6afaee5a0f1b77d29979aa \ |
| --include "transformer/*" --local-dir weights/WorldCrafter-Base |
| hf download TencentARC/WorldCrafter-Fast \ |
| --revision d2ac12a654f8cafde4cb2e0ae7d5127ce2624939 \ |
| --include "text_encoder/*" "tokenizer/*" "vae/*" "scheduler/*" \ |
| --local-dir weights/WorldCrafter-Fast |
| ``` |
|
|
| These commands download only the shared components needed by Context. Existing |
| matching files can be reused. Keep the downloaded configuration and weight files |
| together. |
|
|
| ## Construct the baseline |
|
|
| Combine Base's shared transformer, this repository's UCPE adapter and Fast's |
| text encoder, tokenizer, VAE and scheduler. Use historical video latents as context |
| without a memory encoder; do not load Base/Fast's additional LoRA, camera adapters |
| or RepEncoder. See the [training documentation](https://github.com/TencentARC/WorldCrafter/tree/main/training#worldcrafter-context-baseline). |
|
|
| `camera_adapter.bin` is the original FP32 PyTorch state dictionary, with 400 |
| `blocks.<index>.cam_self_attn.*` tensors and no outer checkpoint wrapper. |
| This release is for custom integration; the current Base/Fast entrypoints and |
| standalone `DiffusionPipeline.from_pretrained` do not load Context automatically. |
|
|
| ## Links |
|
|
| - [WorldCrafter code](https://github.com/TencentARC/WorldCrafter) |
| - [Training documentation](https://github.com/TencentARC/WorldCrafter/tree/main/training) |
| - [Paper](https://arxiv.org/abs/2609.24984) |
| - [Project website](https://drexubery.github.io/WorldCrafter/) |
|
|
