---
license: apache-2.0
base_model:
  - alibaba-pai/Wan2.2-Fun-5B-Control
pipeline_tag: video-to-video
tags:
  - matrix-game-4.0
  - causal-video
  - game-rendering
---

# Matrix-Game 4.0

[Source code](https://github.com/SkyworkAI/Matrix-Game)
| [Project website](https://matrix-game-v4-0.github.io/page/)

Scene-specialized causal video renderers for **Medieval Castle**, **Ancient
Egypt**, and **City Racing**. An executable game determines gameplay, physics
and input, while the model re-renders clean low-poly RGB. HUD and sound remain
separate, frame-aligned presentation layers.

## Contents

| Path | Contents |
| --- | --- |
| `config.json` | Release version, architecture, upstream revision and scene file index |
| `fps-castle/` | Castle checkpoint, sink image and inference configuration |
| `fps-egypt/` | Ancient Egypt checkpoint, sink image and inference configuration |
| `driving_game/` | Racing checkpoint, sink image and inference configuration |
| `base/` | Full streaming TAEHV codec |
| `game/assets/` | Shooter geometry, character data, catalogs and combat sounds |
| `racing/assets/` | City map, routes, captions, vehicle audio and audio credits |
| `releases/` | Exact resource paths, byte lengths and SHA256 manifests |

Each scene checkpoint has 5,001,016,512 BF16 parameters. The weights require
the Matrix-Game 4.0 inference source; they are not drop-in Diffusers pipelines.
Training code and training data are not included. Shared encoders/codecs are
installed once for all scenes. See [game asset details](GAME_ASSETS.md).

## Download and Layout

Use the source release's manifest downloader with a pinned Hub commit revision
and `--repo-id Skywork/Matrix-Game-4.0 --repo-type model` for both model and
game-asset manifests. See `releases/README.md` in the source repository for exact
commands. A scene selection downloads that scene plus its shared dependencies.

The shared root `config.json` is installed as `resources/release-config.json`
and verified by the model manifest. It describes this release; it is not a
Diffusers/Transformers pipeline configuration or the optional DiT
`resources/base/config.json`. Game-asset-only downloads do not fetch it.

The T5 encoder, tokenizer and Wan2.2 VAE are fetched directly from the official
[base-model repository](https://huggingface.co/alibaba-pai/Wan2.2-Fun-5B-Control/tree/b8bc1a65ab71d054ba4636dc0dac104aa4df2686).
The manifest pins that revision and verifies every file's SHA256; no manual
base-model download is needed. These shared files are not duplicated here.

The downloader maps Hub scene folders to the runtime paths
`resources/models/castle/`, `resources/models/ancient-egypt/` and
`resources/models/racing/`. Do not infer the local directory layout from the
Hub folder names alone. The `repo_path` field in `releases/models.json` describes
this mapping. Both length and SHA256 are checked before a file is installed.

## Inference Contract

- Output resolution: 480x864 RGB. One latent chunk corresponds to four RGB
  frames after startup; the startup chunk contains one RGB frame.
- Castle/Egypt use an independent identity sink with fixed-relative RoPE.
  Racing uses the supplied image as output frame zero and an absolute-position
  sink. These policies are not interchangeable.
- Quality preset: Wan VAE encoding/decoding, two denoising steps (1000, 500),
  native DiT. Realtime preset: TAEHV encoding/decoding, one step (1000), optimized
  single-GPU DiT and CUDA graphs.
- CFG is 1 and the scheduler shift is 5. The attention cache contains 21 latent
  frames including the sink. Racing skips denoising on its supplied first frame.
- Shooter captions are fixed. Racing supports fixed or per-chunk dynamic engine
  captions. Clean RGB, not the HUD or keyboard overlay, is the model input.

Use `base/taew2_2.pth` for the full streaming fast codec; a standalone decoder
file is not a replacement for its encoder. Use the exact shared files in the
manifest, not an arbitrary upstream version or Wan2.1's 16-channel VAE.

## Limitations

The models are specialized to the supplied low-poly scenes and camera profiles.
Unseen geometry, rapid camera motion, occlusion and long sessions can cause
appearance drift or geometry errors. Neural output does not change collision
or gameplay. One-step/TAEHV and two-step/Wan have different quality and cost.
There is no guaranteed frame rate, consumer-GPU memory fit, or exact scene
reconstruction. This model card does not claim a performance benchmark.

## License and Attribution

Project scene weights, sink images and game assets: **Apache-2.0**, Skywork AI.
The shared Wan/umT5 resources retain their upstream Apache-2.0 terms:
[Wan2.2-Fun-5B-Control](https://huggingface.co/alibaba-pai/Wan2.2-Fun-5B-Control),
[umT5](https://huggingface.co/google/umt5-xxl).
TAEHV is by Ollin Boer Bohan under MIT; see `licenses/TAEHV-LICENSE` and its
[upstream repository](https://github.com/madebyollin/taehv).
Racing audio includes CC0 and CC BY 3.0 material; retain
`racing/assets/audio/CREDITS.md`. See `THIRD_PARTY_NOTICES.md` for code dependencies.