iapple1 commited on
Commit
fc22c47
·
verified ·
1 Parent(s): 39be9df

Refresh the official English model card

Browse files
Files changed (2) hide show
  1. README.md +40 -24
  2. SHA256SUMS.txt +1 -1
README.md CHANGED
@@ -2,9 +2,10 @@
2
  license: apache-2.0
3
  language:
4
  - en
5
- - zh
6
  base_model: MiniMaxAI/MiniMax-H3
7
  base_model_relation: adapter
 
 
8
  pipeline_tag: image-to-video
9
  tags:
10
  - lora
@@ -12,46 +13,61 @@ tags:
12
  - video-generation
13
  - multi-view
14
  - world-model
15
- - decfg
16
  ---
17
 
18
- # CoCode-World Model
19
 
20
- The **step-4000** CoCode-World task LoRA for **MiniMax-H3 Ref2VA**, trained on synchronized two-player GTA videos with Proxy555 control. The EMA weights are the ones used by the eight published 60-second evaluation examples.
21
 
22
- - **Code and training guide:** https://github.com/IApple233/CoCode-World
23
- - **Evaluation inputs and results:** https://github.com/IApple233/CoCode-World/tree/main/examples/evaluation
24
- - **Public training data:** https://huggingface.co/datasets/iapple1/CoCode-World-Data
25
- - **Project page:** https://iapple233.github.io/CCW/
 
 
 
 
 
 
 
 
 
 
 
 
26
 
27
  ## Download
28
 
29
  ```bash
 
30
  hf download iapple1/CoCode-World-Model --local-dir checkpoints/step-004000
31
  ```
32
 
33
- | File | Use |
34
- |---|---|
35
- | `ema.safetensors` | EMA task LoRA; recommended for inference (FP32 storage) |
36
- | `lora.safetensors` | Non-EMA task LoRA at step 4000 (BF16 storage) |
37
- | `optimizer.pt`, `rng_state.pt` | Saved training state for an exact optimizer resume with matching training data and configuration |
38
- | `trainer_state.json` | Architecture, training settings, tensor names fingerprint and payload checksums |
39
- | `model_manifest.json` | Release provenance and original checkpoint metadata hash |
 
 
 
40
 
41
- These are native checkpoint tensors for the provided loader, not a standalone base model or a drop-in Diffusers `load_lora_weights` export. Download the official MiniMax-H3 model separately. Tensor files are unchanged; local paths in the trainer metadata were normalized for publication.
42
 
43
- ## Training initialization
44
 
45
- **First download the official [MiniMax-H3 weights](https://huggingface.co/MiniMaxAI/MiniMax-H3). Then download DiffSynth-Studio's released [Ref2VA DeCFG training adapter](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter).** Continue task training from that pretrained setup: fuse `model_ref2va.safetensors` into the frozen base at scale 1, then train a fresh rank-64 task LoRA. The training code verifies the exact DeCFG file checksum.
46
 
47
- 训练前先下载 MiniMax-H3 官方权重,再下载 DiffSynth-Studio 开源的 Ref2VA DeCFG LoRA。训练时固定合入 DeCFG 预设,在此基础上训练 CoCode-World 的 rank-64 任务 LoRA。推理时加载原始 MiniMax-H3 和本仓库的任务 LoRA EMA;不合入训练用 DeCFG 预设。
48
 
49
- The published checkpoint was trained with learning rate `1e-5`, 200 warmup steps, rank/alpha `64/64`, EMA decay `0.999`, CP8 × DP4 on 32 GPUs, and a target-row flow-matching objective. See `training_config.json` for the recorded configuration. The original run used 8,552 clips; the public 5,000-clip dataset is a curated subset and does not reproduce the exact original training trajectory.
50
 
51
- ## Inference
52
 
53
- Use the code repository's `scripts/evaluate.sh`. The eight examples include the actual edited first-frame input, all 17 window prompts, original Proxy555 input, seeds, and a compressed preview of the recorded result. Recorded inference used 30 denoising steps, 24 FPS, a 2560 × 720 output, 124-frame windows, 39-frame overlap, and 17 windows cropped to 60 seconds. This is the original task LoRA, not a DMD student.
54
 
55
- ## License
56
 
57
- The task adapter retains this repository's Apache-2.0 license. The MiniMax-H3 base model is separately governed by the [MiniMax-H3 Community License Agreement](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE). The training-only DeCFG adapter retains its upstream terms. No MiniMax-H3 base weights or DeCFG preset weights are redistributed here.
 
2
  license: apache-2.0
3
  language:
4
  - en
 
5
  base_model: MiniMaxAI/MiniMax-H3
6
  base_model_relation: adapter
7
+ datasets:
8
+ - iapple1/CoCode-World-Data
9
  pipeline_tag: image-to-video
10
  tags:
11
  - lora
 
13
  - video-generation
14
  - multi-view
15
  - world-model
 
16
  ---
17
 
18
+ # CoCode-World
19
 
20
+ **Shared 3D Proxy Representation for Multi-Player World Models**
21
 
22
+ [Project Page](https://iapple233.github.io/CCW/) · [Code](https://github.com/IApple233/CoCode-World) · [Dataset](https://huggingface.co/datasets/iapple1/CoCode-World-Data) · [Examples](https://github.com/IApple233/CoCode-World/tree/main/examples/evaluation)
23
+
24
+ ## Model overview
25
+
26
+ CoCode-World generates synchronized video from two player perspectives using a shared 3D proxy representation. A reference image defines the visual appearance, a proxy video provides scene structure and motion, and a text prompt describes the scene.
27
+
28
+ This repository provides the official CoCode-World LoRA checkpoint for **MiniMax-H3 Ref2VA**. The recommended EMA weights are used in the released evaluation examples.
29
+
30
+ | Property | Description |
31
+ |---|---|
32
+ | Base model | MiniMax-H3 Ref2VA |
33
+ | Adapter | Rank-64 LoRA |
34
+ | Checkpoint | Step 4000 |
35
+ | Inputs | Reference image, synchronized proxy video, and text prompt |
36
+ | Output | Two player views arranged side by side |
37
+ | Resolution | 2560 × 720 at 24 FPS |
38
 
39
  ## Download
40
 
41
  ```bash
42
+ pip install -U huggingface_hub
43
  hf download iapple1/CoCode-World-Model --local-dir checkpoints/step-004000
44
  ```
45
 
46
+ The repository includes `ema.safetensors` for inference, `lora.safetensors` for the task adapter, and training state and configuration files. Use the checkpoint loader in the official code repository. Download the MiniMax-H3 base model separately.
47
+
48
+ ## Usage
49
+
50
+ Follow the [installation guide](https://github.com/IApple233/CoCode-World/blob/main/docs/SETUP.md), then download the required models from the code repository:
51
+
52
+ ```bash
53
+ python scripts/download_models.py base
54
+ python scripts/download_models.py cocode
55
+ ```
56
 
57
+ The [evaluation guide](https://github.com/IApple233/CoCode-World/blob/main/docs/EVALUATION.md) explains input preparation and generation. Eight ready-to-use examples include reference images, prompts, proxy videos, and generated results.
58
 
59
+ ## Training
60
 
61
+ Training starts from the official [MiniMax-H3 weights](https://huggingface.co/MiniMaxAI/MiniMax-H3) and [DiffSynth-Studio's Ref2VA DeCFG LoRA](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter). The DeCFG adapter is used as a frozen training preset, followed by training a separate CoCode-World task LoRA. Inference uses the original MiniMax-H3 base and the CoCode-World EMA LoRA.
62
 
63
+ Training code and instructions are available in the [training guide](https://github.com/IApple233/CoCode-World/blob/main/docs/TRAINING.md). The public dataset is a selected subset of the original training collection.
64
 
65
+ ## Intended use and limitations
66
 
67
+ The model supports research on multi-player world models, synchronized multi-view generation, and proxy-conditioned video synthesis. The first-frame reference guides appearance. Generated videos may contain visual artifacts or temporal inconsistencies, and the selected examples do not constitute a quantitative benchmark.
68
 
69
+ ## License and acknowledgments
70
 
71
+ The task adapter is released under Apache 2.0. The MiniMax-H3 base model is governed by the [MiniMax-H3 Community License Agreement](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE). The DeCFG training adapter retains its upstream terms.
72
 
73
+ CoCode-World builds on [MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3), [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio), [Diffusers](https://github.com/huggingface/diffusers), and [Code World Model](https://github.com/buaacyw/code-world-model).
SHA256SUMS.txt CHANGED
@@ -1,5 +1,5 @@
1
  f9e2070c247517b1ddf65f7b11b393484a18da91a958fd18a97bd0f241c3125c LICENSE
2
- 40016785b3089d546b9715e76f6886cbccca98eb3973d4202bd41d6858852bed README.md
3
  9d4e0524ee2b9203c2711f2c0b2a133e17d9ea6cb1280aa9974150f0ba7a038c ema.safetensors
4
  e0d4a3cb1c8cb8cb8ef36cefeb45ecc164556404f4003ec39b1af8f435ca602c lora.safetensors
5
  99c54ce47fe0aa7fa170ddd530b8de7abed0f02dcc970750c63c545021269f33 model_manifest.json
 
1
  f9e2070c247517b1ddf65f7b11b393484a18da91a958fd18a97bd0f241c3125c LICENSE
2
+ 0cf9f82ef2c0845eb5e6f06d45b508016396970928e841b04ae09a8fcad17dd4 README.md
3
  9d4e0524ee2b9203c2711f2c0b2a133e17d9ea6cb1280aa9974150f0ba7a038c ema.safetensors
4
  e0d4a3cb1c8cb8cb8ef36cefeb45ecc164556404f4003ec39b1af8f435ca602c lora.safetensors
5
  99c54ce47fe0aa7fa170ddd530b8de7abed0f02dcc970750c63c545021269f33 model_manifest.json