324 GB
59,821 files
Updated about 1 month ago
Name
Size
prompt_cache
real_latents
teacher_latents
ultravideo_prompts
.gitattributes2.64 kB
xet
LICENSE2.87 kB
xet
LICENSE_MiniMax-H317.6 kB
xet
NOTICE_MiniMax-H3372 Bytes
xet
README.md9.86 kB
xet
README.md

DMAD MiniMax-H3 training data

The preprocessed training data of the DMAD 4-step students of MiniMax-H3 (text-to-audio-video), from the paper DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation. The training code is in train/h3 of the DMAD repository; the released students are at ZhengmingYu/DMAD.

Everything here is what train/h3/scripts/prepare_data.sh produces from UltraVideo (short.csv and the clips_short_1920 clips) and the MiniMax-H3 base model, so training can start without the ~1 GPU-week of preprocessing. No video or audio is stored: the clips are kept only as MiniMax-H3 VAE latents, the captions as text-encoder outputs (the captions themselves are included as text).

Contents

303 GB, 59,816 data files, all tensors in bfloat16. real_latents/latents/, prompt_cache/conditions/ and teacher_latents/ keep their files in sub-folders named by the last two characters of the file name (teacher_latents/23/00000123.pt, real_latents/latents/5b/<clip_id>.pt), because the Hub allows at most 10,000 files per directory; the trainer looks files up in both this and the flat layout, so nothing needs to be moved.

Folder Files Size What
real_latents/ 24,004 .pt + 8 manifests 156 GB UltraVideo clips encoded with the MiniMax-H3 video and audio VAEs (data_process/encode_real_videos.py)
prompt_cache/ 17,902 .pt + metadata.jsonl 28 GB MiniMax-H3 text-encoder outputs of the training captions (data_process/encode_prompts.py)
teacher_latents/ 17,898 .pt 120 GB one MiniMax-H3 sample per training caption, packed token rows (data_process/gen_teacher_data.py)
ultravideo_prompts/ 3 text files 38 MB the caption list, the caption-to-clip map, the training caption indices (data_process/build_prompt_list.py, build_pool_indices.py)

real_latents/latents/<clip_id>.pt — {"video_latent": [24, T', 48, 84], "audio_latent": [2, n, 32], "meta": {...}} for a 768x1344, 24 fps crop of the clip starting at meta["t0"]: 17,911 clips at the 124-frame tier (T' = 37, 5.2 s) and 6,093 at the 107-frame tier (T' = 32); meta also records the source zip, duration and whether the HDR source was tone-mapped. manifest_rank*.jsonl list every clip of the UltraVideo short set (the 18,179 clips shorter than the 107-frame tier, and one clip rejected in a manual quality review, are recorded with "skip"). Training uses the 124-frame tier (17,911 clips).

prompt_cache/conditions/condition_<index>.pt — {"conditioning": {"positive": {"prompt_embeds": [1, L, 5120], "text_token_tags": [L]}}, "prompt": caption, "source_index": index}: the MiniMax-H3 text encoder's output for caption index (the line number in ultravideo_prompts/prompts_dedup.txt), as consumed by the H3 transformer. metadata.jsonl maps id to the caption and the file. The 17,902 cached captions are those of the 124-frame clips plus the four visualization prompts (indices 0-3).

teacher_latents/<index>.pt — {"video_rows": [1, 37296, 96], "audio_rows": [1, 414, 32], "meta": {...}}: one sample of the MiniMax-H3 base model for caption index with its official sampler (31 steps, video shift 12, audio shift 3, seed = index), stored in the transformer's packed token layout at 768x1344 x 124 frames.

ultravideo_prompts/ — prompts_dedup.txt (42,158 unique UltraVideo "Detailed Description" captions, one per line, first-occurrence order), ultravideo_index_to_clipid.jsonl ({"index", "clip_ids"}, the real_caption_map of the training config), pool124_plus_viz_indices.txt (the 17,902 caption indices used in training).

Use

hf download ZhengmingYu/DMAD-H3-data --repo-type dataset --local-dir /path/to/dmad_h3_data

then in train/h3/configs/dmad_minimax_h3.yaml (or dmad_minimax_h3_full_critic.yaml):

dmad:
    prompt_cache_dir:   /path/to/dmad_h3_data/prompt_cache
    real_latent_dir:    /path/to/dmad_h3_data/real_latents
    real_caption_map:   /path/to/dmad_h3_data/ultravideo_prompts/ultravideo_index_to_clipid.jsonl
    teacher_latent_dir: /path/to/dmad_h3_data/teacher_latents

The latents are tied to the MiniMax-H3 VAEs and text encoder; they are not usable with other models. To rebuild or extend the data, see "Data Preparation" in the train/h3 README.

License

  • real_latents/, prompt_cache/ and ultravideo_prompts/ are derived from the UltraVideo dataset and are distributed under the UltraVideo Dataset License Agreement (CC-BY-4.0 with additional restrictions: non-commercial research use only, no redistribution of the source videos, derivative datasets keep these terms). By requesting access you accept that agreement; please also follow the attribution requirements of UltraVideo and its upstream sources (Panda-70M, Koala-36M).
  • teacher_latents/ are outputs of MiniMax-H3 and are AI-generated; see NOTICE_MiniMax-H3 and the MiniMax H3 Community License Agreement.

Citation

@misc{yu2026dmad,
  title         = {DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation},
  author        = {Zhengming Yu and Junkun Yuan and Haotian Yang and Gordon Guocheng Qian and Yizhi Wang and
                   Angtian Wang and Yiding Yang and Bo Liu and Xin Li and Wenping Wang and Chongyang Ma},
  year          = {2026},
  eprint        = {2610.02188},
  archivePrefix = {arXiv}
}

@article{ultravideo,
  title={UltraVideo: High-Quality UHD Video Dataset with Comprehensive Captions},
  author={Xue, Zhucun and Zhang, Jiangning and Hu, Teng and He, Haoyang and Chen, Yinan and Cai, Yuxuan and Wang, Yabiao and Wang, Chengjie and Liu, Yong and Li, Xiangtai and Tao, Dacheng},
  journal={arXiv preprint arXiv:2506.13691},
  year={2025}
}
Total size
324 GB
Files
59,821
Last updated
Oct 13
Pre-warmed CDN
US EU US EU

Contributors