free001style commited on
Commit
5f840e6
·
verified ·
1 Parent(s): 0b46285

Preview model card for DMax-Math-MMD

Browse files

Adds the model card for review. The training section is omitted, and the citation uses @article with the requested arXiv preprint TODO journal placeholder.

Files changed (1) hide show
  1. README.md +94 -0
README.md ADDED
@@ -0,0 +1,94 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: transformers
4
+ base_model: Zigeng/DMax-Math-16B
5
+ base_model_relation: finetune
6
+ datasets:
7
+ - Zigeng/DMax-LLaDA-2.0-Mini-Math-Trajectories
8
+ tags:
9
+ - diffusion
10
+ - dllm
11
+ - mmd
12
+ - math
13
+ - custom_code
14
+ ---
15
+
16
+ # DMax-Math-MMD
17
+
18
+ **Representation-Space MMD for Diffusion Language Models**
19
+
20
+ DMax-Math-MMD is a 16B diffusion language model for mathematical reasoning,
21
+ obtained by MMD post-training of
22
+ [DMax-Math-16B](https://huggingface.co/Zigeng/DMax-Math-16B).
23
+ It builds on LLaDA2.0-mini and uses DMax's hybrid masked–uniform block diffusion.
24
+
25
+ The post-training objective minimizes Maximum Mean Discrepancy (MMD) between
26
+ model samples and reference responses, measured in the representation space of
27
+ a frozen diffusion language model. The project reports increased tokens per
28
+ forward pass while maintaining or improving accuracy on the benchmarks below.
29
+
30
+ [Project code and documentation](https://github.com/yandex-research/dlm-mmd)
31
+ · [DMax-Coder-MMD](https://huggingface.co/yresearch/DMax-Coder-MMD)
32
+
33
+ ## Reference results
34
+
35
+ Results reported in the project README, with decoding threshold **0.85**.
36
+ Each entry is **accuracy (%) / tokens per forward pass (TPF)**.
37
+ Baseline results are attributed to the original DMax paper in the project README.
38
+
39
+ | Method | GSM8K | MATH500 | Minerva-Algebra | ASDIV |
40
+ | --- | :---: | :---: | :---: | :---: |
41
+ | DMax-Math | 92.1 / 5.48 | 75.4 / 5.94 | 91.5 / 7.03 | 92.5 / 5.62 |
42
+ | **DMax-Math-MMD** | 92.1 / **6.15** | **76.0** / **6.84** | **92.1** / **8.19** | **92.9** / **6.20** |
43
+
44
+ Results can vary with hardware, tensor parallelism, and library versions.
45
+ TPF measures decoding parallelism; wall-clock speed also depends on the runtime.
46
+
47
+ ## Evaluation and inference
48
+
49
+ Use DMax's dInfer evaluation pipeline. After following the evaluation environment
50
+ setup in the [project README](https://github.com/yandex-research/dlm-mmd#installation),
51
+ run from the project repository root:
52
+
53
+ ```bash
54
+ conda activate dinfer
55
+ DOMAIN=math MODEL_PATH=yresearch/DMax-Math-MMD bash scripts/eval.sh
56
+ ```
57
+
58
+ The project's math evaluation uses threshold **0.85** and evaluates GSM8K,
59
+ MATH500, Minerva-Algebra, and ASDIV.
60
+
61
+ The checkpoint includes nine Safetensors shards with BF16 parameters and FP32
62
+ router bias buffers, plus its tokenizer, chat template, and custom model code.
63
+ Direct Transformers loading requires `trust_remote_code=True`. Pass
64
+ `dtype=torch.bfloat16` for BF16 loading; the current configuration declares FP32.
65
+
66
+ Generation requires the DMax/dInfer diffusion decoder. The included custom
67
+ backbone does not implement the standard Transformers `.generate()` interface.
68
+
69
+ ## License and acknowledgements
70
+
71
+ The model follows the **Apache-2.0** license of the
72
+ [DMax-Math base checkpoint](https://huggingface.co/Zigeng/DMax-Math-16B)
73
+ and [LLaDA2.0-mini](https://huggingface.co/inclusionAI/LLaDA2.0-mini).
74
+ The included model implementation retains its Apache-2.0 notices.
75
+ The separate MMD training repository is MIT-licensed, with Apache-2.0 third-party components.
76
+
77
+ We thank the authors of [DMax](https://github.com/czg1225/DMax),
78
+ [LLaDA2.0-mini](https://huggingface.co/inclusionAI/LLaDA2.0-mini), and
79
+ [dInfer](https://github.com/inclusionAI/dInfer) for releasing their models, data, and code.
80
+
81
+ ## Citation
82
+
83
+ If you find this work useful in your research, please consider citing:
84
+
85
+ ```bibtex
86
+ @article{drobyshevskiy2026mmd,
87
+ title = {Representation-Space MMD for Diffusion Language Models},
88
+ author = {Drobyshevskiy, Ilya and Sudakov, Ilia and Semenov, Maksim and Kuznedelev, Denis and
89
+ Ignatov, Maksim and Temirchev, Pavel and Balagansky, Nikita and
90
+ Meshchaninov, Viacheslav and Gushchin, Nikita and Baranchuk, Dmitry},
91
+ journal = {arXiv preprint TODO},
92
+ year = {2026}
93
+ }
94
+ ```