free001style commited on
Commit
e8f6700
·
verified ·
1 Parent(s): 8e3f801

Preview model card for DMax-Coder-MMD

Browse files

Adds the model card for review. The training section is omitted, and the citation uses @article with the requested arXiv preprint TODO journal placeholder.

Files changed (1) hide show
  1. README.md +94 -0
README.md ADDED
@@ -0,0 +1,94 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: transformers
4
+ base_model: Zigeng/DMax-Coder-16B
5
+ base_model_relation: finetune
6
+ datasets:
7
+ - Zigeng/DMax-LLaDA-2.0-Mini-Code-Trajectories
8
+ tags:
9
+ - diffusion
10
+ - dllm
11
+ - mmd
12
+ - code
13
+ - custom_code
14
+ ---
15
+
16
+ # DMax-Coder-MMD
17
+
18
+ **Representation-Space MMD for Diffusion Language Models**
19
+
20
+ DMax-Coder-MMD is a 16B diffusion language model for code generation,
21
+ obtained by MMD post-training of
22
+ [DMax-Coder-16B](https://huggingface.co/Zigeng/DMax-Coder-16B).
23
+ It builds on LLaDA2.0-mini and uses DMax's hybrid masked–uniform block diffusion.
24
+
25
+ The post-training objective minimizes Maximum Mean Discrepancy (MMD) between
26
+ model samples and reference responses, measured in the representation space of
27
+ a frozen diffusion language model. The project reports increased tokens per
28
+ forward pass while maintaining or improving accuracy on the benchmarks below.
29
+
30
+ [Project code and documentation](https://github.com/yandex-research/dlm-mmd)
31
+ · [DMax-Math-MMD](https://huggingface.co/yresearch/DMax-Math-MMD)
32
+
33
+ ## Reference results
34
+
35
+ Results reported in the project README, with decoding threshold **0.9**.
36
+ Each entry is **accuracy (%) / tokens per forward pass (TPF)**.
37
+ Baseline results are attributed to the original DMax paper in the project README.
38
+
39
+ | Method | HumanEval-Instruct | MBPP-Instruct |
40
+ | --- | :---: | :---: |
41
+ | DMax-Coder | 83.5 / 7.36 | 79.2 / 5.86 |
42
+ | **DMax-Coder-MMD** | **85.9** / **8.07** | **83.0** / **6.10** |
43
+
44
+ Results can vary with hardware, tensor parallelism, and library versions.
45
+ TPF measures decoding parallelism; wall-clock speed also depends on the runtime.
46
+
47
+ ## Evaluation and inference
48
+
49
+ Use DMax's dInfer evaluation pipeline. After following the evaluation environment
50
+ setup in the [project README](https://github.com/yandex-research/dlm-mmd#installation),
51
+ run from the project repository root:
52
+
53
+ ```bash
54
+ conda activate dinfer
55
+ DOMAIN=code MODEL_PATH=yresearch/DMax-Coder-MMD bash scripts/eval.sh
56
+ ```
57
+
58
+ The project's code evaluation uses threshold **0.9** and evaluates
59
+ HumanEval-Instruct and MBPP-Instruct.
60
+
61
+ The checkpoint includes nine Safetensors shards with BF16 parameters and FP32
62
+ router bias buffers, plus its tokenizer, chat template, and custom model code.
63
+ Direct Transformers loading requires `trust_remote_code=True`. Pass
64
+ `dtype=torch.bfloat16` for BF16 loading; the current configuration declares FP32.
65
+
66
+ Generation requires the DMax/dInfer diffusion decoder. The included custom
67
+ backbone does not implement the standard Transformers `.generate()` interface.
68
+
69
+ ## License and acknowledgements
70
+
71
+ The model follows the **Apache-2.0** license of the
72
+ [DMax-Coder base checkpoint](https://huggingface.co/Zigeng/DMax-Coder-16B)
73
+ and [LLaDA2.0-mini](https://huggingface.co/inclusionAI/LLaDA2.0-mini).
74
+ The included model implementation retains its Apache-2.0 notices.
75
+ The separate MMD training repository is MIT-licensed, with Apache-2.0 third-party components.
76
+
77
+ We thank the authors of [DMax](https://github.com/czg1225/DMax),
78
+ [LLaDA2.0-mini](https://huggingface.co/inclusionAI/LLaDA2.0-mini), and
79
+ [dInfer](https://github.com/inclusionAI/dInfer) for releasing their models, data, and code.
80
+
81
+ ## Citation
82
+
83
+ If you find this work useful in your research, please consider citing:
84
+
85
+ ```bibtex
86
+ @article{drobyshevskiy2026mmd,
87
+ title = {Representation-Space MMD for Diffusion Language Models},
88
+ author = {Drobyshevskiy, Ilya and Sudakov, Ilia and Semenov, Maksim and Kuznedelev, Denis and
89
+ Ignatov, Maksim and Temirchev, Pavel and Balagansky, Nikita and
90
+ Meshchaninov, Viacheslav and Gushchin, Nikita and Baranchuk, Dmitry},
91
+ journal = {arXiv preprint TODO},
92
+ year = {2026}
93
+ }
94
+ ```