Model card: correct usage, base-model attribution, gradient implementation note, reference results

README.md

@@ -33,23 +33,43 @@ This model is part of the code release of *Corrective Diffusion Language Models*
 | Frozen | `lm_head`, `embed_tokens` |
 | Seed | 42 |
 | Hardware | 4 x A100-PCIE-40GB, about 19 minutes |
-| Code | [zhangshuibai/CDLM](https://github.com/zhangshuibai/CDLM), trained at commit `5e52812`; tag `v1.0-corrective-training` contains the same training code. Command: `ARM=cdlm bash training/scripts/train_0.5b_opencodeinstruct.sh` |
+| Code | [zhangshuibai/CDLM](https://github.com/zhangshuibai/CDLM), trained at commit `5e52812`; tags `v1.0-corrective-training` and `v1.0.1-corrective-training` contain the same training code. Command: `ARM=cdlm bash training/scripts/train_0.5b_opencodeinstruct.sh` |
 
 `training_config.yaml` in this repository is the fully resolved configuration the trainer saved for this run.
 
+## Reference results
+
+Scored with the repository's launchers (CRB: 4 datasets x 3 error types at n_replace 1, macro over the 12 cells, confidence threshold 0.9; code generation: n=10 samples, vanilla decoding). The 8 s execution timeout can flip one slow HumanEval+ program, which moves a CRB Pass@1 by at most about 0.001.
+
+| | Pass@1 T=1 | T=2 | T=3 | T=4 | confidence gap | Top-1 | Top-3 | Top-5 |
+|---|---|---|---|---|---|---|---|---|
+| CRB | 0.2196 | 0.3018 | 0.3029 | 0.3070 | 0.1919 | 0.2517 | 0.4883 | 0.6495 |
+
+HumanEval (vanilla decoding): pass@1 0.2165, pass@10 0.4146.
+
+The matched MDLM control on the same data scores 0.1401 / 0.2226 / 0.2352 / 0.2387 (Pass@1 T=1..4), gap 0.0987, Top-1 0.1445 on the same CRB cells.
+
 ## Usage
 
-This is a masked diffusion language model with bidirectional attention. Loading it with `AutoModelForCausalLM` gives a causal Qwen2 model and wrong outputs. Use the evaluation pipeline of the [code repository](https://github.com/zhangshuibai/CDLM), which loads models whose name contains `open-dcoder` with the diffusion Qwen2 implementation (bidirectional attention, shifted logits):
+This is a masked diffusion language model with bidirectional attention. Loading it with `AutoModelForCausalLM` gives a causal Qwen2 model and wrong outputs. Evaluate it with the launchers of the [code repository](https://github.com/zhangshuibai/CDLM) (environment: `evaluation/ENVIRONMENT.md`; pin the model revision listed in `evaluation/README.md`):
 
 ```bash
-git clone https://github.com/zhangshuibai/CDLM && cd CDLM
-# see README.md for the evaluation environment and commands, then pass
-#   --model_name Shuibai12138/Open-Dcoder-0.5B-CDLM-OpenCodeInstruct
+git clone --branch v1.0.1-corrective-training https://github.com/zhangshuibai/CDLM && cd CDLM
+# CRB localisation and correction (n_replace 1: the 48 headline cells)
+bash evaluation/crb/run_crb.sh Shuibai12138/Open-Dcoder-0.5B-CDLM-OpenCodeInstruct cdlm_oci --gpus 0 --nr 1
+# from-scratch code generation (HumanEval, HumanEval+, MBPP, MBPP+)
+bash evaluation/codegen/run_codegen_eval.sh Shuibai12138/Open-Dcoder-0.5B-CDLM-OpenCodeInstruct outputs/cdlm_oci 0
 ```
 
+The launchers load the diffusion Qwen2 implementation because the repository id contains `open-dcoder`.
+
+## Implementation note on the gradient
+
+The 0.5B training code computes the per-token cross-entropy terms with `LigerFusedLinearCrossEntropyLoss(reduction="none")` from liger-kernel 0.5.8. Its backward pass scales every token's gradient by the upstream gradient of the first token, so the per-token weights of the objective (the 1/|S| and 1/t factors and the noise-term weight) are not applied in the update: each micro-batch receives the first token's weight times the unweighted sum of the per-token gradients, a micro-batch whose first target token is unsupervised receives no gradient, and when the first target token is a replaced one the noise term adds gradient on every valid position, clean tokens included. Logged losses are correct. This model was trained with that code; see `training/README.md` ("Effective gradient") in the code repository.
+
 ## Licence and attribution
 
-Model weights: MIT. Trained on [nvidia/OpenCodeInstruct](https://huggingface.co/datasets/nvidia/OpenCodeInstruct) (CC BY 4.0, NVIDIA). Base model: [fredzzp/open-dcoder-0.5B](https://huggingface.co/fredzzp/open-dcoder-0.5B).
+Model weights: MIT. Base model: [fredzzp/open-dcoder-0.5B](https://huggingface.co/fredzzp/open-dcoder-0.5B), licensed under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0); this model is a derivative (continued training) of it, and the base model's licence and notices apply to the parts derived from it. Trained on [nvidia/OpenCodeInstruct](https://huggingface.co/datasets/nvidia/OpenCodeInstruct) (CC BY 4.0, NVIDIA).
 
 ## Citation