Shuibai12138 commited on
Commit
c1f445a
·
verified ·
1 Parent(s): b68c4c8

Model card: point to v1.0.3-corrective-training (pinned base-model revision)

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -37,7 +37,7 @@ Code: [zhangshuibai/CDLM](https://github.com/zhangshuibai/CDLM).
37
  | Precision | bf16 base weights, fp32 LoRA parameters and optimiser states, activation checkpointing |
38
  | Seed | 42 |
39
  | Hardware | 4 x A100-PCIE-40GB, 50.8 min |
40
- | Code | `bash training/llada8b_lora/run_train_opencodeinstruct.sh mdlm` at commit `96906ed`; tag `v1.0.2-corrective-training` has the same trainer and `run_train.sh`, and its launcher passes them the same data and arguments (later commits added comments, a shards-per-rank check, rank and position columns in `data_shards.tsv` and stricter link handling) |
41
 
42
  The trainer's `DataLoader` runs two workers over an unsharded `IterableDataset`, so each chunk is used twice in a
43
  row with independent noise; 2,000 steps see 12,000 distinct chunks. This is how every LLaDA-8B adapter of the
@@ -94,7 +94,7 @@ To reproduce the numbers above, use the code repository ([zhangshuibai/CDLM](htt
94
  (`<owner>/<name>[/<subfolder>][@<revision>]`). The commands below pin the revision that holds these weights.
95
 
96
  ```bash
97
- git clone --branch v1.0.2-corrective-training https://github.com/zhangshuibai/CDLM && cd CDLM
98
  pip install -r training/llada8b_lora/requirements.txt
99
  bash training/llada8b_lora/fetch_crb_inputs.sh # the 24 CRB input files of this experiment
100
  # localisation (confidence gap and Top-K hit rate)
 
37
  | Precision | bf16 base weights, fp32 LoRA parameters and optimiser states, activation checkpointing |
38
  | Seed | 42 |
39
  | Hardware | 4 x A100-PCIE-40GB, 50.8 min |
40
+ | Code | `bash training/llada8b_lora/run_train_opencodeinstruct.sh mdlm` at commit `96906ed`; tag `v1.0.3-corrective-training` has the same trainer (plus the pinned base-model revision, the one this run loaded) and `run_train.sh`, and its launcher passes them the same data and arguments (later commits added comments, a shards-per-rank check, rank and position columns in `data_shards.tsv` and stricter link handling) |
41
 
42
  The trainer's `DataLoader` runs two workers over an unsharded `IterableDataset`, so each chunk is used twice in a
43
  row with independent noise; 2,000 steps see 12,000 distinct chunks. This is how every LLaDA-8B adapter of the
 
94
  (`<owner>/<name>[/<subfolder>][@<revision>]`). The commands below pin the revision that holds these weights.
95
 
96
  ```bash
97
+ git clone --branch v1.0.3-corrective-training https://github.com/zhangshuibai/CDLM && cd CDLM
98
  pip install -r training/llada8b_lora/requirements.txt
99
  bash training/llada8b_lora/fetch_crb_inputs.sh # the 24 CRB input files of this experiment
100
  # localisation (confidence gap and Top-K hit rate)