grammar-v7c -- run #3 (recipe #3, light do-no-harm nudge) -- VERDICT: FAIL
Trained 2026-10-19 on a free Google Colab T4 from mydeskmate/grammar-v7 (run #1).
Same experiment line as v7b (run #2), with the calibration dose cut to about one
quarter: identity_weight 1.5 (not 3.0), hard negatives x2 only (no wikitext
flood), correction replay increased (coedit x0.1, agentlans x0.02), 1 short
epoch, checkpoint selected by best val loss (landed on step 25 of 125).
Official benchmark results (GPU, deterministic scorers)
| metric | live v5 | v7 (run 1) | v7b (run 2) | v7c (run 3) | promotion rule | verdict |
|---|---|---|---|---|---|---|
| Do-No-Harm (clean touched) | 2.39% | 5.00% | 1.40% | 2.80% (14/500) | < 5% | PASS |
| JFLEG GLEU (official, raw) | 62.78 | 63.25 | 59.64 | 61.36 | >= 62.30 | FAIL |
| CoNLL-2014 F0.5 | 47.35 | 52.10 | 40.25 | 48.38 (P 58.51 / R 28.58) | > 47.35 | PASS |
| BEA-2019 dev F0.5 | 33.90 | 35.48 | 29.65 | 35.06 (P 43.60 / R 19.65) | > 33.90 | PASS |
Pre-registered promotion rule: promote only if (CoNLL > 47.35 or BEA > 33.90) AND JFLEG GLEU >= 62.30 AND Do-No-Harm < 5%. v7c passes three of four conditions and misses the fluency hold-line by 0.94 GLEU. NOT promoted; the live site stays on v5. Nothing is hidden: this model is published for provenance only.
Reproducibility
A full re-run from scratch (fresh VM, re-mined hard negatives 478/6000 = 7.97% identical, re-trained 125 steps, best val 0.4024085 identical to 7 significant digits) reproduced the official JFLEG score exactly: 61.36. Determinism across VMs confirmed.
Dose ladder (this experiment line)
| effective identity pressure | GLEU | CoNLL F0.5 | Do-No-Harm |
|---|---|---|---|
| 0% (v7) | 63.25 | 52.10 | 5.00% |
| ~23% (v7c) | 61.36 | 48.38 | 2.80% |
| ~77% (v7b) | 59.64 | 40.25 | 1.40% |
- Downloads last month
- 37
Model tree for mydeskmate/grammar-v7c
Base model
mydeskmate/grammar-v7