grammar-v7c -- run #3 (recipe #3, light do-no-harm nudge) -- VERDICT: FAIL

Trained 2026-10-19 on a free Google Colab T4 from mydeskmate/grammar-v7 (run #1). Same experiment line as v7b (run #2), with the calibration dose cut to about one quarter: identity_weight 1.5 (not 3.0), hard negatives x2 only (no wikitext flood), correction replay increased (coedit x0.1, agentlans x0.02), 1 short epoch, checkpoint selected by best val loss (landed on step 25 of 125).

Official benchmark results (GPU, deterministic scorers)

metric live v5 v7 (run 1) v7b (run 2) v7c (run 3) promotion rule verdict
Do-No-Harm (clean touched) 2.39% 5.00% 1.40% 2.80% (14/500) < 5% PASS
JFLEG GLEU (official, raw) 62.78 63.25 59.64 61.36 >= 62.30 FAIL
CoNLL-2014 F0.5 47.35 52.10 40.25 48.38 (P 58.51 / R 28.58) > 47.35 PASS
BEA-2019 dev F0.5 33.90 35.48 29.65 35.06 (P 43.60 / R 19.65) > 33.90 PASS

Pre-registered promotion rule: promote only if (CoNLL > 47.35 or BEA > 33.90) AND JFLEG GLEU >= 62.30 AND Do-No-Harm < 5%. v7c passes three of four conditions and misses the fluency hold-line by 0.94 GLEU. NOT promoted; the live site stays on v5. Nothing is hidden: this model is published for provenance only.

Reproducibility

A full re-run from scratch (fresh VM, re-mined hard negatives 478/6000 = 7.97% identical, re-trained 125 steps, best val 0.4024085 identical to 7 significant digits) reproduced the official JFLEG score exactly: 61.36. Determinism across VMs confirmed.

Dose ladder (this experiment line)

effective identity pressure GLEU CoNLL F0.5 Do-No-Harm
0% (v7) 63.25 52.10 5.00%
~23% (v7c) 61.36 48.38 2.80%
~77% (v7b) 59.64 40.25 1.40%
Downloads last month
37
Safetensors
Model size
0.2B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for mydeskmate/grammar-v7c

Finetuned
(1)
this model