Update README.md for the distillation release

README.md

@@ -22,9 +22,9 @@ widget:
 
 English | [中文](README_zh.md)
 
-A 143M-parameter bidirectional Chinese↔English translation model, instruction-tuned on the Haidass1.5-143M base — the strongest zh⇄en translator at this scale among general chat-architecture models. The current V3.2 release adds a quality-remix anneal (Stage 2R) on top of the main SFT stage and ships a light repetition-penalty (`repetition_penalty=1.005`) release decoding config.
+A 143M-parameter bidirectional Chinese↔English translation model, instruction-tuned on the Haidass1.5-143M base — the strongest zh⇄en translator at this scale among general chat-architecture models. The current release is trained in three stages (main SFT → quality-remix anneal → distillation from a 27B teacher) and offers two decoding flavors on the same weights: a throughput-friendly **greedy** row and a quality-first **beam** row.
 
-Live demo: [Haidass Translate on Hugging Face Spaces](https://huggingface.co/spaces/umeiko/haidass-translate-app).
+Live demo: [Haidass Translate on Hugging Face Spaces](https://huggingface.co/spaces/umeiko/haidass-translate-app) — greedy / beam / side-by-side compare.
 
 ## FLORES-200 dev
 
@@ -32,50 +32,50 @@ Live demo: [Haidass Translate on Hugging Face Spaces](https://huggingface.co/spa
 |---|---:|:---:|---:|---:|---:|---:|
 | HY-MT1.5-1.8B | 1800M | LLM | 44.65 | 30.98 | 27.68 | 57.96 |
 | Qwen3-0.6B | 600M | LLM | 30.94 | 21.10 | 20.21 | 48.62 |
-| OPUS-MT en-zh | 78M | Seq2Seq | 30.88 | 21.80 | - | - |
-| OPUS-MT zh-en | 78M | Seq2Seq | - | - | 22.99 | 51.03 |
+| **Haidass-Translate-143M-beam** † | **143M** | **LLM** | **30.36** | **21.40** | **20.62** | **46.51** |
 | Qwen2.5-0.5B-Instruct | 500M | LLM | 28.96 | 19.65 | 18.09 | 45.85 |
-| **Haidass-Translate-143M (V3.2)** † | **143M** | **LLM** | **28.56** | **19.58** | **17.86** | **44.05** |
+| **Haidass-Translate-143M** | **143M** | **LLM** | **28.32** | **19.49** | **17.72** | **44.32** |
 | M2M-100-418M | 418M | Seq2Seq | 28.04 | 20.53 | 20.58 | 48.79 |
 | NLLB-200-distilled-600M | 600M | Seq2Seq | 22.44 | 16.74 | 25.71 | 52.28 |
 | Drafter-143M* | 143M | LLM | 12.04 | 9.43 | 5.47 | 27.31 |
 
 *Drafter-143M: a control model with identical configuration, data and training recipe, except that it starts from random initialization instead of the pretrained base — used to quantify the contribution of base-model pretraining.
 
-OPUS-MT models are single-directional — one independent 78M model per direction; "-" marks directions a model does not serve.
-
-† V3.2 scores use the shipped release decoding config (greedy + `repetition_penalty=1.005`); the same checkpoint under pure greedy scores 28.47 / 19.48 / 17.12 / 43.61. All other models are measured with pure greedy decoding. The previous release (4ep) scored 27.56 / 19.27 / 17.17 / 43.20 on this row.
+† The beam row uses beam search:
+```
+en→zh: num_beams=3, length_penalty=1.25
+zh→en: num_beams=4, length_penalty=0.9
+common: do_sample=false, repetition_penalty=1.0, no_repeat_ngram_size=0, max_new_tokens=256
+```
 
 ## FLORES+ devtest
 
-The same models re-evaluated on FLORES+ devtest (released 2026; zero overlap with dev). The V3.2 candidate was evaluated only once per decoding config, after selection was frozen on dev:
+The same models re-evaluated on FLORES+ devtest (released 2026; zero overlap with dev). The candidate was evaluated only once per decoding config, after model selection and decoding parameters were frozen on dev:
 
 | Model | Params | Arch | en→zh BLEU | en→zh chrF++ | zh→en BLEU | zh→en chrF++ |
 |---|---:|:---:|---:|---:|---:|---:|
 | HY-MT1.5-1.8B | 1800M | LLM | 37.36 | 26.08 | 20.33 | 51.48 |
-| OPUS-MT en-zh | 78M | Seq2Seq | 32.23 | 22.40 | - | - |
-| OPUS-MT zh-en | 78M | Seq2Seq | - | - | 23.06 | 51.03 |
 | Qwen3-0.6B | 600M | LLM | 31.76 | 21.48 | 19.66 | 48.14 |
-| **Haidass-Translate-143M (V3.2)** † | **143M** | **LLM** | **29.21** | **19.97** | **17.51** | **43.87** |
+| **Haidass-Translate-143M-beam** † | **143M** | **LLM** | **29.81** | **21.02** | **18.69** | **45.80** |
 | Qwen2.5-0.5B-Instruct | 500M | LLM | 29.32 | 19.95 | 18.04 | 46.00 |
+| **Haidass-Translate-143M** | **143M** | **LLM** | **29.23** | **19.90** | **17.50** | **44.01** |
 | M2M-100-418M | 418M | Seq2Seq | 28.29 | 20.60 | 19.52 | 47.87 |
 | NLLB-200-distilled-600M | 600M | Seq2Seq | 23.07 | 16.94 | 24.30 | 51.48 |
 | Drafter-143M* | 143M | LLM | 10.93 | 9.00 | 5.82 | 26.83 |
 
-† Pure-greedy numbers for the same checkpoint: 29.08 / 19.86 / 16.48 / 43.22.
-
-devtest sentences do not overlap with dev. This model's scores on the new split are essentially unchanged (en→zh 28.56→29.21, zh→en 17.86→17.51), indicating that the results reflect translation ability rather than memorization of a specific benchmark.
+† Beam parameters are identical to the dev table footnote (frozen on dev before this single devtest run).
 
 ## Decontamination
 
-To verify that the scores contain no test-set leakage, we audited all 15.83M training samples: every sentence is cut into consecutive fragments (8 words for English, 10 characters for Chinese), and any training sample sharing any fragment with any test sentence is counted as a hit. Results: 1,147 hits (0.0072%) against FLORES-200 dev, 1,788 (0.0113%) against FLORES+ devtest. Manual inspection shows the hits are common-phrase-level fragment overlaps rather than full-sentence leakage — i.e., the reported scores are not inflated by leakage. The Stage 2R anneal pool added in V3.2 was filtered with the same rule at build time. Audit report (top-50 overlapping samples included for inspection): [audit_report.json](https://huggingface.co/datasets/umeiko/Haidass-Translate-143M-eval/blob/main/audit_report.json) (devtest audit: audit_floresplus_devtest.json in the same repo).
+To verify that the scores contain no test-set leakage, we audited all 15.83M training samples: every sentence is cut into consecutive fragments (8 words for English, 10 characters for Chinese), and any training sample sharing any fragment with any test sentence is counted as a hit. Results: 1,147 hits (0.0072%) against FLORES-200 dev, 1,788 (0.0113%) against FLORES+ devtest. Manual inspection shows the hits are common-phrase-level fragment overlaps rather than full-sentence leakage — i.e., the reported scores are not inflated by leakage. The Stage 2 anneal pool and the Stage 3 distillation pool were built from the same audited sources under the same filtering rule (distillation targets are teacher-generated translations of already-audited source sentences). Audit report (top-50 overlapping samples included for inspection): [audit_report.json](https://huggingface.co/datasets/umeiko/Haidass-Translate-143M-eval/blob/main/audit_report.json) (devtest audit: audit_floresplus_devtest.json in the same repo).
 
-## Training recipe (V3.2)
+## Training recipe
 
 - Base: Haidass1.5-143M (Qwen3 architecture: 30 layers, hidden 576, GQA 9/3, vocab 64,000)
-- Stage 1c (main SFT): 7.837M cleaned zh↔en parallel sentence pairs (15.67M bidirectional samples, translation-only, no general-domain data); official MindSpeed-LLM `--pack --neat-pack` (607,622 full 2048-token sequences with inter-document attention-mask isolation); 16×Ascend 910C, GBS=256, lr 3e-5 cosine on a 5-epoch schedule, 11,867 steps ≈ 6.2B tokens; checkpoint selected on dev: iter_10692 (4.50 epochs)
-- Stage 2R (quality-remix anneal): 715,658 curated pairs — web-mined columns with alignment noise and traditional-Chinese/Cantonese contamination dropped (CCMatrix/CCAligned/WikiMatrix/WikiTitles/MASSIVE), TED and OpenSubtitles Chinese re-translated by Qwen3.8-27B with rule-based gates, BWB web novels book-stratified to 48k, human-authored CoVoST2/BSTC/News/Wikimedia/UN kept; target-side supervision tokens balanced exactly 50/50; 1 epoch = 167 steps from the Stage 1c checkpoint, lr 3e-6→3e-7 cosine; final selection iter167
-- Framework: MindSpeed-LLM v2.3.0 + Megatron-LM core_v0.12.1 (NPU)
+- Stage 1 (main SFT): 7.837M cleaned zh↔en parallel sentence pairs (15.67M bidirectional samples, translation-only, no general-domain data); packing uses the official MindSpeed-LLM `--pack --neat-pack` (607,622 full 2048-token sequences with inter-document attention-mask isolation); 16×Ascend 910C, GBS=256, lr 3e-5 cosine on a 5-epoch schedule, 11,867 steps ≈ 6.2B tokens; checkpoint selected on dev at 4.50 epochs
+- Stage 2 (quality-remix anneal): 715,658 curated pairs — web-mined corpora with alignment noise and heavy traditional-Chinese/Cantonese contamination dropped; TED/OpenSubtitles Chinese re-translated by Qwen3.8-27B; added 48k BWB web-novel pairs (not used in Stage 1), plus CoVoST2/BSTC/News/Wikimedia/UN corpora; continued from the Stage 1 checkpoint, lr 3e-6→3e-7 cosine
+- Stage 3 (distillation): sequence-level distillation from a Qwen3.8-27B teacher — 70% teacher-generated targets + 30% original corpus data, 1,579,486 samples, 60,706 packed rows; continued from the Stage 2 checkpoint, 119 steps ≈ 0.5 epoch, GBS=256, lr 2e-6→2e-7 cosine
+- Training framework: MindSpeed-LLM v2.3.0 + Megatron-LM core_v0.12.1 (NPU)
 
 ## Usage
 
@@ -87,45 +87,62 @@ tok = AutoTokenizer.from_pretrained("DALabCommunity/Haidass-Translate-143M")
 
 msgs = [{"role": "user", "content": "将以下文本翻译为英文：光子甚至比构成原子的物质还要小！"}]
 ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
-out = model.generate(ids, max_new_tokens=512, do_sample=False, repetition_penalty=1.005)
+out = model.generate(ids, max_new_tokens=256, do_sample=False)
 print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
-# Even photons are smaller than the stuff that makes up atoms!
+# Photons are even smaller than the stuff that makes atoms!
 ```
 
-The shipped `generation_config.json` already carries `do_sample=false, num_beams=1, repetition_penalty=1.005`, so plain `model.generate(ids)` reproduces the benchmark configuration.
+The shipped `generation_config.json` is greedy decoding (`do_sample=false, num_beams=1, repetition_penalty=1.0`); plain `model.generate(ids)` reproduces the greedy scores.
 
-Note: the training data follows the qwen3 chat template (an empty `<think></think>` block precedes the assistant turn). Always use the model's built-in `chat_template` at inference; do not hand-craft prompts.
+To reproduce the beam row, use direction-specific beam search:
 
-## Translation samples (V3.2, spot check on FLORES-200 dev)
+```python
+# English → Chinese
+out = model.generate(ids, do_sample=False, num_beams=3, length_penalty=1.25,
+                     repetition_penalty=1.0, max_new_tokens=256)
+# Chinese → English
+out = model.generate(ids, do_sample=False, num_beams=4, length_penalty=0.9,
+                     repetition_penalty=1.0, max_new_tokens=256)
+```
 
-en→zh:
+Beam costs roughly 3–4× the compute of greedy; use the greedy configuration for high-throughput serving.
 
-> Src: Latvia and Slovakia have both delayed the process of joining ACTA.
-> Out: 拉脱维亚和斯洛伐克都推迟了加入ACTA进程。 (sentence chrF++ 73.4 — the previous release mistranslated 斯洛伐克 as Slovenia; fixed)
+Note: the training data follows the qwen3 chat template (an empty `<think></think>` block precedes the assistant turn). Always use the model's built-in `chat_template` at inference.
+
+## Translation samples (current release, spot check on FLORES-200 dev, beam decoding)
 
+en→zh:
+```
+> Src: Latvia and Slovakia have both delayed the process of joining ACTA.
+> Out: 拉脱维亚和斯洛伐克都推迟了加入ACTA的进程。 (sentence chrF++ 85.7)
+```
+```
 > Src: The trial took place at Birmingham Crown Court and concluded on August 3.
 > Out: 审判在伯明翰皇家法院进行，并于8月3日结束。 (sentence chrF++ 73.7)
+```
 
 zh→en:
-
-> Src: 然而，第一个真正的望远镜是 16 世纪晚期在欧洲制造的。
-> Out: However, the first true telescope was made in Europe in the late 16th century. (sentence chrF++ 87.9)
-
+```
 > Src: 这些代理人负责根据《巴基斯坦宪法》第 247 条提供政府和司法服务。
-> Out: These agents are responsible for providing government and judicial services under article 247 of the Pakistan Constitution. (sentence chrF++ 90.8 — the previous release dropped "第 247 条"; now kept)
+> Out: These agents are responsible for providing government and judicial services under article 247 of the Pakistan Constitution. (sentence chrF++ 90.8)
+```
+```
+> Src: 循环系统的主要器官是心脏，心脏负责输送血液。
+> Out: The main organ of the circulatory system is the heart, which pumps blood. (sentence chrF++ 82.8)
+```
 
 ## Known limitations
 
-- zh→en trails en→zh by ~11 BLEU — a pattern shared by every model in the tables, not specific to this model
-- Typical residual errors: occasional entity mix-ups and numeric-detail slips; repetitive looping is largely mitigated by the shipped `repetition_penalty=1.005`, though rare-entity-dense inputs may still degrade
+- zh→en trails en→zh — a pattern shared by every model in the tables, not specific to this model
+- Typical residual errors: occasional entity mix-ups and numeric-detail slips
 - Optimized for zh⇄en translation only; not a general chat model
 
 ## Evaluation
 
 - Metrics: sacreBLEU corpus BLEU (`tokenize=zh` for Chinese targets, `tokenize=13a` for English) + chrF++ (`word_order=2`); prompts byte-identical to the training chat template
 - Benchmark sources: [openlanguagedata/flores_plus](https://huggingface.co/datasets/openlanguagedata/flores_plus) (FLORES+, the maintained version; gated — auto-approved after accepting terms; Simplified Chinese now `cmn_Hans`); [facebook/flores](https://huggingface.co/datasets/facebook/flores) (original archive, unmaintained); login-free mirror [facebookresearch/flores](https://github.com/facebookresearch/flores)
-- Raw predictions: the `eval/` directory in this repo contains per-sentence prediction jsonl (`{direction, src, ref, hyp}`, one file per model per benchmark; V3.2 files are `pred_haidass-v32s2r-iter167_*`, one set per decoding config) for every model in the tables above — all scores can be recomputed with sacreBLEU
+- Raw predictions: the `eval/` directory in this repo contains per-sentence prediction jsonl (`{direction, src, ref, hyp}`, one file per model per benchmark; this release ships `pred_haidass-143m_greedy_*` and `pred_haidass-143m_beam_*`, one set per decoding config) for every model in the tables above — all scores can be recomputed with sacreBLEU
 - Full results & reproduction: the 16-model comparison table, per-sentence predictions, decontamination audits and evaluation scripts live in the companion dataset
   [umeiko/Haidass-Translate-143M-eval](https://huggingface.co/datasets/umeiko/Haidass-Translate-143M-eval)
 
-<sub>Note: all scores are measured on the FLORES Chinese–English subset (eng_Latn ↔ zho_Hans), bidirectional (997 sentences for dev, 1,012 for devtest). V3.2 scores use the shipped release decoding config (greedy + repetition_penalty=1.005); all other models use pure greedy decoding.</sub>
+<sub>Note: all scores are measured on the FLORES Chinese–English subset (eng_Latn ↔ zho_Hans), bidirectional (997 sentences for dev, 1,012 for devtest). The plain 143M row uses pure greedy decoding (`do_sample=false, num_beams=1, repetition_penalty=1.0`); the 143M-beam row uses direction-specific deterministic beam search (en→zh `num_beams=3, length_penalty=1.25`; zh→en `num_beams=4, length_penalty=0.9`); all other models use pure greedy decoding.</sub>