HachimiMT-40-QT

Tiếng Việt

Chinese → Vietnamese translation for xianxia, wuxia and historical web fiction, using the familiar QT/convert register (ta, ngươi, hắn, nàng…). The model has 40,436,352 unique learned parameters (41,026,176 including fixed positional tables).

Release: v1.1, 2026-10-23 (previous: v1.0, 2026-09-21).

What's new in v1.1

Same architecture and tokenizer. v1.1 continues training from v1.0 on new sentences for web slang, idioms, kinship terms, and a subject-role error: when a speaker names themself with a title (朕, 本王, 老夫 …) and then says 不要 or 且, v1.0 often read the sentence as an order to the listener.

Real output, compact FP16 with the helper below (greedy, repetition penalty 1.05):

Source v1.0 v1.1
朕且听你说。 Trẫm hãy nghe ngươi nói. Trẫm tạm nghe ngươi nói.
本王不要看。 Bổn vương đừng nhìn. Bản vương không muốn xem.
…恰好我室友疯狂为你打call… …bạn cùng phòng của ta điên cuồng đánh đập đập ngươi… …bạn cùng phòng của ta điên cuồng cổ vũ ngươi…
…边走边接起电话:“姨父? …vừa đi vừa nghe điện thoại: “Dì ơi?” …vừa đi vừa nhấc máy: “Dượng?”
…但唇亡齿寒的道理,我懂。 …nhưng đạo lý môi vong răng lạnh, ta hiểu. …nhưng đạo lý môi hở răng lạnh, ta hiểu.

Internal checks, v1.0 → v1.1: errors on the targeted terms in held-out sentences 31.6 % → 25.7 %; wrong reading of title + 不要 / 且 sentences 93.2 % → 16.7 %; name reading unchanged or slightly better; general translation quality within 0.3 BLEU on a 1,368-sentence test set.

Still weak: some idioms stay literal (八字没一撇, 放鸽子 → "thả bồ câu"), and a rare item name in quotes can occasionally be translated by meaning instead of read as a name (one case in a hard-name test with the INT8 conversion: 千年灵乳 → "linh nhũ ngàn năm").

The experimental QAT Q4 variant was built for the v1.0 weights only and is not part of v1.1; it remains available with revision="v1.0". Need the previous weights? Pin revision="v1.0", or revision="v1.0.1" for v1.0 plus the CTranslate2 conversion.

Architecture

Property Value
Model Marian encoder–decoder, post-LayerNorm, SiLU FFN
Hidden / FFN size 576 / 2304
Attention heads 8
Encoder Four shared attention/FFN cores, pattern [0,1,2,3] × 3 = 12 passes
LayerNorm Independent parameters for each encoder pass
Decoder 2 layers
Vocabulary Joint Chinese/Vietnamese BPE, 24,000 tokens
Unique parameters, including fixed positional tables 41,026,176
Decoder start / EOS / pad 0 / 2 / 0; start from the actual pad embedding
Positional capacity 512; evaluated input/output caps 256 tokens

Weight sharing reduces stored parameters, not the number of encoder computations. HF's 72.4M badge counts the serialized tensors in the standard Transformers export, which duplicates shared encoder weights across twelve passes. Compact files store each shared tensor once.

Design trade-off. HachimiMT-40-QT is built to shrink the download — about 42 MB as CTranslate2 INT8 or packed Q8, versus about 58 MB for HachimiMT-60/60-QT — while aiming to keep the translation quality of the 60M models. It does not reduce computation: the encoder runs 12 passes (8 layers in HachimiMT-60/60-QT, 6 in HachimiMT-30) with a decoder the same size as HachimiMT-60-QT. In one CPU test with the demo app (INT8, greedy) it took about 1.3× the time of HachimiMT-60/60-QT and about 1.9× that of HachimiMT-30. Choose it when download size or storage matters more than speed.

Files and variants

Path Weight size, decimal MB Purpose
model.safetensors + root config/tokenizer about 145 MB Standard Transformers FP16; twelve materialized passes; no custom model code
packed/f16/ 82.17 MB Compact FP16, shared tensors stored once; reference for integration
packed/q8/ 42.04 MB Per-row Q8 storage, small tensors FP16, logits bias FP32
ct2-int8_float32/ 42.08 MB CTranslate2 INT8 conversion with the corrected decoder start
hachimi40.py — Portable compact loader, weight retie, source/output codec and CPU reference helper
release.json, SHA256SUMS — Model identity, sizes and file integrity

Only the weights are counted above; tokenizer/config/helper files add overhead. Q8 is a storage format: the helper expands it through FP16 and computes in FP32 by default. Expansion does not recover precision lost by quantization. It is not a native INT8 kernel.

The experimental QAT Q4 variant (packed/qat-q4g32/, 32.20 MB) exists only for the v1.0 weights: use revision="v1.0".

Quick start: compact FP16 on CPU

Download only the chosen representation instead of the entire multi-variant repository:

pip install "torch>=2.8" "transformers==5.8.0" "safetensors==0.7.0" "sentencepiece==0.2.1" "huggingface_hub>=1.0"
import sys
import torch
from huggingface_hub import snapshot_download

folder = snapshot_download(
    "ngocdang83/HachimiMT-40-QT",
    revision="v1.1",
    allow_patterns=[
        "hachimi40.py", "source.spm", "target.spm", "vocab.json",
        "tokenizer_config.json", "special_tokens_map.json",
        "target_normalization.json", "packed/f16/*",
    ],
)
sys.path.insert(0, folder)
from hachimi40 import load_compact, translate

torch.set_num_threads(2)
model, tokenizer = load_compact(folder, "f16")  # shared weights, CPU FP32
print(translate(model, tokenizer, ["他抬头看向远处的山门。"]))
print(translate(model, tokenizer, ["山门已开。\n弟子入内。"]))

The helper is ordinary Python source supplied with the release; inspect it before importing downloaded code. No Transformers trust_remote_code=True is required. For Q8, change both packed/f16/* and the loader variant to q8.

The root checkpoint also loads directly with AutoTokenizer.from_pretrained(...) and MarianMTModel.from_pretrained(..., dtype=torch.float16).float() on CPU. Load FP16 first, then upcast: Transformers regenerates Marian's fixed sinusoidal positions in the requested load dtype, so loading directly as FP32 would bypass the evaluated FP16 rounding of those tables. It contains FP16-rounded weights even when computed in FP32. retie_encoder(model) from the helper can restore physical core sharing after loading, and is required before continuing the intended tied-weight training recipe. The plain root model is inference-equivalent without this retie.

Standard Transformers checkpoint

import sys
from pathlib import Path
import torch
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer, MarianMTModel

repo, rev = "ngocdang83/HachimiMT-40-QT", "v1.1"
sys.path.insert(0, str(Path(hf_hub_download(repo, "hachimi40.py", revision=rev)).parent))
from hachimi40 import translate  # wraps generate() with encode_source / decode_target

tokenizer = AutoTokenizer.from_pretrained(repo, revision=rev)
model = MarianMTModel.from_pretrained(repo, revision=rev, dtype=torch.float16).float().eval()
print(translate(model, tokenizer, ["山门已开。\n弟子入内。"]))

Hugging Face's auto-generated pipeline("translation") snippet does not apply here: Transformers 5 no longer has that pipeline task, and calling generate() without the codec leaves a space after opening quotes (“ Ngươi…) and merges source lines into one.

Tokenizer and decoding contract

Do not substitute the 30-QT/60-QT tokenizer. <NL> is an ordinary retained SentencePiece user-defined piece, ID 4, not a Hugging Face special token.

  • Apply encode_source once before tokenization: source line breaks → <NL>.
  • Apply decode_target once after detokenization: <NL> → line breaks, then undo the training convention of one space after an opening quote/bracket.
  • config.json and target_normalization.json name this codec by its path in the training codebase (scripts/postprocess/joint_loop_codec.py), which is not part of this repository; encode_source and decode_target in hachimi40.py are the equivalent implementation.
  • Preserve source text and explicit names/titles. A different QT pronoun choice is not by itself a semantic error.
  • The reference helper uses greedy decoding, repetition penalty 1.05, no n-gram ban and 256-token caps. It rejects overlong sources instead of silently truncating and raises if generation reaches its cap without EOS. Split long text explicitly.
  • Chunk layout preservation is not guaranteed; check line alignment if the application relies on it.

CTranslate2: ct2-int8_float32/ is an INT8 conversion of the same v1.1 checkpoint, made with the corrected decoder start described below. Load it with ctranslate2.Translator, tokenize with the root source.spm/target.spm plus </s>, and apply the same encode_source/decode_target codec. It is the artifact used by the HachimiMT demo Space.

If you convert the checkpoint yourself: CTranslate2's ordinary Marian conversion can use a zero decoder-start embedding. This checkpoint requires the real <pad> embedding; an uncorrected conversion can change translations substantially.

Quality summary

Q8 stays close to FP16 in internal CPU checks (v1.1: on 400 test sentences, BLEU 53.65 → 53.55 and identical output for 85 % of sentences). FP16 or Q8 is recommended for regular use. All variants can make translation mistakes, especially with names and uncommon expressions. Native WebGPU quality and performance have not yet been measured.

License and attribution

CC BY 4.0, matching the HachimiMT-60-QT release policy. Attribute ngocdang83/HachimiMT-40-QT, link the license, and indicate modifications. See LICENSE.md. This license covers the released weights/helper, not a redistribution or relicensing of the training corpus.

Downloads last month
49
Safetensors
Model size
72.4M params
Tensor type
F32
·
F16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Spaces using ngocdang83/HachimiMT-40-QT 2