German dictation punctuation experimental checkpoint

Restore word-final punctuation from German dictation. The head classifies the last subword of each whitespace-delimited word. The decoder inserts marks at original offsets and preserves source characters. The example rejects inputs above 256 tokens including special tokens.

Selected variant all-marks by the frozen validation rule. All parameters were updated on one NVIDIA A100-SXM4-80GB. See provenance.json for initialization, label policy, data revisions, seed and controlled alternatives.

Candidate Mark F1 Added incorrect marks Missed marks Changed-word documents Flagged negatives
final-only 29.5% 32 140 0/112 7/16
all-marks 45.7% 40 112 0/112 8/16
horizon 77.4% 33 44 0/112 9/16
copy-input 0.0% 0 176 0/112 0/16

Final-only trains on sentence-ending marks. All-marks also learns marks inside the dictated message. Horizon is the existing punctuation-restoration-small model. Copy-input returns the input unchanged.

On the 347 public FLEURS German references, mark F1 was 42.0% versus 92.9% for Horizon. Internal label supervision improved the synthetic result over final-only training, but this experimental checkpoint ranked below Horizon on both evaluations.

The training messages and newly generated test messages are fictional. Repeated slot draws within a template are correlated. Split families are disjoint but use the same authoring process. These results do not measure protection or dictation quality on customer messages.

Download and use

Fine-tuned by Mahir Isikli (EasyOctopus). The companion dataset is EasyOctopus/german-dictation-punctuation-data.

Load the tokenizer and classification head with standard Transformers. The model runs locally on a CPU or GPU.

from transformers import AutoModelForTokenClassification, AutoTokenizer

repo_id = "EasyOctopus/german-dictation-punctuation-model"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForTokenClassification.from_pretrained(repo_id).eval()

For the exact decoder used in the evaluation, download the repository and run the included example. This needs Python 3.11 or later, PyTorch and Transformers. It sends no message text to the Hub.

import subprocess
import sys
from pathlib import Path
from huggingface_hub import snapshot_download

folder = Path(snapshot_download("EasyOctopus/german-dictation-punctuation-model", local_dir="german-dictation-punctuation"))
subprocess.run([sys.executable, str(folder / "infer.py"), "könnt ihr mir sagen wann der zug ankommt"], check=True)
subprocess.run([sys.executable, str(folder / "infer.py"), "--self-test"], check=True)

See LICENSE and NOTICE for source attribution. The test predictions and validation selection are documented in eval_results.json and provenance.json.

Retrain the controlled variants

Download this repository with snapshot_download as shown above. From the downloaded directory, run the commands below. The script fetches only the training and validation files it needs from the companion dataset. It verifies the initializer and input fingerprints before fitting. Use --data-dir to use a local dataset checkout.

uv run --script training/retrain.py --prepare-only --output smoke-results
CUDA_VISIBLE_DEVICES=0 uv run --script training/retrain.py --variant all --output ablation-results

The script downloads distilbert/distilbert-base-german-cased at revision 6f96562de9886779ce7d48810dff8f5c4afb6d0b, verifies its files, then initializes the punctuation head with the study seed. Initial weight SHA-256: 764a3ad5aa5f599cd1a93619eba403fb1cce7ad61b5abefd0e2bc129bc34ded5. Each variant uses seed 20261118, 3 epochs, batch size 24 and AdamW with a constant learning rate of 5e-05, weight decay 0.01 and gradient clipping at norm 1. Training uses BF16 autocast when the card supports it. The script retains every final-epoch checkpoint and records the validation-selected variant. CUDA, hardware and library differences can affect numerical results. The original run used PyTorch 2.6.0 with CUDA 12.4 and Transformers 4.49.0.

Downloads last month
16
Safetensors
Model size
66.8M params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for EasyOctopus/german-dictation-punctuation-model

Finetuned
(25)
this model

Dataset used to train EasyOctopus/german-dictation-punctuation-model