FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
Paper • 2506.20920 • Published • 81
SalamandraTA Model Card
SalamandraTA-7b-instruct-WMT26 is a machine translation specialized LLM based on Salamandra-7B, a decoder-only Llama-style architecture. The model was specifically created for the WMT26 General Translation Shared Task submission and released solely for non-commercial research purposes.
Training proceeded in four stages, ordered by data availability:
1. CPT1 — Monolingual continual pre-training with vocabulary adaptation (extended to 256K tokens): covered Salamandra's original language set plus the new WMT 2026 language pairs, with a cap of approximately 1 billion tokens per language (44.91 billion tokens total). Ladin and Ligurian were excluded at this stage due to insufficient monolingual data; Northern Sámi was included with only about 30 million tokens, and Thai was not included because its sample was not ready in time.
2. CPT2 — Mixed monolingual and parallel continual pre-training: narrowed to the WMT 2026 translation directions, combining parallel bitexts with a monolingual top-up per target language (85.1 billion tokens including template overhead, split into two shards, of which only the first was trained on before the compute allocation ran out). Ladin, Ligurian, and Northern Sámi remained excluded due to lack of data.
3. SFT — Supervised fine-tuning: covered the WMT 2026 language set, now including Ligurian, and limited Northern Sámi and Ladin data.
4. PO — Preference optimization (CPO with the SimPO loss, updating all model parameters): applied to six weak directions. For languages where the model already generated fluently (e.g., Ligurian), preference pairs were built from Diverse Beam Search candidates re-ranked by MBR. For languages whose raw generations were degenerate — Ladin, Northern Sámi, Egyptian Arabic, and lower-BLEU Belarusian and Thai — supervised preference pairs were used instead (model's own output as rejected, reference translation as chosen), to avoid reinforcing repetition loops and prompt-echoing.
Note that English to Ladin and English to Ligurian were not included in continual pre-training and are supported through instruction tuning only, relying on transfer from Italian. English to Northern Sámi had only about 30M tokens of Northern Sámi monolingual data in the first continual pre-training stage, was left out of the second, and is not covered by our internal evaluation.
To extend the language coverage, we performed vocabulary adaptation by training a new tokenizer from scratch (SentencePiece BPE) on a corpus spanning both the original Salamandra languages and the ten languages newly introduced for WMT 2026, so that the merge budget is reallocated jointly across all of them. Tokens shared with the original vocabulary retained their embeddings; the remainder were initialized from a Gaussian derived from the mean of the existing embeddings. The vocabulary size remains 256,000, so no architectural change was required.
For further details, we refer to our system description paper, BSC Submission for WMT26 General Machine Translation Shared Task (WMT 2026).
DISCLAIMER: This version of Salamandra is tailored exclusively for translation tasks. It lacks chat capabilities and has not been trained with any chat instructions.
Model Details
Architecture
| Total Parameters | 7,768,117,248 |
| Embedding Parameters | 1,048,576,000 |
| Layers | 32 |
| Hidden size | 4,096 |
| Attention heads | 32 |
| Context length | 8,192 |
| Vocabulary size | 256,000 |
| Precision | bfloat16 |
| Embedding type | RoPE |
| Activation Function | SwiGLU |
| Layer normalization | RMS Norm |
| Flash attention | ✅ |
| Grouped Query Attention | ✅ |
| Num. query groups | 8 |
Intended Use
Direct Use
The Model can be used for general machine translation in the 23 WMT 2026 directions listed under How to use. Other languages seen only in monolingual continual pre-training are not supported as translation directions. The model is released under a Research-Only License that regulates its use. By accessing or using the Model, you acknowledge that you have read, understood, and agree to be bound by the terms of use defined in the LICENSE.
Hardware and Software
Training Framework
Salamandra-7b was continually pre-trained using NVIDIA’s NeMo Framework, which leverages PyTorch Lightning for efficient model training in highly distributed settings.
SalamandraTA-7b-instruct-WMT26 was fully fine-tuned with FastChat and DeepSpeed ZeRO-3. Preference optimization (CPO with the SimPO loss, full-parameter) was applied with TRL and DeepSpeed ZeRO-3 with CPU offload. (TRL 0.22.1, Transformers 4.57.1, DeepSpeed 0.16.8, PyTorch 2.7.0).
Compute Infrastructure
All models were trained on MareNostrum 5, a pre-exascale EuroHPC supercomputer hosted and operated by Barcelona Supercomputing Center.
The accelerated partition is composed of 1,120 nodes with the following specifications:
- 4x Nvidia Hopper GPUs with 64GB HBM2 memory
- 2x Intel Sapphire Rapids 8460Y+ at 2.3Ghz and 32c each (64 cores)
- 4x NDR200 (BW per node 800Gb/s)
- 512 GB of Main memory (DDR5)
- 460GB on NVMe storage
How to use
You can use this model to translate in the following directions:
- Czech to German
- Czech to Ukrainian
- Czech to Vietnamese
- Chinese (Simplified) to Japanese
- English to Arabic (Egyptian)
- English to Eastern Armenian
- English to Belarusian
- English to Chinese (Simplified)
- English to Chinese (Traditional, Taiwan)
- English to Czech
- English to Estonian
- English to German
- English to Icelandic
- English to Indonesian
- English to Japanese
- English to Kazakh
- English to Korean
- English to Ladin (Val Badia)
- English to Ligurian
- English to Northern Sámi
- English to Russian
- English to Thai
- English to Ukrainian
The instruction-following model uses the commonly adopted ChatML template:
<|im_start|>system
{SYSTEM PROMPT}<|im_end|>
<|im_start|>user
{USER PROMPT}<|im_end|>
<|im_start|>assistant
{MODEL RESPONSE}<|im_end|>
<|im_start|>user
[...]
The easiest way to apply it is by using the tokenizer's built-in functions, as shown in the following snippet.
from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch
model_id = "LangTech-MT/salamandraTA-7b-instruct-WMT26"
# Map from human-readable names to model language labels
lang_map = {
'English': 'English',
'Czech': 'Czech',
'Chinese (Simplified)': 'Simplified Chinese',
'Chinese (Traditional, Taiwan)': 'Traditional Chinese',
'Arabic (Egyptian)': 'Egyptian Arabic',
'Eastern Armenian': 'Armenian',
'Belarusian': 'Belarusian',
'Estonian': 'Estonian',
'German': 'German',
'Icelandic': 'Icelandic',
'Indonesian': 'Indonesian',
'Japanese': 'Japanese',
'Kazakh': 'Kazakh',
'Korean': 'Korean',
'Ladin (Val Badia)': 'Ladin',
'Ligurian': 'Ligurian',
'Northern Sámi': 'Northern Sámi',
'Russian': 'Russian',
'Thai': 'Thai',
'Ukrainian': 'Ukrainian',
'Vietnamese': 'Vietnamese',
}
# Input parameters
source = 'English'
target = 'Belarusian'
sentence = "Since the new library opened, more young people in the town have started reading in their free time."
# Map to model's expected language names
source_lang = lang_map[source]
target_lang = lang_map[target]
text = f"Translate the following text from {source_lang} into {target_lang}.\n{source_lang}: {sentence} \n{target_lang}:"
# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
torch_dtype=torch.bfloat16
)
# Construct prompt using chat template
message = [ { "role": "user", "content": text } ]
prompt = tokenizer.apply_chat_template(
message,
tokenize=False,
add_generation_prompt=True
)
inputs = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt")
input_length = inputs.shape[1]
# Generate output
outputs = model.generate(
input_ids=inputs.to(model.device),
max_new_tokens=400,
early_stopping=True,
num_beams=5
)
# Decode and print output
print(tokenizer.decode(outputs[0, input_length:], skip_special_tokens=True))
# З таго часу, як адкрылася новая бібліятэка, больш маладых людзей у горадзе пачалі чытаць у вольны час.
Using this template, each turn is preceded by a <|im_start|> delimiter and the role of the entity
(either user, for content supplied by the user, or assistant for LLM responses), and finished with the <|im_end|> token.
The WMT26 submission itself was not produced with this snippet: it used Minimum Bayes Risk decoding over N=8 candidates per unit drawn by nucleus sampling (temperature 0.6, top-p 0.95, top-k 50, up to 2048 tokens), with an eligibility filter removing truncated, degenerate and off-target-script candidates before MBR selection with COMET (wmt22-comet-da) as utility. See the system description paper for the full inference pipeline.
Machine Translation Prompt
You can use the following prompt template:
Translate the following text from {source} into {target}.
{source}: {source sentence}
{target}:
Show an example
source = 'English'
target = 'Kazakh'
source_sentence = "The committee will publish its final report next month, after reviewing the comments it received from the public."
text = f"Translate the following text from {source} into {target}.\n{source}: {source_sentence} \n{target}:"
# Комиссия қоғамдық пікірлерді қарап шыққаннан кейін келесі айда соңғы есебін жариялайды.
Data
Pretraining data
We conducted continual pretraining on Salamandra-7B in two stages, after adapting its vocabulary to the new tokenizer. All token counts are measured with the new tokenizer.
Stage 1 (CPT1): monolingual. We assembled a monolingual corpus covering all languages already supported by SalamandraTA together with those newly introduced for WMT 2026. Each language was sampled up to a 1B-token cap, and languages with less available data contributed everything available. In total, 52 language datasets amounting to 44.91B tokens were globally shuffled into a single shard, with 0.05% held out for validation and 0.05% for testing. The model was trained for 10,708 updates on this shard.
Most data came from FineWeb for English and FineWeb-2 for the remaining languages. Where FineWeb-2 could not fill the budget we drew on CC-100 (Kazakh, Traditional Chinese, and partially Belarusian and Armenian), the Leipzig Corpora and the Northern Sámi side of OPUS, and regional corpora for Aranese (PILAR, DOGC, WMT24 low-resource Occitan), Valencian (Les Corts Valencianes) and Asturian. Simplified Chinese was obtained by script conversion of the FineWeb-2 Chinese sample. Data was used as released, without additional filtering.
Four coverage gaps should be noted. Thai was not included in this stage, as its sample was not ready when the shard was assembled; this is compensated in Stage 2. Ligurian and Ladin were excluded because no usable monolingual data was available. Northern Sámi was included with only ~30M tokens. Turkish (1B tokens) was included, although it is not a WMT 2026 language, to provide transfer support for Kazakh.
Stage 2 (CPT2): parallel and monolingual. This stage uses instruction-formatted parallel data (~72% of the mix by source-target tokens) and monolingual data (~28%), restricted to the WMT 2026 languages plus Italian and Turkish. The mix contains 85.1B tokens including template overhead, globally shuffled, with the same 0.05%/0.05% held-out split, and was split into two shards of 42.54B tokens each. Training was stopped at the end of the first shard (10,143 updates), when our compute allocation was exhausted, so the model has seen half of the planned Stage 2 data.
Parallel data: we downloaded all corpora permitted for the constrained task with mtdata using the official WMT 2026 recipe, and audited licences, discarding corpora that do not permit derived work even for research. The data was cleaned with ParaCLEAN, which scores every pair with LaBSE alignment and GlotLID language identification, filters pairs below 0.75 alignment or 0.5 language-identification probability on either side, and deduplicates and normalises the remaining data. The result was merged with the existing SalamandraTA parallel data for the same pairs and cross-deduplicated. Each direction was converted to plain-text records by embedding sentence pairs in natural-language translation instructions, sampled uniformly from a pool of 35 templates, with and without an explicit source-language name. Each direction was capped at 5B source-plus-target tokens, giving 52.4B tokens over 937M sentence pairs across 21 directions.
Monolingual data: 1B tokens for each of the 19 Stage 2 languages (fresh samples from the Stage 1 sources, non-overlapping with the Stage 1 shard), for 20B tokens in total. Thai received a double share to compensate for its omission from Stage 1. English monolingual data was not included.
Coverage: Italian (1B monolingual, 5B en-it parallel) was added to support the closely related Ligurian and Ladin, and Turkish (1B monolingual) to support Kazakh. No Arabic variety filtering was applied at either stage: the en-ar data is generic Arabic, predominantly Modern Standard, and Egyptian Arabic is introduced at the instruction-tuning stage. English to Ligurian, Ladin and Northern Sámi had only 1.7M, 9K and 118K tokens respectively after cleaning; these were left out of Stage 2 and first appear in the instruction-tuning data.
Click the expand button below to see the data used in each stage.
Data Sources
Stage 1 (monolingual). Languages above the line were capped at 1B tokens; for those below, all available data was used.
| Languages | Tokens |
|---|---|
| ar, be, bg, ca, cs, da, de, el, en, es, et, eu, fi, fr, gl, hi, hr, hu, hy, id, is, it, ja, kk, ko, lt, lv, nb, nl, pl, pt, ro, ru, sk, sl, sr, sv, tr, uk, vi, zh, zh_TW | ~1B each |
| nn | 768.0M |
| cy | 723.9M |
| ga | 591.1M |
| mt | 425.5M |
| ast | 202.7M |
| oc | 88.8M |
| val (Valencian) | 39.3M |
| se | 30.0M |
| oc-aranes (Aranese) | 28.0M |
| arg (Aragonese) | 16.9M |
| Total | 44.91B |
Stage 2 (parallel). Token counts are source-plus-target, excluding instruction-template overhead; † marks directions truncated at the 5B cap.
| Direction | Pairs | Tokens |
|---|---|---|
| en-cs | 104.1M | 5.00B† |
| en-de | 85.3M | 5.00B† |
| en-id | 88.6M | 5.00B† |
| en-ru | 73.0M | 5.00B† |
| en-zh | 85.6M | 4.34B |
| cs-de | 62.9M | 3.08B |
| en-uk | 28.3M | 2.76B |
| en-ja | 53.4M | 2.59B |
| en-ko | 43.3M | 2.12B |
| en-et | 30.2M | 1.70B |
| en-th | 25.4M | 1.28B |
| zh-ja | 15.5M | 0.79B |
| en-is | 15.7M | 0.77B |
| cs-uk | 14.7M | 0.70B |
| en-be | 16.4M | 0.67B |
| cs-vi | 15.0M | 0.52B |
| en-kk | 11.9M | 0.50B |
| en-hy | 5.6M | 0.31B |
| en-zh_TW | 9.4M | 0.23B |
| en-ar (generic Arabic; base for en-arz) | 73.7M | 5.00B† |
| en-it (transfer support) | 79.5M | 5.00B† |
Excluded from Stage 2 (insufficient data after cleaning; reserved for instruction tuning): en-lij (1.7M tokens), en-se (118K tokens), en-lld (9K tokens).
References
- Penedo, G., Kydlíček, H., Ben allal, L., Lozhkov, A., Mitchell, M., Raffel, C., Von Werra, L., & Wolf, T. (2024). The FineWeb datasets: Decanting the web for the finest text data at scale. arXiv:2406.17557.
- Penedo, G., Kydlíček, H., Sabolčec, V., Messmer, B., Foroutan, N., Kargaran, A. H., Raffel, C., Jaggi, M., Von Werra, L., & Wolf, T. (2025). FineWeb2: One pipeline to scale them all: adapting pre-training data processing to every language. arXiv:2506.20920.
- Wenzek, G., Lachaux, M.-A., Conneau, A., et al. (2020). CCNet: Extracting high quality monolingual datasets from web crawl data. In LREC.
- Eckart, T., & Quasthoff, U. (2013). Statistical corpus and language comparison on comparable corpora. In Building and using comparable corpora, pages 151–165. Springer.
- Tiedemann, J. (2012). Parallel data, tools and interfaces in OPUS. In LREC'12.
- Gowda, T., Zhang, Z., Mattmann, C., & May, J. (2021). Many-to-English machine translation tools, data, and pretrained models. In ACL-IJCNLP 2021: System Demonstrations, pages 306–316.
- Mash, A., Bohman, E. P., & Melero, M. (2026). ParaCLEAN: Improving Translation Quality through Systematic Parallel Data Cleaning. In LREC 2026, pages 6630–6640.
- Feng, F., Yang, Y., Cer, D., Arivazhagan, N., & Wang, W. (2022). Language-agnostic BERT sentence embedding. arXiv:2007.01852.
- Kargaran, A. H., Imani, A., Yvon, F., & Schütze, H. (2023). GlotLID: Language identification for low-resource languages. In EMNLP 2023.
Instruction Tuning Data
The final continual pre-training checkpoint was fully fine-tuned for one epoch on an instruction mixture of 111,106 examples across eight translation-related tasks: sentence-level MT (71,777), paragraph-level MT (4,543), document-level MT (6,789), HTML tag transfer (8,541), terminology-constrained MT (7,147), formality-controlled translation (5,250), multi-reference MT (4,959) and formality register transfer (2,100). The mixture is restricted to translation-centric tasks; no chat or code data was used. Small datasets were used in full, and larger ones uniformly subsampled to 250 to 2,000 examples per language pair with a fixed seed. For the lowest-resource pairs we upsampled existing data (en-arz) or reused parallel sentences from the pre-training stage (en-be, en-lld, en-lij, en-se). Egyptian Arabic is introduced at this stage; it was used for most tasks, except HTML tag transfer, for which generic Arabic was used.
Click the expand button below to see the full list of tasks included in the finetuning data.
Data Sources
| Task | Source | Languages | Count |
|---|---|---|---|
| General Machine Translation | FLORES+ | en paired with (arz, be, cs, de, et, hy, id, is, ja, kk, ko, lij, ru, th, uk, zh, zh_TW); cs-de, cs-uk, cs-vi, zh-ja | 20,937 |
| ALT | en-id, en-ja, en-th, en-vi, en-zh, zh-ja | 12,000 | |
| Tatoeba | cs-de, cs-en, cs-uk, en-et, en-is, en-ja, en-ko, en-ru, en-uk, en-zh, ja-zh | 11,000 | |
| WMT24++ | en-arz, en-cs, en-de, en-et, en-is, en-ja, en-ko, en-ru, en-uk, en-zh | 9,600 | |
| SMOL (SmolSent) | en-arz, en-id, en-ja, en-ko, en-lij, en-th | 5,711 | |
| ArzEn-MultiGenre | en-arz | 5,000 | |
| NTEU | en-cs, en-de, en-et | 2,688 | |
| Resampled WMT26 parallel data | en-be, en-lld, en-lij, en-se | 2,841 | |
| EU press releases | en-cs, en-de, en-et | 1,500 | |
| NusaX-MT | en-id | 500 | |
| Paragraph-level Translation | News Commentary | cs-de, en-ja, en-ru, en-zh, ja-zh | 3,043 |
| NewsPaLM | en-de | 1,000 | |
| AcadTrain | en-de | 500 | |
| Document-level Translation | Europarl | cs-de, cs-et, cs-it, en-et, en-it | 5,000 |
| SMOL (SmolDoc) | en-arz, en-id, en-ja, en-ko, en-lij, en-ru, en-th | 1,304 | |
| Project Gutenberg | de-en | 485 | |
| Multi-reference Translation | Tatoeba | cs-de, cs-en, cs-uk, en-cs, en-de, en-et, en-is, en-ja, en-ko, en-ru, en-uk, en-zh, ja-zh | 4,959 |
| Terminology-aware Translation | MeSpEn (synthetic glossaries) | en-ar, en-de, en-id, en-it, en-ja, en-ko, en-ru, en-th, en-tr, en-zh | 2,033 |
| WMT25 Terminology Task (Track 2) | en-zh_TW | 1,600 | |
| WMT23 Terminology Task | en-cs, en-de, en-zh | 1,500 | |
| EMEA (document-level) | cs-de, en-et | 1,014 | |
| WMT25 Terminology Task (Track 1) | en-de, en-ru | 1,000 | |
| Formality-controlled Translation | Synthetic (Gemma 4) | en paired with (arz, be, cs, de, et, hy, id, is, ja, kk, ko, ru, th, uk, vi, zh, zh_TW); cs-de, cs-uk, cs-vi, zh-ja | 5,250 |
| Formality Register Transfer | Synthetic (Gemma 4) | en paired with (arz, be, cs, de, et, hy, id, is, ja, kk, ko, ru, th, uk, vi, zh, zh_TW); cs-de, cs-uk, cs-vi, zh-ja | 2,100 |
| HTML Tag Transfer | FLORES+ (HTML-augmented) | en-ar, en-cs, en-de, en-et, en-is, en-ja, en-ko, en-ru, en-uk, en-zh | 5,000 |
| News Commentary (HTML-augmented) | ar-en, cs-de, cs-en, de-en, en-ja, en-ru, en-zh, ja-zh | 2,296 | |
| Europarl (HTML-augmented) | cs-de, cs-en, cs-et, de-en, de-et, en-et | 995 | |
| AcadTrain (HTML-augmented) | en-de | 250 | |
| Total | 111,106 |
Hyperparameters: 1 epoch, batch size 1 per device, gradient accumulation 1, learning rate 1e-5, weight decay 0, warmup ratio 0.03, cosine scheduler, model max length 8,192.
Preference Optimization Data
Preference optimization was applied to the six directions with the weakest post-instruction-tuning output: English into Egyptian Arabic, Thai, Belarusian, Northern Sámi, Ladin and Ligurian. Preference pairs were built from a subset of the sentence-level instruction-tuning data. For five of these directions, the reference was used as the chosen example and the instruction-tuned model's output as the rejected one. For Ligurian, Diverse Beam Search was used to generate 32 candidates (32 beams, 8 groups) per source, re-ranked by MBR with chrF++ as utility, and two pairs were formed per source with the best candidate as chosen and the 16th and 32nd as rejected. The set totals 20,090 pairs.
The method combines CPO and SimPO (length normalisation and a target reward margin) and updates all model parameters.
Preference data and hyperparameters
| Direction | Source | Initial Count | Final Count |
|---|---|---|---|
| en-arz | WMT24++ | 960 | 960 |
| FLORES+ | 997 | 997 | |
| SMOL (SmolSent) | 863 | 863 | |
| ArzEn-MultiGenre | 2,180 | 2,180 | |
| en-th | FLORES+ | 997 | 997 |
| SMOL (SmolSent) | 753 | 753 | |
| ALT | 2,250 | 2,250 | |
| en-be | FLORES+ | 997 | 997 |
| Resampled WMT26 parallel data | 2,000 | 2,000 | |
| en-se | Resampled WMT26 parallel data | 748 | 748 |
| en-lld | Resampled WMT26 parallel data | 93 | 93 |
| en-lij | FLORES+ | 997 | 1,994 |
| SMOL (SmolSent) | 863 | 1,726 | |
| Resampled WMT26 parallel data | 1,766 | 3,532 | |
| Total | 16,464 | 20,090 |
Hyperparameters: 1 epoch, batch size 1 per device, gradient accumulation 16, learning rate 5e-7, beta 0.6, loss type simpo, CPO alpha 1.0, SimPO gamma 1.0.
Evaluation
Internal evaluation
The final checkpoint was evaluated with MT-Lens on FLORES+ devtest (22 directions) and WMT24++ (16 directions), using beam search with five beams and batch size five. This setting is simpler than the submission pipeline: it uses no MBR decoding and no candidate filtering. English to Northern Sámi is not included, as no evaluation data is available for that direction. English to Arabic and English to Polish were evaluated in error and are not shared-task directions; they are not reported.
Click to show metrics details
BLEU: sacreBLEU 2.4.0. Signature: nrefs:1|case:mixed|eff:no|tok:13a|smooth:exp|version:2.4.0, except that the tokenizer is selected by target language (tok:zh for Chinese, ja-mecab and ko-mecab for Japanese and Korean), so BLEU for those targets is not comparable with the rest of any table.ChrF: sacreBLEU 2.4.0. Signature: nrefs:1|case:mixed|eff:yes|nc:6|nw:0|space:no|version:2.4.0.COMET: COMET-22, model checkpoint "Unbabel/wmt22-comet-da".
Mean over directions:
| Test set | BLEU | chrF | COMET |
|---|---|---|---|
| FLORES+ devtest (n = 22) | 28.83 | 51.83 | 0.856 |
| WMT24++ (n = 16) | 33.64 | 52.61 | 0.857 |
FLORES+ devtest, per direction
| Direction | BLEU | chrF | COMET |
|---|---|---|---|
| cs-de | 30.54 | 59.24 | 0.863 |
| cs-uk | 24.54 | 53.30 | 0.895 |
| cs-vi | 33.81 | 53.34 | 0.877 |
| en-arz | 16.43 | 49.56 | 0.828 |
| en-be | 16.64 | 47.72 | 0.870 |
| en-cs | 34.88 | 61.00 | 0.911 |
| en-de | 41.57 | 67.06 | 0.871 |
| en-et | 30.18 | 62.03 | 0.914 |
| en-hy | 24.39 | 59.20 | 0.898 |
| en-id | 47.20 | 71.14 | 0.910 |
| en-is | 28.97 | 55.83 | 0.864 |
| en-ja | 33.93 | 41.48 | 0.906 |
| en-kk | 23.80 | 58.75 | 0.898 |
| en-ko | 30.00 | 36.81 | 0.889 |
| en-lij | 28.14 | 55.53 | 0.620 |
| en-lld | 2.58 | 27.54 | 0.498 |
| en-ru | 33.61 | 60.20 | 0.894 |
| en-th | 12.66 | 54.54 | 0.877 |
| en-uk | 32.01 | 60.02 | 0.899 |
| en-zh | 45.16 | 39.49 | 0.874 |
| en-zh_TW | 36.07 | 31.92 | 0.878 |
| zh-ja | 27.23 | 34.54 | 0.895 |
WMT24++, per direction
| Direction | BLEU | chrF | COMET |
|---|---|---|---|
| cs-de | 28.15 | 56.79 | 0.822 |
| cs-uk | 25.52 | 51.91 | 0.845 |
| cs-vi | 26.56 | 47.28 | 0.814 |
| en-cs | 39.68 | 63.79 | 0.882 |
| en-de | 39.65 | 65.17 | 0.849 |
| en-et | 38.71 | 66.25 | 0.896 |
| en-id | 33.17 | 61.38 | 0.845 |
| en-is | 31.20 | 55.65 | 0.849 |
| en-ja | 38.45 | 45.12 | 0.889 |
| en-ko | 40.67 | 45.36 | 0.889 |
| en-ru | 32.09 | 56.60 | 0.832 |
| en-th | 9.57 | 48.48 | 0.833 |
| en-uk | 39.86 | 63.44 | 0.875 |
| en-zh | 56.93 | 52.63 | 0.876 |
| en-zh_TW | 33.31 | 30.25 | 0.848 |
| zh-ja | 24.70 | 31.66 | 0.861 |
WMT26 test set
Automatic scores of the submitted system on the WMT26 General Translation test set, against the organisers' human references, which cover 12 of the 23 directions. The post-edited reference (refPE) is used where available and the from-scratch reference (refA) otherwise. Only surface metrics are reported, as many test documents exceed the input length of the neural metrics. These are our own scores against references released after the evaluation period; the official human evaluation is summarised below. As refPE references are post-edits of machine output, surface metrics may favour systems closer to that output.
Comparison with the constrained-track systems that cover all 12 reference directions, averaged over cs-de, cs-vi, en-cs, en-de, en-et, en-is, en-arz, en-ja, zh-ja, en-lij, en-lld and en-se:
| System | BLEU | chrF |
|---|---|---|
| Lumen | 41.94 | 60.38 |
| Qingqiu-MT-9B | 39.27 | 56.94 |
| SalamandraTA (ours) | 33.01 | 52.00 |
| Gemma 4-4B | 29.73 | 48.33 |
| GPT-OSS-20B | 25.01 | 44.54 |
| Qwen 3.5-9B | 24.32 | 44.82 |
| Tower 9B | 23.46 | 44.64 |
| GLM 4-9B | 23.15 | 42.75 |
| Ministral 3-14B | 18.15 | 42.82 |
chrF per direction (ours)
| cs-de | cs-vi | en-cs | en-de | en-et | en-is | en-arz | en-ja | zh-ja | en-lij | en-lld | en-se | Mean |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 69.9 | 59.5 | 55.0 | 62.1 | 62.2 | 60.6 | 47.8 | 40.1 | 30.2 | 53.1 | 36.0 | 47.6 | 52.0 |
WMT26 official human evaluation
The submission competed in the constrained track. The organisers' human evaluation used contrastive Error Span Annotation (cESA), in which annotators score each translation from 0 to 100. Results are taken from the WMT26 General MT findings paper (Kocmi et al., 2026), Sections 6.2 and 6.3. Ranks are significance clusters: a range such as 3-4 means the system is not significantly different from the systems in those positions.
Among constrained systems, SalamandraTA was ranked second (cluster 2-2) for English to Icelandic, Estonian, Northern Sámi and Ligurian, and third, or in a cluster that includes third, for eight further directions. Its best rank among all systems, constrained and unconstrained, was 7-8 for English to Ligurian. English to Ladin received a human score of 1.6.
Human evaluation, per direction
| Direction | cESA score | Rank (constrained) | Rank (all systems) |
|---|---|---|---|
| cs-de | 65.0 | 3-3 of 11 | 19-23 of 31 |
| cs-uk | 69.6 | 3-4 of 11 | 18-19 of 29 |
| cs-vi | 67.8 | 4-8 of 11 | 18-28 of 31 |
| zh-ja | 57.7 | 6-6 of 12 | 22-23 of 31 |
| en-arz | 76.8 | 4-4 of 10 | 20-21 of 30 |
| en-hy | 52.9 | 3-4 of 10 | 14-16 of 28 |
| en-be | 44.1 | 6-6 of 13 | 20-22 of 31 |
| en-zh | 58.0 | 3-7 of 10 | 19-25 of 29 |
| en-zh_TW | 46.2 | 4-8 of 10 | 21-25 of 28 |
| en-cs | 54.4 | 4-4 of 11 | 20-23 of 30 |
| en-et | 66.1 | 2-2 of 11 | 13-13 of 30 |
| en-de | 55.0 | 3-3 of 11 | 17-19 of 30 |
| en-is | 69.0 | 2-2 of 12 | 11-11 of 31 |
| en-id | 67.3 | 5-6 of 11 | 22-25 of 31 |
| en-ja | 80.2 | 3-4 of 11 | 17-21 of 31 |
| en-kk | 56.3 | 4-4 of 11 | 17-19 of 29 |
| en-ko | 49.4 | 4-5 of 10 | 22-24 of 29 |
| en-lld | 1.6 | 6-7 of 11 | 22-23 of 27 |
| en-lij | 28.6 | 2-2 of 11 | 7-8 of 29 |
| en-se | 33.4 | 2-2 of 10 | 11-12 of 27 |
| en-ru | 57.9 | 5-5 of 11 | 22-24 of 31 |
| en-th | 61.0 | 3-4 of 11 | 19-21 of 31 |
| en-uk | 66.1 | 3-3 of 11 | 17-18 of 30 |
"of N" counts the rows in each ranking table; the all-systems tables include the human reference translations.
Ligurian and Ladin
English to Ligurian and English to Ladin went through the same stages: no data in either continual pre-training stage, Italian as a high-resource relative, first representation at instruction tuning, and preference optimization on top. Their outcomes diverge. On FLORES+, Ligurian rises from 4.42 BLEU after continual pre-training to 27.53 after instruction tuning and ends at 28.14. Ladin declines at every stage, from 2.90 to 2.79 to 2.58 BLEU. After cleaning, en-lij retained 1.7M parallel tokens against 9K for en-lld, and the Ladin corpus was additionally judged of poor quality on inspection. Both directions fall outside WMT24++ coverage, so this rests on FLORES+ alone.
Ethical Considerations and Limitations
Detailed information on the work done to examine the presence of unwanted social and cognitive biases in the base model can be found at Salamandra-7B model card. No specific analysis has yet been carried out in order to evaluate potential biases or limitations in translation accuracy across different languages, dialects, or domains. However, we recognize the importance of identifying and addressing any harmful stereotypes, cultural inaccuracies, or systematic performance discrepancies that may arise in Machine Translation. As such, we plan to continue performing more analyses as we implement the necessary metrics and methods within our evaluation framework MT-Lens. We urge developers to consider potential limitations and conduct safety testing and tuning tailored to their specific applications.
One external result is available: in the WMT26 GENDER1PERSON test suite, which asks systems to produce gender-inclusive forms when translating first-person English text into Czech, Russian and Croatian, this model produced such forms only rarely (WMT26 findings paper, Section 7).
This model was optimized for the WMT 2026 directions only. The second continual pre-training stage stopped at the end of its first shard, half of its planned data, when our compute allocation was exhausted. We hold no internal evaluation data for English to Northern Sámi; its only automatic assessment is the WMT26 test-set reference. English to Ladin had 9K parallel tokens after cleaning and declines at every training stage on FLORES+ (2.58 BLEU for the final model). Preference optimization did not change automatic metrics measurably; we did not run the degeneracy-specific or human evaluation that would test the failure modes it was built to remove.
Additional information
Author
Machine Translation Group, AI Institute, the Barcelona Supercomputing Center.
Contact
For further information, please send an email to ai_institute_mt@bsc.es.
Copyright
Copyright(c) 2026 by Machine Translation Group, AI Institute, the Barcelona Supercomputing Center.
Funding
This work has been promoted and financed by the Government of Catalonia through the Aina project.
It is supported by MLLM4TRA (PID2024-158157OB-C32), funded by MCIN/AEI/10.13039/501100011033/FEDER, UE.
Disclaimer
Be aware that the model may contain biases or other unintended distortions. When third parties deploy systems or provide services based on this model, or use the model themselves, they bear the responsibility for mitigating any associated risks and ensuring compliance with applicable regulations, including those governing the use of Artificial Intelligence.
The Barcelona Supercomputing Center, as the owner and creator of the model, shall not be held liable for any outcomes resulting from third-party use.
License
Research only
See LICENSE file for further details.
Citation
If you use this model, please cite the system description paper:
@inproceedings{mash-etal-2026-bsc,
title = "{BSC} Submission for {WMT}26 General Machine Translation Shared Task",
author = "Mash, Audrey and Orama Ayebakuro, Jonathan and Bohman, Ella Paulina and Liao, Xixian and De Luca Fornaciari, Francesca and Melero, Maite",
booktitle = "Proceedings of the Eleventh Conference on Machine Translation",
year = "2026",
address = "Budapest, Hungary",
publisher = "Association for Computational Linguistics",
pages = "1334--1350",
url = "https://www2.statmt.org/wmt26/pdf/2026.wmt-1.72.pdf"
}
- Downloads last month
- 17
Model tree for LangTech-MT/salamandraTA-7b-instruct-WMT26
Base model
BSC-LT/salamandra-7b