License: free for commercial and non-commercial use, with attribution. If you use this model, its outputs, or anything derived from it in a product, service, paper or dataset, you must credit "AletheiaAING" (https://huggingface.co/AletheiaAING) and keep this notice. This model is derived from Google Gemma and is also subject to the Gemma Terms of Use.

csw: 301M code-switching and idiom model (SFT)

The supervised (SFT) 301M-parameter csw model (fp32, model.safetensors, 1.2 GB), public. See BeardedMonster/csw-sft and BeardedMonster/csw-rl (private) for the experiments, GGUF files and the RL model. The 1B successors are AletheiaAING/gemma-3-1b-csw-best and AletheiaAING/gemma-3-0.8b-csw.

What the model does

Six tasks over 37 African languages: translation from English, code-switching (mix a second language into the base language the way native speakers do) and idiom-rich informal rewriting.

Task What you give it What you get Prompt
eng_m English text translation into the base language Translate the text to Yoruba: TEXT (language name)
eng_hm English text code-switched text <hm> <yor>+<eng>: TEXT
m_hm text in the base language code-switched text <hm> <yor>+<eng>: TEXT
eng_ood English text idiom-rich informal text <ood> <yor>: TEXT
m_ood text in the base language idiom-rich informal text <ood> <yor>: TEXT
hm_ood code-switched text idiom-rich text that keeps the switching <ood> <yor>+<eng>: TEXT

What do m_text, hm and ood mean? (in simple words)

  • m_text (eng_m in the task table): a plain translation of English into the language you want, no mixing. Prompt: Translate the text to Yoruba: TEXT (use the language name).
  • hm (human-style mixing): the text mixes two languages the way real people talk, for example Yoruba with English words in the middle of a sentence. Prompt: <hm> <yor>+<eng>: TEXT (language codes in angle brackets).
  • ood (idiom-rich style): the text is rewritten in a lively, everyday way, full of sayings, proverbs and comparisons, the way a chatty native speaker talks. Prompt: <ood> <yor>: TEXT.

eng_ means the input is English, m_ means the input is already in the base language, and hm_ood means the input is already mixed and you want the idiom-rich rewrite. The usage code below runs m_text, hm and ood on one sentence.

Input format (read this before writing prompts).

  • A single user turn, no system prompt: <bos><start_of_turn>user + prompt + <end_of_turn> + <start_of_turn>model (tokenizer.apply_chat_template produces the same).
  • Translation (eng_m) is the only prompt that uses the language name: Translate the text to Yoruba: TEXT, with the names in the last column of the table below (for example Pidgin for pcm, Cameroon Pidgin for wes, Kirundi for run). Writing the code (Translate the text to yor) is out of distribution.
  • All other tasks use the three-letter code inside angle brackets (<hm> <yor>+<eng>: TEXT, <ood> <yor>: TEXT), never the name.
  • Two spaces after every colon. The tags <yor>, <hm>, <ood>, <dev> are single vocabulary tokens.
  • For <hm> prompts and hm_ood a switch language is required, taken from the table (other pairs were not trained). The model was never trained without a switch language on hm prompts: leaving it out gives an unswitched, idiom-like output.
  • Outputs: eng_m -> yor <translation> (language code, one space, text); other tasks -> <yor><eng><dev>device one | device two</dev> text. If the structure is wrong the output is invalid; resample.

Supported languages and switch languages

The table lists every base language with the switch languages this model was trained with and, in the last column, the language name to use in the translation prompt.

Language Code used in prompts Switch languages (trained pairs) Example prompt (code-switching) Translation prompt (name, not code)
Afrikaans afr English eng, Zulu zul <hm> <afr>+<eng>: TEXT Translate the text to Afrikaans: TEXT
Amharic amh English eng, Arabic ara <hm> <amh>+<eng>: TEXT Translate the text to Amharic: TEXT
Bambara bam French fra, Wolof wol <hm> <bam>+<fra>: TEXT Translate the text to Bambara: TEXT
Cameroonian Pidgin wes French fra, Ewondo ewo, Duala dua <hm> <wes>+<fra>: TEXT Translate the text to Cameroon Pidgin: TEXT
Chichewa nya English eng, Tumbuka tum <hm> <nya>+<eng>: TEXT Translate the text to Chichewa: TEXT
Ewe ewe French fra, English eng <hm> <ewe>+<fra>: TEXT Translate the text to Ewe: TEXT
Fon fon French fra, Yoruba yor <hm> <fon>+<fra>: TEXT Translate the text to Fon: TEXT
Fulah ful French fra, English eng <hm> <ful>+<fra>: TEXT Translate the text to Fulah: TEXT
Ghanaian Pidgin gpe Twi twi, Ewe ewe, Ga gaa <hm> <gpe>+<twi>: TEXT Translate the text to Ghanaian Pidgin: TEXT
Hausa hau English eng, Arabic ara <hm> <hau>+<eng>: TEXT Translate the text to Hausa: TEXT
Igbo ibo English eng, Nigerian Pidgin pcm <hm> <ibo>+<eng>: TEXT Translate the text to Igbo: TEXT
Kikuyu kik English eng, Swahili swh <hm> <kik>+<eng>: TEXT Translate the text to Kikuyu: TEXT
Kinyarwanda kin French fra, English eng, Swahili swh <hm> <kin>+<fra>: TEXT Translate the text to Kinyarwanda: TEXT
Kirundi run French fra, Swahili swh <hm> <run>+<fra>: TEXT Translate the text to Kirundi: TEXT
Kongo kon French fra, Lingala lin <hm> <kon>+<fra>: TEXT Translate the text to Kongo: TEXT
Lingala lin French fra, Kongo kon <hm> <lin>+<fra>: TEXT Translate the text to Lingala: TEXT
Luganda lug English eng, Swahili swh <hm> <lug>+<eng>: TEXT Translate the text to Luganda: TEXT
Luo luo English eng, Swahili swh <hm> <luo>+<eng>: TEXT Translate the text to Luo: TEXT
Malagasy mlg French fra, English eng <hm> <mlg>+<fra>: TEXT Translate the text to Malagasy: TEXT
Nigerian Pidgin pcm Yoruba yor, Igbo ibo, Hausa hau <hm> <pcm>+<yor>: TEXT Translate the text to Pidgin: TEXT
North Ndebele nde English eng, Shona sna <hm> <nde>+<eng>: TEXT Translate the text to North Ndebele: TEXT
Oromo orm Amharic amh, English eng <hm> <orm>+<amh>: TEXT Translate the text to Oromo: TEXT
Sepedi nso English eng, Afrikaans afr <hm> <nso>+<eng>: TEXT Translate the text to Sepedi: TEXT
Sesotho sot English eng, Afrikaans afr <hm> <sot>+<eng>: TEXT Translate the text to Sesotho: TEXT
Setswana tsn English eng, Afrikaans afr <hm> <tsn>+<eng>: TEXT Translate the text to Setswana: TEXT
Shona sna English eng, North Ndebele nde <hm> <sna>+<eng>: TEXT Translate the text to Shona: TEXT
Somali som Arabic ara, English eng, Italian ita, Amharic amh <hm> <som>+<ara>: TEXT Translate the text to Somali: TEXT
Swahili swh English eng, Arabic ara, Sheng sheng <hm> <swh>+<eng>: TEXT Translate the text to Swahili: TEXT
Swati ssw English eng, Zulu zul <hm> <ssw>+<eng>: TEXT Translate the text to Swati: TEXT
Tigrinya tir Amharic amh, English eng <hm> <tir>+<amh>: TEXT Translate the text to Tigrinya: TEXT
Tsonga tso English eng, Zulu zul <hm> <tso>+<eng>: TEXT Translate the text to Tsonga: TEXT
Tumbuka tum English eng, Chichewa nya <hm> <tum>+<eng>: TEXT Translate the text to Tumbuka: TEXT
Twi twi English eng, Ghanaian Pidgin gpe <hm> <twi>+<eng>: TEXT Translate the text to Twi: TEXT
Wolof wol French fra, Arabic ara <hm> <wol>+<fra>: TEXT Translate the text to Wolof: TEXT
Xhosa xho English eng, Afrikaans afr <hm> <xho>+<eng>: TEXT Translate the text to Xhosa: TEXT
Yoruba yor English eng, Nigerian Pidgin pcm <hm> <yor>+<eng>: TEXT Translate the text to Yoruba: TEXT
Zulu zul English eng, Afrikaans afr <hm> <zul>+<eng>: TEXT Translate the text to Zulu: TEXT

What is in this repository

model.safetensors (fp32), tokenizer, config.json, chat template: the SFT model.

How to use it

# pip install torch transformers   (tested with transformers 5.x; also served with vLLM 0.32)
import random, re, sys, torch
from transformers import AutoModelForCausalLM, AutoTokenizer

NAMES = {'yor': 'Yoruba', 'hau': 'Hausa', 'ibo': 'Igbo', 'pcm': 'Pidgin', 'twi': 'Twi', 'ewe': 'Ewe', 'fon': 'Fon', 'wol': 'Wolof', 'swh': 'Swahili', 'kin': 'Kinyarwanda', 'som': 'Somali', 'orm': 'Oromo', 'amh': 'Amharic', 'tir': 'Tigrinya', 'lug': 'Luganda', 'lin': 'Lingala', 'zul': 'Zulu', 'xho': 'Xhosa', 'afr': 'Afrikaans', 'sna': 'Shona', 'nya': 'Chichewa', 'sot': 'Sesotho', 'tsn': 'Setswana', 'mlg': 'Malagasy', 'tum': 'Tumbuka', 'run': 'Kirundi', 'tso': 'Tsonga', 'nso': 'Sepedi', 'ssw': 'Swati', 'nde': 'North Ndebele', 'kik': 'Kikuyu', 'luo': 'Luo', 'bam': 'Bambara', 'kon': 'Kongo', 'ful': 'Fulah', 'wes': 'Cameroon Pidgin', 'gpe': 'Ghanaian Pidgin'}          # base language code -> name used in the TRANSLATION prompt (not the tags)

# switch languages the model was trained with, per base language (anything else is out of distribution)
SWITCH = {'yor': ['eng', 'pcm'], 'hau': ['eng', 'ara'], 'ibo': ['eng', 'pcm'], 'pcm': ['yor', 'ibo', 'hau'], 'twi': ['eng', 'gpe'], 'ewe': ['fra', 'eng'], 'fon': ['fra', 'yor'], 'kin': ['fra', 'eng', 'swh'], 'sna': ['eng', 'nde'], 'zul': ['eng', 'afr'], 'xho': ['eng', 'afr'], 'swh': ['eng', 'ara', 'sheng'], 'orm': ['amh', 'eng'], 'som': ['ara', 'eng', 'ita', 'amh'], 'wol': ['fra', 'ara'], 'tsn': ['eng', 'afr'], 'nya': ['eng', 'tum'], 'afr': ['eng', 'zul'], 'sot': ['eng', 'afr'], 'lin': ['fra', 'kon'], 'run': ['fra', 'swh'], 'bam': ['fra', 'wol'], 'tum': ['eng', 'nya'], 'tso': ['eng', 'zul'], 'nso': ['eng', 'afr'], 'lug': ['eng', 'swh'], 'kik': ['eng', 'swh'], 'luo': ['eng', 'swh'], 'ssw': ['eng', 'zul'], 'nde': ['eng', 'sna'], 'kon': ['fra', 'lin'], 'ful': ['fra', 'eng'], 'mlg': ['fra', 'eng'], 'amh': ['eng', 'ara'], 'tir': ['amh', 'eng'], 'gpe': ['twi', 'ewe', 'gaa'], 'wes': ['fra', 'ewo', 'dua']}

TASKS = ("eng_m", "eng_hm", "m_hm", "eng_ood", "m_ood", "hm_ood")

def build_prompt(task, base, text, switch=None):
    """text = English (eng_*), base-language text (m_*) or code-switched text (hm_ood). Two spaces after every colon."""
    if task == "eng_m":                                   # translation: the language NAME, not the code
        return f"Translate the text to {NAMES[base]}:  {text}"
    if task in ("eng_hm", "m_hm"):
        switch = switch or random.choice(SWITCH[base])
        return f"<hm> <{base}>+<{switch}>:  {text}"
    if task in ("eng_ood", "m_ood"):
        return f"<ood> <{base}>:  {text}"
    if task == "hm_ood":
        if switch is None: raise ValueError("hm_ood needs the switch language of the input text")
        return f"<ood> <{base}>+<{switch}>:  {text}"
    raise ValueError(task)

_LANG = re.compile(r"^((?:<(?!dev>)[a-z]{3,5}>)+)"); _DEV = re.compile(r"<dev>(.*?)</dev>", re.S)

def parse(task, completion):
    """eng_m: 'yor <text>'.  Others: '<yor><eng><dev>device one | device two</dev> <text>'."""
    c = completion.strip()
    if task == "eng_m":
        m = re.match(r"^([a-z]{3,5}) (.*)$", c, re.S)
        return {"ok": bool(m), "tags": [m.group(1)] if m else [], "devices": [], "text": m.group(2).strip() if m else c}
    t = _LANG.match(c)
    if not t: return {"ok": False, "tags": [], "devices": [], "text": c}
    rest = c[t.end():].lstrip(); d = _DEV.match(rest)
    if not d: return {"ok": False, "tags": re.findall(r"<([a-z]{3,5})>", t.group(1)), "devices": [], "text": rest}
    text = rest[d.end():].strip()
    return {"ok": bool(text), "tags": re.findall(r"<([a-z]{3,5})>", t.group(1)), "devices": [x.strip() for x in d.group(1).split("|") if x.strip()], "text": text}

def load(path):
    tok = AutoTokenizer.from_pretrained(path); model = AutoModelForCausalLM.from_pretrained(path, dtype=torch.bfloat16).eval()
    return tok, model.to("cuda" if torch.cuda.is_available() else "cpu")

def generate(tok, model, prompt, sample=False, max_new=700):
    ids = tok(f"<bos><start_of_turn>user\n{prompt}<end_of_turn>\n<start_of_turn>model\n", add_special_tokens=False, return_tensors="pt").input_ids.to(model.device)
    stop = [tok.convert_tokens_to_ids("<end_of_turn>"), tok.eos_token_id]
    kw = dict(do_sample=True, temperature=0.7, top_p=0.95) if sample else dict(do_sample=False)         # greedy is what all reported numbers use
    with torch.no_grad(): out = model.generate(ids, max_new_tokens=max_new, eos_token_id=stop, pad_token_id=tok.pad_token_id or 0, **kw)
    return tok.decode(out[0][ids.shape[1]:], skip_special_tokens=False).replace("<end_of_turn>", "").replace("<eos>", "").replace("<pad>", "").strip()

tok, model = load("AletheiaAING/csw   # or a local download of this repository")
en = "My brother told me that the meeting about the new school fees will start at ten tomorrow, so we must leave early."
print(parse("eng_m", generate(tok, model, build_prompt("eng_m", "yor", en)))["text"])                      # translation
print(parse("eng_hm", generate(tok, model, build_prompt("eng_hm", "yor", en, "eng")))["text"])             # code-switching
print(parse("eng_ood", generate(tok, model, build_prompt("eng_ood", "yor", en)))["text"])                  # idiom-rich rewrite

Greedy decoding is what all reported numbers use. For several candidates use temperature 0.7, top-p 0.95; choosing among 8 candidates with the evaluator did not raise chrF++, so do not expect much from sampling.

Evaluation

Full test set (6,230 items, 34 languages, greedy decoding, chrF++ of the final text, macro over languages; same decoder for all rows). "RL best" is the highest-reward checkpoint that passed the per-task chrF guard, "RL latest" the final RL policy.

Task SFT RL best RL latest
eng_hm 50.05 48.78 49.38
m_hm 55.58 54.02 54.57
eng_ood 35.69 36.47 36.44
m_ood 40.49 41.78 41.75
hm_ood 37.96 38.77 38.82

RL raised the idiom tasks (+0.8 to +1.3 chrF++) and lowered the code-switching tasks (-0.7, -1.0); output-format validity on code-switching rose from 0.949 / 0.954 to 0.996 / 0.999. An independent Gemini 3.8 Flash judge on the full test set scored the RL model slightly lower than the SFT model (-0.088 on a 1-5 scale, every task), so for quality-sensitive use the SFT model is the safer choice.

How it was made

A Gemma-3-270M-based decoder first fine-tuned for English-to-African translation, with language and format tags added to the vocabulary and 6 identity-initialised layers interleaved (24 layers, 301M parameters), then fine-tuned on the project's generated code-switching and idiom corpus (832,421 training rows; the text was generated with a 31B instruction model, filtered by rules and by two LLM judges on about 22% of rows, with judge-corrected copies added). RL (GRPO) used a learned reference-free evaluator (mmBERT-base with 27 rating heads) plus deterministic rule penalties on a fresh prompt pool. Not used: SSA-MTE and AfriMTE (benchmark only), NLLB-produced rows, TranslatePsy. Details and ablations: see the project paper.

Limitations and risks

  • Outputs are machine-generated and not reviewed by native speakers. Training data and references were produced and graded with language models; they can contain unnatural phrasing, invented idioms or proverbs, wrong tone marks and meaning drift. Do not use the outputs where correctness matters without human review.
  • Quality varies by language and by pair. Low-resource languages are weaker, a few languages have very few test items, and Fulfulde and Bambara had almost no m_hm prompts in the RL stage; per-language numbers are noisy.
  • Rewriting, not translation. It can drop or change content and add devices that are not in the source. Pidgin outputs keep much English because English words are normal Pidgin.
  • The learned reward could be gamed. In one run the evaluator's score rose while quality fell (the policy drifted toward copying its input); the released RL model uses rules against that, but quality was measured only with chrF++ and the evaluator, not with human raters.
  • No added safety training beyond what the base model had, and no systematic check for offensive or harmful output.

License and attribution

Commercial use is allowed, with attribution to AletheiaAING (see the notice at the top of this card). Because this model is derived from Google Gemma, the Gemma Terms of Use and prohibited use policy also apply. Training data and quality labels used the outputs of other models (a 31B instruction model for generation, Gemini 3.8 Flash as judge); check the terms of those services for your own use. A full data inventory is in the accompanying project report; if you need the exact sources and licences, ask the authors.

Downloads last month
83
Safetensors
Model size
0.3B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for AletheiaAING/csw

Finetuned
(1,196)
this model