Trilingua (base)

A 70.5M-parameter decoder-only language model for English, Spanish and French, trained from scratch (tokenizer included) for the Arbiter Hacks V1 hackathon. Architecture: pre-norm transformer, RMSNorm, RoPE, SwiGLU, tied embeddings, 10 layers, d_model 640, 10 heads, context 1024.

This is NOT a transformers model; load it with the code at the project repository:

from trilingua.utils import load_model
from trilingua.tokenizer import TrilinguaTokenizer
model = load_model("path/to/this/folder")
tok = TrilinguaTokenizer.from_file("path/to/this/folder/tokenizer.json")

Prompt format: start with a language tag <|en|>, <|es|> or <|fr|>.

Training step: 2385. See the repository README for data, metrics and limitations. Small model: expect fluent-looking but often factually wrong text.

Downloads last month
105
Safetensors
Model size
70.5M params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Datasets used to train aladewealth/trilingua