ai4ug/pearl-chat-85m-base

Pure-JAX / Flax NNX Luganda language model (foundational base checkpoint) trained on verified Ugandan encyclopedic, UBOS 136-district, lg.wikipedia.org, clinical health, and Engero z'Abaganda corpora.

Model architecture

  • Preset: pearl-v5-85m
  • Total parameters: 85,219,968 (74,734,208 transformer body parameters)
  • Geometry: d_model=640, n_layers=16, n_heads=10, n_kv_heads=5, d_ff=1792, max_seq_len=1024, tie_word_embeddings=True
  • Attention mechanism: Hybrid 3:1 Sliding Window Attention (W=256, rope_theta=10000) interleaved with Global GQA every 4th and final layer (global_rope_theta=50000)
  • Tokenizer: 16,384-vocabulary Luganda Byte-BPE (tokenizer.json)
Downloads last month
26
Safetensors
Model size
85.2M params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support