ai4ug/pearl-chat-85m-base
Pure-JAX / Flax NNX Luganda language model (foundational base checkpoint) trained on verified Ugandan encyclopedic, UBOS 136-district, lg.wikipedia.org, clinical health, and Engero z'Abaganda corpora.
Model architecture
- Preset:
pearl-v5-85m - Total parameters:
85,219,968(74,734,208transformer body parameters) - Geometry:
d_model=640,n_layers=16,n_heads=10,n_kv_heads=5,d_ff=1792,max_seq_len=1024,tie_word_embeddings=True - Attention mechanism: Hybrid 3:1 Sliding Window Attention (
W=256,rope_theta=10000) interleaved with Global GQA every4th and final layer (global_rope_theta=50000) - Tokenizer:
16,384-vocabulary Luganda Byte-BPE (tokenizer.json)
- Downloads last month
- 26
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support