solintellegence/sol-lassi

Sol Lassi 600K Base

Sol Lassi is an experimental 600,000-parameter decoder-only language model from Sol Intelligence. It was trained from scratch for 1 billion token exposures with M-SimOW.

This is a small base language model, not an instruction-tuned assistant. No Intelligence Index or leaderboard score is claimed for this checkpoint.

Model summary

Property	Value
Parameters	600,000
Architecture	Dense decoder-only causal language model
Transformer blocks	6 independent blocks
Hidden width	96
Attention	3 heads, 32 dimensions per head
MLP	Gated SiLU, width 104
Vocabulary	2,048-token byte-level BPE
Input/output embeddings	Tied
Training context	128 tokens
Weight format	MLX NPZ
Training tokens	1,000,000,000 exposures

Training

Sol Lassi was trained on a balanced FinePhrase stream composed of its faq, math, table, and tutorial configurations. Training used 128-token sequences, batch size 32, and seed 7.

The optimizer was M-SimOW (momentum beta 0.8) with decoupled weight decay of 0.1. The fixed learning-rate schedule was:

Training span	Learning rate
First 500M tokens	0.006
Next 500M tokens	0.003

The saved run used the FinePhrase-balanced-500m-2k-v2 manifest. It contains 500M training-stream tokens; the model received 1B token exposures. The tokenizer and training metadata are included in this repository.

Quick start

Sol Lassi uses MLX and is intended to run on Apple Silicon.

pip install -r requirements.txt

from huggingface_hub import snapshot_download
from modeling_sol_lassi import load_model, generate

model_dir = snapshot_download("solintellegence/sol-lassi")
model, tokenizer = load_model(model_dir)
print(generate(model, tokenizer, "A tiny language model can", max_new_tokens=48))

Evaluation status

This release does not include Open-SLM, ArithMark, or Intelligence Index evaluations. Its training loss logs are per-minibatch measurements and are not presented as held-out performance.

Intended use and limitations

Sol Lassi is for research on very small language models and optimizer experiments. At 600K parameters, it has substantial capability limits; its output may be incoherent, repetitive, factually incorrect, or unsafe. It is not suitable for high-stakes use.

Repository files

File	Purpose
model.npz	Trained MLX weights
modeling_sol_lassi.py	Standalone loader and generation helper
keystone_mlx/	Architecture implementation required by the loader
tokenizer.json	Frozen 2,048-token tokenizer
tokenizer_config.json	Tokenizer metadata
config.json	Architecture metadata
training_state.json	Training and provenance summary
sol-lassi-banner.png	Sol Lassi model artwork

License

Released under CC BY 4.0. Dataset terms remain with the upstream dataset providers.