DarkGenius01/Intelligence_OS_1p5B-GGUF

Intelligence_OS_1p5B — GGUF

GGUF build of shreyansh-programmer/Intelligence_OS_1p5B, a from-scratch decoder-only sparse MoE model. Runs in LM Studio, llama.cpp, ollama (import), and anything else that speaks GGUF.

The upstream repo ships raw PyTorch checkpoints (intelligence_os_1p5b_latest.pt) written with a custom model.py — there is no transformers model, and no usable tokenizer. This repo contains the converted weights, so you can use the model without reproducing the conversion.

Files

file	size	notes
Intelligence_OS_1p5B-Q4_K_M.gguf	424 MiB	4-bit; the one to use
Intelligence_OS_1p5B-f16.gguf	1 371 MiB	reference build, matches the PyTorch model

What the model actually is

Values read from the checkpoint itself (the upstream config.json does not match the weights):

parameters	718 505 984 (~0.72 B), fp16
layers	24
hidden size	1024
attention	GQA, 16 query / 4 KV heads, head_dim 64
FFN	MoE, 4 SwiGLU experts, top-2, expert hidden 2048
norms	pre-norm RMSNorm, eps 1e-5
positions	RoPE, theta 10000, max context 4096
vocab	50 257 (GPT-2 BPE)
embeddings	tied
training	40 009 steps, 163 876 864 tokens, ~60 h

So it is a 0.72 B model, not 1.5 B: "1.5B" is the theoretical configuration in the upstream config.json (26 layers / hidden 2048 / vocab 49152), which never matches the released weights.

How it was converted

The architecture is structurally Mixtral (GQA + RoPE + pre-norm RMSNorm + SwiGLU MoE top-2), so the tensors were renamed to the MixtralForCausalLM layout (which current llama.cpp maps to its llama architecture, MoE-capable) and converted with the stock convert_hf_to_gguf.py. Scripts are in scripts/.

Two details that are easy to get wrong, and were handled here:

1. RoPE convention. The model rotates pairs of neighbouring values, while convert_hf_to_gguf.py assumes HF-Llama's rotate_half layout and permutes q_proj/k_proj. The conversion therefore pre-applies the inverse permutation — with n_kv_heads (4) for k_proj, not n_heads (16).
2. Tokenizer. The upstream repo only has a 512-token stub. The embedding has 50 257 rows, so the GPT-2 BPE tokenizer is used. This is an inference (no tokenizer files survive in the upstream repo), but generated text is coherent English, which is consistent with it.

Verification: greedy decoding of the f16 build reproduces the PyTorch reference (token-for-token, temperature = 0), and perplexity drops from 3739 (a build with the k_proj bug) to 134.8 (f16) / 134.1 (Q4_K_M) on the same test text.

Usage

LM Studio

lms import Intelligence_OS_1p5B-Q4_K_M.gguf --user-repo local/intelligence-os -c

Or drag the file into the app. Chat template: use Empty (or the Jinja equivalent {% for message in messages %}{{ message['content'] + '\n' }}{% endfor %}) — the model is a base model and was never instruction-tuned, so any chat scaffolding only hurts.

llama.cpp

llama-cli -m Intelligence_OS_1p5B-Q4_K_M.gguf -p "Step 1:" -n 128 --temp 0.2 --repeat-penalty 1.3

ollama

cat > Modelfile <<'EOF'
FROM ./Intelligence_OS_1p5B-Q4_K_M.gguf
TEMPLATE "{{ .Prompt }}"
PARAMETER temperature 0.2
PARAMETER repeat_penalty 1.3
EOF
ollama create intelligence-os -f Modelfile

Sampling matters a lot

The model has a very peaked, repetitive next-token distribution. With LM Studio defaults (temp 0.8, top_k 40, top_p 0.95, repeat_penalty 1.1) it collapses into loops ($16 * $16 * $16 …). Verified on this build:

settings	output for The capital of France is
temp 0	Step 4: Multiply df/du by 2: f'(x) = 4.00 …
temp 0.8 / k40 / p0.95 / rep 1.1	… second second second … space space space … (loop)
temp 0.8 + DRY (0.8 / 1.75 / 2 / 512)	Step 1: … 3/2 + 3/4 = 9/25 …

Recommended: temperature 0.1–0.3, top_k 20–40, top_p 0.9, repeat_penalty 1.2–1.3, and DRY if your runtime exposes it. Keep answers short (100–150 tokens).

Limitations

• Not an assistant. It was trained on 164 M tokens (≈85× less than Chinchilla-optimal for this size) and mostly continues "textbook" patterns: math word problems, Step 1: …, Therefore if …. Ask it a question and you will often get math soup instead of an answer.
• No chat template is embedded: tokenizer.chat_template is intentionally absent because the model has no instruction format.
• Tokenizer is inferred (GPT-2 BPE), see above. If the original training used an extended GPT-2 vocabulary, some prompts will be tokenized differently.
• Upstream describes the model as a research prototype ("status": "architecture scaffold"), and this repo is a faithful conversion of exactly that.
• Converted and quantized on Apple Silicon; the 4-bit build was produced with llama-quantize … Q4_K_M and loads in LM Studio's runtime (llama.cpp 2.70.0).

Reproduction

# 1. repack the PyTorch checkpoint into a Mixtral-style HF directory
python scripts/ios2hf.py intelligence_os_1p5b_latest.pt ./hf --heads 16 --head-dim 64 --tokenizer-dir ./gpt2-tok

# 2. convert + quantize
python convert_hf_to_gguf.py ./hf --outtype f16 --outfile ios-f16.gguf
llama-quantize ios-f16.gguf ios-Q4_K_M.gguf Q4_K_M

scripts/inspect_ckpt.py prints what is inside a checkpoint, scripts/check_gguf.py verifies the resulting GGUF (architecture, hyperparameters, MoE tensors, tokenizer).

Licence

Apache-2.0, inherited from the upstream model. Upstream: shreyansh-programmer/Intelligence_OS_1p5B.