How to use from
llama.cpp
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf GrowBitLabs/zehn-135m:F16
# Run inference directly in the terminal:
llama cli -hf GrowBitLabs/zehn-135m:F16
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf GrowBitLabs/zehn-135m:F16
# Run inference directly in the terminal:
llama cli -hf GrowBitLabs/zehn-135m:F16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf GrowBitLabs/zehn-135m:F16
# Run inference directly in the terminal:
./build/bin/llama cli -hf GrowBitLabs/zehn-135m:F16
Use Docker
docker model run hf.co/GrowBitLabs/zehn-135m:F16
Quick Links

Zehn 135M

Zehn 135M is a compact, decoder-only base language model with 134,997,792 parameters. It combines 12 gated DeltaNet layers, 4 grouped-query attention layers, and sparse mixture-of-experts feed-forward blocks. This repository contains the final pretraining checkpoint at optimizer step 32,000.

This is a completion model, not an instruction-tuned assistant. Prompt it with text to continue; it does not use a chat template.

Model details

Property Value
Parameters 134,997,792
Layers 16 (12 DeltaNet, 4 attention)
Hidden size 768
Vocabulary 16,384 SentencePiece BPE tokens
Attention 12 query heads, 4 key/value heads
MoE 4 routed experts, top-2 routing, 1 shared expert
Training sequence length 1,024 tokens
Configured maximum length 330,000 tokens (not validated at this length)
Checkpoint step 32,000
Export precision FP16

The Transformers implementation is custom code supplied in this repository. Loading it requires trust_remote_code=True.

Usage with Transformers

pip install "torch>=2.2" "transformers>=4.40,<5" safetensors sentencepiece
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "GrowBitLabs/zehn-135m"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.float16,
)

inputs = tokenizer("The future of small language models is", return_tensors="pt")
with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=64,
        temperature=0.8,
        top_k=40,
        top_p=0.95,
        repetition_penalty=1.1,
        do_sample=True,
    )

print(tokenizer.decode(output[0], skip_special_tokens=True))

Included formats

Path Purpose
model.safetensors Hugging Face Transformers FP16 weights
native/model.safetensors Native Zehn FP16 tensor names for custom runtimes
gguf/zehn-135m-step-032000-f16.gguf FP16 GGUF container
metadata/ Training configuration, checkpoint provenance, and validation metrics

The GGUF file declares the custom zehn architecture. Stock llama.cpp cannot run it because Zehn's DeltaNet and MoE operators have not been implemented there. The file is provided for compatible or future runtimes; it is not a Llama-compatible model.

Training

The model was pretrained for 32,000 optimizer steps with an effective batch of 32 sequences of 1,024 tokens, or approximately 1.049 billion training-token positions. The source corpus contained 268,444,137 tokens and 735,831 documents, so samples were revisited across training.

The 1.2 GB text mixture was prepared from the pinned 3ba9d605774198c5868892d7a8deda78031a781f revision of HuggingFaceTB/smollm-corpus:

  • 70% FineWeb-Edu-Dedup educational web text
  • 30% Cosmopedia v2 synthetic textbooks, tutorials, and stories

Key settings were BF16 projection compute, AdamW, a peak learning rate of 2e-4, 1,000 warmup steps, weight decay 0.1, gradient clipping at 1.0, and seed 42. The complete configuration is in metadata/training_config.json.

Evaluation

The final checkpoint produced these token-weighted validation results:

Validation source Cross-entropy Perplexity Tokens
Held-out training mixture 2.9731 19.55 21,152

This is a small, 64-batch validation slice, not a full benchmark result. No claims are made for instruction following, reasoning, coding, factual accuracy, safety, or long-context quality.

Limitations and responsible use

This model can generate incorrect, repetitive, biased, offensive, or unsafe text. Its small scale and limited training corpus constrain knowledge and reasoning. It has not been instruction tuned, preference aligned, or comprehensively safety evaluated. Do not use it as an authority for medical, legal, financial, or other high-impact decisions. Review outputs and add application-level safeguards before deployment.

Although the configuration permits very long inputs, training used 1,024-token sequences. Quality, memory use, and numerical behavior beyond that training length have not been established.

License

The model weights and accompanying code in this repository are released under the Apache License 2.0. See LICENSE. The training datasets retain their own terms; the recorded corpus manifest identifies them as ODC-By and should be reviewed independently.

Downloads last month
313
Safetensors
Model size
0.1B params
Tensor type
F16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support