Instructions to use GrowBitLabs/zehn-135m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use GrowBitLabs/zehn-135m with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="GrowBitLabs/zehn-135m", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("GrowBitLabs/zehn-135m", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use GrowBitLabs/zehn-135m with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf GrowBitLabs/zehn-135m:F16 # Run inference directly in the terminal: llama cli -hf GrowBitLabs/zehn-135m:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf GrowBitLabs/zehn-135m:F16 # Run inference directly in the terminal: llama cli -hf GrowBitLabs/zehn-135m:F16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf GrowBitLabs/zehn-135m:F16 # Run inference directly in the terminal: ./build/bin/llama cli -hf GrowBitLabs/zehn-135m:F16
Use Docker
docker model run hf.co/GrowBitLabs/zehn-135m:F16
- Llama new
- LM Studio
- Jan
- vLLM
How to use GrowBitLabs/zehn-135m with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "GrowBitLabs/zehn-135m" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "GrowBitLabs/zehn-135m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/GrowBitLabs/zehn-135m:F16
- SGLang
How to use GrowBitLabs/zehn-135m with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "GrowBitLabs/zehn-135m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "GrowBitLabs/zehn-135m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "GrowBitLabs/zehn-135m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "GrowBitLabs/zehn-135m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Ollama
How to use GrowBitLabs/zehn-135m with Ollama:
ollama run hf.co/GrowBitLabs/zehn-135m:F16
- Unsloth Desktop
- Docker Model Runner
How to use GrowBitLabs/zehn-135m with Docker Model Runner:
docker model run hf.co/GrowBitLabs/zehn-135m:F16
- Lemonade
How to use GrowBitLabs/zehn-135m with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull GrowBitLabs/zehn-135m:F16
Run and chat with the model
lemonade run user.zehn-135m-F16
List all available models
lemonade list
- Atomic Chat
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf GrowBitLabs/zehn-135m:F16# Run inference directly in the terminal:
llama cli -hf GrowBitLabs/zehn-135m:F16Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf GrowBitLabs/zehn-135m:F16# Run inference directly in the terminal:
./build/bin/llama cli -hf GrowBitLabs/zehn-135m:F16Use Docker
docker model run hf.co/GrowBitLabs/zehn-135m:F16Zehn 135M
Zehn 135M is a compact, decoder-only base language model with 134,997,792 parameters. It combines 12 gated DeltaNet layers, 4 grouped-query attention layers, and sparse mixture-of-experts feed-forward blocks. This repository contains the final pretraining checkpoint at optimizer step 32,000.
This is a completion model, not an instruction-tuned assistant. Prompt it with text to continue; it does not use a chat template.
Model details
| Property | Value |
|---|---|
| Parameters | 134,997,792 |
| Layers | 16 (12 DeltaNet, 4 attention) |
| Hidden size | 768 |
| Vocabulary | 16,384 SentencePiece BPE tokens |
| Attention | 12 query heads, 4 key/value heads |
| MoE | 4 routed experts, top-2 routing, 1 shared expert |
| Training sequence length | 1,024 tokens |
| Configured maximum length | 330,000 tokens (not validated at this length) |
| Checkpoint | step 32,000 |
| Export precision | FP16 |
The Transformers implementation is custom code supplied in this repository. Loading it requires trust_remote_code=True.
Usage with Transformers
pip install "torch>=2.2" "transformers>=4.40,<5" safetensors sentencepiece
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "GrowBitLabs/zehn-135m"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
trust_remote_code=True,
torch_dtype=torch.float16,
)
inputs = tokenizer("The future of small language models is", return_tensors="pt")
with torch.no_grad():
output = model.generate(
**inputs,
max_new_tokens=64,
temperature=0.8,
top_k=40,
top_p=0.95,
repetition_penalty=1.1,
do_sample=True,
)
print(tokenizer.decode(output[0], skip_special_tokens=True))
Included formats
| Path | Purpose |
|---|---|
model.safetensors |
Hugging Face Transformers FP16 weights |
native/model.safetensors |
Native Zehn FP16 tensor names for custom runtimes |
gguf/zehn-135m-step-032000-f16.gguf |
FP16 GGUF container |
metadata/ |
Training configuration, checkpoint provenance, and validation metrics |
The GGUF file declares the custom zehn architecture. Stock llama.cpp cannot run it because Zehn's DeltaNet and MoE operators have not been implemented there. The file is provided for compatible or future runtimes; it is not a Llama-compatible model.
Training
The model was pretrained for 32,000 optimizer steps with an effective batch of 32 sequences of 1,024 tokens, or approximately 1.049 billion training-token positions. The source corpus contained 268,444,137 tokens and 735,831 documents, so samples were revisited across training.
The 1.2 GB text mixture was prepared from the pinned 3ba9d605774198c5868892d7a8deda78031a781f revision of HuggingFaceTB/smollm-corpus:
- 70% FineWeb-Edu-Dedup educational web text
- 30% Cosmopedia v2 synthetic textbooks, tutorials, and stories
Key settings were BF16 projection compute, AdamW, a peak learning rate of 2e-4, 1,000 warmup steps, weight decay 0.1, gradient clipping at 1.0, and seed 42. The complete configuration is in metadata/training_config.json.
Evaluation
The final checkpoint produced these token-weighted validation results:
| Validation source | Cross-entropy | Perplexity | Tokens |
|---|---|---|---|
| Held-out training mixture | 2.9731 | 19.55 | 21,152 |
This is a small, 64-batch validation slice, not a full benchmark result. No claims are made for instruction following, reasoning, coding, factual accuracy, safety, or long-context quality.
Limitations and responsible use
This model can generate incorrect, repetitive, biased, offensive, or unsafe text. Its small scale and limited training corpus constrain knowledge and reasoning. It has not been instruction tuned, preference aligned, or comprehensively safety evaluated. Do not use it as an authority for medical, legal, financial, or other high-impact decisions. Review outputs and add application-level safeguards before deployment.
Although the configuration permits very long inputs, training used 1,024-token sequences. Quality, memory use, and numerical behavior beyond that training length have not been established.
License
The model weights and accompanying code in this repository are released under the Apache License 2.0. See LICENSE. The training datasets retain their own terms; the recorded corpus manifest identifies them as ODC-By and should be reviewed independently.
- Downloads last month
- 313
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf GrowBitLabs/zehn-135m:F16# Run inference directly in the terminal: llama cli -hf GrowBitLabs/zehn-135m:F16