Instructions to use zelapi/lilo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use zelapi/lilo with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="zelapi/lilo", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("zelapi/lilo", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("zelapi/lilo", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use zelapi/lilo with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zelapi/lilo"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zelapi/lilo",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/zelapi/lilo

SGLang

How to use zelapi/lilo with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "zelapi/lilo" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zelapi/lilo",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "zelapi/lilo" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zelapi/lilo",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use zelapi/lilo with Docker Model Runner:
```
docker model run hf.co/zelapi/lilo
```

lilo-100m — Lightweight Local Intelligence & Optimizer

A ~110M parameter language model implementing the DeepSeek-V4 architecture, fine-tuned for chat/instruction following in Indonesian. This is the lilo release of neosantara/wader-100m: same architecture and weights, re-packaged explicitly for CPU-only, low-memory, low-footprint use.

Base: neosantara/wader-100m-base · pretrained on neosantara/IndoTalk
Upstream SFT: neosantara/wader-100m on neosantara/IndoTalkSFT
Trained from scratch — no weights from DeepSeek-V4 were used.

Why lilo

	wader-100m (original)	lilo-100m
Weights on disk	421MB fp32	119MB (int8) / 221MB (fp16)
CPU-friendly runtime	bf16 NaN bugs, no KV cache	fp32/fp16 stable, KV cache, inference_mode
Runs on 2 vCPU / 2GB RAM VPS	untested	measured

Variants

Variant	File	Disk	PPL delta vs fp32	Greedy outputs
`fp16` (default)	`model.fp16.safetensors`	221MB	+0.08%	5/5 identical
`int8`	`model.int8.safetensors`	119MB	+0.32%	5/5 identical

Notes:

int8 is symmetric group-wise quantization (group=64) on all nn.Linear layers + embeddings; norms/router/HC params stay fp32. Scales cost ~7MB.
Both dequantize to fp32 compute on load — the disk footprint is what shrinks, because this CPU (Zen2, AVX2-only, no VNNI) does fp32 GEMM as fast as int8 anyway, and fp16 GEMM is faster still.
fp16 is the default because it is both the closest to the original quality and the fastest on CPU.

CPU speed (measured on 2 vCPU AMD EPYC 7K62 "Zen2", AVX2-only, box under load)

Variant	threads=1	threads=2
fp16	14–22 tok/s	11–12 tok/s
int8	9–19 tok/s	5–8 tok/s

On this tiny model (320 hidden, 8 layers) thread sync overhead dominates — --threads 1 is fastest. On 8+ core machines try 2–4.

Memory safety (no OOM)

Peak RSS measured on a 2 vCPU / 1.9GB RAM VPS: ~0.85GB (int8) / ~1.05GB (fp16) including the Python/torch runtime. Rules baked into run_cpu.sh:

OMP_NUM_THREADS bounded, MALLOC_ARENA_MAX=1 (glibc per-thread arenas can silently eat GBs)
TMPDIR forced to disk, never /tmp when /tmp is tmpfs
prefill chunked at 256 tokens so attention never allocates a big [S,S] matrix

Usage

git lfs install
git clone https://huggingface.co/hzlo0/lilo-100m
cd lilo-100m
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install transformers safetensors tokenizers accelerate

bash run_cpu.sh                              # chat, default fp16
CUDA_VISIBLE_DEVICES="" python chat_lilo.py --variant int8 --threads 1 "Halo, apa kabar?"

Chat template (DeepSeek-V4 style):

<|begin_of_sentence|><|User|>Halo<|Assistant|>

import torch
from lilo_common import load_lilo, generate_lilo, chat

torch.set_num_threads(1)
tok, model = load_lilo(".", variant="fp16")          # or "int8"
print(chat(tok, model, [{"role": "user", "content": "Halo, apa kabar?"}],
           max_new_tokens=200, temperature=0.7, top_p=0.9))

Requires trust_remote_code-free loading: the model code ships as plain files (modeling_deepseek_v4.py) loaded by lilo_common.py. (The same code also works via AutoModelForCausalLM.from_pretrained(..., trust_remote_code=True).)

Architecture

Component	Details
Parameters	~110M (41M embeddings, 69M non-embedding)
Hidden size	320 · Layers 8
Attention	8 heads, 1 KV head (MQA), MLA q_lora_rank=160, grouped low-rank output
MoE	4 routed experts + 1 shared, top-2 routing
Hyper-Connections	hc_mult=4, Sinkhorn routing (replaces residual stream)
Vocab	129,280 (DeepSeek-V4 tokenizer) · Context 2,048

What was fixed for CPU in this release

freqs_cis recomputed on demand — original code left a buffer uninitialized after meta-device loading, corrupting RoPE beyond a few tokens.
KV cache honored — original forward hardcoded use_cache=False, recomputing the whole sequence every decode step. This release caches and steps one token.
return_dict defaults to True — transformers 5.x deprecated config.use_return_dict, which silently returned tuples.
Expert dtype bridge — w2(x.to(w2.weight.dtype)) silently cast activations to int8 under weight-only quantization.
bf16 NaN — model runs fp32/fp16; no bf16 anywhere.

Limitations

Tiny model: 110M params, most capacity in embeddings. Generations are often incoherent or factually wrong.
Undertrained: 5K pretrain + 3K SFT steps (upstream). Production models train 100K+ steps on trillions of tokens.

License

Apache-2.0 (inherited from upstream neosantara/wader-100m).

Downloads last month: 228

Model tree for zelapi/lilo

Base model

neosantara/wader-100m-base

Quantized

(1)

this model

Quantizations

1 model

zelapi
/

lilo