Instructions to use altslate/JugnuLM-110M-R2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use altslate/JugnuLM-110M-R2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="altslate/JugnuLM-110M-R2", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("altslate/JugnuLM-110M-R2", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use altslate/JugnuLM-110M-R2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "altslate/JugnuLM-110M-R2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "altslate/JugnuLM-110M-R2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/altslate/JugnuLM-110M-R2

SGLang

How to use altslate/JugnuLM-110M-R2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "altslate/JugnuLM-110M-R2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "altslate/JugnuLM-110M-R2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "altslate/JugnuLM-110M-R2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "altslate/JugnuLM-110M-R2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use altslate/JugnuLM-110M-R2 with Docker Model Runner:
```
docker model run hf.co/altslate/JugnuLM-110M-R2
```

JugnuLM-110M-R2 / README.md

rajpdus

Model card: cite arXiv:2609.14715

bc9183a verified 5 days ago

preview code

Raw History Blame Contribute Delete

4.55 kB

	---
	license: apache-2.0
	language:
	- en
	library_name: transformers
	pipeline_tag: text-generation
	tags:
	- tiny-lm
	- qwen3
	- value-residual
	- resformer
	- muon
	- ablation
	datasets:
	- HuggingFaceFW/fineweb-edu
	---

	# JugnuLM-110M-R2 (value residuals + Muon)

	What "R2" means. This is rung 2 of the JugnuLM phase-2 ablation ladder. We take
	rung R1 — [JugnuLM-110M-R1](https://huggingface.co/altslate/JugnuLM-110M-R1) (value
	residuals) — and add one change: we swap the optimizer to Muon for the 2D hidden
	matrices, holding everything else fixed (same value-residual architecture, data, tokens,
	schedule shape, batch, seed). The ladder keeps a lever only if it beats the previous rung.
	R2 does, so it's kept.

	## What changed: the Muon optimizer

	R0 and R1 trained with AdamW on every parameter. R2 uses Muon — Newton-Schulz
	orthogonalized momentum updates — on the 2D hidden weight matrices (attention + MLP,
	81.4M params), and keeps AdamW for the rest (tied embedding/head, RMSNorm gains, and
	the 22 value-residual `vr_lambda` scalars, 28.3M params). Muon's advantage is known to be
	largest at exactly this scale (sub-150M) and to shrink as models grow.

	Both optimizers ride one shared cosine LR multiplier: Muon peak `2e-2`, AdamW peak `1.5e-3`,
	each annealed to 10% of peak.

	## Result vs the R1 and R0 rungs

	\| 110M rung \| BLiMP ↑ \| ARC-Easy ↑ \| WikiText-2 byte-ppl ↓ \|
	\|---\|---\|---\|---\|
	\| R0 — baseline \| 81.25 \| 52.48 \| 1.95 \|
	\| R1 — + value residuals \| 81.10 \| 54.67 \| 1.940 \|
	\| R2 — + Muon \| 80.78 \| 56.10 \| 1.932 \|

	**Muon lifts ARC-Easy by +1.43 over R1 (54.67 → 56.10) and lowers perplexity, at a small
	BLiMP dip (−0.32, ~2× SE). Across the two ladder rungs ARC-Easy has climbed +3.6**
	(52.48 → 56.10) while BLiMP stayed ~flat — steadily closing the reasoning gap to the
	125–143M leaderboard models. Evaluated with EleutherAI `lm-evaluation-harness`.

	Training-efficiency note. Muon's convergence lead was largest early — val-perplexity
	was −26% vs R1 at step 1000 — and compressed to ~−1% by the end of the fixed 8.4B-token
	budget (the classic "Muon wins the sprint, converges toward AdamW" pattern). Notably, the
	downstream ARC gain persisted even as the final perplexity gap narrowed.

	## ⚠️ Loading — requires `trust_remote_code`

	Value residuals are a custom attention pathway, so this model must be loaded with the
	included modeling file. Loading it as a stock Qwen3 silently drops the residual and
	understates quality.

	```python
	from transformers import AutoModelForCausalLM, AutoTokenizer
	tok = AutoTokenizer.from_pretrained("altslate/JugnuLM-110M-R2")
	model = AutoModelForCausalLM.from_pretrained("altslate/JugnuLM-110M-R2", trust_remote_code=True)
	ids = tok("The theory of relativity was developed by", return_tensors="pt").input_ids
	print(tok.decode(model.generate(ids, max_new_tokens=30, repetition_penalty=1.3)[0], skip_special_tokens=True))
	```

	## Architecture & training

	Identical architecture to R1: Qwen3-arch + value-residual pathway, 23 layers × 576,
	GQA 9/3, RoPE, SwiGLU, RMSNorm, QK-Norm, tied embeddings, SmolLM2 tokenizer (49,152),
	z-loss 1e-4. Trained from scratch on ~8.4B tokens of FineWeb-Edu (sample-10BT), shared
	cosine schedule (Muon 2e-2, AdamW 1.5e-3, → 10% floor), ~0.5M-token global batch, bf16,
	DDP on 4× RTX PRO 4500 Blackwell. Only the optimizer differs from R1.

	## Intended use & limitations

	Research base model (not instruction-tuned). Fluent English continuation and
	reasonable recall for its size; limited multi-step reasoning, occasional repetition,
	English-only. Not for production.

	## Links

	- Previous rung: [altslate/JugnuLM-110M-R1](https://huggingface.co/altslate/JugnuLM-110M-R1)
	- Baseline: [altslate/JugnuLM-110M](https://huggingface.co/altslate/JugnuLM-110M)
	- Training code & ablation ladder: https://github.com/AltSlate-Labs/jugnu

	## Paper

	This model is described in [Depth and Scale in the Sub-150M Regime: JugnuLM-53M vs JugnuLM-110M](https://arxiv.org/abs/2609.14715)
	(arXiv:2609.14715 · [HF Papers](https://huggingface.co/papers/2609.14715)).

	```bibtex
	@misc{rajput2026jugnu,
	title = {Depth and Scale in the Sub-150M Regime: JugnuLM-53M vs JugnuLM-110M},
	author = {Rajput, Dushyant and Chauhan, Nirdesh and Kosaraju, Siddharth},
	year = {2026},
	eprint = {2609.14715},
	archivePrefix = {arXiv},
	primaryClass = {cs.AI},
	url = {https://arxiv.org/abs/2609.14715}
	}
	```