Instructions to use jaweed123/TinyJLLM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jaweed123/TinyJLLM with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jaweed123/TinyJLLM")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("jaweed123/TinyJLLM")
model = AutoModelForCausalLM.from_pretrained("jaweed123/TinyJLLM", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use jaweed123/TinyJLLM with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jaweed123/TinyJLLM:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf jaweed123/TinyJLLM:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jaweed123/TinyJLLM:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf jaweed123/TinyJLLM:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf jaweed123/TinyJLLM:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf jaweed123/TinyJLLM:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf jaweed123/TinyJLLM:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf jaweed123/TinyJLLM:Q4_K_M

Use Docker

docker model run hf.co/jaweed123/TinyJLLM:Q4_K_M

LM Studio
Jan

vLLM

How to use jaweed123/TinyJLLM with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jaweed123/TinyJLLM"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jaweed123/TinyJLLM",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/jaweed123/TinyJLLM:Q4_K_M

SGLang

How to use jaweed123/TinyJLLM with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jaweed123/TinyJLLM" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jaweed123/TinyJLLM",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jaweed123/TinyJLLM" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jaweed123/TinyJLLM",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Ollama
How to use jaweed123/TinyJLLM with Ollama:
```
ollama run hf.co/jaweed123/TinyJLLM:Q4_K_M
```
Unsloth Desktop
Docker Model Runner
How to use jaweed123/TinyJLLM with Docker Model Runner:
```
docker model run hf.co/jaweed123/TinyJLLM:Q4_K_M
```

Lemonade

How to use jaweed123/TinyJLLM with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull jaweed123/TinyJLLM:Q4_K_M

Run and chat with the model

lemonade run user.TinyJLLM-Q4_K_M

List all available models

lemonade list

Atomic Chat

TinyJLLM — 100M-parameter small language model built from scratch

A decoder-only Transformer (~102.5M parameters) pretrained from random initialization on ~5 GB of FineWeb (sample-10BT), 3 epochs / 108,000 optimizer steps. Every component — tokenizer, data pipeline, model, training loop, evaluation, export — is implemented from scratch in the TinyLLM repository.

Final metrics: validation loss 3.50 (perplexity 33.1); the best checkpoint reached 3.48 / 32.6.

Model family

Model	Stage	Link
`TinyJLLM`	Base (this model)	jaweed123/TinyJLLM
`TinyJLLM-Instruct`	Supervised fine-tuning	jaweed123/TinyJLLM-Instruct
`TinyJLLM-Instruct-DPO`	DPO preference tuning	jaweed123/TinyJLLM-Instruct-DPO

Model details

Property	Value
Parameters	102,450,432 (~102.5M)
Architecture	Llama-style decoder-only: RMSNorm, RoPE (half-split), SwiGLU, tied embeddings, no biases
Layers / heads / head_dim	11 / 12 / 64
Context length	512
Vocabulary	32,000 (custom byte-level BPE, `<pad> <unk> <bos> <eos>` = 0–3)
Pretraining data	FineWeb `sample-10BT`, ~5.37 GB raw, 1.75M documents
Tokens seen	3.54B (3 epochs)
Hardware	RTX 4060 8 GB, ~30K tok/s (`torch.compile`)
Precision	BF16 mixed precision, FP32 master weights

Intended use

Educational reference: inspect a small, complete, honest pretraining run.
Qualitative experimentation: prompt it (a base model — it continues text, it does not follow instructions).
A starting point for post-training (see the Instruct and DPO variants).

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("jaweed123/TinyJLLM")
tokenizer = AutoTokenizer.from_pretrained("jaweed123/TinyJLLM")

prompt = "The future of AI is"
inputs = tokenizer(prompt, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=50, temperature=0.8, top_k=50, top_p=0.95)
print(tokenizer.decode(out[0]))

llama.cpp / GGUF

This repo also ships GGUF files (F16, Q8_0, Q4_K_M) — load directly with llama.cpp or llama-cpp-python.

Training details

Custom 32K byte-level BPE tokenizer (trained on a 512 MB FineWeb sample).
Tokens stored once as uint16 shards (591 train + 6 validation).
AdamW (lr 3e-4, wd 0.1, decay/no-decay groups), 1,000-step warmup + cosine decay to 1e-5, effective batch 32,768 tokens/step, gradient clipping 1.0.
Full run: ~35 h on a single RTX 4060.

Limitations

Small scale ⇒ repetition in long generations, weak instruction following, limited and unreliable world knowledge. Not for factual or consequential use.

Citation

@misc{tinyjllm,
  title  = {TinyJLLM: A 100M-Parameter Small Language Model Built From Scratch},
  author = {Jaweed, Abdul},
  year   = {2026},
  url    = {https://github.com/Abdul-Jaweed/TinyLLM}
}

Acknowledgments

FineWeb (HuggingFaceFW), Hugging Face tokenizers / datasets, PyTorch, and llama.cpp. Built and measured with the TinyLLM pipeline.

Downloads last month: 806

Safetensors

Model size

0.1B params

Tensor type

F32

Model tree for jaweed123/TinyJLLM

Finetunes

1 model

jaweed123
/

TinyJLLM