Instructions to use lillycham/tiny-lm-web-98m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lillycham/tiny-lm-web-98m with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lillycham/tiny-lm-web-98m")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("lillycham/tiny-lm-web-98m")
model = AutoModelForCausalLM.from_pretrained("lillycham/tiny-lm-web-98m", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use lillycham/tiny-lm-web-98m with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lillycham/tiny-lm-web-98m"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lillycham/tiny-lm-web-98m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/lillycham/tiny-lm-web-98m

SGLang

How to use lillycham/tiny-lm-web-98m with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lillycham/tiny-lm-web-98m" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lillycham/tiny-lm-web-98m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lillycham/tiny-lm-web-98m" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lillycham/tiny-lm-web-98m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use lillycham/tiny-lm-web-98m with Docker Model Runner:
```
docker model run hf.co/lillycham/tiny-lm-web-98m
```

tiny-lm-web-98m / README.md

lillycham

Upload folder using huggingface_hub

d938f5a verified 3 days ago

preview code

Raw History Blame Contribute Delete

2.69 kB

---
license: mit
language: en
library_name: transformers
pipeline_tag: text-generation
tags:
  - gpt2
  - tiny-lm
  - educational
datasets:
  - HuggingFaceFW/fineweb-edu
  - SimpleStories/SimpleStories
  - databricks/databricks-dolly-15k
---
# tiny-lm-web-98m

A 98M-parameter GPT-2 small shaped model, trained from scratch on web text. It is from Part 3 of
[tiny-lm](https://github.com/lillycham/tiny-lm), a project that builds a language model step by
step and then looks inside it.

| | |
|---|---|
| Size | 12 layers, width 768, 12 heads, 98.4M parameters |
| Context | 1,024 tokens |
| Tokeniser | byte-level BPE, 16,384 tokens, trained on FineWeb-Edu |
| Training | 2.70B tokens of FineWeb-Edu, 203 minutes on one RTX 5090 |
| Validation loss | 2.983 nats per token, 1.03 bits per character |

For comparison, OpenAI's GPT-2 small is about 1.04 bits per character on the same text.

There are two models in this repository:

- **The root** is the base model. It continues text.
- **`chat/`** is the base model after a 150M-token anneal (25% SimpleStories), then supervised
  fine-tuning on Dolly 15k, then fine-tuning on `System:` lines (rules such as "Answer in one
  sentence", and names for the model and the user).

The weights are a standard `GPT2LMHeadModel`. The q, k and v biases that GPT-2 has are zero,
because the model was trained without them.

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "lillycham/tiny-lm-web-98m"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)
ids = tok("The water cycle is", return_tensors="pt").input_ids
ids = torch.cat([torch.tensor([[tok.eos_token_id]]), ids], dim=1)   # text starts after <|endoftext|>
out = model.generate(ids, max_new_tokens=100, do_sample=True, temperature=0.8)
print(tok.decode(out[0, 1:]))
```

The chat model uses this format, and ends its answer with `<|endoftext|>`. Load it with
`subfolder="chat"`. The `System:` line is optional.

```
System: Answer in one sentence.
User: Why is the sky blue?
Assistant:
```

## Limits

The text is fluent, but most facts are invented. The chat model answers in the right form (a list,
verse or one line) but does not know much. It does not yet take answers from a paragraph in its
context. Do not use it for anything that needs true answers.

## Data and licence

Trained on [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) (ODC-By),
[SimpleStories](https://huggingface.co/datasets/SimpleStories/SimpleStories) (MIT) and, for the chat
model, [Dolly 15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) (CC BY-SA 3.0).
The code and weights are MIT.

	---
	license: mit
	language: en
	library_name: transformers
	pipeline_tag: text-generation
	tags:
	- gpt2
	- tiny-lm
	- educational
	datasets:
	- HuggingFaceFW/fineweb-edu
	- SimpleStories/SimpleStories
	- databricks/databricks-dolly-15k
	---

	# tiny-lm-web-98m

	A 98M-parameter GPT-2 small shaped model, trained from scratch on web text. It is from Part 3 of
	[tiny-lm](https://github.com/lillycham/tiny-lm), a project that builds a language model step by
	step and then looks inside it.

	\| \| \|
	\|---\|---\|
	\| Size \| 12 layers, width 768, 12 heads, 98.4M parameters \|
	\| Context \| 1,024 tokens \|
	\| Tokeniser \| byte-level BPE, 16,384 tokens, trained on FineWeb-Edu \|
	\| Training \| 2.70B tokens of FineWeb-Edu, 203 minutes on one RTX 5090 \|
	\| Validation loss \| 2.983 nats per token, 1.03 bits per character \|

	For comparison, OpenAI's GPT-2 small is about 1.04 bits per character on the same text.

	There are two models in this repository:

	- The root is the base model. It continues text.
	- `chat/` is the base model after a 150M-token anneal (25% SimpleStories), then supervised
	fine-tuning on Dolly 15k, then fine-tuning on `System:` lines (rules such as "Answer in one
	sentence", and names for the model and the user).

	The weights are a standard `GPT2LMHeadModel`. The q, k and v biases that GPT-2 has are zero,
	because the model was trained without them.

	```python
	import torch
	from transformers import AutoModelForCausalLM, AutoTokenizer

	repo = "lillycham/tiny-lm-web-98m"
	tok = AutoTokenizer.from_pretrained(repo)
	model = AutoModelForCausalLM.from_pretrained(repo)

	ids = tok("The water cycle is", return_tensors="pt").input_ids
	ids = torch.cat([torch.tensor([[tok.eos_token_id]]), ids], dim=1) # text starts after <\|endoftext\|>
	out = model.generate(ids, max_new_tokens=100, do_sample=True, temperature=0.8)
	print(tok.decode(out[0, 1:]))
	```

	The chat model uses this format, and ends its answer with `<\|endoftext\|>`. Load it with
	`subfolder="chat"`. The `System:` line is optional.

	```
	System: Answer in one sentence.
	User: Why is the sky blue?
	Assistant:
	```

	## Limits

	The text is fluent, but most facts are invented. The chat model answers in the right form (a list,
	verse or one line) but does not know much. It does not yet take answers from a paragraph in its
	context. Do not use it for anything that needs true answers.

	## Data and licence

	Trained on [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) (ODC-By),
	[SimpleStories](https://huggingface.co/datasets/SimpleStories/SimpleStories) (MIT) and, for the chat
	model, [Dolly 15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) (CC BY-SA 3.0).
	The code and weights are MIT.