teapot1alpha

135M Llama trained from scratch with the SmolLM2 tokenizer. Pretraining was 40B tokens. Supervised training was the Teapot mix. The files at the repo root are the best supervised checkpoint (checkpoints/sft_80pct).

The chat template is the plain System / User / Assistant text this model was trained on.

from transformers import pipeline

pipe = pipeline("text-generation", model="teapotai/teapot1alpha")
pipe([{"role": "user", "content": "Who are you?"}])

Other checkpoints are in checkpoints/. checkpoints/pretrain is the base model. checkpoints/sft_end is the final supervised step. checkpoints/best is the same weights as this repo root.

from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

repo = "teapotai/teapot1alpha"
model = AutoModelForCausalLM.from_pretrained(repo, subfolder="checkpoints/pretrain")
tokenizer = AutoTokenizer.from_pretrained(repo, subfolder="checkpoints/pretrain")
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
Downloads last month
34
Safetensors
Model size
0.1B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support