How to use from
SGLangUse Docker images
docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "teddyoas/mathcode-llm" \
--host 0.0.0.0 \
--port 30000# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "teddyoas/mathcode-llm",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'Quick Links
mathcode-llm
A decoder-only transformer language model built from scratch -- no
pretrained weights, no transformers model classes. Every layer (RMSNorm,
RoPE, grouped-query attention, SwiGLU, KV-cached decoding) is hand-written
PyTorch in model/transformer.py, and the BPE tokenizer was trained from
scratch on our own corpus.
Training data
| Source | Share |
|---|---|
open-web-math (arXiv papers, Math StackExchange) |
50.2% |
codeparrot-clean (Python from GitHub) |
37.7% |
code-search-net (Python / JS / Go / Java functions) |
5.8% |
verifiable-coding-problems-python |
4.8% |
| The Nature of Code (Daniel Shiffman, full 13 chapters) | 1.4% |
600 M training tokens, interleaved at 4096-token granularity so the domain mixture stays stable throughout training.
Architecture
| Parameters | ~110 M (85 M non-embedding) |
| Layers / heads / width | 12 / 12 / 768 |
| Context length | 1024 tokens |
| Vocabulary | 32,768 (byte-level BPE, trained from scratch) |
| Normalisation | RMSNorm (pre-norm) |
| Position | RoPE |
| FFN | SwiGLU |
| Attention | multi-head, KV cache for fast sampling |
| Embeddings | tied |
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("teddyoas/mathcode-llm")
model = AutoModelForCausalLM.from_pretrained("teddyoas/mathcode-llm")
The tokenizer is a PreTrainedTokenizerFast built with the tokenizers
library, so AutoTokenizer loads it directly.
Resuming training
checkpoint/latest.pt holds the model, the optimiser state, the AMP scaler,
the step counter and the token counter:
python train.py --config configs/mathcode.yaml \
--resume checkpoint/latest.pt
Status
Training checkpoint: step 100 / 4577 (13,107,200 tokens seen, val loss inf)
- Downloads last month
- 34
Install from pip and serve model
# Install SGLang from pip: pip install sglang# Start the SGLang server: python3 -m sglang.launch_server \ --model-path "teddyoas/mathcode-llm" \ --host 0.0.0.0 \ --port 30000# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "teddyoas/mathcode-llm", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'