Instructions to use thesreedath/slm-500m-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use thesreedath/slm-500m-base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="thesreedath/slm-500m-base")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("thesreedath/slm-500m-base")
model = AutoModelForCausalLM.from_pretrained("thesreedath/slm-500m-base", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use thesreedath/slm-500m-base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "thesreedath/slm-500m-base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "thesreedath/slm-500m-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/thesreedath/slm-500m-base

SGLang

How to use thesreedath/slm-500m-base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "thesreedath/slm-500m-base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "thesreedath/slm-500m-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "thesreedath/slm-500m-base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "thesreedath/slm-500m-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use thesreedath/slm-500m-base with Docker Model Runner:
```
docker model run hf.co/thesreedath/slm-500m-base
```

slm-500m-base

A 517.8M-parameter Llama-style decoder, pretrained from scratch (fresh weights, fresh tokenizer) on a legal/financial corpus. This is a base model: it completes text, it does not answer questions.

Architecture


parameters	517,795,840
layers	24
hidden size	1280
attention heads	20 (head dim 64, MHA)
MLP	SwiGLU, inner 3456
norm	RMSNorm, pre-norm
positions	RoPE (theta 10000)
context length	1024
vocab	32,768 (byte-level BPE, trained on this corpus)
embeddings	tied
precision	bf16

Training

Data: thesreedath/slm-pretraining-corpus (~2.19B unique tokens; US case law, SEC filings, fineweb-edu), cleaned, MinHash-deduplicated, and decontaminated against CaseHOLD/LexGLUE.
Recipe: AdamW (0.9, 0.95), wd 0.1, LR 0.0003 -> 3e-05 cosine, 200M-token warmup, global batch 524,288 tokens, grad clip 1.0.
Hardware: 8x B200, DDP, torch.compile.
Epochs: n/a | tokens seen: 0.0B

Results

metric	value
validation perplexity	7.912
bits per byte	n/a

Read bits-per-byte, not perplexity, when comparing this to a model with a different tokenizer. Perplexity is per-token: a larger vocabulary packs more text into each token, making each prediction harder and the perplexity higher, for a model that is doing better. We measured exactly that during this run: at step 3000 this model's perplexity (11.51) looked worse than the 125M's (11.33) while its bits-per-byte (0.694) was 3.9% better. Bits-per-byte is tokenizer-invariant and is the honest cross-model number.

What this model is not

A base model is a completer. Give it the start of a passage and it continues in-register. It does not follow instructions, and it does not reliably know facts: at this scale, knowledge capacity is roughly 2 bits per parameter. For question answering, use the fine-tuned variants; for factual reliability, use retrieval.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("thesreedath/slm-500m-base")
model = AutoModelForCausalLM.from_pretrained("thesreedath/slm-500m-base")

ids = tok("The plaintiff alleges that the defendant", return_tensors="pt")
out = model.generate(**ids, max_new_tokens=60, do_sample=True, top_k=50,
                     temperature=0.8, min_new_tokens=40)
print(tok.decode(out[0], skip_special_tokens=True))

min_new_tokens matters: a base model will otherwise sometimes emit EOS immediately.

Downloads last month: 61

Safetensors

Model size

0.5B params

Tensor type

BF16

Model tree for thesreedath/slm-500m-base

Finetunes

4 models