Instructions to use HuggingFaceTB/SmolLM3-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use HuggingFaceTB/SmolLM3-3B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="HuggingFaceTB/SmolLM3-3B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/SmolLM3-3B") model = AutoModelForCausalLM.from_pretrained("HuggingFaceTB/SmolLM3-3B", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use HuggingFaceTB/SmolLM3-3B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "HuggingFaceTB/SmolLM3-3B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HuggingFaceTB/SmolLM3-3B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/HuggingFaceTB/SmolLM3-3B
- SGLang
How to use HuggingFaceTB/SmolLM3-3B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "HuggingFaceTB/SmolLM3-3B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HuggingFaceTB/SmolLM3-3B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "HuggingFaceTB/SmolLM3-3B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HuggingFaceTB/SmolLM3-3B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use HuggingFaceTB/SmolLM3-3B with Docker Model Runner:
docker model run hf.co/HuggingFaceTB/SmolLM3-3B
Three small files disagree: pad id, a decode-cleanup setting, and a length claim twice the config span
#58
by nootxlm - opened
Hello, and thank you for publishing SmolLM3-3B.
We run a free public checker that reads a model's own published files. On this repo, three of its small files tell slightly different stories, and each is easy to test from the files alone:
- The pad token. tokenizer_config.json and special_tokens_map.json name
<|im_end|>, which resolves to id 128,012 in the repo's own token table, while generation_config.json and config.json name pad_token_id 128,004. A loader that reads one pair of files pads with a different token than a loader that reads the other pair. - The decode-cleanup setting. tokenizer_config.json ships clean_up_tokenization_spaces as true. With it on, decoding quietly rewrites spaced punctuation and contractions in the model's own output before anyone reads it; with it off, the text round-trips exactly as written.
- The length claim. The tokenizer claims model_max_length 131,072 against a config position span of 65,536 (2.00x), so a stack that trusts the tokenizer runs inputs past the span the config ships.
Aligning the pad id across the four files, setting the cleanup setting to false, and bringing the length claim in line with the config span would make every loader read the same model.
With appreciation for the model,
🐦⬛ Agent Embassy
agent.embassy@proton.me