Instructions to use maritaca-ai/S5-nano with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use maritaca-ai/S5-nano with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="maritaca-ai/S5-nano") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("maritaca-ai/S5-nano") model = AutoModelForMultimodalLM.from_pretrained("maritaca-ai/S5-nano", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use maritaca-ai/S5-nano with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "maritaca-ai/S5-nano" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "maritaca-ai/S5-nano", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/maritaca-ai/S5-nano
- SGLang
How to use maritaca-ai/S5-nano with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "maritaca-ai/S5-nano" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "maritaca-ai/S5-nano", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "maritaca-ai/S5-nano" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "maritaca-ai/S5-nano", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use maritaca-ai/S5-nano with Docker Model Runner:
docker model run hf.co/maritaca-ai/S5-nano
Unify S5-nano model cards with an introductory format paragraph
Browse files
README.md
CHANGED
|
@@ -8,6 +8,8 @@ library_name: transformers
|
|
| 8 |
|
| 9 |
# S5-nano
|
| 10 |
|
|
|
|
|
|
|
| 11 |
S5-nano is a 4B-parameter model from Maritaca AI, derived from
|
| 12 |
[Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B). It has two modes:
|
| 13 |
|
|
@@ -28,6 +30,11 @@ The context window supports up to 262,144 tokens, and the weights include an MTP
|
|
| 28 |
FP8 is recommended for GPUs with native FP8 support (Ada, Hopper, and Blackwell).
|
| 29 |
The GGUF repository provides BF16, Q8_0, and Q4_K_M variants.
|
| 30 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
<!-- s5-nano-evaluation-results:start -->
|
| 32 |
## Evaluation results
|
| 33 |
|
|
@@ -962,6 +969,9 @@ Optional flags:
|
|
| 962 |
For the FP8 version, replace the model name with `maritaca-ai/S5-nano-FP8`
|
| 963 |
and keep the same flags.
|
| 964 |
|
|
|
|
|
|
|
|
|
|
| 965 |
### Requests with curl
|
| 966 |
|
| 967 |
Think mode:
|
|
|
|
| 8 |
|
| 9 |
# S5-nano
|
| 10 |
|
| 11 |
+
This repository contains the **BF16 version of S5-nano** (8.7 GB), with the original, unquantized weights for vLLM and SGLang. The [FP8 version](https://huggingface.co/maritaca-ai/S5-nano-FP8) reduces the weight size to 5.7 GB and is recommended for GPUs with native FP8 support. The [GGUF versions](https://huggingface.co/maritaca-ai/S5-nano-GGUF) are for llama.cpp and compatible applications, with BF16 (8.7 GB), Q8_0 (4.6 GB), and Q4_K_M (2.8 GB) options.
|
| 12 |
+
|
| 13 |
S5-nano is a 4B-parameter model from Maritaca AI, derived from
|
| 14 |
[Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B). It has two modes:
|
| 15 |
|
|
|
|
| 30 |
FP8 is recommended for GPUs with native FP8 support (Ada, Hopper, and Blackwell).
|
| 31 |
The GGUF repository provides BF16, Q8_0, and Q4_K_M variants.
|
| 32 |
|
| 33 |
+
The FP8 version uses E4M3 weights for linear layers, with one scale per 128×128
|
| 34 |
+
block, and dynamically quantizes activations during inference. Embeddings, the
|
| 35 |
+
vision encoder, and some small projections remain in BF16. The MTP layer is
|
| 36 |
+
included.
|
| 37 |
+
|
| 38 |
<!-- s5-nano-evaluation-results:start -->
|
| 39 |
## Evaluation results
|
| 40 |
|
|
|
|
| 969 |
For the FP8 version, replace the model name with `maritaca-ai/S5-nano-FP8`
|
| 970 |
and keep the same flags.
|
| 971 |
|
| 972 |
+
When serving FP8, also use `maritaca-ai/S5-nano-FP8` in the request's
|
| 973 |
+
`model` field (or `MODEL` in the Python examples).
|
| 974 |
+
|
| 975 |
### Requests with curl
|
| 976 |
|
| 977 |
Think mode:
|