Instructions to use Sparset/Thinklite-27B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Sparset/Thinklite-27B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Sparset/Thinklite-27B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Sparset/Thinklite-27B")
model = AutoModelForMultimodalLM.from_pretrained("Sparset/Thinklite-27B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Sparset/Thinklite-27B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Sparset/Thinklite-27B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sparset/Thinklite-27B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Sparset/Thinklite-27B

SGLang

How to use Sparset/Thinklite-27B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Sparset/Thinklite-27B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sparset/Thinklite-27B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Sparset/Thinklite-27B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sparset/Thinklite-27B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Sparset/Thinklite-27B with Docker Model Runner:
```
docker model run hf.co/Sparset/Thinklite-27B
```

Mintzs commited on 5 days ago

Commit

97ef337

verified ·

1 Parent(s): 844739b

Model card: banner and Sparset table colours

Browse files

Files changed (3) hide show

.gitattributes +1 -0
README.md +12 -12
thinklite-banner.png +3 -0

.gitattributes CHANGED Viewed

@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 tokenizer.json filter=lfs diff=lfs merge=lfs -text

 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 tokenizer.json filter=lfs diff=lfs merge=lfs -text
+thinklite-banner.png filter=lfs diff=lfs merge=lfs -text

README.md CHANGED Viewed

@@ -10,26 +10,26 @@ tags:
 - qwen3.8
 ---
-# Thinklite-27B
 **Qwen3.8-27B that thinks with half the tokens.** At `reasoning_effort="xhigh"`, Thinklite-27B uses 47% less
 reasoning than Qwen3.8-27B on average over six benchmarks and writes half as many output tokens, so its
 time-to-final-output (TTFO) is about 2x shorter. At `reasoning_effort="low"` it uses 28% less reasoning.
 <style>
-.tl-table { display:table !important; width:100% !important; table-layout:fixed !important; border-collapse:separate; border-spacing:0; overflow:hidden; border:1px solid #232835; border-radius:16px; background:#0B0D12; font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,sans-serif; font-size:13px; color:#C9CED8; }
-.tl-table th { padding:10px 6px; text-align:center; font-weight:600; color:#8A92A3; background:#181C26; border-right:1px solid #232835; border-bottom:1px solid #232835; }
-.tl-table td { padding:10px 6px; text-align:center; color:#C9CED8; background:#0B0D12; border-right:1px solid #232835; border-bottom:1px solid #232835; vertical-align:middle; font-variant-numeric:tabular-nums; }
 .tl-table tr > :last-child { border-right:0; }
 .tl-table tbody tr:last-child td { border-bottom:0; }
-.tl-table .group { color:#FFFFFF; font-weight:700; border-bottom:2px solid #3B4E8C; }
-.tl-table .name { padding-left:16px; text-align:left; color:#FFFFFF; font-weight:600; }
-.tl-table .tl { background:#151B2E; }
-.tl-table th.tl { color:#7C9CFF; }
-.tl-table td.tl strong, .tl-table td.tl { color:#FFFFFF; }
-.tl-table strong { color:#FFFFFF; }
-.tl-table .up { color:#7C9CFF; font-weight:700; }
-.tl-table .avg td { border-top:2px solid #3B4E8C; font-weight:700; }
 @media (max-width: 640px) {
   .tl-table { display:block !important; overflow-x:auto !important; table-layout:auto !important; }
   .tl-table th, .tl-table td { min-width:92px; }

 - qwen3.8
 ---
+<img src="thinklite-banner.png" alt="Thinklite-27B: Qwen 3.8 27B made efficient" style="width:100%;max-width:100%;height:auto;display:block;margin-bottom:1em;" />
 **Qwen3.8-27B that thinks with half the tokens.** At `reasoning_effort="xhigh"`, Thinklite-27B uses 47% less
 reasoning than Qwen3.8-27B on average over six benchmarks and writes half as many output tokens, so its
 time-to-final-output (TTFO) is about 2x shorter. At `reasoning_effort="low"` it uses 28% less reasoning.
 <style>
+.tl-table { display:table !important; width:100% !important; table-layout:fixed !important; border-collapse:separate; border-spacing:0; overflow:hidden; border:1px solid rgba(246,246,243,0.16); border-radius:0; background:#090909; font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,sans-serif; font-size:13px; color:rgba(246,246,243,0.80); }
+.tl-table th { padding:10px 6px; text-align:center; font-weight:600; color:rgba(246,246,243,0.55); background:#101010; border-right:1px solid rgba(246,246,243,0.16); border-bottom:1px solid rgba(246,246,243,0.16); }
+.tl-table td { padding:10px 6px; text-align:center; color:rgba(246,246,243,0.80); background:#090909; border-right:1px solid rgba(246,246,243,0.16); border-bottom:1px solid rgba(246,246,243,0.16); vertical-align:middle; font-variant-numeric:tabular-nums; }
 .tl-table tr > :last-child { border-right:0; }
 .tl-table tbody tr:last-child td { border-bottom:0; }
+.tl-table .group { color:#F6F6F3; font-weight:700; border-bottom:2px solid #3E61EB; }
+.tl-table .name { padding-left:16px; text-align:left; color:#F6F6F3; font-weight:600; }
+.tl-table .tl { background:rgba(62,97,235,0.16); }
+.tl-table th.tl { color:#8DA2F5; }
+.tl-table td.tl strong, .tl-table td.tl { color:#F6F6F3; }
+.tl-table strong { color:#F6F6F3; }
+.tl-table .up { color:#8DA2F5; font-weight:700; }
+.tl-table .avg td { border-top:2px solid #3E61EB; font-weight:700; }
 @media (max-width: 640px) {
   .tl-table { display:block !important; overflow-x:auto !important; table-layout:auto !important; }
   .tl-table th, .tl-table td { min-width:92px; }

thinklite-banner.png ADDED Viewed

Git LFS Details

SHA256: f1b99765a58bd2d9d58c6f73da5f48e394a02279874596da7868074cf3127050
Pointer size: 131 Bytes
Size of remote file: 167 kB