Instructions to use fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090")
model = AutoModelForMultimodalLM.from_pretrained("fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090

SGLang

How to use fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090 with Docker Model Runner:
```
docker model run hf.co/fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090
```

Qwen3.8-27B Uncensored NVFP4 RTX 5090

RTX 5090 optimized ModelOpt NVFP4 conversion of orcarouter/Qwen3.8-27B-Uncensored, produced directly from the original BF16 safetensors.

This checkpoint preserves the Qwen3.8 vision-language architecture and native MTP tensors. It is paired with the target-matched fleetml/Qwen3.8-27B-Uncensored-DSpark-RTX5090, now released in ModelOpt NVFP4.

Combined release

Component	Disk size	GPU weight allocation
NVFP4 target	20.59 GB	18.80 GB
NVFP4 DSpark drafter	1.64 GB	1.70 GB
Combined pair	22.24 GB	20.50 GB

The validated Hermes serving profile launches with a 122,880-token context limit. SGLang automatically allocated an 87,798-token active pool and left 2.83 GB of GPU memory free after cache allocation and CUDA graph capture on one RTX 5090.

Context field	Tokens
Architecture maximum	262,144
Validated launch context	122,880
Automatically allocated active pool	87,798

Measured performance

The current NVFP4 target and NVFP4 drafter profile reached 217.73 tokens per second median decode across 18 successful requests on one RTX 5090.

Metric	Result
Requests	18 of 18 successful
Median decode	217.73 tokens per second
Median time to first token	0.113 seconds
P95 time to first token	0.149 seconds
Generated tokens per request	26 to 640
Total generated tokens	3,633

Test conditions were concurrency 1, temperature 0, two warmups, three repeats across six prompts, FP8 E4M3 KV cache, and a fixed request seed. Decode throughput excludes prompt processing and time to first token. The 16,000-token request setting was a ceiling. This result is not a sustained 16,000-token generation measurement.

The earlier deterministic BF16 drafter profile measured 139.32 tokens per second with a 16,384-token active pool. Its full standard decoding, native MTP, public DSpark, and trained DSpark comparison remains available in benchmarks/benchmark-summary.json.

Checkpoint

Field	Value
Source revision	`404ea47aaa5d8a8b00049c9e9750089aca011ab2`
Format	ModelOpt NVFP4 safetensors
Export size	20,593,147,464 bytes
Architecture	`Qwen3_5ForConditionalGeneration`
Native MTP tensors	15
Recommended runtime	SGLang Qwen3.8 image
Recommended hardware	NVIDIA RTX 5090 32 GB

Run with the matched NVFP4 DSpark drafter

docker run --rm --gpus all --ipc=host -p 30000:30000 \
  lmsysorg/sglang:qwen38-27b \
  sglang serve \
  --model-path fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090 \
  --quantization modelopt_fp4 \
  --trust-remote-code \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path fleetml/Qwen3.8-27B-Uncensored-DSpark-RTX5090 \
  --speculative-draft-model-quantization modelopt_fp4 \
  --speculative-dspark-block-size 7 \
  --speculative-num-steps 1 \
  --speculative-eagle-topk 1 \
  --context-length 122880 \
  --max-mamba-cache-size 8 \
  --kv-cache-dtype fp8_e4m3 \
  --attention-backend flashinfer \
  --chunked-prefill-size 1024 \
  --mamba-radix-cache-strategy extra_buffer_lazy \
  --mamba-ssm-dtype bfloat16 \
  --mm-feature-transport cpu \
  --cuda-graph-max-bs-decode 1 \
  --mem-fraction-static 0.86 \
  --max-running-requests 1 \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder \
  --host 0.0.0.0 \
  --port 30000

The active token pool is allocated automatically. Available capacity can vary slightly with the runtime build and GPU state.

Provenance

Base model: Qwen/Qwen3.8-27B
BF16 source: orcarouter/Qwen3.8-27B-Uncensored
Quantization: NVIDIA ModelOpt 0.47.0.dev70, commit c4129b6e
Calibration: 126 tokenized UltraChat conversations
Validation: complete shard, tensor, dtype, MTP, load, API, Hermes, and benchmark checks

Use

This checkpoint is intended for controlled local research, evaluation, and agent development. Use it responsibly and comply with the Apache 2.0 license and applicable law.