Instructions to use IteraCompute/Qwen3.8-27B-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use IteraCompute/Qwen3.8-27B-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="IteraCompute/Qwen3.8-27B-NVFP4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("IteraCompute/Qwen3.8-27B-NVFP4")
model = AutoModelForMultimodalLM.from_pretrained("IteraCompute/Qwen3.8-27B-NVFP4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use IteraCompute/Qwen3.8-27B-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "IteraCompute/Qwen3.8-27B-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IteraCompute/Qwen3.8-27B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/IteraCompute/Qwen3.8-27B-NVFP4

SGLang

How to use IteraCompute/Qwen3.8-27B-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "IteraCompute/Qwen3.8-27B-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IteraCompute/Qwen3.8-27B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "IteraCompute/Qwen3.8-27B-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IteraCompute/Qwen3.8-27B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use IteraCompute/Qwen3.8-27B-NVFP4 with Docker Model Runner:
```
docker model run hf.co/IteraCompute/Qwen3.8-27B-NVFP4
```

Qwen3.8-27B-NVFP4 / release.json

liwhattao

Publish verified Qwen3.8-27B NVFP4 checkpoint and deployment guide

9f5d0c5 verified 15 days ago

Raw History Blame Contribute Delete

7.02 kB

{
  "repository": "IteraCompute/Qwen3.8-27B-NVFP4",
  "release_date": "2026-10-23",
  "source_repository": "gittensor-model-hub/Qwen3.8-27B-NVFP4-RTX5090",
  "source_revision": "8e2c0cd25468ac5c1f85f621c2b8edb15ea1f03a",
  "original_model": "Qwen/Qwen3.8-27B",
  "license": "apache-2.0",
  "distribution": "Unmodified upstream checkpoint hosted by IteraCompute",
  "verification": "Every listed file SHA-256 matches the deployed checkpoint and the pinned source.",
  "files": [
    {
      "path": ".gitattributes",
      "size": 2146,
      "sha256": "94dbfd2161f89d30b089caf714b7c28521d189aa42b869a43ea366a9ab5d5b3f",
      "source_path": ".gitattributes"
    },
    {
      "path": "LICENSE",
      "size": 11544,
      "sha256": "bbedc3fda3305820b977265f01b8619d87570a6739de3a5582c3464840f1e57a",
      "source_path": "LICENSE"
    },
    {
      "path": "SOURCE_MODEL_CARD.md",
      "size": 25931,
      "sha256": "ab664f2faca2722a9906c71f2037d471b0d4769e9a8cfdf037a3328ee4e7068a",
      "source_path": "README.md"
    },
    {
      "path": "README.qwen-upstream.md",
      "size": 65012,
      "sha256": "57e4bdb258ee1a7d2635c5174ebd4e56abe392505cdb5f8bbb356b0dc4293641",
      "source_path": "README.qwen-upstream.md"
    },
    {
      "path": "assets/rtx5090-accuracy.png",
      "size": 86034,
      "sha256": "70013eee4415a5e4c55c36e83f165d39070772976649a5f247223c1730fa9b8c",
      "source_path": "assets/rtx5090-accuracy.png"
    },
    {
      "path": "assets/rtx5090-context-vs.png",
      "size": 230832,
      "sha256": "c10600f7439d4abdb9473ccdaca932e8c7b9eef68c6c6a1ee3a0ee6fa6edc60f",
      "source_path": "assets/rtx5090-context-vs.png"
    },
    {
      "path": "assets/rtx5090-context.png",
      "size": 81913,
      "sha256": "0a0df80b4c1e4276994bd7688ad49b66efb8c845c4364089edc284222a5cff73",
      "source_path": "assets/rtx5090-context.png"
    },
    {
      "path": "assets/rtx5090-decode.png",
      "size": 88744,
      "sha256": "2ad1c493144c263e5ac33b09f406aa69d453e513b05252d08a04fc57c008ed94",
      "source_path": "assets/rtx5090-decode.png"
    },
    {
      "path": "assets/rtx5090-engines.png",
      "size": 265731,
      "sha256": "6edd0152d89dcf323bf5d470ff2953990daee704d279309402a46ca02bf6731e",
      "source_path": "assets/rtx5090-engines.png"
    },
    {
      "path": "assets/rtx5090-hero-engines.png",
      "size": 309038,
      "sha256": "c7599484f01e57740e90e6bb1c2ec630c32646b70a3624d175d2e09e9a2aa413",
      "source_path": "assets/rtx5090-hero-engines.png"
    },
    {
      "path": "assets/rtx5090-hero-final.png",
      "size": 1534902,
      "sha256": "8cc0baee697956e4747981a3fb5a66f93438c95215f91030517f0dc938ea9211",
      "source_path": "assets/rtx5090-hero-final.png"
    },
    {
      "path": "assets/rtx5090-hero.png",
      "size": 123721,
      "sha256": "65fdb7d51c26626e3fb9effd8fdafce6046be70e944f182ba15d331c018d44a5",
      "source_path": "assets/rtx5090-hero.png"
    },
    {
      "path": "assets/rtx5090-overview.png",
      "size": 152106,
      "sha256": "798041c149f0ef5ad6f8d26a002968332b1c4a862120c31da6dc264062ad64c7",
      "source_path": "assets/rtx5090-overview.png"
    },
    {
      "path": "assets/rtx5090-spec.png",
      "size": 122487,
      "sha256": "96d4caee20382e75eeb296e888c758782726e86cad16fc7aa42ec3ded0771722",
      "source_path": "assets/rtx5090-spec.png"
    },
    {
      "path": "assets/rtx5090-speculation.png",
      "size": 171631,
      "sha256": "ac404b7bc6aded8bb2f5176b58af2360ef7e2cb31537320db5b449b6cbec6e18",
      "source_path": "assets/rtx5090-speculation.png"
    },
    {
      "path": "assets/rtx5090-ttft.png",
      "size": 71910,
      "sha256": "ab25dd6883749efb301635dc2ab4bf36e9fee31a2874d39b8c760661ff826abd",
      "source_path": "assets/rtx5090-ttft.png"
    },
    {
      "path": "chat_template.jinja",
      "size": 14221,
      "sha256": "0a20a4673d45476ed88dcb4a60b6af35ca202ae470fee6fd2bd419758aabd9ab",
      "source_path": "chat_template.jinja"
    },
    {
      "path": "config.json",
      "size": 13167,
      "sha256": "62dad33f5c64fedc1826a39f248d91b3ab4734ebdcd0a9981b548073e996d0b2",
      "source_path": "config.json"
    },
    {
      "path": "crc32.txt",
      "size": 86,
      "sha256": "c7841b92ffe3951e2274665a43af62964811d653d962f39901112405424d8fca",
      "source_path": "crc32.txt"
    },
    {
      "path": "generation_config.json",
      "size": 213,
      "sha256": "7ae9e193dbcef99733ccf647c95ef668c35d1a80a8aa88a51ee40a9bcacf5a74",
      "source_path": "generation_config.json"
    },
    {
      "path": "hf_quant_config.json",
      "size": 9050,
      "sha256": "8e7b36023a5a1184f5e0537bd8a30d15a980afaf0c6b3d7b8eb5dfd10d195938",
      "source_path": "hf_quant_config.json"
    },
    {
      "path": "merges.txt",
      "size": 3353259,
      "sha256": "a9d356d7bdf1ef4949e3e748e95b8e10ad9d4e2e838eddc38a0a7b6b94d1db8d",
      "source_path": "merges.txt"
    },
    {
      "path": "model-00001-of-00002.safetensors",
      "size": 9972777720,
      "sha256": "cdd37b0e61eccc8a3d7d08f9d1a4f52856a9d88e4e8b42089bd18a970e3a01ec",
      "source_path": "model-00001-of-00002.safetensors"
    },
    {
      "path": "model-00002-of-00002.safetensors",
      "size": 7943334864,
      "sha256": "713b84b8287e2193290214766c5384fa6475c5626a628e4021c2c9ca90aa61df",
      "source_path": "model-00002-of-00002.safetensors"
    },
    {
      "path": "model.safetensors.index.json",
      "size": 236508,
      "sha256": "28b9c74b60f4e75914845e119e5b6d0f74574373e8d0f99f8af227583cbbf78c",
      "source_path": "model.safetensors.index.json"
    },
    {
      "path": "preprocessor_config.json",
      "size": 390,
      "sha256": "27225450ac9c6529872ee1924fcb0962ff5634834f817040f444118116f4e516",
      "source_path": "preprocessor_config.json"
    },
    {
      "path": "processor_config.json",
      "size": 1191,
      "sha256": "d89ef49ce9cd37fbf510158e13c1ef063d9286411c1ec9049932dbe0487143b1",
      "source_path": "processor_config.json"
    },
    {
      "path": "tokenizer.json",
      "size": 12809320,
      "sha256": "0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3",
      "source_path": "tokenizer.json"
    },
    {
      "path": "tokenizer_config.json",
      "size": 1150,
      "sha256": "c873857aae349387312ff4cb76d4a17a8d5ed79f89523146200a1570739132db",
      "source_path": "tokenizer_config.json"
    },
    {
      "path": "video_preprocessor_config.json",
      "size": 385,
      "sha256": "7768af27c1fafa9cc9011c1dc20067e03f8915e03b63504550e11d5066986d13",
      "source_path": "video_preprocessor_config.json"
    },
    {
      "path": "vocab.json",
      "size": 6722759,
      "sha256": "ce99b4cb2983d118806ce0a8b777a35b093e2000a503ebde25853284c9dfa003",
      "source_path": "vocab.json"
    }
  ]
}

	{
	"repository": "IteraCompute/Qwen3.8-27B-NVFP4",
	"release_date": "2026-10-23",
	"source_repository": "gittensor-model-hub/Qwen3.8-27B-NVFP4-RTX5090",
	"source_revision": "8e2c0cd25468ac5c1f85f621c2b8edb15ea1f03a",
	"original_model": "Qwen/Qwen3.8-27B",
	"license": "apache-2.0",
	"distribution": "Unmodified upstream checkpoint hosted by IteraCompute",
	"verification": "Every listed file SHA-256 matches the deployed checkpoint and the pinned source.",
	"files": [
	{
	"path": ".gitattributes",
	"size": 2146,
	"sha256": "94dbfd2161f89d30b089caf714b7c28521d189aa42b869a43ea366a9ab5d5b3f",
	"source_path": ".gitattributes"
	},
	{
	"path": "LICENSE",
	"size": 11544,
	"sha256": "bbedc3fda3305820b977265f01b8619d87570a6739de3a5582c3464840f1e57a",
	"source_path": "LICENSE"
	},
	{
	"path": "SOURCE_MODEL_CARD.md",
	"size": 25931,
	"sha256": "ab664f2faca2722a9906c71f2037d471b0d4769e9a8cfdf037a3328ee4e7068a",
	"source_path": "README.md"
	},
	{
	"path": "README.qwen-upstream.md",
	"size": 65012,
	"sha256": "57e4bdb258ee1a7d2635c5174ebd4e56abe392505cdb5f8bbb356b0dc4293641",
	"source_path": "README.qwen-upstream.md"
	},
	{
	"path": "assets/rtx5090-accuracy.png",
	"size": 86034,
	"sha256": "70013eee4415a5e4c55c36e83f165d39070772976649a5f247223c1730fa9b8c",
	"source_path": "assets/rtx5090-accuracy.png"
	},
	{
	"path": "assets/rtx5090-context-vs.png",
	"size": 230832,
	"sha256": "c10600f7439d4abdb9473ccdaca932e8c7b9eef68c6c6a1ee3a0ee6fa6edc60f",
	"source_path": "assets/rtx5090-context-vs.png"
	},
	{
	"path": "assets/rtx5090-context.png",
	"size": 81913,
	"sha256": "0a0df80b4c1e4276994bd7688ad49b66efb8c845c4364089edc284222a5cff73",
	"source_path": "assets/rtx5090-context.png"
	},
	{
	"path": "assets/rtx5090-decode.png",
	"size": 88744,
	"sha256": "2ad1c493144c263e5ac33b09f406aa69d453e513b05252d08a04fc57c008ed94",
	"source_path": "assets/rtx5090-decode.png"
	},
	{
	"path": "assets/rtx5090-engines.png",
	"size": 265731,
	"sha256": "6edd0152d89dcf323bf5d470ff2953990daee704d279309402a46ca02bf6731e",
	"source_path": "assets/rtx5090-engines.png"
	},
	{
	"path": "assets/rtx5090-hero-engines.png",
	"size": 309038,
	"sha256": "c7599484f01e57740e90e6bb1c2ec630c32646b70a3624d175d2e09e9a2aa413",
	"source_path": "assets/rtx5090-hero-engines.png"
	},
	{
	"path": "assets/rtx5090-hero-final.png",
	"size": 1534902,
	"sha256": "8cc0baee697956e4747981a3fb5a66f93438c95215f91030517f0dc938ea9211",
	"source_path": "assets/rtx5090-hero-final.png"
	},
	{
	"path": "assets/rtx5090-hero.png",
	"size": 123721,
	"sha256": "65fdb7d51c26626e3fb9effd8fdafce6046be70e944f182ba15d331c018d44a5",
	"source_path": "assets/rtx5090-hero.png"
	},
	{
	"path": "assets/rtx5090-overview.png",
	"size": 152106,
	"sha256": "798041c149f0ef5ad6f8d26a002968332b1c4a862120c31da6dc264062ad64c7",
	"source_path": "assets/rtx5090-overview.png"
	},
	{
	"path": "assets/rtx5090-spec.png",
	"size": 122487,
	"sha256": "96d4caee20382e75eeb296e888c758782726e86cad16fc7aa42ec3ded0771722",
	"source_path": "assets/rtx5090-spec.png"
	},
	{
	"path": "assets/rtx5090-speculation.png",
	"size": 171631,
	"sha256": "ac404b7bc6aded8bb2f5176b58af2360ef7e2cb31537320db5b449b6cbec6e18",
	"source_path": "assets/rtx5090-speculation.png"
	},
	{
	"path": "assets/rtx5090-ttft.png",
	"size": 71910,
	"sha256": "ab25dd6883749efb301635dc2ab4bf36e9fee31a2874d39b8c760661ff826abd",
	"source_path": "assets/rtx5090-ttft.png"
	},
	{
	"path": "chat_template.jinja",
	"size": 14221,
	"sha256": "0a20a4673d45476ed88dcb4a60b6af35ca202ae470fee6fd2bd419758aabd9ab",
	"source_path": "chat_template.jinja"
	},
	{
	"path": "config.json",
	"size": 13167,
	"sha256": "62dad33f5c64fedc1826a39f248d91b3ab4734ebdcd0a9981b548073e996d0b2",
	"source_path": "config.json"
	},
	{
	"path": "crc32.txt",
	"size": 86,
	"sha256": "c7841b92ffe3951e2274665a43af62964811d653d962f39901112405424d8fca",
	"source_path": "crc32.txt"
	},
	{
	"path": "generation_config.json",
	"size": 213,
	"sha256": "7ae9e193dbcef99733ccf647c95ef668c35d1a80a8aa88a51ee40a9bcacf5a74",
	"source_path": "generation_config.json"
	},
	{
	"path": "hf_quant_config.json",
	"size": 9050,
	"sha256": "8e7b36023a5a1184f5e0537bd8a30d15a980afaf0c6b3d7b8eb5dfd10d195938",
	"source_path": "hf_quant_config.json"
	},
	{
	"path": "merges.txt",
	"size": 3353259,
	"sha256": "a9d356d7bdf1ef4949e3e748e95b8e10ad9d4e2e838eddc38a0a7b6b94d1db8d",
	"source_path": "merges.txt"
	},
	{
	"path": "model-00001-of-00002.safetensors",
	"size": 9972777720,
	"sha256": "cdd37b0e61eccc8a3d7d08f9d1a4f52856a9d88e4e8b42089bd18a970e3a01ec",
	"source_path": "model-00001-of-00002.safetensors"
	},
	{
	"path": "model-00002-of-00002.safetensors",
	"size": 7943334864,
	"sha256": "713b84b8287e2193290214766c5384fa6475c5626a628e4021c2c9ca90aa61df",
	"source_path": "model-00002-of-00002.safetensors"
	},
	{
	"path": "model.safetensors.index.json",
	"size": 236508,
	"sha256": "28b9c74b60f4e75914845e119e5b6d0f74574373e8d0f99f8af227583cbbf78c",
	"source_path": "model.safetensors.index.json"
	},
	{
	"path": "preprocessor_config.json",
	"size": 390,
	"sha256": "27225450ac9c6529872ee1924fcb0962ff5634834f817040f444118116f4e516",
	"source_path": "preprocessor_config.json"
	},
	{
	"path": "processor_config.json",
	"size": 1191,
	"sha256": "d89ef49ce9cd37fbf510158e13c1ef063d9286411c1ec9049932dbe0487143b1",
	"source_path": "processor_config.json"
	},
	{
	"path": "tokenizer.json",
	"size": 12809320,
	"sha256": "0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3",
	"source_path": "tokenizer.json"
	},
	{
	"path": "tokenizer_config.json",
	"size": 1150,
	"sha256": "c873857aae349387312ff4cb76d4a17a8d5ed79f89523146200a1570739132db",
	"source_path": "tokenizer_config.json"
	},
	{
	"path": "video_preprocessor_config.json",
	"size": 385,
	"sha256": "7768af27c1fafa9cc9011c1dc20067e03f8915e03b63504550e11d5066986d13",
	"source_path": "video_preprocessor_config.json"
	},
	{
	"path": "vocab.json",
	"size": 6722759,
	"sha256": "ce99b4cb2983d118806ce0a8b777a35b093e2000a503ebde25853284c9dfa003",
	"source_path": "vocab.json"
	}
	]
	}