Instructions to use sakamakismile/Qwen-AgentWorld-35B-A3B-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use sakamakismile/Qwen-AgentWorld-35B-A3B-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="sakamakismile/Qwen-AgentWorld-35B-A3B-NVFP4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("sakamakismile/Qwen-AgentWorld-35B-A3B-NVFP4")
model = AutoModelForMultimodalLM.from_pretrained("sakamakismile/Qwen-AgentWorld-35B-A3B-NVFP4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use sakamakismile/Qwen-AgentWorld-35B-A3B-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "sakamakismile/Qwen-AgentWorld-35B-A3B-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sakamakismile/Qwen-AgentWorld-35B-A3B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/sakamakismile/Qwen-AgentWorld-35B-A3B-NVFP4

SGLang

How to use sakamakismile/Qwen-AgentWorld-35B-A3B-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "sakamakismile/Qwen-AgentWorld-35B-A3B-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sakamakismile/Qwen-AgentWorld-35B-A3B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "sakamakismile/Qwen-AgentWorld-35B-A3B-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sakamakismile/Qwen-AgentWorld-35B-A3B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use sakamakismile/Qwen-AgentWorld-35B-A3B-NVFP4 with Docker Model Runner:
```
docker model run hf.co/sakamakismile/Qwen-AgentWorld-35B-A3B-NVFP4
```

Qwen-AgentWorld-35B-A3B-NVFP4

NVFP4 (W4A4) quantization of Qwen/Qwen-AgentWorld-35B-A3B — Qwen's language world model that simulates agentic environments. Packed to compressed-tensors NVFP4 so the 35B-A3B MoE runs on entry-level Blackwell (SM120) cards at 21.9 GB, served by vLLM with no --quantization flag (auto-detected).

What it is

Qwen-AgentWorld is a world model / environment simulator, not a task-solving agent: given a state and an action, it predicts the next observation. It is trained (CPT → SFT → RL) to simulate seven agentic domains — MCP, Search, Terminal, SWE, Android, Web, OS — so downstream agents can be trained and evaluated against simulated rollouts instead of (costly, risky) real environments. See the paper and the upstream repository.

Quantization


Method	NVFP4 (`nvfp4-pack-quantized`, W4A4, group size 16, FP8-E4M3 scales), `compressed-tensors`
Tool	`llm-compressor` one-shot, 32 calibration samples (`neuralmagic/calibration`, seq 8192)
Quantized	all language-model `Linear` layers, including the 30,720 expert projections
Kept BF16	`lm_head`, the Qwen3-VL vision tower (`visual.*`), the MoE routers (`mlp.gate`, `mlp.shared_expert_gate`), norms
Size	21.9 GB (from ~69 GB BF16)
Architecture	`Qwen3_5MoeForConditionalGeneration` (`qwen3_5_moe`): 40 layers, 256 experts top-8 + shared expert, hybrid linear+full attention, 262k context

Recipe:

from llmcompressor.modifiers.quantization import QuantizationModifier
QuantizationModifier(
    targets="Linear", scheme="NVFP4",
    ignore=["lm_head", "re:.*visual.*", "re:.*mlp.gate$", "re:.*mlp.shared_expert_gate$"],
)

The full multimodal model is loaded so the vision tower is preserved in BF16; the per-expert projections are packed to NVFP4.

Serving (vLLM ≥ 0.22)

Requires a Blackwell (SM120) GPU. TP=4 (4× 16 GB) is the comfortable floor for the MoE NVFP4 GEMM workspace. (TP=2 fits only memory-squeezed: add --enforce-eager --max-model-len 8192 --gpu-memory-utilization 0.95, at a large single-stream speed cost.)

NCCL_P2P_DISABLE=1 vllm serve sakamakismile/Qwen-AgentWorld-35B-A3B-NVFP4 \
  --tensor-parallel-size 4 \
  --disable-custom-all-reduce \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.87 \
  --reasoning-parser qwen3

NVFP4 is auto-detected — do not pass --quantization.
It is a reasoning model: it thinks in <think>…</think> by default before emitting the predicted observation. Recommended sampling: temperature=0.6, top_p=0.95, top_k=20.

Usage — the environment-simulator format

messages = [
  {"role": "system",
   "content": "You are a language world model simulating a Linux terminal environment. "
              "Given the user's command, predict the terminal output."},
  {"role": "user", "content": "Action: execute_bash\nCommand: ls -la /home/user/project/"},
]

The model returns the predicted next observation. Per-domain system-prompt templates for all seven domains live in the upstream repository.

Performance & fidelity

On a single node of 4× RTX PRO 2000 Blackwell (16 GB, SM120), TP=4: ~161 tok/s single-stream. The NVFP4 packing preserves the model's world-modeling behaviour — e.g. it predicts well-formed terminal output (total, permission bits, ./..) and realistic, locale-appropriate search-result listings, and it stays consistent with the supplied environment state.

License & attribution

Apache-2.0, inherited from the base model. Quantized by Lna-Lab. These weights are a faithful NVFP4 packing of Qwen/Qwen-AgentWorld-35B-A3B; all credit for the model itself goes to the Qwen team.