Instructions to use modrill/code-think-q4b-20260908 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use modrill/code-think-q4b-20260908 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="modrill/code-think-q4b-20260908")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("modrill/code-think-q4b-20260908")
model = AutoModelForCausalLM.from_pretrained("modrill/code-think-q4b-20260908", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use modrill/code-think-q4b-20260908 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "modrill/code-think-q4b-20260908"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "modrill/code-think-q4b-20260908",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/modrill/code-think-q4b-20260908

SGLang

How to use modrill/code-think-q4b-20260908 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "modrill/code-think-q4b-20260908" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "modrill/code-think-q4b-20260908",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "modrill/code-think-q4b-20260908" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "modrill/code-think-q4b-20260908",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use modrill/code-think-q4b-20260908 with Docker Model Runner:
```
docker model run hf.co/modrill/code-think-q4b-20260908
```

modrill commited on Sep 9

Commit

1142555

verified ·

1 Parent(s): e731f1f

Add provenance/POLICY.json

Browse files

Files changed (1) hide show

provenance/POLICY.json +169 -0

provenance/POLICY.json ADDED Viewed

	@@ -0,0 +1,169 @@

+{
+  "arms": [
+    {
+      "arm_id": "Q4B-THINK-MIX",
+      "data": {
+        "assistant_target_tokens": 64848450,
+        "manifest_file_sha256": "fc22e609554e898f2ae570f75bce208ae3428aada9931dc43bc7da3735f95885",
+        "manifest_path": "/workspace/code-sft-runs/t30b2507-q4b-tn-v2-lr1e4-e1-20260907/arms/Q4B-THINK-MIX/data/mix_l0.2_s0.8_tail151643_2EP_READY.json",
+        "manifest_sha256": "fc22e609554e898f2ae570f75bce208ae3428aada9931dc43bc7da3735f95885",
+        "max_assistant_target_tokens_per_example": 16384,
+        "payload_rows": 8310,
+        "payload_sha256": "6097d7bf8bc18621ed4fbc2c7f4e6a68b65186eb1d25964fd632580f97dabdfd",
+        "source_revision": "3a7a0a0636ec96e3c1ec42ebe79ade467caa040d"
+      },
+      "gpu": 2,
+      "model": {
+        "config_sha256": "304b2545a258d35620f1d4bf46940c0471d9baa00715ff8e77f84c2fca5057c1",
+        "hf_id": "Qwen/Qwen3-4B-Base",
+        "local_path": "/workspace/code-sft-infra/models/qwen3-4b-base",
+        "revision": "906bfd4b4dc7f14ee4320094d8b41684abff8539",
+        "tie_word_embeddings": true,
+        "tokenizer_sha256": "c0382117ea329cdf097041132f6d735924b697924d6f6fc3945713e96ce87539"
+      },
+      "model_id": "qwen3-4b-base",
+      "renderer": {
+        "assistant_termination": {
+          "final_text_suffix": "<|endoftext|>",
+          "final_token_suffix": [
+            151643
+          ],
+          "turn_end_token_id": 151645
+        },
+        "chat_template_kwargs": {
+          "enable_thinking": true
+        },
+        "chat_template_sha256": "b608d01889daca5f325cc44d1cbabf73ec74d4d53c9950725cc46068a7006a68",
+        "encoding_policy": {
+          "add_special_tokens": false,
+          "assistant_termination_in_loss": true,
+          "assistant_termination_in_target_census": true,
+          "incomplete_target": "reject",
+          "padding_label_mask": "attention_or_pad_positions_never_token_id"
+        },
+        "eos_token": "<|endoftext|>",
+        "eos_token_id": 151643,
+        "mode": "think",
+        "runtime_tokenizer_manifest_sha256": "66615d472270d50fca0a4b4b299ab2663a9944102f0afd4626fcee46214fe248",
+        "runtime_tokenizer_path": "/workspace/code-sft-runs/t30b2507-q4b-tn-v2-lr1e4-e1-20260907/tokenizers/qwen3-v4-tail151643",
+        "stop_token_ids": [
+          151643,
+          151645
+        ],
+        "system_prompt": "You are an expert Python programmer. You will be given a question (problem specification) and will generate a correct Python program that matches the specification and passes all tests. You will NOT return anything except for the program.",
+        "system_prompt_policy": "exact_evalscope_live_code_bench_system_message",
+        "system_prompt_policy_sha256": "209896b91040ef82e511aa0c7c95a2f9f5dadc9d3c4879f7c66da4ddca34295b",
+        "system_prompt_sha256": "41b9af0ebedb936e74e0996175f76f0116c334375ec3a7c914166f126c02f489",
+        "tokenizer_hf_id": "Qwen/Qwen3-4B-Base",
+        "tokenizer_revision": "906bfd4b4dc7f14ee4320094d8b41684abff8539",
+        "tokenizer_sha256": "c0382117ea329cdf097041132f6d735924b697924d6f6fc3945713e96ce87539"
+      },
+      "renderer_authority": {
+        "path": "/workspace/code-sft-runs/t30b2507-q4b-tn-v2-lr1e4-e1-20260907/tokenizers/qwen3-v4-tail151643.authority.json",
+        "sha256": "6b1461404d033b79f05a8cbb373f05a495d506a567b35b18cbf89930f345422a"
+      },
+      "renderer_id": "qwen3-8b-think-lcb-system-v2",
+      "source_id": "rstar-coder"
+    }
+  ],
+  "generation_behavior_gate": null,
+  "owner_authorization": "PI 2026-09-09 Q4B-THINK-MIX Mix Distillation 0.2:0.8",
+  "recipe": {
+    "adapter": {
+      "alpha": 128,
+      "dropout": 0.0,
+      "method": "lora",
+      "modules_to_save": null,
+      "rank": 64,
+      "target_modules": [
+        "q_proj",
+        "k_proj",
+        "v_proj",
+        "o_proj",
+        "gate_proj",
+        "up_proj",
+        "down_proj"
+      ]
+    },
+    "boundary": {
+      "fp32_master": true,
+      "mode": "trainable_token_rows_both_sides",
+      "row_lr": 0.0001,
+      "row_weight_decay": 0.0,
+      "token_ids": [
+        151643,
+        151667,
+        151668
+      ],
+      "untie_tied_embeddings": true
+    },
+    "checkpoint_target_tokens": [
+      32445505,
+      48759220,
+      64848450
+    ],
+    "epochs": 2,
+    "gradient_checkpointing": true,
+    "max_sequence_length": 32768,
+    "model_runtime": {
+      "bf16": true,
+      "finetuning_type": "lora",
+      "gradient_checkpointing": true,
+      "lora_alpha": 128,
+      "lora_dropout": 0.0,
+      "lora_rank": 64,
+      "lora_targets": [
+        "q_proj",
+        "k_proj",
+        "v_proj",
+        "o_proj",
+        "gate_proj",
+        "up_proj",
+        "down_proj"
+      ],
+      "modules_to_save": null,
+      "trainable_dtype": "float32",
+      "trainable_token_indices": {
+        "embed_tokens": [
+          151643,
+          151667,
+          151668
+        ],
+        "lm_head": [
+          151643,
+          151667,
+          151668
+        ]
+      }
+    },
+    "optimizer": {
+      "betas": [
+        0.9,
+        0.95
+      ],
+      "eps": 1e-08,
+      "gradient_clip_norm": 1.0,
+      "learning_rate": 0.0001,
+      "name": "adamw",
+      "weight_decay": 0.1
+    },
+    "packing": {
+      "enabled": false
+    },
+    "precision": "bfloat16",
+    "repeat_examples": false,
+    "scheduler": "cosine_by_assistant_target_token_dose",
+    "seed": 42,
+    "target_assistant_tokens": 64848450,
+    "tokens_per_optimizer_update": 65536,
+    "trainable_dtype": "float32",
+    "truncation": false,
+    "warmup_target_tokens": 3890907
+  },
+  "runtime": {
+    "checkpoint_root": "/workspace/code-sft-runs/t30b2507-q4b-tn-v2-lr1e4-e1-20260907/arms/Q4B-THINK-MIX/out",
+    "state_root": "/workspace/code-sft-runs/t30b2507-q4b-tn-v2-lr1e4-e1-20260907/arms/Q4B-THINK-MIX"
+  },
+  "schema": "CODE_SFT_RSTAR_DUAL_TRAINING_POLICY_V1",
+  "status": "SEALED"
+}