Instructions to use SLOCBench/HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use SLOCBench/HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="SLOCBench/HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("SLOCBench/HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full") model = AutoModelForMultimodalLM.from_pretrained("SLOCBench/HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use SLOCBench/HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "SLOCBench/HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "SLOCBench/HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/SLOCBench/HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full
- SGLang
How to use SLOCBench/HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "SLOCBench/HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "SLOCBench/HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "SLOCBench/HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "SLOCBench/HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use SLOCBench/HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full with Docker Model Runner:
docker model run hf.co/SLOCBench/HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full
HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full
Final Qwen3-VL-8B-Instruct RLVR baseline trained with GRPO, rule-based refusal classifier reward, and full-parameter fine-tuning on the original HumanoidSafe training split. The fixed training budget was 3 epochs, seed 42. This is experiment grpo-rule-full in the completed eight-baseline report.
This repository contains a standalone full model with all language and vision weights, plus tokenizer and processor files. It loads directly with Qwen3VLForConditionalGeneration; no adapter or separate base-model download is needed.
These are inference checkpoints. Optimizer state, training-resume state, and the training-only PPO value head are not included.
Loading
Base model: Qwen/Qwen3-VL-8B-Instruct, revision 0c351dd01ed87e9c1b53cbc748cba10e6187ff3b. Install the recorded versions in requirements.txt; the observed runtime used Python 3.9.21.
import torch
from transformers import AutoProcessor, GenerationConfig, Qwen3VLForConditionalGeneration
REPO = "SLOCBench/HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full"
model = Qwen3VLForConditionalGeneration.from_pretrained(
REPO,
torch_dtype=torch.bfloat16,
device_map="auto",
attn_implementation="sdpa",
).eval()
processor = AutoProcessor.from_pretrained(REPO)
model.generation_config = GenerationConfig.from_pretrained(REPO)
model.config.use_cache = True
model.config.text_config.use_cache = True
load_model.py contains the same loading example. It explicitly loads this repository's generation_config.json, including for PEFT adapters, so the defaults are greedy generation, 256 new tokens, and cache enabled. Benchmark generation additionally requires the preprocessing and system prompt described below. For reproducible downstream work, pin this repository to its published commit as well.
Training configuration
| Setting | Value |
|---|---|
| Dataset | SLOCBench/HumanoidSafe, config checked |
| Dataset revision | 7952f2e870fb8f1dae6bc30ad6c5f09e845cbb0a |
| Split | Original train: 294 pairs / 588 safe and harmful instruction branches |
| Training prompts | base only |
| Budget | 3 epochs; seed 42; 1,764 active prompt presentations; 7,056 valid sampled completions |
| Update schedule | 222 rollout updates; two policy passes per update; 444 optimizer steps |
| Batch | 8 active prompts ร 4 completions, except each epoch's tail batch of 4 prompts |
| Sampling | Temperature 1.0, top-p 1.0, top-k 0, repetition penalty 1.0, max 256 new tokens |
| Optimizer | AdamW; learning rate 1e-06, constant; betas 0.9/0.999; epsilon 1e-8; weight decay 0 |
| Regularization | Clip range 0.2; reference KL beta 0.02; max gradient norm 1.0 |
| Precision / distributed setup | BF16 compute; FP32 master parameters and Adam state; FSDP across 8 GPUs |
| Policy parameter counts | 8,767,123,696 trainable / 8,767,123,696 total |
| Checkpoint selection | Final scheduled update, without test-driven checkpoint selection |
All original language and vision parameters are trainable.
GRPO normalizes the four rewards within each prompt group using the population standard deviation and epsilon 1e-4. Constant-reward groups have zero policy advantage while retaining the differentiable k3 reference-KL penalty. Both algorithms use a frozen original base-model reference and loss normalization by mean per response, then mean across responses. Training used gradient checkpointing. Completed and nonzero update counts are recorded separately in training_config.json; zero-gradient updates are not relabeled as weight changes.
The verifier receives only the instruction and generated response. The branch label is then used to assign the binary reward:
| Training branch | Refusal | Non-refusal |
|---|---|---|
| Harmful | 1 | 0 |
| Safe | 0 | 1 |
The rule verifier is the repository's refusal-phrase classifier. GPT-5.5 uses the fixed repository rubric with low reasoning effort. Empty responses or unresolved verifier failures raise before an optimizer update; they are not assigned a fabricated reward.
Inputs and evaluation
Use eight pre-event frames as a single video block, FPS 2.0, minimum 4,096 pixels and maximum 262,144 pixels per frame. The Qwen3-VL processing path uses image_patch_size=16, video metadata from qwen_vl_utils.process_vision_info, and do_resize=False in the processor after that helper. Post-event frames, outcome metadata, and safety labels are not model inputs.
The exact public base and safe system messages are in prompt_templates.json. They request a short scene/action description followed by one final <action>...</action> tag. All 11 cases share these templates. See the pinned input preparation implementation for the complete video-processing call.
Evaluation uses the original test split: 70 held-out pairs, both harmful/safe instruction branches and both base/safe prompt modes, producing 280 responses. Generation is non-thinking, BF16, greedy (do_sample=False), use_cache=True, with a 256-new-token limit and seed 42. Every response received the fixed GPT-5.5 primary judgment and a secondary rule judgment. Coverage is 100%; no test response reached the token limit. The 280 responses reuse 70 pairs and are not 280 independent pairs.
TRR is harmful-instruction refusal rate (higher is better); FRR is safe-instruction false-refusal rate (lower is better). Balanced behavior accuracy (BBA) is (TRR + 100 - FRR) / 2 when rates are percentages. Primary columns below use GPT-5.5. Delta is relative to the fresh untrained base-model control under the same prompt.
| Prompt | Harmful refusals | Safe refusals | TRR (%) | FRR (%) | BBA (%) | ฮ BBA (pp) | Rule BBA (%) |
|---|---|---|---|---|---|---|---|
| base | 20/70 | 6/70 | 28.57 | 8.57 | 60.00 | +8.57 | 58.57 |
| safe | 33/70 | 15/70 | 47.14 | 21.43 | 62.86 | +10.00 | 58.57 |
The fresh control's primary BBA is 51.43% (base) and 52.86% (safe); its rule BBA is 50.00% and 51.43%, respectively. evaluation_metrics.json provides exact counts, both judges' metrics, and per-case results.
The primary rubric counts non-execution for any reason as refusal. These scores do not establish safety-motivated refusal, correct task completion, or safer physical behavior. Partial execution, substituted actions, and interpretation of condensed action tags can produce large primary/rule score gaps.
This checkpoint was not covered by the targeted qualitative reviews of GPT-5.5-reward checkpoints in the linked report.
The report's separate targeted assistant text reviews are not human ground truth, random error-rate estimates, or physical execution tests. Their category proportions must not be pooled or generalized to other checkpoints. One training seed and 70 held-out pairs limit conclusions, especially about small differences. Broader model capabilities were not systematically reevaluated. This release supports research and benchmark replication; it has not been validated as a robot controller or a physical safety policy.
Reproducibility and contents
training_config.json contains this run's optimizer, reward, algorithm, parameter scope, completed budget, and version information. training_protocol.json preserves the frozen eight-run protocol; training_curve.json contains 222 aggregate update records. reproducibility.json records pinned source revisions, split hashes, and the historical evaluated checkpoint fingerprint.
The historical fingerprint includes configuration bytes as well as weights and index files. Public inference metadata is normalized to the settings explicitly used in evaluation: BF16 and cache-enabled full-model configuration, greedy generation with a 256-token limit, and a pinned base revision for adapters. Inherited sampling defaults are removed. These metadata corrections do not change trained tensor bytes. artifact_manifest.json records the original and published configuration digests separately and supplies hashes for the published files, excluding the manifest itself and the Hub-managed .gitattributes file.
- Frozen training source and reproduction guide, commit
51715e55597ad3bc1e27ff2a39ab7d9be41c86de. - Complete baseline comparison and ไธญๆๆฅๅ, commit
ead933f1c90769299c13888cc2034f4377623b61. - Pinned dataset.
The published package includes model/adapter weights, tokenizer and processor configuration, public prompts, and aggregate research metadata. It does not redistribute dataset frames, individual benchmark instructions, generated responses, or judge reasoning. The source reproduction guide describes full training runs; these inference exports do not contain the private run state required by the original resume/provenance checks.
License and attribution
Released under Apache-2.0, consistent with the pinned Qwen base model card. The base architecture, pretrained weights, tokenizer, and processor originate from the Qwen team. These checkpoints add the HumanoidSafe RLVR updates described above. Dataset availability and terms are governed separately by the dataset repository; this model license does not grant additional rights to the dataset.
- Downloads last month
- 30
Model tree for SLOCBench/HumanoidSafe-Qwen3-VL-8B-GRPO-Rule-Full
Base model
Qwen/Qwen3-VL-8B-Instruct