Instructions to use kittyhy/PhyRubric-Judge-32B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use kittyhy/PhyRubric-Judge-32B with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-VL-32B-Instruct") model = PeftModel.from_pretrained(base_model, "kittyhy/PhyRubric-Judge-32B") - Notebooks
- Google Colab
- Kaggle
PhyRubric-Judge-32B
LoRA adapter (r = 16, all linear layers of the language model) on Qwen/Qwen3-VL-32B-Instruct, fine-tuned on PhyRubric-Train (3,788 clips from four video generators, labelled with the PhyRubric rubric protocol).
Given the caption used to generate a video and 16 uniformly sampled frames of the clip, the judge returns
{"sa_score": 1-5, "pc_assessable": true|false, "pc_score": 1-5, "reason": "..."}
sa_score is semantic adherence (does the clip show what the caption describes), pc_score is physical commonsense
(does everyday physics hold), pc_assessable says whether the central physical event of the caption appears at all,
and reason is a timestamped explanation. The judge never sees a rubric.
This is the 32B variant of PhyRubric-Judge: same data, prompt and recipe, with the base model swapped for Qwen3-VL-32B-Instruct. It is not part of the paper's main results.
Training: 2 epochs (900 steps), lr 1e-4, LoRA rank 16 / alpha 32, bf16, 16 frames at most 448x448, ms-swift 4.5; 4.6 h and 70 GiB on one NVIDIA GH200. Inference in bf16 needs about 70 GB of GPU memory.
Agreement with human labels on PhyRubric-Bench (2,433 videos, soft score)
| Judge | SA Pearson | SA AUC | PC Pearson | PC AUC |
|---|---|---|---|---|
| PhyRubric-Judge (8B) | 0.62 | 0.85 | 0.31 | 0.68 |
| PhyRubric-Judge-32B | 0.66 | 0.87 | 0.35 | 0.69 |
Paired bootstrap over the same videos: SA +0.034 [+0.016, +0.052], PC +0.043 [+0.011, +0.074].
Usage
Code, data and the evaluation scripts are at https://github.com/chenyil6/phyRubric.
hf download kittyhy/PhyRubric-Judge-32B --local-dir models/PhyRubric-Judge-32B
CKPT=models/PhyRubric-Judge-32B RUN=judge bash scripts/infer.sh # scores PhyRubric-Bench and prints the metrics
scripts/infer.sh runs judge/infer.py, which loads the base model with this adapter through ms-swift and applies the
same frame sampling as training (FPS=24 FPS_MAX_FRAMES=16 VIDEO_MAX_PIXELS=448*448). The system prompt the judge was
trained with is data/system_prompt.txt in the repository.
- Downloads last month
- 13
Model tree for kittyhy/PhyRubric-Judge-32B
Base model
Qwen/Qwen3-VL-32B-Instruct