PhyRubric-Judge-32B

LoRA adapter (r = 16, all linear layers of the language model) on Qwen/Qwen3-VL-32B-Instruct, fine-tuned on PhyRubric-Train (3,788 clips from four video generators, labelled with the PhyRubric rubric protocol).

Given the caption used to generate a video and 16 uniformly sampled frames of the clip, the judge returns

{"sa_score": 1-5, "pc_assessable": true|false, "pc_score": 1-5, "reason": "..."}

sa_score is semantic adherence (does the clip show what the caption describes), pc_score is physical commonsense (does everyday physics hold), pc_assessable says whether the central physical event of the caption appears at all, and reason is a timestamped explanation. The judge never sees a rubric.

This is the 32B variant of PhyRubric-Judge: same data, prompt and recipe, with the base model swapped for Qwen3-VL-32B-Instruct. It is not part of the paper's main results.

Training: 2 epochs (900 steps), lr 1e-4, LoRA rank 16 / alpha 32, bf16, 16 frames at most 448x448, ms-swift 4.5; 4.6 h and 70 GiB on one NVIDIA GH200. Inference in bf16 needs about 70 GB of GPU memory.

Agreement with human labels on PhyRubric-Bench (2,433 videos, soft score)

Judge SA Pearson SA AUC PC Pearson PC AUC
PhyRubric-Judge (8B) 0.62 0.85 0.31 0.68
PhyRubric-Judge-32B 0.66 0.87 0.35 0.69

Paired bootstrap over the same videos: SA +0.034 [+0.016, +0.052], PC +0.043 [+0.011, +0.074].

Usage

Code, data and the evaluation scripts are at https://github.com/chenyil6/phyRubric.

hf download kittyhy/PhyRubric-Judge-32B --local-dir models/PhyRubric-Judge-32B
CKPT=models/PhyRubric-Judge-32B RUN=judge bash scripts/infer.sh      # scores PhyRubric-Bench and prints the metrics

scripts/infer.sh runs judge/infer.py, which loads the base model with this adapter through ms-swift and applies the same frame sampling as training (FPS=24 FPS_MAX_FRAMES=16 VIDEO_MAX_PIXELS=448*448). The system prompt the judge was trained with is data/system_prompt.txt in the repository.

Downloads last month
13
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for kittyhy/PhyRubric-Judge-32B

Adapter
(19)
this model