Junjun2333/HPSv3-PlusPlus

HPSv3++: Scaling Reward Models Across the Full Spectrum of Diffusion Model Capabilities

HPSv3++ is a capability-aware and RL-iteration-aware text-to-image (T2I) reward model, built on the Qwen/Qwen3-VL-8B-Instruct backbone with a Capability Encoder, a FiLM conditioning head, and a three-layer RankNet reward head.

A Capability Encoder implicitly infers the generative ability of the model that produced an image, while the RL iteration step is supplied as an explicit condition; the two are jointly modulated through FiLM so that a single reward model produces calibrated preference scores across generators of differing capability and different stages of RL optimization.

HPDv3++ is an independently usable human preference training dataset and benchmark with approximately 212K preference pairs covering text fidelity and aesthetic quality. See the dataset guide.

Files

File	Description
hpsv3++.pth	Final HPSv3++ reward-model weights (17.6 GB)
config.json	Model configuration

Conditioning at inference

• Model capability is inferred implicitly from the image; you do not pass it in.
• RL iteration is passed explicitly as a normalized scalar in [0, 1].
  • General preference scoring / ranking: use 0.0 (pre-RL setting).
  • As the reward inside T2I RL fine-tuning: ramp the iteration condition linearly from 0.3 to 1.0 over training (the setting used in the paper).
• Use the mean (mu) output as the scalar reward.

Citation

Please cite the paper when using the HPSv3++ model or HPDv3++ dataset. Download BibTeX.

@article{liu2026hpsv3plusplus,
  title   = {HPSv3++: Scaling Reward Models Across the Full Spectrum of Diffusion Model Capabilities},
  author  = {Liu, Yijun and Huang, Jie and Xue, Zeyue and Li, Yuming and He, Ruizhe and Li, Haoran and Ge, Shijia and Fu, Siming},
  journal = {arXiv preprint arXiv:2606.14657},
  year    = {2026},
  doi     = {10.48550/arXiv.2606.14657},
  url     = {https://arxiv.org/abs/2606.14657}
}