Junjun2333 commited on
Commit
ac26578
·
verified ·
1 Parent(s): 2df0408

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +44 -0
README.md ADDED
@@ -0,0 +1,44 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model:
4
+ - Qwen/Qwen3-VL-8B-Instruct
5
+ pipeline_tag: image-text-to-text
6
+ tags:
7
+ - reward-model
8
+ - text-to-image
9
+ - human-preference
10
+ - rlhf
11
+ ---
12
+
13
+ # HPSv3++: Scaling Reward Models Across the Full Spectrum of Diffusion Model Capabilities
14
+
15
+ HPSv3++ is a **capability-aware and RL-iteration-aware** text-to-image (T2I) reward model, built on the `Qwen/Qwen3-VL-8B-Instruct` backbone with a Capability Encoder, a FiLM conditioning head, and a three-layer RankNet reward head.
16
+
17
+ A Capability Encoder implicitly infers the generative ability of the model that produced an image, while the RL iteration step is supplied as an explicit condition; the two are jointly modulated through FiLM so that a single reward model produces calibrated preference scores across generators of differing capability and different stages of RL optimization.
18
+
19
+ The training/evaluation dataset, HPDv3++, is released separately: [Junjun2333/HPDv3-PlusPlus](https://huggingface.co/datasets/Junjun2333/HPDv3-PlusPlus).
20
+
21
+ ## Files
22
+
23
+ | File | Description |
24
+ |---|---|
25
+ | `hpsv3++.pth` | Final HPSv3++ reward-model weights (17.6 GB) |
26
+ | `config.json` | Model configuration |
27
+
28
+ ## Conditioning at inference
29
+
30
+ - **Model capability** is inferred implicitly from the image; you do not pass it in.
31
+ - **RL iteration** is passed explicitly as a normalized scalar in `[0, 1]`.
32
+ - General preference scoring / ranking: use `0.0` (pre-RL setting).
33
+ - As the reward inside T2I RL fine-tuning: ramp the iteration condition linearly from `0.3` to `1.0` over training (the setting used in the paper).
34
+ - Use the mean (`mu`) output as the scalar reward.
35
+
36
+ ## Citation
37
+
38
+ ```bibtex
39
+ @misc{hpsv3pp,
40
+ title = {HPSv3++: Scaling Reward Models Across the Full Spectrum of Diffusion Model Capabilities},
41
+ author = {HPSv3++ Team},
42
+ year = {2026}
43
+ }
44
+ ```