| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3-VL-4B-Instruct |
| library_name: transformers |
| inference: false |
| tags: |
| - worl |
| - nvrm |
| - reward-model |
| - video-reward-model |
| - qwen3-vl |
| - safetensors |
| --- |
| |
| # WORL-NVRM |
|
|
| **Next-Video Reward Model (NVRM)** scores a candidate video continuation in the |
| context of preceding video and a text prompt. It provides a relative preference |
| signal for reinforcement learning of video world models. |
|
|
| <img src="assets/nvrm-preferences.jpg" alt="Examples of higher- and lower-ranked video continuations" width="100%"> |
|
|
| Related policy releases: [WORL-Helios](https://huggingface.co/Efficient-Large-Model/WORL-Helios) |
| and [WORL-Lingbot](https://huggingface.co/Efficient-Large-Model/WORL-Lingbot). |
|
|
| ## Model details |
|
|
| | Field | Description | |
| | --- | --- | |
| | Base model | [Qwen3-VL-4B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct) | |
| | Base revision | `ebb281ec70b05090aa6165b016eac8ec08e71b17` | |
| | Architecture | 36-layer Qwen3-VL backbone with a one-dimensional reward head | |
| | Release format | Merged BF16 backbone and separate scalar head, all in Safetensors | |
| | Input | Text prompt, preceding video context and candidate continuation | |
| | Output | One unnormalized scalar reward; higher indicates stronger relative preference | |
| | Intended use | Video preference research and reward evaluation in video RL | |
|
|
| The training LoRA is already merged into the backbone. The reward head is in |
| `nvrm_head.safetensors`, under key `rm_head.weight`. This is a reward model, |
| not a text-generation model; loading it as a generic chat model does not produce |
| the documented reward. |
|
|
| ## Training data and method |
|
|
| NVRM uses pairwise video preferences with a Bradley–Terry objective. The WORL |
| manuscript describes approximately 312.4K preference pairs drawn from Sekai |
| Real Walking, Sekai Game, OmniWorld, Object-centric Revisiting, OpenScene and |
| General categories. Preference construction uses automated multimodal judgments, |
| order-consistency checks and graph-based filtering. |
|
|
| The team reports that initial scene imagery for its self-created OpenScene |
| data was generated with gpt-image. This repository distributes model artifacts; |
| the full preference corpus is not bundled with the weights. |
|
|
| <img src="assets/nvrm-training.png" alt="NVRM pairwise preference training" width="100%"> |
|
|
| ## Download and use |
|
|
| ```bash |
| hf download Efficient-Large-Model/WORL-NVRM --local-dir artifacts/WORL-NVRM |
| ``` |
|
|
| Use the WORL NVRM reward runtime, which loads the merged `Qwen3VLModel` backbone |
| and the separate scalar head. The complete chat formatting and preprocessing |
| contract is required; a generic Transformers pipeline is not a replacement for |
| that runtime. The WORL source release is being prepared separately. |
|
|
| | Preprocessing / scoring | Release contract | |
| | --- | --- | |
| | Candidate continuation | 5 seconds | |
| | Frame sampling | 1 FPS | |
| | Resize | Height 360 | |
| | Frame encoding | JPEG quality 85 | |
| | Image pixel bounds | 1,024–76,800 | |
| | Thinking / hidden-state normalization | Disabled / disabled | |
| | Reward | Last-token scalar, raw BF16 logit | |
|
|
| The source runtime profile uses Transformers 4.57.6, Accelerate 1.13.0, |
| qwen-vl-utils 0.0.14 and FlashAttention 2.7.4.post1. The artifact does not require |
| `trust_remote_code`. Pin a Hub revision for reproducible use and verify the |
| files against `worl-bundle.json` or `RELEASE_MANIFEST.json`. |
|
|
| ## Files and provenance |
|
|
| The repository contains three backbone shards, the reward head, tokenizer and |
| processor assets, figures, license notices and integrity manifests. |
|
|
| Weights and model/processor configurations are unchanged from |
| [qyoo/NVRM-world at 03f850630c0b](https://huggingface.co/qyoo/NVRM-world/tree/03f850630c0b66aef8f54279b82c39c2b058a5ce). |
| Documentation and release manifests were updated for this repository. |
| `RELEASE_MANIFEST.json` records the source revision and checksums. |
|
|
| The source release reports matching preprocessing tensors and raw BF16 reward |
| output against its inference fixture. That is a conversion/runtime check, not |
| a measure of general reward quality. This publication adds no new model-quality |
| or hardware benchmark. |
|
|
| ## Limitations |
|
|
| Rewards are relative and distribution-dependent, not calibrated probabilities. |
| The model can miss physical or temporal errors, learn superficial preferences, |
| and be vulnerable to reward hacking. Biases in the base model, candidate videos |
| and preference judgments can affect rankings. Behavior beyond the documented |
| preprocessing profile is not established by this release. |
|
|
| NVRM is intended for research and comparative reward evaluation. Its output |
| does not establish factual correctness, content safety or suitability for |
| safety-critical decisions. |
|
|
| ## License |
|
|
| This release retains the source model's |
| [Apache License 2.0](LICENSE.txt). Preserve [NOTICE.txt](NOTICE.txt) when |
| redistributing it. The merged Qwen3-VL backbone is also distributed under |
| Apache-2.0. |
|
|
