File size: 2,544 Bytes
c4c3778 b94c879 c4c3778 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | ---
library_name: transformers
pipeline_tag: text-generation
base_model: FrenzyMath/Reaper-SFT
base_model_relation: finetune
tags:
- qwen3
- reaper
- policy-value
- dual-head
---
# Reaper-1
Reaper-1 is the final policy-value checkpoint in the Reaper model family. The repository stores a Qwen3 1.7B policy model together with a separate categorical value head.
This checkpoint is part of the [Reaper model collection](https://huggingface.co/collections/FrenzyMath/reaper-6ad03ad88957749d6114b8f6).
## Checkpoint structure
- `policy_model/`: `Qwen3ForCausalLM` policy checkpoint
- `value_head.pt`: state dictionary for a linear value head with 32 output bins
- `dual_head_meta.json`: policy/value loss weights and value-head configuration
- Tokenizer and chat-template files are stored at the repository root
The included metadata specifies `policy_loss_weight=1.0`, `value_loss_weight=0.1`, and `value_dropout=0.1`.
## Model family
| Checkpoint | Stage |
| --- | --- |
| [FrenzyMath/Reaper-Base](https://huggingface.co/FrenzyMath/Reaper-Base) | Base checkpoint |
| [FrenzyMath/Reaper-SFT](https://huggingface.co/FrenzyMath/Reaper-SFT) | Supervised fine-tuning |
| [FrenzyMath/Reaper-1](https://huggingface.co/FrenzyMath/Reaper-1) | Final policy-value checkpoint |
## Load the policy model
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "FrenzyMath/Reaper-1"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
policy_model = AutoModelForCausalLM.from_pretrained(
repo_id,
subfolder="policy_model",
torch_dtype="auto",
device_map="auto",
)
```
## Load the value head
```python
import json
import torch
from huggingface_hub import hf_hub_download
meta_path = hf_hub_download(repo_id, "dual_head_meta.json")
head_path = hf_hub_download(repo_id, "value_head.pt")
with open(meta_path, encoding="utf-8") as handle:
metadata = json.load(handle)
value_head = torch.nn.Linear(
policy_model.config.hidden_size,
metadata["num_value_bins"],
)
value_head.load_state_dict(
torch.load(head_path, map_location="cpu", weights_only=True)
)
value_head.eval()
```
The policy and value head are stored separately. Integrating value predictions with policy hidden states depends on the calling application.
## Intended use and limitations
This checkpoint is intended for research with policy-value modeling. Users should independently evaluate it for their target domain. Training-data details, benchmark results, and a license declaration are not included in the current release.
|