rajbsk commited on
Commit
6cea3dc
·
verified ·
1 Parent(s): c06cbf9

Update config.json to reflect float16 instead of bfloat16

Browse files

## What this changes

Sets `"torch_dtype": "float16"` in `config.json` (was `"bfloat16"`). No other changes.

## Why

The released weights are fp16, but `config.json` says bf16:

- The safetensors weights are stored as **F16**.
- RLinf trains this model in fp16 (`fp16: True` in the Megatron config) and evaluates it with `rollout.model.precision: fp16`.

vLLM, SGLang and HF Transformers read `torch_dtype` by default. So unless users pass `--dtype float16`, the model is served in **bf16**. That causes a consistent drop in accuracy.

## Results

Eval setup is the same as the paper's: temperature 1.0, top_p 1.0, 32K context, Qwen3-30B-A3B judge, Avg@4 on the ASearcher test sets. With this setup we reproduce the paper's Qwen3-4B baseline to within about 0.2 points on average.

Avg@4 (x100):

| Benchmark | Paper | bf16 (current default) | fp16 (this PR) |
|---|---|---|---|
| NQ | 56.1 | 54.3 | 55.2 |
| TriviaQA | 78.5 | 76.2 | 77.8 |
| PopQA | 48.5 | 47.4 | 47.9 |
| HotpotQA | 64.2 | 59.9 | 64.1 |
| 2Wiki | 75.0 | 70.7 | 75.1 |
| Bamboogle | 61.8 | 57.2 | 63.4 |
| MuSiQue | 28.9 | 25.6 | 28.4 |
| **Average** | **59.0** | **55.9** | **58.8** |

- With fp16, the paper's numbers reproduce: 58.8 vs 59.0 on average, and each dataset is within run-to-run noise.
- bf16 loses about 3 points on average, and 4 to 6 points on the multi-hop sets (HotpotQA, 2Wiki, Bamboogle). Small numeric differences likely add up over the long multi-turn trajectories.

## Workaround until merged

Serve with `--dtype float16` (vLLM / SGLang) or `torch_dtype=torch.float16` (Transformers).

Thanks for releasing WideSeek-R1!

Files changed (1) hide show
  1. config.json +1 -1
config.json CHANGED
@@ -22,7 +22,7 @@
22
  "rope_theta": 1000000,
23
  "sliding_window": null,
24
  "tie_word_embeddings": true,
25
- "torch_dtype": "bfloat16",
26
  "transformers_version": "4.51.0",
27
  "use_cache": true,
28
  "use_sliding_window": false,
 
22
  "rope_theta": 1000000,
23
  "sliding_window": null,
24
  "tie_word_embeddings": true,
25
+ "torch_dtype": "float16",
26
  "transformers_version": "4.51.0",
27
  "use_cache": true,
28
  "use_sliding_window": false,