Instructions to use 96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use 96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive")
model = AutoModelForCausalLM.from_pretrained("96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use 96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive

SGLang

How to use 96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use 96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive with Docker Model Runner:
```
docker model run hf.co/96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive
```

Qwen3-4B-MathRL-Multilingual-Additive / README.md

96kevinli29

Redesign card: figure, key results, quick start

63d7807 verified 3 days ago

preview code

Raw History Blame Contribute Delete

10.1 kB

---
language:
  - en
  - zh
  - es
  - fr
  - ar
  - ru
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags:
  - qwen3
  - causal-lm
  - reinforcement-learning
  - ppo
  - rlvr
  - math
  - reasoning
  - multilingual
  - multilingual-reasoning
  - chain-of-thought
  - verl
base_model: 96kevinli29/Qwen3-4B-SFT-Math-Multilingual
datasets:
  - 96kevinli29/DAPO-Math-Multilingual-6Lang
model-index:
  - name: Qwen3-4B-MathRL-Multilingual-Additive
    results:
      - task:
          type: text-generation
        dataset:
          name: AIME-2026 + AMC-2023, five non-English languages (zh, es, fr, ar, ru)
          type: 96kevinli29/Multilingual-Math-Reasoning-Eval
        metrics:
          - name: Delivered J@1 (correct, in the requested language, terminated)
            type: accuracy
            value: 59.8
          - name: Answer accuracy
            type: accuracy
            value: 60.1
          - name: Reasoning in the requested language
            type: accuracy
            value: 99.6
---
<div align="center">

# Qwen3-4B-MathRL-Multilingual-Additive

**Qwen3-4B that reasons in the language of the problem, and finishes**

<a href="https://arxiv.org/abs/2609.37104"><img alt="arXiv: 2609.37104" src="https://img.shields.io/badge/arXiv-2609.37104-b31b1b"></a> <a href="https://huggingface.co/Qwen/Qwen3-4B"><img alt="base: Qwen3-4B" src="https://img.shields.io/badge/base-Qwen3--4B-475467"></a> <img alt="stage: RL additive" src="https://img.shields.io/badge/stage-RL_additive-1baf7a"> <img alt="languages: 6" src="https://img.shields.io/badge/languages-6-2a78d6"> <img alt="license: Apache 2.0" src="https://img.shields.io/badge/license-Apache_2.0-1baf7a">

</div>

<img src="https://huggingface.co/96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive/resolve/main/assets/lineage_4b.png" alt="Bar chart, non-English mean. Reasoning in the requested language: released 40%, SFT 99%, RL additive 100%, RL gated 100%. Did not finish: 5%, 21%, 1%, 2%. Delivered: 29%, 60%, 60%, 61%." width="100%">

<p align="center"><sub>Mean over Chinese, Spanish, French, Arabic and Russian · 70 problems × 16 samples per language</sub></p>

- **What it is.** [Qwen3-4B-SFT-Math-Multilingual](https://huggingface.co/96kevinli29/Qwen3-4B-SFT-Math-Multilingual) after 200 PPO steps on localized math prompts with the **additive reward `R = 0.7 · C + 0.3 · ℓ`**: correctness `C` plus a language score `ℓ` that is paid whether or not the answer is right.
- **What it fixes.** The SFT model reasons in the right language but 21.1% of its non-English responses never finish. Here that is **1.2%**, the reasoning stays in the requested language (99.6%), and the median response shrinks from 7,673 to 4,072 tokens.
- **What to know.** It is one of two matched runs that differ only in reward; [Qwen3-4B-MathRL-Multilingual-Gated](https://huggingface.co/96kevinli29/Qwen3-4B-MathRL-Multilingual-Gated) (gated reward) performs comparably. A third run rewarded for correctness alone also stops the looping but drifts back to English.

## Results

| | Released | SFT | **RL additive** | RL gated |
| :--- | :---: | :---: | :---: | :---: |
| Answer correct | 76.2% | 61.4% | **60.1%** | 61.7% |
| Reasoning in the requested language | 39.8% | 99.4% | **99.6%** | 99.7% |
| Did not finish | 4.7% | 21.1% | **1.2%** | 1.8% |
| **Delivered** (J@1) | 28.6% | 60.2% | **59.8%** | 61.4% |
| **Delivered**, best of 16 (J@16) | 35.7% | 84.6% | **83.1%** | 83.7% |
| Median response, tokens | 5,824 | 7,673 | **4,072** | 4,463 |

70 competition problems (30 AIME-2026 + 40 AMC-2023) per language from [Multilingual-Math-Reasoning-Eval](https://huggingface.co/datasets/96kevinli29/Multilingual-Math-Reasoning-Eval), 16 samples each at temperature 0.7, 24,576-token budget. A response is **delivered** when the answer is correct, the visible reasoning is in the requested language, and it terminates (`J = C ∧ L ∧ T`, scored per response). Tables show the mean over Chinese, Spanish, French, Arabic and Russian.

<details>
<summary><b>Per language · other benchmarks</b></summary>

| | Correct | In language | Did not finish | J@1 | J@16 | Median tokens |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| English | 65.7% | 100.0% | 0.2% | 65.7% | 85.7% | 3,510 |
| Chinese | 58.7% | 99.7% | 0.6% | 58.5% | 81.4% | 3,596 |
| Spanish | 61.5% | 99.8% | 0.8% | 61.4% | 81.4% | 3,856 |
| French | 61.3% | 99.8% | 0.3% | 61.3% | 85.7% | 3,818 |
| Arabic | 57.3% | 98.6% | 2.1% | 56.1% | 82.9% | 4,566 |
| Russian | 61.7% | 99.9% | 2.0% | 61.7% | 84.3% | 4,524 |

| Delivered (J), non-English mean | Released | SFT | **RL additive** | RL gated |
| :--- | :---: | :---: | :---: | :---: |
| PolyMath, 444 problems | 16.8% | 34.1% | **35.4%** | 36.8% |
| MMLU-ProX-Lite, 588 questions | 24.0% | 49.0% | **56.2%** | 55.5% |

</details>

## Quick start

```python
from datasets import load_dataset
from transformers import AutoModelForCausalLM, AutoTokenizer
name = "96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name, torch_dtype="auto", device_map="auto")
# a French problem with a French instruction: "…en raisonnant entièrement en français."
prompt = load_dataset("96kevinli29/Multilingual-Math-Reasoning-Eval", "fr")["amc23"][0]["prompt"]
text = tok.apply_chat_template([{"role": "user", "content": prompt}], tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=24576, do_sample=True, temperature=0.7)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
```

Qwen3 chat template with thinking on; the reasoning appears inside `<think> … </think>`. Give hard problems a large `max_new_tokens`. Loads with transformers 4.x and 5.x.

<details>
<summary><b>Prompt format in six languages</b></summary>

| | Instruction that opens the user turn |
| :--- | :--- |
| English | Solve the following math problem step by step, reasoning entirely in English. |
| Chinese | 请全程用中文逐步推理并解决以下数学问题。 |
| Spanish | Resuelve el siguiente problema matemático paso a paso, razonando enteramente en español. |
| French | Résous le problème de mathématiques suivant étape par étape, en raisonnant entièrement en français. |
| Arabic | حل المسألة الرياضية التالية خطوة بخطوة مع التفكير بالكامل باللغة العربية. |
| Russian | Решите следующую математическую задачу шаг за шагом, рассуждая полностью на русском языке. |

The full prompts (instruction, problem and answer format, all in the target language) are in the evaluation set.

</details>

## Model family

| Stage | Checkpoint | |
| :--- | :--- | :--- |
| I · Release | [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B) · [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) | solves the problem, reasons in English |
| II · SFT | [Qwen3-4B-SFT-Math-Multilingual](https://huggingface.co/96kevinli29/Qwen3-4B-SFT-Math-Multilingual) | reasons in the requested language, sometimes loops |
| II · SFT | [Qwen3-8B-SFT-Math-Multilingual](https://huggingface.co/96kevinli29/Qwen3-8B-SFT-Math-Multilingual) | the same result at 8B |
| III · RL | [Qwen3-4B-MathRL-Multilingual-Gated](https://huggingface.co/96kevinli29/Qwen3-4B-MathRL-Multilingual-Gated) | keeps the language, stops the looping |
| III · RL | [Qwen3-4B-MathRL-Multilingual-Additive](https://huggingface.co/96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive) ← this model | keeps the language, stops the looping |

Data: [SFT-Math-Multilingual-43k](https://huggingface.co/datasets/96kevinli29/SFT-Math-Multilingual-43k) · [SFT-Math-Multilingual-56k](https://huggingface.co/datasets/96kevinli29/SFT-Math-Multilingual-56k) · [DAPO-Math-Multilingual-6Lang](https://huggingface.co/datasets/96kevinli29/DAPO-Math-Multilingual-6Lang) · [Multilingual-Math-Reasoning-Eval](https://huggingface.co/datasets/96kevinli29/Multilingual-Math-Reasoning-Eval)

## Training

| | |
| :--- | :--- |
| Algorithm | PPO, GAE with γ = λ = 1, clip ratio 0.2, no KL term |
| Reward | `0.7 · C + 0.3 · ℓ`, plus a soft penalty for overlong responses |
| Prompts | [DAPO-Math-Multilingual-6Lang](https://huggingface.co/datasets/96kevinli29/DAPO-Math-Multilingual-6Lang), 16,754 localized DAPO-Math-17K prompts |
| Batch | 32 prompts × 8 rollouts per step, 16,384-token response cap |
| Learning rate | 1e-6 actor, 1e-5 critic |
| Steps | 200 |
| Hardware · framework | 8 GPUs on one node · [verl](https://github.com/volcengine/verl), FSDP |

`C` is rule-based correctness of the final answer; `ℓ ∈ [0, 1]` is a language-identification score of the response against the language of the prompt.

## Limitations

- **Accuracy** stays below the released Qwen3-4B on the same problems (76.2% → 60.1%). The cost enters with SFT; RL does not recover it, and it is now what limits delivery.
- 200 PPO steps, one seed, one model size.
- Mathematics and six languages only.

## Citation

```bibtex
@article{li2026posttraining,
  title   = {What Does Post-Training Change in Multilingual Reasoning?},
  author  = {Li, Hongyang and Li, Xiao and Wu, Caesar and Danoy, Gr{\'e}goire and Bouvry, Pascal},
  journal = {arXiv preprint arXiv:2609.37104},
  year    = {2026}
}
@misc{qwen3-4b-mathrl-multilingual-additive-2026,
  title        = {{Qwen3-4B-MathRL-Multilingual-Additive}: PPO from multilingual-SFT {Qwen3-4B} with a additive language reward},
  author       = {Li, Hongyang and Li, Xiao and Wu, Caesar and Danoy, Gr{\'e}goire and Bouvry, Pascal and {Sea-Fill Community}},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive}},
  note         = {Maintained by Sea-Fill Community.}
}
```

Built and released by the authors together with the [Sea-Fill Community](https://huggingface.co/SeaFill2025), which maintains this release.

	---
	language:
	- en
	- zh
	- es
	- fr
	- ar
	- ru
	license: apache-2.0
	library_name: transformers
	pipeline_tag: text-generation
	tags:
	- qwen3
	- causal-lm
	- reinforcement-learning
	- ppo
	- rlvr
	- math
	- reasoning
	- multilingual
	- multilingual-reasoning
	- chain-of-thought
	- verl
	base_model: 96kevinli29/Qwen3-4B-SFT-Math-Multilingual
	datasets:
	- 96kevinli29/DAPO-Math-Multilingual-6Lang
	model-index:
	- name: Qwen3-4B-MathRL-Multilingual-Additive
	results:
	- task:
	type: text-generation
	dataset:
	name: AIME-2026 + AMC-2023, five non-English languages (zh, es, fr, ar, ru)
	type: 96kevinli29/Multilingual-Math-Reasoning-Eval
	metrics:
	- name: Delivered J@1 (correct, in the requested language, terminated)
	type: accuracy
	value: 59.8
	- name: Answer accuracy
	type: accuracy
	value: 60.1
	- name: Reasoning in the requested language
	type: accuracy
	value: 99.6
	---

	<div align="center">

	# Qwen3-4B-MathRL-Multilingual-Additive

	Qwen3-4B that reasons in the language of the problem, and finishes

	<a href="https://arxiv.org/abs/2609.37104"><img alt="arXiv: 2609.37104" src="https://img.shields.io/badge/arXiv-2609.37104-b31b1b"></a> <a href="https://huggingface.co/Qwen/Qwen3-4B"><img alt="base: Qwen3-4B" src="https://img.shields.io/badge/base-Qwen3--4B-475467"></a> <img alt="stage: RL additive" src="https://img.shields.io/badge/stage-RL_additive-1baf7a"> <img alt="languages: 6" src="https://img.shields.io/badge/languages-6-2a78d6"> <img alt="license: Apache 2.0" src="https://img.shields.io/badge/license-Apache_2.0-1baf7a">

	</div>

	<img src="https://huggingface.co/96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive/resolve/main/assets/lineage_4b.png" alt="Bar chart, non-English mean. Reasoning in the requested language: released 40%, SFT 99%, RL additive 100%, RL gated 100%. Did not finish: 5%, 21%, 1%, 2%. Delivered: 29%, 60%, 60%, 61%." width="100%">

	<p align="center"><sub>Mean over Chinese, Spanish, French, Arabic and Russian · 70 problems × 16 samples per language</sub></p>

	- What it is. [Qwen3-4B-SFT-Math-Multilingual](https://huggingface.co/96kevinli29/Qwen3-4B-SFT-Math-Multilingual) after 200 PPO steps on localized math prompts with the additive reward `R = 0.7 · C + 0.3 · ℓ`: correctness `C` plus a language score `ℓ` that is paid whether or not the answer is right.
	- What it fixes. The SFT model reasons in the right language but 21.1% of its non-English responses never finish. Here that is 1.2%, the reasoning stays in the requested language (99.6%), and the median response shrinks from 7,673 to 4,072 tokens.
	- What to know. It is one of two matched runs that differ only in reward; [Qwen3-4B-MathRL-Multilingual-Gated](https://huggingface.co/96kevinli29/Qwen3-4B-MathRL-Multilingual-Gated) (gated reward) performs comparably. A third run rewarded for correctness alone also stops the looping but drifts back to English.

	## Results

	\| \| Released \| SFT \| RL additive \| RL gated \|
	\| :--- \| :---: \| :---: \| :---: \| :---: \|
	\| Answer correct \| 76.2% \| 61.4% \| 60.1% \| 61.7% \|
	\| Reasoning in the requested language \| 39.8% \| 99.4% \| 99.6% \| 99.7% \|
	\| Did not finish \| 4.7% \| 21.1% \| 1.2% \| 1.8% \|
	\| Delivered (J@1) \| 28.6% \| 60.2% \| 59.8% \| 61.4% \|
	\| Delivered, best of 16 (J@16) \| 35.7% \| 84.6% \| 83.1% \| 83.7% \|
	\| Median response, tokens \| 5,824 \| 7,673 \| 4,072 \| 4,463 \|

	70 competition problems (30 AIME-2026 + 40 AMC-2023) per language from [Multilingual-Math-Reasoning-Eval](https://huggingface.co/datasets/96kevinli29/Multilingual-Math-Reasoning-Eval), 16 samples each at temperature 0.7, 24,576-token budget. A response is delivered when the answer is correct, the visible reasoning is in the requested language, and it terminates (`J = C ∧ L ∧ T`, scored per response). Tables show the mean over Chinese, Spanish, French, Arabic and Russian.

	<details>
	<summary><b>Per language · other benchmarks</b></summary>

	\| \| Correct \| In language \| Did not finish \| J@1 \| J@16 \| Median tokens \|
	\| :--- \| :---: \| :---: \| :---: \| :---: \| :---: \| :---: \|
	\| English \| 65.7% \| 100.0% \| 0.2% \| 65.7% \| 85.7% \| 3,510 \|
	\| Chinese \| 58.7% \| 99.7% \| 0.6% \| 58.5% \| 81.4% \| 3,596 \|
	\| Spanish \| 61.5% \| 99.8% \| 0.8% \| 61.4% \| 81.4% \| 3,856 \|
	\| French \| 61.3% \| 99.8% \| 0.3% \| 61.3% \| 85.7% \| 3,818 \|
	\| Arabic \| 57.3% \| 98.6% \| 2.1% \| 56.1% \| 82.9% \| 4,566 \|
	\| Russian \| 61.7% \| 99.9% \| 2.0% \| 61.7% \| 84.3% \| 4,524 \|

	\| Delivered (J), non-English mean \| Released \| SFT \| RL additive \| RL gated \|
	\| :--- \| :---: \| :---: \| :---: \| :---: \|
	\| PolyMath, 444 problems \| 16.8% \| 34.1% \| 35.4% \| 36.8% \|
	\| MMLU-ProX-Lite, 588 questions \| 24.0% \| 49.0% \| 56.2% \| 55.5% \|

	</details>

	## Quick start

	```python
	from datasets import load_dataset
	from transformers import AutoModelForCausalLM, AutoTokenizer

	name = "96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive"
	tok = AutoTokenizer.from_pretrained(name)
	model = AutoModelForCausalLM.from_pretrained(name, torch_dtype="auto", device_map="auto")

	# a French problem with a French instruction: "…en raisonnant entièrement en français."
	prompt = load_dataset("96kevinli29/Multilingual-Math-Reasoning-Eval", "fr")["amc23"][0]["prompt"]
	text = tok.apply_chat_template([{"role": "user", "content": prompt}], tokenize=False, add_generation_prompt=True)
	inputs = tok(text, return_tensors="pt").to(model.device)
	out = model.generate(**inputs, max_new_tokens=24576, do_sample=True, temperature=0.7)
	print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
	```

	Qwen3 chat template with thinking on; the reasoning appears inside `<think> … </think>`. Give hard problems a large `max_new_tokens`. Loads with transformers 4.x and 5.x.

	<details>
	<summary><b>Prompt format in six languages</b></summary>

	\| \| Instruction that opens the user turn \|
	\| :--- \| :--- \|
	\| English \| Solve the following math problem step by step, reasoning entirely in English. \|
	\| Chinese \| 请全程用中文逐步推理并解决以下数学问题。 \|
	\| Spanish \| Resuelve el siguiente problema matemático paso a paso, razonando enteramente en español. \|
	\| French \| Résous le problème de mathématiques suivant étape par étape, en raisonnant entièrement en français. \|
	\| Arabic \| حل المسألة الرياضية التالية خطوة بخطوة مع التفكير بالكامل باللغة العربية. \|
	\| Russian \| Решите следующую математическую задачу шаг за шагом, рассуждая полностью на русском языке. \|

	The full prompts (instruction, problem and answer format, all in the target language) are in the evaluation set.

	</details>

	## Model family

	\| Stage \| Checkpoint \| \|
	\| :--- \| :--- \| :--- \|
	\| I · Release \| [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B) · [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) \| solves the problem, reasons in English \|
	\| II · SFT \| [Qwen3-4B-SFT-Math-Multilingual](https://huggingface.co/96kevinli29/Qwen3-4B-SFT-Math-Multilingual) \| reasons in the requested language, sometimes loops \|
	\| II · SFT \| [Qwen3-8B-SFT-Math-Multilingual](https://huggingface.co/96kevinli29/Qwen3-8B-SFT-Math-Multilingual) \| the same result at 8B \|
	\| III · RL \| [Qwen3-4B-MathRL-Multilingual-Gated](https://huggingface.co/96kevinli29/Qwen3-4B-MathRL-Multilingual-Gated) \| keeps the language, stops the looping \|
	\| III · RL \| [Qwen3-4B-MathRL-Multilingual-Additive](https://huggingface.co/96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive) ← this model \| keeps the language, stops the looping \|

	Data: [SFT-Math-Multilingual-43k](https://huggingface.co/datasets/96kevinli29/SFT-Math-Multilingual-43k) · [SFT-Math-Multilingual-56k](https://huggingface.co/datasets/96kevinli29/SFT-Math-Multilingual-56k) · [DAPO-Math-Multilingual-6Lang](https://huggingface.co/datasets/96kevinli29/DAPO-Math-Multilingual-6Lang) · [Multilingual-Math-Reasoning-Eval](https://huggingface.co/datasets/96kevinli29/Multilingual-Math-Reasoning-Eval)

	## Training

	\| \| \|
	\| :--- \| :--- \|
	\| Algorithm \| PPO, GAE with γ = λ = 1, clip ratio 0.2, no KL term \|
	\| Reward \| `0.7 · C + 0.3 · ℓ`, plus a soft penalty for overlong responses \|
	\| Prompts \| [DAPO-Math-Multilingual-6Lang](https://huggingface.co/datasets/96kevinli29/DAPO-Math-Multilingual-6Lang), 16,754 localized DAPO-Math-17K prompts \|
	\| Batch \| 32 prompts × 8 rollouts per step, 16,384-token response cap \|
	\| Learning rate \| 1e-6 actor, 1e-5 critic \|
	\| Steps \| 200 \|
	\| Hardware · framework \| 8 GPUs on one node · [verl](https://github.com/volcengine/verl), FSDP \|

	`C` is rule-based correctness of the final answer; `ℓ ∈ [0, 1]` is a language-identification score of the response against the language of the prompt.

	## Limitations

	- Accuracy stays below the released Qwen3-4B on the same problems (76.2% → 60.1%). The cost enters with SFT; RL does not recover it, and it is now what limits delivery.
	- 200 PPO steps, one seed, one model size.
	- Mathematics and six languages only.

	## Citation

	```bibtex
	@article{li2026posttraining,
	title = {What Does Post-Training Change in Multilingual Reasoning?},
	author = {Li, Hongyang and Li, Xiao and Wu, Caesar and Danoy, Gr{\'e}goire and Bouvry, Pascal},
	journal = {arXiv preprint arXiv:2609.37104},
	year = {2026}
	}

	@misc{qwen3-4b-mathrl-multilingual-additive-2026,
	title = {{Qwen3-4B-MathRL-Multilingual-Additive}: PPO from multilingual-SFT {Qwen3-4B} with a additive language reward},
	author = {Li, Hongyang and Li, Xiao and Wu, Caesar and Danoy, Gr{\'e}goire and Bouvry, Pascal and {Sea-Fill Community}},
	year = {2026},
	publisher = {Hugging Face},
	howpublished = {\url{https://huggingface.co/96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive}},
	note = {Maintained by Sea-Fill Community.}
	}
	```

	Built and released by the authors together with the [Sea-Fill Community](https://huggingface.co/SeaFill2025), which maintains this release.