AtmosRegime-Qwen3-8B-SFT-RL

Qwen3-8B fine-tuned on a few verified worked examples and then trained with reinforcement learning, from the paper Language models misapply atmospheric formulas until trained on a few worked examples. In the paper, this route repairs mechanisms that reinforcement learning alone could not: outside-regime accuracy on them rises by 61 points.

Code and data: https://github.com/acodercat/AtmosRegime. The other models from the paper are in the AtmosRegime collection.

Training

  • Base model: Qwen3-8B, thinking off.
  • Step 1, supervised fine-tuning (one epoch) on worked examples. A worked example is a program the base model wrote when the correct formula was stated in the prompt, verified by execution and paired with the problem without the formula. The mechanisms that reinforcement learning alone had not learned received 0, 1, 2 or 4 examples each, assigned at random.
  • Step 2, reinforcement learning: DAPO in verl with the answer reward, the same settings as the other routes (learning rate 1e-6, 128 problems ร— 8 samples per step, 120 steps).
  • Data: training fold 1 of the study's templates (29 of the 70 mechanisms); seed 42.

Use

The model was trained on one prompt format and should be given the same one. It answers with a Python program that defines solve(). Run it with thinking disabled.

# git clone https://github.com/acodercat/AtmosRegime && cd AtmosRegime   (for the prompt)
from transformers import AutoModelForCausalLM, AutoTokenizer
from atmoscoder_rl.prompts import chat

name = "codercat/AtmosRegime-Qwen3-8B-SFT-RL"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name, torch_dtype="auto", device_map="auto")

problem = "Atmospheric CO2 rises from 280 ppm to 560 ppm. Calculate the radiative forcing in W m-2."
text = tok.apply_chat_template(chat(problem), tokenize=False, add_generation_prompt=True, enable_thinking=False)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=3072, do_sample=True, temperature=0.3)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

Execution and grading of the returned program are in atmoscoder_rl/evaluation/ of the repository.

Limitations

  • This checkpoint is one fold and one seed of a randomised design. Numbers in the paper compare mechanisms with and without examples across both folds and two seeds, so this model alone will not reproduce them.
  • The repair was shown on problems from the same templates. Whether it carries over to independently sourced problems of the same mechanism was not tested.
  • It is not a general-purpose assistant for atmospheric science.

Citation

@unpublished{ran2026regime,
  title  = {Language models misapply atmospheric formulas until trained on a few worked examples},
  author = {Ran, Maohao and Zhang, Yanting and Ma, Chendong and Song, Jun},
  year   = {2026},
  note   = {Manuscript under review}
}

Licence

Apache 2.0, as for the base model Qwen3-8B.

Downloads last month
125
Safetensors
Model size
8B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for codercat/AtmosRegime-Qwen3-8B-SFT-RL

Finetuned
Qwen/Qwen3-8B
Finetuned
(2,330)
this model

Collection including codercat/AtmosRegime-Qwen3-8B-SFT-RL