Attoloop/PermitForge-Qwen3.8-27B-GRPO

PermitForge Qwen3.8-27B GRPO adapter

A LoRA adapter trained with GRPO on PermitForge, an OpenEnv environment where an agent chooses emissions controls and stack heights for a synthetic plant so that it meets every air-quality standard at the lowest annualized cost.

The result is a null result: no measurable change. On the same 10 held-out tasks, three repeats each, native verification for every submission, the base model saves 43.94% of the draft design's cost and this adapter saves 43.90%. The paired difference is -0.04 percentage points (95% CI -2.8 to +2.6). It is published so the run can be checked and continued, not because it is better.

	Base Qwen3.8-27B	This adapter
Mean saved fraction, 10 held-out tasks x 3	0.4394	0.4390
95% CI (station-clustered bootstrap)	0.3742 to 0.4950	0.3845 to 0.5005
Feasible submissions	30 / 30	30 / 30

Full tables, curves, the failures met on the way and the cost ledger: train/cloud/RESULTS.md in the repository. The eval card is eval-card.json here and results/eval-v2-qwen.json in the repository.

Training

Setting	Value
Base model	Qwen/Qwen3.8-27B, revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0, bf16, thinking off
Adapter	LoRA rank 16, alpha 32, dropout 0, on q/k/v/o of the full-attention layers
Method	TRL 1.12 GRPOTrainer with environment_factory, with one method overridden to compute log-probabilities in 2,048-token chunks
Steps	30, one task per step, 6 rollouts per task, gradient accumulation 6
Optimizer	paged AdamW 8-bit, learning rate 3e-5 with linear decay to 0, beta 0
Episode	40 tool turns, 30 evaluations, 32,768 tokens per reply, 40,960 completion tokens
Data	30 train tasks, 3 per train station, chosen by hash. Reward is the feasible saved fraction of draft cost
Hardware	one H200 on Hugging Face Jobs, in five-step jobs after the first job died at step 15
Cost	$82 of training, $62 of evaluation, $164 for every cloud job, billed at $5 per H200 hour

Files

• adapter_model.safetensors, adapter_config.json: the final adapter after step 30.
• checkpoints/checkpoint-{10,15,20,25,30}: complete trainer checkpoints (adapter, optimizer, scheduler, RNG).
• trainer_state.json, metrics.json, training_options.json, run.json: the trainer log, the run's metrics and its frozen configuration.
• tokenizer.json, tokenizer_config.json, chat_template.jinja: the tokenizer the run used.

The training rollouts, the base and trained evaluation episodes with their native grades, the console logs and the job ledger are in the dataset Attoloop/PermitForge under training/.

Use

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.8-27B", revision="1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "Attoloop/PermitForge-Qwen3.8-27B-GRPO")
tokenizer = AutoTokenizer.from_pretrained("Attoloop/PermitForge-Qwen3.8-27B-GRPO")

To reproduce the run or evaluate the adapter on the environment, follow train/cloud/README.md in the repository.