talzoomanzoo/ttrl-permuted

Permuted-UID TTRL

LoRA adapter for Qwen/Qwen3-1.7B, saved after training step 2 of the Permuted-UID TTRL experiment. This repository contains adapter weights, not a merged base model.

The reward combines answer agreement with a UID auxiliary bonus. After centering, clipping, and gating, the realized UID bonuses are randomly permuted among majority-agreeing responses within each prompt, preserving their per-prompt distribution.

Training used 30 AIME 2026 prompts, 8 responses per prompt, 2 epochs, seed 42, and a maximum response length of 16,384 tokens. LoRA rank is 16 and alpha is 32. The run used 4 NVIDIA A40 GPUs. AIME 2026 was used for both adaptation and final evaluation; evaluation on that dataset is not held-out transfer evaluation.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-1.7B", torch_dtype="auto", device_map="auto"
)
model = PeftModel.from_pretrained(base, "talzoomanzoo/ttrl-permuted")
model.eval()
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-1.7B")

Source run: aime2026-permuted-a40-seed42-4063898-uid_permuted, checkpoint global_step_2.