AF_RL — AndroidFlux GRPO LoRA adapters

GRPO-trained LoRA adapters for Qwen3-VL-2B-Instruct on Android GUI error recovery. Each adapter lives in its own subfolder.

subfolder	data split	reward model	final reward
scalar-firsterror-20260817	first_error (2,662)	Bradley-Terry scalar head	+4.608
discrete-firsterror-20260817	first_error (2,662)	judge-token margin	−2.273
scalar-preerror-20260818	pre_error (2,669)	Bradley-Terry scalar head	+4.982
discrete-preerror-20260818	pre_error (2,669)	judge-token margin	−5.053

Reward numbers are not comparable across rows. The two reward models emit different scales, and the two splits are different states. Compare policies by downstream task performance, not by these values.

• first_error — the screen immediately after the first erroneous action; the policy must recover.
• pre_error — the screen before the erroneous action; the policy must avoid the mistake.

import torch
from transformers import AutoModelForImageTextToText
from peft import PeftModel

base = AutoModelForImageTextToText.from_pretrained(
    "Qwen/Qwen3-VL-2B-Instruct",
    revision="89644892e4d85e24eaac8bacfd4f463576704203",
    dtype=torch.bfloat16, device_map="cuda")
model = PeftModel.from_pretrained(
    base, "YeMoKoo/AF_RL", subfolder="scalar-preerror-20260818")

Qwen3-VL is a vision-language model, so AutoModelForCausalLM / AutoPeftModelForCausalLM do not work — use AutoModelForImageTextToText. After loading, assert sum("lora" in n.lower() for n, _ in model.named_parameters()) == 392.

All four were trained from Gyubeum/AndroidFlux_RL_Train_Test @ 4545695c26951a4884e8f18e6468d69cbdf9d174 for 1 epoch with identical hyperparameters — only the reward model and the data split differ. See each subfolder's README.md for full details.

Intended comparison (run elsewhere, on held-out data): base Qwen/Qwen3-VL-2B-Instruct vs. scalar-GRPO vs. discrete-GRPO.

Qwen3-VL-4B adapters (2026-09-24/25)

Four offline-GRPO LoRA adapters for Qwen3-VL-4B-Instruct, trained with the same recipe, reward model (Gyubeum/Qwen3-VL-8B-Instruct-UI-Genie-scoring-128k-balanced, Bradley-Terry scalar + GPRS) and budget (2,400 prompts, 150 steps); only the training-data mix differs. HV65 success rate (65 human-verified AndroidWorld tasks):

subfolder	training data	clean	error_t	mean
(base Qwen3-VL-4B-Instruct, no RL)	—	20.0	21.5	20.8
grpo4b-ui100-20260924	UI-Genie 100%	23.1	16.9	20.0
grpo4b-t100-20260924	AndroidFlux t 100%	18.5	30.8	24.6
grpo4b-ui50-t25-t1-25-20260924	UI 50 / t 25 / t-1 25	29.2	21.5	25.4
grpo4b-ui40-t35-t1-25-20260925	UI 40 / t 35 / t-1 25	27.7	26.2	26.9

Load with subfolder= exactly as above, but with base Qwen/Qwen3-VL-4B-Instruct @ ebb281ec70b05090aa6165b016eac8ec08e71b17, and assert 504 LoRA tensors (not 392). Each subfolder's README.md has the full recipe and data provenance; trainer_state.json has the per-step logs.