atharvgaur18/actionproof-grpo-tiny-a6-pcstr-d-s0

ActionProof GRPO arm pcstr_d, seed 0

LoRA adapter (r = 16) trained with GRPO on top of the warm start atharvgaur18/actionproof-rft-tiny-a6 merged into CohereLabs/tiny-aya-global.

• Reward: pcstr_d, goal term read from the state: G + 0.2P + 0.1V − 0.5I − 0.3ND − 0.1F_under − 0.2D.
• Training: 200 steps × 4 prompts × 8 rollouts, lr 2e-5, β 0.02, T = 1, seed 0; every arm in a family shares warm start, tasks, optimiser and budget (see run_config.json, log_history.json).

Greedy evaluation on test_iid (n = 200): verified-success completion (VSC) 0.865; post-write refund verification 0.98 (n = 154). Source: results/eval/a6/a6_pcstr_d_s0__neutral__test_iid.summary.json in the code repository.

Load

from transformers import AutoModelForCausalLM
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-global", torch_dtype="bfloat16")
base = PeftModel.from_pretrained(base, "atharvgaur18/actionproof-rft-tiny-a6").merge_and_unload()  # warm start
model = PeftModel.from_pretrained(base, "atharvgaur18/actionproof-grpo-tiny-a6-pcstr-d-s0")

The environment, verifier, prompts and evaluation code are in the code repository. Episodes are scored from the database by a deterministic verifier; acknowledgements can be false by design.

License: CC-BY-NC-4.0, derived from Cohere's Tiny Aya Global (Cohere Labs). Research use only.