atharvgaur18/actionproof-grpo-tiny-v2-pcstr-d-s0

ActionProof GRPO arm pcstr_d, seed 0

LoRA adapter (r = 16) trained with GRPO on top of the warm start atharvgaur18/actionproof-rft-tiny-v2 merged into CohereLabs/tiny-aya-global.

• Reward: pcstr_d, goal term read from the state: G + 0.2P + 0.1V − 0.5I − 0.3ND − 0.1F_under − 0.2D.
• Training: 200 steps × 4 prompts × 8 rollouts, lr 2e-5, β 0.02, T = 1, seed 0; every arm in a family shares warm start, tasks, optimiser and budget (see run_config.json, log_history.json).

Greedy evaluation on test_iid (n = 200): verified-success completion (VSC) 0.850. Source: results/eval/r3/grpo_tiny_v2_pcstr_d_s0__neutral__test_iid.summary.json in the code repository.

Load

from transformers import AutoModelForCausalLM
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-global", torch_dtype="bfloat16")
base = PeftModel.from_pretrained(base, "atharvgaur18/actionproof-rft-tiny-v2").merge_and_unload()  # warm start
model = PeftModel.from_pretrained(base, "atharvgaur18/actionproof-grpo-tiny-v2-pcstr-d-s0")

The environment, verifier, prompts and evaluation code are in the code repository. Episodes are scored from the database by a deterministic verifier; acknowledgements can be false by design.

License: CC-BY-NC-4.0, derived from Cohere's Tiny Aya Global (Cohere Labs). Research use only.