atharvgaur18's picture
ActionProof adapter and model card
aad54c3 verified
|
Raw History Blame Contribute Delete
1.49 kB
---
license: cc-by-nc-4.0
base_model: CohereLabs/tiny-aya-global
library_name: peft
tags: [lora, grpo, tool-use, agents, reinforcement-learning, actionproof]
---
# ActionProof warm start `rft_tiny_v2`
LoRA adapter (r = 16) on [CohereLabs/tiny-aya-global](https://huggingface.co/CohereLabs/tiny-aya-global). v2 warm start: rejection-sampling fine-tuning on Aya Expanse 32B trajectories plus expert refund demonstrations (half of them verify the refund).
GRPO arms in the ActionProof collection were trained as a fresh LoRA on this adapter **merged** into the base.
Design and registration: `PREREGISTRATION_A4.md` in the [code repository](https://github.com/atharvgaur1845/actionproof-v2).
Greedy evaluation on `test_iid` (n = 200): verified-success completion (VSC) **0.555**. Source: `results/eval/r3/rft_tiny_v2__neutral__test_iid.summary.json` in the code repository.
## Load
```python
from transformers import AutoModelForCausalLM
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-global", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "atharvgaur18/actionproof-rft-tiny-v2")
```
The environment, verifier, prompts and evaluation code are in the [code repository](https://github.com/atharvgaur1845/actionproof-v2). Episodes are scored from
the database by a deterministic verifier; acknowledgements can be false by design.
License: CC-BY-NC-4.0, derived from Cohere's Tiny Aya Global (Cohere Labs). Research use only.