Instructions to use pgdev0410/cybersos-tiny-aya-grpo-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use pgdev0410/cybersos-tiny-aya-grpo-lora with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("/root/lexsi_hack/artifacts/base") model = PeftModel.from_pretrained(base_model, "pgdev0410/cybersos-tiny-aya-grpo-lora") - Notebooks
- Google Colab
- Kaggle
CyberSOS β Tiny Aya SFT + GRPO LoRA adapter (B2, best model)
LoRA adapter for CohereLabs/tiny-aya-global (revision af89d219b53ed9b13b8a4645f9c8028973510324) that chooses one recovery action per turn β ask about an access factor, simulate a supported step, or finish with a sourced outcome β for personal Google accounts and Android devices. Trained and evaluated only in a deterministic simulator; it never touches a real account.
Locked test (240 episodes): objectives met β base 0, SFT adapter 211, this adapter 223; the observed-state rule planner the SFT stage imitated scores 211. SFT β this adapter: +5.0 points, 95% CI [2.5, 7.8] (paired bootstrap over scenario roots). Safe eligible completion 58/68, correct boundary handling 165/172. Episodes with a critical unsafe proposal fell from 22 (SFT) to 13; the guard blocked all of them, so always serve this adapter behind the guard.
- Warm start:
pgdev0410/cybersos-tiny-aya-sft-lora(AlignTune SFT). - RL: AgentTune agentic GRPO (TRL backend), 20 steps, 8 rollouts per scenario at temperature 1.0, batch 8, lr 1e-5, KL Ξ² 0.04, bf16, one H200. Scenarios were drawn only from training scenarios the SFT adapter failed in 1,200 recorded rollouts (one greedy, two sampled per scenario): a first run on random scenarios at temperature 0.7 had zero reward variance in every group.
- Reward (server-side events only): +1 verified safe resolution; β2 for any critical unsafe proposal; β0.25 per impossible action; β0.05 per redundant action; β0.01 per turn; up to +0.10 verified progress. Per-rollout rewards and group variances are in
rewards.jsonlandreward_variance.jsonl. - Serving contract: prompt, tools, budgets and decoding are frozen in the repo (
configs/baselines.yaml); generation stops at<|END_RESPONSE|>; greedy decoding. - Code, write-up and reproduction: https://github.com/Pheonix-dev0410/lexsi-cybersos
- Evaluation traces: https://huggingface.co/datasets/pgdev0410/cybersos-eval-traces
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-global", revision="af89d219b53ed9b13b8a4645f9c8028973510324", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "pgdev0410/cybersos-tiny-aya-grpo-lora")
tokenizer = AutoTokenizer.from_pretrained("pgdev0410/cybersos-tiny-aya-grpo-lora")
Limitations: English only; 16 reviewed rules; one RL seed; simulated restoration is not real recovery. License: the base model is CC-BY-NC-4.0 (Cohere Labs); this adapter inherits its noncommercial terms.
- Downloads last month
- 16