Instructions to use FatUnc6969/tiny-aya-borderland-genA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use FatUnc6969/tiny-aya-borderland-genA with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-global") model = PeftModel.from_pretrained(base_model, "FatUnc6969/tiny-aya-borderland-genA") - Notebooks
- Google Colab
- Kaggle
Tiny Aya Global + Borderland A (verified derivations)
A adds checked derivations or provenance reasons to the agent-task targets. GSM8K and Aya rows are identical in A and B.
LoRA (r=32, alpha=64, attention+MLP) on CohereLabs/tiny-aya-global, trained from base with Lexsi AgentTune distillation on 4,003 rows (Borderland 900 + DAgger 121, procedural notes-across-a-wipe tasks 1,272, tool calling 1,060, GSM8K 500, Aya multilingual 150), 2 epochs, lr 1e-4. The A and B adapters use identical rows and hyper-parameters; B's agent-task targets are A's with the reasoning removed; GSM8K and Aya targets are unchanged.
The Borderland chain test uses a teacher-played prefix through room 3, ending immediately before notebook-writing. A then writes its own notebook, is wiped, and opens 58/64 locks (90.6%). This is not an autonomous full-game score. Separate room-4 interventions use empty notes (25%) or the twin world's teacher-written notes (100% follows-twin), not swapped model-written notes. Known weaknesses: over-calls tools on irrelevant requests (BFCL irrelevance 21%), shorter maths reasoning than base.
Held-out results (greedy, single seed, 95% bootstrap CI):
| Base | A (verified derivations) | B (plain traces) | |
|---|---|---|---|
| Own notebook -> memory wipe -> open lock after teacher-played prefix (64 worlds, chance 25%) | - | 90.6% | 21.9% |
| Borderland room success (16x7) | 0% | 38.4% | 6.2% |
| Note-taking across a wipe, trained task types | 2.1% | 62.5% | 52.1% |
| BFCL v3 AST (400) | 44.5% | 54.5% | 67.0% |
| GSM8K (500) | 74.8% | 57.6% | 60.0% |
| Global-MMLU-Lite, 8 languages (200) | 50.5% | 54.5% | 51.0% |
Write-up, data, code and every per-item log: https://huggingface.co/datasets/FatUnc6969/lexi-borderland-data ยท https://github.com/VasuBansal7576/lexi-borderland
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-global", torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "FatUnc6969/tiny-aya-borderland-genA")
tok = AutoTokenizer.from_pretrained("FatUnc6969/tiny-aya-borderland-genA")
- Downloads last month
- 23