README.md

---
library_name: peft
tags:
- peft
- lora
- qwen3.5
- gemma
- swe-qa
- software-engineering
---

# Evolve LoRA adapters

This private repository stores the 24 final LoRA adapter directories used in
the Evolve layered SWE-QA experiments. It contains Qwen3.5-9B,
Qwen3.5-35B-A3B, and Gemma-4-26B-A4B adapters for Direct SFT and v4 layered
SFT at H2, H4, H5, and H6.

## Layout

```text
adapters/
├── qwen35-9b/
│   ├── direct/{h2,h4,h5,h6}/
│   └── v4/{h2,h4,h5,h6}/
├── qwen35-35b-a3b/
│   ├── direct/{h2,h4,h5,h6}/
│   └── v4/{h2,h4,h5,h6}/
└── gemma4-26b-a4b/
    ├── direct/{h2,h4,h5,h6}/
    └── v4/{h2,h4,h5,h6}/
```

Each level directory is the complete training `final_adapter` directory. In
addition to the loadable top-level `adapter_model.safetensors` and
`adapter_config.json`, retained intermediate checkpoint subdirectories are
included for reproducibility.

| Base model | Direct | v4 | Total |
|---|---:|---:|---:|
| Qwen3.5-9B | 4 adapters, ~12 GB | 4 adapters, ~12 GB | 8 adapters, ~24 GB |
| Qwen3.5-35B-A3B | 4 adapters, ~6.3 GB | 4 adapters, ~6.3 GB | 8 adapters, ~12.6 GB |
| Gemma-4-26B-A4B | 4 adapters, ~11.2 GB | 4 adapters, ~11.2 GB | 8 adapters, ~22.4 GB |

## Loading an adapter

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-9B")
model = PeftModel.from_pretrained(
    base,
    "Yingxuan/evolve",
    subfolder="adapters/qwen35-9b/v4/h6",
)
```

The adapter configuration records PEFT 0.19.1, LoRA rank 64, alpha 128, and
dropout 0.05. Use the matching Qwen3.5 or Gemma base model for each adapter
family.

Training code, processed data, evaluation predictions, metrics, and complete
evaluation trajectories are maintained at
[zoe-yyx/evolve](https://github.com/zoe-yyx/evolve).