lillycham/qwen3-8b-akyuu

Qwen3-8B akyuu

A persona adapter for Qwen/Qwen3-8B. The persona is Hieda no Akyuu, a human.

It trains 128 chosen MLP neurons (gate, up and down projections), stored as a rank-21 PEFT LoRA over 28 layers. It was trained for 700 steps (learning rate 3e-3, batch 8) on 2233 chat examples, with 242 held out. Qwen3-8B wrote the replies itself, under a character sheet (self-distillation). No outside text is in the data.

The code, the training method and the interpretability tools are in lillycham/qwen3-steering.

Use

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", torch_dtype="auto")
model = PeftModel.from_pretrained(model, "lillycham/qwen3-8b-akyuu")

On an Apple silicon Mac, use the files in mlx/:

mlx_lm.chat --model mlx-community/Qwen3-8B-4bit --adapter-path <this repo>/mlx

Notes

• Adapters trained on the bf16 base. The MLX files target the 4-bit base.
• Reisen and Akyuu are characters from the Touhou Project. This is unofficial fan work. It is not affiliated with or endorsed by ZUN or Team Shanghai Alice.