README.md

---
library_name: transformers
license: apache-2.0
base_model: vera6/affine-5g4yy75zuz-t6
tags:
  - affine
  - sn120
  - qwen3.5-moe
  - grpo
---

# Affine SN120 candidate p28 (HiAlpha-GRPO on p24 / t6)

Reason-v4 GRPO checkpoint trained in the `/mining/ralph` loop on an 8×H100
PCIe box. Pushed for a live SN120 duel against the sitting king
`vera6/affine-5g4yy75zuz-t6` @ `8e3f1695e058837ed80fec3238ff439fdc2d0f0e`
(reign 36). Architecture is stock `Qwen3_5MoeForConditionalGeneration`
(hidden 2048, 40 layers, 256 experts / 8 active). No custom modeling code,
no `auto_map`, no `*.py`.

## Lineage

1. Live king `vera6/affine-5g4yy75zuz-t6` @ `8e3f1695e058837ed80fec3238ff439fdc2d0f0e`
2. **p24** — HiAlpha-GRPO on that king (`cand_p24-grpo-king-affine-5g4yy75zu`).
   Local v4 screen vs t6: n=400, margin **−0.00009**, z **−0.09**, med |z| 139,
   B-pass 0.46 (tie / BELOW_BAR).
3. **p28** — same recipe on p24 (this repo). Merged LoRA → bf16 safetensors
   via `/mining/sim/merge_lora2.py` (nonzero delta + 333 visual tensors).

Experiment path: `/mining/ralph/runs/p28-grpo-p24-grpo-king-affine-5g4`

## Method

Teacher-anchored **Reason v4** GRPO (`train_reason_grpo.py`).

Per-sample reward:

```
a_i    = lpC(y_i | z_A) − lpC(y_i | ∅)     # k=3 teacher refs
Reason = τ · log((1/k) · Σ exp(a_i/τ))     # τ=0.03
reward = Reason + length_shape(|z|)        # penalize |z|≥220 only
```

Winner-only tail-boost 2.0 on the best group member. Ranked quantity is the
thought `z` (action `y` is not the score). Teacher is the frozen Affine
teacher `zai-org/GLM-4.5-Air-FP8`, two local vLLM TP=2 endpoints.

## Data

- Pool: `/mining/ralph/data/grpo.jsonl`
- Kept 871 / 1200 rows (char budget 12288)
- Affine public turn corpus D (SWE-style coding turns), same chat contract
  the validator uses (`THOUGHT` + last closed bash fence)

## Hyperparameters

| knob | value |
|---|---|
| method | HiAlpha-GRPO (LoRA) |
| lr | 5e-6 |
| LoRA r / α / dropout | 16 / 128 / 0.05 |
| target modules | q,k,v,o,gate,up,down `_proj` |
| group size G | 4 |
| steps | 200 |
| max new tokens (z sample) | 64 |
| max seq | 6144 |
| KL coef | 0.0 |
| tail-boost | 2.0 winner-only |
| length shape | penalty on \|z\|≥220, no length bonus |
| τ / k_refs | 0.03 / 3 |
| dtype | bfloat16 |

Train wall: 4524 s (~75 min). Last-20 mean reward 0.044. Trainable 8.36M /
34.7B (0.024%). GPUs 4–7 for LoRA (`device_map=auto`); teachers on 0–1 and 2–3.

## Local sim vs live king (Reason v4, k=3, τ=0.03)

Screened 2026-08-19 against the **same** king SHA still sitting at push time
(`vera6/affine-5g4yy75zuz-t6` @ `8e3f1695…`). `gate_screen.py` / `fast_screen`
n=160:

| | cand | king |
|---|---|---|
| mean Reason | 0.01184 | 0.00995 |
| med \|z\| | 144 | 145 |
| B-pass | 0.46 | 0.49 |

- margin **+0.00189** · SE **0.00149** · z **+1.27** · n=160
- win_frac 0.40 · median Δ 0 · p90 Δ +0.00943 · top-5% contrib +0.00288
- gates: thought length **PASS** (≥80), causality B **PASS** (≥0.30)
- live crown bar = max(2·SE, δ=0.002) ≈ **0.0030** on this slice
- local verdict: **BELOW_BAR** (n=160 is a rejector vs live n_turns=1300)

p28 was the closest this-loop candidate vs t6. Live duel is n=1300 k=3;
this n=160 slice does **not** prove a crown.

## Serving

Same family the eval pod loads: Qwen3.5-MoE, canonical sharded safetensors
(`model-00001-of-00002` + `model-00002-of-00002` + `model.safetensors.index.json`
+ `model-visual.safetensors`). No `--trust-remote-code`. Local screen served
this merge with vLLM TP=2.

## Hardware

8× NVIDIA H100 80GB PCIe. Merge on CPU (`device_map=cpu`).