Affine SN120 candidate p28 (HiAlpha-GRPO on p24 / t6)

Reason-v4 GRPO checkpoint trained in the /mining/ralph loop on an 8ร—H100 PCIe box. Pushed for a live SN120 duel against the sitting king vera6/affine-5g4yy75zuz-t6 @ 8e3f1695e058837ed80fec3238ff439fdc2d0f0e (reign 36). Architecture is stock Qwen3_5MoeForConditionalGeneration (hidden 2048, 40 layers, 256 experts / 8 active). No custom modeling code, no auto_map, no *.py.

Lineage

  1. Live king vera6/affine-5g4yy75zuz-t6 @ 8e3f1695e058837ed80fec3238ff439fdc2d0f0e
  2. p24 โ€” HiAlpha-GRPO on that king (cand_p24-grpo-king-affine-5g4yy75zu). Local v4 screen vs t6: n=400, margin โˆ’0.00009, z โˆ’0.09, med |z| 139, B-pass 0.46 (tie / BELOW_BAR).
  3. p28 โ€” same recipe on p24 (this repo). Merged LoRA โ†’ bf16 safetensors via /mining/sim/merge_lora2.py (nonzero delta + 333 visual tensors).

Experiment path: /mining/ralph/runs/p28-grpo-p24-grpo-king-affine-5g4

Method

Teacher-anchored Reason v4 GRPO (train_reason_grpo.py).

Per-sample reward:

a_i    = lpC(y_i | z_A) โˆ’ lpC(y_i | โˆ…)     # k=3 teacher refs
Reason = ฯ„ ยท log((1/k) ยท ฮฃ exp(a_i/ฯ„))     # ฯ„=0.03
reward = Reason + length_shape(|z|)        # penalize |z|โ‰ฅ220 only

Winner-only tail-boost 2.0 on the best group member. Ranked quantity is the thought z (action y is not the score). Teacher is the frozen Affine teacher zai-org/GLM-4.5-Air-FP8, two local vLLM TP=2 endpoints.

Data

  • Pool: /mining/ralph/data/grpo.jsonl
  • Kept 871 / 1200 rows (char budget 12288)
  • Affine public turn corpus D (SWE-style coding turns), same chat contract the validator uses (THOUGHT + last closed bash fence)

Hyperparameters

knob value
method HiAlpha-GRPO (LoRA)
lr 5e-6
LoRA r / ฮฑ / dropout 16 / 128 / 0.05
target modules q,k,v,o,gate,up,down _proj
group size G 4
steps 200
max new tokens (z sample) 64
max seq 6144
KL coef 0.0
tail-boost 2.0 winner-only
length shape penalty on |z|โ‰ฅ220, no length bonus
ฯ„ / k_refs 0.03 / 3
dtype bfloat16

Train wall: 4524 s (~75 min). Last-20 mean reward 0.044. Trainable 8.36M / 34.7B (0.024%). GPUs 4โ€“7 for LoRA (device_map=auto); teachers on 0โ€“1 and 2โ€“3.

Local sim vs live king (Reason v4, k=3, ฯ„=0.03)

Screened 2026-08-19 against the same king SHA still sitting at push time (vera6/affine-5g4yy75zuz-t6 @ 8e3f1695โ€ฆ). gate_screen.py / fast_screen n=160:

cand king
mean Reason 0.01184 0.00995
med |z| 144 145
B-pass 0.46 0.49
  • margin +0.00189 ยท SE 0.00149 ยท z +1.27 ยท n=160
  • win_frac 0.40 ยท median ฮ” 0 ยท p90 ฮ” +0.00943 ยท top-5% contrib +0.00288
  • gates: thought length PASS (โ‰ฅ80), causality B PASS (โ‰ฅ0.30)
  • live crown bar = max(2ยทSE, ฮด=0.002) โ‰ˆ 0.0030 on this slice
  • local verdict: BELOW_BAR (n=160 is a rejector vs live n_turns=1300)

p28 was the closest this-loop candidate vs t6. Live duel is n=1300 k=3; this n=160 slice does not prove a crown.

Serving

Same family the eval pod loads: Qwen3.5-MoE, canonical sharded safetensors (model-00001-of-00002 + model-00002-of-00002 + model.safetensors.index.json

  • model-visual.safetensors). No --trust-remote-code. Local screen served this merge with vLLM TP=2.

Hardware

8ร— NVIDIA H100 80GB PCIe. Merge on CPU (device_map=cpu).

Downloads last month
13
Safetensors
Model size
35B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for elevateecho/sn120-9fbb78acd964

Finetuned
(35)
this model