JimmmmmL/chord-0.8b-student

CHORD 0.8B Student (e16)

Distilled 0.8B student encoder for CHORD, a coherence-aware corpus-level reference distance for open-ended text generation. This checkpoint mimics the PEOL-coherence representation of the Qwen3.5-27B teacher (layer −3 / block 62) at ~35x fewer parameters, for use as a cheap drop-in featurizer in the CHORD pipeline.

Status: private / unpublished. This checkpoint underlies results in an ICLR 2026 submission that has not yet been decided. Please do not redistribute outside the project team until the paper's status is resolved.

Model details

• Architecture: Qwen3_5Model, hidden_size=1024, dtype float32
• Params: ~0.8B
• Teacher: Qwen3.5-27B, PEOL-coherence probe, layer −3 (block 62)
• Training: LoRA-distilled then merged to a full checkpoint (this repo contains the merged weights, not an adapter)
• Source repo: text-evaluation-metric (private, JimmmmmL), commit 071713b, run id distill16 (2026-07-13)
• model.safetensors sha256: 8b7b72ea893a45a4453c15585c136964202d84d48c8918d8e6cc92f2886de13a

Training data

Corpus = prior distill15 corpus (verbatim) + 768 fresh native-length ELF-L documents (seed 20260714) + 400 packed windows at W=929 (the median native eval length). Total 13,688 documents; battery/eval documents excluded from training. See train_expanded16.jsonl / build_report16.json in the source repo for the exact manifest.

Evaluation (from the source repo's experiment ledger, run distill16)

• Counterfactual battery, confidence-half: 36/40 (teacher: 38/40; full battery incl. selection-half: 10/12 semantic + 28/28 form — matches teacher counts)
• cap532 packed-corpus MMD ladder (the main real-generator diagnostic) correctly ordered for the first time at this scale: human ≪ AR < ELF-L < MDLM < SEDD ≈ LangFlow (previous students e13/e15 had ELF-L inverted below AR)
• This is the checkpoint backing the restored 0.8B column in the paper's main distillation-cost table

Usage

from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("JimmmmmL/chord-0.8b-student")
tokenizer = AutoTokenizer.from_pretrained("JimmmmmL/chord-0.8b-student")

Feature extraction follows the same PEOL-coherence prompting and layer-(-3) pooling convention as the 27B teacher; see the source repo's experiments/real_dlm/distill_peol.py and mauve_attack featurization code for the exact protocol.