ling30-m4-r16e4 — LoRA for inclusionAI/Ling-3.0-tiny

A LoRA adapter that tunes Ling-3.0-tiny (a 7.9B-param MoE with 128 routed experts and hybrid linear attention, bailing_hybrid architecture) for short-form literary realist fiction — character-driven prose vignettes in the vein of mid-century American short stories.

This is the clean-data retrain of the earlier m3 adapter (ling30-m3-r16clean, not published) (ling30-m3-r16clean): identical recipe, but trained on a corpus whose "uncanny" stylistic tics (machines/p objects written as if alive, un-nameable sensations, inverted personification) were machine-scrubbed out before training.

• r16 / α32, dropout 0.05, attention projections only (q_a_proj, q_b_proj, kv_a_proj_with_mqa, kv_b_proj, q/k/v/o_proj) — 21 MB of bf16 deltas
• 4 epochs run, checkpoint 186 (= end of epoch 3) shipped, lr 1e-4, cosine schedule, bf16, micro-batch 1 × grad-accum 8, eager attention, peft==0.20.0, transformers==4.57.6
• Checkpoint 186 was selected by a paired per-epoch generation battery (uncanny-tic census + zlib repetition + duplicate-loop scan over 24 fixed prompts, greedy and sampled) — not by eval loss. It dominated every axis: greedy tics 1.08/1kw and sampled 1.78/1kw (human reference corpus: 1.41; the dirty-data m3 model: 3.2–7.9), zero duplicate-loop sentences, zlib in the clean band.

Training data

~500 short genre vignettes, published as the dataset baiango/genre500 (494 train + 6 val docs; the scrubbed files this run trained on are included in its source/ folder). The corpus is synthetic: prose distilled from frontier teacher models, machine-adjudicated, then scrubbed for surreal-uncanny stylistic tics before this training run. No books, no scraped fiction.

Usage

The base model uses remote code (trust_remote_code=True) — its bailing_hybrid architecture is not in stock transformers.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "inclusionAI/Ling-3.0-tiny",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, "<this-repo>")

tok = AutoTokenizer.from_pretrained("inclusionAI/Ling-3.0-tiny", trust_remote_code=True)

prompt = "Write a scene: a ferry conductor on the last crossing of the night."
ids = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=400, do_sample=True, temperature=1.0, top_p=0.95)
print(tok.decode(out[0], skip_special_tokens=True))

To bake the adapter in permanently:

merged = model.merge_and_unload()
merged.save_pretrained("ling30-m4-merged")

merge gotcha (transformers ≥4.53): the base ships a generation_config.json with top_p/top_k under do_sample=False; save_pretrained rejects it. Unset them before saving: merged.generation_config.top_p = None; merged.generation_config.top_k = None. Also re-set config.model_type = "bailing_hybrid" and torch_dtype after saving — 4.57 can rename torch_dtype → dtype.

A pre-quantized UD-Q4_K_M GGUF (q8_0 embeddings/output, imatrix-calibrated on in-domain prose) built from the merged weights is published at baiango/ling30-m4r16.

Notes

• Trained on a single 24 GB GPU (RTX 3090); the micro-batch-1 + eager-attention recipe is load-bearing for this architecture (batched token-label shuffling pollutes loss).
• The per-epoch battery showed the clean corpus removes the epoch-4 repetition cliff seen in the dirty-data run; epochs are roughly free on scrubbed data (sampled census flat 1.8–2.3 across epochs).
• Trained via HF Trainer; trainer_state.json (loss curves) is included for provenance.