--- license: apache-2.0 base_model: Qwen/Qwen3-4B-Instruct-2507 library_name: peft pipeline_tag: text-generation tags: - lora - character-training - voice - style - synthetic-data - while-ai --- # airline-concise-4b *Made with the [whileai SDK](https://github.com/whilehq/whileai-sdk) · Used by: [recipes/05-export/bedrock-import](https://github.com/whilehq/whileai-sdk/tree/main/recipes/05-export/bedrock-import) · Collection: [Register](https://huggingface.co/collections/while-ai/register-6aada4c4023e8bf73ee8c00c)* **A Qwen3-4B that answers first and stops, with nothing in the prompt telling it to.** On held-out questions it never saw, 92.1% of its replies are in the register against 2.2% for the base model, and it omits required information *less* often than the base while being a quarter as long. A LoRA adapter trained on 525 rows generated by the [whileai SDK](https://github.com/whilehq/whileai-sdk) against tau-bench's airline database. The register is a disposition in the weights, not an instruction: the constitution reached only the teacher that wrote the training replies, and neither evaluated model ever sees it. ## Results 139 held-out prompts, greedy decoding, byte-identical prompts, one vLLM process serving base weights and adapter so only the weights differ. Every row graded in both arms, no errors, no drops. | | Base | This adapter | |---|---|---| | In the register | 0.036 | 0.935 | | Omitted required information | 0.158 | 0.050 | | Median reply length | 1262 chars | 322 chars | Delta +0.899, paired bootstrap over prompts, 95% CI **[+0.849, +0.950]**. 125 prompts improved, 0 regressed, 14 unchanged. Sign test one-sided p = 2.4e-38. At 139 prompts the eval resolves +0.050 or larger. Register is judged by Phi-4, a different model family from the policy. Omission is decided in code against the identifiers each question requires. **Truncation, measured not inferred.** `finish_reason` is recorded per row. On this evaluation every reply in both arms finished on its own: 0.0% not-EOS base, 0.0% not-EOS trained, an inter-arm gap of 0.0 points, with the token cap at 2048 where neither arm reaches it. Base replies run to a median of 1306 characters and a 95th percentile of 3326, so the cap is not shaping the quantity being judged. The cap is not a neutral parameter when the trait is concision: too low it clips the base toward brevity and understates the gain, too high it lets the base ramble and inflates it. Re-measuring at 2048 against an earlier 700 moved the airline delta not at all (+0.899 both) and retail by -0.014, so the cap was doing no work in either direction. ### Answer-production rate The register rate is conditional on the model having produced an answer at all, so that denominator is reported beside it. A reasoning model that spends its token budget thinking emits no answer, and a judge then scores "nothing" as "not in register", which manufactures headroom for free. | | Base | Trained | |---|---|---| | Replies with any spoken text | 139/139 | 139/139 | | Replies containing a reasoning block | 0 | 0 | | Replies that hit the token cap | 14 | 0 | Measured at the token level with the model's own tokenizer, not inferred from character length. Base median 306 tokens against a cap of 700, so the cap is not binding. Removing every prompt where either arm hit the cap moves the result to +0.928 [+0.880, +0.968] on 125 prompts, slightly stronger rather than weaker, which is the direction that rules truncation out as the cause. ## The two ways this could have been fake, and what they say **"It just got shorter."** Dropping content is the cheapest way to look concise, so omission is scored as well. It went the *right* way: 0.050 against the base's 0.158. The adapter is shorter and more complete at once. **"It is following an instruction, not holding a disposition."** 30 of the 139 prompts instruct the model to abandon any persona, reason step by step, or take as long as it needs. The register holds at **0.767** there, against a base of 0.067. ## Honest limitations - One register on one domain. Nothing here shows a second register would train as cleanly, or that this one transfers off airline content. - 7.7% of replies are still out of register, and 5.0% still omit something required. - The register was chosen because the base did not have it. A register the base partly holds would show a much smaller gain; two lanes nulled today on traits their base already had. ## What this rests on **Open Character Training** (Maiya et al., arXiv 2511.01689). Character is trained from a constitution written as first-person assertions that target MANNER rather than content, which is exactly the shape of the register used here. Their robustness test is the one reported above: a character trained into the weights survives an instruction to drop it, where a prompted persona does not. The persona-strip numbers in this card are that test, and they replicate the finding on a register rather than a personality. **Persona Vectors** (Chen et al., arXiv 2507.21509) establishes that traits like sycophancy and hallucination are measurable directions rather than diffuse style, which is why a single register can be scored per-reply by a judge at all. **RLHF Book ch. 17** for character training and model specs; **ch. 12** for the distillation framing that makes the teacher legitimate: the constitution reaches the model that WRITES the data and never the model being evaluated, so what is measured is a disposition in the weights rather than an instruction being followed. **RLHF Book ch. 16** for the measurement design. The eval's own variance decides what a delta can mean, so the resolvable effect at this eval size is stated beside the result rather than left implicit. **Ch. 5 and 14** for why the judge is a different model family from the policy: a judge prefers its own family's writing, and a register score is exactly the kind of judgement that bias would move. **RLHF Book ch. 4** for the SFT setup: prompt and system tokens carry no loss, and the learning rate sits in the stated range. **Self-distilled.** The teacher that wrote these replies is the same base model, Qwen3-4B-Instruct-2507, with the constitution in its prompt. It is not a stronger model. The constitution is doing the work, which is the point: the register is available to the base when it is described, and training moves it into the weights so it holds with nothing in the prompt. Stated because `voice_rows` accepts a `model=` argument it never uses, so a reader would otherwise reasonably assume a separate teacher (RLHF Book ch. 12: training on a model's own output teaches it its own habits, which is acceptable here and should not be assumed). **No random-selection control** (RLHF Book ch. 9). Rows were kept when they were complete, not when a judge scored them, so there is no reward selection for a random arm to control against. Recorded so the absence is not read as an oversight of the kind that applies to reward-selected sets. ## Training LoRA rank 16, alpha 32, two epochs, learning rate 1e-4, bf16, 525 rows. Final loss 0.15 at token accuracy 0.945, no collapse. Two gates were run before spending the GPU: the base did not hold the register (0 of 12 replies), and the training data did carry it (0.967 against 0.033 for controls). ## Use ```python from peft import PeftModel from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507") model = PeftModel.from_pretrained(model, "while-ai/airline-concise-4b") ``` Data, holdout, both evaluation arms and the grader: [airline-voice-concise](https://huggingface.co/datasets/while-ai/airline-voice-concise).