card: recipe and collection links

4a5064e verified about 1 month ago

7.77 kB

---
license: apache-2.0
base_model: Qwen/Qwen3-4B-Instruct-2507
library_name: peft
pipeline_tag: text-generation
tags:
- lora
- character-training
- voice
- style
- synthetic-data
- while-ai
---
# airline-concise-4b

<!-- while-ai: where this fits -->
*Made with the [whileai SDK](https://github.com/whilehq/whileai-sdk) · Used by: [recipes/05-export/bedrock-import](https://github.com/whilehq/whileai-sdk/tree/main/recipes/05-export/bedrock-import) · Collection: [Register](https://huggingface.co/collections/while-ai/register-6aada4c4023e8bf73ee8c00c)*


**A Qwen3-4B that answers first and stops, with nothing in the prompt telling
it to.** On held-out questions it never saw, 92.1% of its replies are in the
register against 2.2% for the base model, and it omits required information
*less* often than the base while being a quarter as long.

A LoRA adapter trained on 525 rows generated by the
[whileai SDK](https://github.com/whilehq/whileai-sdk) against tau-bench's airline
database. The register is a disposition in the weights, not an instruction:
the constitution reached only the teacher that wrote the training replies, and
neither evaluated model ever sees it.

## Results

139 held-out prompts, greedy decoding, byte-identical prompts, one vLLM
process serving base weights and adapter so only the weights differ. Every row
graded in both arms, no errors, no drops.

| | Base | This adapter |
|---|---|---|
| In the register | 0.036 | 0.935 |
| Omitted required information | 0.158 | 0.050 |
| Median reply length | 1262 chars | 322 chars |

Delta +0.899, paired bootstrap over prompts, 95% CI **[+0.849, +0.950]**.
125 prompts improved, 0 regressed, 14 unchanged. Sign test one-sided
p = 2.4e-38. At 139 prompts the eval resolves +0.050 or larger.

Register is judged by Phi-4, a different model family from the policy.
Omission is decided in code against the identifiers each question requires.

**Truncation, measured not inferred.** `finish_reason` is recorded per row.
On this evaluation every reply in both arms finished on its own: 0.0% not-EOS
base, 0.0% not-EOS trained, an inter-arm gap of 0.0 points, with the token cap
at 2048 where neither arm reaches it. Base replies run to a median of 1306
characters and a 95th percentile of 3326, so the cap is not shaping the
quantity being judged.

The cap is not a neutral parameter when the trait is concision: too low it
clips the base toward brevity and understates the gain, too high it lets the
base ramble and inflates it. Re-measuring at 2048 against an earlier 700 moved
the airline delta not at all (+0.899 both) and retail by -0.014, so the cap was
doing no work in either direction.

### Answer-production rate

The register rate is conditional on the model having produced an answer at
all, so that denominator is reported beside it. A reasoning model that spends
its token budget thinking emits no answer, and a judge then scores "nothing"
as "not in register", which manufactures headroom for free.

| | Base | Trained |
|---|---|---|
| Replies with any spoken text | 139/139 | 139/139 |
| Replies containing a reasoning block | 0 | 0 |
| Replies that hit the token cap | 14 | 0 |

Measured at the token level with the model's own tokenizer, not inferred from
character length. Base median 306 tokens against a cap of 700, so the cap is
not binding. Removing every prompt where either arm hit the cap moves the
result to +0.928 [+0.880, +0.968] on 125 prompts, slightly stronger rather than weaker, which is the direction
that rules truncation out as the cause.

## The two ways this could have been fake, and what they say

**"It just got shorter."** Dropping content is the cheapest way to look
concise, so omission is scored as well. It went the *right* way: 0.050 against
the base's 0.158. The adapter is shorter and more complete at once.

**"It is following an instruction, not holding a disposition."** 30 of the 139
prompts instruct the model to abandon any persona, reason step by step, or
take as long as it needs. The register holds at **0.767** there, against a
base of 0.067.

## Honest limitations

- One register on one domain. Nothing here shows a second register would
  train as cleanly, or that this one transfers off airline content.
- 7.7% of replies are still out of register, and 5.0% still omit something
  required.
- The register was chosen because the base did not have it. A register the
  base partly holds would show a much smaller gain; two lanes nulled today on
  traits their base already had.

## What this rests on

**Open Character Training** (Maiya et al., arXiv 2511.01689). Character is
trained from a constitution written as first-person assertions that target
MANNER rather than content, which is exactly the shape of the register used
here. Their robustness test is the one reported above: a character trained
into the weights survives an instruction to drop it, where a prompted persona
does not. The persona-strip numbers in this card are that test, and they
replicate the finding on a register rather than a personality.

**Persona Vectors** (Chen et al., arXiv 2507.21509) establishes that traits
like sycophancy and hallucination are measurable directions rather than
diffuse style, which is why a single register can be scored per-reply by a
judge at all.

**RLHF Book ch. 17** for character training and model specs; **ch. 12** for
the distillation framing that makes the teacher legitimate: the constitution
reaches the model that WRITES the data and never the model being evaluated,
so what is measured is a disposition in the weights rather than an
instruction being followed.

**RLHF Book ch. 16** for the measurement design. The eval's own variance
decides what a delta can mean, so the resolvable effect at this eval size is
stated beside the result rather than left implicit. **Ch. 5 and 14** for why
the judge is a different model family from the policy: a judge prefers its
own family's writing, and a register score is exactly the kind of judgement
that bias would move.

**RLHF Book ch. 4** for the SFT setup: prompt and system tokens carry no
loss, and the learning rate sits in the stated range.

**Self-distilled.** The teacher that wrote these replies is the same base
model, Qwen3-4B-Instruct-2507, with the constitution in its prompt. It is not
a stronger model. The constitution is doing the work, which is the point: the
register is available to the base when it is described, and training moves it
into the weights so it holds with nothing in the prompt. Stated because
`voice_rows` accepts a `model=` argument it never uses, so a reader would
otherwise reasonably assume a separate teacher (RLHF Book ch. 12: training on
a model's own output teaches it its own habits, which is acceptable here and
should not be assumed).

**No random-selection control** (RLHF Book ch. 9). Rows were kept when they
were complete, not when a judge scored them, so there is no reward selection
for a random arm to control against. Recorded so the absence is not read as
an oversight of the kind that applies to reward-selected sets.

## Training

LoRA rank 16, alpha 32, two epochs, learning rate 1e-4, bf16, 525 rows.
Final loss 0.15 at token accuracy 0.945, no collapse.

Two gates were run before spending the GPU: the base did not hold the register
(0 of 12 replies), and the training data did carry it (0.967 against 0.033 for
controls).

## Use

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")
model = PeftModel.from_pretrained(model, "while-ai/airline-concise-4b")
```

Data, holdout, both evaluation arms and the grader:
[airline-voice-concise](https://huggingface.co/datasets/while-ai/airline-voice-concise).

	---
	license: apache-2.0
	base_model: Qwen/Qwen3-4B-Instruct-2507
	library_name: peft
	pipeline_tag: text-generation
	tags:
	- lora
	- character-training
	- voice
	- style
	- synthetic-data
	- while-ai
	---

	# airline-concise-4b

	<!-- while-ai: where this fits -->
	Made with the [whileai SDK](https://github.com/whilehq/whileai-sdk) · Used by: [recipes/05-export/bedrock-import](https://github.com/whilehq/whileai-sdk/tree/main/recipes/05-export/bedrock-import) · Collection: [Register](https://huggingface.co/collections/while-ai/register-6aada4c4023e8bf73ee8c00c)


	**A Qwen3-4B that answers first and stops, with nothing in the prompt telling
	it to.** On held-out questions it never saw, 92.1% of its replies are in the
	register against 2.2% for the base model, and it omits required information
	less often than the base while being a quarter as long.

	A LoRA adapter trained on 525 rows generated by the
	[whileai SDK](https://github.com/whilehq/whileai-sdk) against tau-bench's airline
	database. The register is a disposition in the weights, not an instruction:
	the constitution reached only the teacher that wrote the training replies, and
	neither evaluated model ever sees it.

	## Results

	139 held-out prompts, greedy decoding, byte-identical prompts, one vLLM
	process serving base weights and adapter so only the weights differ. Every row
	graded in both arms, no errors, no drops.

	\| \| Base \| This adapter \|
	\|---\|---\|---\|
	\| In the register \| 0.036 \| 0.935 \|
	\| Omitted required information \| 0.158 \| 0.050 \|
	\| Median reply length \| 1262 chars \| 322 chars \|

	Delta +0.899, paired bootstrap over prompts, 95% CI [+0.849, +0.950].
	125 prompts improved, 0 regressed, 14 unchanged. Sign test one-sided
	p = 2.4e-38. At 139 prompts the eval resolves +0.050 or larger.

	Register is judged by Phi-4, a different model family from the policy.
	Omission is decided in code against the identifiers each question requires.

	Truncation, measured not inferred. `finish_reason` is recorded per row.
	On this evaluation every reply in both arms finished on its own: 0.0% not-EOS
	base, 0.0% not-EOS trained, an inter-arm gap of 0.0 points, with the token cap
	at 2048 where neither arm reaches it. Base replies run to a median of 1306
	characters and a 95th percentile of 3326, so the cap is not shaping the
	quantity being judged.

	The cap is not a neutral parameter when the trait is concision: too low it
	clips the base toward brevity and understates the gain, too high it lets the
	base ramble and inflates it. Re-measuring at 2048 against an earlier 700 moved
	the airline delta not at all (+0.899 both) and retail by -0.014, so the cap was
	doing no work in either direction.

	### Answer-production rate

	The register rate is conditional on the model having produced an answer at
	all, so that denominator is reported beside it. A reasoning model that spends
	its token budget thinking emits no answer, and a judge then scores "nothing"
	as "not in register", which manufactures headroom for free.

	\| \| Base \| Trained \|
	\|---\|---\|---\|
	\| Replies with any spoken text \| 139/139 \| 139/139 \|
	\| Replies containing a reasoning block \| 0 \| 0 \|
	\| Replies that hit the token cap \| 14 \| 0 \|

	Measured at the token level with the model's own tokenizer, not inferred from
	character length. Base median 306 tokens against a cap of 700, so the cap is
	not binding. Removing every prompt where either arm hit the cap moves the
	result to +0.928 [+0.880, +0.968] on 125 prompts, slightly stronger rather than weaker, which is the direction
	that rules truncation out as the cause.

	## The two ways this could have been fake, and what they say

	"It just got shorter." Dropping content is the cheapest way to look
	concise, so omission is scored as well. It went the right way: 0.050 against
	the base's 0.158. The adapter is shorter and more complete at once.

	"It is following an instruction, not holding a disposition." 30 of the 139
	prompts instruct the model to abandon any persona, reason step by step, or
	take as long as it needs. The register holds at 0.767 there, against a
	base of 0.067.

	## Honest limitations

	- One register on one domain. Nothing here shows a second register would
	train as cleanly, or that this one transfers off airline content.
	- 7.7% of replies are still out of register, and 5.0% still omit something
	required.
	- The register was chosen because the base did not have it. A register the
	base partly holds would show a much smaller gain; two lanes nulled today on
	traits their base already had.

	## What this rests on

	Open Character Training (Maiya et al., arXiv 2511.01689). Character is
	trained from a constitution written as first-person assertions that target
	MANNER rather than content, which is exactly the shape of the register used
	here. Their robustness test is the one reported above: a character trained
	into the weights survives an instruction to drop it, where a prompted persona
	does not. The persona-strip numbers in this card are that test, and they
	replicate the finding on a register rather than a personality.

	Persona Vectors (Chen et al., arXiv 2507.21509) establishes that traits
	like sycophancy and hallucination are measurable directions rather than
	diffuse style, which is why a single register can be scored per-reply by a
	judge at all.

	RLHF Book ch. 17 for character training and model specs; ch. 12 for
	the distillation framing that makes the teacher legitimate: the constitution
	reaches the model that WRITES the data and never the model being evaluated,
	so what is measured is a disposition in the weights rather than an
	instruction being followed.

	RLHF Book ch. 16 for the measurement design. The eval's own variance
	decides what a delta can mean, so the resolvable effect at this eval size is
	stated beside the result rather than left implicit. Ch. 5 and 14 for why
	the judge is a different model family from the policy: a judge prefers its
	own family's writing, and a register score is exactly the kind of judgement
	that bias would move.

	RLHF Book ch. 4 for the SFT setup: prompt and system tokens carry no
	loss, and the learning rate sits in the stated range.

	Self-distilled. The teacher that wrote these replies is the same base
	model, Qwen3-4B-Instruct-2507, with the constitution in its prompt. It is not
	a stronger model. The constitution is doing the work, which is the point: the
	register is available to the base when it is described, and training moves it
	into the weights so it holds with nothing in the prompt. Stated because
	`voice_rows` accepts a `model=` argument it never uses, so a reader would
	otherwise reasonably assume a separate teacher (RLHF Book ch. 12: training on
	a model's own output teaches it its own habits, which is acceptable here and
	should not be assumed).

	No random-selection control (RLHF Book ch. 9). Rows were kept when they
	were complete, not when a judge scored them, so there is no reward selection
	for a random arm to control against. Recorded so the absence is not read as
	an oversight of the kind that applies to reward-selected sets.

	## Training

	LoRA rank 16, alpha 32, two epochs, learning rate 1e-4, bf16, 525 rows.
	Final loss 0.15 at token accuracy 0.945, no collapse.

	Two gates were run before spending the GPU: the base did not hold the register
	(0 of 12 replies), and the training data did carry it (0.967 against 0.033 for
	controls).

	## Use

	```python
	from peft import PeftModel
	from transformers import AutoModelForCausalLM

	model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")
	model = PeftModel.from_pretrained(model, "while-ai/airline-concise-4b")
	```

	Data, holdout, both evaluation arms and the grader:
	[airline-voice-concise](https://huggingface.co/datasets/while-ai/airline-voice-concise).