no-e / README.md
jwjesse's picture
Update model card
f77c73d verified
|
Raw History Blame Contribute Delete
4.49 kB
---
base_model: Qwen/Qwen3-1.7B
library_name: transformers
license: apache-2.0
language:
- en
tags:
- lipogram
- constrained-generation
- qlora
datasets:
- jwjesse/slm-lipogram
---
# no-e
A small model (Qwen3-1.7B, fine-tuned with QLoRA) that rewrites an English passage **without the letter "e"** (either case), keeping the names, the facts and at least 70% of the original length. Names are copied as written, so an "e" inside a name from the passage is allowed.
It is a research prototype: it cuts the stray e’s a lot but rarely reaches zero. A "stray e" below is an "e" outside names copied exactly from the passage (the rule check masks such names before counting).
## How to use
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "jwjesse/no-e"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype="auto", device_map="auto")
passage = "..." # 50 to 100 words of English
prompt = "Rewrite this passage without using the letter e:\n\n" + passage
text = tok.apply_chat_template([{"role": "user", "content": prompt}], tokenize=False,
add_generation_prompt=True, enable_thinking=False)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=400, do_sample=False) # greedy
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
```
Training and evaluation used exactly that prompt. The [demo](https://huggingface.co/spaces/jwjesse/no-e-demo) sends "Rewrite this passage in English without using the letter e:" instead,
because the added words reduce the occasional non-English output.
## Training
- Data: [jwjesse/slm-lipogram](https://huggingface.co/datasets/jwjesse/slm-lipogram), config `us_open1`: 150 examples. The passages are the openings of US CNN articles and Wikipedia introductions (50 to 100 words);
the rewrites were written by GPT-6.1 Sol and kept only if they passed code checks (no "e" outside names copied exactly from the source, look-alike letters count, ASCII only, real English words, at least 70% of the source length) and a meaning judge (Claude Sonnet 5.5, score 4 or 5).
- Method: 4-bit QLoRA, rank 16, learning rate 2e-4, 10 epochs, seed 0, on one L4 GPU. The adapter is merged into the weights; the repo holds a single merged BF16 `model.safetensors` (1.7B parameters, 3.4 GB), not an adapter.
## Evaluation
150 held-out passages from the same two sources (50 of them adversarial), none in the training data. Greedy decoding, plain prompt.
| Model | Median stray e’s | Mean | Rewrites with only real words |
|---|---|---|---|
| Base Qwen3-1.7B | 38 | 37.8 | 100.0% |
| Previous model (188 BBC examples) | 11 | 11.0 | 84.7% |
| **This model** | **10** | **10.7** | **92.0%** |
- 11 of 150 passages end with 3 or fewer stray e’s; **0%** are fully e-free.
- Data curve, same recipe on the first N training rows (median stray e’s): 19 rows 35, 38 rows 30, 75 rows 14, 150 rows 10. It was still falling at 150, so more data would probably help.
- The previous model (trained on 188 BBC news rewrites) is the comparison point: about the same stray e’s on this set, the previous one has fewer rewrites with only real words (84.7% vs 92.0%), and on 81 of 150 passages this model leaves fewer e’s (58 more, 11 ties).
## Limitations
- It rarely reaches zero e’s. Expect a handful of stray e’s per passage.
- Occasional non-English output: 2 of 150 eval outputs had Chinese characters, one fully Chinese. Saying "in English" in the prompt helps.
- Meaning is not reliably measured. AI judges missed clear meaning failures when checked against hand ratings; the judge score here (3.59 of 5) is only a rough guide. Read the outputs.
- It sometimes dodges the letter with shorthand or invented-looking words rather than a natural rewrite (92% of its rewrites use only real words; 8% contain at least one non-word or e-dodging shorthand).
- Tested on 50 to 100 word news and encyclopedia passages in US English only.
## Links
- Code, evaluation and write-up: https://labs.gauntletai.com/jessewells/SLM
- Demo: https://huggingface.co/spaces/jwjesse/no-e-demo
- Dataset: https://huggingface.co/datasets/jwjesse/slm-lipogram
The weights are marked Apache 2.0, the licence of the Qwen3 base model. The training text includes CC BY-SA Wikipedia passages and CNN article text (© CNN), so check the rights position before any use beyond research; see the dataset card.