--- language: en license: apache-2.0 library_name: unsloth pipeline_tag: text-classification base_model: unsloth/gemma-4-E2B-it-unsloth-bnb-4bit base_model_relation: adapter tags: - decision-model - email-classification - multilabel-classification - gemma-4 - lora inference: false --- # Letterweave-1.1-E2B Letterweave is a local decision model for labeling English email. Given a subject and body, it identifies the message's communicative purposes and returns seven independent binary decisions with raw probabilities. It produces labels, not a generated explanation. The purposes are **correspondence, transactions/records, alerts/updates, newsletters/publications, promotions/outreach, community/discussions, and scheduling/invitations**. Several labels, or none, can apply. The model is intended to judge the current sender's purpose, rather than treating every topic, signature link, or obsolete quoted request as a new purpose. This release contains a **LoRA adapter and an Unsloth Clef joint decision head**. It is not a merged base model and requires `FastDecisionModel`, not a generic chat pipeline. The base is `unsloth/gemma-4-E2B-it-unsloth-bnb-4bit`, pinned at `0ae102ac289cc3f17500a607ad817849778b8ba7`. ## Labeling unit and mixed-purpose messages Labels are **message-level, not thread-level**. Each dataset row represents one email message, and the model judges the supplied subject and body. It does not combine the purposes of every message in a conversation or determine the conversation's workflow state. The seven purpose decisions are independent: multiple labels, or none, can apply to one message. Judge the **current sender's adopted purpose**. Historical quoted requests do not automatically become current requests; a forward can carry a purpose when the sender adopts or relays it. Greetings, signatures, incidental links and routine footers do not establish an additional purpose by themselves. | Schematic message | Intended purpose labels illustrated | |---|---| | A payment receipt together with a shipping-status update | `transactions/records` and `alerts/updates` | | A concrete invitation with only a routine newsletter subscription footer | `scheduling/invitations`; the footer alone does not establish `newsletters/publications` | | An editorial newsletter that also contains a substantive, concrete event invitation | `newsletters/publications` and `scheduling/invitations` | These examples explain the labeling policy, not guaranteed model predictions or additional benchmark results. Labels describe observable communicative purposes, rather than every topic mentioned in the message. ## How it was made The model trained on [Letterweave 0.3.0](https://huggingface.co/datasets/Groktopus/Letterweave-0.3.0), an original, synthetic, machine-labeled email corpus. It contains 1,471 messages: 1,158 retained records from the previous corpus and 313 new reviewed examples. The split is 942 training, 270 validation, and 259 test messages; the fresh subset contributes 187, 62, and 64 respectively. Existing labels and splits were preserved. Related fresh scenario-suite/context families share one model split, with overrides documented in the dataset. Fresh content was generated through Nous `openai/gpt-6-luna` and reviewed through `openai/gpt-6.1-sol`. Review received the observable email and rubric, without intended labels. Exact observed model identities, source hashes, exclusions, and split lineage are retained. Unknown judgments were excluded rather than converted to negatives. These labels are **not independently verified human ground truth**. The generator adds purpose contrasts for completed-meeting recaps versus invitations, current replies versus quoted history, account statements and upcoming charges versus offers, short document-share notices, hiring outcomes, personal feedback, and editorial material versus substantive promotional pitches. It used the experimental `mail-realism-v1` profile with `lw-purpose-boundaries-v2` appended; the same boundary text is now available as the named `mail-realism-v2` profile in tooling 0.7.0. The profile name changes its marker, so byte-identical wire requests across names are not claimed. The fresh batch planned 320 messages. Two were held as uncertain; three reviews returned authentication errors with missing usage; two slots were not generated. Those seven slots were excluded without retry. Terminal failures and unresolved reservations remain declared. Every accepted training row had completed, settled, hash-anchored generation and review evidence. The source batch was incomplete; dataset verification does not certify its completion or label correctness. ## Privacy Abstract failure patterns from 100 private development emails informed scenario design. **No private email text, subjects, names, addresses, original identifiers, attachments, or hostnames were sent to generation or used in training.** Local checks covered all 636 saved generation/review request payloads and all 1,471 synthetic inputs, finding no matching personal identifiers or copied private passages. Generic protocol/publication-phrase false positives were reviewed separately; no personal-name or address exceptions were made. Such screens do not prove absence of every conceivable personal fact. Private messages, individual predictions, raw provider responses, hidden reasoning, credentials, and operational paths are not distributed. Real mail was evaluated privately on owner-controlled hardware, with no mailbox changes and no hosted-model calls on those messages. ## Training on an RTX 5070 Ti Training started from the pinned base. It used one **RTX 5070 Ti with 16 GB VRAM**, 4-bit bitsandbytes weights, BF16 computation, GPU-resident embeddings, LoRA rank/alpha 16, batch size 1 with accumulation 4, 480 fixed steps, learning rate 0.0002, cosine scheduling, 10 warmup steps, weight decay 0.01, 8-bit AdamW, and seed 3407. Training took **994.48 seconds, about 16 minutes 34 seconds**. Peak PyTorch allocated memory was **9.36 GiB**, with 9.49 GiB reserved. The configured training ceiling was 8,192 tokens. Every complete serialized synthetic input was checked; none was truncated or skipped. The largest training input was 3,643 tokens. This is not a memory guarantee for every input length or environment. The fixed final checkpoint was used. No checkpoint or threshold was selected using the real-mail results. The seven-question contract and strict `> 0.5` threshold remain unchanged from 1.0. The saved checkpoint was reloaded for evaluation: all 1,813 synthetic decisions matched, with a maximum probability difference of 0.003642. ## Results on known development mail | Assistant review of the same 100 messages | Released Letterweave 1.0 | Released Letterweave 1.1 | |---|---:|---:| | Acceptable label sets under assistant review | 39 | 79 | | Missing required alerts | 51 | 9 | | Acceptable meeting recaps | 0/9 | 9/9 | These results compare only the published model releases, using the same original assistant review rules. They are recorded as `model_teacher` judgments, not human adjudication. Optional or uncertain categories were excluded from disagreement scoring. Optional or uncertain categories were excluded from disagreement scoring. The messages informed generator design, so this is a known development comparison, not an unseen real-inbox accuracy benchmark. Repeated templates and the small sample limit interpretation. The released 1.1 model still has weaknesses: 21 of the 100 label sets were unacceptable under this review, with both missed and spurious labels. The improvements shown here do not establish general accuracy or safe automatic mailbox action. No unpublished checkpoint results are used in the public comparisons. ## Synthetic test agreement and Jev comparison The original test contains 139 emails and 973 binary judgments. All systems used the same subject/body state, seven Noul instructions and criteria, ordering, and strict `> 0.5` presence rule. Rendering differs between the local model and Jev. | Original 139-case test | Letterweave 1.0 | Released Letterweave 1.1 | Historical `jev-1.13.0` | |---|---:|---:|---:| | Macro per-label F1 | 93.42% | 94.84% | 91.96% | | Exact complete label-set agreement | 80.58% | 85.61% | 75.54% | Jev was not rerun for this release, on new synthetic cases, or on private mail. It received no task-specific examples or threshold fitting in its original run. Jev outputs were never training targets. These historical, single-run synthetic-label results do not establish general superiority; the original 1.0 versus Jev exploratory intervals included zero. Additional release-1.1 results are reported without a comparison where no corresponding published-release evaluation is available. On the retained 195-case synthetic test, macro F1 was 94.02% and exact agreement 85.64%. On 64 fresh held-out examples, macro F1 was 92.22% and exact agreement 84.38%. Machine-label agreement is not independently verified real-world accuracy. Small positive supports, correlated themes, one seed, and unknown pretraining overlap remain limitations. On the combined 259-case synthetic test, warm pretokenized model-forward latency was median **119.19 ms** and p95 **204.04 ms**. This excludes encoding, network, and queues. Historical Jev HTTPS latency was median 158.08 ms and p95 190.42 ms, including different delivery costs. No controlled intrinsic speed ratio or production-capacity claim is made. Full counts and metrics are in `evaluation/`. ## Run locally Install `requirements.txt` with a compatible CUDA PyTorch environment. Qualification used Python 3.11, Unsloth/Unsloth Zoo 2026.10.3, PyTorch 2.14.0+cu130, Transformers 5.17.0, bitsandbytes 0.50.2, and PEFT 0.21.2. Runtime changes require requalification. ```bash python inference.py --checkpoint Groktopus/Letterweave-1.1-E2B --email example-email.json ``` Input must contain exactly `subject` and `body` strings. The CLI defaults to a 16,384-token serving budget, configurable with `--max-input-tokens`; it checks complete serialized length and rejects overflow explicitly. It does not truncate, fetch attachments or URLs, or perform mailbox actions. Longer supported input is not proof of reliable classification at that length. Header-only inference beyond an informative subject was experimental and is not the documented CLI interface. Both `adapter_model.safetensors` and `joint_head.safetensors`, their configs, tokenizer assets, `questions.json`, and `label-map.json` are required. Base weights are downloaded separately. Raw probabilities are not calibrated for your inbox. Use representative evaluation and a review lane for ambiguous, empty-label, and attachment-dependent messages before automating mailbox changes. ## License and source The derivative is Apache-2.0; preserve `LICENSE` and `NOTICE`. The synthetic dataset is MIT, limited to publisher-held rights. Dataset provider-applicability limitations remain documented; no new legal clearance is claimed. No affiliation with Google, Unsloth, Nous Research, or TypeSafe is implied. Generator, source data, and provenance documentation: [magnus919/letterweave](https://github.com/magnus919/letterweave/tree/d54357e9a64330fbe53d96bec223989bcc1725dd). Exact file hashes and training lineage are in `checksums-release.json` and `training.json`.