Text Classification
Safetensors
PEFT
English
unsloth
multi-label-classification
email-classification
decision-model
lora
4-bit precision
Instructions to use Groktopus/Letterweave-1.0-E2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Groktopus/Letterweave-1.0-E2B with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Unsloth Desktop
|
Download README.md from Groktopus/Letterweave-1.0-E2B: direct link, hf CLI and curl.
- Browser
- Download file 14.3 kB
-
https://huggingface.co/Groktopus/Letterweave-1.0-E2B/resolve/main/README.md
- Command line
-
hf download hf://Groktopus/Letterweave-1.0-E2B/README.md
-
curl -L -o README.md https://huggingface.co/Groktopus/Letterweave-1.0-E2B/resolve/main/README.md
14.3 kB
| license: apache-2.0 | |
| language: | |
| - en | |
| library_name: unsloth | |
| pipeline_tag: text-classification | |
| base_model: unsloth/gemma-4-E2B-it-unsloth-bnb-4bit | |
| base_model_relation: adapter | |
| datasets: | |
| - Groktopus/Letterweave-0.1.0 | |
| tags: | |
| - multi-label-classification | |
| - email-classification | |
| - decision-model | |
| - lora | |
| - peft | |
| - 4-bit | |
| inference: false | |
| # Letterweave-1.0-E2B | |
| Letterweave-1.0-E2B is an English email-purpose decision model published by **Groktopus**. Given an email subject and body, it returns seven independent presence/absence judgments. Multiple purposes may apply to the same email; an empty label set is also valid. | |
| This release contains a **LoRA adapter and a learned Clef joint decision head** for Google Gemma 4 E2B, trained through Unsloth. It requires the pinned base model below; it is not a complete merged language model or a general chat assistant. Load it with Unsloth's `FastDecisionModel`, rather than a generic text-generation pipeline. Standard Hub inference-provider compatibility, GGUF conversion, CPU inference, and other backends have not been qualified. | |
| **Version:** 1.0. **License:** Apache 2.0. **Training data:** [Letterweave-0.1.0](https://huggingface.co/datasets/Groktopus/Letterweave-0.1.0), a separately MIT-licensed synthetic email corpus. The released model uses raw checkpoint probabilities; no deployment acceptance threshold is calibrated for real mail. | |
| ## Labeling unit and mixed-purpose messages | |
| Labels are **message-level, not thread-level**. Each dataset row represents one email message, and the model judges the supplied subject and body. It does not combine the purposes of every message in a conversation or determine the conversation's workflow state. | |
| The seven purpose decisions are independent: multiple labels, or none, can apply to one message. Judge the **current sender's adopted purpose**. Historical quoted requests do not automatically become current requests; a forward can carry a purpose when the sender adopts or relays it. Greetings, signatures, incidental links and routine footers do not establish an additional purpose by themselves. | |
| | Schematic message | Intended purpose labels illustrated | | |
| |---|---| | |
| | A payment receipt together with a shipping-status update | `transactions/records` and `alerts/updates` | | |
| | A concrete invitation with only a routine newsletter subscription footer | `scheduling/invitations`; the footer alone does not establish `newsletters/publications` | | |
| | An editorial newsletter that also contains a substantive, concrete event invitation | `newsletters/publications` and `scheduling/invitations` | | |
| These examples explain the labeling policy, not guaranteed model predictions or additional benchmark results. Labels describe observable communicative purposes, rather than every topic mentioned in the message. | |
| ## The seven purposes | |
| | Label | What it describes | | |
| |---|---| | |
| | `correspondence` | Substantive interpersonal exchange, beyond greetings, signatures or routine mechanics. | | |
| | `transactions/records` | A discrete exchange, account record or commitment; an offer is not a receipt. | | |
| | `alerts/updates` | An event-triggered operational change, including completed-payment confirmation. | | |
| | `newsletters/publications` | Recurring or curated editorial publication, rather than an account notice alone. | | |
| | `promotions/outreach` | Substantive persuasive outreach, pitch or solicitation. | | |
| | `community/discussions` | Group discussion or participation; bulk distribution alone is insufficient. | | |
| | `scheduling/invitations` | A current concrete invitation, scheduling, rescheduling or cancellation purpose. | | |
| For example, a completed-payment confirmation may be both `transactions/records` and `alerts/updates`. A group discussion can also contain substantive `correspondence`. Labels describe the current sender's adopted purpose: obsolete quoted invitations should not automatically become current invitations. Missing context is not evidence that a purpose is absent. | |
| The exact versioned questions are in [questions.json](questions.json), with ID-to-label mapping in [label-map.json](label-map.json) and the source rubric in [taxonomy.json](taxonomy.json). Use those questions to reproduce the measured contract. Question wording, candidate descriptions and grouping are part of the model's evaluated interface. | |
| ## Intended use and limits | |
| Use this model for experimentation with email categorization, local purpose tagging, and private shadow evaluation. It judges the supplied **subject/body**, not a complete mailbox or conversational workflow. It does not determine whether a message requires action, whether someone owes a reply, or whether a thread should be archived. It sends no email and changes no mailbox labels. | |
| Emails are untrusted input, not instructions to execute. Keep side effects, authorization, overrides and fallback behavior in application code. The model can add an incorrect label, miss a purpose or assign unjustified probability. Do not use this pilot as a sole basis for consequential automated decisions. | |
| English synthetic messages are the measured domain. Real inboxes, other languages, attachments, long threads, privacy-sensitive content and adversarial messages have not been independently qualified. Although the adapter accepts custom questions, performance on other questions is not established. | |
| ## How it was made | |
| The model starts from `unsloth/gemma-4-E2B-it-unsloth-bnb-4bit` at revision **`0ae102ac289cc3f17500a607ad817849778b8ba7`**, derived from `google/gemma-4-E2B-it`. A fresh deterministically initialized decision head and rank-16 LoRA were trained on Letterweave's seven binary purpose judgments. No earlier task-specific checkpoint was reused. Base weights are dynamic 4-bit BitsAndBytes quantized; computation uses BF16. The added head and adapter have their own saved tensor precision. | |
| Training used **573 emails**, with **143 validation emails** and **139 test emails**. Existing family-based membership was preserved, with no family crossing splits. The seven labels produce 4,011 training judgments and 973 held-out test judgments. Model state contains only subject/body. Record identifiers, split/family metadata, scenario intent and teacher rationale are not model inputs. | |
| Letterweave is synthetic and **machine-labeled, not human gold**. Its 855 retained records come from 821 families. Source generation was observed under Luna and Sol model aliases, with Sol review evidence for every retained record; raw provider model-version strings are preserved in the dataset. The source generator and immutable release are at [magnus919/letterweave](https://github.com/magnus919/letterweave/tree/0a91613a19bbf6f5591f91b26a23eeb5505bb038). Teacher agreement, schema verification and family separation do not establish real-inbox correctness or eliminate correlated factual themes. | |
| | Training setting | Value | | |
| |---|---| | |
| | Hardware | One NVIDIA GeForce RTX 5070 Ti, 16 GB, with another GPU service co-resident | | |
| | Sequence budget | 2,048 tokens; largest complete corpus input 1,613 tokens | | |
| | Optimizer | 8-bit AdamW | | |
| | LoRA rank / alpha / dropout | 16 / 16 / 0 | | |
| | Batch size / gradient accumulation | 1 / 4 | | |
| | Steps / seed | 240 / 3407 | | |
| | Encoder / head learning rate | 0.0002 / 0.0001 | | |
| | Warmup / schedule / weight decay | 10 steps / cosine / 0.01 | | |
| | Checkpoint selection | Final checkpoint only; no early stopping or test-driven selection | | |
| | Trainer runtime | **455.36 seconds — approximately 7m 35s**, excluding setup and evaluation | | |
| | Peak process GPU allocation / reservation | **8.65 / 8.79 GiB**, excluding other processes | | |
| | Residency | All model parameters on CUDA, including 768 MiB input embeddings | | |
| All records and all seven questions passed input qualification without skipping or truncation. An initial attempt ended before scoring after a host GPU/driver failure; the completed attempt restarted from the untouched base with the same recipe. It produced the final checkpoint reported here. | |
| A scalar temperature was fitted on validation for separate probability analysis. It was **not folded into this released raw checkpoint**, and it does not change binary argmax decisions. The default example treats a probability strictly above 0.5 as present, with an exact tie absent. That is an evaluation rule, not a calibrated automatic-label policy for an inbox. | |
| ## RTX 5070 Ti benchmark and Jev comparison | |
| Both systems evaluated the same **139 test emails / 973 binary judgments**, with identical subject/body state, seven Noul instructions, true/false criteria, ordering and a strict `> 0.5` presence rule. Jev used pinned **`jev-1.13.0`**, with no task-specific examples, prompt tuning or threshold fitting. Native input rendering differs. Jev outputs were used only for evaluation, never as training targets. | |
| **These scores measure agreement with synthetic Letterweave labels, not independently adjudicated inbox accuracy.** | |
| | Measure | Letterweave-1.0-E2B | Jev 1.13 | | |
| |---|---:|---:| | |
| | Macro per-label F1 | **93.42%** | 91.96% | | |
| | Micro precision | **90.25%** | 87.35% | | |
| | Micro recall | 97.71% | **98.17%** | | |
| | Micro F1 | **93.83%** | 92.44% | | |
| | Exact complete label-set agreement | **112/139 — 80.58%** | 105/139 — 75.54% | | |
| | Binary teacher agreement | **945/973 — 97.12%** | 938/973 — 96.40% | | |
| | False additions / missed labels | **23 / 5** | 31 / **4** | | |
| | Observed median latency | **117.44 ms** | 158.08 ms | | |
| | Observed p95 latency | **122.18 ms** | 190.42 ms | | |
| **Timing boundaries differ.** Local timing covers one warm, pretokenized seven-question call, including collation, device transfer, model forward and result transfer; it excludes tokenization, HTTP/network and service queues. The first email is excluded, leaving 138 observations. Jev timing covers HTTPS through parsed JSON, including TLS/network, for 139 observations. Requests were sequential. These are measured configurations, not a controlled intrinsic model-speed comparison or a production capacity test. | |
| Paired exact-set results: both matched on 92 emails; only Letterweave matched on 20; only Jev matched on 13; neither matched on 14. An exploratory 2,000-draw bootstrap over 133 test families estimated Letterweave-minus-Jev differences: | |
| - Macro F1: **+1.46 percentage points**, 95% interval **−2.46 to +5.68**. | |
| - Exact-set agreement: **+5.04 percentage points**, 95% interval **−2.90 to +13.67**. | |
| Both intervals include zero. The observed quality difference is inconclusive at this sample size; the results do not establish general superiority over Jev. One seed, selected synthetic scenarios, small label supports, residual theme dependence and unknown pretraining overlap limit interpretation. | |
| A fixed train-only TF-IDF logistic baseline achieved 74.72% macro F1 and 59.71% exact-set agreement. See [evaluation/results.json](evaluation/results.json) for all per-label results and paired measurements. Letterweave's largest error concentration was `alerts/updates`: 10 false additions and one missed label; newsletter recall was 9/11. | |
| ## Load and classify | |
| The tested environment used Python 3.11, Unsloth/Unsloth Zoo 2026.10.3, PyTorch 2.14.0+cu130, Transformers 5.17.0, BitsAndBytes 0.50.2 and PEFT 0.21.2. [requirements.txt](requirements.txt) records the core tested package versions. Install a CUDA-compatible PyTorch build and Unsloth according to their platform instructions; the requirement list is not a universal installer or a CPU-support claim. | |
| Download this repository locally; then use its complete checkpoint and trusted questions: | |
| ```python | |
| import json | |
| import unsloth # import before transformers | |
| import torch | |
| from pathlib import Path | |
| from huggingface_hub import snapshot_download | |
| from unsloth import FastDecisionModel | |
| checkpoint = Path(snapshot_download("Groktopus/Letterweave-1.0-E2B")) | |
| model, tokenizer = FastDecisionModel.from_pretrained( | |
| str(checkpoint), | |
| max_seq_length=2048, | |
| load_in_4bit=True, | |
| dtype=torch.bfloat16, | |
| offload_embedding=False, | |
| ) | |
| FastDecisionModel.for_inference(model) | |
| questions = json.loads((checkpoint / "questions.json").read_text()) | |
| label_map = json.loads((checkpoint / "label-map.json").read_text()) | |
| state = { | |
| "subject": "Payment receipt", | |
| "body": "Your payment is complete. This receipt confirms the purchase.", | |
| } | |
| # Production callers should enforce the evaluated input budget; | |
| # see inference.py for a complete token-budget check. | |
| answers = FastDecisionModel.predict(model, tokenizer, state, questions) | |
| probabilities = {label: answers[qid]["noul"] for label, qid in label_map.items()} | |
| labels = [label for label, p in probabilities.items() if p > 0.5] | |
| print({"labels": labels, "probabilities": probabilities}) | |
| ``` | |
| The loader downloads the pinned base weights when they are not already cached. [inference.py](inference.py) supplies a subject/body JSON CLI, validates the complete input budget before inference, and returns all seven raw probabilities. The repository's smoke test checks that this exact interface runs; the receipt example is illustrative, not a new quality benchmark. Reworded questions and longer inputs require new evaluation. No automatic mailbox integration is included. | |
| ## Export verification, licensing and attribution | |
| Reloading the saved checkpoint preserved **all 973 binary decisions** and the exact-set score. Raw probabilities differed by up to **0.002258** (about 0.23 percentage points); 11 judgments exceeded the strict 0.001 tolerance, so strict probability identity did not pass. Decisions and probability parity are reported separately in [training.json](training.json). | |
| Google Gemma 4 and this model release are Apache 2.0 licensed. Retain [LICENSE](LICENSE), [NOTICE](NOTICE) and applicable upstream attribution when redistributing. This is an independent Groktopus modification and is not endorsed by Google or Unsloth. Training data has its own MIT license; source provenance and its limitations are documented in the linked dataset card. | |
| Before enabling real-email labeling, evaluate a prediction-blind, independently labeled inbox sample under this seven-purpose rubric, measure false additions and missed labels, and test a private shadow adapter with conservative fallbacks. The public pilot does not establish production calibration, privacy compliance or safe automatic actions. | |