--- license: gemma base_model: google/gemma-3-1b-it datasets: - while-ai/ecommerce-intent library_name: peft pipeline_tag: text-classification tags: - intent-classification - ecommerce - lora - qlora - function-calling language: - en --- # Model Card for ecommerce-1b *Made with the [whileai SDK](https://github.com/whilehq/whileai-sdk) ยท Collection: [Ecommerce Intent Detection](https://huggingface.co/collections/while-ai/ecommerce-intent-detection-6aa80c55172c4862c0d09b20)* **Release:** v1.2 **Frontier-approaching e-commerce intent detection in a 1B model.** Given a live customer conversation, `ecommerce-1b` returns a single JSON object with the customer's payment intent and its structured details. It runs inline on live traffic, at a cost and latency where calling a frontier model is not an option. On a hard, held-out benchmark it reaches **75.3% macro-averaged intent-type accuracy**. On the 412-row frontier comparison subset it reaches **71.4%, 89% of GPT-5's 80.2%**, at **$0.18 per 1M output tokens** and about a second per response. ## Model Details ### Model Description - **Developed by:** While - **Model type:** E-commerce payment-intent classifier; structured JSON output over seven intent types - **Language:** English - **License:** Gemma (inherited from the base model) - **Finetuned from:** `google/gemma-3-1b-it`, via 4-bit QLoRA ### Model Sources - **Repository:** https://huggingface.co/while-ai/ecommerce-1b - **Training data:** https://huggingface.co/datasets/while-ai/ecommerce-intent ## Uses ### Direct Use Drop-in intent detection for e-commerce and payments support. Feed the conversation so far; the model returns one JSON object per message: whether an actionable payment intent is present, which of seven types it is, and the extracted details. Cheap and fast enough to call on every turn of every conversation. ### Out-of-Scope Use English-language e-commerce and payments only. It is a narrow intent classifier, not a general assistant, and should not be the sole authority for executing a payment without a downstream verification step. ## Bias, Risks, and Limitations Trained and evaluated on synthetic role-play conversations from a multi-agent marketplace simulation, labeled in a separate model pass under a locked labeling policy; labels are model-assigned and carry residual noise, and live-traffic distribution may differ. Accuracy is currently strongest on refunds, exchanges, and transfers; checkout and bill-pay are the focus of the current data round. ### Recommendations Pair the model with a verification layer before any payment executes, and re-benchmark on your own traffic before relying on the reported numbers in production. ## How to Get Started with the Model ```python from peft import AutoPeftModelForCausalLM from transformers import AutoTokenizer model = AutoPeftModelForCausalLM.from_pretrained("while-ai/ecommerce-1b") tok = AutoTokenizer.from_pretrained("while-ai/ecommerce-1b") ``` Prompt it with the system prompt shipped in `prompt_template.txt` plus the conversation so far, and decode greedily (`temperature=0`). ### Output format The model returns one JSON object per message: ```json {"intent_detected": true, "core_type": "reverse", "details": {"action": "refund", "merchant": "Amazon", "amount": 81.40, "reason_code": "scam"}, "confidence": 0.85, "reason": "Customer wants a refund on a suspicious charge.", "source_message_seqs": [2]} ``` `core_type` is one of `spend`, `send`, `exchange`, `recur`, `bill`, `reverse`, or `none`, and each type carries its own `details` fields. ## Training Details ### Training Data The differentiator is the data. While builds e-commerce intent training data as **randomized conversational simulations**: role-played customers with independently sampled personas, tones, financial situations, life stages, devices, and behaviors, including adversarial actors, simulated turn by turn between two independently drawn models. Generation is label-blind (the generating models never see the intent schema), labels are assigned in a separate pass, and every candidate passes a structural data gate: deduplicated by message signature, with zero train/eval leakage. This release was fine-tuned on **17,144 conversations**. The held-out evaluation set contains **1,977 conversations** and has no conversation or identifier overlap with the training split. ### Training Procedure #### Training Hyperparameters - **Method:** 4-bit QLoRA (via [unsloth](https://github.com/unslothai/unsloth)) - **Epochs:** 1 - **Effective batch size:** 16 - **Learning rate:** 2e-4 - **Sequence length:** 12288 (long conversations fit) ## Evaluation ### Testing Data, Factors & Metrics Held-out eval of 1,977 conversations, zero train/eval leakage, macro-averaged (equal weight per intent), scored deterministically against gold: intent type by exact match, order details as the fraction of gold fields reproduced (partial credit per field). The frontier panel is scored on a 412-row balanced subset of the same eval. ### Results ![While e-commerce intent evaluation: accuracy, per-action accuracy, and cost to serve](1b-v1.png) - Fine-tuning takes intent-type accuracy from **14.4% to 75.3%** on the public six-action rubric, macro-averaged across the 1,977-row held-out evaluation. - Intent detection reaches **80.7%**, and structured order-detail extraction reaches **66.3%**, both macro-averaged on the same public rubric. - Action-level intent-type accuracy is **71.8% for bill payment** and **78.5% for recurring payments**. - On the no-action boundary, **9.5%** of true no-action conversations are marked actionable. ## Technical Specifications ### Model Architecture and Objective A LoRA adapter over `gemma-3-1b-it`, trained response-only to emit one structured intent object per turn. ### Compute Infrastructure Served as an OpenAI-compatible endpoint (base + adapter) under vLLM. Measured on a Modal L4 GPU: **$0.18 per 1M output tokens** at batched capacity and **~1s per response** (p50 984 ms, p95 1,191 ms), 55 to 140x below frontier list prices. ## Model Card Contact While, https://huggingface.co/while-ai