README.md

---
license: gemma
base_model: google/gemma-3-1b-it
datasets:
  - while-ai/ecommerce-intent
library_name: peft
pipeline_tag: text-classification
tags:
  - intent-classification
  - ecommerce
  - lora
  - qlora
  - function-calling
language:
  - en
---

# Model Card for ecommerce-1b

<!-- while-ai: where this fits -->
*Made with the [whileai SDK](https://github.com/whilehq/whileai-sdk) · Collection: [Ecommerce Intent Detection](https://huggingface.co/collections/while-ai/ecommerce-intent-detection-6aa80c55172c4862c0d09b20)*


**Release:** v1.2

**Frontier-approaching e-commerce intent detection in a 1B model.** Given a live customer conversation, `ecommerce-1b` returns a single JSON object with the customer's payment intent and its structured details. It runs inline on live traffic, at a cost and latency where calling a frontier model is not an option.

On a hard, held-out benchmark it reaches **75.3% macro-averaged intent-type accuracy**. On the 412-row frontier comparison subset it reaches **71.4%, 89% of GPT-5's 80.2%**, at **$0.18 per 1M output tokens** and about a second per response.

## Model Details

### Model Description

- **Developed by:** While
- **Model type:** E-commerce payment-intent classifier; structured JSON output over seven intent types
- **Language:** English
- **License:** Gemma (inherited from the base model)
- **Finetuned from:** `google/gemma-3-1b-it`, via 4-bit QLoRA

### Model Sources

- **Repository:** https://huggingface.co/while-ai/ecommerce-1b
- **Training data:** https://huggingface.co/datasets/while-ai/ecommerce-intent

## Uses

### Direct Use

Drop-in intent detection for e-commerce and payments support. Feed the conversation so far; the model returns one JSON object per message: whether an actionable payment intent is present, which of seven types it is, and the extracted details. Cheap and fast enough to call on every turn of every conversation.

### Out-of-Scope Use

English-language e-commerce and payments only. It is a narrow intent classifier, not a general assistant, and should not be the sole authority for executing a payment without a downstream verification step.

## Bias, Risks, and Limitations

Trained and evaluated on synthetic role-play conversations from a multi-agent marketplace simulation, labeled in a separate model pass under a locked labeling policy; labels are model-assigned and carry residual noise, and live-traffic distribution may differ. Accuracy is currently strongest on refunds, exchanges, and transfers; checkout and bill-pay are the focus of the current data round.

### Recommendations

Pair the model with a verification layer before any payment executes, and re-benchmark on your own traffic before relying on the reported numbers in production.

## How to Get Started with the Model

```python
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model = AutoPeftModelForCausalLM.from_pretrained("while-ai/ecommerce-1b")
tok   = AutoTokenizer.from_pretrained("while-ai/ecommerce-1b")
```

Prompt it with the system prompt shipped in `prompt_template.txt` plus the conversation so far, and decode greedily (`temperature=0`).

### Output format

The model returns one JSON object per message:

```json
{"intent_detected": true, "core_type": "reverse",
 "details": {"action": "refund", "merchant": "Amazon", "amount": 81.40, "reason_code": "scam"},
 "confidence": 0.85, "reason": "Customer wants a refund on a suspicious charge.",
 "source_message_seqs": [2]}
```

`core_type` is one of `spend`, `send`, `exchange`, `recur`, `bill`, `reverse`, or `none`, and each type carries its own `details` fields.

## Training Details

### Training Data

The differentiator is the data. While builds e-commerce intent training data as **randomized conversational simulations**: role-played customers with independently sampled personas, tones, financial situations, life stages, devices, and behaviors, including adversarial actors, simulated turn by turn between two independently drawn models. Generation is label-blind (the generating models never see the intent schema), labels are assigned in a separate pass, and every candidate passes a structural data gate: deduplicated by message signature, with zero train/eval leakage.

This release was fine-tuned on **17,144 conversations**. The held-out evaluation set contains **1,977 conversations** and has no conversation or identifier overlap with the training split.

### Training Procedure

#### Training Hyperparameters

- **Method:** 4-bit QLoRA (via [unsloth](https://github.com/unslothai/unsloth))
- **Epochs:** 1
- **Effective batch size:** 16
- **Learning rate:** 2e-4
- **Sequence length:** 12288 (long conversations fit)

## Evaluation

### Testing Data, Factors & Metrics

Held-out eval of 1,977 conversations, zero train/eval leakage, macro-averaged (equal weight per intent), scored deterministically against gold: intent type by exact match, order details as the fraction of gold fields reproduced (partial credit per field). The frontier panel is scored on a 412-row balanced subset of the same eval.

### Results

![While e-commerce intent evaluation: accuracy, per-action accuracy, and cost to serve](1b-v1.png)

- Fine-tuning takes intent-type accuracy from **14.4% to 75.3%** on the public six-action rubric, macro-averaged across the 1,977-row held-out evaluation.
- Intent detection reaches **80.7%**, and structured order-detail extraction reaches **66.3%**, both macro-averaged on the same public rubric.
- Action-level intent-type accuracy is **71.8% for bill payment** and **78.5% for recurring payments**.
- On the no-action boundary, **9.5%** of true no-action conversations are marked actionable.

## Technical Specifications

### Model Architecture and Objective

A LoRA adapter over `gemma-3-1b-it`, trained response-only to emit one structured intent object per turn.

### Compute Infrastructure

Served as an OpenAI-compatible endpoint (base + adapter) under vLLM. Measured on a Modal L4 GPU: **$0.18 per 1M output tokens** at batched capacity and **~1s per response** (p50 984 ms, p95 1,191 ms), 55 to 140x below frontier list prices.

## Model Card Contact

While, https://huggingface.co/while-ai