pecca-core/demo-router-e5-logreg

What it does

Routes a synthetic customer-support email of the fictional "Northbridge Bank" to one of 60 RFI topics. It is the demo model of Pecca, which replaces repetitive LLM calls with small, auditable models. Not for production.

How it was trained

Pecca ran a tournament on the same cross-validated splits and measured macro-F1:

candidate	macro-F1	std	fit s	latency ms
tfidf_linear ★	0.915	0.011	0.9	0.4
e5_logreg	0.909	0.011	9.7	0.2

On this synthetic data tfidf_linear won the tournament by 0.006 macro-F1. This published model is e5_logreg anyway: multilingual-e5 embeddings are more robust on real multilingual data, where templated phrasing is not repeated as it is here.

Training data: pecca-core/demo-support-emails (6,000 rows; labels = human tag where present, else the simulated LLM's answer).

Metrics

	macro-F1
this model (5-fold CV on training labels)	0.909 ± 0.011
simulated "LLM" vs human labels (hold-out)	0.834

(The "LLM" is a noisy label generator with ~15% errors, not a real model. Synthetic data is far easier than real email.)

Threshold & expected fallback

Calibration: temperature on human_labels. Threshold 0.25 (smallest confidence with precision >= 0.95 on hold-out rows). Expected fallback to the LLM: 0% of calls.

How to load with Pecca

import pecca
from huggingface_hub import snapshot_download
model = pecca.load(snapshot_download("pecca-core/demo-router-e5-logreg"))
print(model.predict(["Card lost\n\nI lost my debit card ending ****1234, please block it."]))

The multilingual-e5-base encoder is downloaded from its own repository on first use.

Limitations

Synthetic, code-mixed data (non-English emails keep the topic phrase in English). Latency/metrics are not representative of real workloads. No real customer data was used.