notes-slots

Compact token-classification model that extracts scheduling/task slots from short English notes — participants, datetimes, priorities, and recurrences — and runs fully client-side via Transformers.js.

The shipped artifact is an INT8-quantized ONNX bundle (~7.6 MB) intended for in-browser WASM inference, not a PyTorch checkpoint.

Model details

Base model	microsoft/xtremedistil-l6-h256-uncased (MIT)
Architecture	BertForTokenClassification — 6 layers, hidden size 256, 8 heads, intermediate 1024, vocab 9567 (pruned from 30522), max positions 512 (~7.6M params)
Task	Token classification (BIO slot tagging)
Schema version	slot-labels-v0.3.0
Model version	0.2.0
Languages	English
Runtime	Transformers.js v4, WASM device, dtype q8
Bundle size	7.56 MB
transformers (training)	4.57.6
License	CC BY 4.0

Labels (9, BIO)

O, B-PARTICIPANT, I-PARTICIPANT, B-PRIORITY, I-PRIORITY, B-DATETIME, I-DATETIME, B-RECURRENCE, I-RECURRENCE

A bundled transitions.json carries a BIO-constraint transition matrix (0 for allowed transitions, a large negative penalty for invalid I- continuations) for constrained Viterbi decoding on top of the raw token logits. The bundled JS loader applies it automatically. The matrix is deliberately constraint-only: empirical bigram priors double-count label frequency already present in the emissions and were measured to cost ~7 F1 points of recall on the quantized model.

Intended use

• In scope: extracting participants / datetimes / priorities / recurrence cues from short, informal English notes and reminders (calendar, to-do, email intent style text).
• Out of scope: long documents, languages other than English, normalization of extracted spans into structured datetimes (use a downstream parser such as chrono-node for that), and any high-stakes decisioning.

Usage (Transformers.js)

import { pipeline } from "@huggingface/transformers";

const tagger = await pipeline("token-classification", "jottypro/notes-slots", {
  dtype: "q8",
});

const out = await tagger("call Sarah next Friday at 5pm, high priority, every week");
console.log(out);

The ONNX weights live at onnx/model_quantized.onnx, which is the layout Transformers.js expects when loading from the Hub.

Training

• Data: AmazonScience/MASSIVE en-US (config en-US, revision d2362678…), filtered to the calendar / datetime / email / lists scenarios with MASSIVE slots remapped onto the local 4-slot schema (e.g. person/relation/email_address → PARTICIPANT, date/time/time_zone → DATETIME, general_frequency → RECURRENCE), combined with synthetic productivity and realistic note generators.
• Augmentation: light, training-split only (AUGMENT_FACTOR=2) — random filler-word prefix, trailing punctuation, occasional O-token dropout; deduplicated against originals.
• Hyperparameters: 10 epochs with early stopping (patience 2, restore best by F1), batch size 64, learning rate 5e-5, cosine schedule, warmup ratio 0.1, weight decay 0.01, label smoothing 0.05, max sequence length 128, seed 42.
• Distillation: the shipped student is knowledge-distilled from a bert-base-uncased teacher fine-tuned on the same data (teacher eval F1 ~0.90); loss = 0.5·CE(gold) + 0.5·T²·KL(teacher‖student), T=2.
• Vocab pruning: wordpiece vocab pruned 30522 → 9567 (pieces observed in domain data, fixtures, regression texts, plus a wikitext-103 frequency sweep; all single-char and digit pieces retained). Pruned tokenizer emits token-identical output on the full test set, so metrics are unchanged.
• Quantization: dynamic, per-channel QInt8 with reduce_range, applied to MatMul and Gather ops via ONNX Runtime. Static QDQ quantization was evaluated (MinMax/Entropy/Percentile calibration) and consistently lost 8-12 F1 points on this model size; dynamic + reduce_range won the matrix. Weight-only QAT (straight-through fake-quant fine-tune) was also evaluated and rejected: it landed below the post-training baseline (0.823 vs 0.828) because it cannot model ORT's runtime dynamic activation quantization.

Evaluation

Token-level metrics (seqeval) on the held-out test split (n ≈ 559). fp32 is argmax-decoded; q8 uses the constrained-Viterbi decode that ships in the JS loader. The q8 column reflects the artifact actually shipped in this repo.

Metric	fp32	q8 (shipped)
Accuracy	0.9545	0.9444
Precision	0.8235	0.7925
Recall	0.8777	0.8662
F1	0.8498	0.8278
DATETIME F1	0.8175	0.7925
PARTICIPANT F1	0.9355	0.9331
PRIORITY F1	0.7526	0.6735
RECURRENCE F1	0.8649	0.8651

Limitations and bias

• Quantization cost: with dynamic per-channel INT8 (reduce_range) and constrained-Viterbi decoding the gap to fp32 is ~2 F1 points (0.850 → 0.828). Decoding with plain argmax instead of the bundled transitions costs a further ~2 points; empirical-prior Viterbi costs ~7. PRIORITY is the most quantization-sensitive entity (0.75 → 0.67).
• Domain: trained on short calendar/task-style English notes plus synthetic data; expect degradation on long-form text, other domains, or other languages.
• Synthetic data: part of the training distribution is generated, so phrasing diversity and demographic coverage of names/relations is limited and may carry generator biases.
• No span normalization: the model tags spans only; converting a DATETIME span to an actual timestamp is a downstream concern.

License and attribution

Released under CC BY 4.0, consistent with the MASSIVE training data (CC BY 4.0). The base model microsoft/xtremedistil-l6-h256-uncased is MIT.

Part of the training data is derived from the MASSIVE dataset; CC BY 4.0 requires attribution to that source:

@misc{fitzgerald2022massive,
  title  = {MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages},
  author = {FitzGerald, Jack and others},
  year   = {2022},
  eprint = {2204.08582},
  archivePrefix = {arXiv}
}