spoken-point-minilm

A sentence-similarity model that tells whether a spoken sentence makes a given point. It is all-MiniLM-L12-v2, fine-tuned on short written points paired with real and synthetic speech.

General-purpose embedding models score topic overlap. To them, "We had slow support tickets with the vendor" looks about as close to "Delay came from the vendor API change" as a true paraphrase does. This model is trained to separate the two: a sentence that makes the point scores high, and a sentence on the same topic that makes a different claim scores low. It keeps the base model's size (33M parameters, 34 MB as int8 ONNX) and speed, so it runs in a browser.

Use it

In the browser with Transformers.js (int8, 34 MB):

import { pipeline } from '@huggingface/transformers';
const embed = await pipeline('feature-extraction', 'struan-assembly/spoken-point-minilm', { dtype: 'q8' });
const [a, b] = await Promise.all(['Data migration is finished', 'All the old records have been moved over and verified']
  .map(t => embed(t, { pooling: 'mean', normalize: true })));
const similarity = a.data.reduce((s, v, i) => s + v * b.data[i], 0);

In Python with sentence-transformers:

from sentence_transformers import SentenceTransformer
model = SentenceTransformer('struan-assembly/spoken-point-minilm')
a, b = model.encode(['Data migration is finished', 'All the old records have been moved over and verified'], normalize_embeddings=True)
similarity = a @ b

Training

• Data: 849 points, each paired with spoken sentences that make it and near misses that don't (same topic but a different claim, a negation, a different number, or a vague mention).
  • 597 synthetic points across business, technical, teaching and everyday speech.
  • 252 points from real meeting speech in the AMI Meeting Corpus (CC BY 4.0). The points are annotators' decisions, actions and problems. The positives are the utterances they linked to each one, and the near misses are utterances from the same meeting linked to a different item.
• Objective: MultipleNegativesRankingLoss with near misses as hard negatives, 8 epochs.

Evaluation

On 84 held-out points (252 sentences that make them and 168 near misses), compared with the base model:

	all-MiniLM-L12-v2	spoken-point-minilm
Ranks a true paraphrase above a near miss	51.1%	86.1%
Paraphrases scoring 0.48 or higher	65.5%	74.2%
Near misses scoring 0.48 or higher	66.7%	17.3%

The base model ranks paraphrases above near misses at about chance level. The fine-tuned model separates them well, though not perfectly.

Limitations

• English only. Tuned for short spoken points, not documents or search queries.
• Scores run lower than the base model's, so thresholds should be set for this model and not carried over.
• Hard cases remain: vague statements can still score close to vague points ("we stay on the new timeline" vs "Budget is still on track"), and the same topic with a different outcome can still score high ("I reproduced the crash" vs "Fixed the crash").
• It scores sentences one at a time. It can't use what is said later, so "we discussed adding an alert" scores as making the point even if the speaker goes on to reject it.

License and credits

Apache 2.0, the same as the base model, all-MiniLM-L12-v2 by the sentence-transformers team. Training uses the AMI Meeting Corpus (CC BY 4.0, University of Edinburgh and the AMI Consortium). Fine-tuned by Struan Assembly, LLC.