Planning model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,96 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
+
license: mit
|
| 5 |
+
pipeline_tag: text-classification
|
| 6 |
+
tags:
|
| 7 |
+
- intent-classification
|
| 8 |
+
- multi-label-classification
|
| 9 |
+
- voice-notes
|
| 10 |
+
- asr
|
| 11 |
+
- routing
|
| 12 |
+
- mcp
|
| 13 |
+
- agents
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
# Voice-Note-Intent-Classifier
|
| 17 |
+
|
| 18 |
+
> **Status (2026-10-15): planning.** No weights have been released yet. This card
|
| 19 |
+
> documents the project plan and will be updated at each phase.
|
| 20 |
+
|
| 21 |
+
A small multi-label text classifier that reads a voice-note **transcript** and
|
| 22 |
+
predicts its **intent** — shopping list, to-do, calendar event, email draft, AI
|
| 23 |
+
prompt, and so on — so that an agent system can route it to the right integration.
|
| 24 |
+
|
| 25 |
+
## Use case
|
| 26 |
+
|
| 27 |
+
A voice app records a note and transcribes it. The transcript then needs to become
|
| 28 |
+
an action in another service. This model is the routing step in the middle:
|
| 29 |
+
|
| 30 |
+
```
|
| 31 |
+
voice note ─► ASR ─► transcript ─► [ this classifier ] ─► router
|
| 32 |
+
│ one dispatch per predicted label
|
| 33 |
+
┌──────────────┬──────────────┬───────────────┼──────────────┐
|
| 34 |
+
▼ ▼ ▼ ▼ ▼
|
| 35 |
+
shopping-list to-do calendar email AI prompt …
|
| 36 |
+
subagent subagent subagent subagent subagent
|
| 37 |
+
│ │ │ │ │
|
| 38 |
+
▼ ▼ ▼ ▼ ▼
|
| 39 |
+
list MCP to-do MCP calendar MCP email MCP agent runner
|
| 40 |
+
```
|
| 41 |
+
|
| 42 |
+
Each subagent holds an MCP connection to the target service and extracts the
|
| 43 |
+
details it needs. For example, a note classified `todo` goes to a subagent
|
| 44 |
+
connected to the user's to-do list, which turns it into to-do items. The
|
| 45 |
+
classifier only decides where the note goes; it never performs the action.
|
| 46 |
+
|
| 47 |
+
## Labels (taxonomy v0.1.0, draft)
|
| 48 |
+
|
| 49 |
+
| Label | Meaning |
|
| 50 |
+
|---|---|
|
| 51 |
+
| `shopping_list` | Items to buy |
|
| 52 |
+
| `todo` | Tasks to do, with no fixed time slot |
|
| 53 |
+
| `note_to_self` | Information to remember; nothing to do |
|
| 54 |
+
| `calendar_event` | Something at a specific date/time that occupies a slot |
|
| 55 |
+
| `email_draft` | Text to send as an email, or an instruction to write one |
|
| 56 |
+
| `idea` | A single discrete idea, stated briefly |
|
| 57 |
+
| `brainstorm` | Extended exploratory thinking across options |
|
| 58 |
+
| `content_draft` | Material for publication: blog post, article, script, outline |
|
| 59 |
+
| `ai_prompt` | Instructions addressed to an AI model or agent |
|
| 60 |
+
|
| 61 |
+
Under consideration: `message_draft`, `reminder`, `journal`, `meeting_notes`,
|
| 62 |
+
`research_query`, `technical_spec`, `expense_log`, `contact_info`.
|
| 63 |
+
|
| 64 |
+
The task is **multi-label**: one note can carry several intents ("add bread to the
|
| 65 |
+
list and I've got the dentist Thursday at four" → `shopping_list` +
|
| 66 |
+
`calendar_event`).
|
| 67 |
+
|
| 68 |
+
## Planned approach
|
| 69 |
+
|
| 70 |
+
- **Input:** raw ASR output — no guaranteed punctuation or casing; fillers, false
|
| 71 |
+
starts and recognition errors expected.
|
| 72 |
+
- **Output:** a score per label, labels above per-label calibrated thresholds, and
|
| 73 |
+
an abstain flag when nothing is confident (the router then falls back to an LLM
|
| 74 |
+
or asks the user).
|
| 75 |
+
- **Training data:** synthetic, LLM-generated transcripts styled to resemble real
|
| 76 |
+
ASR output, with compound-intent examples and hard negatives on confusable
|
| 77 |
+
pairs. To be released as a public dataset.
|
| 78 |
+
- **Evaluation:** a privately held set of real, hand-labelled voice notes. Reported
|
| 79 |
+
scores will be on real data, not on synthetic test data.
|
| 80 |
+
- **Baseline to beat:** an LLM zero-shot classifier using the same taxonomy.
|
| 81 |
+
- **Base model candidates:** `answerdotai/ModernBERT-base`,
|
| 82 |
+
`microsoft/deberta-v3-small`, SetFit on `BAAI/bge-small-en-v1.5`.
|
| 83 |
+
- **Release format:** safetensors plus ONNX for CPU and in-browser inference.
|
| 84 |
+
- **Language:** English only in v1.
|
| 85 |
+
|
| 86 |
+
## Roadmap
|
| 87 |
+
|
| 88 |
+
| # | Phase | Status |
|
| 89 |
+
|---|---|---|
|
| 90 |
+
| 0 | Taxonomy definition | in progress |
|
| 91 |
+
| 1 | LLM baseline on real held-out set | — |
|
| 92 |
+
| 2 | Synthetic dataset v1 | — |
|
| 93 |
+
| 3 | Training | — |
|
| 94 |
+
| 4 | Evaluation and threshold calibration | — |
|
| 95 |
+
| 5 | Release (weights, ONNX, eval results) | — |
|
| 96 |
+
| 6 | Integration into a voice-to-action router | — |
|