langarita commited on
Commit
08d6471
·
verified ·
1 Parent(s): 01fee3f

Add int8 question classifier

Browse files
Files changed (6) hide show
  1. .gitattributes +1 -0
  2. README.md +53 -0
  3. labels.json +1 -0
  4. model.onnx +3 -0
  5. tokenizer.json +3 -0
  6. tokenizer_config.json +23 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,53 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
4
+ language: [en, es, fr, de, pt]
5
+ pipeline_tag: text-classification
6
+ library_name: onnx
7
+ tags: [question-detection, meetings, onnx, int8, transformers.js]
8
+ ---
9
+
10
+ # Dictaria question classifier
11
+
12
+ Detects whether the latest chunk of live meeting speech asks something the listener should answer.
13
+ Dictaria runs it on the user's device (browser or desktop app) with onnxruntime-web: no text leaves the device to be classified.
14
+
15
+ ## Labels
16
+
17
+ `question`, `request`, `social`, `rhetorical`, `statement`, `fragment` (in `labels.json` order).
18
+ Dictaria treats `question + request` probability >= 0.5 as "answer this".
19
+
20
+ - `question`: a real question someone else is expected to answer.
21
+ - `request`: asks for information without question form ("walk me through...", "cuéntame...").
22
+ - `social`: comprehension checks, tag questions, small talk, logistics, action requests.
23
+ - `rhetorical`: the speaker answers it or says it for emphasis.
24
+ - `statement`: asks nothing.
25
+ - `fragment`: cut off mid-thought or filler only.
26
+
27
+ ## Model
28
+
29
+ - Base: `paraphrase-multilingual-MiniLM-L12-v2` (Apache 2.0) with a 6-way classification head.
30
+ - Fine-tuned on about 3,820 synthetic meeting-speech examples (90% training, 10% validation) in English, Spanish, French, German and Portuguese, labeled by an AI model.
31
+ - Exported to ONNX and quantized to int8 per channel (`model.onnx`, 118 MB).
32
+ - Input: at most 96 tokens (BOS + the last 94 tokens of the speech + EOS).
33
+
34
+ ## Results
35
+
36
+ 340 independently written test cases, never used for training or tuning, scored through the browser code path with the int8 model:
37
+
38
+ | Language | Accuracy |
39
+ |---|---|
40
+ | English | 97.5% (78/80) |
41
+ | Spanish | 96.3% (77/80) |
42
+ | French | 95.0% (57/60) |
43
+ | German | 96.7% (58/60) |
44
+ | Portuguese | 98.3% (59/60) |
45
+ | All | 96.8% (329/340) |
46
+
47
+ Speed: 12 to 30 ms per check in a browser worker (wasm, one thread).
48
+
49
+ ## Limitations
50
+
51
+ - Judges only the text it is given; it does not know whether a question was already answered.
52
+ - Trained on synthetic speech, not real transcripts; speech-recognition errors in real meetings may lower accuracy.
53
+ - Only the five languages above were tested.
labels.json ADDED
@@ -0,0 +1 @@
 
 
1
+ ["question", "request", "social", "rhetorical", "statement", "fragment"]
model.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:645e73822d8b63025d092bbe851f9b66c67e46314cd55395a920b65003655915
3
+ size 118318872
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3a56def25aa40facc030ea8b0b87f3688e4b3c39eb8b45d5702b3a1300fe2a20
3
+ size 17082734
tokenizer_config.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "bos_token": "<s>",
4
+ "cls_token": "<s>",
5
+ "do_lower_case": true,
6
+ "eos_token": "</s>",
7
+ "is_local": true,
8
+ "mask_token": "<mask>",
9
+ "max_length": 128,
10
+ "model_max_length": 512,
11
+ "pad_to_multiple_of": null,
12
+ "pad_token": "<pad>",
13
+ "pad_token_type_id": 0,
14
+ "padding_side": "right",
15
+ "sep_token": "</s>",
16
+ "stride": 0,
17
+ "strip_accents": null,
18
+ "tokenize_chinese_chars": true,
19
+ "tokenizer_class": "TokenizersBackend",
20
+ "truncation_side": "left",
21
+ "truncation_strategy": "longest_first",
22
+ "unk_token": "<unk>"
23
+ }