ConceptualMachines commited on
Commit
c7072e6
·
verified ·
1 Parent(s): 5c6cb6b

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +157 -0
README.md ADDED
@@ -0,0 +1,157 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ base_model: microsoft/deberta-v3-base
4
+ language:
5
+ - en
6
+ library_name: onnx
7
+ tags:
8
+ - token-classification
9
+ - intent-classification
10
+ - onnx
11
+ - daw
12
+ - music-production
13
+ ---
14
+
15
+ # MAGDA command model
16
+
17
+ Turns a typed request into a [MAGDA](https://github.com/Conceptual-Machines/magda-core)
18
+ DAW command, offline and on-device. "put @serum on a new track" becomes
19
+ `track(name="", new=true).fx.add(name="<serum>")`, which the DAW executes as one
20
+ undoable transaction.
21
+
22
+ Fine-tuned from `microsoft/deberta-v3-base`. English only.
23
+
24
+ ## What it actually does
25
+
26
+ **It never writes the command.** It does perception only, in a single forward
27
+ pass, and emits two things:
28
+
29
+ 1. which of 38 commands this is
30
+ 2. a BIO tag per word saying what role that word plays
31
+
32
+ ```
33
+ "can you drop @1176 onto the vocals"
34
+ O O O PLUGIN O O TRACK_NAME intent = add_plugin
35
+ ```
36
+
37
+ Deterministic code then assembles the DSL from those labels. The model cannot
38
+ emit malformed output, because it does not emit output — a property a
39
+ generative model would have to buy back with constrained decoding.
40
+
41
+ Plugin references never reach the model as identities: the host rewrites every
42
+ `@mention` to an opaque `<alias>` token before tokenization and substitutes the
43
+ real one back by position. So an unseen plugin generalizes for free, and the
44
+ vocabulary does not grow with the plugin registry.
45
+
46
+ ## Results
47
+
48
+ Scored as exact-match on the fully rendered command, not on tags.
49
+
50
+ | | in-distribution | **held-out phrasing** |
51
+ |---|---|---|
52
+ | previous model (51k-param conv net, from scratch) | 98.2% | **46.5%** |
53
+ | **this model** | 100% | **91.5%** |
54
+
55
+ The two columns matter more than the numbers. The in-distribution set is
56
+ authored from the same case list the training templates come from, so a high
57
+ score there means the model recalls its templates. The held-out set is 71 cases
58
+ hand-written from the command list alone, in the register people actually type:
59
+ lowercase, filler, contractions, typos. That is the number that predicts
60
+ behaviour on input nobody has seen.
61
+
62
+ The previous model had no language prior — trained from scratch on synthetic
63
+ English, it named tracks after whichever word sat in the name position during
64
+ training:
65
+
66
+ ```
67
+ "can you mute the guitar" -> track(name="Can You").track.set(mute=true)
68
+ "push Keys up to 2 dB" -> track(name="Push").track.set(volume_db=2)
69
+ "the bass needs @pro_q_3" -> track(name="Bass Needs").track.set(colour="None")
70
+ ```
71
+
72
+ Politeness wrappers scored 10%. With a pretrained encoder they score 100%.
73
+
74
+ ## Files
75
+
76
+ | file | size | what |
77
+ |---|---|---|
78
+ | `command_model.onnx` | 442 MB | encoder + intent head + slot head, one graph, opset 17 |
79
+ | `tokenizer.json` | 8 MB | HF fast tokenizer, including the `<alias>` special tokens |
80
+ | `maps.json` | 1 KB | intent and BIO tag id maps |
81
+
82
+ Inputs are `input_ids` and `attention_mask`, both `int64 [batch, seq]` with
83
+ dynamic sequence length. Outputs are `intent_logits [batch, 38]` and
84
+ `slot_logits [batch, seq, 18]`.
85
+
86
+ ## Quantization warning
87
+
88
+ The embedding is int8; **the attention and FFN deliberately are not.**
89
+
90
+ | build | size | accuracy |
91
+ |---|---|---|
92
+ | fp32 | 736 MB | 96.5% |
93
+ | int8 embedding only (this) | 442 MB | 96.1% |
94
+ | int8 everything | 538 MB | **0.9%** |
95
+
96
+ DeBERTa-v3's disentangled attention is quantization-hostile. Running
97
+ `quantize_dynamic` over the whole graph produces a model that loads, runs at
98
+ full speed, and emits garbage. If you re-quantize this, re-measure it.
99
+
100
+ ## Usage
101
+
102
+ Two details will cost you accuracy if you get them wrong.
103
+
104
+ **Words, not subwords.** The tags are per word, and the deterministic
105
+ reconstructor consumes per-word tags. Label the first subword of each word and
106
+ read the tag back at that same index. Reading tags positionally shifts every
107
+ tag after the first multi-piece word — it still produces plausible-looking
108
+ output, so it fails quietly.
109
+
110
+ **The alias tokens must stay atomic.** `<alias>` and `<alias.param>` are
111
+ registered as additional special tokens. If your tokenizer splits them into
112
+ `<`, `alias`, `>`, every plugin slot breaks.
113
+
114
+ ```python
115
+ from transformers import AutoTokenizer
116
+ import onnxruntime as ort, numpy as np
117
+
118
+ tok = AutoTokenizer.from_pretrained("microsoft/deberta-v3-base", from_slow=True)
119
+ tok.add_special_tokens({"additional_special_tokens": ["<alias>", "<alias.param>"]})
120
+ sess = ort.InferenceSession("command_model.onnx")
121
+
122
+ words = ["put", "<alias>", "on", "a", "new", "track"]
123
+ enc = tok(words, is_split_into_words=True, return_tensors="np")
124
+ intent_logits, slot_logits = sess.run(None, {
125
+ "input_ids": enc["input_ids"].astype(np.int64),
126
+ "attention_mask": enc["attention_mask"].astype(np.int64),
127
+ })
128
+ ```
129
+
130
+ A reference implementation in C++ (ONNX Runtime plus a hand-written
131
+ SentencePiece Unigram tokenizer) is in `magda/agents/command_model_onnx.cpp`
132
+ and `unigram_tokenizer.cpp` in the MAGDA repo.
133
+
134
+ ## Training
135
+
136
+ 24,000 template-generated English examples with LLM-paraphrased inputs, 4
137
+ epochs, lr 3e-5 for the encoder and 1e-3 for the two heads. Checkpoints are
138
+ selected on a hand-authored held-out set, never on the reported one — the
139
+ val split is drawn from the same templates as training and saturates at 100%
140
+ after one epoch, which makes it useless for ranking checkpoints.
141
+
142
+ ## Limitations
143
+
144
+ - **English only.** The word splitter in front of the model is ASCII-only, so
145
+ non-English input reaches it as an empty sequence. This is a limitation of
146
+ the surrounding pipeline, not the encoder, which is multilingual-capable.
147
+ - **Plugin references need the `@` sigil.** `track with @serum` works;
148
+ `track with serum` does not, and degrades into an unrelated command rather
149
+ than failing cleanly.
150
+ - **Closed command set.** 38 commands. It cannot express anything outside them
151
+ and will map an unsupported request onto the nearest one it has.
152
+ - Sentences where the verb trails its object ("i don't need the perc track
153
+ anymore, delete it") are a known weak spot.
154
+
155
+ ## License
156
+
157
+ MIT, following `microsoft/deberta-v3-base`.