Lucasgitguy-co commited on
Commit
e7e5ec3
·
verified ·
1 Parent(s): d9a49ed

Fleet-Clean release

Browse files
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tagger/tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
+ whistle.cact filter=lfs diff=lfs merge=lfs -text
LICENSE ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ Apache License
3
+ Version 2.0, January 2004
4
+ http://www.apache.org/licenses/
5
+
6
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
7
+
8
+ 1. Definitions.
9
+
10
+ "License" shall mean the terms and conditions for use, reproduction,
11
+ and distribution as defined by Sections 1 through 9 of this document.
12
+
13
+ "Licensor" shall mean the copyright owner or entity authorized by
14
+ the copyright owner that is granting the License.
15
+
16
+ "Legal Entity" shall mean the union of the acting entity and all
17
+ other entities that control, are controlled by, or are under common
18
+ control with that entity. For the purposes of this definition,
19
+ "control" means (i) the power, direct or indirect, to cause the
20
+ direction or management of such entity, whether by contract or
21
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
22
+ outstanding shares, or (iii) beneficial ownership of such entity.
23
+
24
+ "You" (or "Your") shall mean an individual or Legal Entity
25
+ exercising permissions granted by this License.
26
+
27
+ "Source" form shall mean the preferred form for making modifications,
28
+ including but not limited to software source code, documentation
29
+ source, and configuration files.
30
+
31
+ "Object" form shall mean any form resulting from mechanical
32
+ transformation or translation of a Source form, including but
33
+ not limited to compiled object code, generated documentation,
34
+ and conversions to other media types.
35
+
36
+ "Work" shall mean the work of authorship, whether in Source or
37
+ Object form, made available under the License, as indicated by a
38
+ copyright notice that is included in or attached to the work
39
+ (an example is provided in the Appendix below).
40
+
41
+ "Derivative Works" shall mean any work, whether in Source or Object
42
+ form, that is based on (or derived from) the Work and for which the
43
+ editorial revisions, annotations, elaborations, or other modifications
44
+ represent, as a whole, an original work of authorship. For the purposes
45
+ of this License, Derivative Works shall not include works that remain
46
+ separable from, or merely link (or bind by name) to the interfaces of,
47
+ the Work and Derivative Works thereof.
48
+
49
+ "Contribution" shall mean any work of authorship, including
50
+ the original version of the Work and any modifications or additions
51
+ to that Work or Derivative Works thereof, that is intentionally
52
+ submitted to Licensor for inclusion in the Work by the copyright owner
53
+ or by an individual or Legal Entity authorized to submit on behalf of
54
+ the copyright owner. For the purposes of this definition, "submitted"
55
+ means any form of electronic, verbal, or written communication sent
56
+ to the Licensor or its representatives, including but not limited to
57
+ communication on electronic mailing lists, source code control systems,
58
+ and issue tracking systems that are managed by, or on behalf of, the
59
+ Licensor for the purpose of discussing and improving the Work, but
60
+ excluding communication that is conspicuously marked or otherwise
61
+ designated in writing by the copyright owner as "Not a Contribution."
62
+
63
+ "Contributor" shall mean Licensor and any individual or Legal Entity
64
+ on behalf of whom a Contribution has been received by Licensor and
65
+ subsequently incorporated within the Work.
66
+
67
+ 2. Grant of Copyright License. Subject to the terms and conditions of
68
+ this License, each Contributor hereby grants to You a perpetual,
69
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
70
+ copyright license to reproduce, prepare Derivative Works of,
71
+ publicly display, publicly perform, sublicense, and distribute the
72
+ Work and such Derivative Works in Source or Object form.
73
+
74
+ 3. Grant of Patent License. Subject to the terms and conditions of
75
+ this License, each Contributor hereby grants to You a perpetual,
76
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77
+ (except as stated in this section) patent license to make, have made,
78
+ use, offer to sell, sell, import, and otherwise transfer the Work,
79
+ where such license applies only to those patent claims licensable
80
+ by such Contributor that are necessarily infringed by their
81
+ Contribution(s) alone or by combination of their Contribution(s)
82
+ with the Work to which such Contribution(s) was submitted. If You
83
+ institute patent litigation against any entity (including a
84
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
85
+ or a Contribution incorporated within the Work constitutes direct
86
+ or contributory patent infringement, then any patent licenses
87
+ granted to You under this License for that Work shall terminate
88
+ as of the date such litigation is filed.
89
+
90
+ 4. Redistribution. You may reproduce and distribute copies of the
91
+ Work or Derivative Works thereof in any medium, with or without
92
+ modifications, and in Source or Object form, provided that You
93
+ meet the following conditions:
94
+
95
+ (a) You must give any other recipients of the Work or
96
+ Derivative Works a copy of this License; and
97
+
98
+ (b) You must cause any modified files to carry prominent notices
99
+ stating that You changed the files; and
100
+
101
+ (c) You must retain, in the Source form of any Derivative Works
102
+ that You distribute, all copyright, patent, trademark, and
103
+ attribution notices from the Source form of the Work,
104
+ excluding those notices that do not pertain to any part of
105
+ the Derivative Works; and
106
+
107
+ (d) If the Work includes a "NOTICE" text file as part of its
108
+ distribution, then any Derivative Works that You distribute must
109
+ include a readable copy of the attribution notices contained
110
+ within such NOTICE file, excluding those notices that do not
111
+ pertain to any part of the Derivative Works, in at least one
112
+ of the following places: within a NOTICE text file distributed
113
+ as part of the Derivative Works; within the Source form or
114
+ documentation, if provided along with the Derivative Works; or,
115
+ within a display generated by the Derivative Works, if and
116
+ wherever such third-party notices normally appear. The contents
117
+ of the NOTICE file are for informational purposes only and
118
+ do not modify the License. You may add Your own attribution
119
+ notices within Derivative Works that You distribute, alongside
120
+ or as an addendum to the NOTICE text from the Work, provided
121
+ that such additional attribution notices cannot be construed
122
+ as modifying the License.
123
+
124
+ You may add Your own copyright statement to Your modifications and
125
+ may provide additional or different license terms and conditions
126
+ for use, reproduction, or distribution of Your modifications, or
127
+ for any such Derivative Works as a whole, provided Your use,
128
+ reproduction, and distribution of the Work otherwise complies with
129
+ the conditions stated in this License.
130
+
131
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
132
+ any Contribution intentionally submitted for inclusion in the Work
133
+ by You to the Licensor shall be under the terms and conditions of
134
+ this License, without any additional terms or conditions.
135
+ Notwithstanding the above, nothing herein shall supersede or modify
136
+ the terms of any separate license agreement you may have executed
137
+ with Licensor regarding such Contributions.
138
+
139
+ 6. Trademarks. This License does not grant permission to use the trade
140
+ names, trademarks, service marks, or product names of the Licensor,
141
+ except as required for reasonable and customary use in describing the
142
+ origin of the Work and reproducing the content of the NOTICE file.
143
+
144
+ 7. Disclaimer of Warranty. Unless required by applicable law or
145
+ agreed to in writing, Licensor provides the Work (and each
146
+ Contributor provides its Contributions) on an "AS IS" BASIS,
147
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148
+ implied, including, without limitation, any warranties or conditions
149
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150
+ PARTICULAR PURPOSE. You are solely responsible for determining the
151
+ appropriateness of using or redistributing the Work and assume any
152
+ risks associated with Your exercise of permissions under this License.
153
+
154
+ 8. Limitation of Liability. In no event and under no legal theory,
155
+ whether in tort (including negligence), contract, or otherwise,
156
+ unless required by applicable law (such as deliberate and grossly
157
+ negligent acts) or agreed to in writing, shall any Contributor be
158
+ liable to You for damages, including any direct, indirect, special,
159
+ incidental, or consequential damages of any character arising as a
160
+ result of this License or out of the use or inability to use the
161
+ Work (including but not limited to damages for loss of goodwill,
162
+ work stoppage, computer failure or malfunction, or any and all
163
+ other commercial damages or losses), even if such Contributor
164
+ has been advised of the possibility of such damages.
165
+
166
+ 9. Accepting Warranty or Additional Liability. While redistributing
167
+ the Work or Derivative Works thereof, You may choose to offer,
168
+ and charge a fee for, acceptance of support, warranty, indemnity,
169
+ or other liability obligations and/or rights consistent with this
170
+ License. However, in accepting such obligations, You may act only
171
+ on Your own behalf and on Your sole responsibility, not on behalf
172
+ of any other Contributor, and only if You agree to indemnify,
173
+ defend, and hold each Contributor harmless for any liability
174
+ incurred by, or claims asserted against, such Contributor by reason
175
+ of your accepting any such warranty or additional liability.
176
+
177
+ END OF TERMS AND CONDITIONS
178
+
179
+ APPENDIX: How to apply the Apache License to your work.
180
+
181
+ To apply the Apache License to your work, attach the following
182
+ boilerplate notice, with the fields enclosed by brackets "[]"
183
+ replaced with your own identifying information. (Don't include
184
+ the brackets!) The text should be enclosed in the appropriate
185
+ comment syntax for the file format. We also recommend that a
186
+ file or class name and description of purpose be included on the
187
+ same "printed page" as the copyright notice for easier
188
+ identification within third-party archives.
189
+
190
+ Copyright [yyyy] [name of copyright owner]
191
+
192
+ Licensed under the Apache License, Version 2.0 (the "License");
193
+ you may not use this file except in compliance with the License.
194
+ You may obtain a copy of the License at
195
+
196
+ http://www.apache.org/licenses/LICENSE-2.0
197
+
198
+ Unless required by applicable law or agreed to in writing, software
199
+ distributed under the License is distributed on an "AS IS" BASIS,
200
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
201
+ See the License for the specific language governing permissions and
202
+ limitations under the License.
NOTICE ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Fleet-Clean
2
+ Copyright 2026 Lucasgitguy-co. Licensed under the Apache License, Version 2.0 (see LICENSE).
3
+
4
+ Components:
5
+ - Fleet-2 (Lucasgitguy-co, Apache-2.0), itself derived from Whistle by Cactus Compute (Apache-2.0),
6
+ Sky-Kim/whistle-finetuned-ko (Apache-2.0) and Lucasgitguy-co/Fleet-1 (Apache-2.0); training targets by
7
+ openai/whisper-large-v3-turbo (MIT, Copyright (c) 2022 OpenAI), no Whisper weights included.
8
+ - The cleanup tagger's encoder: convaiinnovations/laya-multilingual (Apache-2.0, Convai Innovations), built on
9
+ jhu-clsp/mmBERT-base (MIT License).
10
+ - Tag data: DisfluencySpeech (Apache-2.0), VoxPopuli (CC0 1.0), Zeroth-Korean (CC BY 4.0, Atlas Guide Inc. and
11
+ Gridspace Inc.), FLEURS (CC BY 4.0, Conneau et al. 2022). No audio is redistributed.
README.md ADDED
@@ -0,0 +1,127 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language: [en, de, fr, es, it, nl, pl, ko]
4
+ pipeline_tag: automatic-speech-recognition
5
+ base_model:
6
+ - Lucasgitguy-co/Fleet-2
7
+ - convaiinnovations/laya-multilingual
8
+ datasets:
9
+ - Lucasgitguy-co/fleet-clean-tags
10
+ - amaai-lab/DisfluencySpeech
11
+ - facebook/voxpopuli
12
+ - Bingsu/zeroth-korean
13
+ - google/fleurs
14
+ tags: [speech-to-text, disfluency-removal, punctuation, token-classification, whistle, cactus, mmbert, laya]
15
+ ---
16
+
17
+ # Fleet-Clean-0.36B
18
+
19
+ Speech in, clean text out. It is two open models with 363M parameters in total:
20
+
21
+ 1. **[Fleet-2](https://huggingface.co/Lucasgitguy-co/Fleet-2)** (56M, Cactus Whistle) writes down what was said.
22
+ 2. **A word tagger** (307M) cleans that transcript. It is the encoder of [convaiinnovations/laya-multilingual](https://huggingface.co/convaiinnovations/laya-multilingual) (mmBERT) with a new head that decides, for every word:
23
+ - keep or delete it (fillers, discourse markers, repeats, false starts)
24
+ - which mark follows it (none , . ? !)
25
+ - whether its first letter changes case
26
+
27
+ The tagger only removes Fleet-2's words and fixes punctuation and capitals. **It never writes a word of its own**, so it can't invent content or answer what was dictated.
28
+
29
+ **GGUF version:** [Fleet-Clean-0.36B-GGUF](https://huggingface.co/Lucasgitguy-co/Fleet-Clean-0.36B-GGUF) · **Training data:** [fleet-clean-tags](https://huggingface.co/datasets/Lucasgitguy-co/fleet-clean-tags) · **Collection:** [Speech recognition](https://huggingface.co/collections/Lucasgitguy-co/speech-recognition-6aef2dcbe0ea208136b0ddc9)
30
+
31
+ ## Usage
32
+ ```bash
33
+ pip install cactus-compute torch "transformers>=5" safetensors
34
+ hf download Lucasgitguy-co/Fleet-Clean-0.36B --local-dir fleet-clean && cd fleet-clean
35
+ python fleet_clean.py clip.wav # clean transcript
36
+ python fleet_clean.py --json clip.wav # raw, clean and the per-word tags
37
+ python fleet_clean.py --text "so I I think, you know, we should go"
38
+ python fleet_clean.py --punct full clip.wav # also re-punctuate words away from deletions
39
+ ```
40
+ Audio is 16 kHz WAV (`ffmpeg -i in.m4a -ar 16000 -ac 1 clip.wav`). Fleet-2 detects the language, or you can pass `--language`. The delete threshold is 0.5. It was calibrated on validation data so that fluent speech keeps its words, and `--threshold` changes it.
41
+
42
+ ## Examples
43
+ These are from the DisfluencySpeech test set (English):
44
+
45
+ - **Fleet-2:** Yeah. Well, robots are darling. That would be fun if you could get them trained. Otherwise, they're pretty smelly.
46
+ **Fleet-Clean:** Yeah, robots are darling. That would be fun if you could get them trained. Otherwise, they're pretty smelly.
47
+ - **Fleet-2:** Well, McNew, New, and Lara, I guess it's leaving one of them is a leaving.
48
+ **Fleet-Clean:** McNew, New, and Lara, I guess one of them is a leaving.
49
+ - **Fleet-2:** Well, I don't want to. I don't want this to be a unilateral conversation, but yes, we do.
50
+ **Fleet-Clean:** I don't want to. I don't want this to be a unilateral conversation, but yes, we do.
51
+
52
+ ## Evaluation
53
+ **Real disfluent English speech.** Fleet-2 transcribes the audio, and the output is compared with DisfluencySpeech's own cleaned transcript:
54
+
55
+ | DisfluencySpeech test (250 clips) | Fleet-2 alone | Fleet-Clean |
56
+ |---|---:|---:|
57
+ | WER against the cleaned transcript (`transcript_c`), punctuation ignored | 18.8% | **14.6%** |
58
+ | Punctuation F1 against `transcript_c` | **0.6700** | 0.6695 |
59
+
60
+ **Fluent speech (FLEURS test).** These are the same 100 clips per language as Fleet-2's card. A cleanup model should leave fluent speech alone:
61
+
62
+ | Language | Words deleted from fluent speech | WER before | WER after |
63
+ |---|---:|---:|---:|
64
+ | English | 0.47% | 14.5% | 14.3% |
65
+ | German | 0.05% | 22.0% | 22.0% |
66
+ | French | 0.04% | 24.9% | 25.0% |
67
+ | Spanish | 0.05% | 11.9% | 11.9% |
68
+ | Italian | 0.06% | 16.7% | 16.7% |
69
+ | Dutch | 0.19% | 30.1% | 30.0% |
70
+ | Polish | 0.29% | 41.6% | 41.6% |
71
+ | Korean | 0.27% | 42.8% (CER) | 42.9% (CER) |
72
+
73
+ <details>
74
+ <summary>Synthetic disfluencies (held-out tag test rows)</summary>
75
+
76
+ Fillers, discourse markers, repeats, restarts and self-corrections were inserted into Fleet-2 transcripts of held-out VoxPopuli / Zeroth / FLEURS clips. The table shows whether the tagger finds exactly the inserted words. "Weak" means F1 below 0.80.
77
+
78
+ | Language | Delete F1 | Precision | Recall |
79
+ |---|---:|---:|---:|
80
+ | English | 0.981 | 0.967 | 0.996 |
81
+ | German | 0.984 | 0.968 | 1.000 |
82
+ | French | 0.970 | 0.946 | 0.995 |
83
+ | Spanish | 0.971 | 0.978 | 0.964 |
84
+ | Italian | 0.969 | 0.959 | 0.979 |
85
+ | Dutch | 0.952 | 0.920 | 0.986 |
86
+ | Polish | 0.990 | 0.980 | 1.000 |
87
+ | Korean | 0.979 | 0.975 | 0.984 |
88
+ </details>
89
+
90
+ Limits:
91
+ - **Real disfluent speech was only tested in English.** The other seven languages were trained and tested on synthetic disfluencies, so their numbers show how well the tagger removes those inserted patterns, not how it handles real conversation.
92
+ - **Fleet-2 already leaves out most "uh/um".** It learned from Whisper transcripts. What is left to clean is mostly "you know", "I mean", "like", repeats and restarts.
93
+ - **Recognition errors stay.** The tagger can only delete or re-punctuate words, so it can't fix a misheard word.
94
+ - Clips are up to 30 s per call. The tagger reads up to 512 tokens and leaves anything past that unchanged.
95
+
96
+ ## How it was built
97
+ <details>
98
+ <summary>Data, training and gates</summary>
99
+
100
+ - **Transcripts:** Fleet-2 (`whistle.cact`, native Cactus engine, automatic language detection) transcribed:
101
+ - DisfluencySpeech train/validation/test
102
+ - VoxPopuli validation (7 languages, a split Fleet-2 never trained on)
103
+ - Zeroth-Korean train minus Fleet-2's own training rows
104
+ - FLEURS Korean train minus Fleet-2's rows
105
+ - **Labels:**
106
+ - **English deletions:** each Fleet-2 word is aligned to DisfluencySpeech's verbatim transcript, and that transcript's words are mapped to its cleaned version.
107
+ - **Punctuation and case:** taken from the human reference wherever Fleet-2's word matches it. Zeroth has no punctuation, so its rows only teach deletion.
108
+ - **Synthetic rows:** disfluencies are only ever inserted, so the clean text is always recoverable by deletion.
109
+ - **Training:**
110
+ - Laya-multilingual's encoder, with its 256k-token embedding table frozen, plus a new 768→10 head.
111
+ - PyTorch on an Apple M4 (MPS), fp32.
112
+ - Best validation checkpoint kept, then the delete threshold calibrated on validation only.
113
+ - **Release gates:**
114
+ - G1: cleaning lowers WER on real disfluent English, and punctuation F1 doesn't drop. WER dropped from 18.76% to 14.61%, but punctuation F1 went from 0.6700 to 0.6695 (−0.0005), so this part of the gate did not pass. It was published anyway by the owner's decision, and the number is reported as measured.
115
+ - G2: at most 1.0% of words deleted from fluent FLEURS speech in every language. Passed in all 8.
116
+ - **Punctuation mode:** by default, Fleet-2's own punctuation is kept, and the tagger's punctuation and case are used only next to removed words. This was chosen on DisfluencySpeech *validation*, where re-punctuating everything scored lower against the human clean transcripts. `--punct full` re-punctuates every word; on fluent validation speech in all 8 languages, full mode was slightly more accurate than Fleet-2's own marks.
117
+ </details>
118
+
119
+ ## License and attribution
120
+ Apache-2.0. See `NOTICE` for every component:
121
+ - Fleet-2 / Cactus Whistle (Apache-2.0)
122
+ - Laya-multilingual (Apache-2.0), which builds on mmBERT (MIT)
123
+ - DisfluencySpeech (Apache-2.0)
124
+ - VoxPopuli (CC0)
125
+ - Zeroth-Korean and FLEURS (CC BY 4.0)
126
+
127
+ No audio is included.
fleet_clean.py ADDED
@@ -0,0 +1,102 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Fleet-Clean: speech -> Fleet-2 transcript -> Laya word tagger -> clean text.
3
+
4
+ python fleet_clean.py clip.wav # print the clean transcript
5
+ python fleet_clean.py --json clip.wav other.wav # raw + clean + per-word tags as JSON lines
6
+ python fleet_clean.py --text "so I I think, you know, we should go"
7
+
8
+ Speech runs in Cactus' native engine on whistle.cact (pip install cactus-compute); cleanup runs the tagger in
9
+ PyTorch (pip install torch "transformers>=5" safetensors). Both models sit next to this file.
10
+ The tagger only deletes words and changes punctuation / first-letter case: it never writes a word of its own.
11
+ """
12
+ import argparse
13
+ import json
14
+ import sys
15
+ from pathlib import Path
16
+
17
+ HERE = Path(__file__).resolve().parent
18
+ sys.path.insert(0, str(HERE))
19
+ from fleet_clean_words import C_KEEP, PUNCT_MODES, apply_tags, resolve, trailing_class # noqa: E402
20
+
21
+
22
+ class Tagger:
23
+ def __init__(self, path=HERE / "tagger", device=None):
24
+ import torch
25
+ from safetensors.torch import load_file
26
+ from transformers import AutoTokenizer, ModernBertModel
27
+ self.torch = torch
28
+ self.device = device or ("mps" if torch.backends.mps.is_available() else "cpu")
29
+ self.encoder = ModernBertModel.from_pretrained(path).to(self.device).eval()
30
+ head = load_file(str(Path(path) / "head.safetensors"))
31
+ self.w, self.b = head["weight"].to(self.device), head["bias"].to(self.device)
32
+ self.tok = AutoTokenizer.from_pretrained(path)
33
+ import json
34
+ labels = json.loads((Path(path) / "labels.json").read_text())
35
+ self.threshold = labels.get("delete_threshold", 0.5)
36
+ self.punct_mode = labels.get("punct_mode", "boundary")
37
+
38
+ def tag(self, words, delete_threshold=None):
39
+ delete_threshold = self.threshold if delete_threshold is None else delete_threshold
40
+ torch = self.torch
41
+ if not words:
42
+ return [], [], []
43
+ enc = self.tok(words, is_split_into_words=True, truncation=True, max_length=512)
44
+ first = [None] * len(words)
45
+ for t, w in enumerate(enc.word_ids()):
46
+ if w is not None and first[w] is None:
47
+ first[w] = t
48
+ x = torch.tensor([enc["input_ids"]], device=self.device)
49
+ with torch.no_grad():
50
+ h = self.encoder(input_ids=x, attention_mask=torch.ones_like(x)).last_hidden_state[0].float()
51
+ logits = (h @ self.w.T + self.b).cpu()
52
+ p_del = logits[:, :2].softmax(-1)[:, 1]
53
+ d, p, c = [], [], []
54
+ for w, t in zip(words, first):
55
+ if t is None: # past the 512-token window: keep the word as written
56
+ d.append(0), p.append(trailing_class(w)), c.append(C_KEEP)
57
+ else:
58
+ d.append(int(p_del[t] >= delete_threshold))
59
+ p.append(int(logits[t, 2:7].argmax()))
60
+ c.append(int(logits[t, 7:10].argmax()))
61
+ return d, p, c
62
+
63
+ def clean(self, text, delete_threshold=None, punct_mode=None):
64
+ words = text.split()
65
+ d, p, c = self.tag(words, delete_threshold)
66
+ P, C = resolve(words, d, p, c, punct_mode or self.punct_mode)
67
+ return apply_tags(words, d, P, C), {"words": words, "delete": d, "punct": P, "case": C}
68
+
69
+
70
+ def transcribe(path, weights=HERE / "whistle.cact", language=None):
71
+ import cactus
72
+ res = cactus.transcribe(str(path), language=language, weights=str(weights))
73
+ return res["text"].strip(), res.get("language")
74
+
75
+
76
+ def main(argv=None):
77
+ ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
78
+ ap.add_argument("audio", nargs="*", help="WAV files (16 kHz mono; other rates need cactus-compute[mic])")
79
+ ap.add_argument("--text", help="clean this text instead of transcribing audio")
80
+ ap.add_argument("--language", default=None, help="en de fr es it nl pl ko (default: detect)")
81
+ ap.add_argument("--threshold", type=float, default=None, help="delete probability threshold (default: calibrated)")
82
+ ap.add_argument("--punct", choices=PUNCT_MODES, default=None,
83
+ help="boundary (default): fix punctuation only where words were removed; full: re-punctuate everything")
84
+ ap.add_argument("--device", default=None)
85
+ ap.add_argument("--json", action="store_true")
86
+ a = ap.parse_args(argv)
87
+ if not a.text and not a.audio:
88
+ ap.error("give a WAV file or --text")
89
+ tagger = Tagger(device=a.device)
90
+ jobs = [("text", a.text)] if a.text else [("audio", p) for p in a.audio]
91
+ for kind, item in jobs:
92
+ raw, lang = transcribe(item, language=a.language) if kind == "audio" else (item, a.language)
93
+ clean, tags = tagger.clean(raw, a.threshold, a.punct)
94
+ if a.json:
95
+ print(json.dumps({"source": str(item) if kind == "audio" else None, "language": lang, "raw": raw,
96
+ "clean": clean, **tags}, ensure_ascii=False))
97
+ else:
98
+ print(clean)
99
+
100
+
101
+ if __name__ == "__main__":
102
+ main()
fleet_clean_words.py ADDED
@@ -0,0 +1,125 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """The word rule shared by training, the torch runner and the GGUF runner.
2
+
3
+ A transcript is split on whitespace into words. The tagger sees the transcript exactly as Fleet-2 wrote it and
4
+ predicts, per word:
5
+ delete 0 keep / 1 delete
6
+ punct the word's trailing mark in the clean text: 0 none, 1 ",", 2 ".", 3 "?", 4 "!"
7
+ case 0 keep / 1 upper-case the first letter / 2 lower-case the first letter (relative to Fleet-2's casing)
8
+
9
+ Applying the tags (`apply_tags`) replaces each kept word's trailing , . ? ! (one or more) with the predicted
10
+ mark and fixes the first letter. Marks inside a word ("1,000", "3.5", "U.S.") are never touched: only a trailing
11
+ run of , . ? ! is replaced, and a dot is kept when the word is an abbreviation like "U.S." or "e.g.".
12
+ Nothing else changes, so the clean text is always Fleet-2's own words (minus the deleted ones).
13
+ """
14
+ import re
15
+
16
+ PUNCT = ("", ",", ".", "?", "!")
17
+ P_NONE, P_COMMA, P_DOT, P_Q, P_EXCL = range(5)
18
+ C_KEEP, C_UPPER, C_LOWER = range(3)
19
+ IGNORE = -100
20
+
21
+ _TRAIL = re.compile(r"[,.?!]+$")
22
+ _ABBREV = re.compile(r"^(?:[^\W\d_]\.){2,}$|^(?:e\.g|i\.e|etc|vs|z\.B|bzw|usw|ca|Dr|Mr|Mrs|Ms|St|Nr|nr|bijv)\.$",
23
+ re.IGNORECASE)
24
+ _EDGE = re.compile(r"^[^\w']+|[^\w']+$")
25
+
26
+
27
+ def split_words(text):
28
+ return (text or "").split()
29
+
30
+
31
+ def norm_word(w):
32
+ return _EDGE.sub("", w.lower()).replace("’", "'")
33
+
34
+
35
+ def core(word):
36
+ """The word without its trailing , . ? ! run (abbreviations keep their dots)."""
37
+ if _ABBREV.match(word):
38
+ return word
39
+ stripped = _TRAIL.sub("", word)
40
+ if not stripped:
41
+ return word # a bare "." / "?" token stays itself
42
+ if _ABBREV.match(stripped + "."):
43
+ return stripped + "." # "U.S.," -> "U.S."
44
+ return stripped
45
+
46
+
47
+ def trailing_class(word):
48
+ """The punctuation class a word currently ends with (last mark of the trailing run decides)."""
49
+ if _ABBREV.match(word):
50
+ return P_NONE
51
+ m = _TRAIL.search(word)
52
+ if not m or m.group(0) == word:
53
+ return P_NONE
54
+ return PUNCT.index(m.group(0)[-1]) if m.group(0)[-1] in PUNCT else P_NONE
55
+
56
+
57
+ def first_alpha(word):
58
+ for i, ch in enumerate(word):
59
+ if ch.isalpha():
60
+ return i
61
+ return -1
62
+
63
+
64
+ def case_class(src, dst):
65
+ """How dst's first letter relates to src's: keep / upper / lower."""
66
+ i, j = first_alpha(src), first_alpha(dst)
67
+ if i < 0 or j < 0:
68
+ return C_KEEP
69
+ a, b = src[i], dst[j]
70
+ if a == b or a.upper() == a.lower():
71
+ return C_KEEP
72
+ if b.isupper() and a.islower():
73
+ return C_UPPER
74
+ if b.islower() and a.isupper():
75
+ return C_LOWER
76
+ return C_KEEP
77
+
78
+
79
+ def recase(word, case):
80
+ i = first_alpha(word)
81
+ if i < 0 or case == C_KEEP:
82
+ return word
83
+ ch = word[i].upper() if case == C_UPPER else word[i].lower()
84
+ return word[:i] + ch + word[i + 1:]
85
+
86
+
87
+ def apply_tags(words, delete, punct, case):
88
+ out = []
89
+ for w, d, p, c in zip(words, delete, punct, case):
90
+ if d:
91
+ continue
92
+ base = core(w)
93
+ if _ABBREV.match(base): # abbreviation: keep its own dot, only add , ? ! after it
94
+ out.append(recase(base, c) + (PUNCT[p] if p in (P_COMMA, P_Q, P_EXCL) else ""))
95
+ else:
96
+ out.append(recase(base, c) + PUNCT[p])
97
+ return " ".join(out)
98
+
99
+
100
+ PUNCT_MODES = ("boundary", "full")
101
+
102
+
103
+ def resolve(words, delete, punct, case, mode="boundary"):
104
+ """Which predicted marks / cases to use.
105
+
106
+ full the heads' punctuation and case on every kept word.
107
+ boundary (default) Fleet-2's own marks and case, except next to deletions: the head's punctuation on the
108
+ last kept word before a deleted run (or at the end after one), the head's case on the first kept
109
+ word after a deleted run. Chosen on DisfluencySpeech validation, where full re-punctuation scored
110
+ lower against the human clean transcripts.
111
+ """
112
+ if mode == "full":
113
+ return list(punct), list(case)
114
+ if mode != "boundary":
115
+ raise ValueError(f"unknown punctuation mode {mode}")
116
+ P, C = [trailing_class(w) for w in words], [C_KEEP] * len(words)
117
+ kept = [i for i in range(len(words)) if not delete[i]]
118
+ for a, i in enumerate(kept):
119
+ nxt = kept[a + 1] if a + 1 < len(kept) else len(words)
120
+ prv = kept[a - 1] if a > 0 else -1
121
+ if nxt != i + 1:
122
+ P[i] = punct[i]
123
+ if prv != i - 1:
124
+ C[i] = case[i]
125
+ return P, C
requirements.txt ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ cactus-compute
2
+ torch
3
+ transformers>=5
4
+ safetensors
results/calibrate.json ADDED
@@ -0,0 +1,136 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "chosen": 0.5,
3
+ "met_rule": true,
4
+ "sweep": {
5
+ "0.5": {
6
+ "false_delete_B": {
7
+ "nl": 0.0009852216748768472,
8
+ "pl": 0.002324230098779779,
9
+ "fr": 0.0,
10
+ "ko": 0.0005162622612287042,
11
+ "en": 0.0015353121801432957,
12
+ "it": 0.0028790786948176585,
13
+ "de": 0.0041841004184100415,
14
+ "es": 0.0009350163627863488
15
+ },
16
+ "delete_f1_AC": 0.9348850096979773
17
+ },
18
+ "0.55": {
19
+ "false_delete_B": {
20
+ "nl": 0.0009852216748768472,
21
+ "pl": 0.002324230098779779,
22
+ "fr": 0.0,
23
+ "ko": 0.0005162622612287042,
24
+ "en": 0.0015353121801432957,
25
+ "it": 0.0028790786948176585,
26
+ "de": 0.0041841004184100415,
27
+ "es": 0.0009350163627863488
28
+ },
29
+ "delete_f1_AC": 0.9389121338912134
30
+ },
31
+ "0.6": {
32
+ "false_delete_B": {
33
+ "nl": 0.0009852216748768472,
34
+ "pl": 0.002324230098779779,
35
+ "fr": 0.0,
36
+ "ko": 0.0,
37
+ "en": 0.0015353121801432957,
38
+ "it": 0.0028790786948176585,
39
+ "de": 0.0035863717872086074,
40
+ "es": 0.0009350163627863488
41
+ },
42
+ "delete_f1_AC": 0.9406827084499161
43
+ },
44
+ "0.65": {
45
+ "false_delete_B": {
46
+ "nl": 0.0009852216748768472,
47
+ "pl": 0.002324230098779779,
48
+ "fr": 0.0,
49
+ "ko": 0.0,
50
+ "en": 0.0015353121801432957,
51
+ "it": 0.0028790786948176585,
52
+ "de": 0.002988643156007173,
53
+ "es": 0.0009350163627863488
54
+ },
55
+ "delete_f1_AC": 0.9417721518987342
56
+ },
57
+ "0.7": {
58
+ "false_delete_B": {
59
+ "nl": 0.0009852216748768472,
60
+ "pl": 0.002324230098779779,
61
+ "fr": 0.0,
62
+ "ko": 0.0,
63
+ "en": 0.0015353121801432957,
64
+ "it": 0.0028790786948176585,
65
+ "de": 0.002988643156007173,
66
+ "es": 0.0009350163627863488
67
+ },
68
+ "delete_f1_AC": 0.9418407679277244
69
+ },
70
+ "0.75": {
71
+ "false_delete_B": {
72
+ "nl": 0.0009852216748768472,
73
+ "pl": 0.002324230098779779,
74
+ "fr": 0.0,
75
+ "ko": 0.0,
76
+ "en": 0.0015353121801432957,
77
+ "it": 0.0019193857965451055,
78
+ "de": 0.002390914524805738,
79
+ "es": 0.0009350163627863488
80
+ },
81
+ "delete_f1_AC": 0.941243258586432
82
+ },
83
+ "0.8": {
84
+ "false_delete_B": {
85
+ "nl": 0.0009852216748768472,
86
+ "pl": 0.002324230098779779,
87
+ "fr": 0.0,
88
+ "ko": 0.0,
89
+ "en": 0.0015353121801432957,
90
+ "it": 0.0019193857965451055,
91
+ "de": 0.002390914524805738,
92
+ "es": 0.0009350163627863488
93
+ },
94
+ "delete_f1_AC": 0.9426533523537803
95
+ },
96
+ "0.85": {
97
+ "false_delete_B": {
98
+ "nl": 0.0,
99
+ "pl": 0.002324230098779779,
100
+ "fr": 0.0,
101
+ "ko": 0.0,
102
+ "en": 0.0015353121801432957,
103
+ "it": 0.0019193857965451055,
104
+ "de": 0.002390914524805738,
105
+ "es": 0.0009350163627863488
106
+ },
107
+ "delete_f1_AC": 0.9411764705882353
108
+ },
109
+ "0.9": {
110
+ "false_delete_B": {
111
+ "nl": 0.0,
112
+ "pl": 0.002324230098779779,
113
+ "fr": 0.0,
114
+ "ko": 0.0,
115
+ "en": 0.0015353121801432957,
116
+ "it": 0.0019193857965451055,
117
+ "de": 0.0017931858936043037,
118
+ "es": 0.0009350163627863488
119
+ },
120
+ "delete_f1_AC": 0.9388696655132641
121
+ },
122
+ "0.95": {
123
+ "false_delete_B": {
124
+ "nl": 0.0,
125
+ "pl": 0.002324230098779779,
126
+ "fr": 0.0,
127
+ "ko": 0.0,
128
+ "en": 0.0015353121801432957,
129
+ "it": 0.0019193857965451055,
130
+ "de": 0.0017931858936043037,
131
+ "es": 0.0009350163627863488
132
+ },
133
+ "delete_f1_AC": 0.9373725604427614
134
+ }
135
+ }
136
+ }
results/eval.json ADDED
@@ -0,0 +1,321 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "threshold": 0.5,
3
+ "punct_mode": "boundary",
4
+ "tagger": "/Users/lucasvandeweerd/fleet-clean/models/tagger",
5
+ "G1": {
6
+ "clips": 250,
7
+ "wer_raw_vs_c": 0.18759078647022204,
8
+ "wer_clean_vs_c": 0.14608840008300478,
9
+ "punct_f1_raw": 0.6699588477366255,
10
+ "punct_f1_clean": 0.6694843617920541,
11
+ "examples": [
12
+ {
13
+ "raw": "Yeah. Well, robots are darling. That would be fun if you could get them trained. Otherwise, they're pretty smelly.",
14
+ "clean": "Yeah, robots are darling. That would be fun if you could get them trained. Otherwise, they're pretty smelly.",
15
+ "transcript_c": "Yeah. rabbits are darling. That would be fun if you could get them trained. Otherwise they're pretty smelly."
16
+ },
17
+ {
18
+ "raw": "Well, McNew, New, and Lara, I guess it's leaving one of them is a leaving.",
19
+ "clean": "McNew, New, and Lara, I guess one of them is a leaving.",
20
+ "transcript_c": "McNeil, and Lehrer, I guess he's leaving, one of them is leaving."
21
+ },
22
+ {
23
+ "raw": "Well, I don't want to. I don't want this to be a unilateral conversation, but yes, we do.",
24
+ "clean": "I don't want to. I don't want this to be a unilateral conversation, but yes, we do.",
25
+ "transcript_c": "I don't want this to be a unilateral conversation but, yes, we do."
26
+ },
27
+ {
28
+ "raw": "Well, what about peppers, you know, made out of them, you know, Bembusta from the banks of the Nile?",
29
+ "clean": "What about peppers, made out of them, Bembusta from the banks of the Nile?",
30
+ "transcript_c": "what about, papyrus, made out of bamboo stuff, from the banks of the Nile."
31
+ },
32
+ {
33
+ "raw": "Because women are in every few now. I mean, I can think of a feud that they are not involved in.",
34
+ "clean": "Because women are in every few now. I can think of a feud that they are not involved in.",
35
+ "transcript_c": "Because women are in every field now, I can't think of a field that they're not involved in."
36
+ },
37
+ {
38
+ "raw": "Yeah, I think it's getting better, not worse. Well, I should probably get back to my job.",
39
+ "clean": "Yeah, I think it's getting better, not worse. I should probably get back to my job.",
40
+ "transcript_c": "Yeah, I think it's getting better, not worse. I should probably get back to my job."
41
+ },
42
+ {
43
+ "raw": "They go, well, they got a lot of benefits that would go with it.",
44
+ "clean": "They got a lot of benefits that would go with it.",
45
+ "transcript_c": "They go, they got a lot of benefits that would go with it so."
46
+ },
47
+ {
48
+ "raw": "What it is for us to make, mark off models, you know. It's like Jesus Christ.",
49
+ "clean": "What it is for us to make, mark off models, It's like Jesus Christ.",
50
+ "transcript_c": "what it is for us to make mark off models, It's like, Jesus Christ."
51
+ },
52
+ {
53
+ "raw": "And the baby just screams. I mean, the baby is like 17 months and she just screams.",
54
+ "clean": "And the baby just screams. The baby is 17 months and she just screams.",
55
+ "transcript_c": "And, the baby just screams. the baby is seventeen months and she just screams."
56
+ },
57
+ {
58
+ "raw": "Yeah, that seems to be the norm of most movies, but what the hey? This is America.",
59
+ "clean": "Yeah, that seems to be the norm of most movies, but what the this is America.",
60
+ "transcript_c": "Yeah, that seems to be the norm of most movies, but what the hey. This is America."
61
+ },
62
+ {
63
+ "raw": "No, we, well, I, and of course, you know, you have the rates that come up in between the bricks or in things like that.",
64
+ "clean": "No, we, and of course, you have the rates that come up in between the bricks or in things like that.",
65
+ "transcript_c": "No we, I, And, of course you have the weeds that come up in between the bricks and things like that."
66
+ },
67
+ {
68
+ "raw": "Or, you know, whatever the case may be and may not have any idea of what the standup punishment would be for it.",
69
+ "clean": "whatever the case may be and may not have any idea of what the standup punishment would be for it.",
70
+ "transcript_c": "Or, whatever the case may be. and may not have any idea of what the standard punishment would be for it."
71
+ }
72
+ ],
73
+ "pass": false
74
+ },
75
+ "G2": {
76
+ "en": {
77
+ "clips": 100,
78
+ "deleted_pct": 0.472193074501574,
79
+ "wer_raw": 0.14496573537163943,
80
+ "wer_clean": 0.143384290985767,
81
+ "deleted_examples": [
82
+ "kind",
83
+ "of",
84
+ "well,",
85
+ "to",
86
+ "make",
87
+ "exhal.",
88
+ "That",
89
+ "is,",
90
+ "it"
91
+ ],
92
+ "pass": true
93
+ },
94
+ "de": {
95
+ "clips": 100,
96
+ "deleted_pct": 0.05230125523012552,
97
+ "wer_raw": 0.22021276595744682,
98
+ "wer_clean": 0.21968085106382979,
99
+ "deleted_examples": [
100
+ "die"
101
+ ],
102
+ "pass": true
103
+ },
104
+ "fr": {
105
+ "clips": 100,
106
+ "deleted_pct": 0.04424778761061947,
107
+ "wer_raw": 0.24926315789473685,
108
+ "wer_clean": 0.2496842105263158,
109
+ "deleted_examples": [
110
+ "puis,"
111
+ ],
112
+ "pass": true
113
+ },
114
+ "es": {
115
+ "clips": 100,
116
+ "deleted_pct": 0.049554013875123884,
117
+ "wer_raw": 0.11927528938097634,
118
+ "wer_clean": 0.11877201811776547,
119
+ "deleted_examples": [
120
+ "elo"
121
+ ],
122
+ "pass": true
123
+ },
124
+ "it": {
125
+ "clips": 100,
126
+ "deleted_pct": 0.05672149744753262,
127
+ "wer_raw": 0.16657381615598885,
128
+ "wer_clean": 0.1671309192200557,
129
+ "deleted_examples": [
130
+ "basso,"
131
+ ],
132
+ "pass": true
133
+ },
134
+ "nl": {
135
+ "clips": 100,
136
+ "deleted_pct": 0.18552875695732837,
137
+ "wer_raw": 0.30062530062530063,
138
+ "wer_clean": 0.30014430014430016,
139
+ "deleted_examples": [
140
+ "zes",
141
+ "soort",
142
+ "een",
143
+ "gewoon"
144
+ ],
145
+ "pass": true
146
+ },
147
+ "pl": {
148
+ "clips": 100,
149
+ "deleted_pct": 0.29086678301337987,
150
+ "wer_raw": 0.4161603888213852,
151
+ "wer_clean": 0.4161603888213852,
152
+ "deleted_examples": [
153
+ "tak,",
154
+ "Tak",
155
+ "poda",
156
+ "poda,",
157
+ "je,"
158
+ ],
159
+ "pass": true
160
+ },
161
+ "ko": {
162
+ "clips": 100,
163
+ "deleted_pct": 0.27002700270027,
164
+ "wer_raw": 0.42800114057599087,
165
+ "wer_clean": 0.4294268605645851,
166
+ "deleted_examples": [
167
+ "이제",
168
+ "계정과",
169
+ "사람과"
170
+ ],
171
+ "pass": true
172
+ }
173
+ },
174
+ "G3": {
175
+ "de/B": {
176
+ "rows": 87,
177
+ "delete_f1": 0.0,
178
+ "delete_precision": 0.0,
179
+ "delete_recall": 0.0,
180
+ "punct_acc": 0.9661266568483063,
181
+ "case_acc": 0.9783705839942322
182
+ },
183
+ "de/C": {
184
+ "rows": 86,
185
+ "delete_f1": 0.9838337182448037,
186
+ "delete_precision": 0.9681818181818181,
187
+ "delete_recall": 1.0,
188
+ "punct_acc": 0.9675226586102719,
189
+ "case_acc": 0.9763663220088626,
190
+ "weak": false
191
+ },
192
+ "en/A": {
193
+ "rows": 231,
194
+ "delete_f1": 0.7516339869281046,
195
+ "delete_precision": 0.749185667752443,
196
+ "delete_recall": 0.7540983606557377,
197
+ "punct_acc": 0.9229443623830625,
198
+ "case_acc": 0.9723174914257717
199
+ },
200
+ "en/B": {
201
+ "rows": 86,
202
+ "delete_f1": 0.0,
203
+ "delete_precision": 0.0,
204
+ "delete_recall": 0.0,
205
+ "punct_acc": 0.9594742606790799,
206
+ "case_acc": 0.985373781148429
207
+ },
208
+ "en/C": {
209
+ "rows": 85,
210
+ "delete_f1": 0.9811320754716981,
211
+ "delete_precision": 0.966542750929368,
212
+ "delete_recall": 0.9961685823754789,
213
+ "punct_acc": 0.9552733296521259,
214
+ "case_acc": 0.9841703056768559,
215
+ "weak": false
216
+ },
217
+ "es/B": {
218
+ "rows": 66,
219
+ "delete_f1": 0.0,
220
+ "delete_precision": 0.0,
221
+ "delete_recall": 0.0,
222
+ "punct_acc": 0.9549707602339181,
223
+ "case_acc": 0.9825885084155542
224
+ },
225
+ "es/C": {
226
+ "rows": 65,
227
+ "delete_f1": 0.9710144927536232,
228
+ "delete_precision": 0.9781021897810219,
229
+ "delete_recall": 0.9640287769784173,
230
+ "punct_acc": 0.9550970873786407,
231
+ "case_acc": 0.9819494584837545,
232
+ "weak": false
233
+ },
234
+ "fr/B": {
235
+ "rows": 88,
236
+ "delete_f1": 0.0,
237
+ "delete_precision": 0.0,
238
+ "delete_recall": 0.0,
239
+ "punct_acc": 0.9582753824756607,
240
+ "case_acc": 0.9953917050691244
241
+ },
242
+ "fr/C": {
243
+ "rows": 87,
244
+ "delete_f1": 0.9699769053117783,
245
+ "delete_precision": 0.9459459459459459,
246
+ "delete_recall": 0.995260663507109,
247
+ "punct_acc": 0.9608026131591227,
248
+ "case_acc": 0.9939703153988868,
249
+ "weak": false
250
+ },
251
+ "it/B": {
252
+ "rows": 65,
253
+ "delete_f1": 0.0,
254
+ "delete_precision": 0.0,
255
+ "delete_recall": 0.0,
256
+ "punct_acc": 0.9494415049970606,
257
+ "case_acc": 0.9936342592592593
258
+ },
259
+ "it/C": {
260
+ "rows": 63,
261
+ "delete_f1": 0.9686411149825784,
262
+ "delete_precision": 0.9586206896551724,
263
+ "delete_recall": 0.9788732394366197,
264
+ "punct_acc": 0.9510703363914373,
265
+ "case_acc": 0.9933734939759036,
266
+ "weak": false
267
+ },
268
+ "ko/B": {
269
+ "rows": 158,
270
+ "delete_f1": 0.0,
271
+ "delete_precision": 0.0,
272
+ "delete_recall": 0.0,
273
+ "punct_acc": 0.947136563876652,
274
+ "case_acc": 0.9995583038869258
275
+ },
276
+ "ko/C": {
277
+ "rows": 148,
278
+ "delete_f1": 0.9793977812995246,
279
+ "delete_precision": 0.9747634069400631,
280
+ "delete_recall": 0.9840764331210191,
281
+ "punct_acc": 0.9454976303317536,
282
+ "case_acc": 0.9995318352059925,
283
+ "weak": false
284
+ },
285
+ "nl/B": {
286
+ "rows": 62,
287
+ "delete_f1": 0.0,
288
+ "delete_precision": 0.0,
289
+ "delete_recall": 0.0,
290
+ "punct_acc": 0.9695945945945946,
291
+ "case_acc": 0.9917763157894737
292
+ },
293
+ "nl/C": {
294
+ "rows": 62,
295
+ "delete_f1": 0.9517241379310345,
296
+ "delete_precision": 0.92,
297
+ "delete_recall": 0.9857142857142858,
298
+ "punct_acc": 0.9670608108108109,
299
+ "case_acc": 0.990953947368421,
300
+ "weak": false
301
+ },
302
+ "pl/B": {
303
+ "rows": 73,
304
+ "delete_f1": 0.0,
305
+ "delete_precision": 0.0,
306
+ "delete_recall": 0.0,
307
+ "punct_acc": 0.9626865671641791,
308
+ "case_acc": 0.983633387888707
309
+ },
310
+ "pl/C": {
311
+ "rows": 73,
312
+ "delete_f1": 0.9897435897435898,
313
+ "delete_precision": 0.9796954314720813,
314
+ "delete_recall": 1.0,
315
+ "punct_acc": 0.9643449419568823,
316
+ "case_acc": 0.9787581699346405,
317
+ "weak": false
318
+ }
319
+ },
320
+ "pass": false
321
+ }
results/train.json ADDED
@@ -0,0 +1,78 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "history": [
3
+ {
4
+ "delete_f1": 0.886832363828662,
5
+ "delete_precision": 0.8431372549019608,
6
+ "delete_recall": 0.9353039598438372,
7
+ "false_delete_rate": 0.009375,
8
+ "punct_macro_f1": 0.577000537751614,
9
+ "case_acc": 0.9843201041221619,
10
+ "score": 0.8160510019008127,
11
+ "step": 132
12
+ },
13
+ {
14
+ "delete_f1": 0.9208275862068965,
15
+ "delete_precision": 0.9110262008733624,
16
+ "delete_recall": 0.9308421639709983,
17
+ "false_delete_rate": 0.004897836538461538,
18
+ "punct_macro_f1": 0.617543747215973,
19
+ "case_acc": 0.9879997372627258,
20
+ "score": 0.8421236902285317,
21
+ "step": 264
22
+ },
23
+ {
24
+ "delete_f1": 0.9287292817679558,
25
+ "delete_precision": 0.9200875752599891,
26
+ "delete_recall": 0.9375348577802566,
27
+ "false_delete_rate": 0.004387019230769231,
28
+ "punct_macro_f1": 0.6114112442431484,
29
+ "case_acc": 0.9879002571105957,
30
+ "score": 0.8426802610405666,
31
+ "step": 396
32
+ },
33
+ {
34
+ "delete_f1": 0.928709055876686,
35
+ "delete_precision": 0.9168478260869565,
36
+ "delete_recall": 0.9408812046848857,
37
+ "false_delete_rate": 0.004597355769230769,
38
+ "punct_macro_f1": 0.7164910954761785,
39
+ "case_acc": 0.988927960395813,
40
+ "score": 0.8780427039162259,
41
+ "step": 528
42
+ },
43
+ {
44
+ "delete_f1": 0.9299750899529476,
45
+ "delete_precision": 0.9230769230769231,
46
+ "delete_recall": 0.9369771332961517,
47
+ "false_delete_rate": 0.004206730769230769,
48
+ "punct_macro_f1": 0.7014204620419096,
49
+ "case_acc": 0.9888947606086731,
50
+ "score": 0.8734301042011768,
51
+ "step": 660
52
+ },
53
+ {
54
+ "delete_f1": 0.9308909242298085,
55
+ "delete_precision": 0.9265193370165746,
56
+ "delete_recall": 0.9353039598438372,
57
+ "false_delete_rate": 0.0039963942307692304,
58
+ "punct_macro_f1": 0.6994555898127952,
59
+ "case_acc": 0.9884969592094421,
60
+ "score": 0.8729478244173485,
61
+ "step": 792
62
+ },
63
+ {
64
+ "delete_f1": 0.9308909242298085,
65
+ "delete_precision": 0.9265193370165746,
66
+ "delete_recall": 0.9353039598438372,
67
+ "false_delete_rate": 0.0039963942307692304,
68
+ "punct_macro_f1": 0.6995397903507097,
69
+ "case_acc": 0.9884969592094421,
70
+ "score": 0.87297589126332,
71
+ "step": 795
72
+ }
73
+ ],
74
+ "best_score": 0.8780427039162259,
75
+ "steps": 795,
76
+ "delete_weight": 4.308142190372098,
77
+ "seconds": 2693.4392421245575
78
+ }
tagger/config.json ADDED
@@ -0,0 +1,79 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "ModernBertModel"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 2,
8
+ "classifier_activation": "gelu",
9
+ "classifier_bias": false,
10
+ "classifier_dropout": 0.0,
11
+ "classifier_pooling": "mean",
12
+ "cls_token_id": 1,
13
+ "decoder_bias": true,
14
+ "deterministic_flash_attn": false,
15
+ "dtype": "float32",
16
+ "embedding_dropout": 0.0,
17
+ "eos_token_id": 1,
18
+ "global_attn_every_n_layers": 3,
19
+ "gradient_checkpointing": false,
20
+ "hidden_activation": "gelu",
21
+ "hidden_size": 768,
22
+ "initializer_cutoff_factor": 2.0,
23
+ "initializer_range": 0.02,
24
+ "intermediate_size": 1152,
25
+ "layer_norm_eps": 1e-05,
26
+ "layer_types": [
27
+ "full_attention",
28
+ "sliding_attention",
29
+ "sliding_attention",
30
+ "full_attention",
31
+ "sliding_attention",
32
+ "sliding_attention",
33
+ "full_attention",
34
+ "sliding_attention",
35
+ "sliding_attention",
36
+ "full_attention",
37
+ "sliding_attention",
38
+ "sliding_attention",
39
+ "full_attention",
40
+ "sliding_attention",
41
+ "sliding_attention",
42
+ "full_attention",
43
+ "sliding_attention",
44
+ "sliding_attention",
45
+ "full_attention",
46
+ "sliding_attention",
47
+ "sliding_attention",
48
+ "full_attention"
49
+ ],
50
+ "local_attention": 128,
51
+ "mask_token_id": 4,
52
+ "max_position_embeddings": 8192,
53
+ "mlp_bias": false,
54
+ "mlp_dropout": 0.0,
55
+ "model_type": "modernbert",
56
+ "norm_bias": false,
57
+ "norm_eps": 1e-05,
58
+ "num_attention_heads": 12,
59
+ "num_hidden_layers": 22,
60
+ "pad_token_id": 0,
61
+ "position_embedding_type": "sans_pos",
62
+ "repad_logits_with_grad": false,
63
+ "rope_parameters": {
64
+ "full_attention": {
65
+ "rope_theta": 160000,
66
+ "rope_type": "default"
67
+ },
68
+ "sliding_attention": {
69
+ "rope_theta": 160000,
70
+ "rope_type": "default"
71
+ }
72
+ },
73
+ "sep_token_id": 1,
74
+ "sparse_pred_ignore_index": -100,
75
+ "sparse_prediction": false,
76
+ "tie_word_embeddings": true,
77
+ "transformers_version": "5.23.0",
78
+ "vocab_size": 256000
79
+ }
tagger/head.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:24a473e01162ce9a2ad0586983d8d9b6c6cd6611e62a6a169b7400667cecb8b9
3
+ size 30896
tagger/labels.json ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "delete": [
3
+ "keep",
4
+ "delete"
5
+ ],
6
+ "punct": [
7
+ "",
8
+ ",",
9
+ ".",
10
+ "?",
11
+ "!"
12
+ ],
13
+ "case": [
14
+ "keep",
15
+ "upper_first",
16
+ "lower_first"
17
+ ],
18
+ "layout": "logits[0:2] delete, [2:7] punct, [7:10] case",
19
+ "label_token": "first sub-token of each whitespace-separated word",
20
+ "step": 528,
21
+ "validation": {
22
+ "delete_f1": 0.928709055876686,
23
+ "delete_precision": 0.9168478260869565,
24
+ "delete_recall": 0.9408812046848857,
25
+ "false_delete_rate": 0.004597355769230769,
26
+ "punct_macro_f1": 0.7164910954761785,
27
+ "case_acc": 0.988927960395813,
28
+ "score": 0.8780427039162259,
29
+ "step": 528
30
+ },
31
+ "base": "convaiinnovations/laya-multilingual encoder",
32
+ "delete_threshold": 0.5,
33
+ "delete_threshold_rule": "smallest of 0.5..0.95 with <= 0.5% of fluent validation words deleted in every language",
34
+ "punct_mode": "boundary",
35
+ "punct_mode_rule": "chosen on DisfluencySpeech validation (scripts/punct_modes.py): full re-punctuation scored below Fleet-2's own marks there; 'full' is available as an option"
36
+ }
tagger/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:627aced6cf34dbff1c35abc80d5119d6777db612f31d4456070efdd8c985891c
3
+ size 1227771776
tagger/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f34b44a6dc8a0eef651bfa10e8325057530152606a69d9c3b6ae02efb1558f0a
3
+ size 34363286
tagger/tokenizer_config.json ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "bos_token": "<bos>",
4
+ "clean_up_tokenization_spaces": false,
5
+ "cls_token": "<bos>",
6
+ "eos_token": "<eos>",
7
+ "extra_special_tokens": [
8
+ "<start_of_turn>",
9
+ "<end_of_turn>"
10
+ ],
11
+ "is_local": true,
12
+ "local_files_only": false,
13
+ "mask_token": "<mask>",
14
+ "model_input_names": [
15
+ "input_ids",
16
+ "attention_mask"
17
+ ],
18
+ "model_max_length": 8192,
19
+ "pad_token": "<pad>",
20
+ "padding_side": "right",
21
+ "sep_token": "<eos>",
22
+ "spaces_between_special_tokens": false,
23
+ "tokenizer_class": "TokenizersBackend",
24
+ "unk_token": "<unk>"
25
+ }
whistle.cact ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5379f387fed44c6093a627d124a67dc0a5dc40a1e2bc41f270d041e5b6662898
3
+ size 30009274