WIlfLin commited on
Commit
2737bcb
·
verified ·
1 Parent(s): 457ddf8

Release S1-mini Chinese-data full fine-tuning and model card

Browse files
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
LICENSE ADDED
@@ -0,0 +1,217 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Copyright 2026 Superwhisper
2
+
3
+ This model is a derivative work of Qwen3-0.6B, Copyright 2024 Alibaba Cloud,
4
+ licensed under the Apache License, Version 2.0.
5
+
6
+ Apache License
7
+ Version 2.0, January 2004
8
+ http://www.apache.org/licenses/
9
+
10
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
11
+
12
+ 1. Definitions.
13
+
14
+ "License" shall mean the terms and conditions for use, reproduction,
15
+ and distribution as defined by Sections 1 through 9 of this document.
16
+
17
+ "Licensor" shall mean the copyright owner or entity authorized by
18
+ the copyright owner that is granting the License.
19
+
20
+ "Legal Entity" shall mean the union of the acting entity and all
21
+ other entities that control, are controlled by, or are under common
22
+ control with that entity. For the purposes of this definition,
23
+ "control" means (i) the power, direct or indirect, to cause the
24
+ direction or management of such entity, whether by contract or
25
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
26
+ outstanding shares, or (iii) beneficial ownership of such entity.
27
+
28
+ "You" (or "Your") shall mean an individual or Legal Entity
29
+ exercising permissions granted by this License.
30
+
31
+ "Source" form shall mean the preferred form for making modifications,
32
+ including but not limited to software source code, documentation
33
+ source, and configuration files.
34
+
35
+ "Object" form shall mean any form resulting from mechanical
36
+ transformation or translation of a Source form, including but
37
+ not limited to compiled object code, generated documentation,
38
+ and conversions to other media types.
39
+
40
+ "Work" shall mean the work of authorship, whether in Source or
41
+ Object form, made available under the License, as indicated by a
42
+ copyright notice that is included in or attached to the work
43
+ (an example is provided in the Appendix below).
44
+
45
+ "Derivative Works" shall mean any work, whether in Source or Object
46
+ form, that is based on (or derived from) the Work and for which the
47
+ editorial revisions, annotations, elaborations, or other modifications
48
+ represent, as a whole, an original work of authorship. For the purposes
49
+ of this License, Derivative Works shall not include works that remain
50
+ separable from, or merely link (or bind by name) to the interfaces of,
51
+ the Work and Derivative Works thereof.
52
+
53
+ "Contribution" shall mean any work of authorship, including
54
+ the original version of the Work and any modifications or additions
55
+ to that Work or Derivative Works thereof, that is intentionally
56
+ submitted to Licensor for inclusion in the Work by the copyright owner
57
+ or by an individual or Legal Entity authorized to submit on behalf of
58
+ the copyright owner. For the purposes of this definition, "submitted"
59
+ means any form of electronic, verbal, or written communication sent
60
+ to the Licensor or its representatives, including but not limited to
61
+ communication on electronic mailing lists, source code control systems,
62
+ and issue tracking systems that are managed by, or on behalf of, the
63
+ Licensor for the purpose of discussing and improving the Work, but
64
+ excluding communication that is conspicuously marked or otherwise
65
+ designated in writing by the copyright owner as "Not a Contribution."
66
+
67
+ "Contributor" shall mean Licensor and any individual or Legal Entity
68
+ on behalf of whom a Contribution has been received by Licensor and
69
+ subsequently incorporated within the Work.
70
+
71
+ 2. Grant of Copyright License. Subject to the terms and conditions of
72
+ this License, each Contributor hereby grants to You a perpetual,
73
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
74
+ copyright license to reproduce, prepare Derivative Works of,
75
+ publicly display, publicly perform, sublicense, and distribute the
76
+ Work and such Derivative Works in Source or Object form.
77
+
78
+ 3. Grant of Patent License. Subject to the terms and conditions of
79
+ this License, each Contributor hereby grants to You a perpetual,
80
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
81
+ (except as stated in this section) patent license to make, have made,
82
+ use, offer to sell, sell, import, and otherwise transfer the Work,
83
+ where such license applies only to those patent claims licensable
84
+ by such Contributor that are necessarily infringed by their
85
+ Contribution(s) alone or by combination of their Contribution(s)
86
+ with the Work to which such Contribution(s) was submitted. If You
87
+ institute patent litigation against any entity (including a
88
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
89
+ or a Contribution incorporated within the Work constitutes direct
90
+ or contributory patent infringement, then any patent licenses
91
+ granted to You under this License for that Work shall terminate
92
+ as of the date such litigation is filed.
93
+
94
+ 4. Redistribution. You may reproduce and distribute copies of the
95
+ Work or Derivative Works thereof in any medium, with or without
96
+ modifications, and in Source or Object form, provided that You
97
+ meet the following conditions:
98
+
99
+ (a) You must give any other recipients of the Work or
100
+ Derivative Works a copy of this License; and
101
+
102
+ (b) You must cause any modified files to carry prominent notices
103
+ stating that You changed the files; and
104
+
105
+ (c) You must retain, in the Source form of any Derivative Works
106
+ that You distribute, all copyright, patent, trademark, and
107
+ attribution notices from the Source form of the Work,
108
+ excluding those notices that do not pertain to any part of
109
+ the Derivative Works; and
110
+
111
+ (d) If the Work includes a "NOTICE" text file as part of its
112
+ distribution, then any Derivative Works that You distribute must
113
+ include a readable copy of the attribution notices contained
114
+ within such NOTICE file, excluding those notices that do not
115
+ pertain to any part of the Derivative Works, in at least one
116
+ of the following places: within a NOTICE text file distributed
117
+ as part of the Derivative Works; within the Source form or
118
+ documentation, if provided along with the Derivative Works; or,
119
+ within a display generated by the Derivative Works, if and
120
+ wherever such third-party notices normally appear. The contents
121
+ of the NOTICE file are for informational purposes only and
122
+ do not modify the License. You may add Your own attribution
123
+ notices within Derivative Works that You distribute, alongside
124
+ or as an addendum to the NOTICE text from the Work, provided
125
+ that such additional attribution notices cannot be construed
126
+ as modifying the License.
127
+
128
+ You may add Your own copyright statement to Your modifications and
129
+ may provide additional or different license terms and conditions
130
+ for use, reproduction, or distribution of Your modifications, or
131
+ for any such Derivative Works as a whole, provided Your use,
132
+ reproduction, and distribution of the Work otherwise complies with
133
+ the conditions stated in this License.
134
+
135
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
136
+ any Contribution intentionally submitted for inclusion in the Work
137
+ by You to the Licensor shall be under the terms and conditions of
138
+ this License, without any additional terms or conditions.
139
+ Notwithstanding the above, nothing herein shall supersede or modify
140
+ the terms of any separate license agreement you may have executed
141
+ with Licensor regarding such Contributions.
142
+
143
+ 6. Trademarks. This License does not grant permission to use the trade
144
+ names, trademarks, service marks, or product names of the Licensor,
145
+ except as required for reasonable and customary use in describing the
146
+ origin of the Work and reproducing the content of the NOTICE file.
147
+
148
+ 7. Disclaimer of Warranty. Unless required by applicable law or
149
+ agreed to in writing, Licensor provides the Work (and each
150
+ Contributor provides its Contributions) on an "AS IS" BASIS,
151
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
152
+ implied, including, without limitation, any warranties or conditions
153
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
154
+ PARTICULAR PURPOSE. You are solely responsible for determining the
155
+ appropriateness of using or redistributing the Work and assume any
156
+ risks associated with Your exercise of permissions under this License.
157
+
158
+ 8. Limitation of Liability. In no event and under no legal theory,
159
+ whether in tort (including negligence), contract, or otherwise,
160
+ unless required by applicable law (such as deliberate and grossly
161
+ negligent acts) or agreed to in writing, shall any Contributor be
162
+ liable to You for damages, including any direct, indirect, special,
163
+ incidental, or consequential damages of any character arising as a
164
+ result of this License or out of the use or inability to use the
165
+ Work (including but not limited to damages for loss of goodwill,
166
+ work stoppage, computer failure or malfunction, or any and all
167
+ other commercial damages or losses), even if such Contributor
168
+ has been advised of the possibility of such damages.
169
+
170
+ 9. Accepting Warranty or Additional Liability. While redistributing
171
+ the Work or Derivative Works thereof, You may choose to offer,
172
+ and charge a fee for, acceptance of support, warranty, indemnity,
173
+ or other liability obligations and/or rights consistent with this
174
+ License. However, in accepting such obligations, You may act only
175
+ on Your own behalf and on Your sole responsibility, not on behalf
176
+ of any other Contributor, and only if You agree to indemnify,
177
+ defend, and hold each Contributor harmless for any liability
178
+ incurred by, or claims asserted against, such Contributor by reason
179
+ of your accepting any such warranty or additional liability.
180
+
181
+ END OF TERMS AND CONDITIONS
182
+
183
+ APPENDIX: How to apply the Apache License to your work.
184
+
185
+ To apply the Apache License to your work, attach the following
186
+ boilerplate notice, with the fields enclosed by brackets "[]"
187
+ replaced with your own identifying information. (Don't include
188
+ the brackets!) The text should be enclosed in the appropriate
189
+ comment syntax for the file format. We also recommend that a
190
+ file or class name and description of purpose be included on the
191
+ same "printed page" as the copyright notice for easier
192
+ identification within third-party archives.
193
+
194
+ Copyright [yyyy] [name of copyright owner]
195
+
196
+ Licensed under the Apache License, Version 2.0 (the "License");
197
+ you may not use this file except in compliance with the License.
198
+ You may obtain a copy of the License at
199
+
200
+ http://www.apache.org/licenses/LICENSE-2.0
201
+
202
+ Unless required by applicable law or agreed to in writing, software
203
+ distributed under the License is distributed on an "AS IS" BASIS,
204
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
205
+ See the License for the specific language governing permissions and
206
+ limitations under the License.
207
+
208
+ ------------------------------------------------------------------------
209
+
210
+ ADDITIONAL TERM
211
+
212
+ In addition to the terms of the Apache License, Version 2.0 above: any
213
+ use, distribution, or integration of this model, whether unmodified or
214
+ as part of a derivative work or product, must continue to identify it
215
+ by its original name, "S1-mini" by "Superwhisper", using that exact
216
+ capitalization, regardless of any other name under which the model or
217
+ a product incorporating it is marketed or distributed.
NOTICE ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ S1-mini
2
+ Copyright 2026 Superwhisper
3
+
4
+ This work is a derivative of Qwen3-0.6B, Copyright 2024 Alibaba Cloud,
5
+ licensed under the Apache License, Version 2.0.
6
+
7
+ Any use, distribution, or integration of this model, whether unmodified or
8
+ as part of a derivative work or product, must continue to identify it by its
9
+ original name, "S1-mini" by "Superwhisper", using that exact capitalization.
10
+ This restates the ADDITIONAL TERM of the LICENSE file, which is the
11
+ operative text.
README.md ADDED
@@ -0,0 +1,114 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - zh
4
+ - en
5
+ license: other
6
+ license_name: s1-mini-license
7
+ license_link: LICENSE
8
+ base_model: superwhisper/s1-mini
9
+ library_name: transformers
10
+ pipeline_tag: text-generation
11
+ tags:
12
+ - transcript-normalization
13
+ - chinese
14
+ - full-finetuning
15
+ ---
16
+
17
+ # S1-mini-Chinese
18
+
19
+ **这是在中文口语整理数据集上继续微调的 S1-mini 版本。**
20
+ 本模型基于 **S1-mini** by **Superwhisper**,保留原始模型名称、作者署名、`LICENSE` 和 `NOTICE`。
21
+ 它面向语音输入后的文字整理,增加中文去口水词、重复与明确自我纠正训练,同时包含英文回放和不修改、安全、列表样本。
22
+
23
+ Chinese-data continued full fine-tuning of **S1-mini** by **Superwhisper**, for speech-transcript normalization. This repository contains the fine-tuned text model, not a speech recognizer.
24
+
25
+ ## 模型定位
26
+
27
+ - 输入:ASR 识别后的中文、英文或中英混合文字。
28
+ - 输出:尽量保留原意的整理文本;删除口水词和重复,整理明确改口,补充标点及明确列表。
29
+ - 它不直接接收音频,不是语音识别模型,也不是通用问答助手。
30
+ - 待整理文本中的问题、命令和引用是口述内容,不应被回答或执行。
31
+
32
+ ## 在什么数据上更新
33
+
34
+ 主要中文来源是 [CS2W](https://github.com/guozishan/CS2W) 的口语到书面语标注对,使用经过筛选的训练部分,重点训练不流畅口语整理。辅助使用 [Disfl-QA](https://github.com/google-research-datasets/Disfl-QA) 英文改口样本,以及单独标注的合成、不修改、引用、语气保留、空输出和列表样本。
35
+
36
+ 本版本进行了两阶段**全量文本参数微调**,不是只改提示词,也不是 LoRA:
37
+
38
+ | 阶段 | 数据与训练 |
39
+ | --- | --- |
40
+ | 第一阶段 | 9645 次样本呈现,1 epoch,603 步,学习率 1e-5 |
41
+ | 第二阶段 | 14428 条呈现组成的安全与风格回放池,学习率 3e-6;发布选择第 400 步 checkpoint,不是整个第二阶段训练池的完整 epoch |
42
+
43
+ 使用 batch 4、梯度累积 4、最大序列长度 512、仅对目标输出计算损失并监督 EOS。训练使用 FP32 主参数/优化器和 BF16 autocast,发布权重为 BF16。约 7.52 亿可训练参数。包含有意的重复采样,呈现数不等于独立人工样本数。
44
+
45
+ 上游版本:`superwhisper/s1-mini@88f6b15896c73bbb13a3b596e0afe8ea0d5150b4`。
46
+ 具体参数、训练数据哈希和来源版本见 [`training-provenance.json`](training-provenance.json) 与 [`data-provenance.json`](data-provenance.json)。未使用用户私有录音或私人输入历史。
47
+
48
+ ## 文本测试结果
49
+
50
+ 冻结的公开来源测试包含中文整理 200 条、中文不修改参考 100 条和英文 Disfl-QA 100 条。以下为**原始模型输出**,不包含线上保护校验:
51
+
52
+ | 指标 | 上游 S1-mini | S1-mini-Chinese |
53
+ | --- | ---: | ---: |
54
+ | 中文整理:去标点后整句完全匹配 | 46/200 | 82/200 |
55
+ | 中文整理:词法字符错误率,越低越好 | 21.06% | 17.22% |
56
+ | 英文整理:去标点后整句完全匹配 | 23/100 | 64/100 |
57
+ | 英文整理:词法字符错误率,越低越好 | 26.17% | 9.02% |
58
+ | 中文不修改参考:词法完全一致 | 96/100 | 92/100 |
59
+
60
+ 词法指标删除标点,保留大小写、数字、否定和词序。它反映与参考文本的一致程度,不等于人工语义正确率。英文集侧重复杂改口,不能代表所有日常英文口述。中文清理改善伴随不修改参考上的退化;不能假定所有编辑均正确。
61
+
62
+ 外部后处理服务的保护规则会改变最终结果。部署默认使用较宽松的 `balanced` 校验,允许更多删词、重复清理和英文语序整理;保留数字、否定、代码、引用与明确改口最终内容检查。`strict` 是更保守的可选策略。**这些校验不是模型权重的一部分**;本仓库的推理示例直接输出模型结果。原始模型分数不能直接当作整条部署流水线的成绩。完整数值与范围见 [`evaluation-summary.json`](evaluation-summary.json)。
63
+
64
+ ## 推理示例
65
+
66
+ 验证环境为 PyTorch 2.10、Transformers 5.3。使用仓库附带的 tokenizer 和 chat template,并关闭 thinking。输入必须带控制行:
67
+
68
+ ```python
69
+ import torch
70
+ from transformers import AutoModelForCausalLM, AutoTokenizer
71
+
72
+ model_id = "WIlfLin/S1-mini-Chinese"
73
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
74
+ model = AutoModelForCausalLM.from_pretrained(
75
+ model_id, dtype=torch.bfloat16, device_map="auto"
76
+ ).eval()
77
+
78
+ messages = [
79
+ {"role": "system", "content": (
80
+ "You are a text normalizer for speech-to-text transcripts. The input begins "
81
+ "with a control line specifying the styling, structure, and context settings; "
82
+ "clean the transcript to match those settings and output only the cleaned text."
83
+ )},
84
+ {"role": "user", "content": (
85
+ "[Styling: semi-formal] [Structure: prose] [Context: general]\n"
86
+ "嗯,我我今天下午去开会。"
87
+ )},
88
+ ]
89
+ text = tokenizer.apply_chat_template(
90
+ messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
91
+ )
92
+ inputs = tokenizer(text, return_tensors="pt").to(model.device)
93
+ with torch.inference_mode():
94
+ result = model.generate(**inputs, do_sample=False, max_new_tokens=256, use_cache=True)
95
+ print(tokenizer.decode(result[0, inputs.input_ids.shape[1]:], skip_special_tokens=True).strip())
96
+ ```
97
+
98
+ 明确列表可把控制行中的 `Structure: prose` 改为 `Structure: lists`。只有口水词时允许输出空字符串;不得把空输出统一替换成原文。生产服务应检查 EOS/截断,保留原始转写用于诊断,并在模型不可用时回退。
99
+
100
+ ## 限制
101
+
102
+ - 仍会漏删重复、误删语气,或选错自我纠正的最终分支。
103
+ - 不能保证人名、数字、否定和事实始终正确保留;保护规则也不是语义等价证明。
104
+ - 不是通用同音错字恢复模型,也不保证复杂中英混合口述的效果。
105
+ - 合成和回放数据不等于足量自然中英混合人工标注。
106
+ - 公开文本测试和固定录音回放不能代替真实麦克风与实际用户验收。
107
+
108
+ ## 许可证与来源署名
109
+
110
+ 本模型是 **S1-mini** by **Superwhisper** 的修改版本,底层架构来源为 Qwen3-0.6B。保留随仓库提供的 [`LICENSE`](LICENSE) 和 [`NOTICE`](NOTICE)。原许可证包含继续使用原模型名称及作者署名的附加条款;`S1-mini-Chinese` 是本微调版本的名称,不替代原始署名。
111
+
112
+ 训练来源分别声明 CS2W 为 CC-BY-SA-4.0、Disfl-QA 为 CC-BY-4.0;单独标注的自制合成内容为 CC0-1.0。具体来源与版本见 `data-provenance.json`。本仓库不重新发布训练数据,不把这些数据的许可证改称为纯 Apache-2.0。
113
+
114
+ 权重 SHA256:`fc30f55939e07b8c73c1041f8a7a772e761ad70b448f5cf5b5cf5e38579351e4`。
chat_template.jinja ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- if tools %}
2
+ {{- '<|im_start|>system\n' }}
3
+ {%- if messages[0].role == 'system' %}
4
+ {{- messages[0].content + '\n\n' }}
5
+ {%- endif %}
6
+ {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
7
+ {%- for tool in tools %}
8
+ {{- "\n" }}
9
+ {{- tool | tojson }}
10
+ {%- endfor %}
11
+ {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
12
+ {%- else %}
13
+ {%- if messages[0].role == 'system' %}
14
+ {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
15
+ {%- endif %}
16
+ {%- endif %}
17
+ {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
18
+ {%- for message in messages[::-1] %}
19
+ {%- set index = (messages|length - 1) - loop.index0 %}
20
+ {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
21
+ {%- set ns.multi_step_tool = false %}
22
+ {%- set ns.last_query_index = index %}
23
+ {%- endif %}
24
+ {%- endfor %}
25
+ {%- for message in messages %}
26
+ {%- if message.content is string %}
27
+ {%- set content = message.content %}
28
+ {%- else %}
29
+ {%- set content = '' %}
30
+ {%- endif %}
31
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
32
+ {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
33
+ {%- elif message.role == "assistant" %}
34
+ {%- set reasoning_content = '' %}
35
+ {%- if message.reasoning_content is string %}
36
+ {%- set reasoning_content = message.reasoning_content %}
37
+ {%- else %}
38
+ {%- if '</think>' in content %}
39
+ {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
40
+ {%- set content = content.split('</think>')[-1].lstrip('\n') %}
41
+ {%- endif %}
42
+ {%- endif %}
43
+ {%- if loop.index0 > ns.last_query_index %}
44
+ {%- if loop.last or (not loop.last and reasoning_content) %}
45
+ {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
46
+ {%- else %}
47
+ {{- '<|im_start|>' + message.role + '\n' + content }}
48
+ {%- endif %}
49
+ {%- else %}
50
+ {{- '<|im_start|>' + message.role + '\n' + content }}
51
+ {%- endif %}
52
+ {%- if message.tool_calls %}
53
+ {%- for tool_call in message.tool_calls %}
54
+ {%- if (loop.first and content) or (not loop.first) %}
55
+ {{- '\n' }}
56
+ {%- endif %}
57
+ {%- if tool_call.function %}
58
+ {%- set tool_call = tool_call.function %}
59
+ {%- endif %}
60
+ {{- '<tool_call>\n{"name": "' }}
61
+ {{- tool_call.name }}
62
+ {{- '", "arguments": ' }}
63
+ {%- if tool_call.arguments is string %}
64
+ {{- tool_call.arguments }}
65
+ {%- else %}
66
+ {{- tool_call.arguments | tojson }}
67
+ {%- endif %}
68
+ {{- '}\n</tool_call>' }}
69
+ {%- endfor %}
70
+ {%- endif %}
71
+ {{- '<|im_end|>\n' }}
72
+ {%- elif message.role == "tool" %}
73
+ {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
74
+ {{- '<|im_start|>user' }}
75
+ {%- endif %}
76
+ {{- '\n<tool_response>\n' }}
77
+ {{- content }}
78
+ {{- '\n</tool_response>' }}
79
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
80
+ {{- '<|im_end|>\n' }}
81
+ {%- endif %}
82
+ {%- endif %}
83
+ {%- endfor %}
84
+ {%- if add_generation_prompt %}
85
+ {{- '<|im_start|>assistant\n' }}
86
+ {%- if enable_thinking is defined and enable_thinking is false %}
87
+ {{- '<think>\n\n</think>\n\n' }}
88
+ {%- endif %}
89
+ {%- endif %}
config.json ADDED
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Qwen3ForCausalLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 151643,
8
+ "dtype": "bfloat16",
9
+ "eos_token_id": 151645,
10
+ "head_dim": 128,
11
+ "hidden_act": "silu",
12
+ "hidden_size": 1024,
13
+ "initializer_range": 0.02,
14
+ "intermediate_size": 3072,
15
+ "layer_types": [
16
+ "full_attention",
17
+ "full_attention",
18
+ "full_attention",
19
+ "full_attention",
20
+ "full_attention",
21
+ "full_attention",
22
+ "full_attention",
23
+ "full_attention",
24
+ "full_attention",
25
+ "full_attention",
26
+ "full_attention",
27
+ "full_attention",
28
+ "full_attention",
29
+ "full_attention",
30
+ "full_attention",
31
+ "full_attention",
32
+ "full_attention",
33
+ "full_attention",
34
+ "full_attention",
35
+ "full_attention",
36
+ "full_attention",
37
+ "full_attention",
38
+ "full_attention",
39
+ "full_attention",
40
+ "full_attention",
41
+ "full_attention",
42
+ "full_attention",
43
+ "full_attention"
44
+ ],
45
+ "max_position_embeddings": 40960,
46
+ "max_window_layers": 28,
47
+ "model_type": "qwen3",
48
+ "num_attention_heads": 16,
49
+ "num_hidden_layers": 28,
50
+ "num_key_value_heads": 8,
51
+ "pad_token_id": null,
52
+ "rms_norm_eps": 1e-06,
53
+ "rope_parameters": {
54
+ "rope_theta": 1000000,
55
+ "rope_type": "default"
56
+ },
57
+ "sliding_window": null,
58
+ "tie_word_embeddings": false,
59
+ "transformers_version": "5.3.0",
60
+ "use_cache": true,
61
+ "use_sliding_window": false,
62
+ "vocab_size": 151936
63
+ }
data-provenance.json ADDED
@@ -0,0 +1,86 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "CS2W": {
3
+ "source": "https://github.com/guozishan/CS2W",
4
+ "revision": "be6e0519b433cfa9d8388b23308f1380716254fd",
5
+ "license_declared": "CC-BY-SA-4.0"
6
+ },
7
+ "Disfl-QA": {
8
+ "source": "https://github.com/google-research-datasets/Disfl-QA",
9
+ "revision": "1f0c16171c77b3d3408be92c485f11b8998a9189",
10
+ "license_declared": "CC-BY-4.0 for dataset"
11
+ },
12
+ "stage1_manifest": {
13
+ "stage": "pilot-v1-not-natural-mixed-gold",
14
+ "seed": 20261104,
15
+ "rows": 9645,
16
+ "sources": {
17
+ "authored-synthetic CC0-1.0": 2407,
18
+ "CS2W CC-BY-SA-4.0": 3983,
19
+ "CS2W-derived identity CC-BY-SA-4.0": 1653,
20
+ "Disfl-QA-derived identity CC-BY-4.0": 359,
21
+ "Disfl-QA CC-BY-4.0": 1200,
22
+ "authored CC0-1.0": 43
23
+ },
24
+ "tasks": {
25
+ "filler": 1326,
26
+ "cleanup": 5183,
27
+ "identity": 2596,
28
+ "empty": 325,
29
+ "mixed": 192,
30
+ "repair": 18,
31
+ "list": 5
32
+ },
33
+ "rejected": {
34
+ "cs2w_out_of_stage1_task": 1225,
35
+ "cs2w_identity_or_non_deletion_review": 545,
36
+ "heldout_near_jaccard_0.8": 22,
37
+ "heldout_exact_cross_side": 8,
38
+ "duplicate_pair": 79,
39
+ "conservative_disflqa_id_family": 862,
40
+ "conflicting_target_group_rows": 9
41
+ },
42
+ "source_sha256": {
43
+ "CS2W-train.jsonl": "acf249f7e8077337fc8e4029e7c381e117f3bc780fef48c820e4f328e78f563a",
44
+ "CS2W-val.jsonl": "1d89f155c7caecee9d63fc938c42226d43b52fe43f5644ee6c9190b38272ee9a",
45
+ "CS2W-test.jsonl": "32ea92028c3f990b304e2dc5838f487a86a7725b99c9f12fc9696446ecfaab80",
46
+ "Disfl-QA-train.json": "5407199d0c039de5b50cfc16d1ed4a3299c9127cb549da4e4a650b30f4e000eb",
47
+ "Disfl-QA-dev.json": "b60e075b810b27a5130fd0aa2cfbc85753b71a0b30dcd2585f540f0a6afe6492",
48
+ "Disfl-QA-test.json": "404801de916ebcb2caa82661dfd189c0520e2766db6838f6ff548088650e565e"
49
+ },
50
+ "development_rows": 328,
51
+ "evaluation_policy": "Official test excluded from training and development selection; acceptance set used for development, not independent final proof",
52
+ "limits": [
53
+ "Synthetic mixed/noise templates are not natural mixed-language gold",
54
+ "Target-to-target identities require semantic quality review",
55
+ "Deletion-only source filter does not itself guarantee semantic preservation",
56
+ "Disfl-QA 22-character ID grouping is conservative, not a confirmed source paragraph mapping"
57
+ ]
58
+ },
59
+ "stage2_manifest": {
60
+ "stage": "v2 safety replay",
61
+ "rows": 14428,
62
+ "tasks": {
63
+ "identity": 2596,
64
+ "list_identity": 500,
65
+ "cleanup": 5183,
66
+ "identity_replay": 2500,
67
+ "filler": 1325,
68
+ "mixed": 192,
69
+ "focus_identity": 64,
70
+ "quoted_identity": 384,
71
+ "empty": 325,
72
+ "list": 505,
73
+ "demonstrative_identity": 240,
74
+ "repetition_identity": 32,
75
+ "demonstrative_cleanup": 120,
76
+ "acknowledgement_identity": 132,
77
+ "emphasis_identity": 96,
78
+ "quoted_cleanup": 192,
79
+ "repair": 18,
80
+ "deictic_list_identity": 24
81
+ },
82
+ "synthetic_additions": 4796,
83
+ "limits": "Acknowledgement and safety repetitions are deliberate sampling weights; additions are synthetic, not independent human gold."
84
+ },
85
+ "notes": "Only filtered training splits and labelled authored/synthetic replay entered SFT. Evaluation splits excluded; exact cross-side and near-duplicate filtering applied as documented. Sampling repetitions are intentional. Dataset texts are not redistributed in this model repository."
86
+ }
evaluation-summary.json ADDED
@@ -0,0 +1,99 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "upstream_raw": {
3
+ "all": {
4
+ "n": 400,
5
+ "lexical_exact": 165,
6
+ "exact": 69,
7
+ "cer": 0.2098846787479407,
8
+ "truncated": 0
9
+ },
10
+ "test_en": {
11
+ "n": 100,
12
+ "lexical_exact": 23,
13
+ "exact": 18,
14
+ "cer": 0.26165355841166316,
15
+ "truncated": 0
16
+ },
17
+ "test_identity": {
18
+ "n": 100,
19
+ "lexical_exact": 96,
20
+ "exact": 38,
21
+ "cer": 0.0037907505686125853,
22
+ "truncated": 0
23
+ },
24
+ "test_zh": {
25
+ "n": 200,
26
+ "lexical_exact": 46,
27
+ "exact": 13,
28
+ "cer": 0.2106490478041197,
29
+ "truncated": 0
30
+ }
31
+ },
32
+ "chinese_finetuned_raw": {
33
+ "all": {
34
+ "n": 400,
35
+ "lexical_exact": 238,
36
+ "exact": 110,
37
+ "cer": 0.10236133992311916,
38
+ "truncated": 0
39
+ },
40
+ "test_en": {
41
+ "n": 100,
42
+ "lexical_exact": 64,
43
+ "exact": 57,
44
+ "cer": 0.09015921734126223,
45
+ "truncated": 0
46
+ },
47
+ "test_identity": {
48
+ "n": 100,
49
+ "lexical_exact": 92,
50
+ "exact": 38,
51
+ "cer": 0.014404852160727824,
52
+ "truncated": 0
53
+ },
54
+ "test_zh": {
55
+ "n": 200,
56
+ "lexical_exact": 82,
57
+ "exact": 15,
58
+ "cer": 0.1721725612125923,
59
+ "truncated": 0
60
+ }
61
+ },
62
+ "balanced_postprocessor_offline": {
63
+ "all": {
64
+ "n": 400,
65
+ "lexical_exact": 208,
66
+ "exact": 86,
67
+ "cer": 0.214497528830313,
68
+ "truncated": 0
69
+ },
70
+ "test_en": {
71
+ "n": 100,
72
+ "lexical_exact": 34,
73
+ "exact": 32,
74
+ "cer": 0.28697487051601767,
75
+ "truncated": 0
76
+ },
77
+ "test_identity": {
78
+ "n": 100,
79
+ "lexical_exact": 94,
80
+ "exact": 40,
81
+ "cer": 0.0075815011372251705,
82
+ "truncated": 0
83
+ },
84
+ "test_zh": {
85
+ "n": 200,
86
+ "lexical_exact": 80,
87
+ "exact": 14,
88
+ "cer": 0.1737271667314419,
89
+ "truncated": 0
90
+ }
91
+ },
92
+ "scope": "400 frozen public text rows: Chinese cleanup 200, Chinese identity 100, English Disfl-QA 100; lexical CER strips punctuation, not semantic accuracy. Postprocessor is separately calibrated and not an independent blind evaluation.",
93
+ "test_manifest": {
94
+ "rows": 400,
95
+ "seed": 20261107,
96
+ "sha256": "55d21ed3b91bd19093f6e4620d76565ec36afd9ad3c32a05c95f1e43a785fd88",
97
+ "selection": "200 Chinese cleanup + 100 disjoint Chinese target-to-target identity + 100 English cleanup, official test splits only. No subsequent tuning on this test."
98
+ }
99
+ }
generation_config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token_id": 151643,
3
+ "do_sample": false,
4
+ "eos_token_id": [
5
+ 151645,
6
+ 151643
7
+ ],
8
+ "pad_token_id": 151643,
9
+ "transformers_version": "5.3.0"
10
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fc30f55939e07b8c73c1041f8a7a772e761ad70b448f5cf5b5cf5e38579351e4
3
+ size 1503300328
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506
3
+ size 11422650
tokenizer_config.json ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": null,
5
+ "clean_up_tokenization_spaces": false,
6
+ "eos_token": "<|im_end|>",
7
+ "errors": "replace",
8
+ "extra_special_tokens": [
9
+ "<|im_start|>",
10
+ "<|im_end|>",
11
+ "<|object_ref_start|>",
12
+ "<|object_ref_end|>",
13
+ "<|box_start|>",
14
+ "<|box_end|>",
15
+ "<|quad_start|>",
16
+ "<|quad_end|>",
17
+ "<|vision_start|>",
18
+ "<|vision_end|>",
19
+ "<|vision_pad|>",
20
+ "<|image_pad|>",
21
+ "<|video_pad|>"
22
+ ],
23
+ "is_local": true,
24
+ "model_max_length": 131072,
25
+ "pad_token": "<|endoftext|>",
26
+ "split_special_tokens": false,
27
+ "tokenizer_class": "Qwen2Tokenizer",
28
+ "unk_token": null
29
+ }
training-provenance.json ADDED
@@ -0,0 +1,55 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "stage1": {
3
+ "model": "superwhisper/s1-mini",
4
+ "data": "train.jsonl",
5
+ "output": "S1-mini-Chinese stage1",
6
+ "style": "s1",
7
+ "epochs": 1,
8
+ "steps": 0,
9
+ "lr": 1e-05,
10
+ "micro_batch": 4,
11
+ "accumulation": 4,
12
+ "max_length": 512,
13
+ "seed": 20261104,
14
+ "save_every": 300,
15
+ "lora": false,
16
+ "base_model": "superwhisper/s1-mini",
17
+ "train_sha256": "bc3910d4e72a5bd79b11abfcbf37872c0cf86d7e533b2d4aeec79cfb1822bba7",
18
+ "rows": 9645,
19
+ "rejected_overlength": 0,
20
+ "eos_only_samples": 325,
21
+ "total_steps": 603,
22
+ "trainable_parameters": 751632384,
23
+ "precision": "FP32 master parameters and optimizer; BF16 CUDA autocast; BF16 inference export",
24
+ "python_torch": "2.10.0+cu128",
25
+ "completed_steps": 603,
26
+ "training_seconds": 164.68058762699366
27
+ },
28
+ "stage2_checkpoint": {
29
+ "model": "S1-mini-Chinese stage1",
30
+ "data": "train-safety-v2.jsonl",
31
+ "output": "S1-mini-Chinese stage2_checkpoint",
32
+ "style": "s1",
33
+ "epochs": 1,
34
+ "steps": 0,
35
+ "lr": 3e-06,
36
+ "micro_batch": 4,
37
+ "accumulation": 4,
38
+ "max_length": 512,
39
+ "seed": 20261104,
40
+ "save_every": 400,
41
+ "lora": false,
42
+ "base_model": "S1-mini-Chinese stage1",
43
+ "train_sha256": "819c6e80285423b3d8fa575ba7327244d03df6992440d77928d79b07f3dd7712",
44
+ "rows": 14428,
45
+ "rejected_overlength": 0,
46
+ "eos_only_samples": 325,
47
+ "total_steps": 902,
48
+ "trainable_parameters": 751632384,
49
+ "precision": "FP32 master parameters and optimizer; BF16 CUDA autocast; BF16 inference export",
50
+ "python_torch": "2.10.0+cu128",
51
+ "step": 400
52
+ },
53
+ "upstream_revision": "88f6b15896c73bbb13a3b596e0afe8ea0d5150b4",
54
+ "checkpoint_weights_sha256": "fc30f55939e07b8c73c1041f8a7a772e761ad70b448f5cf5b5cf5e38579351e4"
55
+ }