anocha-mcomp commited on
Commit
460a09f
·
verified ·
1 Parent(s): 5644614

Update model card from Confluence: Qwen-SEA-LION-v4-32B-IT

Browse files
Files changed (1) hide show
  1. qwen-sea-lion-v4-32b-it.md +8 -8
qwen-sea-lion-v4-32b-it.md CHANGED
@@ -16,7 +16,7 @@ Qwen-SEA-LION-v4-32B-IT inherits the following features from Qwen3-32B:
16
 
17
  SEA-LION stands for *Southeast Asian Languages In One Network*.
18
 
19
- We performed continued pre-training in English and SEA languages on Qwen3-32B, a decoder model using the Gemma 3 architecture, and post-training to create Qwen-SEA-LION-v4-32B-IT.
20
 
21
  For tokenization, the model employs the default tokenizer used in Qwen3-32B.
22
 
@@ -27,7 +27,7 @@ For tokenization, the model employs the default tokenizer used in Qwen3-32B.
27
  - **Context Length:** 128k tokens
28
  - **Language(s) (NLP):** Burmese, English, Indonesian, Khmer, Lao, Malay, Mandarin, Tagalog, Tamil, Thai, and Vietnamese
29
  - **License:** [Qwen Terms of Service](https://qwen.ai/termsservice) / [Qwen Usage Policy](https://qwen.ai/usagepolicy)
30
- - **Continue pretrained from model:** [Qwen-3-32B](https://huggingface.co/Qwen/Qwen3-32B)
31
 
32
  ## Uses
33
 
@@ -56,7 +56,7 @@ Use the code below to get started with the model using the 🤗 Transformers lib
56
  ```
57
  from transformers import AutoModelForCausalLM, AutoTokenizer
58
 
59
- model_name = "aisingapore/Qwen-SEA-LION-v4-32B"
60
 
61
  # load the tokenizer and the model
62
  tokenizer = AutoTokenizer.from_pretrained(model_name)
@@ -104,13 +104,13 @@ print("content:", content)
104
 
105
  **Training Datasets**
106
 
107
- The instruction fine-tuning dataset combines our SEA-Instruct, Infinity-Instruct, and OpenMath-Instruct 2 with open-source datasets. For the Online RL datasets, open sourced datasets such as nvidia/Llama-Nemotron-Post-Training-Dataset (RL set) and zwhe99/DeepMath-103K were used.
108
 
109
  #### Training Procedure
110
 
111
  **Training regime**
112
 
113
- Our post-training workflow consists of multiple stages: instruction fine-tuning, model merging, online RL for both instruction following and math using DRGPPO, and then followed by on-policy alignment via APO.
114
 
115
  ## Uses
116
 
@@ -130,7 +130,7 @@ Our post-training workflow consists of multiple stages: instruction fine-tuning,
130
 
131
  *Additional Remarks:*
132
 
133
- - TTFT and Toks per Sec: measured with vLLM on localhost and concurrency = 1.
134
  - Reported results are the median (p50) values, calculated across 10 requests.
135
  (11 requests were run and the first result was dropped, to eliminate cold-start delays)
136
  - Model size taken from vLLM upon loading
@@ -192,7 +192,7 @@ The following metrics were used:
192
 
193
  ### Results
194
 
195
- [Attachment: image-20251017-024331.png]Leaderboard snapshotted on 17 Oct 2025 10:44am
196
 
197
  For details on Qwen-SEA-LION-v4-27B-IT performance, please refer to the SEA-HELM leaderboard, <https://leaderboard.sea-lion.ai/> .
198
 
@@ -204,4 +204,4 @@ For more info, please contact us using this [sealion@aisingapore.org](mailto:sea
204
 
205
  ## Team
206
 
207
- Ahn Jeongmi, Antonyrex Sajeban, Chan Hok Teng Adwin, Cheng Zi Yi Nicholas, Choa Hsueh Mei Esther, Heng Jonathan, Huang Yuli, Hulagadri Adithya Venkatadri, Jann Railey Estrada Montalan, Kang Siow Wei Bryan, Lau Wayne, Lee Chwan Ren, Leong Wai Yi, Leong Wei Qi, Limkonchotiwat Peerat, Muhammad Ridzuan Bin Mokhtar, Nagarajan Karthik, Ng Boon Cheong Raymond, Ngee Chia Tai, Ngui Jian Gang, Nguyen Thanh Ngan, Ong Jin Jie Brandon, Ong Tat-Wee David, Ong Zhi Hao, Pereira Mark, Rengarajan Hamsawardhini, Susanto Yosephine, Sutaveephamochanon Anocha, Tan Choon Meng, Tan Chor Phin Evelyn, Tan Siao Wei Jessica, Tan Yixian, Tee Jun Yun, Teng Kok Wai Walter, Teo Eng Sipp Leslie, Tjhi William, Yeo Yeow Tong, Yong Xianbin, Zhang Zhou, Liew Rachel, Liu Bing Jie Darius, Zhou Lin (NCS), Gopalakrishnan Roshan (NCS), Anda Cuahtemoc (NCS), Sri Devi Wijaya (NCS), Nandi Partha (NCS)
 
16
 
17
  SEA-LION stands for *Southeast Asian Languages In One Network*.
18
 
19
+ We performed continued pre-training in English and SEA languages on Qwen3-32B, a decoder model using the Qwen3 architecture, and post-training to create Qwen-SEA-LION-v4-32B-IT.
20
 
21
  For tokenization, the model employs the default tokenizer used in Qwen3-32B.
22
 
 
27
  - **Context Length:** 128k tokens
28
  - **Language(s) (NLP):** Burmese, English, Indonesian, Khmer, Lao, Malay, Mandarin, Tagalog, Tamil, Thai, and Vietnamese
29
  - **License:** [Qwen Terms of Service](https://qwen.ai/termsservice) / [Qwen Usage Policy](https://qwen.ai/usagepolicy)
30
+ - **Continue pretrained from model:** [Qwen3-32B](https://huggingface.co/Qwen/Qwen3-32B)
31
 
32
  ## Uses
33
 
 
56
  ```
57
  from transformers import AutoModelForCausalLM, AutoTokenizer
58
 
59
+ model_name = "aisingapore/Qwen-SEA-LION-v4-32B-IT"
60
 
61
  # load the tokenizer and the model
62
  tokenizer = AutoTokenizer.from_pretrained(model_name)
 
104
 
105
  **Training Datasets**
106
 
107
+ The instruction fine-tuning dataset combines a collection of OSS & synthetic data.
108
 
109
  #### Training Procedure
110
 
111
  **Training regime**
112
 
113
+ Our post-training workflow consists of instruction fine-tuning and model merging.
114
 
115
  ## Uses
116
 
 
130
 
131
  *Additional Remarks:*
132
 
133
+ - TTFT and Tokens per Second: measured with vLLM on localhost and concurrency = 1.
134
  - Reported results are the median (p50) values, calculated across 10 requests.
135
  (11 requests were run and the first result was dropped, to eliminate cold-start delays)
136
  - Model size taken from vLLM upon loading
 
192
 
193
  ### Results
194
 
195
+ [Attachment: image-20251017-071644.png]Leaderboard snapshotted on 17 Oct 2025 10:44am
196
 
197
  For details on Qwen-SEA-LION-v4-27B-IT performance, please refer to the SEA-HELM leaderboard, <https://leaderboard.sea-lion.ai/> .
198
 
 
204
 
205
  ## Team
206
 
207
+ Ahn Jeongmi, Antonyrex Sajeban, Chan Hok Teng Adwin, Cheng Zi Yi Nicholas, Choa Hsueh Mei Esther, Heng Jonathan, Huang Yuli, Hulagadri Adithya Venkatadri, Jann Railey Estrada Montalan, Kang Siow Wei Bryan, Lau Wayne, Lee Chwan Ren, Leong Wai Yi, Leong Wei Qi, Limkonchotiwat Peerat, Muhammad Ridzuan Bin Mokhtar, Nagarajan Karthik, Ng Boon Cheong Raymond, Ngee Chia Tai, Ngui Jian Gang, Nguyen Thanh Ngan, Ong Jin Jie Brandon, Ong Tat-Wee David, Ong Zhi Hao, Pereira Mark, Rengarajan Hamsawardhini, Susanto Yosephine, Sutaveephamochanon Anocha, Tan Choon Meng, Tan Chor Phin Evelyn, Tan Siao Wei Jessica, Tan Yixian, Tee Jun Yun, Teng Kok Wai Walter, Teo Eng Sipp Leslie, Tjhi William, Yeo Yeow Tong, Yong Xianbin, Zhang Zhou, Liew Rachel, Liu Bing Jie Darius, Tep Kilian Rithi (GoTo)