anocha-mcomp commited on
Commit
5d274fd
·
verified ·
1 Parent(s): 140a91f

Update model card from Confluence: Model Cards: Qwen-SEA-LION-v4-4B-VL

Browse files
Files changed (1) hide show
  1. model_cards_qwen-sea-lion-v4-4b-vl.md +13 -16
model_cards_qwen-sea-lion-v4-4b-vl.md CHANGED
@@ -1,10 +1,10 @@
1
  # Qwen-SEA-LION-v4-4B-VL
2
 
3
- Last update: 2025-11-28
4
 
5
  SEA-LION is a collection of Large Language Models (LLMs) which have been pretrained and instruct-tuned for the Southeast Asia (SEA) region.
6
 
7
- **Qwen-SEA-LION-v4-4B-VL** is a 4-billion parameter Vision-Language Model (VLM) built upon the Qwen3-VL-4B-Instruct architecture. To ensure **domain adaptation** for the region, the model underwent rigorous supervised fine-tuning (SFT) on a curated dataset of approximately **8 million** instruction-image pairs. This extensive post-training instills **Multicultural** fluency, covering 7 key SEA languages: Burmese, Indonesian, Malay, Tagalog, Tamil, Thai, and Vietnamese.
8
 
9
  Qwen-SEA-LION-v4-4B-VL inherits the following features from Qwen3-VL:
10
 
@@ -19,7 +19,7 @@ Qwen-SEA-LION-v4-4B-VL inherits the following features from Qwen3-VL:
19
 
20
  SEA-LION stands for *Southeast Asian Languages In One Network*.
21
 
22
- We performed post-training in English and SEA languages on Qwen3-VL-4B-Instruct, a decoder model using the Qwen3 architecture, and post-training to create Qwen-SEA-LION-v4-4B-VL.
23
 
24
  For tokenization, the model employs the default tokenizer used in Qwen3-VL.
25
 
@@ -28,14 +28,13 @@ For tokenization, the model employs the default tokenizer used in Qwen3-VL.
28
  - **Shared by:** AI Products Pillar, AI Singapore
29
  - **Model type:** Decoder
30
  - **Context length:** 256k
31
- - **Language(s):** fine-tuned on Burmese, Indonesian, Malay, Tagalog, Tamil, Thai, and Vietnamese
32
  - **License:** [Apache-2.0](https://choosealicense.com/licenses/apache-2.0/)
33
  - **Finetuned from model:** [Qwen3-VL-4B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct)
34
 
35
  ### Model Sources
36
 
37
- - **Repository:** <https://huggingface.co/collections/aisingapore/sea-lion-v4>
38
- - **Demo:** [playground.sea-lion.ai](https://playground.sea-lion.ai/chat)
39
 
40
  ## Uses
41
 
@@ -168,25 +167,23 @@ The following metrics were used for text capabilities:
168
  | Natural Language Inference | Accuracy |
169
  | LINDSEA | Accuracy |
170
  | Global MMLU Lite | Accuracy |
171
- | Thai Exam | Accuracy |
172
  | Kalahi | Accuracy |
173
  | SEA-IFEval | Accuracy |
174
  | SEA-MTBench | Win rate against a reference |
175
 
176
  ### Results
177
 
178
- <<leaderboard_snapshot>>
179
-
180
- For details on Qwen-SEA-LION-v4-4B-VL performance, please refer to the SEA-HELM leaderboard, <https://leaderboard.sea-lion.ai/>.
181
 
182
  ### Retaining VL Capabilities
183
 
184
- We also evaluated the model on two types of tasks using datasets specifically focused on Southeast Asian examples to benchmark against the original base models (Qwen3-VL-4B/8B).
185
 
186
- - Visual Question Answering (VQA): We utilized Multiple Choice Question (MCQ) style tasks, including MARVL, CVQA, and WorldCuisines.
187
  - Image Captioning: We employed the XM3600 dataset, evaluating strictly on examples relevant to the SEA region.
188
 
189
- Key Insight: Despite our fine-tuning process focusing primarily on text data (approximately 8 million regional Q&A and instruction pairs), our evaluation confirms that Qwen-SEA-LION-v4 (4B/8B) successfully retains the high-performance vision-language capabilities of the original base models.
190
 
191
  #### Factors
192
 
@@ -196,12 +193,12 @@ The evaluation was done **zero-shot** with native prompts.
196
 
197
  The following metrics were used to measure performance:
198
 
199
- - **Normalized accuracy** was the primary metric for the VQA tasks (CVQA, MARVL, and WorldCuisines).
200
  - **RefCLIP Score** was used for the XM3600 image captioning task.
201
 
202
  ### Results
203
 
204
- [Attachment: image-20251126-101525.png]The VL capabilities were retained despite training using text only.
205
 
206
  ## More Information
207
 
@@ -211,7 +208,7 @@ For more info, please contact us at [sealion@aisingapore.org](mailto:sealion@ais
211
 
212
  ## Team
213
 
214
- Ahn Jeongmi, Antonyrex Sajeban, Chan Hok Teng Adwin, Cheng Zi Yi Nicholas, Choa Hsueh Mei Esther, Heng Jonathan, Huang Yuli, Hulagadri Adithya Venkatadri, Jann Railey Estrada Montalan, Kang Siow Wei Bryan, Lau Wayne, Lee Chwan Ren, Leong Wai Yi, Leong Wei Qi, Liew Rachel, Limkonchotiwat Peerat, Liu Bing Jie Darius, Muhammad Ridzuan Bin Mokhtar, Nagarajan Karthik, Ng Boon Cheong Raymond, Ngee Chia Tai, Ngui Jian Gang, Nguyen Thanh Ngan, Ong Tat-Wee David, Ong Zhi Hao, Pereira Mark, Poon Joseph, Rengarajan Hamsawardhini, Susanto Yosephine, Sutaveephamochanon Anocha, Tan Choon Meng, Tan Chor Phin Evelyn, Tan Siao Wei Jessica, Tan Yixian, Tee Jun Yun, Teng Kok Wai Walter, Teo Eng Sipp Leslie, Tjhi William, Yeo Yeow Tong, Yong Xianbin, Zhang Zhou
215
 
216
  ## Contact
217
 
 
1
  # Qwen-SEA-LION-v4-4B-VL
2
 
3
+ Last update: 2025-12-4
4
 
5
  SEA-LION is a collection of Large Language Models (LLMs) which have been pretrained and instruct-tuned for the Southeast Asia (SEA) region.
6
 
7
+ **Qwen-SEA-LION-v4-4B-VL** is a 4-billion parameter Vision-Language Model (VLM) built upon the Qwen3-VL-4B-Instruct architecture. To ensure **domain adaptation** for the region, the model underwent rigorous supervised fine-tuning (SFT) on a curated dataset of approximately **9 million** instruction-text pairs. This extensive post-training instills **multilingual** and **multicultural** fluency, covering English and 7 key SEA languages: Burmese, Indonesian, Filipino, Malay, Tamil, Thai, and Vietnamese.
8
 
9
  Qwen-SEA-LION-v4-4B-VL inherits the following features from Qwen3-VL:
10
 
 
19
 
20
  SEA-LION stands for *Southeast Asian Languages In One Network*.
21
 
22
+ We performed post-training in English and SEA languages on Qwen3-VL-4B-Instruct, a decoder model using the Qwen3 architecture, to create Qwen-SEA-LION-v4-4B-VL.
23
 
24
  For tokenization, the model employs the default tokenizer used in Qwen3-VL.
25
 
 
28
  - **Shared by:** AI Products Pillar, AI Singapore
29
  - **Model type:** Decoder
30
  - **Context length:** 256k
31
+ - **Language(s):** fine-tuned on Burmese, Indonesian, Filipino, Malay, Tamil, Thai, and Vietnamese
32
  - **License:** [Apache-2.0](https://choosealicense.com/licenses/apache-2.0/)
33
  - **Finetuned from model:** [Qwen3-VL-4B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct)
34
 
35
  ### Model Sources
36
 
37
+ - **Repository:** [SEA-LION v4 - an aisingapore Collection](https://huggingface.co/collections/aisingapore/sea-lion-v4)
 
38
 
39
  ## Uses
40
 
 
167
  | Natural Language Inference | Accuracy |
168
  | LINDSEA | Accuracy |
169
  | Global MMLU Lite | Accuracy |
170
+ | ThaiExam | Accuracy |
171
  | Kalahi | Accuracy |
172
  | SEA-IFEval | Accuracy |
173
  | SEA-MTBench | Win rate against a reference |
174
 
175
  ### Results
176
 
177
+ [Attachment: 8B & 4B screenshot.png]For details on Qwen-SEA-LION-v4-4B-VL performance, please refer to the <https://leaderboard.sea-lion.ai/>.
 
 
178
 
179
  ### Retaining VL Capabilities
180
 
181
+ We also evaluated our models on two types of tasks using datasets specifically focused on Southeast Asian examples to benchmark and compared our models' performances against the original base models (Qwen3-VL-4B/8B).
182
 
183
+ - Visual Question Answering (VQA): We utilised Multiple Choice Question (MCQ) style tasks, including MARVL, CVQA, and WorldCuisines.
184
  - Image Captioning: We employed the XM3600 dataset, evaluating strictly on examples relevant to the SEA region.
185
 
186
+ Key Insight: Despite our fine-tuning process focusing primarily on text data (approximately 8 million regional Q&A and instruction pairs), our evaluations confirm that Qwen-SEA-LION-v4 (4B/8B) successfully retains the high-performance vision-language capabilities of the original base models.
187
 
188
  #### Factors
189
 
 
193
 
194
  The following metrics were used to measure performance:
195
 
196
+ - **Normalised accuracy** was the primary metric for the VQA tasks (CVQA, MARVL, and WorldCuisines).
197
  - **RefCLIP Score** was used for the XM3600 image captioning task.
198
 
199
  ### Results
200
 
201
+ [Attachment: image-20251126-101525.png]The VL capabilities were retained despite training using text only
202
 
203
  ## More Information
204
 
 
208
 
209
  ## Team
210
 
211
+ Ahn Jeongmi, Antonyrex Sajeban, Chan Hok Teng Adwin, Cheng Zi Yi Nicholas, Choa Hsueh Mei Esther, Heng Jonathan, Huang Yuli, Jann Railey Estrada Montalan, Kang Siow Wei Bryan, Lau Wayne, Lee Chwan Ren, Leong Wai Yi, Leong Wei Qi, Liew Rachel, Limkonchotiwat Peerat, Muhammad Ridzuan Bin Mokhtar, Nagarajan Karthik, Ng Boon Cheong Raymond, Ngee Chia Tai, Ngui Jian Gang, Nguyen Thanh Ngan, Ong Tat-Wee David, Ong Zhi Hao, Pereira Mark, Poon Joseph, Rengarajan Hamsawardhini, Susanto Yosephine, Sutaveephamochanon Anocha, Tan Choon Meng, Tan Chor Phin Evelyn, Tan Siao Wei Jessica, Tan Yixian, Tee Jun Yun, Teng Kok Wai Walter, Teo Eng Sipp Leslie, Tjhi William, Yeo Yeow Tong, Yong Xianbin, Zhang Zhou
212
 
213
  ## Contact
214