Instructions to use anocha-mcomp/lora_model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use anocha-mcomp/lora_model with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("anocha-mcomp/lora_model", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Unsloth Desktop
Update model card from Confluence: Model Card: Gemma-SEA-LION-v4-4B-VL
Browse files
model_card_gemma-sea-lion-v4-4b-vl.md
CHANGED
|
@@ -6,11 +6,10 @@ SEA-LION is a collection of Large Language Models (LLMs) which have been pretrai
|
|
| 6 |
|
| 7 |
**Gemma-SEA-LION-v4-4B-VL** is a 4-billion parameter Vision-Language Model (VLM) built upon the gemma-3-4b-it architecture. To ensure **domain adaptation** for the region, the model underwent rigorous post-training on a curated dataset of approximately **8.54 million** instruction-text pairs. This curated dataset included various SEA Languages such as Burmese, Malay, Tagalog and Tamil. This extensive post-training instills **multilingual** and **multicultural** fluency for both SEA languages and English.
|
| 8 |
|
| 9 |
-
Gemma-SEA-LION-v4-4B-VL inherits the
|
| 10 |
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
- Large 128K context length
|
| 14 |
|
| 15 |
## Model Details
|
| 16 |
|
|
@@ -100,19 +99,10 @@ The instruction fine-tuning text dataset comprises of a collection of OSS & synt
|
|
| 100 |
- <https://huggingface.co/datasets/allenai/Dolci-Instruct-SFT-Tool-Use>
|
| 101 |
- <https://huggingface.co/datasets/Agent-Ark/Toucan-1.5M>
|
| 102 |
|
| 103 |
-
**Datasets for OCR:**
|
| 104 |
-
|
| 105 |
-
- <https://huggingface.co/datasets/Goekdeniz-Guelmez/unsloth-LaTeX_OCR-for-mlx-vlm> (68,700)
|
| 106 |
-
- <https://huggingface.co/datasets/sushanthr/fineweb_ocr/viewer/default/train?row=0> (150,000)
|
| 107 |
-
- <https://huggingface.co/datasets/sushanthr/wiki_ocr> (50,000)
|
| 108 |
-
- <https://huggingface.co/datasets/Chanrith123333/khmer_english_ocr_image_line> (11.3 million)
|
| 109 |
-
- <https://huggingface.co/datasets/chuuhtetnaing/burmese_ocr_data_hf> (9070)
|
| 110 |
-
- <https://huggingface.co/datasets/SoyVitou/KhmerST-Dataset-OCR-Cropped> (3560)
|
| 111 |
-
|
| 112 |
**Datasets for Reinforcement Learning:**
|
| 113 |
|
| 114 |
-
-
|
| 115 |
-
-
|
| 116 |
|
| 117 |
### Training Procedure
|
| 118 |
|
|
@@ -179,36 +169,32 @@ For details on Gemma-SEA-LION-v4-4B-VL performance, please refer to the SEA-HELM
|
|
| 179 |
|
| 180 |
### Tool calling
|
| 181 |
|
| 182 |
-
We evaluated the tool calling capabilities of our model
|
| 183 |
-
|
| 184 |
-
#### Results
|
| 185 |
-
|
| 186 |
-
### OCR
|
| 187 |
|
| 188 |
-
####
|
| 189 |
|
| 190 |
-
|
| 191 |
|
| 192 |
-
|
| 193 |
|
| 194 |
-
|
| 195 |
-
- **Image Captioning:** We employed the XM3600 dataset, evaluating strictly on examples relevant to the SEA region.
|
| 196 |
-
- ~~OCR~~
|
| 197 |
|
| 198 |
-
|
| 199 |
|
| 200 |
-
###
|
| 201 |
|
| 202 |
-
|
| 203 |
|
| 204 |
-
|
|
|
|
|
|
|
|
|
|
| 205 |
|
| 206 |
-
|
| 207 |
|
| 208 |
-
|
| 209 |
-
- **RefCLIP Score** was used for the XM3600 image captioning task.
|
| 210 |
|
| 211 |
-
### Results
|
| 212 |
|
| 213 |
## More Information
|
| 214 |
|
|
@@ -220,8 +206,6 @@ For more info, please contact us at [sealion@aisingapore.org](mailto:sealion@ais
|
|
| 220 |
|
| 221 |
Ahmed Dabeer, Ahn Jeongmi, Antonyrex Sajeban, Chan Hok Teng Adwin, Cheng Zi Yi Nicholas, Choa Hsueh Mei Esther, Heng Jonathan, Huang Yuli, Jann Railey Estrada Montalan, Kang Siow Wei Bryan, Lee Chwan Ren, Leong Wai Yi, Leong Wei Qi, Liew Rachel, Limkonchotiwat Peerat, Muhammad Ridzuan Bin Mokhtar, Nagarajan Karthik, Ng Boon Cheong Raymond, Ngee Chia Tai, Ngui Jian Gang, Nguyen Thanh Ngan, Ong Tat-Wee David, Ong Zhi Hao, Pereira Mark, Poon Joseph, Rengarajan Hamsawardhini, Susanto Yosephine, Sutaveephamochanon Anocha, Tan Choon Meng, Tan Chor Phin Evelyn, Tan Siao Wei Jessica, Tan Yixian, Tee Jun Yun, Teng Kok Wai Walter, Teo Eng Sipp Leslie, Tjhi William, Yeo Yeow Tong, Yong Xianbin, Zhang Zhou, Elliott Chris (Google), Mohseni Mohammadreza (Google), Sharan Mayank (Google), Wei Fanny (Google), Tang Jiuqiang (Google), Xu Xiang (Google), Yu Ting (Google), Loh Michelle (Google), Mangal Saurabh (Google), Mukherjee Pratyusha (Google), Sim Stephanie (Google)
|
| 222 |
|
| 223 |
-
~~Gemma team??? Elliott Chris (Google), Mohseni Mohammadreza (Google), Sharan Mayank (Google), Wei Fanny (Google), Tang Jiuqiang (Google), Xu Xiang (Google), Yu Ting (Google), Loh Michelle (Google), Mangal Saurabh (Google), Mukherjee Pratyusha (Google), Sim Stephanie (Google)~~
|
| 224 |
-
|
| 225 |
## Acknowledgement
|
| 226 |
|
| 227 |
This project is supported by the National Research Foundation Singapore and Infocomm Media Development Authority (IMDA), Singapore under its National Large Language Model Funding Initiative.
|
|
|
|
| 6 |
|
| 7 |
**Gemma-SEA-LION-v4-4B-VL** is a 4-billion parameter Vision-Language Model (VLM) built upon the gemma-3-4b-it architecture. To ensure **domain adaptation** for the region, the model underwent rigorous post-training on a curated dataset of approximately **8.54 million** instruction-text pairs. This curated dataset included various SEA Languages such as Burmese, Malay, Tagalog and Tamil. This extensive post-training instills **multilingual** and **multicultural** fluency for both SEA languages and English.
|
| 8 |
|
| 9 |
+
Gemma-SEA-LION-v4-4B-VL inherits the image and text capabilities from gemma-3-4b-it alongside its large context length of 128K tokens. Additionally, beyond extending the multilingual capabilities of the original gemma model for SEA languages, we experimented with:
|
| 10 |
|
| 11 |
+
1. Adding function calling to the model to allow for this model to be used in tool calling applications
|
| 12 |
+
2. The OCR capabilities in Thai, Chinese and English
|
|
|
|
| 13 |
|
| 14 |
## Model Details
|
| 15 |
|
|
|
|
| 99 |
- <https://huggingface.co/datasets/allenai/Dolci-Instruct-SFT-Tool-Use>
|
| 100 |
- <https://huggingface.co/datasets/Agent-Ark/Toucan-1.5M>
|
| 101 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 102 |
**Datasets for Reinforcement Learning:**
|
| 103 |
|
| 104 |
+
- <https://huggingface.co/datasets/nvidia/Nemotron-Cascade-RL-Instruction-Following>
|
| 105 |
+
- <https://huggingface.co/datasets/openai/gsm8k>
|
| 106 |
|
| 107 |
### Training Procedure
|
| 108 |
|
|
|
|
| 169 |
|
| 170 |
### Tool calling
|
| 171 |
|
| 172 |
+
We evaluated the tool calling capabilities of our model using the Berkeley Function Calling Leaderboard (BFCL) V4 evaluation.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 173 |
|
| 174 |
+
#### Factors
|
| 175 |
|
| 176 |
+
The evaluation was done using the codes from the BFCL v4 repository. Modifications were made to the agentic web search task:
|
| 177 |
|
| 178 |
+
- The Brave Search API was used instead of the DuckDuckGo Search API. The Brave Search API is also a privacy focused search engine and is similar to the DuckDuckGo Search API.
|
| 179 |
|
| 180 |
+
**Note:** Gemma-3-4b-it was run in prompt mode as it does not support function calling.
|
|
|
|
|
|
|
| 181 |
|
| 182 |
+
#### Results
|
| 183 |
|
| 184 |
+
### OCR
|
| 185 |
|
| 186 |
+
We also evaluated the OCR capabilities using the components of the OCRBenchv2 and ThaiOCRBench that focused primarily on text extraction. These included the following tasks:
|
| 187 |
|
| 188 |
+
- Table/Chart/Document parsing
|
| 189 |
+
- Fine grained OCR
|
| 190 |
+
- Full page OCR
|
| 191 |
+
- Text recognition
|
| 192 |
|
| 193 |
+
#### Factors
|
| 194 |
|
| 195 |
+
The evaluation was done zero-shot using native prompts.
|
|
|
|
| 196 |
|
| 197 |
+
#### Results
|
| 198 |
|
| 199 |
## More Information
|
| 200 |
|
|
|
|
| 206 |
|
| 207 |
Ahmed Dabeer, Ahn Jeongmi, Antonyrex Sajeban, Chan Hok Teng Adwin, Cheng Zi Yi Nicholas, Choa Hsueh Mei Esther, Heng Jonathan, Huang Yuli, Jann Railey Estrada Montalan, Kang Siow Wei Bryan, Lee Chwan Ren, Leong Wai Yi, Leong Wei Qi, Liew Rachel, Limkonchotiwat Peerat, Muhammad Ridzuan Bin Mokhtar, Nagarajan Karthik, Ng Boon Cheong Raymond, Ngee Chia Tai, Ngui Jian Gang, Nguyen Thanh Ngan, Ong Tat-Wee David, Ong Zhi Hao, Pereira Mark, Poon Joseph, Rengarajan Hamsawardhini, Susanto Yosephine, Sutaveephamochanon Anocha, Tan Choon Meng, Tan Chor Phin Evelyn, Tan Siao Wei Jessica, Tan Yixian, Tee Jun Yun, Teng Kok Wai Walter, Teo Eng Sipp Leslie, Tjhi William, Yeo Yeow Tong, Yong Xianbin, Zhang Zhou, Elliott Chris (Google), Mohseni Mohammadreza (Google), Sharan Mayank (Google), Wei Fanny (Google), Tang Jiuqiang (Google), Xu Xiang (Google), Yu Ting (Google), Loh Michelle (Google), Mangal Saurabh (Google), Mukherjee Pratyusha (Google), Sim Stephanie (Google)
|
| 208 |
|
|
|
|
|
|
|
| 209 |
## Acknowledgement
|
| 210 |
|
| 211 |
This project is supported by the National Research Foundation Singapore and Infocomm Media Development Authority (IMDA), Singapore under its National Large Language Model Funding Initiative.
|