Instructions to use anocha-mcomp/lora_model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use anocha-mcomp/lora_model with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("anocha-mcomp/lora_model", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Unsloth Desktop
Update model card from Confluence: Working Gemma-SEA-LION-v4-27B-IT Model Card - Draft1
Browse files
working_gemma-sea-lion-v4-27b-it_model_card_-_draft1.md
CHANGED
|
@@ -1,7 +1,215 @@
|
|
| 1 |
-
#
|
| 2 |
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
|
| 5 |
```
|
| 6 |
-
|
| 7 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Model Card for Gemma-SEA-LION-v4-27B-IT (Instruct model)
|
| 2 |
|
| 3 |
+
SEA-LION is a collection of Large Language Models (LLMs) which have been pretrained and instruct-tuned for the Southeast Asia (SEA) region.
|
| 4 |
+
|
| 5 |
+
## Model Details
|
| 6 |
+
|
| 7 |
+
### Model Description
|
| 8 |
+
|
| 9 |
+
SEA-LION stands for Southeast Asian Languages In One Network.
|
| 10 |
+
|
| 11 |
+
We continue pre-train Gemma 3 27B Instruct in English, Chinese, Filipino, Indonesian, Malay, Tamil, Thai, Vietnamese, Burmese, Khmer and Lao using 500B tokens from mix sources of web-data, code, opensource dataset and synthetically generated dataset, to create Gemma-SEA-LION-v4-27B-IT.
|
| 12 |
+
|
| 13 |
+
**(Add more Post-training part from Tai or Wei Yi)**
|
| 14 |
+
We conduct post-training using a variety of Reinforcement Learning algorithm. The post-training data for instruction fine-tuning consists of our own SEA-Instruct, Infinity-Instruct, OpenMath-Instruct 2, and other datasets. The post-training process includes multiple stages of instruction fine-tuning, an alignment stage, and several merge stages (need Tai).
|
| 15 |
+
|
| 16 |
+
For tokenization, the model employs the default tokenizer used in Gemma 3 27B Instruct. ~~The model has a context length of 128k tokens.~~
|
| 17 |
+
|
| 18 |
+
- **Developed by:** Products Pillar, AI Singapore
|
| 19 |
+
- **Funded by:** Singapore NRF
|
| 20 |
+
- **Shared by:** Products Pillar, AI Singapore
|
| 21 |
+
- **Model type:** Decoder
|
| 22 |
+
- **Context Length:** 128k tokens
|
| 23 |
+
- **Language(s) (NLP):** Burmese, Chinese, English, Filipino, Indonesian, Malay, Tamil, Thai, Vietnamese, Khmer and Lao,
|
| 24 |
+
- **License:** [Gemma Terms of Use](https://ai.google.dev/gemma/terms)
|
| 25 |
+
- **Finetuned from model [optional]:** [Gemma-3-27B-IT](https://huggingface.co/google/gemma-3-27b-it)
|
| 26 |
+
|
| 27 |
+
### Model Sources [optional]
|
| 28 |
+
|
| 29 |
+
- **Repository:** {{ repo | default("[More Information Needed]", true)}}
|
| 30 |
+
- **Paper [optional]:** {{ paper | default("[More Information Needed]", true)}}
|
| 31 |
+
- **Demo [optional]:** {{ demo | default("[More Information Needed]", true)}}
|
| 32 |
+
|
| 33 |
+
## Uses
|
| 34 |
+
|
| 35 |
+
### Direct Use
|
| 36 |
+
|
| 37 |
+
{{ direct_use | default("[More Information Needed]", true)}}
|
| 38 |
+
|
| 39 |
+
### Downstream Use [optional]
|
| 40 |
+
|
| 41 |
+
{{ downstream_use | default("[More Information Needed]", true)}}
|
| 42 |
+
|
| 43 |
+
### Out-of-Scope Use
|
| 44 |
+
|
| 45 |
+
The model has *not* been aligned for safety. Developers and users should perform their own safety fine-tuning and related security measures. In no event shall the authors be held liable for any claims, damages, or other liabilities arising from the use of the released weights and codes.
|
| 46 |
+
|
| 47 |
+
## Bias, Risks, and Limitations
|
| 48 |
+
|
| 49 |
+
### Caveats || Risks
|
| 50 |
+
|
| 51 |
+
It is important for users to be aware that our model exhibits certain limitations that warrant consideration. Like many LLMs, the model can hallucinate and occasionally generates irrelevant content, introducing fictional elements that are not grounded in the provided context. Users should also exercise caution in interpreting and validating the model's responses due to the potential inconsistencies.
|
| 52 |
+
|
| 53 |
+
## Limitations
|
| 54 |
+
|
| 55 |
+
In terms of vision capability, Gemma-SEA-LION-v4-27B-IT has been trained and fine-tuned exclusively on the text back-end. As a result, its vision capabilities are expected to be comparable to those of Gemma 3 IT 27B, and may not exhibit significant improvements or differences in this area. (<https://huggingface.co/google/gemma-3-27b-it> )
|
| 56 |
+
|
| 57 |
+
### Safety
|
| 58 |
+
|
| 59 |
+
Current SEA-LION models, including this commercially permissive release, have not been aligned for safety. Developers and users should perform their own safety fine-tuning and related security measures. In no event shall the authors be held liable for any claim, damages, or other liability arising from the use of the released weights and codes.
|
| 60 |
+
|
| 61 |
+
### Recommendations
|
| 62 |
+
|
| 63 |
+
{{ bias_recommendations | default("Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.", true)}}
|
| 64 |
+
|
| 65 |
+
## How to Get Started with the Model
|
| 66 |
+
|
| 67 |
+
But not ready to use yet! Use the code below to get started with the model using the 🤗 Transformers library.
|
| 68 |
|
| 69 |
```
|
| 70 |
+
from transformers import pipeline
|
| 71 |
+
import torch
|
| 72 |
+
|
| 73 |
+
pipe = pipeline(
|
| 74 |
+
"text-generation",
|
| 75 |
+
# model="aisingapore/Gemma-SEA-LION-v4-27B-IT",
|
| 76 |
+
device="cuda",
|
| 77 |
+
torch_dtype=torch.bfloat16
|
| 78 |
+
)
|
| 79 |
+
|
| 80 |
+
messages = [
|
| 81 |
+
{
|
| 82 |
+
"role": "system",
|
| 83 |
+
"content": [{"type": "text", "text": "You are a helpful assistant."}]
|
| 84 |
+
},
|
| 85 |
+
{
|
| 86 |
+
"role": "user",
|
| 87 |
+
"content": [
|
| 88 |
+
{"type": "text", "text": "Write a poem on southeast asian countries in Thai."}
|
| 89 |
+
]
|
| 90 |
+
}
|
| 91 |
+
]
|
| 92 |
+
|
| 93 |
+
output = pipe(text=messages, max_new_tokens=200)
|
| 94 |
+
print(output[0]["generated_text"][-1]["content"])
|
| 95 |
+
```
|
| 96 |
+
Later, additional things like image-text-to-text example
|
| 97 |
+
|
| 98 |
+
## Training Details
|
| 99 |
+
|
| 100 |
+
### Training Data
|
| 101 |
+
|
| 102 |
+
The dataset comprises English, Chinese, Tagalog, Indonesian, Malay, Tamil, Thai, Vietnamese, Burmese, Khmer, and Lao languages, collected from a mixture of sources including web data, code, open-source datasets, and synthetically generated datasets, amounting to a total of 500 billion tokens.
|
| 103 |
+
|
| 104 |
+
### Training Procedure
|
| 105 |
+
|
| 106 |
+
#### Preprocessing [optional]
|
| 107 |
+
|
| 108 |
+
{{ preprocessing | default("[More Information Needed]", true)}}
|
| 109 |
+
|
| 110 |
+
#### Training Hyperparameters
|
| 111 |
+
|
| 112 |
+
- **Training regime:** {{ training_regime | default("[More Information Needed]", true)}}
|
| 113 |
+
|
| 114 |
+
#### Speeds, Sizes, Times [optional]
|
| 115 |
+
|
| 116 |
+
{{ speeds_sizes_times | default("[More Information Needed]", true)}}
|
| 117 |
+
|
| 118 |
+
## Evaluation
|
| 119 |
+
|
| 120 |
+
### Testing Data, Factors & Metrics
|
| 121 |
+
|
| 122 |
+
We evaluated Gemma-SEA-LION-v4-27B-IT on both general language capabilities and instruction-following capabilities.
|
| 123 |
+
|
| 124 |
+
#### Testing Data
|
| 125 |
+
|
| 126 |
+
General
|
| 127 |
+
|
| 128 |
+
For the evaluation of general language capabilities, we employed the [SEA-HELM evaluation benchmark](https://arxiv.org/abs/2502.14301) across a variety of tasks. These tasks include Question Answering (QA), Sentiment Analysis (Sentiment), Toxicity Detection (Toxicity), Translation in both directions (Eng>Lang & Lang>Eng), Abstractive Summarisation (Abssum), Causal Reasoning (Causal), Natural Language Inference (NLI), and linguistic diagnostics (LINDSEA).
|
| 129 |
+
|
| 130 |
+
Instruction-following
|
| 131 |
+
|
| 132 |
+
We evaluated the models on instruction-following capabilities with two datasets, SEA-IFEval (based on [IFEval](https://arxiv.org/abs/2311.07911)) and SEA-MTBench (based on [MT-Bench](https://arxiv.org/abs/2306.05685)). The two datasets were originally in English, the linguists and native speakers in the team worked together to filter, localise and translate the datasets into the respective target languages to ensure that the examples remained reasonable, meaningful and natural.
|
| 133 |
+
|
| 134 |
+
Counts for Selected Tasks in SEA-HELM
|
| 135 |
+
|
| 136 |
+
| **Competency** | **Task** | **Count** |
|
| 137 |
+
| NLP Classic - NLG | Translation and Abstractive Summarisation | 11467 |
|
| 138 |
+
| NLP Classic - NLR | Natural Language Inference and Causal Reasoning | 7050 |
|
| 139 |
+
| NLP Classic - NLU | Metaphor Understanding, Question Answering, Paraphrase and Sentiment Analysis | 10810 |
|
| 140 |
+
| LLM-Specific: Instruction Following | Instruction/Format Following (IF-Eval | 630 |
|
| 141 |
+
| LLM-Specific: Chat | Multi-Turn Chat (MT-Bench) | 381 |
|
| 142 |
+
|
| 143 |
+
#### Metrics
|
| 144 |
+
|
| 145 |
+
The evaluation was done **zero-shot** with native prompts on a sample of 100-1000 instances for each dataset.
|
| 146 |
+
|
| 147 |
+
### Results
|
| 148 |
+
|
| 149 |
+
For details on Gemma-SEA-LION-v4-27B-IT performance, please refer to the SEA-HELM leaderboard, <https://leaderboard.sea-lion.ai/>.
|
| 150 |
+
|
| 151 |
+
#### Summary
|
| 152 |
+
|
| 153 |
+
{{ results_summary | default("", true) }}
|
| 154 |
+
|
| 155 |
+
## Model Examination [optional]
|
| 156 |
+
|
| 157 |
+
{{ model_examination | default("[More Information Needed]", true)}}
|
| 158 |
+
|
| 159 |
+
## Environmental Impact
|
| 160 |
+
|
| 161 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://www.google.com/url?q=https%3A%2F%2Fmlco2.github.io%2Fimpact%23compute) presented in [Lacoste et al. (2019)](https://www.google.com/url?q=https%3A%2F%2Farxiv.org%2Fabs%2F1910.09700).
|
| 162 |
+
|
| 163 |
+
- **Hardware Type:** H256 from 32 nodes
|
| 164 |
+
- **Hours used:** {{ hours_used | default("[More Information Needed]", true)}}
|
| 165 |
+
- **Cloud Provider:** {{ cloud_provider | default("[More Information Needed]", true)}}
|
| 166 |
+
- **Compute Region:** {{ cloud_region | default("[More Information Needed]", true)}}
|
| 167 |
+
- **Carbon Emitted:** {{ co2_emitted | default("[More Information Needed]", true)}}
|
| 168 |
+
|
| 169 |
+
## Technical Specifications [optional]
|
| 170 |
+
|
| 171 |
+
### Model Architecture and Objective
|
| 172 |
+
|
| 173 |
+
H256 from 32 nodes
|
| 174 |
+
|
| 175 |
+
### Compute Infrastructure
|
| 176 |
+
|
| 177 |
+
{{ compute_infrastructure | default("[More Information Needed]", true)}}
|
| 178 |
+
|
| 179 |
+
#### Hardware
|
| 180 |
+
|
| 181 |
+
{{ hardware_requirements | default("[More Information Needed]", true)}}
|
| 182 |
+
|
| 183 |
+
#### Software
|
| 184 |
+
|
| 185 |
+
{{ software | default("[More Information Needed]", true)}}
|
| 186 |
+
|
| 187 |
+
## ~~Citation [optional]~~
|
| 188 |
+
|
| 189 |
+
**~~BibTeX:~~**
|
| 190 |
+
|
| 191 |
+
~~{{ citation_bibtex | default("[More Information Needed]", true)}}~~
|
| 192 |
+
|
| 193 |
+
**~~APA:~~**
|
| 194 |
+
|
| 195 |
+
~~{{ citation_apa | default("[More Information Needed]", true)}}~~
|
| 196 |
+
|
| 197 |
+
## ~~Glossary [optional]~~
|
| 198 |
+
|
| 199 |
+
~~{{ glossary | default("[More Information Needed]", true)}}~~
|
| 200 |
+
|
| 201 |
+
## More Information [optional]
|
| 202 |
+
|
| 203 |
+
This is the repository for the commercial instruction-tuned model. The model has *not* been aligned for safety. Developers and users should perform their own safety fine-tuning and related security measures. In no event shall the authors be held liable for any claims, damages, or other liabilities arising from the use of the released weights and codes.
|
| 204 |
+
|
| 205 |
+
For more info, please contact us using this [SEA-LION Inquiry Form](https://forms.gle/sLCUVb95wmGf43hi6)
|
| 206 |
+
|
| 207 |
+
[Link to SEA-LION's GitHub repository](https://github.com/aisingapore/sealion)
|
| 208 |
+
|
| 209 |
+
## Model Card Authors [optional]
|
| 210 |
+
|
| 211 |
+
Adithya Venkatadri Hulagadri, Adwin Chan Hok Teng, Anocha Sutaveephamochanon, Brandon Ong Jin Jie, Bryan Siow Wei Kang, David Ong Tat-Wee, Esther Choa Hsueh Mei, Evelyn Tan Chor Phin, Hamsawardhini Rengarajan, Huang Yuli, Jann Railey Estrada Montalan, Jessica Tan Siao Wei, Jonathan Heng, Karthik Nagarajan, Lee Chwan Ren, Leong Wai Yi, Leong Wei Qi, Leslie Teo, Mark Pereira, Muhammad Ridzuan Bin Mokhtar, Ngee Chia Tai, Ngui Jian Gang, Nguyen Thanh Ngan, Nicholas Cheng Zi Yi, Ong Zhi Hao, Peerat Limkonchotiwat, Raymond Ng Boon Cheong, Sajeban Antonyrex, Susanto Yosephine, Tan Choon Meng, Walter Teng Kok Wai, Wayne Lau, William Tjhi Chandra, Yeo Yeow Tong, Yong Xianbin, Liew Rachel, Liu Bing Jie Darius, Teo Wei Yi
|
| 212 |
+
|
| 213 |
+
## Model Card Contact
|
| 214 |
+
|
| 215 |
+
{{ model_card_contact | default("[More Information Needed]", true)}}
|