Hugging Face's logo

LLaMAX
/
LLaMAX2-7B-XNLI

Text Generation
Transformers
PyTorch
llama
Multilingual
text-generation-inference
Model card Files
xet
 Community
1
LLaMAX commited on
Commit
e35e02e
·
verified ·
1 Parent(s): 7916855

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +57 -53
README.md CHANGED
@@ -1,53 +1,57 @@
1
-
2
- ### Model Sources
3
- - **Paper**: LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages
4
- - **Link**: https://arxiv.org/pdf/2407.05975
5
- - **Repository**: https://github.com/CONE-MT/LLaMAX/
6
-
7
- ### Model Description
8
-
9
- 🔥 LLaMAX-7B-X-NLI is a NLI model with multilingual capability, which is fully fine-tuned the powerful multilingual model [LLaMAX-7B](https://huggingface.co/LLaMAX/LLaMAX-7B) on MultiNLI dataset.
10
-
11
- 🔥 Compared with fine-tuning Llama-2 on the same setting, LLaMAX-7B-X-CSQA improves the average accuracy up to 5.6% on the XNLI dataset.
12
-
13
- ### Experiments
14
-
15
-
16
- | XNLI | Avg. | Sw | Ur | Hi | Th | Ar | Tr | El | Vi | Zh | Ru | Bg | De | Fr | Es | En |
17
- |-------------------|-------|------|------|------|------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------|------|------|------|------|------|------|------|------|------|
18
- | Llama2-7B-X-XNLI | 70.6 | 44.6 | 55.1 | 62.2 | 58.4 | 64.7 | 64.9 | 65.6 | 75.4 | 75.9 | 78.9 | 78.6 | 80.7 | 81.7 | 83.1 | 89.5 |
19
- | LLaMAX-7B-X-XNLI | 76.2 | 66.7 | 65.3 | 69.1 | 66.2 | 73.6 | 71.8| 74.3 | 77.4 | 78.3 | 80.3 | 81.6 | 82.2 | 83.0 | 84.1 | 89.7 |
20
-
21
-
22
-
23
- ### Model Usage
24
-
25
- Code Example:
26
- ```angular2html
27
- from transformers import AutoTokenizer, LlamaForCausalLM
28
-
29
- model = LlamaForCausalLM.from_pretrained(PATH_TO_CONVERTED_WEIGHTS)
30
- tokenizer = AutoTokenizer.from_pretrained(PATH_TO_CONVERTED_TOKENIZER)
31
-
32
- query = "Premise: She doesn’t really understand. Hypothesis: Actually, she doesn’t get it. Label:"
33
- inputs = tokenizer(query, return_tensors="pt")
34
-
35
- generate_ids = model.generate(inputs.input_ids, max_length=30)
36
- tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
37
- # => Entailment
38
- ```
39
-
40
- ### Citation
41
- if our model helps your work, please cite this paper:
42
-
43
- ```
44
- @misc{lu2024llamaxscalinglinguistichorizons,
45
- title={LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages},
46
- author={Yinquan Lu and Wenhao Zhu and Lei Li and Yu Qiao and Fei Yuan},
47
- year={2024},
48
- eprint={2407.05975},
49
- archivePrefix={arXiv},
50
- primaryClass={cs.CL},
51
- url={https://arxiv.org/abs/2407.05975},
52
- }
53
- ```
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - Multilingual
4
+ ---
5
+
6
+ ### Model Sources
7
+ - **Paper**: LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages
8
+ - **Link**: https://arxiv.org/pdf/2407.05975
9
+ - **Repository**: https://github.com/CONE-MT/LLaMAX/
10
+
11
+ ### Model Description
12
+
13
+ 🔥 LLaMAX-7B-X-NLI is a NLI model with multilingual capability, which is fully fine-tuned the powerful multilingual model [LLaMAX-7B](https://huggingface.co/LLaMAX/LLaMAX-7B) on MultiNLI dataset.
14
+
15
+ 🔥 Compared with fine-tuning Llama-2 on the same setting, LLaMAX-7B-X-CSQA improves the average accuracy up to 5.6% on the XNLI dataset.
16
+
17
+ ### Experiments
18
+
19
+
20
+ | XNLI | Avg. | Sw | Ur | Hi | Th | Ar | Tr | El | Vi | Zh | Ru | Bg | De | Fr | Es | En |
21
+ |-------------------|-------|------|------|------|------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------|------|------|------|------|------|------|------|------|------|
22
+ | Llama2-7B-X-XNLI | 70.6 | 44.6 | 55.1 | 62.2 | 58.4 | 64.7 | 64.9 | 65.6 | 75.4 | 75.9 | 78.9 | 78.6 | 80.7 | 81.7 | 83.1 | 89.5 |
23
+ | LLaMAX-7B-X-XNLI | 76.2 | 66.7 | 65.3 | 69.1 | 66.2 | 73.6 | 71.8| 74.3 | 77.4 | 78.3 | 80.3 | 81.6 | 82.2 | 83.0 | 84.1 | 89.7 |
24
+
25
+
26
+
27
+ ### Model Usage
28
+
29
+ Code Example:
30
+ ```angular2html
31
+ from transformers import AutoTokenizer, LlamaForCausalLM
32
+
33
+ model = LlamaForCausalLM.from_pretrained(PATH_TO_CONVERTED_WEIGHTS)
34
+ tokenizer = AutoTokenizer.from_pretrained(PATH_TO_CONVERTED_TOKENIZER)
35
+
36
+ query = "Premise: She doesn’t really understand. Hypothesis: Actually, she doesn’t get it. Label:"
37
+ inputs = tokenizer(query, return_tensors="pt")
38
+
39
+ generate_ids = model.generate(inputs.input_ids, max_length=30)
40
+ tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
41
+ # => Entailment
42
+ ```
43
+
44
+ ### Citation
45
+ if our model helps your work, please cite this paper:
46
+
47
+ ```
48
+ @misc{lu2024llamaxscalinglinguistichorizons,
49
+ title={LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages},
50
+ author={Yinquan Lu and Wenhao Zhu and Lei Li and Yu Qiao and Fei Yuan},
51
+ year={2024},
52
+ eprint={2407.05975},
53
+ archivePrefix={arXiv},
54
+ primaryClass={cs.CL},
55
+ url={https://arxiv.org/abs/2407.05975},
56
+ }
57
+ ```