Instructions to use Lutech-AI/I-SPIn with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Lutech-AI/I-SPIn with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import ISPIn model = ISPIn.from_pretrained("Lutech-AI/I-SPIn", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit ·
9c46e5e
1
Parent(s): e269e00
Update README.md
Browse files
README.md
CHANGED
|
@@ -188,10 +188,10 @@ ISPIn(
|
|
| 188 |
## Evaluation results
|
| 189 |
| Dataset | Metric | Performance |
|
| 190 |
|:--------------------------------------:|--------------|-------------|
|
| 191 |
-
| RTE3-ITA | Accuracy | 68% |
|
| 192 |
-
| RTE3-ITA | Min F1-Score | 60% |
|
| 193 |
-
| [RTE-2009](https://live.european-language-grid.eu/catalogue/corpus/8121/download/) | Accuracy | 59% |
|
| 194 |
-
| [RTE-2009](https://live.european-language-grid.eu/catalogue/corpus/8121/download/) | Min F1-Score | 31% |
|
| 195 |
| [SNLI](https://nlp.stanford.edu/projects/snli/) (IT) translated w/NLLB | Accuracy | 74% |
|
| 196 |
| [MNLI-Matched](https://cims.nyu.edu/~sbowman/multinli/) (IT) translated w/NLLB | Accuracy | 72% |
|
| 197 |
| [MNLI-Mismatched](https://cims.nyu.edu/~sbowman/multinli/) (IT) translated w/NLLB | Accuracy | 73%
|
|
|
|
| 188 |
## Evaluation results
|
| 189 |
| Dataset | Metric | Performance |
|
| 190 |
|:--------------------------------------:|--------------|-------------|
|
| 191 |
+
| [RTE3-ITA](https://github.com/gilnoh/RTEFormatWork/tree/master/RTE3-ITdata-original-format) | Accuracy | 68% |
|
| 192 |
+
| [RTE3-ITA](https://github.com/gilnoh/RTEFormatWork/tree/master/RTE3-ITdata-original-format) | Min F1-Score | 60% |
|
| 193 |
+
| [RTE-2009-ITA](https://live.european-language-grid.eu/catalogue/corpus/8121/download/) | Accuracy | 59% |
|
| 194 |
+
| [RTE-2009-ITA](https://live.european-language-grid.eu/catalogue/corpus/8121/download/) | Min F1-Score | 31% |
|
| 195 |
| [SNLI](https://nlp.stanford.edu/projects/snli/) (IT) translated w/NLLB | Accuracy | 74% |
|
| 196 |
| [MNLI-Matched](https://cims.nyu.edu/~sbowman/multinli/) (IT) translated w/NLLB | Accuracy | 72% |
|
| 197 |
| [MNLI-Mismatched](https://cims.nyu.edu/~sbowman/multinli/) (IT) translated w/NLLB | Accuracy | 73%
|