README.md

---
license: apache-2.0
tags:
- setfit
- sentence-transformers
- text-classification
pipeline_tag: text-classification
library_name: sentence-transformers
metrics:
  - accuracy
  - f1
  - precision
  - recall
language:
  - en 
  - fr 
  - ko
  - zh 
  - ja
  - pt
  - ru
datasets:
  - imdb
model-index:
  - name: germla/satoken
    results:
      - task: 
          type: text-classification
          name: sentiment-analysis
        dataset:
          type: imdb
          name: imdb
          split: test
        metrics:
          - type: accuracy
            value: 73.976
            name: Accuracy
          - type: f1
            value: 73.1667079105832
            name: F1
          - type: precision
            value: 75.51506895964584
            name: Precision
          - type: recall
            value: 70.96
            name: Recall
      - task:
          type: text-classification
          name: sentiment-analysis
        dataset:
          type: sepidmnorozy/Russian_sentiment
          name: sepidmnorozy/Russian_sentiment
          split: train
        metrics:
          - type: accuracy
            value: 75.66371681415929
            name: Accuracy
          - type: f1
            value: 83.64218714253031
            name: F1
          - type: precision
            value: 75.25730753396459
            name: Precision
          - type: recall
            value: 94.129763130793
            name: Recall
---

# Satoken

This is a [SetFit model](https://github.com/huggingface/setfit) trained on multilingual datasets (mentioned below) for Sentiment classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

It is utilized by [Germla](https://github.com/germla) for it's feedback analysis tool. (specifically the Sentiment analysis feature)

For other models (specific language-basis) check [here](https://github.com/germla/satoken#available-models)

# Usage

To use this model for inference, first install the SetFit library:

```bash
python -m pip install setfit
```

You can then run inference as follows:

```python
from setfit import SetFitModel

# Download from Hub and run inference
model = SetFitModel.from_pretrained("germla/satoken")
# Run inference
preds = model(["i loved the spiderman movie!", "pineapple on pizza is the worst 🤮"])
```

# Training Details

## Training Data

- [IMDB](https://huggingface.co/datasets/imdb)
- [RuReviews](https://github.com/sismetanin/rureviews)
- [chABSA](https://github.com/chakki-works/chABSA-dataset)
- [Glyph](https://github.com/zhangxiangxiao/glyph)
- [nsmc](https://github.com/e9t/nsmc)
- [Allocine](https://huggingface.co/datasets/allocine)
- [Portuguese Tweets for Sentiment Analysis](https://www.kaggle.com/datasets/augustop/portuguese-tweets-for-sentiment-analysis)

## Training Procedure

We made sure to have a balanced dataset.
The model was trained on only 35% (50% for chinese) of the train split of all datasets.

### Preprocessing

- Basic Cleaning (removal of dups, links, mentions, hashtags, etc.)
- Removal of stopwords using [nltk](https://www.nltk.org/)

### Speeds, Sizes, Times

The training procedure took 6hours on the NVIDIA T4 GPU.

## Evaluation

### Testing Data, Factors & Metrics

- [IMDB test split](https://huggingface.co/datasets/imdb)

# Environmental Impact

- Hardware Type: NVIDIA T4 GPU
- Hours used: 6
- Cloud Provider: Amazon Web Services
- Compute Region: ap-south-1 (Mumbai)
- Carbon Emitted: 0.39 [kg co2 eq.](https://mlco2.github.io/impact/#co2eq)