model update

121d40c almost 4 years ago

3.07 kB

datasets:
  - tweet_eval
metrics:
  - f1
  - accuracy
model-index:
  - name: cardiffnlp/roberta-base-hate
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: tweet_eval
          type: hate
          split: test
        metrics:
          - name: Micro F1 (tweet_eval/hate)
            type: micro_f1_tweet_eval/hate
            value: 0.5131313131313131
          - name: Macro F1 (tweet_eval/hate)
            type: micro_f1_tweet_eval/hate
            value: 0.4634195952763752
          - name: Accuracy (tweet_eval/hate)
            type: accuracy_tweet_eval/hate
            value: 0.5131313131313131
pipeline_tag: text-classification
widget:
  - text: >-
      Get the all-analog Classic Vinyl Edition of "Takin Off" Album from
      {@herbiehancock@} via {@bluenoterecords@} link below {{URL}}
    example_title: topic_classification 1
  - text: Yes, including Medicare and social security saving👍
    example_title: sentiment 1
  - text: All two of them taste like ass.
    example_title: offensive 1
  - text: If you wanna look like a badass, have drama on social media
    example_title: irony 1
  - text: Whoever just unfollowed me you a bitch
    example_title: hate 1
  - text: >-
      I love swimming for the same reason I love meditating...the feeling of
      weightlessness.
    example_title: emotion 1
  - text: Beautiful sunset last night from the pontoon @TupperLakeNY
    example_title: emoji 1

cardiffnlp/roberta-base-hate

This model is a fine-tuned version of roberta-base on the tweet_eval (hate) via tweetnlp. Training split is train and parameters have been tuned on the validation split validation.

Following metrics are achieved on the test split test (link).

F1 (micro): 0.5131313131313131
F1 (macro): 0.4634195952763752
Accuracy: 0.5131313131313131

Usage

Install tweetnlp via pip.

pip install tweetnlp

Load the model in python.

import tweetnlp
model = tweetnlp.Classifier("cardiffnlp/roberta-base-hate", max_length=128)
model.predict('Get the all-analog Classic Vinyl Edition of "Takin Off" Album from {@herbiehancock@} via {@bluenoterecords@} link below {{URL}}')

Reference

@inproceedings{camacho-collados-etal-2022-tweetnlp,
    title={{T}weet{NLP}: {C}utting-{E}dge {N}atural {L}anguage {P}rocessing for {S}ocial {M}edia},
    author={Camacho-Collados, Jose and Rezaee, Kiamehr and Riahi, Talayeh and Ushio, Asahi and Loureiro, Daniel and Antypas, Dimosthenis and Boisson, Joanne and Espinosa-Anke, Luis and Liu, Fangyu and Mart{'\i}nez-C{'a}mara, Eugenio and others},
    author = "Ushio, Asahi  and
      Camacho-Collados, Jose",
    booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations",
    month = nov,
    year = "2022",
    address = "Abu Dhabi, U.A.E.",
    publisher = "Association for Computational Linguistics",
}