lindazeng979/codemixed-sentiment-models

Code-Mixed Sentiment Analysis Models (LLM-augmented)

A working collection of ~104 multilingual BERT and XLM-T models for code-mixed (code-switched) sentiment analysis, covering Spanish–English (Spanglish) and Malayalam–English. Models are fine-tuned on a mix of naturally occurring and LLM-generated (augmented) code-mixed data.

Status: development repository. This is the repository owner's research workspace, kept public for convenience. The checkpoints here are organized for the owner's iteration, not packaged as polished, benchmarked releases. Expect naming to be terse.

Repository structure

Models and tokenizers are organized as Language / Type / Architecture / folder:

<Language-Pair>/<Data-Type>/<Architecture>/<model-or-tokenizer folder>

• Language pair — Spanish-English, Malayalam-English
• Data type — how the training data was produced:
  • Natural — naturally occurring code-mixed data (folder tag NCM)
  • Synthetic — fully LLM-generated code-mixed data (folder tag SCM)
  • Mix — a blend of natural and synthetic data (folder tag MIX)
  • Zero-shot — evaluated/trained in a zero-shot setup
• Architecture — mBERT or XLM-T
• Kind — a folder is a tokenizer if its name contains tokenizer; otherwise it's a model. Tokenizers are stored as their own folders next to the models (not merged in).

Example paths:

spanish-english/natural/mBERT/spanglish_NCM_..._mBERT_mBERT/       # model
spanish-english/natural/mBERT/spanglish_NCM_tokenizer_..._mBERT/   # its tokenizer
malayalam-english/mix/XLM-T/malay_MIX_..._mBERT_XLMT/              # an XLM-T model
malayalam-english/zero-shot/XLM-T/malay_zero_shot_XLMT/           # zero-shot XLM-T

Naming note: in the raw folder names the architecture is the last arch token — e.g. ..._mBERT_XLMT is an XLM-T model, ..._mBERT_mBERT is an mBERT model.

important-models/

A separately discovered directory of what are likely the most important models in this repo. Like the folder below, these do not follow the clean Language / Type / Architecture layout, but the folder naming is consistent enough to identify each model. These checkpoints were created Feb 28 – Apr 11, 2024 (the majority on Apr 11, 2024).

preliminary-experiments/

A separate top-level folder holding my earlier, informally organized experiments. It contains initial mBERT Spanglish model runs, along with some zero-shot, few-shot, and natural baselines and initial synthetic and mix models. These were created in February 2024. These folders do not follow the Language / Type / Architecture / folder layout used elsewhere in the repo, but the folder naming is still fairly consistent, so you can generally tell the language, data type, and architecture from a folder's name.

Rough timeline

The creation dates give a quick way to tell the informal folders apart:

• preliminary-experiments/ — February 2024
• important-models/ — Feb 28 – Apr 11, 2024 (mostly Apr 11)
• everything in the structured Language / Type / Architecture tree — generally March–May 2024

How to load a specific model

Because there are many models in one repo, pass the subfolder argument. Tokenizers live in separate folders, so point the tokenizer loader at the matching tokenizer folder:

from transformers import AutoModelForSequenceClassification, AutoTokenizer

REPO = "lindazeng979/codemixed-sentiment-models"

model = AutoModelForSequenceClassification.from_pretrained(
    REPO, subfolder="spanish-english/natural/mBERT/<model_folder_name>"
)
tokenizer = AutoTokenizer.from_pretrained(
    REPO, subfolder="spanish-english/natural/mBERT/<tokenizer_folder_name>"
)

Browse the file tree to find exact folder names: https://huggingface.co/lindazeng979/codemixed-sentiment-models/tree/main

Intended use & limitations

These models classify sentiment in code-mixed (code-switched) Spanglish and Malayalam–English social media text. They are used to investigate LLM-based data augmentation for low-resource, code-switched sentiment analysis, how training on naturally occurring code-mixed data (natural), fully LLM-generated code-mixed data (synthetic), and blends of the two (mix) affects downstream sentiment performance across mBERT and XLM-T backbones. For the data, experimental setup, and evaluation, see the GitHub repo and the paper.

Limitations: checkpoints here are organized to accompany the paper rather than as individual releases, so results and quality vary across data types and architectures. Models trained on synthetic-only data may carry artifacts of the generating LLM, and sentiment models can reflect biases in their training data.

Citation

If you use these models, please cite the paper:

@inproceedings{zeng-2024-leveraging,
    title = "Leveraging Large Language Models for Code-Mixed Data Augmentation in Sentiment Analysis",
    author = "Zeng, Linda",
    editor = "Hale, James and Chawla, Kushal and Garg, Muskan",
    booktitle = "Proceedings of the Second Workshop on Social Influence in Conversations (SICon 2024)",
    month = nov,
    year = "2024",
    address = "Miami, Florida, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.sicon-1.6/",
    doi = "10.18653/v1/2024.sicon-1.6",
    pages = "85--101"
}

Paper: https://aclanthology.org/2024.sicon-1.6/

Data & code: https://github.com/lindazeng979/LLM-CMSA