| --- |
| license: gpl-3.0 |
| language: |
| - en |
| metrics: |
| - f1 |
| --- |
| |
| # Model Card |
|
|
| ## Model Details |
|
|
| - Model Name: IssueReportClassifier-NLBSE22 |
| - Base Model: RoBERTa |
| - Dataset: NLBSE22 |
| - Model Type: Fine-tuned |
| - Model Version: 1.0 |
| - Model Date: 2023-03-21 |
|
|
| ## Model Description |
|
|
| IssueReportClassifier-NLBSE22 is a RoBERTa model which is fine-tuned on the NLBSE22 dataset. |
| The model is trained to classify issue reports from GitHub into three categories: bug, enhancement, and question. |
| The model is trained on a dataset of labeled issue reports and is designed to predict the category of a new issue report based on its text content (title and body). |
|
|
| ## Dataset |
|
|
|
|
| | Category | Training Set | Test Set | |
| |------------|--------------|-------------| |
| | bug | 361,239 (50%) | 40,152 (49.9%) | |
| | enhancement | 299,287 (41.4%) | 33,290 (41.3%) | |
| | question | 62,373 (8.6%) | 7,076 (8.8%) | |
|
|
|
|
| ## Data preprocessing |
| The data used for training was preprocessed with [ekphrasis](https://github.com/cbaziotis/ekphrasis), adding some regular expressions to remove code, images and URLs. |
| Check out our [GitHub](https://github.com/collab-uniba/Issue-Report-Classification-Using-RoBERTa) code for more information about this. |
|
|
|
|
| ## Metrics |
|
|
| The model is evaluated using the following metrics: |
|
|
| - Accuracy |
| - Precision |
| - Recall |
| - F1 Score (micro and macro average) |
|
|
| ## References |
|
|
| - [NLBSE22 Dataset](https://nlbse2022.github.io/tools/) |
|
|
| ## Cite our work |
|
|
| ``` |
| @inproceedings{Colavito-2022, |
| title = {Issue Report Classification Using Pre-trained Language Models}, |
| booktitle = {2022 IEEE/ACM 1st International Workshop on Natural Language-Based Software Engineering (NLBSE)}, |
| author = {Colavito, Giuseppe and Lanubile, Filippo and Novielli, Nicole}, |
| year = {2022}, |
| month = may, |
| pages = {29--32}, |
| doi = {10.1145/3528588.3528659}, |
| abstract = {This paper describes our participation in the tool competition organized in the scope of the 1st International Workshop on Natural Language-based Software Engineering. We propose a supervised approach relying on fine-tuned BERT-based language models for the automatic classification of GitHub issues. We experimented with different pre-trained models, achieving the best performance with fine-tuned RoBERTa (F1 = .8591).}, |
| keywords = {Issue classification, BERT, deep learning, labeling unstructured data, |
| software maintenance and evolution}, |
| } |
| ``` |
|
|
