---
license: mit
datasets:
- eriktks/conll2003
language:
- en
metrics:
- accuracy
base_model:
- distilbert/distilbert-base-uncased
pipeline_tag: token-classification
library_name: transformers
tags:
- ner
- named-entity-recognition
---

# Results

This repository contains the training results of a fine-tuned [**DistilBERT (distilbert-base-uncased)**](https://huggingface.co/distilbert/distilbert-base-uncased) model on the **CoNLL-2003** dataset for Named Entity Recognition (NER).

The model achieves the following results on the validation set:

- **Loss:** 0.2416
- **Accuracy:** 0.9433 (94.33%)

The trained model is available [here](https://huggingface.co/mohammad748/distilbert-conll2003-ner).

## Model Description

The model performs token classification to identify named entities in English text, including persons (PER), organizations (ORG), locations (LOC), and miscellaneous entities (MISC).

## Intended Uses & Limitations

The model is intended for English Named Entity Recognition. Its performance may be limited when applied to specialized terminology or text outside the training domain.

## Training and Evaluation Data

The model was trained and evaluated using the [CoNLL-2003](https://huggingface.co/datasets/eriktks/conll2003) dataset, which provides separate training, validation, and test splits.

The text was tokenized using the DistilBERT tokenizer with a maximum sequence length of 128 tokens. Word-level entity labels were aligned with the corresponding tokens.

## Training Procedure

The pretrained DistilBERT encoder was frozen, and the token classification head was trained using the Hugging Face Trainer API. The best model checkpoint was selected based on validation accuracy.

### Training Hyperparameters

The following hyperparameters were used during training:

- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- optimizer: AdamW
- weight_decay: 0.01
- lr_scheduler_type: linear
- num_epochs: 3
- max_sequence_length: 128
- eval_strategy: epoch
- save_strategy: epoch
- logging_steps: 50

### Training Results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 0.278032 | 1.0 | 878 | 0.275000 | 0.935337 |
| 0.236482 | 2.0 | 1756 | 0.248705 | 0.941511 |
| 0.237519 | 3.0 | 2634 | 0.241559 | 0.943264 |

*Training loss values represent the most recent logged losses at each epoch.*

### Final Evaluation Results

- **Validation Loss:** 0.241559
- **Validation Accuracy:** 94.33%
- **Overall Training Loss:** 0.267937
- **Total Training Steps:** 2,634
- **Total Epochs:** 3

### Frameworks

- PyTorch
- Hugging Face Transformers
- Hugging Face Datasets
- Accelerate
- Evaluate