Upload folder using huggingface_hub
Browse files- README.md +13 -19
- model.safetensors +1 -1
README.md
CHANGED
|
@@ -7,7 +7,8 @@ tags:
|
|
| 7 |
- squad
|
| 8 |
- extractive-qa
|
| 9 |
metrics:
|
| 10 |
-
-
|
|
|
|
| 11 |
model-index:
|
| 12 |
- name: tds-bert-qa-squad
|
| 13 |
results:
|
|
@@ -18,37 +19,30 @@ model-index:
|
|
| 18 |
name: SQuAD v1.1
|
| 19 |
type: squad
|
| 20 |
metrics:
|
| 21 |
-
- name:
|
| 22 |
-
type:
|
| 23 |
-
value:
|
|
|
|
|
|
|
|
|
|
| 24 |
---
|
| 25 |
-
# BERT Extractive QA on SQuAD v1.1
|
| 26 |
|
| 27 |
## Intended Use
|
| 28 |
-
English Extractive Question Answering using the SQuAD v1.1 dataset, developed for an academic course assignment.
|
| 29 |
|
| 30 |
## Model and Training Details
|
| 31 |
- **Base Model:** [bert-base-uncased](https://huggingface.co/bert-base-uncased)
|
| 32 |
- **Dataset:** [rajpurkar/squad](https://huggingface.co/datasets/rajpurkar/squad) (Subsampled to 15k training examples).
|
| 33 |
- **Adaptation Strategy:** Full Fine-tuning with differentiated learning rates; random seed 42; 2 epochs; batch size 16.
|
| 34 |
- **Optimizer Configuration:** Discriminative learning rates (Encoder LR: 2e-5, QA Head LR: 1e-3).
|
| 35 |
-
- **Label Alignment:** Used `offset_mapping` to align character-level answer boundaries to WordPiece subtokens.
|
| 36 |
|
| 37 |
## Evaluation Results
|
| 38 |
Evaluation performed on the subsampled validation set (2,000 examples):
|
| 39 |
-
|
| 40 |
-
- **
|
| 41 |
-
|
| 42 |
-
### Comparison with Baseline
|
| 43 |
-
| Model Variant | Validation Loss |
|
| 44 |
-
| :--- | :---: |
|
| 45 |
-
| Partial Fine-Tuning (Top 2 layers + Head) | 2.135518 |
|
| 46 |
-
| **Full Fine-Tuning (This Model)** | **1.322171** |
|
| 47 |
-
|
| 48 |
-
## Limitations
|
| 49 |
-
Trained on a 15k subset of SQuAD v1.1. Performance will degrade on non-English text, informal web text, or questions without an explicit answer in the context (unlike SQuAD v2.0). Sequences exceeding 384 tokens are truncated.
|
| 50 |
|
| 51 |
## References
|
| 52 |
- Rajpurkar et al. (2016), [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://arxiv.org/abs/1606.05250).
|
| 53 |
- Devlin et al. (2019), [BERT: Pre-training of Deep Bidirectional Transformers](https://arxiv.org/abs/1810.04805).
|
| 54 |
-
- Hugging Face Transformers Documentation.
|
|
|
|
| 7 |
- squad
|
| 8 |
- extractive-qa
|
| 9 |
metrics:
|
| 10 |
+
- exact_match
|
| 11 |
+
- f1
|
| 12 |
model-index:
|
| 13 |
- name: tds-bert-qa-squad
|
| 14 |
results:
|
|
|
|
| 19 |
name: SQuAD v1.1
|
| 20 |
type: squad
|
| 21 |
metrics:
|
| 22 |
+
- name: Exact Match
|
| 23 |
+
type: exact_match
|
| 24 |
+
value: 60.70
|
| 25 |
+
- name: F1 Score
|
| 26 |
+
type: f1
|
| 27 |
+
value: 71.74
|
| 28 |
---
|
| 29 |
+
# BERT Extractive QA on SQuAD v1.1 (Winning Model)
|
| 30 |
|
| 31 |
## Intended Use
|
| 32 |
+
English Extractive Question Answering using the SQuAD v1.1 dataset, developed for an academic course assignment.
|
| 33 |
|
| 34 |
## Model and Training Details
|
| 35 |
- **Base Model:** [bert-base-uncased](https://huggingface.co/bert-base-uncased)
|
| 36 |
- **Dataset:** [rajpurkar/squad](https://huggingface.co/datasets/rajpurkar/squad) (Subsampled to 15k training examples).
|
| 37 |
- **Adaptation Strategy:** Full Fine-tuning with differentiated learning rates; random seed 42; 2 epochs; batch size 16.
|
| 38 |
- **Optimizer Configuration:** Discriminative learning rates (Encoder LR: 2e-5, QA Head LR: 1e-3).
|
|
|
|
| 39 |
|
| 40 |
## Evaluation Results
|
| 41 |
Evaluation performed on the subsampled validation set (2,000 examples):
|
| 42 |
+
- **Exact Match (EM):** 60.70
|
| 43 |
+
- **F1-Score:** 71.74
|
| 44 |
+
- **Validation Loss:** 1.319787
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
|
| 46 |
## References
|
| 47 |
- Rajpurkar et al. (2016), [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://arxiv.org/abs/1606.05250).
|
| 48 |
- Devlin et al. (2019), [BERT: Pre-training of Deep Bidirectional Transformers](https://arxiv.org/abs/1810.04805).
|
|
|
model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 435596064
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9b53be6f114865206c44878e14a46be8e3541994fc89b8cde24d42d0aa6473fc
|
| 3 |
size 435596064
|