BradechiO commited on
Commit
794ef5b
·
verified ·
1 Parent(s): c9b55f8

Upload folder using huggingface_hub

Browse files
Files changed (2) hide show
  1. README.md +13 -19
  2. model.safetensors +1 -1
README.md CHANGED
@@ -7,7 +7,8 @@ tags:
7
  - squad
8
  - extractive-qa
9
  metrics:
10
- - loss
 
11
  model-index:
12
  - name: tds-bert-qa-squad
13
  results:
@@ -18,37 +19,30 @@ model-index:
18
  name: SQuAD v1.1
19
  type: squad
20
  metrics:
21
- - name: Validation Loss
22
- type: loss
23
- value: 1.322171
 
 
 
24
  ---
25
- # BERT Extractive QA on SQuAD v1.1
26
 
27
  ## Intended Use
28
- English Extractive Question Answering using the SQuAD v1.1 dataset, developed for an academic course assignment. Not intended for production-critical NLP applications without domain-specific validation.
29
 
30
  ## Model and Training Details
31
  - **Base Model:** [bert-base-uncased](https://huggingface.co/bert-base-uncased)
32
  - **Dataset:** [rajpurkar/squad](https://huggingface.co/datasets/rajpurkar/squad) (Subsampled to 15k training examples).
33
  - **Adaptation Strategy:** Full Fine-tuning with differentiated learning rates; random seed 42; 2 epochs; batch size 16.
34
  - **Optimizer Configuration:** Discriminative learning rates (Encoder LR: 2e-5, QA Head LR: 1e-3).
35
- - **Label Alignment:** Used `offset_mapping` to align character-level answer boundaries to WordPiece subtokens.
36
 
37
  ## Evaluation Results
38
  Evaluation performed on the subsampled validation set (2,000 examples):
39
-
40
- - **Validation Loss:** 1.322171
41
-
42
- ### Comparison with Baseline
43
- | Model Variant | Validation Loss |
44
- | :--- | :---: |
45
- | Partial Fine-Tuning (Top 2 layers + Head) | 2.135518 |
46
- | **Full Fine-Tuning (This Model)** | **1.322171** |
47
-
48
- ## Limitations
49
- Trained on a 15k subset of SQuAD v1.1. Performance will degrade on non-English text, informal web text, or questions without an explicit answer in the context (unlike SQuAD v2.0). Sequences exceeding 384 tokens are truncated.
50
 
51
  ## References
52
  - Rajpurkar et al. (2016), [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://arxiv.org/abs/1606.05250).
53
  - Devlin et al. (2019), [BERT: Pre-training of Deep Bidirectional Transformers](https://arxiv.org/abs/1810.04805).
54
- - Hugging Face Transformers Documentation.
 
7
  - squad
8
  - extractive-qa
9
  metrics:
10
+ - exact_match
11
+ - f1
12
  model-index:
13
  - name: tds-bert-qa-squad
14
  results:
 
19
  name: SQuAD v1.1
20
  type: squad
21
  metrics:
22
+ - name: Exact Match
23
+ type: exact_match
24
+ value: 60.70
25
+ - name: F1 Score
26
+ type: f1
27
+ value: 71.74
28
  ---
29
+ # BERT Extractive QA on SQuAD v1.1 (Winning Model)
30
 
31
  ## Intended Use
32
+ English Extractive Question Answering using the SQuAD v1.1 dataset, developed for an academic course assignment.
33
 
34
  ## Model and Training Details
35
  - **Base Model:** [bert-base-uncased](https://huggingface.co/bert-base-uncased)
36
  - **Dataset:** [rajpurkar/squad](https://huggingface.co/datasets/rajpurkar/squad) (Subsampled to 15k training examples).
37
  - **Adaptation Strategy:** Full Fine-tuning with differentiated learning rates; random seed 42; 2 epochs; batch size 16.
38
  - **Optimizer Configuration:** Discriminative learning rates (Encoder LR: 2e-5, QA Head LR: 1e-3).
 
39
 
40
  ## Evaluation Results
41
  Evaluation performed on the subsampled validation set (2,000 examples):
42
+ - **Exact Match (EM):** 60.70
43
+ - **F1-Score:** 71.74
44
+ - **Validation Loss:** 1.319787
 
 
 
 
 
 
 
 
45
 
46
  ## References
47
  - Rajpurkar et al. (2016), [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://arxiv.org/abs/1606.05250).
48
  - Devlin et al. (2019), [BERT: Pre-training of Deep Bidirectional Transformers](https://arxiv.org/abs/1810.04805).
 
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f6ddb659565bbf5f3eee93559abfd93c8b26d3c26eb4f85eca183f2e9da4a346
3
  size 435596064
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9b53be6f114865206c44878e14a46be8e3541994fc89b8cde24d42d0aa6473fc
3
  size 435596064