Hugging Face's logo

rafmacalaba
/
gliner_datause_v0

Token Classification
GLiNER
PyTorch
ner
data-use
Model card Files
xet
 Community
rafmacalaba commited on
Commit
b259fc0
·
verified ·
1 Parent(s): 20ba6d6

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +29 -29
README.md CHANGED
@@ -21,7 +21,7 @@ Fine-tune of `urchade/gliner_large-v2.1` for data-use mention extraction
21
  - base model: `urchade/gliner_large-v2.1`
22
  - dataset: `rafmacalaba/data-use-mentions-v2` (gliner config)
23
  - corpus: `all`
24
- - epochs: 3
25
  - learning rate: 5e-06
26
  - batch size: 16
27
  - precision: bf16
@@ -30,48 +30,48 @@ Fine-tune of `urchade/gliner_large-v2.1` for data-use mention extraction
30
 
31
  | thr | tp | fp | fn | precision | recall | f0.5 | f1 |
32
  | --- | --- | --- | --- | --- | --- | --- | --- |
33
- | 0.10 | 16858 | 18193 | 247 | 0.4810 | 0.9856 | 0.5358 | 0.6464 |
34
- | 0.20 | 16779 | 13849 | 326 | 0.5478 | 0.9809 | 0.6009 | 0.7030 |
35
- | 0.30 | 16685 | 11278 | 420 | 0.5967 | 0.9754 | 0.6469 | 0.7404 |
36
- | 0.40 | 16525 | 9229 | 580 | 0.6416 | 0.9661 | 0.6878 | 0.7711 |
37
- | 0.50 | 16104 | 6956 | 1001 | 0.6984 | 0.9415 | 0.7364 | 0.8019 |
38
- | 0.60 | 14741 | 4323 | 2364 | 0.7732 | 0.8618 | 0.7895 | 0.8151 |
39
- | 0.70 | 10845 | 1783 | 6260 | 0.8588 | 0.6340 | 0.8019 | 0.7295 |
40
 
41
- **Best F0.5**: 0.8019 (thr=0.7)
42
- **Best F1**: 0.8151 (thr=0.6)
43
 
44
  ## Evaluation breakdown (holdout)
45
 
46
  | group | examples | spans | thr | precision | recall | f0.5 | f1 |
47
  | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
48
- | overall | 15332 | 17332 | 0.70 | 0.8588 | 0.6340 | 0.8019 | 0.7295 |
49
- | prwp | 7758 | 11375 | 0.70 | 0.8617 | 0.6443 | 0.8072 | 0.7373 |
50
- | fcv | 7574 | 5957 | 0.70 | 0.8530 | 0.6139 | 0.7913 | 0.7139 |
51
- | general_prwp | 7758 | 11375 | 0.70 | 0.8617 | 0.6443 | 0.8072 | 0.7373 |
52
- | fcv_pads_east_africa | 5351 | 3920 | 0.70 | 0.8433 | 0.5900 | 0.7766 | 0.6942 |
53
- | jdc_operational | 145 | 110 | 0.70 | 0.8095 | 0.6239 | 0.7640 | 0.7047 |
54
- | refugee_pads | 573 | 377 | 0.70 | 0.8759 | 0.6622 | 0.8228 | 0.7542 |
55
- | reliefweb | 1505 | 1550 | 0.70 | 0.8730 | 0.6611 | 0.8204 | 0.7525 |
56
 
57
  ### Per-label (overall)
58
 
59
  | label | examples | spans | thr | precision | recall | f0.5 | f1 |
60
  | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
61
- | NAMED_DATA | 15332 | 7107 | 0.70 | 0.8303 | 0.7211 | 0.8059 | 0.7718 |
62
- | DESCRIPTIVE_DATA | 15332 | 8601 | 0.70 | 0.8167 | 0.4660 | 0.7098 | 0.5934 |
63
- | VAGUE_DATA | 15332 | 1624 | 0.70 | 0.5038 | 0.5302 | 0.5089 | 0.5167 |
64
 
65
  <details>
66
  <summary>Origin breakdown (per-origin metrics)</summary>
67
 
68
  | origin | examples | spans | thr | precision | recall | f0.5 | f1 |
69
  | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
70
- | fcv_pads_east_africa | 5351 | 3920 | 0.70 | 0.8433 | 0.5900 | 0.7766 | 0.6942 |
71
- | general_prwp | 7758 | 11375 | 0.70 | 0.8617 | 0.6443 | 0.8072 | 0.7373 |
72
- | jdc_operational | 145 | 110 | 0.70 | 0.8095 | 0.6239 | 0.7640 | 0.7047 |
73
- | refugee_pads | 573 | 377 | 0.70 | 0.8759 | 0.6622 | 0.8228 | 0.7542 |
74
- | reliefweb | 1505 | 1550 | 0.70 | 0.8730 | 0.6611 | 0.8204 | 0.7525 |
75
 
76
  </details>
77
 
@@ -80,8 +80,8 @@ Fine-tune of `urchade/gliner_large-v2.1` for data-use mention extraction
80
 
81
  | label | examples | spans | thr | precision | recall | f0.5 | f1 |
82
  | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
83
- | NAMED_DATA | 15332 | 7107 | 0.70 | 0.8303 | 0.7211 | 0.8059 | 0.7718 |
84
- | DESCRIPTIVE_DATA | 15332 | 8601 | 0.70 | 0.8167 | 0.4660 | 0.7098 | 0.5934 |
85
- | VAGUE_DATA | 15332 | 1624 | 0.70 | 0.5038 | 0.5302 | 0.5089 | 0.5167 |
86
 
87
  </details>
 
21
  - base model: `urchade/gliner_large-v2.1`
22
  - dataset: `rafmacalaba/data-use-mentions-v2` (gliner config)
23
  - corpus: `all`
24
+ - epochs: 5
25
  - learning rate: 5e-06
26
  - batch size: 16
27
  - precision: bf16
 
30
 
31
  | thr | tp | fp | fn | precision | recall | f0.5 | f1 |
32
  | --- | --- | --- | --- | --- | --- | --- | --- |
33
+ | 0.10 | 16553 | 64389 | 552 | 0.2045 | 0.9677 | 0.2428 | 0.3377 |
34
+ | 0.20 | 16274 | 32132 | 831 | 0.3362 | 0.9514 | 0.3861 | 0.4968 |
35
+ | 0.30 | 15597 | 18446 | 1508 | 0.4582 | 0.9118 | 0.5088 | 0.6099 |
36
+ | 0.40 | 13992 | 10141 | 3113 | 0.5798 | 0.8180 | 0.6156 | 0.6786 |
37
+ | 0.50 | 10189 | 4475 | 6916 | 0.6948 | 0.5957 | 0.6724 | 0.6414 |
38
+ | 0.60 | 4852 | 1332 | 12253 | 0.7846 | 0.2837 | 0.5798 | 0.4167 |
39
+ | 0.70 | 1272 | 245 | 15833 | 0.8385 | 0.0744 | 0.2745 | 0.1366 |
40
 
41
+ **Best F0.5**: 0.6724 (thr=0.5)
42
+ **Best F1**: 0.6786 (thr=0.4)
43
 
44
  ## Evaluation breakdown (holdout)
45
 
46
  | group | examples | spans | thr | precision | recall | f0.5 | f1 |
47
  | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
48
+ | overall | 15332 | 17332 | 0.50 | 0.6948 | 0.5957 | 0.6724 | 0.6414 |
49
+ | prwp | 7758 | 11375 | 0.50 | 0.7719 | 0.6282 | 0.7382 | 0.6927 |
50
+ | fcv | 7574 | 5957 | 0.50 | 0.5642 | 0.5318 | 0.5574 | 0.5475 |
51
+ | general_prwp | 7758 | 11375 | 0.50 | 0.7719 | 0.6282 | 0.7382 | 0.6927 |
52
+ | fcv_pads_east_africa | 5351 | 3920 | 0.50 | 0.5332 | 0.5332 | 0.5332 | 0.5332 |
53
+ | jdc_operational | 145 | 110 | 0.50 | 0.4776 | 0.5872 | 0.4961 | 0.5267 |
54
+ | refugee_pads | 573 | 377 | 0.50 | 0.4732 | 0.5442 | 0.4859 | 0.5062 |
55
+ | reliefweb | 1505 | 1550 | 0.50 | 0.7178 | 0.5212 | 0.6675 | 0.6039 |
56
 
57
  ### Per-label (overall)
58
 
59
  | label | examples | spans | thr | precision | recall | f0.5 | f1 |
60
  | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
61
+ | NAMED_DATA | 15332 | 7107 | 0.60 | 0.6049 | 0.4388 | 0.5623 | 0.5087 |
62
+ | DESCRIPTIVE_DATA | 15332 | 8601 | 0.50 | 0.4997 | 0.2891 | 0.4361 | 0.3663 |
63
+ | VAGUE_DATA | 15332 | 1624 | 0.40 | 0.2019 | 0.1078 | 0.1719 | 0.1405 |
64
 
65
  <details>
66
  <summary>Origin breakdown (per-origin metrics)</summary>
67
 
68
  | origin | examples | spans | thr | precision | recall | f0.5 | f1 |
69
  | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
70
+ | fcv_pads_east_africa | 5351 | 3920 | 0.50 | 0.5332 | 0.5332 | 0.5332 | 0.5332 |
71
+ | general_prwp | 7758 | 11375 | 0.50 | 0.7719 | 0.6282 | 0.7382 | 0.6927 |
72
+ | jdc_operational | 145 | 110 | 0.50 | 0.4776 | 0.5872 | 0.4961 | 0.5267 |
73
+ | refugee_pads | 573 | 377 | 0.50 | 0.4732 | 0.5442 | 0.4859 | 0.5062 |
74
+ | reliefweb | 1505 | 1550 | 0.50 | 0.7178 | 0.5212 | 0.6675 | 0.6039 |
75
 
76
  </details>
77
 
 
80
 
81
  | label | examples | spans | thr | precision | recall | f0.5 | f1 |
82
  | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
83
+ | NAMED_DATA | 15332 | 7107 | 0.60 | 0.6049 | 0.4388 | 0.5623 | 0.5087 |
84
+ | DESCRIPTIVE_DATA | 15332 | 8601 | 0.50 | 0.4997 | 0.2891 | 0.4361 | 0.3663 |
85
+ | VAGUE_DATA | 15332 | 1624 | 0.40 | 0.2019 | 0.1078 | 0.1719 | 0.1405 |
86
 
87
  </details>