vadimbelsky commited on
Commit
b1a3621
·
verified ·
1 Parent(s): 0caa6ee

Model card: link the canonical leaderboard; declare the fine-tune relation to the base model

Browse files
Files changed (1) hide show
  1. README.md +9 -5
README.md CHANGED
@@ -22,7 +22,7 @@ model-index:
22
  value: 0.7766
23
  name: F1, character-level, label-agnostic, averaged over 7 datasets
24
  - type: precision
25
- value: 0.7100
26
  name: Precision
27
  - type: recall
28
  value: 0.8780
@@ -78,8 +78,12 @@ each dataset (1,000 resamples for each model's F1; 2,000 for the paired differen
78
  | precision | 0.710 | 0.744 |
79
  | recall | **0.878** | 0.783 |
80
  | touch rate (not ranked; lower is better) | **0.021** | 0.288 |
81
- | latency, median ms per row, batch size 1 | **20.3** | 140.6 |
82
- | latency, 95th percentile ms per row | **33.5** | 199.7 |
 
 
 
 
83
 
84
  The paired difference in average F1 is **+0.0233, 95 % CI [+0.0147, +0.0322]** (2,000
85
  resamples, seed 0); the interval excludes zero.
@@ -90,8 +94,8 @@ Three qualifications belong with that result:
90
  entity leaves the device unredacted while a false positive masks a word unnecessarily, so
91
  the trade is in the intended direction — but it is a trade, and the F1 average conceals it.
92
  - **Latency was measured on a shared workstation** under a higher load average for this model
93
- (1-minute load 20.7) than for the comparison model (11.1). Absolute milliseconds depend on the
94
- host; the ordering is the informative part.
95
  - **Touch rate** is the share of characters belonging to format-fixed identifiers that a model
96
  marks anyway. It is reported, not ranked.
97
 
 
22
  value: 0.7766
23
  name: F1, character-level, label-agnostic, averaged over 7 datasets
24
  - type: precision
25
+ value: 0.7104
26
  name: Precision
27
  - type: recall
28
  value: 0.8780
 
78
  | precision | 0.710 | 0.744 |
79
  | recall | **0.878** | 0.783 |
80
  | touch rate (not ranked; lower is better) | **0.021** | 0.288 |
81
+ | latency, median ms per row, batch size 1 | **20.3** † | 140.6 |
82
+ | latency, 95th percentile ms per row | **33.5** † | 199.7 |
83
+
84
+ † Measured on the previous release (v15, revision `e98225e07225`), which has the same
85
+ architecture and is exported to int8 ONNX the same way; latency has not been re-measured for
86
+ this build.
87
 
88
  The paired difference in average F1 is **+0.0233, 95 % CI [+0.0147, +0.0322]** (2,000
89
  resamples, seed 0); the interval excludes zero.
 
94
  entity leaves the device unredacted while a false positive masks a word unnecessarily, so
95
  the trade is in the intended direction — but it is a trade, and the F1 average conceals it.
96
  - **Latency was measured on a shared workstation** under a higher load average for this model
97
+ (1-minute load 20.7) than for the comparison model (11.1), and for this model on the previous
98
+ release (†). Absolute milliseconds depend on the host; the ordering is the informative part.
99
  - **Touch rate** is the share of characters belonging to format-fixed identifiers that a model
100
  marks anyway. It is reported, not ranked.
101