Document v3 comparison limits
Browse files
README.md
CHANGED
|
@@ -71,6 +71,20 @@ With the default abstention rule, English coverage is 96.54% and accuracy on
|
|
| 71 |
retained estimates is 98.54%. Hindi coverage is 97.99% and retained accuracy is
|
| 72 |
99.18%.
|
| 73 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 74 |
These results do not establish accuracy for every state, script, community, or
|
| 75 |
time period. The Hindi test shares the land-record source used for training,
|
| 76 |
although normalized names are partitioned. Religion labels derived from caste
|
|
|
|
| 71 |
retained estimates is 98.54%. Hindi coverage is 97.99% and retained accuracy is
|
| 72 |
99.18%.
|
| 73 |
|
| 74 |
+
A paired audit recalibrated the released v2 model on v3's calibration
|
| 75 |
+
partition. V3 improved accuracy by 1.19 percentage points (95% name-cluster
|
| 76 |
+
bootstrap interval: 0.95 to 1.43), Muslim recall by 15.62 points (13.55 to
|
| 77 |
+
17.75), Muslim F1 by 0.086 (0.071 to 0.103), and Brier score by 0.0122 (0.0106
|
| 78 |
+
to 0.0139). Muslim precision was 2.04 points lower (-3.44 to -0.67), reflecting
|
| 79 |
+
v2's more conservative recalibrated operating point.
|
| 80 |
+
|
| 81 |
+
This comparison supports the complete v3 pipeline on the available SEPRI
|
| 82 |
+
population. It does not isolate the effect of architecture from changes in
|
| 83 |
+
training data and calibration. The evaluation partition was held out from
|
| 84 |
+
parameter fitting and calibration but was inspected during architecture
|
| 85 |
+
development, so it is developmental rather than pristine confirmatory
|
| 86 |
+
evidence.
|
| 87 |
+
|
| 88 |
These results do not establish accuracy for every state, script, community, or
|
| 89 |
time period. The Hindi test shares the land-record source used for training,
|
| 90 |
although normalized names are partitioned. Religion labels derived from caste
|