soodoku commited on
Commit
96f9cd5
·
verified ·
1 Parent(s): 6a1f48c

Document v3 comparison limits

Browse files
Files changed (1) hide show
  1. README.md +14 -0
README.md CHANGED
@@ -71,6 +71,20 @@ With the default abstention rule, English coverage is 96.54% and accuracy on
71
  retained estimates is 98.54%. Hindi coverage is 97.99% and retained accuracy is
72
  99.18%.
73
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
74
  These results do not establish accuracy for every state, script, community, or
75
  time period. The Hindi test shares the land-record source used for training,
76
  although normalized names are partitioned. Religion labels derived from caste
 
71
  retained estimates is 98.54%. Hindi coverage is 97.99% and retained accuracy is
72
  99.18%.
73
 
74
+ A paired audit recalibrated the released v2 model on v3's calibration
75
+ partition. V3 improved accuracy by 1.19 percentage points (95% name-cluster
76
+ bootstrap interval: 0.95 to 1.43), Muslim recall by 15.62 points (13.55 to
77
+ 17.75), Muslim F1 by 0.086 (0.071 to 0.103), and Brier score by 0.0122 (0.0106
78
+ to 0.0139). Muslim precision was 2.04 points lower (-3.44 to -0.67), reflecting
79
+ v2's more conservative recalibrated operating point.
80
+
81
+ This comparison supports the complete v3 pipeline on the available SEPRI
82
+ population. It does not isolate the effect of architecture from changes in
83
+ training data and calibration. The evaluation partition was held out from
84
+ parameter fitting and calibration but was inspected during architecture
85
+ development, so it is developmental rather than pristine confirmatory
86
+ evidence.
87
+
88
  These results do not establish accuracy for every state, script, community, or
89
  time period. The Hindi test shares the land-record source used for training,
90
  although normalized names are partitioned. Religion labels derived from caste