Hugging Face's logo

Buckets:
Mercity
/
Desicion-Model-Custom

Files
xet
Mercity/Desicion-Model-Custom / jevals /logs /jevals_score.log
Pranav2748's picture
Pranav2748
download
raw
7.72 kB
== pubmedqa (noul)
Gemini 3.8 Flash DS 73.0 [ 62.1, 82.2] acc 0.925 ECE 0.020
Jev DS 69.0 [ 59.8, 76.5] acc 0.913 ECE 0.050
OURS rl60_think (cal) DS 63.5 [ 51.2, 74.8] acc 0.900 ECE 0.045
Qwen3.8 Flash DS 62.4 [ 50.9, 72.5] acc 0.897 ECE 0.025
GLM-5.3 DS 60.6 [ 50.0, 69.8] acc 0.887 ECE 0.020
OURS v2_joint_think (cal) DS 60.2 [ 47.7, 71.9] acc 0.883 ECE 0.054
OURS rl60_think (raw T=1) DS 58.9 [ 44.4, 72.3] acc 0.900 ECE 0.094
OURS rl60 (raw T=1) DS 58.7 [ 48.3, 68.0] acc 0.863 ECE 0.027
OURS rl60 (cal) DS 58.6 [ 47.3, 68.8] acc 0.863 ECE 0.036
OURS rl40 (raw T=1) DS 58.3 [ 47.7, 67.6] acc 0.860 ECE 0.025
OURS rl40 (cal) DS 58.1 [ 46.8, 68.3] acc 0.860 ECE 0.038
Mistral Medium 3.5 DS 58.0 [ 45.3, 69.8] acc 0.888 ECE 0.052
OURS v2_joint (raw T=1) DS 58.0 [ 47.5, 67.3] acc 0.857 ECE 0.025
OURS v2_joint (cal) DS 57.8 [ 46.4, 68.0] acc 0.857 ECE 0.033
Mercury 2.5 DS 55.7 [ 44.8, 65.4] acc 0.871 ECE 0.047
OURS v2_joint_think (raw T=1) DS 53.8 [ 38.5, 67.9] acc 0.883 ECE 0.106
OURS v1_joint (cal) DS 50.3 [ 38.8, 60.2] acc 0.823 ECE 0.066
OURS v1_joint (raw T=1) DS 49.2 [ 36.7, 59.9] acc 0.823 ECE 0.070
OURS base (raw T=1) DS 48.1 [ 37.8, 57.2] acc 0.807 ECE 0.065
DeepSeek V4.1 Flash DS 47.5 [ 34.7, 59.1] acc 0.837 ECE 0.062
OURS v2_2b_ro (raw T=1) DS 47.0 [ 36.0, 56.8] acc 0.820 ECE 0.015
OURS v2_2b_ro (cal) DS 44.6 [ 31.5, 56.5] acc 0.820 ECE 0.073
OURS v2_2b (raw T=1) DS 43.6 [ 32.5, 53.5] acc 0.790 ECE 0.063
OURS base_2b_think (cal) DS 43.6 [ 29.8, 55.6] acc 0.843 ECE 0.026
OURS v2_2b (cal) DS 41.6 [ 27.9, 53.5] acc 0.790 ECE 0.086
OURS base (cal) DS 36.4 [ 29.4, 42.8] acc 0.807 ECE 0.157
OURS base_2b_think (raw T=1) DS 33.5 [ 15.0, 50.1] acc 0.843 ECE 0.157
OURS base_2b (cal) DS 7.9 [ 4.4, 11.1] acc 0.683 ECE 0.126
OURS base_2b (raw T=1) DS 0.1 [ -12.1, 12.9] acc 0.683 ECE 0.226
Label prior DS 0.0 [ 0.0, 0.0] acc 0.620 ECE 0.000
== helpsteer2 (score)
Jev DS 9.2 [ -4.4, 21.0] acc 0.413 ECE 0.197
GLM-5.3 DS 7.8 [ -4.6, 18.6] acc 0.430 ECE 0.129
OURS v1_joint (cal) DS 7.5 [ -1.2, 16.7] acc 0.393 ECE 0.168
OURS v1_joint (raw T=1) DS 6.8 [ -2.2, 16.2] acc 0.393 ECE 0.180
OURS v2_joint (raw T=1) DS 6.1 [ -2.4, 14.4] acc 0.430 ECE 0.076
OURS rl40 (raw T=1) DS 6.0 [ -2.5, 14.4] acc 0.420 ECE 0.093
OURS rl60 (raw T=1) DS 6.0 [ -2.4, 14.4] acc 0.437 ECE 0.096
Gemini 3.8 Flash DS 4.6 [ -12.2, 19.0] acc 0.424 ECE 0.226
OURS base (cal) DS 4.1 [ -6.4, 14.0] acc 0.403 ECE 0.089
OURS v2_2b_ro (raw T=1) DS 2.7 [ -4.6, 10.2] acc 0.403 ECE 0.107
OURS rl60 (cal) DS 2.4 [ -7.2, 11.7] acc 0.437 ECE 0.130
OURS v2_joint (cal) DS 2.3 [ -7.3, 11.8] acc 0.430 ECE 0.136
OURS rl40 (cal) DS 2.1 [ -7.6, 11.6] acc 0.420 ECE 0.151
OURS base (raw T=1) DS 2.0 [ -9.8, 13.1] acc 0.403 ECE 0.151
OURS v2_2b (raw T=1) DS 0.7 [ -6.5, 8.3] acc 0.350 ECE 0.171
Label prior DS 0.0 [ 0.0, 0.0] acc 0.417 ECE 0.000
OURS v2_2b_ro (cal) DS -0.4 [ -8.2, 7.9] acc 0.403 ECE 0.138
Qwen3.8 Flash DS -1.4 [ -15.4, 11.5] acc 0.361 ECE 0.258
OURS v2_2b (cal) DS -4.0 [ -11.6, 4.4] acc 0.350 ECE 0.224
Mercury 2.5 DS -5.5 [ -16.2, 4.0] acc 0.417 ECE 0.315
OURS base_2b (cal) DS -6.3 [ -12.5, -0.7] acc 0.323 ECE 0.011
OURS v2_joint_think (cal) DS -13.0 [ -26.6, 0.6] acc 0.407 ECE 0.345
Mistral Medium 3.5 DS -13.7 [ -28.3, -0.1] acc 0.439 ECE 0.336
DeepSeek V4.1 Flash DS -19.0 [ -36.0, -3.3] acc 0.347 ECE 0.335
OURS rl60_think (cal) DS -20.1 [ -35.3, -4.9] acc 0.413 ECE 0.378
OURS base_2b_think (cal) DS -24.4 [ -38.1, -11.7] acc 0.337 ECE 0.450
OURS v2_joint_think (raw T=1) DS -26.2 [ -41.2, -10.4] acc 0.407 ECE 0.438
OURS base_2b (raw T=1) DS -26.5 [ -32.4, -19.8] acc 0.323 ECE 0.412
OURS rl60_think (raw T=1) DS -43.0 [ -61.4, -24.7] acc 0.413 ECE 0.519
OURS base_2b_think (raw T=1) DS -53.5 [ -70.3, -37.2] acc 0.337 ECE 0.663
== banking77 (choice)
Gemini 3.8 Flash DS 74.1 [ 68.4, 79.8] acc 0.846 ECE 0.032
Jev DS 67.8 [ 61.3, 74.5] acc 0.797 ECE 0.098
GLM-5.3 DS 66.8 [ 62.0, 71.7] acc 0.789 ECE 0.062
DeepSeek V4.1 Flash DS 63.9 [ 58.5, 69.6] acc 0.759 ECE 0.028
Qwen3.8 Flash DS 62.4 [ 55.9, 68.9] acc 0.768 ECE 0.095
OURS v1_joint (cal) DS 61.5 [ 55.9, 67.1] acc 0.733 ECE 0.050
OURS rl60 (cal) DS 61.3 [ 55.5, 67.3] acc 0.731 ECE 0.028
OURS v2_joint (cal) DS 61.3 [ 55.4, 67.3] acc 0.734 ECE 0.021
OURS rl40 (cal) DS 61.3 [ 55.5, 67.3] acc 0.732 ECE 0.021
OURS rl60 (raw T=1) DS 61.0 [ 54.7, 67.3] acc 0.731 ECE 0.059
OURS v2_joint (raw T=1) DS 60.9 [ 54.6, 67.3] acc 0.734 ECE 0.058
OURS rl40 (raw T=1) DS 60.9 [ 54.6, 67.2] acc 0.732 ECE 0.060
OURS v1_joint (raw T=1) DS 60.6 [ 54.2, 66.9] acc 0.733 ECE 0.079
Mistral Medium 3.5 DS 59.5 [ 54.3, 64.8] acc 0.745 ECE 0.081
OURS base (cal) DS 59.3 [ 53.7, 64.9] acc 0.719 ECE 0.043
OURS v2_joint_think (cal) DS 59.3 [ 53.9, 65.1] acc 0.733 ECE 0.119
OURS rl60_think (cal) DS 59.3 [ 53.6, 64.9] acc 0.733 ECE 0.134
OURS base (raw T=1) DS 56.2 [ 49.3, 63.1] acc 0.719 ECE 0.140
OURS v2_2b (cal) DS 54.2 [ 48.4, 59.9] acc 0.668 ECE 0.034
Mercury 2.5 DS 54.0 [ 47.8, 60.6] acc 0.703 ECE 0.104
OURS v2_2b (raw T=1) DS 53.9 [ 47.8, 60.0] acc 0.668 ECE 0.048
OURS v2_2b_ro (cal) DS 53.9 [ 48.4, 59.8] acc 0.679 ECE 0.046
OURS v2_2b_ro (raw T=1) DS 53.9 [ 48.0, 60.1] acc 0.679 ECE 0.027
OURS v2_joint_think (raw T=1) DS 51.9 [ 43.2, 61.3] acc 0.733 ECE 0.220
OURS rl60_think (raw T=1) DS 49.5 [ 40.3, 58.8] acc 0.733 ECE 0.246
OURS base_2b_think (raw T=1) DS 36.5 [ 25.7, 47.3] acc 0.687 ECE 0.313
OURS base_2b (cal) DS 31.9 [ 28.8, 35.2] acc 0.570 ECE 0.262
OURS base_2b_think (cal) DS 28.8 [ 25.9, 31.6] acc 0.687 ECE 0.407
OURS base_2b (raw T=1) DS 28.6 [ 20.9, 36.3] acc 0.570 ECE 0.304
Label prior DS 0.0 [ 0.0, 0.0] acc 0.013 ECE 0.000

Xet Storage Details

Size:
7.72 kB
·
Xet hash:
9c17bebe3e711a50b5d42aae91e0fb6ced3bcaf2970dbaebf8122772d36a3ada

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.