freshwater-quantization-models / paper_claim_check.csv
Datadragon110's picture
Upload folder using huggingface_hub
ea05031 verified
Raw History Blame Contribute Delete
1.82 kB
claim,paper,measured,delta,tol,status
Train-folder images (paper: 1485 + 262),1747.0,1750.0,3.0,0.0,MISMATCH
Test-folder images,697.0,700.0,3.0,0.0,MISMATCH
Test files that also exist in Train (should be 0),0.0,700.0,700.0,0.0,MISMATCH
"Test images with a near-duplicate in Train, % (should be ~0)",0.0,100.0,100.0,1.0,MISMATCH
classes,7.0,7.0,0.0,0.0,MATCH
EfficientNetB0 total params,4058538.0,4058538.0,0.0,0.0,MATCH
EfficientNetB0 frozen params,4049571.0,4049571.0,0.0,0.0,MATCH
MobileNetV3Small total params,445039.0,445039.0,0.0,0.0,MATCH
MobileNetV3Small frozen params,441000.0,441000.0,0.0,0.0,MATCH
"EfficientNetB0 fp32 acc, PAPER protocol (leaky Test/)",90.24,93.286,3.046,3.0,MISMATCH
"EfficientNetB0 INT8 acc, PAPER protocol (leaky Test/)",69.44,79.143,9.703,3.0,MISMATCH
"EfficientNetB0 fp32 acc, LEAKAGE-FREE grouped CV",90.24,88.004,-2.236,1.993,MISMATCH
"EfficientNetB0 INT8 acc, LEAKAGE-FREE grouped CV",69.44,74.176,4.736,6.209,MATCH
"MobileNetV3Small fp32 acc, PAPER protocol (leaky Test/)",81.78,83.714,1.934,3.0,MATCH
"MobileNetV3Small INT8 acc, PAPER protocol (leaky Test/)",59.08,65.714,6.634,3.0,MISMATCH
"MobileNetV3Small fp32 acc, LEAKAGE-FREE grouped CV",81.78,79.287,-2.493,4.511,MATCH
"MobileNetV3Small INT8 acc, LEAKAGE-FREE grouped CV",59.08,60.007,0.927,10.445,MATCH
EfficientNetB0 fp32 size (KiB),15700.1,15700.116,0.016,157.001,MATCH
EfficientNetB0 INT8 size (KiB),4808.1,4808.125,0.025,48.081,MATCH
MobileNetV3Small fp32 size (KiB),1690.8,1690.846,0.046,16.908,MATCH
MobileNetV3Small INT8 size (KiB),603.7,603.717,0.017,6.037,MATCH
size reduction EffNet fp32 -> INT8 (x),3.3,3.265,-0.035,0.1,MATCH
size reduction MNv3 fp32 -> INT8 (x),2.8,2.801,0.001,0.1,MATCH
size reduction EffNet fp32 -> MNv3 INT8 (x),26.0,26.006,0.006,0.5,MATCH
param ratio EffNet / MNv3 (x),9.1,9.12,0.02,0.1,MATCH