Native-Bird2 / results /RESULTS.md
UncleanCode's picture
Upload via Autoresearch export_data
989d4ef verified
|
Raw History Blame Contribute Delete
5.51 kB

Native-Bird x SigLIP image-text alignment: full run (2026-10-18, 1x H100)

Test = 809 held-out images; gallery = every test caption in that language (eng 809, ibo 751, hau 769, yor 764). Chance R@1 ~0.13%. Best checkpoint by val mean R: native_bird (frozen text), epoch 3 (val 60.46) vs unfreeze-top-2 epoch 2 (val 60.35): a near tie.

run lang i2t R@1 i2t R@5 i2t R@10 t2i R@1 t2i R@5 t2i R@10 mean R
untrained heads eng 0.37 1.11 1.48 0.12 0.87 1.73 0.95
untrained heads ibo 0.0 0.8 2.4 0.13 0.8 1.6 0.96
untrained heads hau 0.26 0.78 1.69 0.13 1.04 1.69 0.93
untrained heads yor 0.26 0.92 2.36 0.26 0.92 1.05 0.96
mCLIP zero-shot eng 58.34 84.67 92.21 57.85 83.31 89.99 77.73
mCLIP zero-shot ibo 6.13 14.38 20.91 4.26 11.98 16.11 12.29
mCLIP zero-shot hau 3.38 6.63 8.71 1.82 4.16 6.11 5.13
mCLIP zero-shot yor 2.36 4.84 6.41 1.31 2.75 4.84 3.75
Native-Bird frozen eng 46.48 77.38 88.5 44.99 76.64 87.02 70.17
Native-Bird frozen ibo 39.28 70.71 82.29 39.01 69.64 81.49 63.74
Native-Bird frozen hau 33.16 61.64 74.64 32.25 59.43 74.9 56.0
Native-Bird frozen yor 25.52 53.8 64.66 23.56 52.75 65.84 47.69
Native-Bird unfreeze top-2 eng 44.99 78.74 89.0 46.23 76.14 87.89 70.5
Native-Bird unfreeze top-2 ibo 37.95 70.57 83.49 39.68 70.17 83.36 64.2
Native-Bird unfreeze top-2 hau 31.73 60.47 74.9 30.3 60.73 74.38 55.42
Native-Bird unfreeze top-2 yor 23.3 52.88 64.66 24.87 53.27 64.4 47.23
frozen, test_unfiltered eng 46.48 77.38 88.5 44.99 76.64 87.02 70.17
frozen, test_unfiltered ibo 36.59 67.37 79.6 35.6 66.75 79.23 60.86
frozen, test_unfiltered hau 31.52 59.21 72.06 30.9 57.73 72.81 54.04
frozen, test_unfiltered yor 24.6 52.78 63.54 22.5 51.79 64.4 46.6
unfreeze-2, test_unfiltered eng 44.99 78.74 89.0 46.23 76.14 87.89 70.5
unfreeze-2, test_unfiltered ibo 35.72 67.86 81.09 36.96 67.12 80.22 61.5
unfreeze-2, test_unfiltered hau 29.91 58.71 73.05 28.31 58.22 72.44 53.44
unfreeze-2, test_unfiltered yor 22.74 50.31 62.42 23.98 52.41 62.79 45.77

Smoke (CPU, 128 images, 1 epoch, mean R): untrained 4.6/3.5/2.8/5.3, mCLIP 92.3/22.5/7.7/6.8, trained Native-Bird 42.1/42.4/38.2/33.9 (eng/ibo/hau/yor). Not comparable to the 809-image gallery.

Igbo image search (best checkpoint, 809 test images)

All 751 held-out Igbo captions as queries: hit@1 37.42, hit@5 68.44, hit@10 80.56, median rank 2. English: hit@1 44.99, hit@5 76.64, hit@10 87.02.

Igbo query English Igbo top-5 top-5 overlap with English query
Nkịta na-agba ọsọ n'elu ahịhịa A dog running on the grass 697582336_601462e052.jpg, 2114739371_83aa8bdb0e.jpg, 1514957266_a19827c538.jpg, 240583223_e26e17ee96.jpg, 3358380484_b99b48f0c9.jpg 3/5
Ụmụaka na-egwu egwu n'ime mmiri Children playing in the water 1095476286_87d4f8664e.jpg, 3041689520_c481bdb20e.jpg, 2951092164_4940b9a517.jpg, 3545427060_c16a8b7dfd.jpg, 3747543364_bf5b548527.jpg 5/5
Otu nwoke na-agba ịnyịnya ígwè A man riding a bicycle 3649916507_b88a3d2082.jpg, 3391924827_53b31542ce.jpg, 3079073247_97e88e2eb7.jpg, 3665169936_01ae94c9fd.jpg, 2192411521_9c7e488c5e.jpg 4/5
Otu nwanyị yi uwe na-acha uhie uhie A woman wearing a red dress 495055747_a75872762a.jpg, 613900608_2e49415772.jpg, 3534952095_975cca0056.jpg, 2695085448_a11833df95.jpg, 1821238649_2fda79d6d7.jpg 4/5
Ndị mmadụ nọ n'elu ugwu People on top of a mountain 2176980976_7054c99621.jpg, 3340575518_137ce2695f.jpg, 1428578577_82864facae.jpg, 3229821595_77ace81c6b.jpg, 95783195_e1ba3f57ca.jpg 4/5
Otu nwata nwoke na-awụli elu A boy jumping in the air 3348811097_0e09baa26f.jpg, 2061354254_faa5bd294b.jpg, 3229913073_e7857a5966.jpg, 3741827382_71e93298d0.jpg, 3396036947_0af6c3aab7.jpg 3/5
Nkịta abụọ na-egwu egwu Two dogs playing together 3473534758_1ae3847781.jpg, 522486784_978021d537.jpg, 78984436_ad96eaa802.jpg, 255741044_1102982213.jpg, 3264678536_46601d25f0.jpg 2/5
Ọtụtụ mmadụ nọ n'okporo ụzọ A crowd of people on a street 3638374272_444f5e0457.jpg, 256444892_efcb3bd824.jpg, 3225037367_a71fa86319.jpg, 3433470650_a8b1c27173.jpg, 1685990174_09c4fb7df8.jpg 4/5
held-out Igbo caption English reference GT rank in top-5
Ndị ikom abụọ na-egwuri egwu na egwuregwu egwuregwu ọnụ . Two men are playing in a sports game together . 8 no
A isi awọ agba nkịta na-eje ije na mmiri ájá na a osimiri . A grey colored dog walks in wet sand at a beach . 14 no
Nkịta na-ejide Frisbee na ndị na-ekiri ya na ndabere . A dog catching a Frisbee with onlookers in the background . 3 yes
Ụmụ agbọghọ abụọ na-acha odo odo na-egwuri egwu ọnụ . Two blond girls are playing together . 1 yes
Nkịta na-awakpo nwoke yi ngwá ọrụ nchebe . A dog attacking a man wearing protective gear . 3 yes
Nkịta ojii na aja aja na-agba ọsọ gafee ahịhịa. A black and brown dog running across the grass . 4 yes