Download docs/LABELING_GUIDE_NATIVE.md from sapinsapin/wikalid: direct link, hf CLI and curl.
- Browser
- Download file 7.37 kB
-
https://huggingface.co/sapinsapin/wikalid/resolve/main/docs/LABELING_GUIDE_NATIVE.md
- Command line
-
hf download hf://sapinsapin/wikalid/docs/LABELING_GUIDE_NATIVE.md
-
curl -L -o LABELING_GUIDE_NATIVE.md https://huggingface.co/sapinsapin/wikalid/resolve/main/docs/LABELING_GUIDE_NATIVE.md
Native-speaker language labeling guide
You will see about 675 short texts (at most 300 characters each). For each one, say which language it is written in. There are no right-or-wrong "tricks": we want your honest native judgement. About 50 rows are repeated on purpose (to measure consistency) - just label every row on its own, as if you had never seen it.
Time: about 1-1.5 minutes per 10 rows, so roughly 1.5 hours in total. You can stop and continue later; leave unfinished rows empty.
What to fill in (columns E-H of the sheet; columns A-C are the item)
| column | what to put |
|---|---|
annotator_label |
one code from the list below (dropdown in the Excel file) |
code_switched |
yes if a second language also appears in the text (e.g. Taglish), otherwise no |
confidence |
high (sure), medium, or low (guessing) |
note |
optional; required for oth_ph and oth (write the language name), helpful for anything odd |
Codes
| code | use it for |
|---|---|
| tgl | Tagalog / Filipino, including Taglish whose sentence frame is Tagalog |
| eng | English, including Taglish whose frame is English |
| ceb | Cebuano / Bisaya |
| hil | Hiligaynon / Ilonggo |
| war | Waray |
| ilo | Ilocano |
| bcl | Bikol (any variety: Naga, Legazpi, Rinconada, Albay, Sorsogon...) |
| pam | Kapampangan |
| pag | Pangasinan |
| cbk | Chavacano (Zamboanga, Cavite, Ternate, Davao...) |
| krj | Kinaray-a |
| akl | Aklanon |
| sgd | Surigaonon |
| tsg | Tausug |
| oth_ph | another Philippine language (write which in note) |
| spa | Spanish |
| ind | Indonesian |
| msa | Malay |
| oth | any other language (write which in note) |
| mixed | two languages in clearly separate parts and no single frame language |
| not_language | only names, numbers, codes, symbols, gibberish, or a list of proper nouns |
| unsure | you really cannot tell (this is a fine answer - please use it instead of guessing) |
Edge cases
- Taglish (code-switching). Pick the language that supplies the sentence frame - the grammar words (Tagalog: ang, ng, sa, na, ay, mga, ko, mo, niya, hindi, para sa...; English: the, is, to, of, and, for...). "Nag-meeting kami kanina about the project" ->
tgl, code_switched = yes. "The mayor said na hindi sila aalis" ->engif the English part carries the frame, otherwisetgl; use your judgement and mark confidencemediumif it is half-half. Usemixedonly if two separate chunks (e.g. a headline in English followed by a Tagalog sentence) have no single frame. - Names, numbers, places, acronyms (Juan dela Cruz, PNP, P20,000, Kalibo) do not decide the language. Look at the rest of the text. If nothing else is left, use
not_language. - Quotations. If a Tagalog sentence quotes English words, or an English sentence quotes a Tagalog phrase, label the language of the whole text frame (the part outside the quotation marks), and set code_switched = yes. If the whole text is only the quote, label the quote.
- Very short strings (2-4 words, headlines): label what a native reader would say it is. If it is equally valid as English and Tagalog (e.g. "Ok na", "Bahay"), use your best guess and
low. - Close languages. Cebuano vs Hiligaynon vs Waray, Tagalog vs Bikol, Indonesian vs Malay, Spanish vs Chavacano: answer from your own knowledge, and use
lowif the text is too short or too standard to tell. Chavacano uses Spanish words with Philippine grammar (ta, ya, di, mi, na, e.g. "Ya mira yo con el mujer"); plain Spanish with Spanish grammar isspa. - Old/formal/religious spelling, accents (e.g. Cebuano with stress accents) - label the language normally.
- Mixed junk (OCR errors, broken words): label the language if you can read it, otherwise
unsure. - Do not search the internet or use a translator. Do not change the text column. Do not sort or delete rows.
What happens with your labels
They become a private evaluation set and training candidates for the OurLID language identifier. They are only used to measure and improve the model; the excerpts themselves are not published.
Gabay sa pag-label ng mga katutubong tagapagsalita
Makakakita ka ng humigit-kumulang 675 maiikling teksto (hindi hihigit sa 300 character bawat isa). Para sa bawat isa, sabihin kung anong wika ang pagkakasulat nito. Gusto namin ang tapat mong pagkaintindi bilang katutubong tagapagsalita. May mga 50 hilera na sadyang inulit para masukat ang pagkakapare-pareho - i-label ang bawat hilera na parang ngayon mo lang nakita.
Oras: mga 1-1.5 minuto bawat 10 hilera, kaya mga 1.5 oras lahat. Puwedeng huminto at ipagpatuloy mamaya; iwanang blangko ang hindi pa nagagawa.
Ano ang pupunan (mga column E-H; ang A-C ay ang aytem)
| column | ilalagay |
|---|---|
annotator_label |
isang code mula sa listahan sa itaas (may dropdown sa Excel) |
code_switched |
yes kung may ikalawang wika rin sa teksto (hal. Taglish), kung hindi no |
confidence |
high (sigurado), medium, o low (hula lang) |
note |
opsyonal; kailangan para sa oth_ph at oth (isulat ang pangalan ng wika) |
Mga espesyal na kaso
- Taglish. Piliin ang wikang nagbibigay ng balangkas ng pangungusap - ang mga salitang pambalarila (Tagalog: ang, ng, sa, na, ay, mga, ko, mo, niya, hindi, para sa...; English: the, is, to, of, and, for...). "Nag-meeting kami kanina about the project" ->
tgl, code_switched = yes. Kung kalahati-kalahati, pumili ayon sa pakiramdam mo at lagyan ngmedium. Gamitin angmixedlang kung magkahiwalay ang dalawang bahagi (hal. English na headline na sinundan ng Tagalog na pangungusap) at walang iisang balangkas. - Mga pangalan, numero, lugar, akronim (Juan dela Cruz, PNP, P20,000, Kalibo) ay hindi nagpapasya ng wika. Tingnan ang natitirang teksto. Kung wala nang natira,
not_language. - Mga sipi (quotes). Kung ang pangungusap na Tagalog ay may sipi sa English (o kabaligtaran), i-label ang wika ng buong balangkas (ang labas ng panipi), at
code_switched = yes. Kung sipi lang ang buong teksto, i-label ang sipi. - Napakaikling teksto (2-4 salita, headline): sagutin kung ano ang sasabihin ng isang katutubong bumabasa. Kung pantay na puwedeng English o Tagalog, magbigay ng pinakamahusay na hula at lagyan ng
low. - Magkakalapit na wika. Cebuano / Hiligaynon / Waray, Tagalog / Bikol, Indonesian / Malay, Spanish / Chavacano: sumagot ayon sa sarili mong kaalaman;
lowkung masyadong maikli o masyadong karaniwan. Ang Chavacano ay gumagamit ng salitang Espanyol na may balarilang Pilipino (ta, ya, di, mi, na, hal. "Ya mira yo con el mujer"); ang purong Espanyol na may balarilang Espanyol ayspa. - Lumang/pormal/panrelihiyong baybay, may accent (hal. Cebuano na may tuldik) - i-label ang wika gaya ng dati.
- Sirang teksto (OCR, putol na salita): i-label kung nababasa mo, kung hindi
unsure. - Huwag mag-search sa internet o gumamit ng translator. Huwag baguhin ang text column. Huwag mag-sort o magbura ng hilera.
Ang unsure ay katanggap-tanggap na sagot - mas mabuti ito kaysa manghula.
Ano ang mangyayari sa mga label mo
Gagamitin ang mga ito bilang pribadong evaluation set at training candidates ng OurLID na language identifier. Ginagamit lamang ang mga ito para sukatin at pahusayin ang model; hindi ilalathala ang mga sipi.