|
Download docs/LABELING_GUIDE_NATIVE.md from sapinsapin/wikalid: direct link, hf CLI and curl.
- Browser
- Download file 7.37 kB
-
https://huggingface.co/sapinsapin/wikalid/resolve/main/docs/LABELING_GUIDE_NATIVE.md
- Command line
-
hf download hf://sapinsapin/wikalid/docs/LABELING_GUIDE_NATIVE.md
-
curl -L -o LABELING_GUIDE_NATIVE.md https://huggingface.co/sapinsapin/wikalid/resolve/main/docs/LABELING_GUIDE_NATIVE.md
7.37 kB
| # Native-speaker language labeling guide | |
| You will see about 675 short texts (at most 300 characters each). For each one, say **which language it is written in**. There are no right-or-wrong "tricks": we want your honest native judgement. About 50 rows are repeated on purpose (to measure consistency) - just label every row on its own, as if you had never seen it. | |
| Time: about 1-1.5 minutes per 10 rows, so roughly 1.5 hours in total. You can stop and continue later; leave unfinished rows empty. | |
| ## What to fill in (columns E-H of the sheet; columns A-C are the item) | |
| | column | what to put | | |
| |---|---| | |
| | `annotator_label` | one code from the list below (dropdown in the Excel file) | | |
| | `code_switched` | `yes` if a second language also appears in the text (e.g. Taglish), otherwise `no` | | |
| | `confidence` | `high` (sure), `medium`, or `low` (guessing) | | |
| | `note` | optional; **required** for `oth_ph` and `oth` (write the language name), helpful for anything odd | | |
| ## Codes | |
| | code | use it for | | |
| |---|---| | |
| | tgl | Tagalog / Filipino, including Taglish whose *sentence frame* is Tagalog | | |
| | eng | English, including Taglish whose frame is English | | |
| | ceb | Cebuano / Bisaya | | |
| | hil | Hiligaynon / Ilonggo | | |
| | war | Waray | | |
| | ilo | Ilocano | | |
| | bcl | Bikol (any variety: Naga, Legazpi, Rinconada, Albay, Sorsogon...) | | |
| | pam | Kapampangan | | |
| | pag | Pangasinan | | |
| | cbk | Chavacano (Zamboanga, Cavite, Ternate, Davao...) | | |
| | krj | Kinaray-a | | |
| | akl | Aklanon | | |
| | sgd | Surigaonon | | |
| | tsg | Tausug | | |
| | oth_ph | another Philippine language (write which in `note`) | | |
| | spa | Spanish | | |
| | ind | Indonesian | | |
| | msa | Malay | | |
| | oth | any other language (write which in `note`) | | |
| | mixed | two languages in clearly separate parts and no single frame language | | |
| | not_language | only names, numbers, codes, symbols, gibberish, or a list of proper nouns | | |
| | unsure | you really cannot tell (this is a fine answer - please use it instead of guessing) | | |
| ## Edge cases | |
| 1. **Taglish (code-switching).** Pick the language that supplies the sentence *frame* - the grammar words (Tagalog: *ang, ng, sa, na, ay, mga, ko, mo, niya, hindi, para sa...*; English: *the, is, to, of, and, for...*). "Nag-meeting kami kanina about the project" -> `tgl`, code_switched = yes. "The mayor said na hindi sila aalis" -> `eng` if the English part carries the frame, otherwise `tgl`; use your judgement and mark confidence `medium` if it is half-half. Use `mixed` only if two separate chunks (e.g. a headline in English followed by a Tagalog sentence) have no single frame. | |
| 2. **Names, numbers, places, acronyms** (Juan dela Cruz, PNP, P20,000, Kalibo) do not decide the language. Look at the rest of the text. If nothing else is left, use `not_language`. | |
| 3. **Quotations.** If a Tagalog sentence quotes English words, or an English sentence quotes a Tagalog phrase, label the language of the **whole text frame** (the part outside the quotation marks), and set code_switched = yes. If the whole text is only the quote, label the quote. | |
| 4. **Very short strings** (2-4 words, headlines): label what a native reader would say it is. If it is equally valid as English and Tagalog (e.g. "Ok na", "Bahay"), use your best guess and `low`. | |
| 5. **Close languages.** Cebuano vs Hiligaynon vs Waray, Tagalog vs Bikol, Indonesian vs Malay, Spanish vs Chavacano: answer from your own knowledge, and use `low` if the text is too short or too standard to tell. Chavacano uses Spanish words with Philippine grammar (*ta, ya, di, mi, na*, e.g. "Ya mira yo con el mujer"); plain Spanish with Spanish grammar is `spa`. | |
| 6. **Old/formal/religious spelling, accents** (e.g. Cebuano with stress accents) - label the language normally. | |
| 7. **Mixed junk** (OCR errors, broken words): label the language if you can read it, otherwise `unsure`. | |
| 8. Do not search the internet or use a translator. Do not change the text column. Do not sort or delete rows. | |
| ## What happens with your labels | |
| They become a private evaluation set and training candidates for the OurLID language identifier. They are only used to measure and improve the model; the excerpts themselves are not published. | |
| --- | |
| # Gabay sa pag-label ng mga katutubong tagapagsalita | |
| Makakakita ka ng humigit-kumulang 675 maiikling teksto (hindi hihigit sa 300 character bawat isa). Para sa bawat isa, sabihin kung **anong wika ang pagkakasulat nito**. Gusto namin ang tapat mong pagkaintindi bilang katutubong tagapagsalita. May mga 50 hilera na sadyang inulit para masukat ang pagkakapare-pareho - i-label ang bawat hilera na parang ngayon mo lang nakita. | |
| Oras: mga 1-1.5 minuto bawat 10 hilera, kaya mga 1.5 oras lahat. Puwedeng huminto at ipagpatuloy mamaya; iwanang blangko ang hindi pa nagagawa. | |
| ## Ano ang pupunan (mga column E-H; ang A-C ay ang aytem) | |
| | column | ilalagay | | |
| |---|---| | |
| | `annotator_label` | isang code mula sa listahan sa itaas (may dropdown sa Excel) | | |
| | `code_switched` | `yes` kung may ikalawang wika rin sa teksto (hal. Taglish), kung hindi `no` | | |
| | `confidence` | `high` (sigurado), `medium`, o `low` (hula lang) | | |
| | `note` | opsyonal; **kailangan** para sa `oth_ph` at `oth` (isulat ang pangalan ng wika) | | |
| ## Mga espesyal na kaso | |
| 1. **Taglish.** Piliin ang wikang nagbibigay ng *balangkas ng pangungusap* - ang mga salitang pambalarila (Tagalog: *ang, ng, sa, na, ay, mga, ko, mo, niya, hindi, para sa...*; English: *the, is, to, of, and, for...*). "Nag-meeting kami kanina about the project" -> `tgl`, code_switched = yes. Kung kalahati-kalahati, pumili ayon sa pakiramdam mo at lagyan ng `medium`. Gamitin ang `mixed` lang kung magkahiwalay ang dalawang bahagi (hal. English na headline na sinundan ng Tagalog na pangungusap) at walang iisang balangkas. | |
| 2. **Mga pangalan, numero, lugar, akronim** (Juan dela Cruz, PNP, P20,000, Kalibo) ay hindi nagpapasya ng wika. Tingnan ang natitirang teksto. Kung wala nang natira, `not_language`. | |
| 3. **Mga sipi (quotes).** Kung ang pangungusap na Tagalog ay may sipi sa English (o kabaligtaran), i-label ang wika ng **buong balangkas** (ang labas ng panipi), at `code_switched = yes`. Kung sipi lang ang buong teksto, i-label ang sipi. | |
| 4. **Napakaikling teksto** (2-4 salita, headline): sagutin kung ano ang sasabihin ng isang katutubong bumabasa. Kung pantay na puwedeng English o Tagalog, magbigay ng pinakamahusay na hula at lagyan ng `low`. | |
| 5. **Magkakalapit na wika.** Cebuano / Hiligaynon / Waray, Tagalog / Bikol, Indonesian / Malay, Spanish / Chavacano: sumagot ayon sa sarili mong kaalaman; `low` kung masyadong maikli o masyadong karaniwan. Ang Chavacano ay gumagamit ng salitang Espanyol na may balarilang Pilipino (*ta, ya, di, mi, na*, hal. "Ya mira yo con el mujer"); ang purong Espanyol na may balarilang Espanyol ay `spa`. | |
| 6. **Lumang/pormal/panrelihiyong baybay, may accent** (hal. Cebuano na may tuldik) - i-label ang wika gaya ng dati. | |
| 7. **Sirang teksto** (OCR, putol na salita): i-label kung nababasa mo, kung hindi `unsure`. | |
| 8. Huwag mag-search sa internet o gumamit ng translator. Huwag baguhin ang text column. Huwag mag-sort o magbura ng hilera. | |
| Ang `unsure` ay katanggap-tanggap na sagot - mas mabuti ito kaysa manghula. | |
| ## Ano ang mangyayari sa mga label mo | |
| Gagamitin ang mga ito bilang pribadong evaluation set at training candidates ng OurLID na language identifier. Ginagamit lamang ang mga ito para sukatin at pahusayin ang model; hindi ilalathala ang mga sipi. | |