|
Download docs/NATIVE_REVIEW.md from sapinsapin/wikalid: direct link, hf CLI and curl.
- Browser
- Download file 1.19 kB
-
https://huggingface.co/sapinsapin/wikalid/resolve/main/docs/NATIVE_REVIEW.md
- Command line
-
hf download hf://sapinsapin/wikalid/docs/NATIVE_REVIEW.md
-
curl -L -o NATIVE_REVIEW.md https://huggingface.co/sapinsapin/wikalid/resolve/main/docs/NATIVE_REVIEW.md
1.19 kB
| # What needs native-speaker review before any release | |
| 1. **Gold set**: re-label/adjudicate 299 docs / 593 spans (single non-native annotator now); priorities: hil (30), war (8), cbk (6), `oth` (Aklanon, Surigaonon etc.), `nol` boundaries. | |
| 2. **Tier-W training rows** (web, GlotLID labels): audit 200 random rows per language, esp. hil, ceb, ilo, bcl, pam, pag; record label noise %. | |
| 3. **Seed marker lexicons** (`src/ourlid/seeds.py`) and the **induced closed-class lexicons** (`data/work/induced_lexicon.json`): confirm top function words per language, flag Spanish/English loans and topic words. | |
| 4. **Frame vs majority convention** for quoted English and the Taglish `cs_level` thresholds (0.03 / 0.25 English share): does the community agree with "light/heavy"? | |
| 5. **Benchmark**: sanity-check 300 random items per category for label correctness (pam/pag/cbk Wikipedia sentences contain English/Spanish), natural-sounding SMS spellings, and add real social-media Taglish. | |
| 6. **Unsupported varieties**: decide which of akl, sgd, mdh, cuyonon, sur etc. should become classes, and who can supply licensed text. | |
| 7. Orthography: pre-1960s/old spellings, Chavacano orthographies, diacritics. | |