|
Download docs/NATIVE_REVIEW.md from sapinsapin/wikalid: direct link, hf CLI and curl.
- Browser
- Download file 1.19 kB
-
https://huggingface.co/sapinsapin/wikalid/resolve/main/docs/NATIVE_REVIEW.md
- Command line
-
hf download hf://sapinsapin/wikalid/docs/NATIVE_REVIEW.md
-
curl -L -o NATIVE_REVIEW.md https://huggingface.co/sapinsapin/wikalid/resolve/main/docs/NATIVE_REVIEW.md
1.19 kB
What needs native-speaker review before any release
- Gold set: re-label/adjudicate 299 docs / 593 spans (single non-native annotator now); priorities: hil (30), war (8), cbk (6),
oth(Aklanon, Surigaonon etc.),nolboundaries. - Tier-W training rows (web, GlotLID labels): audit 200 random rows per language, esp. hil, ceb, ilo, bcl, pam, pag; record label noise %.
- Seed marker lexicons (
src/ourlid/seeds.py) and the induced closed-class lexicons (data/work/induced_lexicon.json): confirm top function words per language, flag Spanish/English loans and topic words. - Frame vs majority convention for quoted English and the Taglish
cs_levelthresholds (0.03 / 0.25 English share): does the community agree with "light/heavy"? - Benchmark: sanity-check 300 random items per category for label correctness (pam/pag/cbk Wikipedia sentences contain English/Spanish), natural-sounding SMS spellings, and add real social-media Taglish.
- Unsupported varieties: decide which of akl, sgd, mdh, cuyonon, sur etc. should become classes, and who can supply licensed text.
- Orthography: pre-1960s/old spellings, Chavacano orthographies, diacritics.