Spaces:
Sleeping
Sleeping
Commit History
-
feat: typo correction, Kinyarwanda-first detection, memory/learning system
utils/normalizer.py (NEW) - Runs on raw text before any model or keyword check - Fixes 60+ common English typos (medecine→medicine, kiled→killed, docotr→doctor, pharmaice→pharmacy, scamed→scammed, accaunt→account...) - Fixes Kinyarwanda variant spellings (nab[iy]→nabi, ikibaz[ao]→ikibazo...) - Expands SMS/WhatsApp abbreviations (pls→please, thx→thanks, 50k→50000, mrk→murakoze, mrh→muraho, nd→ndi, knd→kandi...) - Collapses repeated chars (baaad→baad, noooo→no) - Fixes leet-speak digit substitutions (injec5ion→injection) - Fixes missing spaces after punctuation utils/lang_detect.py (UPDATED) - Kinyarwanda-first: assumes rw unless clear English evidence - Expanded to 120+ vocabulary words and richer morphological patterns - Single vocab/pattern hit with no strong English signal → rw - Code-switching (mixed rw+en) → treated as rw, NLLB handles it - Short messages like 'nabi' or 'bibi cyane' now correctly detected as rw models/memory.py (NEW) - SQLite-bas…bb47b9e