Spaces:
Runtime error
Runtime error
Commit History
-
Clean up dataset - Remove source files and backups from deployment
This commit removes unnecessary source files from the repository to reduce deployment size and improve HuggingFace Spaces build time. ## Removed Files: - **Source Datasets**: data/washyourmouthoutwithsoap/ (build.json, _en.json) - 2 large JSON files no longer needed after merge - **Backup Files**: data/*.ldnoobw_only (20 files) - Original LDNOOBW-only wordlists preserved locally but excluded from deployment ## What Remains: - **58 merged language files** - All final profanity wordlists needed for runtime - Proper attribution headers maintained in each merged file ## Updated .gitignore: - Added data/washyourmouthoutwithsoap/ to ignore list - Added data/*.ldnoobw_only to ignore list ## Benefits: - Reduced repository size for faster deployments - Kept only production-ready merged wordlists - Source files still available locally for reference The profanity detector will continue to work with all 58 languages using only the merged files. 🤖 Generated with [Claude Code](https://cla…53ac297 -
Merge washyourmouthoutwithsoap dataset - Expand to 58 languages with 6,936 new words
This commit integrates the Wash Your Mouth Out With Soap (WYMOSP) profanity dataset with our existing LDNOOBW wordlists, significantly expanding coverage: ## Dataset Improvements: - Expanded from 25 to 58 supported languages (+133% increase) - Added 34 new languages from WYMOSP (MIT licensed) - Merged 20 overlapping languages with additional words - Total of 6,936 new profanity words added across all languages ## Language Coverage: - **New Languages (34)**: be, bg, ca, cy, el, et, eu, gd, gl, hr, hy, id, is, kn, la, lt, lv, mk, ml, mn, mr, ms, mt, my, ro, sk, sl, sq, sr, te, uk, uz, vi, zu - **Enhanced Languages (20)**: cs (+136), da (+131), de (+113), en (+0 already merged), es (+122), fa (+126), fi (+123), fr (+109), hi (+137), hu (+121), it (+112), ja (+100), ko (+113), nl (+123), pl (+124), pt (+121), ru (+133), sv (+128), th (+133), tr (+125) ## Technical Changes: - Created backup files (*.ldnoobw_only) for all merged language files - Added proper attribution headers …43fb695
-
added dataset
589ab6a