External Viet74K dictionary (63K multi-syllable entries) improves word F1
from 97.36% to 97.62% (+0.26%), narrowing gap to SOTA from 0.70% to 0.44%.
Includes configurable dictionary source, error analysis, and annotation
consistency analysis of 4+ syllable compounds.
Add VLSP 2013 word segmentation, feature ablation study, and Hydra config
- Add VLSP 2013 WTK dataset (75K train, 2K test sentences)
- Train word segmentation on VLSP 2013: 97.17% word F1
- Run systematic feature ablation study (13 experiments):
- Bigrams most critical (-2.86% when removed)
- Left context second most important (-1.21%)
- Type features provide zero benefit (+0.01% when removed)
- Minimal 12-template model achieves 97.01% (99.8% of full)
- Refactor training script from Click to Hydra configuration
- Add feature group toggles for ablation via config overrides
- Move scripts/ to src/, add Hydra conf/ structure
- Add 16 research paper extracts to references/
- Add research/ literature survey (41 papers, SOTA comparison)
- Update technical report with ablation results and VLSP 2013 eval
Add trained models (POS tagger and word segmentation)
