Text Classification
fastText
defuss-lid
language-identification
edge
kyr0 commited on
Commit
0e4d9d9
·
verified ·
1 Parent(s): 52a01e8

Card and notice update, add build manifest

Browse files
Files changed (3) hide show
  1. .DS_Store +0 -0
  2. NOTICE.md +2 -2
  3. README.md +1 -2
.DS_Store DELETED
Binary file (6.15 kB)
 
NOTICE.md CHANGED
@@ -15,7 +15,7 @@ Edit data (not redistributed here):
15
  - Wikinews articles (CirrusSearch content index of 2026-10-25): Wikinews contributors; CC BY 2.5 and CC BY 4.0
16
  (en, es), CC BY 2.5 (de), CC BY 4.0 (ru, it), CC BY 2.5 and CC BY-SA 4.0 (fr), as the editions' page footers state.
17
 
18
- Test data only, never fitted: NTREX-128 (CC BY-SA 4.0, Microsoft), FLORES-200 devtest, MASSIVE test, the Tatoeba
19
- test split.
20
 
21
  License: CC BY-SA 3.0, like lid.176 (ShareAlike: adaptations keep this license).
 
15
  - Wikinews articles (CirrusSearch content index of 2026-10-25): Wikinews contributors; CC BY 2.5 and CC BY 4.0
16
  (en, es), CC BY 2.5 (de), CC BY 4.0 (ru, it), CC BY 2.5 and CC BY-SA 4.0 (fr), as the editions' page footers state.
17
 
18
+ Test data only, never fitted or edited on: NTREX-128 (CC BY-SA 4.0, Microsoft), FLORES-200 devtest, MASSIVE test,
19
+ the Tatoeba test split.
20
 
21
  License: CC BY-SA 3.0, like lid.176 (ShareAlike: adaptations keep this license).
README.md CHANGED
@@ -20,8 +20,7 @@ datasets:
20
 
21
  # defuss-lid fastText DNO-PQ packs
22
 
23
- > fastText lid.176 for a few languages, 82% to 89% smaller, with 20% to 21% fewer errors than lid.176 across the
24
- > measured test slices.
25
 
26
  Two fastText language-identification packs, `der_3` (English, German, Russian) and `deferi_6` (plus French, Italian
27
  and Spanish), made from Meta's `lid.176.ftz` by the DNO-PQ (Direct Neural Optimization - Product Quantization)
 
20
 
21
  # defuss-lid fastText DNO-PQ packs
22
 
23
+ > fastText lid.176 for a few languages, 82% to 89% smaller, with 20% to 21% fewer errors than lid.176 across the measured test slices.
 
24
 
25
  Two fastText language-identification packs, `der_3` (English, German, Russian) and `deferi_6` (plus French, Italian
26
  and Spanish), made from Meta's `lid.176.ftz` by the DNO-PQ (Direct Neural Optimization - Product Quantization)