Instructions to use kyr0/fastText-pq-dno with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- fastText
How to use kyr0/fastText-pq-dno with fastText:
from huggingface_hub import hf_hub_download import fasttext model = fasttext.load_model(hf_hub_download("kyr0/fastText-pq-dno", "model.bin")) - Notebooks
- Google Colab
- Kaggle
Card and notice update, add build manifest
Browse files
.DS_Store
DELETED
|
Binary file (6.15 kB)
|
|
|
NOTICE.md
CHANGED
|
@@ -15,7 +15,7 @@ Edit data (not redistributed here):
|
|
| 15 |
- Wikinews articles (CirrusSearch content index of 2026-10-25): Wikinews contributors; CC BY 2.5 and CC BY 4.0
|
| 16 |
(en, es), CC BY 2.5 (de), CC BY 4.0 (ru, it), CC BY 2.5 and CC BY-SA 4.0 (fr), as the editions' page footers state.
|
| 17 |
|
| 18 |
-
Test data only, never fitted: NTREX-128 (CC BY-SA 4.0, Microsoft), FLORES-200 devtest, MASSIVE test,
|
| 19 |
-
test split.
|
| 20 |
|
| 21 |
License: CC BY-SA 3.0, like lid.176 (ShareAlike: adaptations keep this license).
|
|
|
|
| 15 |
- Wikinews articles (CirrusSearch content index of 2026-10-25): Wikinews contributors; CC BY 2.5 and CC BY 4.0
|
| 16 |
(en, es), CC BY 2.5 (de), CC BY 4.0 (ru, it), CC BY 2.5 and CC BY-SA 4.0 (fr), as the editions' page footers state.
|
| 17 |
|
| 18 |
+
Test data only, never fitted or edited on: NTREX-128 (CC BY-SA 4.0, Microsoft), FLORES-200 devtest, MASSIVE test,
|
| 19 |
+
the Tatoeba test split.
|
| 20 |
|
| 21 |
License: CC BY-SA 3.0, like lid.176 (ShareAlike: adaptations keep this license).
|
README.md
CHANGED
|
@@ -20,8 +20,7 @@ datasets:
|
|
| 20 |
|
| 21 |
# defuss-lid fastText DNO-PQ packs
|
| 22 |
|
| 23 |
-
> fastText lid.176 for a few languages, 82% to 89% smaller, with 20% to 21% fewer errors than lid.176 across the
|
| 24 |
-
> measured test slices.
|
| 25 |
|
| 26 |
Two fastText language-identification packs, `der_3` (English, German, Russian) and `deferi_6` (plus French, Italian
|
| 27 |
and Spanish), made from Meta's `lid.176.ftz` by the DNO-PQ (Direct Neural Optimization - Product Quantization)
|
|
|
|
| 20 |
|
| 21 |
# defuss-lid fastText DNO-PQ packs
|
| 22 |
|
| 23 |
+
> fastText lid.176 for a few languages, 82% to 89% smaller, with 20% to 21% fewer errors than lid.176 across the measured test slices.
|
|
|
|
| 24 |
|
| 25 |
Two fastText language-identification packs, `der_3` (English, German, Russian) and `deferi_6` (plus French, Italian
|
| 26 |
and Spanish), made from Meta's `lid.176.ftz` by the DNO-PQ (Direct Neural Optimization - Product Quantization)
|