|
Download README.md from javohirmat/uzmorph: direct link, hf CLI and curl.
- Browser
- Download file 4.44 kB
-
https://huggingface.co/javohirmat/uzmorph/resolve/main/README.md
- Command line
-
hf download hf://javohirmat/uzmorph/README.md
-
curl -L -o README.md https://huggingface.co/javohirmat/uzmorph/resolve/main/README.md
4.44 kB
| license: gpl-3.0 | |
| language: | |
| - uz | |
| tags: | |
| - morphology | |
| - apertium | |
| - morphological-analysis | |
| - uzbek | |
| library_name: uzmorph | |
| # uzmorph: Uzbek morphological analyzer and generator | |
| This is an improved version of the [Apertium](https://www.apertium.org/) Uzbek transducer (apertium-uzb, upstream commit `00522e3`), packaged with a Python layer: | |
| - a confident guesser for unknown words; | |
| - a proper-noun layer; | |
| - a corpus-trained reading ranker that replaces Constraint Grammar `-1`; | |
| - UD-style output (lemma, UPOS, features); | |
| - input in official Latin, Cyrillic or the 2026 Latin alphabet. | |
| **Code, documentation, evaluation protocol and issues:** https://github.com/javohirmat/uzmorph (release [v0.9.0](https://github.com/javohirmat/uzmorph/releases/tag/v0.9.0)). | |
| ## Files in this repository | |
| | path | contents | | |
| |---|---| | |
| | `dist/uzmorph-0.9.0-py3-none-any.whl` | the Python package (includes everything in `data/`) | | |
| | `dist/uzmorph-0.9.0.tar.gz` | sdist: the full source, including the modified `apertium-uzb/` (Corresponding Source of the transducers) | | |
| | `data/uzb.automorf.bin`, `uzb.autogen.bin`, `uzb_guesser.automorf.bin` | compiled lttoolbox transducers: analyser, generator, guesser | | |
| | `data/*.json.gz`, `data/name_sufprior.json` | ranker, guesser and capitalisation models, and the proper-noun suffix prior | | |
| | `data/names_list.tsv` | proper-noun stems verified against Wikidata (ids included) | | |
| | `data/SOURCE.md` | where the Corresponding Source is, and the written offer | | |
| **The models are counts only.** They hold word types, Apertium tags and suffix strings with frequencies or probabilities, derived from [tahrirchi/uz-books-v2](https://huggingface.co/datasets/tahrirchi/uz-books-v2) (dataset card: MIT). They contain no sentences or running text. Everything here is licensed GPL-3.0-only, the same as the package and upstream apertium-uzb. | |
| ## Usage | |
| ```bash | |
| sudo apt install lttoolbox | |
| pip install https://huggingface.co/javohirmat/uzmorph/resolve/main/dist/uzmorph-0.9.0-py3-none-any.whl | |
| ``` | |
| ```python | |
| import uzmorph | |
| for t in uzmorph.analyze("Kecha doʻstlarim bilan Samarqandga bordik."): | |
| if t.is_word: | |
| print(t.text, t.best.lemma, t.best.upos, t.best.feats) | |
| print(uzmorph.generate("kitob", "<n><pl><px1pl><abl>")) # ['kitoblarimizdan'] | |
| ``` | |
| ## Results | |
| The numbers are the same as in the GitHub README; every number there comes from a file in the repository. | |
| **Top-1 accuracy, held-out test split** (156 sentences, 1,731 scored tokens, four genres; an unanalysed token counts as wrong): | |
| | system | lemma+feats | lemma | UPOS* | | |
| |---|---|---|---| | |
| | upstream apertium-uzb 00522e3 + CG `-1` | 79.7 | 86.5 | 75.4 | | |
| | previous build (v8d + CG `-1` + guesser) | 83.9 | 90.4 | 77.1 | | |
| | **uzmorph 0.9** | **93.5** | **96.1** | 79.1 | | |
| | uzmorph 0.9, dictionary words only | 88.0 | 90.5 | 74.1 | | |
| | ZirhUzMorph 0.1.1, all analyses | 87.8 | 92.2 | 57.3 | | |
| | ZirhUzMorph 0.1.1, known stems only | 83.5 | 87.9 | 56.9 | | |
| \*UPOS is not comparable across tools: ZirhUzMorph only distinguishes N and V. | |
| On the dev split (1,062 tokens), uzmorph scores 94.1, Zirh 87.5 and upstream + CG 78.1. | |
| **Coverage** (token coverage of the uz-books-v2 word-frequency list, anchored to the upstream 87.51): | |
| - dictionary: 90.69; | |
| - with the confident guesser: 94.80. | |
| **Non-words** (299 lower-case non-words; lower is better): | |
| - 1% are accepted by the dictionary; | |
| - 17% with the confident guesser; | |
| - ZirhUzMorph accepts 15% (known stems). | |
| **Generation:** 0 of 462 checked verb forms give a spurious -ish- variant (previous build: 275). | |
| ## Limitations | |
| - **The evaluation gold is mostly model-built:** | |
| - 1,525 labels were auto-accepted where the tools agreed (150-token spot-check); | |
| - about 1,120 were AI-adjudicated and never reviewed by a person; | |
| - 250 rows were checked by a native speaker. | |
| - **Conventions:** Finite 3rd-person verbs default to Number=Sing. Without that ranker rule, dev drops to 87.5 (equal to Zirh). | |
| - **UPOS is weak (79%).** | |
| - **Guesser readings are often wrong:** 57–63% are fully right. | |
| - **The proper-noun layer** accepts any capitalised unknown word in mid-sentence. | |
| - **The evaluation sentences are private** (copyrighted books, news, Telegram) and are not in this repository or on GitHub. Researchers can request them from the maintainer. | |
| ## Citation and contact | |
| Please cite the GitHub repository (see its `CITATION.cff`) and Apertium. | |
| Contact: Javohir Matniyazov, javohirmatniyazovv@gmail.com. | |