Download NOTICE from picur/picur-tokenizer: direct link, hf CLI and curl.
- Browser
- Download file 2.43 kB
-
https://huggingface.co/picur/picur-tokenizer/resolve/main/NOTICE
- Command line
-
hf download hf://picur/picur-tokenizer/NOTICE
-
curl -L -o NOTICE https://huggingface.co/picur/picur-tokenizer/resolve/main/NOTICE
2.43 kB
| picur-tokenizer | |
| Copyright 2026 Lajos Bencz | |
| Licensed under the Apache License, Version 2.0 (see LICENSE). | |
| This tokenizer is a derivative work. Its seed lexicon and its morpheme | |
| segmentation were derived from the two resources below, whose terms require | |
| attribution and are reproduced here. | |
| -------------------------------------------------------------------------- | |
| morphdb.hu -- Creative Commons Attribution 2.5 | |
| Copyright (c) Nemeth Laszlo, Godo Ferenc | |
| https://github.com/morphdb/morphdb.hu | |
| http://creativecommons.org/licenses/by/2.5/ | |
| Supplied the declared lexicon: 351,917 headwords, 37,042 compound seams, | |
| derivational splits, alternation classes and vowel harmony per entry. These | |
| decided which pieces the fit protects and which it evicts. | |
| -------------------------------------------------------------------------- | |
| hunmorph-foma -- BSD 2-Clause | |
| Copyright (c) 2016 r0ller. All rights reserved. | |
| https://github.com/r0ller/hunmorph-foma | |
| Redistribution and use in source and binary forms, with or without | |
| modification, are permitted provided that the following conditions are met: | |
| 1. Redistributions of source code must retain the above copyright notice, | |
| this list of conditions and the following disclaimer. | |
| 2. Redistributions in binary form must reproduce the above copyright notice, | |
| this list of conditions and the following disclaimer in the documentation | |
| and/or other materials provided with the distribution. | |
| THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" | |
| AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE | |
| IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE | |
| ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE | |
| LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR | |
| CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF | |
| SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS | |
| INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN | |
| CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) | |
| ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE | |
| POSSIBILITY OF SUCH DAMAGE. | |
| The finite-state transducers analysed corpus words into lemma plus inflection | |
| tags. Those analyses bootstrapped the morpheme segmentation the fit is scored | |
| against. | |
| -------------------------------------------------------------------------- | |