| ukr-static-m3-64k |
| Copyright (c) 2026 Oleksandr Kholodniak |
|
|
| This model contains material derived from the following third-party works. |
|
|
| 1. BAAI/bge-m3 (https://huggingface.co/BAAI/bge-m3) |
| The token vectors were distilled from this model and then trained against its outputs. |
|
|
| MIT License |
|
|
| Copyright (c) 2022 staoxiao |
|
|
| Permission is hereby granted, free of charge, to any person obtaining a copy |
| of this software and associated documentation files (the "Software"), to deal |
| in the Software without restriction, including without limitation the rights |
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell |
| copies of the Software, and to permit persons to whom the Software is |
| furnished to do so, subject to the following conditions: |
|
|
| The above copyright notice and this permission notice shall be included in all |
| copies or substantial portions of the Software. |
|
|
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR |
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, |
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE |
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER |
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, |
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE |
| SOFTWARE. |
|
|
| 2. XLM-RoBERTa tokenizer (https://huggingface.co/FacebookAI/xlm-roberta-large) |
| BGE-M3 uses this SentencePiece tokenizer. The subword pieces and the normalisation table in |
| tokenizer.json come from it. |
|
|
| MIT License |
|
|
| Copyright (c) Facebook, Inc. and its affiliates. |
|
|
| Permission is hereby granted, free of charge, to any person obtaining a copy |
| of this software and associated documentation files (the "Software"), to deal |
| in the Software without restriction, including without limitation the rights |
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell |
| copies of the Software, and to permit persons to whom the Software is |
| furnished to do so, subject to the following conditions: |
|
|
| The above copyright notice and this permission notice shall be included in all |
| copies or substantial portions of the Software. |
|
|
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR |
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, |
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE |
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER |
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, |
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE |
| SOFTWARE. |
|
|
| 3. Training text (no passages are included in this release) |
|
|
| FineWeb-2 (https://huggingface.co/datasets/HuggingFaceFW/fineweb-2), subset ukr_Cyrl. |
| Contains information from FineWeb-2, which is made available under the ODC Attribution License |
| (https://opendatacommons.org/licenses/by/1-0/). Use of FineWeb-2 is also subject to the Common Crawl |
| Terms of Use (https://commoncrawl.org/terms-of-use). |
|
|
| Ukrainian Wikipedia (https://huggingface.co/datasets/wikimedia/wikipedia), dump 20231101.uk, licensed |
| under the Creative Commons Attribution-ShareAlike 4.0 International License |
| (https://creativecommons.org/licenses/by-sa/4.0/). |
|
|
| Word frequencies from both sources were used to choose the vocabulary, so tokenizer.json contains |
| single Ukrainian words and subword pieces. It contains no sentences or passages from either source. |
|
|
