File size: 2,037 Bytes
4b750ba | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 | # NOTICE — Neuratal NTokenizer v0.3-polished model
This NOTICE covers the attribution and ShareAlike terms attached to the model
artifact `ntokenizer-0.3-256k.ntok` and to the `tokenizer.json` export included in
this repository.
1. The model is a machine-learned derivative of a 1.96 GB training corpus dominated
by CC BY-SA/GFDL text from Wikimedia projects (simplewiki, trwiki, hiwiki,
eswikisource, arwikisource, enwikibooks, jawikisource, kowikisource, zhwiktionary,
jawiktionary, kowiktionary, trwiktionary and others).
2. It is therefore distributed under **CC BY-SA 4.0**, not Apache-2.0.
ShareAlike applies to adaptations of the model.
3. Attribution pattern: `"<project> contributors, <license>, <URL>"`. Example:
"Simple English Wikipedia contributors, CC BY-SA 4.0, https://simple.wikipedia.org".
The complete per-source table (dumps, licenses, URLs, SHA-256) is in
`data/v0.2/LICENSES.md` of the source repository
(<https://github.com/Neuratal/neuratal-tokenizer>) and summarized in this repo's
`training_manifest.json` (fingerprints only; no corpus text included).
4. CC BY sources (arXiv per-article CC BY 4.0 — 187 papers; enwikinews CC BY
2.5/4.0; Tatoeba CC BY 2.0 FR) require attribution, no ShareAlike.
5. Public-domain sources (Project Gutenberg) and permissively licensed code
(CPython PSF 2.0, TypeScript Apache-2.0, Go BSD-3-Clause, Rust MIT OR Apache-2.0)
impose only notice retention, satisfied by this NOTICE.
6. No training corpus is distributed. No benchmark file is distributed. The
benchmark used in evaluation is a development/validation set.
7. Founder and creator: Rüzgar Erdem. Neuratal is an independent AI research and
software project; it is not claimed to be an incorporated company. The model
is an independent implementation; no third-party tokenizer vocabulary, merge
table, or implementation was used.
8. If you cannot comply with CC BY-SA 4.0, do not redistribute the model.
The NTokenizer source code remains MIT OR Apache-2.0.
|