Add model card (README pointing to source and purpose)
Browse files
README.md
ADDED
|
@@ -0,0 +1,65 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- tr
|
| 4 |
+
- en
|
| 5 |
+
tags:
|
| 6 |
+
- transformer
|
| 7 |
+
- pytorch
|
| 8 |
+
- llm
|
| 9 |
+
- decoder-only
|
| 10 |
+
- neural-network
|
| 11 |
+
- gqa
|
| 12 |
+
- rope
|
| 13 |
+
- swiglu
|
| 14 |
+
- apache-2.0
|
| 15 |
+
- neuratal
|
| 16 |
+
pretty_name: Neuratal NTransformer v1.4.0
|
| 17 |
+
license: apache-2.0
|
| 18 |
+
---
|
| 19 |
+
|
| 20 |
+
# NTransformer — Neuratal's open-source PyTorch Transformer
|
| 21 |
+
|
| 22 |
+
**NTransformer is not a pretrained model; this page documents the project, not model weights.**
|
| 23 |
+
|
| 24 |
+
NTransformer is an independently developed, PyTorch-based, decoder-only
|
| 25 |
+
Transformer research/engineering framework by Neuratal (https://neuratal.com).
|
| 26 |
+
|
| 27 |
+
- **Not pretrained.** No weights are distributed with the repository; outputs from an untrained preset are gibberish.
|
| 28 |
+
- **NTokenizer-native.** Vocab 262,144; token IDs consumed verbatim; SHA-256 fingerprint-verified checkpoints against the NTokenizer v0.3 artifact.
|
| 29 |
+
- **Apache-2.0.** NTokenizer is separate (CC-BY-SA-4.0); NTransformer is Apache-2.0.
|
| 30 |
+
- **Full NTokenizer integration:** the entire pipeline — token IDs consumed verbatim, SHA-256 fingerprint bound to checkpoints, no BOS/EOS injection, correct special-token map. **Designed to operate as the language model for NTokenizer v0.3-polished.**
|
| 31 |
+
- **Source + docs:** https://github.com/Neuratal/NTransformer
|
| 32 |
+
- **README:** https://github.com/Neuratal/NTransformer/blob/master/README.md
|
| 33 |
+
- **Latest release:** https://github.com/Neuratal/NTransformer/releases/tag/v1.4.0
|
| 34 |
+
|
| 35 |
+
## What it is
|
| 36 |
+
|
| 37 |
+
A research/engineering framework covering architecture, inference, training,
|
| 38 |
+
and checkpointing — everything you need to train, evaluate and iterate on a
|
| 39 |
+
decoder-only Transformer without vendoring a tokenizer, model hub, or hosted
|
| 40 |
+
inference service. NTransformer is **not** a pretrained model; no trained
|
| 41 |
+
weights are distributed. No quality claims are made. Use as a research tool,
|
| 42 |
+
not as a production ML stack.
|
| 43 |
+
|
| 44 |
+
- **Architecture block:** decoder-only pre-norm; GQA, RoPE, SwiGLU, RMSNorm,
|
| 45 |
+
factorized tied embeddings — all LLaMA-style.
|
| 46 |
+
- **Long-context** paths for 16K/32K sequences with gradient checkpointing.
|
| 47 |
+
- **Tested:** 460 tests, real release, honest release notes.
|
| 48 |
+
|
| 49 |
+
## Why this is here
|
| 50 |
+
|
| 51 |
+
NTransformer needs a presence on Hugging Face so that the project card, tags and references live where serious ML practitioners browse. This page intentionally does **not** host a model; it points to the open-source repository.
|
| 52 |
+
|
| 53 |
+
## Links
|
| 54 |
+
|
| 55 |
+
- **GitHub repository:** https://github.com/Neuratal/NTransformer
|
| 56 |
+
- **NTokenizer (CC-BY-SA-4.0):** https://huggingface.co/Neuratal/ntokenizer
|
| 57 |
+
- **About Neuratal:** https://neuratal.com/about
|
| 58 |
+
- **License:** Apache-2.0
|
| 59 |
+
|
| 60 |
+
## Limitations (honest)
|
| 61 |
+
|
| 62 |
+
- No trained weights.
|
| 63 |
+
- Not production support; research-oriented.
|
| 64 |
+
- Multi-GPU/NCCL not verified on the development hardware.
|
| 65 |
+
- Benchmarks are development-machine numbers; do not treat as broad-hardware measurements.
|