File size: 2,760 Bytes
c7930e0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
---
language:
- tr
- en
tags:
- transformer
- pytorch
- llm
- decoder-only
- neural-network
- gqa
- rope
- swiglu
- apache-2.0
- neuratal
pretty_name: Neuratal NTransformer v1.4.0
license: apache-2.0
---

# NTransformer — Neuratal's open-source PyTorch Transformer

**NTransformer is not a pretrained model; this page documents the project, not model weights.**

NTransformer is an independently developed, PyTorch-based, decoder-only
Transformer research/engineering framework by Neuratal (https://neuratal.com).

- **Not pretrained.** No weights are distributed with the repository; outputs from an untrained preset are gibberish.
- **NTokenizer-native.** Vocab 262,144; token IDs consumed verbatim; SHA-256 fingerprint-verified checkpoints against the NTokenizer v0.3 artifact.
- **Apache-2.0.** NTokenizer is separate (CC-BY-SA-4.0); NTransformer is Apache-2.0.
- **Full NTokenizer integration:** the entire pipeline — token IDs consumed verbatim, SHA-256 fingerprint bound to checkpoints, no BOS/EOS injection, correct special-token map. **Designed to operate as the language model for NTokenizer v0.3-polished.**
- **Source + docs:** https://github.com/Neuratal/NTransformer
- **README:** https://github.com/Neuratal/NTransformer/blob/master/README.md
- **Latest release:** https://github.com/Neuratal/NTransformer/releases/tag/v1.4.0

## What it is

A research/engineering framework covering architecture, inference, training,
and checkpointing — everything you need to train, evaluate and iterate on a
decoder-only Transformer without vendoring a tokenizer, model hub, or hosted
inference service. NTransformer is **not** a pretrained model; no trained
weights are distributed. No quality claims are made. Use as a research tool,
not as a production ML stack.

- **Architecture block:** decoder-only pre-norm; GQA, RoPE, SwiGLU, RMSNorm,
  factorized tied embeddings — all LLaMA-style.
- **Long-context** paths for 16K/32K sequences with gradient checkpointing.
- **Tested:** 460 tests, real release, honest release notes.

## Why this is here

NTransformer needs a presence on Hugging Face so that the project card, tags and references live where serious ML practitioners browse. This page intentionally does **not** host a model; it points to the open-source repository.

## Links

- **GitHub repository:** https://github.com/Neuratal/NTransformer
- **NTokenizer (CC-BY-SA-4.0):** https://huggingface.co/Neuratal/ntokenizer
- **About Neuratal:** https://neuratal.com/about
- **License:** Apache-2.0

## Limitations (honest)

- No trained weights.
- Not production support; research-oriented.
- Multi-GPU/NCCL not verified on the development hardware.
- Benchmarks are development-machine numbers; do not treat as broad-hardware measurements.