NOVA8-53M — a language model that improves itself

Runs only with the NOVA-EVO program. NOVA is not a transformer and does not load in transformers, llama.cpp, Ollama or LM Studio. The program, the whole experiment and every measurement are on GitHub: https://github.com/roppikevo/AI-NOVA-EVO — how to run it is below, under "Running it".

NOVA-EVO is an independent research project by roppik: a small language model built from zero — its own core, tokenizer, training and evaluation — living inside a system that tries to make it better by itself, under rules it is not allowed to change. This file is the current champion of that system.

What makes it different

It learns on its own. A program called the director runs day and night on one consumer graphics card (RTX 4060, 8 GB). It picks what to try next, trains a challenger from the current champion and hands it to an independent judge. Only a challenger that is measurably better on texts it has never seen, confirmed on a second hidden set, without losing its programming skills, is released. Then it is checked once more on fresh text (probation); if it does worse there, the system steps back on its own.

It cannot cheat its own rules. The judge's thresholds and hidden test texts form a sealed constitution. The director may change its strategy, never the rules; if the rules or the hidden texts change, everything stops until the creator has looked. Only the creator changes and re-seals them.

It remembers and draws lessons. Every attempt, its cost and the verdict are kept in a ledger. The director does not repeat what failed on the champion in front of it, measures its own noise (how much two runs of the same experiment differ) and tests one setting at a time to learn what actually helps. When nothing untried is left, it runs small tournaments of new core designs instead of repeating itself.

You can tell it what to learn. Give it a topic — python -m nova.novafile learn <model>.nova "photosynthesis" — and it collects material (local teacher models, Wikipedia, its own invisible web browser), keeps part of it aside as an exam, trains, and lets the judge decide. A better core comes out as a new .nova file whose history lists what it learned. It also finds its own weak spots (its weakest language, the hardest level of its code exam) and queues topics for itself.

It writes at constant speed. NOVA's core is recurrent: it carries a fixed-size state (88 kB here) instead of a growing cache, so the next token costs the same after 100 or after 10 000 tokens, on an ordinary CPU.

How this champion came to be

Version	What happened	Judge's verdict
NOVA8-53M-v1	the same build as the 24 M champion at width 704, trained from scratch (~24 h, one RTX 4060)	accepted, +2.10 % (hidden set confirmed +2.14 %)
NOVA8-53M-v2	the director's own idea: continue training with less web text	accepted, +1.85 %
NOVA8-53M-v3	another improvement the director found and released by itself	accepted, current champion

The project publishes its failures as well: ideas of ours that the director turned down, 34 hours the loop once wasted repeating itself, a larger 88 M core that learned its training texts by heart instead of generalising, and a first attempt to learn a topic that the judge rejected because the model forgot a little elsewhere. All of it, with the numbers, is in the GitHub repository.

Results (hidden texts, loss per token, lower is better)

Slovak	Czech	Polish	English	Python	Rust
2.896	3.100	3.227	3.367	3.021	2.120

Code exam: 60 of 79 small Python tasks solved (level 1: 46/55, level 2: 14/20, level 3: 0/4).

parameters	64.5 M (all tensors, token table included)
core	generation 8, recurrent memory blocks alternating with slot memory, width 704 × 7 blocks
languages	Slovak, Czech, Polish, English; Python and Rust code
tokenizer	own, 16 384 tokens
state	88 kB, fixed — it does not grow with the text

Be fair to it: this is a small research model. It writes fluent-looking text in four languages and simple functions, but it is not a chat assistant and its facts are often wrong. What is interesting here is how it is built and how it improves, not what it knows today.

Files

file	size
NOVA8-53M-v3-int8.nova	66 MB	default: weights in 8 bits, loss +0.0001 against the original
NOVA8-53M-v3-f16.nova	130 MB	16-bit weights, identical answers
tokenizer.json, MODEL.json, SHA256SUMS		tokenizer, release record, checksums

.nova is NOVA's own single-file format: weights, tokenizer, configuration and the model's learning history in one file, read straight from the disk. It exists because NOVA's core is not an architecture that GGUF tools run.

Running it

Python 3.10+, no graphics card needed:

git clone --depth 1 https://github.com/roppikevo/AI-NOVA-EVO.git
cd AI-NOVA-EVO
python install.py                      # private .venv with the libraries (or ./install.sh, install.bat)
source .venv/bin/activate              # Windows: .venv\Scripts\activate

curl -L -o NOVA8-53M-v3-int8.nova https://huggingface.co/devilart1980/NOVA8-53M/resolve/main/NOVA8-53M-v3-int8.nova
python -m nova.novafile info NOVA8-53M-v3-int8.nova      # what is inside and what it has learned
python -m nova.novafile ask  NOVA8-53M-v3-int8.nova "Bratislava je" --lang sk --tokens 60

Languages for --lang: sk, cs, pl, en, py, rs. The repository's README explains the rest (section "Running a .nova file"), and how to run the whole self-improving system on your own graphics card.

Training data

FineWeb-2 (Slovak, Czech, Polish) and FineWeb-Edu (English), ODC-By 1.0, built from Common Crawl; Wikipedia (CC BY-SA 4.0); the Python standard library and Rust sources; texts written by open teacher models running locally. The model may reproduce patterns from these sources; attribution to them is kept here.

Licence

Free for research, experiments, study and other non-commercial use under the PolyForm Noncommercial License 1.0.0, like the NOVA-EVO code. Commercial use needs the written permission of the creator.

Follow the experiment

Everything — the code, the rules, every release with its numbers, the successes and the failures — is described in the GitHub repository: https://github.com/roppikevo/AI-NOVA-EVO. Questions and feedback: open an issue there.