Card: checkpoints/1217308k (step 62,895)

README.md

@@ -16,6 +16,12 @@ datasets:
 
 # Nirca Mini
 
+- **Small:** about 109 million parameters.
+- **Trained from scratch with quantization-aware training, in ternary:** from step 14,655 of 62,895, so for 98% of its 1,217,308,397 tokens; the ternary weights were learned in training, not quantized after it.
+
+Nirca was inspired by [mini-AGI](https://github.com/volotat/mini-AGI), and its architecture is
+deliberately derived from it: Nirca began as a port of mini-AGI to MLX.
+
 Nirca Mini is a small chat model trained from scratch, with ternary weights:
 its linear layers' weights are -1, 0 or +1, with a scale and a bias for each
 block of 128 weights. It reads and writes bytes, so it needs no tokenizer. It
@@ -46,8 +52,7 @@ This card is written by program from the checkpoint's own record, at step
 
 ## Architecture
 
-Nirca was inspired by [mini-AGI](https://github.com/volotat/mini-AGI), and its architecture is
-deliberately derived from it: Nirca began as a port of mini-AGI to MLX.
+What it keeps from [mini-AGI](https://github.com/volotat/mini-AGI), and what it changes:
 
 - **Kept:** bytes, no tokenizer; latent recurrence; one shared pool of experts
   every pass routes into; a halting head; first pretraining text mini-AGI's