---
license: mit
tags:
- arithmetic
- llama
- custom-architecture
- gguf
- safetensors
- micro-model
library_name: pytorch
pipeline_tag: text-generation
model_name: pizdecM2
---

# pizdecM2

**pizdecM2** is an experimental micro-model (~520 parameters) built to evaluate character-level arithmetic representations, custom Llama-compatible GGUF metadata mapping, and PyTorch Safetensors export pipelines.

Despite its tiny size, the architecture achieves **100% accuracy** on its specialized target domain.

---

## ⚠️ Important Note & Model Scope

> **CRITICAL:** This model is designed strictly for a single specific task and **WILL NOT** understand regular text, multi-digit operations, or non-addition prompts.

* **Supported Input Format:** Exactly 4-character prompts formatted as `x+y=` where $x$ and $y$ are single digits such that $x + y < 10$.
  * **Valid examples:** `2+3=`, `0+0=`, `1+8=`, `4+5=`
  * **Unsupported inputs:** `5+5=`, `10+2=`, `2*3=`, `hello`
* **Context Length:** Fixed at **4 tokens**.
* **Output:** Predicts a single character token representing the result ($0 \dots 9$).

---

## Model Metrics & Specs

* **Model Name:** `pizdecM2`
* **Total Parameters:** 520 (100% Trainable)
* **Accuracy:** **100.00%** (55/55 evaluated test samples)
* **Vocabulary Size:** 13 tokens (`0`-`9`, `+`, `=`, `\n`)
* **Embedding Size:** 10
* **FFN Hidden Dimension:** 14
* **Available Formats:**
  * `model.safetensors` (~2.52 KB)
  * `model.gguf` (~2.97 KB)
  * `tokenizer.json` (~0.13 KB)

---

## Architecture Breakdown

| Layer | Shape | Param Count |
| :--- | :--- | :--- |
| `embed_tokens.weight` | [13, 10] | 130 |
| `pos_emb` | [4, 10] | 40 |
| `ffn.weight` | [14, 10] | 140 |
| `ffn.bias` | [14] | 14 |
| `lm_head.weight` | [13, 14] | 182 |
| `lm_head.bias` | [13] | 13 |
| `pad` | [1] | 1 |
| **Total** | | **520** |

---

## How to Run Inference

```python
import json
import torch
import torch.nn as nn
from safetensors.torch import load_file

# Define architecture
class LlamaCompatible500P(nn.Module):
    def __init__(self):
        super().__init__()
        self.embed_tokens = nn.Embedding(13, 10)
        self.pos_emb = nn.Parameter(torch.randn(4, 10) * 0.1)
        self.ffn = nn.Linear(10, 14)
        self.act = nn.SiLU()
        self.lm_head = nn.Linear(14, 13)
        self.pad = nn.Parameter(torch.zeros(1))

    def forward(self, x):
        B, T = x.size()
        h = self.embed_tokens(x) + self.pos_emb[:T, :]
        h_flat = h.mean(dim=1)
        feat = self.act(self.ffn(h_flat))
        return self.lm_head(feat)

# Load Tokenizer & Model Weights
with open("tokenizer.json", "r", encoding="utf-8") as f:
    vocab = json.load(f)["vocab"]

char_to_id = {ch: i for i, ch in enumerate(vocab)}
id_to_char = {i: ch for i, ch in enumerate(vocab)}

model = LlamaCompatible500P()
model.load_state_dict(load_file("model.safetensors"))
model.eval()

# Run Prediction
prompt = "3+5="
tokens = torch.tensor([[char_to_id[c] for c in prompt]])

with torch.no_grad():
    logits = model(tokens)
    pred_id = torch.argmax(logits, dim=-1).item()
    print(f"Input: {prompt} | Output: {id_to_char[pred_id]}")
```
# License
MIT License