| --- |
| license: mit |
| tags: |
| - arithmetic |
| - llama |
| - custom-architecture |
| - gguf |
| - safetensors |
| - micro-model |
| library_name: pytorch |
| pipeline_tag: text-generation |
| model_name: pizdecM2 |
| --- |
| |
| # pizdecM2 |
|
|
| **pizdecM2** is an experimental micro-model (~520 parameters) built to evaluate character-level arithmetic representations, custom Llama-compatible GGUF metadata mapping, and PyTorch Safetensors export pipelines. |
|
|
| Despite its tiny size, the architecture achieves **100% accuracy** on its specialized target domain. |
|
|
| --- |
|
|
| ## ⚠️ Important Note & Model Scope |
|
|
| > **CRITICAL:** This model is designed strictly for a single specific task and **WILL NOT** understand regular text, multi-digit operations, or non-addition prompts. |
|
|
| * **Supported Input Format:** Exactly 4-character prompts formatted as `x+y=` where $x$ and $y$ are single digits such that $x + y < 10$. |
| * **Valid examples:** `2+3=`, `0+0=`, `1+8=`, `4+5=` |
| * **Unsupported inputs:** `5+5=`, `10+2=`, `2*3=`, `hello` |
| * **Context Length:** Fixed at **4 tokens**. |
| * **Output:** Predicts a single character token representing the result ($0 \dots 9$). |
|
|
| --- |
|
|
| ## Model Metrics & Specs |
|
|
| * **Model Name:** `pizdecM2` |
| * **Total Parameters:** 520 (100% Trainable) |
| * **Accuracy:** **100.00%** (55/55 evaluated test samples) |
| * **Vocabulary Size:** 13 tokens (`0`-`9`, `+`, `=`, `\n`) |
| * **Embedding Size:** 10 |
| * **FFN Hidden Dimension:** 14 |
| * **Available Formats:** |
| * `model.safetensors` (~2.52 KB) |
| * `model.gguf` (~2.97 KB) |
| * `tokenizer.json` (~0.13 KB) |
|
|
| --- |
|
|
| ## Architecture Breakdown |
|
|
| | Layer | Shape | Param Count | |
| | :--- | :--- | :--- | |
| | `embed_tokens.weight` | [13, 10] | 130 | |
| | `pos_emb` | [4, 10] | 40 | |
| | `ffn.weight` | [14, 10] | 140 | |
| | `ffn.bias` | [14] | 14 | |
| | `lm_head.weight` | [13, 14] | 182 | |
| | `lm_head.bias` | [13] | 13 | |
| | `pad` | [1] | 1 | |
| | **Total** | | **520** | |
|
|
| --- |
|
|
| ## How to Run Inference |
|
|
| ```python |
| import json |
| import torch |
| import torch.nn as nn |
| from safetensors.torch import load_file |
| |
| # Define architecture |
| class LlamaCompatible500P(nn.Module): |
| def __init__(self): |
| super().__init__() |
| self.embed_tokens = nn.Embedding(13, 10) |
| self.pos_emb = nn.Parameter(torch.randn(4, 10) * 0.1) |
| self.ffn = nn.Linear(10, 14) |
| self.act = nn.SiLU() |
| self.lm_head = nn.Linear(14, 13) |
| self.pad = nn.Parameter(torch.zeros(1)) |
| |
| def forward(self, x): |
| B, T = x.size() |
| h = self.embed_tokens(x) + self.pos_emb[:T, :] |
| h_flat = h.mean(dim=1) |
| feat = self.act(self.ffn(h_flat)) |
| return self.lm_head(feat) |
| |
| # Load Tokenizer & Model Weights |
| with open("tokenizer.json", "r", encoding="utf-8") as f: |
| vocab = json.load(f)["vocab"] |
| |
| char_to_id = {ch: i for i, ch in enumerate(vocab)} |
| id_to_char = {i: ch for i, ch in enumerate(vocab)} |
| |
| model = LlamaCompatible500P() |
| model.load_state_dict(load_file("model.safetensors")) |
| model.eval() |
| |
| # Run Prediction |
| prompt = "3+5=" |
| tokens = torch.tensor([[char_to_id[c] for c in prompt]]) |
| |
| with torch.no_grad(): |
| logits = model(tokens) |
| pred_id = torch.argmax(logits, dim=-1).item() |
| print(f"Input: {prompt} | Output: {id_to_char[pred_id]}") |
| ``` |
| # License |
| MIT License |
|
|
