---
language: en
license: apache-2.0
base_model: HuggingFaceTB/SmolLM2-135M-Instruct
library_name: transformers
pipeline_tag: text-generation
tags:
- hypertensor
- ceci-graft
- smollm2
- experimental
---

# minSode (minSøde): "my sweet"

An experimental weight-surgery variant of [SmolLM2-135M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct),
made with the HyperTensor CECI "graft" procedure. It is a research artifact: it does **not** outperform its base model.

## What was changed

Only the layer-5 MLP (`gate_proj`, `up_proj`, `down_proj`) was changed: `W5 <- W5 + 0.5 * (W15 - W5) @ P` (`P @ (...)` for `down_proj`), where `P` projects onto the top-230 right-singular vectors of layer 5's `q_proj`. Created by `scripts/graft_proof.py` ("Kimære").

The other 269 of 272 tensors are bit-identical to the base model. Weights are stored in float32. The tokenizer and chat template are the base model's.

## Measured quality (audit, 2026-10-09)

Teacher-forced next-token loss on 30 web and 30 code documents (512 tokens each, `data/corpus/{web,code}.dev.jsonl`), compared with the base model:

| text | base NLL | this model NLL | perplexity change | top-1 agreement with base | KL(base‖model) |
|---|---|---|---|---|---|
| web | 2.865 | 2.896 | +3.1% | 89.6% | 0.044 |
| code | 1.783 | 1.838 | +5.6% | 90.9% | 0.063 |
| all | 2.324 | 2.367 | +4.3% | 90.3% | 0.053 |

| original HyperTensor test | base | this model |
|---|---|---|
| graft_proof 15-sentence PPL (mean of per-sentence PPL) | 60.96 | 60.68 |
| graft_benchmark MMLU-lite (50 items) | 31/50 | 32/50 (+1 / -0 items, sign-test p=1.00) |
| graft_benchmark BoolQ (15 items) | 6/15 | 4/15 (+1 / -3, p=0.62) |

The earlier card claimed "105% PPL recovery, outperforms original". The "% recovery" figures compared this model with a copy of the base whose MLP had been zeroed.
This model was never ablated, so they do not measure recovery. On real text the model is slightly worse than its base, and the small
MMLU-lite/BoolQ differences are a few items out of 50/15 and are not statistically significant.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("NagusameCS/minSode")
model = AutoModelForCausalLM.from_pretrained("NagusameCS/minSode")
msgs = [{"role": "user", "content": "What is the capital of France?"}]
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True)
out = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

### Ollama

The repo includes `minSode-Q8_0.gguf` and a `Modelfile`:

```bash
hf download NagusameCS/minSode --local-dir minSode && cd minSode
ollama create minsode -f Modelfile && ollama run minsode
# or directly: ollama run hf.co/NagusameCS/minSode
```

Source: [HyperTensor](https://github.com/NagusameCS/HyperTensor).