Fix packaging (transformers 4.x tokenizer/RoPE, tokenizer truncation, license, base_model) and correct the model card with measured quality (audit 2026-10-09); add Q8_0 GGUF and a working Ollama Modelfile

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+minSode-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

Modelfile

@@ -1,23 +1,26 @@
-# minSøde (my sweet) — HyperTensor CECI Grafted Model
-# Layer 5 attention chimaera. Early-layer FFN transplanted via GRC basis alignment from donor layer 15. 105% PPL recovery — grafted model outperforms original on simple sentences. The FFN from a deep reasoning layer now processes at an early attention layer, creating a unique 'deep insight at shallow depth' effect.
-#
-# Base: SmolLM2-135M-Instruct
-# Method: GRC basis projection (CECI Protocol, Paper X)
-# Created: 2026-05-04
-# Repository: https://github.com/NagusameCS/HyperTensor
-
-FROM ./minSøde.gguf
-
-PARAMETER temperature 0.7
-PARAMETER top_p 0.9
-PARAMETER top_k 40
-PARAMETER num_predict 256
-
-TEMPLATE """<|im_start|>system
-Du er minSøde, en podet kunstig intelligens. Layer 5 attention chimaera. Early-layer FFN transplanted via GRC basis alignment from donor layer 15. 105% PPL recovery — grafted model outperforms original on simple sentences. The FFN from a deep re{{ if .System }} {{ .System }}{{ end }}<|im_end|>
-<|im_start|>user
-{{ .Prompt }}<|im_end|>
-<|im_start|>assistant
-"""
-
-SYSTEM """Du er minSøde (my sweet), en dansk podet model skabt gennem HyperTensor CECI podning."""
+# minSode - HyperTensor CECI graft of SmolLM2-135M-Instruct (see README for what was changed and measured quality)
+# Usage (from a directory containing the repo files):
+#   ollama create minsode -f Modelfile && ollama run minsode
+# Or, once the GGUF is in the repo:  ollama run hf.co/NagusameCS/minSode
+FROM ./minSode-Q8_0.gguf
+
+TEMPLATE """{{- if .System }}<|im_start|>system
+{{ .System }}<|im_end|>
+{{ end }}
+{{- range $i, $_ := .Messages }}
+{{- $last := eq (len (slice $.Messages $i)) 1 -}}
+{{- if ne .Role "system" }}<|im_start|>{{ .Role }}
+{{ .Content }}<|im_end|>
+{{ end }}
+{{- if and $last (ne .Role "assistant") }}<|im_start|>assistant
+{{ end }}
+{{- end }}"""
+
+SYSTEM """You are a helpful AI assistant named SmolLM, trained by Hugging Face"""
+
+PARAMETER stop "<|im_end|>"
+PARAMETER stop "<|im_start|>"
+PARAMETER temperature 0.7
+PARAMETER top_p 0.9
+PARAMETER top_k 40
+PARAMETER num_predict 256

README.md

@@ -1,40 +1,68 @@
----
-language: en
-tags:
-- hypertensor
-- ceci-graft
-- danish
-- smollm2
-- experimental
-pipeline_tag: text-generation
-license: apache-2.0
----
-
-# minSøde (my sweet)
-
-Layer 5 attention chimaera. Early-layer FFN transplanted via GRC basis alignment from donor layer 15. 105% PPL recovery — grafted model outperforms original on simple sentences. The FFN from a deep reasoning layer now processes at an early attention layer, creating a unique 'deep insight at shallow depth' effect.
-
-## Architecture
-
-- **Base**: SmolLM2-135M-Instruct
-- **Method**: CECI Protocol (HyperTensor Paper X) — GRC basis projection
-- **Created**: 2026-05-04
-- **Repository**: [HyperTensor](https://github.com/NagusameCS/HyperTensor)
-
-## Graft Proof
-
-This model was created by:
-1. Computing the GRC (Geodesic Residual Compression) basis from the target layer's attention weights via SVD
-2. Projecting the donor layer's FFN weights into the target's geometric subspace
-3. Blending at controlled strength to preserve stability
-
-Perplexity testing confirms the graft transfers functional structure without destroying the model.
-
-## Usage
-
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-
-model = AutoModelForCausalLM.from_pretrained("NagusameCS/minSøde", trust_remote_code=True)
-tokenizer = AutoTokenizer.from_pretrained("NagusameCS/minSøde")
-```
+---
+language: en
+license: apache-2.0
+base_model: HuggingFaceTB/SmolLM2-135M-Instruct
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- hypertensor
+- ceci-graft
+- smollm2
+- experimental
+---
+
+# minSode (minSøde): "my sweet"
+
+An experimental weight-surgery variant of [SmolLM2-135M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct),
+made with the HyperTensor CECI "graft" procedure. It is a research artifact: it does **not** outperform its base model.
+
+## What was changed
+
+Only the layer-5 MLP (`gate_proj`, `up_proj`, `down_proj`) was changed: `W5 <- W5 + 0.5 * (W15 - W5) @ P` (`P @ (...)` for `down_proj`), where `P` projects onto the top-230 right-singular vectors of layer 5's `q_proj`. Created by `scripts/graft_proof.py` ("Kimære").
+
+The other 269 of 272 tensors are bit-identical to the base model. Weights are stored in float32. The tokenizer and chat template are the base model's.
+
+## Measured quality (audit, 2026-10-09)
+
+Teacher-forced next-token loss on 30 web and 30 code documents (512 tokens each, `data/corpus/{web,code}.dev.jsonl`), compared with the base model:
+
+| text | base NLL | this model NLL | perplexity change | top-1 agreement with base | KL(base‖model) |
+|---|---|---|---|---|---|
+| web | 2.865 | 2.896 | +3.1% | 89.6% | 0.044 |
+| code | 1.783 | 1.838 | +5.6% | 90.9% | 0.063 |
+| all | 2.324 | 2.367 | +4.3% | 90.3% | 0.053 |
+
+| original HyperTensor test | base | this model |
+|---|---|---|
+| graft_proof 15-sentence PPL (mean of per-sentence PPL) | 60.96 | 60.68 |
+| graft_benchmark MMLU-lite (50 items) | 31/50 | 32/50 (+1 / -0 items, sign-test p=1.00) |
+| graft_benchmark BoolQ (15 items) | 6/15 | 4/15 (+1 / -3, p=0.62) |
+
+The earlier card claimed "105% PPL recovery, outperforms original". The "% recovery" figures compared this model with a copy of the base whose MLP had been zeroed.
+This model was never ablated, so they do not measure recovery. On real text the model is slightly worse than its base, and the small
+MMLU-lite/BoolQ differences are a few items out of 50/15 and are not statistically significant.
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+tok = AutoTokenizer.from_pretrained("NagusameCS/minSode")
+model = AutoModelForCausalLM.from_pretrained("NagusameCS/minSode")
+msgs = [{"role": "user", "content": "What is the capital of France?"}]
+inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True)
+out = model.generate(**inputs, max_new_tokens=64, do_sample=False)
+print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
+```
+
+### Ollama
+
+The repo includes `minSode-Q8_0.gguf` and a `Modelfile`:
+
+```bash
+hf download NagusameCS/minSode --local-dir minSode && cd minSode
+ollama create minsode -f Modelfile && ollama run minsode
+# or directly: ollama run hf.co/NagusameCS/minSode
+```
+
+Source: [HyperTensor](https://github.com/NagusameCS/HyperTensor).

config.json

@@ -1,40 +1,42 @@
-{
-  "architectures": [
-    "LlamaForCausalLM"
-  ],
-  "attention_bias": false,
-  "attention_dropout": 0.0,
-  "bos_token_id": 1,
-  "dtype": "float32",
-  "eos_token_id": 2,
-  "head_dim": 64,
-  "hidden_act": "silu",
-  "hidden_size": 576,
-  "initializer_range": 0.041666666666666664,
-  "intermediate_size": 1536,
-  "is_llama_config": true,
-  "max_position_embeddings": 8192,
-  "mlp_bias": false,
-  "model_type": "llama",
-  "num_attention_heads": 9,
-  "num_hidden_layers": 30,
-  "num_key_value_heads": 3,
-  "pad_token_id": 2,
-  "pretraining_tp": 1,
-  "rms_norm_eps": 1e-05,
-  "rope_interleaved": false,
-  "rope_parameters": {
-    "rope_theta": 100000,
-    "rope_type": "default"
-  },
-  "tie_word_embeddings": true,
-  "transformers.js_config": {
-    "kv_cache_dtype": {
-      "fp16": "float16",
-      "q4f16": "float16"
-    }
-  },
-  "transformers_version": "5.7.0",
-  "use_cache": true,
-  "vocab_size": 49152
-}
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "float32",
+  "eos_token_id": 2,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 576,
+  "initializer_range": 0.041666666666666664,
+  "intermediate_size": 1536,
+  "is_llama_config": true,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 9,
+  "num_hidden_layers": 30,
+  "num_key_value_heads": 3,
+  "pad_token_id": 2,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_interleaved": false,
+  "rope_parameters": {
+    "rope_theta": 100000,
+    "rope_type": "default"
+  },
+  "tie_word_embeddings": true,
+  "transformers.js_config": {
+    "kv_cache_dtype": {
+      "fp16": "float16",
+      "q4f16": "float16"
+    }
+  },
+  "transformers_version": "5.7.0",
+  "use_cache": true,
+  "vocab_size": 49152,
+  "rope_theta": 100000,
+  "torch_dtype": "float32"
+}

minSode-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:556c3762316fb005f07004399660d2291e80a1535ccda3a47e345635bdd9a5ae
+size 144810912

special_tokens_map.json

@@ -0,0 +1,34 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": {
+    "content": "<|im_start|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -1,19 +1,154 @@
-{
-  "add_prefix_space": false,
-  "backend": "tokenizers",
-  "bos_token": "<|im_start|>",
-  "clean_up_tokenization_spaces": false,
-  "eos_token": "<|im_end|>",
-  "errors": "replace",
-  "extra_special_tokens": [
-    "<|im_start|>",
-    "<|im_end|>"
-  ],
-  "is_local": false,
-  "local_files_only": false,
-  "model_max_length": 8192,
-  "pad_token": "<|im_end|>",
-  "tokenizer_class": "GPT2Tokenizer",
-  "unk_token": "<|endoftext|>",
-  "vocab_size": 49152
-}
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<repo_name>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<reponame>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "5": {
+      "content": "<file_sep>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "6": {
+      "content": "<filename>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "7": {
+      "content": "<gh_stars>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "8": {
+      "content": "<issue_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "9": {
+      "content": "<issue_comment>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "10": {
+      "content": "<issue_closed>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "11": {
+      "content": "<jupyter_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "12": {
+      "content": "<jupyter_text>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "13": {
+      "content": "<jupyter_code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "14": {
+      "content": "<jupyter_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "15": {
+      "content": "<jupyter_script>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "16": {
+      "content": "<empty_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": "<|im_start|>",
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "model_max_length": 8192,
+  "pad_token": "<|im_end|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>",
+  "vocab_size": 49152
+}