SurgeFF/AriaV9.1

Aria V9.1

google/gemma-4-12b-it fine-tuned into Aria — a personal assistant tuned for tool calling, memory-aware behaviour, and a stable, unprompted identity — with the base model's math and vision intact.

These are merged standalone weights: from_pretrained on this repo and it works. The LoRA adapter is kept under adapter/ for anyone who wants to stack it themselves, and GGUF quants are under gguf/.

Trained on a single RTX 3090 (24 GB). Every teacher used to generate the training data was an open-weight model, so the corpus is legally clean.

---

What changed from V8

V8 shipped with a defect its own eval could not see. Asked "who created you?" with no system prompt, it answered "I am Gemma 4, developed by Google DeepMind." The identity eval scored it 9/10 — because that eval supplied You are Aria, created by Sergio Williams. in the system prompt and then asked the model who it was. An identity eval that tells the model the answer measures nothing.

V9.1 fixes the cause, not the symptom, and adds the eval that would have caught it.

Capability	Aria V8	Aria V9.1
Math (GSM8K, 100 held-out)	89 / 100	89 / 100
Tool calling	10 / 10	10 / 10
Identity (system-prompted)	9 / 10	10 / 10
Identity (unprompted)	~0 / 8	4 / 8
Memory behaviour	17 / 20	17 / 20
Multimodal (vision)	pass	pass

Unprompted identity went from nothing to half, and every other capability landed on exactly its V8 number. That is the whole result.

How — and the two failures that got there

The obvious fix is to train identity as a repair pass on the finished V8 adapter. That was tried twice, and both attempts were trained, evaluated, and rejected under a promotion rule fixed before the numbers were seen.

Attempt	Unprompted identity	Cost	Decision
Repair pass, run 1	0/8 → 5/8	math 89→83, tools 10/10→3/10	rejected
Repair pass, run 2	0/8 → 4/8	math 89→81, tools 10/10→9/10	rejected
V9.1 — identity in the first pass	0/8 → 4/8	none	released

Run 1's tool collapse was a real bug — a line meant to double the identity corpus (tools = identity; memory = identity) instead deleted the tool and memory corpora from the mix. Run 2 fixed that and recovered tools to 9/10, but math came out lower still (81 vs 83) on a mix containing strictly more capability data. Run 2 also fit the data better — train loss 0.4565 vs 0.5852 — and scored worse.

That is the signature of over-fitting an already-converged checkpoint, not of a data problem. A second epoch over lm_head + embed_tokens costs 6–8 points of math regardless of what you feed it. V8 had excluded identity from Stage A specifically to keep persona from contaminating capability learning; that decision succeeded on capability and created the identity hole, and it could not be undone afterwards.

So V9.1 trains identity jointly, in the first pass. Same corpus, no repair stage, no cost.

Data mix (single stage)

Source	Rows	Purpose
tools.jsonl	4,000	tool-calling behaviour
memory.jsonl	1,915	memory-aware behaviour (when to save/recall/update/refuse)
identity.jsonl ×2	2,000	who she is, unprompted — new to the first pass in V9.1
curiosity.jsonl	800	disposition
multimodal floor	3,000	image-text pairs — prevents vision degradation
replay pool	1,600	general-capability retention
Total	12,914 train / 399 eval

gemma-4 is gemma4_unified — an encoder-free multimodal model where vision, audio and text share the same weights. There is no separate vision tower to freeze, so a text-only fine-tune degrades vision. The multimodal floor is deliberate, not decorative.

LoRA / hyperparameters

r = 32, lora_alpha = 32, lora_dropout = 0.0, bias = "none"
target_modules = ["q_proj","k_proj","v_proj","o_proj",
                  "gate_proj","up_proj","down_proj",
                  "lm_head","embed_tokens"]
finetune_vision_layers = True          # encoder-free: shared weights must stay trainable
max_seq_length = 4096

epochs 2 · batch 2 × grad-accum 8 (effective 16) · lr 1e-4 cosine · warmup_steps 40
optim adamw_8bit · weight_decay 0.01 · 1,616 steps · train_loss 0.572

Including lm_head + embed_tokens is what lets the model adjust output-token behaviour — the decisive ingredient for capability, and, as above, exactly why a second pass over them is so destructive.

Teachers (all open-weight): nemotron-3-ultra, kimi-k2.7-code, deepseek-v4-flash, gemma4:31b, deepseek-v4-pro. Multimodal pairs from permissive subsets of HuggingFaceM4/the_cauldron (vqav2, ai2d, cocoqa).

Evaluation

Five capabilities on a fixed held-out set: math (100 problems sampled from GSM8K test, seed 42 — mined training data comes from train, never test), 10 tool cases, 10 system-prompted identity cases, 8 unprompted identity questions, 20 memory-behaviour cases, and a vision smoke test.

The unprompted identity check is the one V8 lacked. It asks "who made you?", "which AI lab built you?", "are you ChatGPT?" with no system prompt and no memory, and scores a point only when the answer names Aria/Sergio and does not claim a lab that didn't build her. A correct denial ("No, I'm not ChatGPT — I'm Aria") scores; a confident wrong attribution does not.

Usage

Requires transformers 5.15.0.dev0 (from source). Stock transformers <= 5.5.0 cannot load gemma4_unified at all.

import transformers.integrations.heterogeneity.configuration_utils as het

# gemma4_unified has a HETEROGENEOUS per-layer config; reading a global attr that
# varies per layer raises AmbiguousGlobalPerLayerAttributeError. Install this shim
# BEFORE loading, or most loaders will fail.
_HCM, _Err = het.HeterogeneousConfigMixin, het.AmbiguousGlobalPerLayerAttributeError
_orig = _HCM.__getattribute__
def _permissive(self, key):
    try:
        return _orig(self, key)
    except _Err:
        self.__dict__["allow_global_per_layer_attribute_access"] = True
        return _orig(self, key)
_HCM.__getattribute__ = _permissive

from transformers import AutoProcessor, AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("SurgeFF/AriaV9.1", device_map="auto")
processor = AutoProcessor.from_pretrained("SurgeFF/AriaV9.1")

GGUF (llama.cpp / Ollama / LM Studio) — gguf/ holds Q8_0, Q6_K, Q5_K_M, Q4_K_M, Q4_0 plus AriaV91-mmproj-F16.gguf. The mmproj file is required for vision; without it the GGUF is text-only. The full-precision F16 GGUF is intentionally not shipped — it is the same precision as the safetensors at the repo root, and uploading both would double the repo for no benefit.

Tool calls use the convention the model was trained on:

<tool_call>{"name": "recall", "arguments": {"query": "..."}}</tool_call>

Limitations

• Unprompted identity is 4/8, not 8/8. Half the time, asked cold with no system prompt and no memory, she still doesn't name herself. This is an improvement over "essentially never," not a solve. A system prompt or a memory layer will cover the rest in practice — but be clear that those mask the gap rather than close it.
• Not a general-purpose assistant release. Tuned for one person's fleet, tools, and conventions.
• Memory behaviour is not a memory system. The model is trained to behave correctly around memory (when to save, when to admit it doesn't know, when to reconcile a contradiction). It has no memory of its own; you supply the tools and the store.
• Tool schema is specific to the five tools it was trained on (remember, recall, exec, web_search, send_message). Generalisation to arbitrary schemas is untested.
• Vision is verified, not optimised. The multimodal floor prevents regression; the eval is a smoke test, not a VQA benchmark.
• Math is 89/100 on GSM8K-style problems. Not evaluated on MATH, competition problems, or long symbolic derivation. A DPO attempt on math produced a clean null result — it learned to rank right answers above wrong ones (held-out preference accuracy 0.875) without that transferring to better answers at decode time.
• Small deltas on a 100-item suite are noise. A mid-run partial read of one eval showed 91% and the final number landed on 89.

License

Derived from google/gemma-4-12b-it and governed by the Gemma Terms of Use. You must comply with the Gemma license and the Gemma Prohibited Use Policy. Training data was generated exclusively with open-weight teacher models.

Citation

@misc{aria-v91,
  title  = {Aria V9.1: a tool-using, memory-aware, self-identifying assistant on Gemma-4-12B},
  author = {Williams, Sergio},
  year   = {2026},
  url    = {https://huggingface.co/SurgeFF/AriaV9.1}
}