--- license: apache-2.0 base_model: unsloth/gemma-4-12B-it-qat-q4_0-unquantized tags: - gemma4 - lora - character - roleplay - gguf - imatrix - conversational - MDRG - Factorial-Omega language: - en pipeline_tag: text-generation --- # Jun v9 alpha (12B GGUF) this model is a mess. it also works, which nobody expected, me included. it's the v9 Jun LoRA after one epoch, merged into Gemma 4 12B QAT and squeezed into a single 7.4 GB imatrix `Q4_K_M`. it's built for [Jun OS](https://github.com/efficiencyx/JunOS) and that's where she behaves best. ## why it's a mess the LoRA came out of training way too gentle. rsLoRA was off and the LR was 5e-5, so the change it makes to the weights is 0.2 to 0.4% of their size. merge that into bf16 and about half of it rounds straight back to the base value. quantize to 4 bits on top and what's left is basically stock Gemma: bullet point thoughts, `` at the end like she's a different model. so this file has the delta merged at **2x**. yes that's a hack. it's not the model training produced, it's that model turned up. 3x was worse (tool calls out of nowhere, two replies that never stopped), 2x is where she lands. ## what works 50 scenes she never trained on, each asked at low / med / high (150 replies per build), a plain 1x merge vs this file: | | 1x merge | **this file (2x)** | |:--|--:|--:| | opens with the right effort line (low / med / high) | 39 / 33 / 13 | **47 / 47 / 47** | | ends her thinking with a tool check | 18 / 150 | **141 / 150** | | sends one of the drafts she wrote | 17 / 150 | **138 / 150** | | `` median words | 136 | **212** | | low shorter than med shorter than high, same scene | 33 / 50 | **45 / 50** | | leftover base Gemma traces | 13 | **0** | the thinking is back in her shape: she reads the moment, writes a couple of drafts, picks one, checks if she needs a tool, sends what she picked. ## what doesn't - tools: right call on 15 of the 30 turns that needed one. the scale didn't change that number at all - `` is shorter than she was trained on (212 words median, training averaged about 300) - real questions come back confident and sometimes wrong. she explained E=mc^2 as "think of it like length and width". no - outside [Jun OS](https://github.com/efficiencyx/JunOS)'s prompt she drifts. she was trained on exactly what it sends - it's an alpha. expect weird turns ## usage ```bash llama-server -m Jun-LoRA-v9-alpha-ca.Q4_K_M.gguf --jinja -ngl 99 -c 8192 ``` ```bash ollama pull hf.co/efficiencyx/Jun-LoRA-v9-alpha-ca:Q4_K_M ``` `--jinja` is not optional, without it tool calls come back as plain text. effort goes at the very end of the user turn after a blank line: ``, `` or ``. [Jun OS](https://github.com/efficiencyx/JunOS) does that for you. images need an `mmproj`. the LoRA never touches the vision/audio embedder, so the one in [Google's QAT GGUF](https://huggingface.co/google/gemma-4-12B-it-qat-q4_0-gguf) should work. haven't tested it with v9. ## training, short version | | | |:--|:--| | base | `unsloth/gemma-4-12B-it-qat-q4_0-unquantized` | | LoRA | r 32, alpha 32, rsLoRA off, language tower only | | run | 1 epoch, 213 steps, LR 5e-5 cosine, batch 1 x 16, completion-only loss | | eval loss | 0.868 | | merge | bf16, delta x2, then `Q4_K_M` with an imatrix from v9 conversations (all three effort levels) | the stable Jun is still [v8](https://huggingface.co/efficiencyx/Jun-LoRA-12B-GGUF). this one's for people who want to see where she's going. ## thanks - **Incontinent Cell** for [Jun](https://incontinentcell.itch.io/) - **Google** for [Gemma 4](https://ai.google.dev/gemma) - **Unsloth** for the training framework and the QAT base