nohurry

/

sillytavern

Like 49

I'm terribly sorry that I only saw this a month later, and thank you so much for your reply!

My only (small) gripe is that it hides the NPC card / location with xml, but then (rarely) acts as if I am aware of "John's" pseudo character card.

This is actually a really good point! I never told Gemma that whatever is inside a XML comment is hidden from the user. I'll mess around with it for a bit!

[..] from context, but not always, creating some hilarious moments.

Glad it did!

Mistral... suffers. Even finetunes often praised for instruct adherence e.g. Cydonia forget random parts of Voyage at 12-16k ctx

Yeah... it sadly seems to be the case for any Mistral Small 3.2 / Mistral Magistral Small model I've tried. It remains tight up to 8K context in BF16.

Voyage 3.0 is too much for a mxfp4_moe quantization of gemma 4 26b a4b finetunes to handle

I've just released Voyage 4.0 EXP (experimental releases). I suggest you give EXP2 a try; it increases instruction adherane through recall and is a lot more modular. If some instructions cause Gemma 4 26B-A4B to loop, it is easy to turn them off as every instruction is split into it's own prompt block.

Once again, thank you very much for your review (it was a joy to read, please keep them coming!), and glad you liked the work!

I updated the preset again (Voyage v5), this time it should be tighter for Mistral Small 3.2 based models!
Mistral Small 3.2 didn't seem to like the XML approach, markdown-based formatting for the preset seems to work better.
Even got some coherent answers out of Gemma 4 E2B IT QAT this time.

My only (small) gripe is that it hides the NPC card / location with xml

I hope my workaround this time works well enough for you; I instructed the model that XML comments are hidden information.

I gave V5 a few tests with some of the models in my primary rotation - a Gemma 4 26b a4b finetune, a Gemma 4 31b finetune, a Magistral Small 24B 2509 finetune, as well as a couple outliers - GLM 4.7 Flash, REAP50'd 4.5 Air, a Gemma 3 12b tune, and a Qwen 3.5 9B tune (just for fun)

1. It works good for Gemma 4. The MoE Gemma in particular seems to benefit most from it. Logic of characters and narration had considerably improved in comparison to V3, which in turn offered improvement over stock freeform prompting. The positive rephrasing and simplifed instructions work wonders in shortening reasoning traces. It seems to nudge the characters away from default "assistant" into an actual actor l, which in turn somewhat reduces the dreaded Gemma turn endings.

Those alone were my primary gripe with Gemma 4, so for me it is a massive benefit.

2. GLM Flash sees a measurable improvement in story coherence over medium context (the hidden trackers seem to help it considerably), though it has an issue keeping track of individually declared instructions. Sometimes it focuses on the structure of the task itself, reasons over the 26 instructions list, and then outputs it verbatim. More of a GLM issue though.

3. My Mistral/ Magistral 24B builds seem to have issues when reasoning is injected. Could be an issue with quant - 4bit ffn gate, 5 bit up/down, q6/8 attn, and quantized KV (q8_0/q5_1). Without reasoning injected, it works fine, though the primary effect it has on Mistral is introducing the hidden notes (which it occasionally forgets to open or close with the respective tag, but it's... Well, Mistral 24B)

4. Surprisingly even Qwen 3.5 9B manages to benefit, though the instruction scaffolding often causes it to loop. It swings wildly from genuinely intriguing answers to absolute incoherence, none of which were observed with stock prompting. Not sure what exactly causes it.

My endorsement of this preset for Gemma 4 still stands, thank you much for the effort of making the new version

Thank you very much for testing such a wide range of models!
This is genuinely helpful feedback, some of it actionable on my end.

Gemma 4

Happy to hear that it has worked well for you. Gemma is incredibly worried when it comes to negative instructions, so turning it positive and giving it permission to be creative allows the model to actually explore directions and whatnot. I also found that explaining the social function (e.g. that speech and body pose is an action to get what they want) helped most models, especially Gemma.

I'm still not entirely happy with it though; therapy speak ("I don't have to wear a mask around you") and accountant speak (e.g. ledgers) seems to creep in sometimes, characters with jobs or professions become flat quickly. For the next version I want to experiment with "Occam's Razor" and finding the reason behind why it turns so flat or business-like by default. I suspect it's simply the model not knowing what to do, defaulting to Assistant.

Gemma 3

I wonder, how did the model perform here? Did it have any noticeable degration?

Mistral Small 3.2 / Magistral Small

For those models, quantinized KV really hurts performance more than running a smaller quant (e.g. Q4_K_S). The reasoning prompts causing issues is because it's too vague; even Mistral Large 4 (Le Chonk) needs incredibly precise instructions to function well. Instead of writing "Implement the card component", you need to write "Implement the GeneralCard component in file X, and look at the implementations of class A in file Y and class B in file Z to ensure you got the right method signatures for the functionality."

I wish I could do more for the open-close tags though... you could try to use a regex that finds unclosed tags after X amount of characters and makes sure it's injected.

GLM 4.7 Flash

Medium context... what is the size? 16K? 32K?

"and then outputs it verbatim." -> I'll check if I can add a guardrail for this.

Qwen 3.5

I'm quite suprised, I didn't test the model at all. Seems like a fun challenge to take on! Qwen 3.5 models are extremely prone to loops when incorrect samplers are used or when there is too much repetition in their reasoning. I think the latter is likely causing it, but I'll give it a closer look. If you have the ability to run Qwen3.6 27B at Q3 on your hardware, it is worth a try.

Those alone were my primary gripe

If you have other ones, I gladly hear them! It's hard to see the trees in the forest, so hearing what else could be improved is quite useful.

thank you much for the effort of making the new version

Likewise for these detailed reviews and your thoughts!

Gemma 3
I wonder, how did the model perform here? Did it have any noticeable degration?

In my particular case it was following instructions as well as Mistral Small 3.2 (forgetting a random few each time) and occasionally broke perspective, but the prose saw an improvement. I'm now considering trying it with gemma 3 27b to see if a stronger model (with less fanatical assistant personality baked in, at least in comparison to G4) would have a better time with it.

GLM 4.7 Flash
Medium context... what is the size? 16K? 32K?
It was about 25k-ish. That's after summaryception compression on an already ongoing long session, so it had a lot of concepts to be playing with simultaneously despite not having that many tokens. I noticed that it often confuses action/item propriety (as do all LLMs to an extent), but disabling some of the instructions helps it regain some attention budget. I didn't have much time to sort out which particular ones are to blame. It's not a huge issue, but it does necessitate an occasional reroll.

In my experience, however, even the tiny boi GLM 4.7 Flash is ahead of Gemma 4 in terms of literary and RP applications. So it can work wonders with less already.

Qwen 3.5
I'm quite suprised, I didn't test the model at all. Seems like a fun challenge to take on! Qwen 3.5 models are extremely prone to loops when incorrect samplers are used or when there is too much repetition in their reasoning. I think the latter is likely causing it, but I'll give it a closer look. If you have the ability to run Qwen3.6 27B at Q3 on your hardware, it is worth a try.

What surprises me most is that it wasn't even the bigger 27B/35B's, it was the small boi again. It did loop - such is its DNA, afterall - but when it didn't, it was writing much better prose and employed some turns I haven't seen my reference model (Skyfall) churn out. Occasionally. I have some Gemma3 / Qwen 3.5-8 27B's to test it against, but I'm unsure whether it would be fair - I have to run pretty low quants to fit into my system. Whether it would be base model, the particular artifact's brain compression, or the prompting structure itself playing tricks would be anyone's guess at this point

If you have other ones, I gladly hear them! It's hard to see the trees in the forest, so hearing what else could be improved is quite useful.

I think there could be some quality gained in letting the LLM reason over whether the particular response within a scene requires a reactive beat (shorter response length, such as active dialogue, quest actions, etc.) or a longer descriptive write-up (scene transitions, character monologues, chapter summaries etc.) G4 really likes finding patterns, and when it reads its previous replies it starts emulating their lengths.

Whereas with MS 24B it usually drifts - getting either longer or considerably more choppy depending on finetune - in my experience, G4 26B A4B likes to "lock in" to a particular response length. It can then pour out several paragraphs of monologue, complete with multiple questions, in an active back and forth conversation. Or do the opposite and describe a complex scene with a few simple sentences.

Alright, v6 is out!

It's a really big update this time, worth all the work that went into it.
I hope this release is just as enjoyable as the previous ones.
https://www.reddit.com/r/SillyTavernAI/comments/1xgcxcc/universal_preset_voyage_v6/

I've spent a little more tokens on average per prompt in return for much more clarity. More info on that here:
https://www.reddit.com/r/SillyTavernAI/comments/1xaptdu/discussing_prompting_techniques_october_2/

(if reddit isn't an option, I will happily download the threads and upload it here.)

Gemma 4 E2B is now outputting trackers nearly always correctly.
I hope it will work fine for Mistral Small 3.2 and akin, I'm slightly worried about it outputting random items or renaming.
Even so, worst case scenario just disable the regex.

As for Qwen3.5, I managed to reduce the looping but I can't fully stop it.
Turning off reasoning preserves a suprising amount of quality due to how I wrote the preset.
I recommend you give it a try without reasoning enabled.

Message length

I've experimented with some prompts, but in the end I think I need to delve deeper into sillytavern macros to fix this. Maybe a random?