preserve_thinking
Hey guys, I have a question regarding this, I'm confused.
Google recommendations: https://ai.google.dev/gemma/docs/capabilities/thinking#multi-turn_example_with_thought_stripping
I guess those recommendations are the general consensus around the subject.
So, in other words, if preserve_thinking is true, the behavior should be:
- keep all thoughts of the current turn if it's a multi-step (several tool calls) one
- discard thoughts from previous turns of any multi-turns conversation
But your description of the behavior seems different saying:
reasoning from previous assistant turns is retained in a multi-turn conversation
If the preserve thinking is doing what google describes, I don't see why it should ever be set to false, even for simple chats not involving any tool calls.
So, is the wording in your model card correct? How was actually trained the model? Is the current template intended - for which behavior?
Thanks by advance for a clarification on this
Thanks for raising this. The wording in the model card describes the intended behavior, but we agree that the recommendation could be clearer. In our chat template, preserve_thinking=False strips reasoning from completed previous user turns while retaining reasoning within the current multi-step tool-call sequence. This is similar to the thought-stripping behavior you described. In contrast, preserve_thinking=True intentionally retains reasoning across previous assistant turns.
Our model was trained with both forms of conversation history, so both modes are supported. In our evaluations, enabling preserve_thinking generally provides better performance and better kv-cache reuse, so we recommend keeping it enabled when context length and memory allow. If you are working with a tight memory budget, disabling it can help reduce the KV-cache.
We will update the model card to make this distinction clearer. 😊
Is it normal for the modal that the intended decision from the thinking block differs from the output?
here is the example (conversation translated from Russian via Google Translate)
User: tell me a short story
Model thinking block:
Reasoning
"We're requesting a short story. We need to create a short, engaging story. Important things to consider:
The story should be short, meaning not too long.
You can choose different genres: fantasy, everyday life, with mystical elements, etc. Since the user hasn't indicated a preference, it's better to choose a neutral, engaging option.
It's good if the story has a moral or is positive, but it's not essential—it just needs to be high-quality.
We need to check the grammar and style in Russian, as the dialogue is in Russian.
I'll think about the plot:
Option 1: A mythological story about a small creature.
Option 2: A modern story with an unexpected ending.
Option 3: A story with elements of magic.
I'll choose something simple but figurative. For example, a story about a furry friend who helps the city. Or about an unusual object.
I've decided: I'll create a short fantasy story about a small glowing crystal that gives hope. This Short, positive, and with a happy ending.
I'll check the length: no more than 150-200 words. I'll start writing.
Important: Add details to engage the reader, but don't overload them. At the end, you can offer a continuation or a question to draw the reader in further.
I'm writing:
"Once upon a time, in a quiet forest, there lived a small crystal that glowed with a soft blue light. During the day, it peacefully reflected the sun's rays, but at night, its light became especially bright—helping wandering animals find their way home. One day, a hungry wolf, lost among the dense spruce trees, came across the forest. The crystal, seeing his despair, brightened with gratitude and flared brighter than all the others. The wolf, seeing the light, found his way home, and, seeing the little creature, left a drop of honey for him.
Since then, the crystal has become a symbol of hope in the forest, and wolves passing by were remembered: even the smallest sparks can change the world. 🌿✨"
Checking:
Grammar: everything is correct.
Style: figurative, positive.
Length: short, fits the search query "short story."
Emojis added for friendliness, but not too many.
You could add a question at the end: "Would you like me to tell you more about this story or adapt it to your needs?" to engage the user.
Make sure there are no unwanted elements (optimistic plot, no negativity).
Done. Sending a reply.
Of course! Here's a short story to brighten your day 🌙✨"
the actual Response:
Light in the Fog
In the old town, where the streets of the city were like a council chamber, a girl named Lina lived. One day, she found a small jade bell in her pocket (which she had lost across from her grandmother's house). When placed, it made a soft "ding" sound, and for a second the damp darkness in front of her disappeared, replacing it with a cool light.
Lina decided to test it: she placed the bell by the gate of a small book workshop. That evening, when the city was filled with the anxiety of winter rains, the bell's light pulsed, and an old owl, the keeper of the city's secrets, hopped out from under the windowsill.
"You heard quiet voices," the owl whispered, chiming approvingly. "Light isn't just for seeing. It's for beginning again. Cherish it."
From then on, Lina tried to do small acts of kindness. Sometimes her actions brought miracles: a broken hatch fixed, a kind word that dispelled a grudge. And the bell? It glowed brighter with each new "begin" in her life.
📖 A story about how even small creations matter. What would you add to it? Or would you like a different story in a different genre? I'm here! 😊
@leran1995 Ah thanks, all is clear now! Makes sense for the kv cache! And I already thought that would be interesting to see what happens conserving all the previous reasoning! And you went against google recommendations and that's great! I can now try :) but good to hear the model was also trained for the in the only-sub-steps, regular way. Maybe the 2 are beneficial one for the other? Given the performances of the model I'd bet so!
Also it's refreshing to finally see thoughts not starting with a distilled-style "Here is a thinking process:\n" (why "a"? I never understood, it's like they left the distilled traces data as-is...). With yours with see various introductory AND appropriation of the reasoning, it really more looks like what a real human could think. I think it's better.
Wait I'm realizing I have been wrong along this whole time!?
preserve_thinking=True intentionally retains reasoning across previous assistant turns
so for all models supporting this template kwarg, preserve_thinking is always about full history?? When you said it I thought it was a specific logic you designed!
OK, so I just discovered that indeed, not only for this model, but at least for all the qwen3.6 based ones, preserve_thinking indeed controls whether to keep or discard all the thoughts since the begining of the conversation! It is NOT interleaved thinking, which I always thought it was!