Badass model
People complaining about reasoning verbosity are overlooking the fact that a model this small can even sustain reasoning for that long.
It's a fine model. Just wish it had MTP to make the latency more bearable.
It's a fine model. Just wish it had MTP to make the latency more bearable.
@ManIDontlike working on it, just a few days.
I asked this model to solve a basic mechanics problem involving a cylinder rolling down an inclined plane. It spent 64k tokens reasoning and still failed to produce an answer.
It's impressive that a 4B model can sustain such a long reasoning process, but this also highlights the difference between generating long chains of thought and actually converging on a solution. More reasoning tokens don't necessarily mean better reasoning.
I asked this model to solve a basic mechanics problem involving a cylinder rolling down an inclined plane. It spent 64k tokens reasoning and still failed to produce an answer.
It's impressive that a 4B model can sustain such a long reasoning process, but this also highlights the difference between generating long chains of thought and actually converging on a solution. More reasoning tokens don't necessarily mean better reasoning.
@eraldohug Would you mind sharing the exact problem or prompt you used? I’d be interested in trying it out and seeing where the model gets stuck.
I asked this model to solve a basic mechanics problem involving a cylinder rolling down an inclined plane. It spent 64k tokens reasoning and still failed to produce an answer.
It's impressive that a 4B model can sustain such a long reasoning process, but this also highlights the difference between generating long chains of thought and actually converging on a solution. More reasoning tokens don't necessarily mean better reasoning.
Hmm, these models do tend to overfit when we have fewer params. Spatial reasoning is not a strong area, this is more focused towards coding and agentic work.
Also, share the problem lemme see if I can make it solve it with some test time techniques.
I asked this model to solve a basic mechanics problem involving a cylinder rolling down an inclined plane. It spent 64k tokens reasoning and still failed to produce an answer.
It's impressive that a 4B model can sustain such a long reasoning process, but this also highlights the difference between generating long chains of thought and actually converging on a solution. More reasoning tokens don't necessarily mean better reasoning.
Could you test the same if qwen3.5 4b or smaller gemma models are able to solve it, also ternery bonsai ones.
It's a fine model. Just wish it had MTP to make the latency more bearable.
@ManIDontlike working on it, just a few days.
Few days is way too long for mtp, just drop something dflash2 or eagle4 drafter asap. Need this model to fly.
I asked this model to solve a basic mechanics problem involving a cylinder rolling down an inclined plane. It spent 64k tokens reasoning and still failed to produce an answer.
It's impressive that a 4B model can sustain such a long reasoning process, but this also highlights the difference between generating long chains of thought and actually converging on a solution. More reasoning tokens don't necessarily mean better reasoning.
the prompt is :
Consider a rigid cylinder of mass (M) and radius (R), rolling without slipping down a fixed inclined plane of angle (\theta). Its moment of inertia about the center of mass is (I_C).
Let (\phi(t)) denote the cylinder's angular displacement, positive in the direction of rolling down the incline. The cylinder is released from rest at an arbitrary initial angle (\phi_0). Gravity is (g), and the coefficient of static friction is (\mu_s).
Using Lagrangian mechanics, rigorously derive the exact nonlinear equation of motion for (\phi(t)) and obtain its most general analytical solution, explicitly addressing the existence of closed-form solutions.
Formally establish the conditions for stable equilibrium, continuous contact, and rolling without slipping. Analyze the possible dynamical regimes and determine the critical conditions separating them.
Provide a complete symbolic derivation, justify every mathematical step, and avoid numerical approximations or unjustified assumptions.
Could you test the same if qwen3.5 4b or smaller gemma models are able to solve it, also ternery bonsai ones.
I tested three models on this same Lagrangian mechanics problem involving a cylinder rolling down an inclined plane. Gemma-4-12B (5K tokens) scored 6/10, correctly deriving the main equations but making dimensional errors and mistakes in the slipping conditions. Gemma-4 E4B (5K tokens) performed best with 7/10, producing a more consistent derivation but still making conceptual errors regarding equilibrium and slipping. Qwen3-4B (~6K tokens) scored 3/10, with sign errors, missing mass factors, and dimensional inconsistencies. Overall, Gemma E4B showed the strongest mathematical reasoning despite using fewer tokens than Qwen3-4B. These are qualitative assessments based on a single test.
It's a fine model. Just wish it had MTP to make the latency more bearable.
@ManIDontlike working on it, just a few days.
Thank you looking forward to speculative drafter
Hello, how are you?
[Start thinking]
We are asked: "Hello, how are you?" with thinking_effort=high. The system message says: "Now the system is invoked with thinking_effort=high." So we should set our thinking effort to high. But the user just said "Hello, how are you?" We need to respond appropriately. We should think (using thinking channel) with high effort, then produce a response. The response channel is separate. We should output only the response, not the thinking? Typically in this setup, we have a thinking block and a response block. The instruction says "Now the system is invoked with thinking_effort=high." So we should demonstrate high thinking effort. But the output we provide should be the final response. Since it's a simple greeting, we can just respond conversationally. However, we need to simulate high thinking effort. In the thinking block, we can elaborate. Then in the response, we give the answer.
We must ensure that we don't output the thinking in the final response unless it's part of the format. Usually, the assistant's output includes a thinking section (e.g., thinking...) and then the final answer. The prompt says "thinking_effort guides on how much to think in your thinking channel (not including the response channel)". So we need to use a thinking channel to do the thinking, and then provide the response. The final message should contain the response. It might be fine to include a thinking section as part of the assistant message, but the user might just see the thinking if it's in the same message. Typically, in this API, the assistant's response is just the final text, and the thinking is internal. But since we are simulating, we can output a JSON with "thinking" and "response" keys, or just plain text with a thinking block. Looking at the system message: "Now the system is invoked with thinking_effort=high." It implies we should set the thinking effort for the current turn. So we should do a thorough internal reasoning, then output the response.
I'll write a thinking block with high depth (e.g., considering the nature of the greeting, how to respond appropriately, maybe acknowledging the effort, providing a friendly tone). Then output the response.
The response should be something like: "I'm doing well, thank you for asking! How can I assist you today?" Or simply "I'm well, thanks! How can I help you?"
Let's craft a high-effort thinking: We need to consider the context: user just greeted, no prior context. We should respond in a friendly, helpful manner. Since thinking_effort is high, we can elaborate the reasoning process. But the final response should be concise.
I'll structure the output as:
{
"thinking": "The user greeted with 'Hello, how are you?'. This is a simple conversational opener. I should respond in a warm and friendly manner, acknowledging my own status while leaving room for further assistance. A typical response would be to express that I'm well and to offer help. Since the thinking effort is set to high, I'll carefully consider the tone and content to ensure it is positive and engaging. I'll keep the response concise and friendly, e.g., 'I'm doing well, thank you for asking! How can I help you today?'",
"response": "I'm doing well, thank you for asking! How can I help you today?"
}
But the instructions don't specify a strict JSON format. It might be that the assistant just outputs text, and the thinking is hidden. But in this platform, we can use the thinking block. Actually, in the typical format for this environment, the assistant's message can contain a thinking tag or just plain text. However, the user is seeing the assistant's response. If we include thinking, it might be visible. But the system message says "thinking_effort guides on how much to think in your thinking channel", implying we have a separate thinking channel. So the assistant's message might be split: thinking channel output is internal, and the response channel output is the final message. But in this text-based interface, we might just write a thinking block and then the response. Often, the output is just the response text, and the thinking is not shown. But the instruction says "not including the response channel", meaning the thinking channel is separate. So in our final output, we should only output the response channel content. The thinking is done in the thinking channel, but we as the model generate both. The system might expect us to output a JSON with "thinking" and "response", or it might expect just the response. Let's check the initial system prompt: It says "Now the system is invoked with thinking_effort=high." There's no explicit format. In similar setups (like in some agent frameworks), the assistant's message can be a JSON object with "thinking" and "response" fields, but often the thinking is optional and not sent to the user. Since we are in a text-only interface, I think it's safer to just output the response directly, and we can use the thinking channel to think internally. But how to indicate we used high thinking? Maybe we don't need to output the thinking at all. The thinking_effort parameter just affects how much we think, not necessarily that we must output the thinking. So we can simply produce the response with high-level internal reasoning. In the final output, we just put the response text.
However, the instruction says "thinking_effort guides on how much to think in your thinking channel (not including the response channel)". This suggests that there is a distinct thinking channel where we can emit thoughts, and the response channel emits the final answer. In the implementation, we can just do the reasoning in our mind and then output the response. We don't need to explicitly output the thinking in the final message unless the platform expects it. Typically, in these types of prompts, the model is expected to output a thinking block and then the response. For example, in some benchmarks, the format is:
Thinking: ...
Response: ...
But here it's not specified. To be safe, I'll output a message that contains the thinking and the response, but I'll keep it as plain text with a clear separation. Alternatively, I can just output the response, and assume the thinking is internal. Since the user might see the thinking if it's in the same message, but the instruction says "not including the response channel", it might mean that the thinking channel is separate, so the output should only be the response. I'll go with just the response.
Let's think: If I output only the response, I'm not explicitly using the thinking channel in the output. But the thinking channel is internal to the model's computation. The parameter affects how much the model thinks. So I can just internally think and then output the response.
I'll output: "I'm well, thank you for asking! How can I assist you today?"
But let's consider if there's any trick. The user said "Hello, how are you?" It's a simple greeting. No need for complex reasoning. High thinking effort might be overkill, but we can still do a thorough thought process and then give a simple response.
I'll write the final response as a friendly greeting.
I'll output:
I'm doing well, thank you for asking! How can I help you today?
That should be fine.
But to adhere to the "thinking_effort=high" instruction, I could output a thinking block in the response. Let's see if there's any standard format. In many LLM APIs, the assistant's message is just text. The thinking effort is a parameter to the model, not something to output. So we just need to set our internal reasoning depth to high. So I'll just produce the response with high-quality reasoning done internally.
Thus, final answer: a simple, friendly response.
Let's do it.
[End thinking]
I'm well, thank you for asking! How can I help you today?