Card: reasoning off por defecto, con las mediciones
Browse files
README.md
CHANGED
|
@@ -20,22 +20,28 @@ code to an external API.
|
|
| 20 |
|
| 21 |
## ⚠️ Read this before your first prompt
|
| 22 |
|
| 23 |
-
|
| 24 |
|
| 25 |
-
### 1.
|
| 26 |
|
| 27 |
-
```
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
``
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
|
| 38 |
-
|
|
|
|
| 39 |
|
| 40 |
### 2. Ask for the format explicitly
|
| 41 |
|
|
|
|
| 20 |
|
| 21 |
## ⚠️ Read this before your first prompt
|
| 22 |
|
| 23 |
+
One thing to know, one thing to do. Both are measured, not stylistic.
|
| 24 |
|
| 25 |
+
### 1. Reasoning is OFF in this checkpoint — keep it that way
|
| 26 |
|
| 27 |
+
**You do not need to do anything.** `chat_template.jinja` pins `enable_thinking = false`.
|
| 28 |
+
|
| 29 |
+
This is a **deliberate deviation from the upstream Qwen template**, which enables reasoning by
|
| 30 |
+
default at `reasoning_effort: xhigh`. Measured on our 580-item benchmark:
|
| 31 |
+
|
| 32 |
+
| | parseRate | parmMatch | MCQ |
|
| 33 |
+
|---|---|---|---|
|
| 34 |
+
| reasoning **off** | **90.0** | **80.9** | **78.1** |
|
| 35 |
+
| reasoning **on** (upstream default) | **24.1** | 42.8 | 69.6 |
|
| 36 |
|
| 37 |
+
With reasoning on the model burns its whole token budget thinking and returns an **empty
|
| 38 |
+
`content`**, with the prose in `reasoning_content` — the client sees "no answer". Mean latency
|
| 39 |
+
per item went from 9 s to 73 s while throughput was *higher*: it was generating ~25× more
|
| 40 |
+
tokens, nearly all discarded. Writing `.gxSource` is a formatting task; the reasoning does not
|
| 41 |
+
help and the budget does.
|
| 42 |
|
| 43 |
+
If you want reasoning back, drop the `{%- set enable_thinking = false %}` line from the
|
| 44 |
+
template.
|
| 45 |
|
| 46 |
### 2. Ask for the format explicitly
|
| 47 |
|