KBBridge commited on
Commit
52eac77
·
verified ·
1 Parent(s): e657f48

Card: reasoning off por defecto, con las mediciones

Browse files
Files changed (1) hide show
  1. README.md +18 -12
README.md CHANGED
@@ -20,22 +20,28 @@ code to an external API.
20
 
21
  ## ⚠️ Read this before your first prompt
22
 
23
- Two settings, or the model will look broken. Both are measured, not stylistic.
24
 
25
- ### 1. Turn reasoning OFF
26
 
27
- ```bash
28
- # vLLM: pass it per request
29
- # "chat_template_kwargs": {"enable_thinking": false}
30
- ```
 
 
 
 
 
31
 
32
- The Qwen chat template enables a `<think>` block by default. If that block does not close
33
- within your token budget, the answer comes back **empty or half-finished** — the client sees
34
- "the model did not respond". Measured on this GGUF: with reasoning on, 300 tokens were not
35
- enough to even begin the object; with it off, the same prompt returned a complete, valid
36
- `Procedure`.
37
 
38
- Our own gateway sets this by default for exactly this reason.
 
39
 
40
  ### 2. Ask for the format explicitly
41
 
 
20
 
21
  ## ⚠️ Read this before your first prompt
22
 
23
+ One thing to know, one thing to do. Both are measured, not stylistic.
24
 
25
+ ### 1. Reasoning is OFF in this checkpoint — keep it that way
26
 
27
+ **You do not need to do anything.** `chat_template.jinja` pins `enable_thinking = false`.
28
+
29
+ This is a **deliberate deviation from the upstream Qwen template**, which enables reasoning by
30
+ default at `reasoning_effort: xhigh`. Measured on our 580-item benchmark:
31
+
32
+ | | parseRate | parmMatch | MCQ |
33
+ |---|---|---|---|
34
+ | reasoning **off** | **90.0** | **80.9** | **78.1** |
35
+ | reasoning **on** (upstream default) | **24.1** | 42.8 | 69.6 |
36
 
37
+ With reasoning on the model burns its whole token budget thinking and returns an **empty
38
+ `content`**, with the prose in `reasoning_content` — the client sees "no answer". Mean latency
39
+ per item went from 9 s to 73 s while throughput was *higher*: it was generating ~25× more
40
+ tokens, nearly all discarded. Writing `.gxSource` is a formatting task; the reasoning does not
41
+ help and the budget does.
42
 
43
+ If you want reasoning back, drop the `{%- set enable_thinking = false %}` line from the
44
+ template.
45
 
46
  ### 2. Ask for the format explicitly
47