Spaces:
Running
Running
| <html lang="en"> | |
| <head> | |
| <meta charset="utf-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1"> | |
| <title>Does this model need a reasoning parser?</title> | |
| <!-- The Space renders inside an iframe and huggingface.co sends | |
| x-frame-options: DENY, so any link without a target tries to load a | |
| refused page in the frame and reads as broken. --> | |
| <base target="_blank"> | |
| <style> | |
| :root { | |
| --bg: #fbfbfa; --panel: #ffffff; --ink: #1a1a19; --muted: #6b6b66; | |
| --line: #e4e3df; --accent: #3d5a80; | |
| --problem: #a03030; --problem-bg: #fdf3f2; | |
| --check: #8a6212; --check-bg: #fdf9ee; | |
| --clear: #2f6b45; --clear-bg: #f1f8f3; | |
| --mono: ui-monospace, "SF Mono", "Cascadia Mono", Menlo, monospace; | |
| } | |
| @media (prefers-color-scheme: dark) { | |
| :root { | |
| --bg: #16171a; --panel: #1d1f23; --ink: #e8e6e3; --muted: #9a978f; | |
| --line: #2e3137; --accent: #8ab0d9; | |
| --problem: #e08a86; --problem-bg: #2a1e1e; | |
| --check: #d9b76a; --check-bg: #29241a; | |
| --clear: #8fc9a6; --clear-bg: #1b2620; | |
| } | |
| } | |
| * { box-sizing: border-box; } | |
| body { | |
| margin: 0; background: var(--bg); color: var(--ink); | |
| font: 16px/1.65 ui-sans-serif, system-ui, -apple-system, "Segoe UI", sans-serif; | |
| -webkit-font-smoothing: antialiased; | |
| } | |
| .wrap { max-width: 50rem; margin: 0 auto; padding: 3.5rem 1.25rem 5rem; } | |
| .brand { display: inline-block; margin-bottom: 1.75rem; } | |
| .brand img { height: 1.5rem; width: auto; display: block; filter: invert(1); opacity: .78; } | |
| @media (prefers-color-scheme: dark) { .brand img { filter: none; opacity: .9; } } | |
| header h1 { | |
| font-size: clamp(1.6rem, 4.5vw, 2.2rem); line-height: 1.2; | |
| letter-spacing: -0.02em; margin: 0 0 .75rem; font-weight: 620; | |
| } | |
| header p { color: var(--muted); margin: 0 0 .5rem; max-width: 42rem; } | |
| form { display: flex; gap: .5rem; flex-wrap: wrap; margin: 2rem 0 .75rem; } | |
| input[type=text] { | |
| flex: 1 1 20rem; min-width: 0; padding: .7rem .85rem; | |
| border: 1px solid var(--line); border-radius: .5rem; | |
| background: var(--panel); color: var(--ink); | |
| font: inherit; font-family: var(--mono); font-size: .92rem; | |
| } | |
| input[type=text]:focus { outline: 2px solid var(--accent); outline-offset: -1px; border-color: transparent; } | |
| button { | |
| padding: .7rem 1.4rem; border: 0; border-radius: .5rem; | |
| background: var(--accent); color: #fff; font: inherit; font-weight: 560; cursor: pointer; | |
| } | |
| button:hover { filter: brightness(1.08); } | |
| button:disabled { opacity: .55; cursor: progress; } | |
| .examples { font-size: .86rem; color: var(--muted); margin-bottom: 2.5rem; line-height: 2; } | |
| .examples .group { display: inline-block; min-width: 6.5rem; } | |
| .examples button { | |
| background: none; color: var(--accent); padding: 0 .15rem; font-size: .86rem; | |
| font-family: var(--mono); text-decoration: underline; text-underline-offset: 2px; font-weight: 400; | |
| } | |
| .card { | |
| background: var(--panel); border: 1px solid var(--line); | |
| border-radius: .7rem; padding: 1.1rem 1.3rem; margin: .85rem 0; | |
| } | |
| .card h3 { margin: 0 0 .5rem; font-size: 1.02rem; font-weight: 600; } | |
| .card p { margin: .5rem 0; } | |
| .flag { | |
| font-family: var(--mono); font-size: .95rem; background: var(--bg); | |
| border: 1px solid var(--line); border-radius: .4rem; | |
| padding: .55rem .8rem; margin: .6rem 0; overflow-x: auto; white-space: nowrap; | |
| } | |
| .problem { border-left: 3px solid var(--problem); background: var(--problem-bg); } | |
| .problem h3 { color: var(--problem); } | |
| .check { border-left: 3px solid var(--check); background: var(--check-bg); } | |
| .check h3 { color: var(--check); } | |
| .clear { border-left: 3px solid var(--clear); background: var(--clear-bg); } | |
| .clear h3 { color: var(--clear); } | |
| .meta { | |
| display: flex; flex-wrap: wrap; gap: .35rem 1.5rem; font-size: .86rem; | |
| color: var(--muted); padding-bottom: .9rem; margin-bottom: .3rem; | |
| border-bottom: 1px solid var(--line); | |
| } | |
| .meta code { color: var(--ink); } | |
| code { font-family: var(--mono); font-size: .86em; } | |
| ul { margin: .5rem 0; padding-left: 1.1rem; } | |
| li { margin: .2rem 0; } | |
| li code { font-size: .84rem; } | |
| h2 { font-size: 1.05rem; font-weight: 600; letter-spacing: -0.01em; margin: 3rem 0 .75rem; } | |
| table { border-collapse: collapse; width: 100%; font-size: .87rem; } | |
| th, td { text-align: left; padding: .45rem .7rem .45rem 0; border-bottom: 1px solid var(--line); vertical-align: top; } | |
| th { font-weight: 600; color: var(--muted); font-size: .8rem; text-transform: uppercase; letter-spacing: .04em; } | |
| .scroll { overflow-x: auto; } | |
| footer { margin-top: 3.5rem; padding-top: 1.5rem; border-top: 1px solid var(--line); font-size: .87rem; color: var(--muted); } | |
| footer a { color: var(--accent); } | |
| .spin { color: var(--muted); font-size: .9rem; } | |
| </style> | |
| </head> | |
| <body> | |
| <div class="wrap"> | |
| <header> | |
| <a class="brand" href="https://assert.gr" rel="noopener"> | |
| <img src="./assert-logo.png" alt="ASSERT" width="420" height="87"> | |
| </a> | |
| <h1>Does this model need a reasoning parser?</h1> | |
| <p> | |
| A model that thinks needs vLLM's <code>--reasoning-parser</code> to split that | |
| thinking out of the answer. Getting it wrong fails in two ways, and only one | |
| of them is visible. | |
| </p> | |
| <p> | |
| <strong>Missing</strong> parser: the user sees the model's thinking in the | |
| reply — obvious, and quickly fixed. <strong>Wrong</strong> parser: it claims | |
| the entire output, the request finishes with <code>stop</code>, and | |
| <code>content</code> comes back <strong>empty</strong>. Nothing errors. | |
| </p> | |
| <p> | |
| This reads the repo's <strong>chat template only</strong> — a few kilobytes, | |
| fetched from your browser — and asks the one question that decides it: does | |
| the assistant's turn carry a marker some parser closes on? | |
| </p> | |
| </header> | |
| <form id="form"> | |
| <input type="text" id="repo" value="google/gemma-4-12B-it" | |
| placeholder="owner/model" autocomplete="off" spellcheck="false" | |
| aria-label="Model repo id"> | |
| <button type="submit" id="go">Check</button> | |
| </form> | |
| <div class="examples"> | |
| <span class="group">Siblings that differ:</span> | |
| <button type="button" data-ex="Qwen/Qwen3.6-35B-A3B">Qwen3.6-35B needs one</button> · | |
| <button type="button" data-ex="Qwen/Qwen3-VL-30B-A3B-Instruct">Qwen3-VL-30B must not have one</button> | |
| <br> | |
| <span class="group">Other shapes:</span> | |
| <button type="button" data-ex="google/gemma-4-12B-it">gemma channel</button> · | |
| <button type="button" data-ex="openai/gpt-oss-20b">gpt-oss analysis</button> · | |
| <button type="button" data-ex="microsoft/Phi-4-mini-instruct">no thinking at all</button> | |
| <br> | |
| <span class="group">Our packs:</span> | |
| <button type="button" data-ex="aleada/Nemotron-3.5-Lightning-30B-A3B-W4A16">Nemotron-3.5</button> · | |
| <button type="button" data-ex="aleada/Qwen3.8-27B-W4A16">Qwen3.8-27B</button> | |
| </div> | |
| <div id="out"></div> | |
| <h2>Why the family name cannot decide it</h2> | |
| <p> | |
| Two models from one family, one release apart, answer differently: | |
| <code>Qwen3.6-35B-A3B</code> emits <code><think></code> and needs the | |
| <code>qwen3</code> parser; <code>Qwen3-VL-30B</code> emits nothing of the sort | |
| and is <em>broken</em> by it. A quantized pack inherits its source's template, | |
| so the same split runs through every repack of them. | |
| </p> | |
| <p> | |
| The markers below were read out of vLLM's <code>vllm/reasoning/</code> modules, | |
| not from documentation — a parser matches what its code matches. Only markers | |
| that <strong>discriminate a reasoning section</strong> are listed: an earlier | |
| version of this table included <code><|end|></code>, gpt-oss's turn | |
| terminator, and recommended a parser for a model that does no thinking at all. | |
| </p> | |
| <div class="scroll"> | |
| <table> | |
| <thead><tr><th>Parser</th><th>Closes on</th></tr></thead> | |
| <tbody id="rules"></tbody> | |
| </table> | |
| </div> | |
| <footer> | |
| <p> | |
| <strong>A verdict of "none" is an answer, not an absence.</strong> It means | |
| the template carries no reasoning marker, so the model should be served | |
| <em>without</em> the flag — adding one is the silent failure described above. | |
| </p> | |
| <p> | |
| Markers extracted from vLLM <code>0.23.1rc1.dev552+g4559c43a9</code>; they are | |
| not a stable public API upstream. Parsers that hold their markers as token ids | |
| rather than strings cannot be matched this way — <code>qwen3</code> is listed | |
| because it closes on the same pair as the DeepSeek family, and that is stated | |
| rather than inferred. Gated and private repos cannot be read. This never loads | |
| the model and never runs it. | |
| </p> | |
| <p> | |
| Built from the tooling behind the | |
| <a href="https://huggingface.co/aleada" rel="noopener">aleada</a> W4A16 packs. | |
| See also the <a href="https://huggingface.co/spaces/aleada/pack-precision-map" rel="noopener">precision map</a>, the <a href="https://huggingface.co/spaces/aleada/pack-integrity-check" rel="noopener">pack integrity checker</a> and the <a href="https://huggingface.co/spaces/aleada/model-search-that-answers" rel="noopener">model search</a>. | |
| </p> | |
| </footer> | |
| </div> | |
| <script type="module" src="./app.js"></script> | |
| </body> | |
| </html> | |