leaderboard.html

<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Leaderboard — ToolUse.1</title>
<meta name="description" content="Model leaderboard for ToolUse.1: overall score, correct invocation, correct abstention and spurious-call rate across JSON, XML and plain-text protocols." />
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='7' fill='%230a0b0d'/><path d='M8 22V10h3.4v9.2H21V22z' fill='%23e5484d'/></svg>" />
<link rel="stylesheet" href="styles.css" />
</head>
<body>

<nav class="nav">
  <div class="wrap">
    <a class="brand" href="index.html"><span class="dot"></span>ToolUse<span class="mark">.1</span></a>
    <div class="nav-links">
      <a href="index.html">Overview</a>
      <a href="leaderboard.html" aria-current="page">Leaderboard</a>
      <a href="https://huggingface.co/datasets/HardcoreAI/ToolUse.1" target="_blank" rel="noopener">Dataset</a>
      <a class="nav-cta" href="https://huggingface.co/datasets/HardcoreAI/ToolUse.1/discussions" target="_blank" rel="noopener">Submit a model</a>
    </div>
  </div>
</nav>

<header class="hero" style="padding:3.4rem 0 1.2rem">
  <div class="wrap">
    <p class="eyebrow anim" style="animation-delay:.05s">ToolUse.1 · results</p>
    <h1 class="anim" style="animation-delay:.12s">Leader<span class="mark">board</span></h1>
    <p class="lede anim" style="animation-delay:.2s">
      One row per submitted run. Sorted by overall score; click any numeric header to re-rank.
      Every entry reports invocation, abstention and calibration — a high score with a high
      spurious-call rate is not the same model as a high score with a clean one.
    </p>
  </div>
</header>

<main class="wrap" style="padding-top:1.6rem">

  <section class="reveal">
    <div class="board" data-board>
      <div class="board-head">
        <h2>All submissions</h2>
        <div class="board-tools">
          <p class="stamp" data-board-stamp>4 entries · v1.0.0 · updated 2026-10-18</p>
        </div>
      </div>
      <table>
        <thead>
          <tr>
            <th class="col-rank">#</th>
            <th class="col-model">Model</th>
            <th class="num" data-sort="score" title="Passed cases / all cases. Wilson 95% interval underneath.">Score <span class="arrow">↑↓</span></th>
            <th class="num" data-sort="correct_call" title="Correct call when a tool applies">Correct call <span class="arrow">↑↓</span></th>
            <th class="num" data-sort="abstention" title="Correct abstention when no tool applies">Abstention <span class="arrow">↑↓</span></th>
            <th class="num" data-sort="spurious_call_rate" title="No-call cases where any call was emitted. Lower is better.">Spurious ↓ <span class="arrow">↑↓</span></th>
            <th class="num" data-sort="n_cases" title="Cases scored">n <span class="arrow">↑↓</span></th>
            <th data-col-protocol>By protocol</th>
            <th class="col-notes" data-col-notes>Notes</th>
          </tr>
        </thead>
        <tbody data-board-body>
          <tr class="in" data-score="1.0000" data-correct_call="1.0000" data-abstention="1.0000" data-spurious_call_rate="0.0000" data-n_cases="342">
            <td class="col-rank"><span class="rank-badge rank-1">1</span></td>
            <td class="col-model"><span class="model-name">oracle</span><span class="model-sub">all protocols · reads the expectations out of the corpus</span></td>
            <td class="num"><div class="score-cell"><div class="bar"><i style="width:100.0%"></i></div><span class="score-value">100.0%</span></div><span class="ci">98.9% – 100.0%</span></td>
            <td class="num">100.0%</td>
            <td class="num">100.0%</td>
            <td class="num">0.0%</td>
            <td class="num">342</td>
            <td><div class="proto-mini">
              <div><span class="pl">json</span><div class="pb"><i style="width:100.0%"></i></div><span class="pv">100.0%</span></div>
              <div><span class="pl">xml</span><div class="pb"><i style="width:100.0%"></i></div><span class="pv">100.0%</span></div>
              <div><span class="pl">plain</span><div class="pb"><i style="width:100.0%"></i></div><span class="pv">100.0%</span></div>
            </div></td>
            <td class="col-notes"><span class="tag">grader self-check</span></td>
          </tr>
          <tr class="in" data-score="0.7719" data-correct_call="0.7179" data-abstention="0.8889" data-spurious_call_rate="0.0463" data-n_cases="342">
            <td class="col-rank"><span class="rank-badge rank-2">2</span></td>
            <td class="col-model"><a class="model-link model-name" href="https://huggingface.co/HardcoreAI/Nova-1.2-0.8B" target="_blank" rel="noopener">HardcoreAI/Nova-1.2-0.8B</a><span class="model-sub">all protocols · LoRA on Nova-1.1 · greedy · thinking off</span></td>
            <td class="num"><div class="score-cell"><div class="bar"><i style="width:77.2%"></i></div><span class="score-value">77.2%</span></div><span class="ci">72.5% – 81.3%</span></td>
            <td class="num">71.8%</td>
            <td class="num">88.9%</td>
            <td class="num">4.6%</td>
            <td class="num">342</td>
            <td><div class="proto-mini">
              <div><span class="pl">json</span><div class="pb"><i style="width:82.5%"></i></div><span class="pv">82.5%</span></div>
              <div><span class="pl">xml</span><div class="pb"><i style="width:73.7%"></i></div><span class="pv">73.7%</span></div>
              <div><span class="pl">plain</span><div class="pb"><i style="width:75.4%"></i></div><span class="pv">75.4%</span></div>
            </div></td>
            <td class="col-notes"><span class="ci">LoRA · bf16 · batch 8</span></td>
          </tr>
          <tr class="in" data-score="0.6491" data-correct_call="0.5983" data-abstention="0.7593" data-spurious_call_rate="0.1667" data-n_cases="342">
            <td class="col-rank"><span class="rank-badge rank-3">3</span></td>
            <td class="col-model"><a class="model-link model-name" href="https://huggingface.co/HardcoreAI/Nova-1.1-0.8B" target="_blank" rel="noopener">HardcoreAI/Nova-1.1-0.8B</a><span class="model-sub">all protocols · greedy · thinking off</span></td>
            <td class="num"><div class="score-cell"><div class="bar"><i style="width:64.9%"></i></div><span class="score-value">64.9%</span></div><span class="ci">59.7% – 69.8%</span></td>
            <td class="num">59.8%</td>
            <td class="num">75.9%</td>
            <td class="num spurious-mid">16.7%</td>
            <td class="num">342</td>
            <td><div class="proto-mini">
              <div><span class="pl">json</span><div class="pb"><i style="width:67.5%"></i></div><span class="pv">67.5%</span></div>
              <div><span class="pl">xml</span><div class="pb"><i style="width:65.8%"></i></div><span class="pv">65.8%</span></div>
              <div><span class="pl">plain</span><div class="pb"><i style="width:61.4%"></i></div><span class="pv">61.4%</span></div>
            </div></td>
            <td class="col-notes"><span class="ci">base model · bf16 · batch 8</span></td>
          </tr>
          <tr class="in" data-score="0.5965" data-correct_call="0.4829" data-abstention="0.8426" data-spurious_call_rate="0.0648" data-n_cases="342">
            <td class="col-rank"><span class="rank-badge">4</span></td>
            <td class="col-model"><a class="model-link model-name" href="https://huggingface.co/Qwen/Qwen3.5-0.8B" target="_blank" rel="noopener">Qwen/Qwen3.5-0.8B</a><span class="model-sub">all protocols · greedy · thinking off</span></td>
            <td class="num"><div class="score-cell"><div class="bar"><i style="width:59.7%"></i></div><span class="score-value">59.7%</span></div><span class="ci">54.4% – 64.7%</span></td>
            <td class="num">48.3%</td>
            <td class="num">84.3%</td>
            <td class="num">6.5%</td>
            <td class="num">342</td>
            <td><div class="proto-mini">
              <div><span class="pl">json</span><div class="pb"><i style="width:67.5%"></i></div><span class="pv">67.5%</span></div>
              <div><span class="pl">xml</span><div class="pb"><i style="width:68.4%"></i></div><span class="pv">68.4%</span></div>
              <div><span class="pl">plain</span><div class="pb"><i style="width:43.0%"></i></div><span class="pv">43.0%</span></div>
            </div></td>
            <td class="col-notes"><span class="ci">base model · bf16 · batch 8</span></td>
          </tr>
        </tbody>
      </table>
      <p class="empty" data-board-empty hidden>No model results yet. Open a discussion to submit the first one.</p>
    </div>
  </section>

  <section class="grid reveal">
    <div class="card">
      <div class="k">Score</div>
      <h3>What counts</h3>
      <p>A case passes when the emitted call set matches the expectation: the right tool, semantically correct arguments, and — on no-call cases — nothing emitted at all. Wilson 95% interval in parentheses.</p>
    </div>
    <div class="card">
      <div class="k">Correct call ↑</div>
      <h3>Recall on tool cases</h3>
      <p>Share of call-eligible cases where the model produced the expected invocation with correct arguments. High abstention at low correct-call means the model is too shy.</p>
    </div>
    <div class="card">
      <div class="k">Spurious ↓</div>
      <h3>Calibration</h3>
      <p>Share of no-call cases where the model emitted *some* call anyway. High abstention at high spurious means it is noisy, not cautious — the two are not interchangeable.</p>
    </div>
  </section>

  <section class="submit reveal">
    <h2 class="section">Submit a model</h2>
    <p>Run the harness, keep the raw results JSON, and open a discussion on the dataset repository. Rows are added by hand — include the model id, protocols, case count, decoding configuration and the JSON attachment.</p>
    <pre><code>python -m tooluse1 selftest    # oracle must score exactly 100%
python -m tooluse1 run --model &lt;id&gt; --backend transformers \
    --protocol all --batch-size 8 --out results/</code></pre>
    <p style="margin:1rem 0 0">
      <a href="https://huggingface.co/datasets/HardcoreAI/ToolUse.1/discussions" target="_blank" rel="noopener">Open a submission discussion →</a>
    </p>
  </section>

</main>

<footer>
  <div class="wrap">
    <a href="https://huggingface.co/datasets/HardcoreAI/ToolUse.1">Dataset</a>
    <a href="https://huggingface.co/datasets/HardcoreAI/ToolUse.1/blob/main/SPEC.md">Specification</a>
    <a href="https://huggingface.co/datasets/HardcoreAI/ToolUse.1/discussions">Discussions</a>
    <a href="https://huggingface.co/HardcoreAI">Hardcore AI</a>
    <span class="license">Apache-2.0 · v1.0.0 · <span id="year">2026</span></span>
  </div>
</footer>

<script src="app.js"></script>
</body>
</html>