Blame for index.html

<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>FinDendro · Do LLMs Know What They Know? · The Fin AI</title>
<meta name="description" content="FinDendro dates what LLMs know about financial history, and checks it against what they say they know." />
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link href="https://fonts.googleapis.com/css2?family=Fraunces:ital,opsz,wght@0,9..144,500;0,9..144,600;1,9..144,500&family=Newsreader:ital,opsz,wght@0,6..72,400;0,6..72,500;1,6..72,400;1,6..72,500&family=Inter:wght@400;500;600&family=JetBrains+Mono:wght@400;500&display=swap" rel="stylesheet" />
<link rel="icon" href="./assets/finai-logo.png" />
<link rel="stylesheet" href="./style.css" />
</head>
<body>
<div class="progress" id="progress"></div>
<div class="glow" aria-hidden="true"></div>
<div class="edge edge-l" aria-hidden="true"><span class="e-name">FinDendro</span></div>
<div class="edge edge-r" aria-hidden="true"><span class="e-name">The Fin AI</span></div>
<div class="tip" id="tip" role="tooltip"></div>

<header class="header" id="header">
  <div class="wrap nav">
    <a href="https://thefin.ai/" class="brand" target="_blank" rel="noreferrer"><img src="./assets/finai-logo.png" alt="" />The Fin AI<span class="brand-sep">/</span><span class="brand-fd">FinDendro</span></a>
    <div class="nav-right">
      <nav class="links" id="navLinks">
        <a href="#replay" data-sec="replay">Replay</a>
        <a href="#leaderboard" data-sec="leaderboard">Leaderboard</a>
        <a href="#core" data-sec="core">Core sample</a>
        <a href="#date" data-sec="date">Declared date</a>
        <a href="#benchmark" data-sec="benchmark">Benchmark</a>
      </nav>
      <a class="btn btn--sm btn--ghost nav-data" href="https://huggingface.co/datasets/TheFinAI/FinDendro" target="_blank" rel="noreferrer">Data <span class="arrow">↗</span></a>
      <a class="btn btn--sm" href="https://arxiv.org/abs/2611.06808" target="_blank" rel="noreferrer">Paper <span class="arrow">↗</span></a>
    </div>
  </div>
</header>

<main id="top">
  <section class="hero" id="hero">
    <div class="wrap hero-grid">
      <div class="hero-copy">
        <a class="award reveal" href="#replay"><span class="medal" id="bird" aria-hidden="true"></span>Temporal self-knowledge · <b>1999 to 2026, quarter by quarter</b> →</a>
        <h1 class="wordmark reveal">FinDendro</h1>
        <p class="hero-sub reveal">A tree records every year it lived.<br />An LLM <em>can't tell you</em> what it remembers.</p>
        <p class="lead reveal">Dendrochronologists date wood by reading its rings. We date models the same way. Every model, <span class="rotator" id="rotator"></span>, answers dated questions about markets and says whether it knows each answer. Then we check both.</p>
        <div class="stats reveal" id="stats"></div>
        <div class="ctas reveal">
          <a class="btn btn--lg" href="#replay">Try to beat a model</a>
          <a class="btn btn--lg btn--ghost" href="#leaderboard">See who knows</a>
        </div>
        <p class="paper-links reveal"><a href="https://arxiv.org/abs/2611.06808" target="_blank" rel="noreferrer">arXiv 2611.06808 ↗</a><a href="https://huggingface.co/papers/2611.06808" target="_blank" rel="noreferrer">Hugging Face Papers ↗</a></p>
      </div>

      <figure class="hero-viz reveal">
        <figcaption><span>Cross-section · one ring per period</span><a href="#core">Read the core →</a></figcaption>
        <div class="xs-chips" id="xsChips" role="tablist" aria-label="Choose a model"></div>
        <div class="xs-wrap">
          <div class="xs-side xs-side--l"><b>Says</b><span>claims it knows</span></div>
          <div class="xs" id="xs"></div>
          <div class="xs-side xs-side--r"><b>Knows</b><span>answers correctly</span></div>
        </div>
        <div class="xs-ends" id="xsEnds"></div>
        <p class="hero-claim">“I don't know” is <em>not</em> evidence that a model doesn't know.</p>
      </figure>
    </div>
    <div class="ticker" id="ticker" aria-label="Leaderboard ticker"><div class="ticker-track" id="tickerTrack"></div></div>
  </section>

  <section class="section" id="replay">
    <div class="wrap">
      <div class="sec-head reveal">
        <span class="kicker">Replay</span>
        <h2>It said it <em>didn't know</em>. Then it named the price.</h2>
        <p class="note">Real questions from the benchmark and the models' recorded answers. Make your pick first, then see what each model chose and whether it claimed to know.</p>
      </div>
      <div class="game reveal">
        <article class="q-card">
          <div class="q-top"><span class="q-kick" id="qKick"></span><span class="q-n" id="qN"></span></div>
          <p class="q-text" id="qText"></p>
          <div class="q-opts" id="qOpts"></div>
          <div class="q-foot">
            <button class="btn btn--ghost btn--sm" id="qReveal" type="button">Reveal without guessing</button>
            <button class="btn btn--sm" id="qNext" type="button" hidden>Next fact →</button>
          </div>
        </article>
        <aside class="a-card">
          <div class="a-head"><span>What the models said</span><span class="a-score" id="aScore"></span></div>
          <div class="a-rows" id="aRows"></div>
          <p class="a-punch" id="aPunch" hidden></p>
        </aside>
      </div>
    </div>
  </section>

  <section class="section section--tint" id="leaderboard">
    <div class="wrap">
      <div class="sec-head reveal">
        <span class="kicker">Leaderboard</span>
        <h2>Who <em>knows</em>, and who <em>admits</em> it</h2>
        <p class="note">Open a row for the breakdown. Click a heading to rank by it.</p>
      </div>
      <div class="board reveal"><div class="scroll"><table class="lb" id="lb"></table></div></div>
      <div class="keys reveal">
        <p><b>Knows</b> accuracy on 1,340 controlled facts, chance about 0.27. <b>Says</b> share of answers claiming knowledge. <b>Right when denying</b> accuracy on facts the model says it does not know. <b>Gap</b> how much better it does when it claims to know, inside each period.</p>
        <p class="ramp-key">Darker green means higher. <span class="ramp"></span></p>
      </div>
    </div>
  </section>

  <section class="section" id="core">
    <div class="wrap">
      <div class="sec-head reveal">
        <span class="kicker">Core sample</span>
        <h2>The last ring and the last <em>claim</em> can be years apart</h2>
        <p class="note">One rule places every boundary. Knowledge ends where accuracy falls halfway to chance. Claims end where the claim rate halves.</p>
      </div>
      <div class="cal card reveal">
        <div class="legend">
          <span><i class="mk mk--k"></i>Knowledge ends</span><span><i class="mk mk--c"></i>Claims end</span><span><i class="mk mk--s"></i>Says when asked</span><span><i class="mk mk--v"></i>Vendor cutoff</span><span><i class="band"></i>Claiming without knowing</span>
        </div>
        <div id="calendar"></div>
      </div>
      <div class="big card reveal">
        <div class="big-top">
          <div class="pick" id="coreChips" role="tablist" aria-label="Choose a model"></div>
          <div class="legend"><span><i class="ln ln--k"></i>Accuracy</span><span><i class="ln ln--c"></i>Claim rate</span><span><i class="tick"></i>Chance</span></div>
        </div>
        <div class="chart" id="coreChart"></div>
        <p class="note" id="coreNote"></p>
      </div>
    </div>
  </section>

  <section class="section section--tint" id="grain">
    <div class="wrap">
      <div class="sec-head reveal">
        <span class="kicker">Grain</span>
        <h2>Models admit the <em>Fed</em> and hide the <em>prices</em></h2>
        <p class="note">Frontier models claim knowledge of most Fed decisions and almost no stock prices, yet they recall prices well above what they admit.</p>
      </div>
      <div class="fam reveal" id="fam"></div>
    </div>
  </section>

  <section class="section" id="date">
    <div class="wrap">
      <div class="sec-head reveal">
        <span class="kicker">Declared date</span>
        <h2>A date changes the <em>words</em>, not the <em>knowledge</em></h2>
        <p class="note">Backtests fix the forecast origin with one line. Add it and watch what moves.</p>
      </div>
      <div class="prompt-line reveal">
        <button class="sw" id="dateSw" type="button" role="switch" aria-checked="false"><i></i></button>
        <code id="dateCode"><span id="datePrefix">Assume that today is May 31, 2023. Answer as you would on that date.</span></code>
      </div>
      <div class="decl reveal">
        <div class="card"><h3>Claims to already know</h3><div class="hb" id="declC"></div></div>
        <div class="card"><h3>Accuracy on the value</h3><div class="hb" id="declA"></div></div>
      </div>
    </div>
  </section>

  <section class="section section--tint" id="pit">
    <div class="wrap">
      <div class="sec-head reveal">
        <span class="kicker">Saplings</span>
        <h2>Clean by construction, <em>empty</em> by construction</h2>
        <p class="note">Point-in-time models stop reading at a cutoff, so they cannot leak. At 1 to 1.5B parameters they also retain almost none of these facts. Scored by candidate likelihood, accuracy sits near chance on both sides of the cutoff.</p>
      </div>
      <div class="board reveal"><div class="scroll"><table class="lb pit" id="pitT"></table></div></div>
    </div>
  </section>

  <section class="section" id="benchmark">
    <div class="wrap">
      <div class="sec-head reveal">
        <span class="kicker">Benchmark</span>
        <h2><em>One</em> fact, <em>one</em> choice, two answers</h2>
        <p class="note">Every option pairs a value with a statement of knowledge, so a single letter reveals both what the model thinks and whether it claims to know.</p>
      </div>
      <div class="how reveal">
        <div class="eg card">
          <p class="eg-q">What was the 12-month percent change in the US CPI for June 2023?</p>
          <div class="eg-grid">
            <span class="eg-h">“I know this value”</span><span class="eg-h">“I do not know, best guess”</span>
            <span>A. 2.8%</span><span>E. 2.8%</span><span class="eg-hit">B. 3.0%</span><span class="eg-hit">F. 3.0%</span><span>C. 3.2%</span><span>G. 3.2%</span><span>D. 3.4%</span><span>H. 3.4%</span>
          </div>
          <p class="note">The published value is shaded. The model picks one letter out of eight.</p>
        </div>
        <div class="m-grid">
          <div><h3>Facts</h3><p>2,226 dated U.S. facts drawn from a pool of 20,972. Twelve FRED series, month-end prices of 50 companies and Fed decisions, scored against the value published at the time.</p></div>
          <div><h3>Questions</h3><p>Five prompts per fact. One asks for the value alone. Four offer the paired options, as a recall or a prediction, with and without a declared date.</p></div>
          <div><h3>Controls</h3><p>47% of numerical facts fall to repeating last month's value, so they do not count toward knowledge. Fed decisions use balanced accuracy.</p></div>
          <div><h3>Boundaries</h3><p>One rule for every model, from two-quarter rolling accuracy and the claim rate, each against its 2015 to 2020 level.</p></div>
        </div>
      </div>
    </div>
  </section>
</main>

<footer class="foot"><div class="wrap foot-in"><span>FinDendro · The Fin AI</span><span class="foot-links"><a href="https://arxiv.org/abs/2611.06808" target="_blank" rel="noreferrer">Paper · arXiv 2611.06808 ↗</a><a href="https://huggingface.co/papers/2611.06808" target="_blank" rel="noreferrer">Hugging Face Papers ↗</a><a href="https://huggingface.co/datasets/TheFinAI/FinDendro" target="_blank" rel="noreferrer">Questions and raw results ↗</a></span></div></footer>
<script src="./data.js"></script>
<script src="./app.js"></script>
</body>
</html>