File size: 9,703 Bytes
beba946
61a25fe
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
beba946
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="A reproducible benchmark of English, Spanish and Portuguese information retrieval methods.">
<title>Multilingual ModelBench | Rodrigo Pantoja</title>
<link rel="stylesheet" href="styles.css">
<script defer src="app.js"></script>
</head>
<body>
<header class="hero">
  <div class="shell">
    <div class="eyebrow"><span class="dot"></span> RODRIGO PANTOJA <span class="slash">/</span> APPLIED AI & RESEARCH ENGINEERING</div>
    <div class="hero-grid">
      <div>
        <p class="series">PORTFOLIO LAB <strong>005</strong> <span class="edition">RESEARCH SERIES</span></p>
        <h1><span class="mark">◈</span> Multilingual<br><span class="gradient">ModelBench</span></h1>
        <p class="lede">Evaluate retrieval methods across English, Spanish and Portuguese. Inspect real benchmark runs, diagnose difficult cross-language queries and reproduce every reported metric.</p>
        <div class="tags"><span>EN · ES · PT</span><span>Information retrieval</span><span>Reproducible Python</span><span>Open benchmark</span></div>
      </div>
      <div class="hero-panel" aria-label="Evaluation coverage">
        <div class="hero-panel-title"><span class="live-dot"></span> EXPERIMENT SNAPSHOT <span class="mono">v3.0</span></div>
        <div class="mini-grid">
          <div><strong>48</strong><small>Query cases</small></div>
          <div><strong>4</strong><small>Executed baselines</small></div>
          <div><strong>6</strong><small>Language directions</small></div>
          <div><strong>5</strong><small>Research domains</small></div>
        </div>
        <div class="hero-note">Results are from a small curated evaluation set, not a universal ranking. Semantic models remain <b>not evaluated</b>.</div>
      </div>
    </div>
  </div>
</header>
<main class="shell">
  <section class="toolbar" aria-label="Benchmark controls">
    <div class="toolbar-left">
      <div class="control-label">EVALUATION MODE</div>
      <div class="segmented" role="group" aria-label="Dataset split">
        <button id="splitMono" type="button" class="active" aria-pressed="true">Within-language <span class="small-count">30</span></button>
        <button id="splitCross" type="button" aria-pressed="false">Cross-language <span class="small-count">18</span></button>
      </div>
    </div>
    <div class="toolbar-right"><span id="lastStatus" class="status-text" role="status">Loading verified benchmark data...</span><button type="button" id="exportBtn" class="outline-button">↓ Export summary CSV</button></div>
  </section>
  <div id="errorBanner" class="error-banner" hidden role="alert"></div>
  <nav class="tabbar" aria-label="Views">
    <button type="button" class="tab active" data-view="overview" aria-selected="true">01 <span>Overview</span></button>
    <button type="button" class="tab" data-view="explorer" aria-selected="false">02 <span>Query explorer</span></button>
    <button type="button" class="tab" data-view="methodology" aria-selected="false">03 <span>Methodology</span></button>
  </nav>

  <section id="overview" class="view active">
    <div class="section-heading"><div><p class="micro">AT A GLANCE</p><h2>Evaluation dashboard</h2></div><p class="muted" id="splitDescription">30 monolingual query-to-passage tests</p></div>
    <div class="stats" id="stats"></div>
    <div class="wide-grid">
      <section class="panel"><div class="panel-top"><div><p class="micro">LEADERBOARD</p><h3>Model comparison</h3></div><label class="compact-label">Metric<select id="metricSelect"><option value="accuracy_at_1">Accuracy@1</option><option value="mrr">Mean reciprocal rank</option></select></label></div>
        <div id="leaderboard" class="bars"></div>
        <div class="note-inline">These are measured lexical baselines from the stored Python experiment.</div>
      </section>
      <section class="panel insight-panel"><div class="panel-top"><div><p class="micro">EXPERIMENT INSIGHT</p><h3>What the scores reveal</h3></div><span class="outlined-tag">DIAGNOSTIC</span></div>
        <div id="insight" class="insight-text"></div>
        <div class="callout-text">Accuracy@1 measures whether the relevant passage ranked first. MRR also rewards finding it in second or third place.</div>
      </section>
    </div>
    <section class="panel breakdown"><div class="panel-top"><div><p class="micro">PERFORMANCE BY SEGMENT</p><h3>Where retrieval succeeds or fails</h3></div><div class="selectors"><label class="compact-label">Method<select id="modelSelect"></select></label><label class="compact-label">Group by<select id="groupSelect"><option value="by_language">Query language</option><option value="by_domain">Domain</option><option value="by_language_pair">Language direction</option></select></label></div></div><div id="breakdown" class="breakdown-grid"></div><p class="footnote">Small subgroup sample sizes are displayed with each result. Comparisons across tiny groups can be unstable.</p></section>
    <section class="panel"><div class="panel-top"><div><p class="micro">EVALUATION REGISTRY</p><h3>Neural candidates for the next phase</h3></div><span class="outlined-tag">NO FABRICATED METRICS</span></div><div id="semanticRows" class="semantic-grid"></div><p class="footnote">These models are configured in the reproducible Python script but have not been executed in this benchmark. They are never included in the ranking.</p></section>
  </section>

  <section id="explorer" class="view">
    <div class="section-heading"><div><p class="micro">EXPLAINABILITY LAB</p><h2>Inspect every retrieval decision</h2></div><p class="muted">Every case contains a question and three candidate passages.</p></div>
    <div class="explorer-grid">
      <section class="panel explorer-control"><div class="panel-top"><div><p class="micro">EVALUATION CASES</p><h3>Choose an experiment</h3></div><span id="caseCount" class="outlined-tag"></span></div>
        <label class="field-label" for="querySearch">Search questions</label><input id="querySearch" type="search" placeholder="Search question, case ID or domain">
        <div class="filter-grid"><label class="compact-label">Language<select id="langFilter"></select></label><label class="compact-label">Domain<select id="domainFilter"></select></label></div>
        <div id="caseList" class="case-list" role="list"></div>
      </section>
      <section class="panel case-detail"><div class="panel-top"><div><p class="micro">SCORE EXPLANATION</p><h3>Passage ranking</h3></div><label class="compact-label">Method<select id="caseModelSelect"></select></label></div>
        <div id="caseDetail"></div>
      </section>
    </div>
  </section>

  <section id="methodology" class="view">
    <div class="section-heading"><div><p class="micro">REPRODUCIBLE RESEARCH</p><h2>Methodology and limitations</h2></div><p class="muted">Transparent by design, with the data and runner included.</p></div>
    <div class="method-grid">
      <section class="panel"><p class="micro">01 / DATASET</p><h3>Research Retrieval Eval Mini</h3><p>48 manually curated query-to-passage tasks: 30 within-language examples and 18 cross-language examples. Every query has one positive passage, one difficult negative passage and one easier negative passage.</p><p>English, Spanish and Portuguese; five domains: research, artificial intelligence, business, education and Latin America.</p></section>
      <section class="panel"><p class="micro">02 / METHODS</p><h3>Four measured baselines</h3><p>Character TF-IDF (3 to 5 character n-grams), word TF-IDF (1 to 2 word n-grams), hybrid TF-IDF (60% character and 40% word), and BM25.</p><p>Embeddings are future candidates, not secretly inferred scores.</p></section>
      <section class="panel"><p class="micro">03 / METRICS</p><h3>How to interpret the numbers</h3><p><b>Accuracy@1</b>: the fraction of queries where the positive passage ranks first. <b>MRR</b>: the average inverse rank of the positive passage.</p><p>Scores are based on three candidates per question. This is a controlled diagnostic, not open-web search evaluation.</p></section>
      <section class="panel"><p class="micro">04 / LIMITATIONS</p><h3>What we cannot conclude</h3><p>The curated dataset is small. Content and negative examples may contain lexical patterns that favor certain methods. Some cross-language cognates make character matching unusually effective.</p><p>This does not establish broad model superiority, statistical significance, generalization or reliable performance on unseen production data.</p></section>
    </div>
    <section class="panel reproduce"><div><p class="micro">RUN IT YOURSELF</p><h3>Reproducibility in four commands</h3><p>These files are included in your published Hugging Face repository. No Python is required to view the dashboard.</p></div>
      <div class="code-area"><code>pip install -r requirements-benchmark.txt</code><code>python benchmark.py --models lexical --split monolingual</code><code>python benchmark.py --models lexical --split cross-language</code><code>python benchmark.py --models all --split cross-language</code></div>
      <p class="footnote">The last command attempts neural models and needs additional optional dependencies and downloaded weights. Output is written to your local JSON files, not automatically uploaded to the Space.</p>
    </section>
  </section>
  <footer class="footer"><div><b>Multilingual ModelBench</b><span> · Research portfolio by Rodrigo Pantoja · 2026</span></div><div>Code: MIT · Included dataset: CC BY 4.0 · Static hosting</div></footer>
</main>
</body>
</html>