Spaces:
Running
Running
Upload 5 files
Browse files- LICENSE +21 -0
- README.md +70 -5
- app.js +35 -0
- engine.js +125 -0
- index.html +64 -18
LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
MIT License
|
| 2 |
+
|
| 3 |
+
Copyright (c) 2026 Rodrigo Pantoja
|
| 4 |
+
|
| 5 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 6 |
+
of this software and associated documentation files (the "Software"), to deal
|
| 7 |
+
in the Software without restriction, including without limitation the rights
|
| 8 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 9 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 10 |
+
furnished to do so, subject to the following conditions:
|
| 11 |
+
|
| 12 |
+
The above copyright notice and this permission notice shall be included in all
|
| 13 |
+
copies or substantial portions of the Software.
|
| 14 |
+
|
| 15 |
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 16 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 17 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 18 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 19 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 20 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 21 |
+
SOFTWARE.
|
README.md
CHANGED
|
@@ -1,12 +1,77 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
| 8 |
license: mit
|
| 9 |
short_description: Evidence-based document search with clear source citations.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
---
|
| 11 |
|
| 12 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: ResearchRAG
|
| 3 |
+
emoji: 📚
|
| 4 |
+
colorFrom: blue
|
| 5 |
+
colorTo: indigo
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
| 8 |
license: mit
|
| 9 |
short_description: Evidence-based document search with clear source citations.
|
| 10 |
+
tags:
|
| 11 |
+
- nlp
|
| 12 |
+
- research
|
| 13 |
+
- document-retrieval
|
| 14 |
+
- multilingual
|
| 15 |
+
- explainable-ai
|
| 16 |
---
|
| 17 |
|
| 18 |
+
# ResearchRAG · Browser Edition
|
| 19 |
+
|
| 20 |
+
**Evidence-based multilingual document search with traceable source excerpts.**
|
| 21 |
+
|
| 22 |
+
A practical, free and public [Hugging Face Static Space](https://huggingface.co/docs/hub/spaces-sdks-static) developed as part of Rodrigo Pantoja's applied AI portfolio.
|
| 23 |
+
|
| 24 |
+
## What it does
|
| 25 |
+
|
| 26 |
+
- Loads example documents or accepts pasted text and TXT, Markdown, CSV files.
|
| 27 |
+
- Builds a browser-local passage index with source identifiers such as `D1:C2`.
|
| 28 |
+
- Retrieves relevant passages using BM25-style word relevance, query term coverage and character-trigram overlap.
|
| 29 |
+
- Extracts relevant **existing** source sentences, each tied to a document and passage citation.
|
| 30 |
+
- Displays retrieval scores for transparent interpretation and exports a Markdown evidence report.
|
| 31 |
+
- Supports English, Spanish and Portuguese token matching (but is not a full multilingual semantic search model).
|
| 32 |
+
- Runs on the visitor's machine without a paid compute instance or external generative API.
|
| 33 |
+
|
| 34 |
+
## Try it
|
| 35 |
+
|
| 36 |
+
1. Open the **App** tab.
|
| 37 |
+
2. Click **Load 3 example documents**.
|
| 38 |
+
3. Click **Find evidence**, or ask: `Why is source verification important in research?`.
|
| 39 |
+
4. Read the extractive evidence and use citation labels to identify source excerpts.
|
| 40 |
+
5. Download the Markdown report, or replace the demos with your own text.
|
| 41 |
+
|
| 42 |
+
## Privacy and supported formats
|
| 43 |
+
|
| 44 |
+
All submitted document text remains in the current browser tab for local indexing and search. This app does not intentionally upload source text to any backend or external AI API. Hosting platforms and your browser may still handle normal web connection metadata. Do not assume absolute security on a shared device.
|
| 45 |
+
|
| 46 |
+
**Accepted uploads:** `.txt`, `.md`, `.csv` (up to 1 MB/file, 15 documents maximum). Paste text instead of uploading, if desired.
|
| 47 |
+
|
| 48 |
+
**Not directly supported in this edition:** PDF, DOCX and scanned-image OCR. Copy/export their text before importing it. No documents persist across page refreshes.
|
| 49 |
+
|
| 50 |
+
## Reproducible method
|
| 51 |
+
|
| 52 |
+
1. Clean and segment document text into passages, usually around 650 characters (plus short overlap).
|
| 53 |
+
2. Tokenize and remove common stopwords for Spanish, English and Portuguese.
|
| 54 |
+
3. Index per-passage word frequencies and character 3-grams.
|
| 55 |
+
4. Rank passages using a weighted mix of BM25-like lexical relevance (63%), query term coverage (27%) and character similarity (10%).
|
| 56 |
+
5. Select a small set of extractive evidence sentences, retaining the passage and document names.
|
| 57 |
+
6. Export passages and citations without generating unsourced answers.
|
| 58 |
+
|
| 59 |
+
`engine.js` is independent of the user interface, enabling deterministic Node.js tests.
|
| 60 |
+
|
| 61 |
+
## Important limitations
|
| 62 |
+
|
| 63 |
+
This is an **explainable lexical retrieval system**, not generative question answering, deep semantic embeddings, RAG generation with an LLM, factual verification, or a trained search model. It can miss paraphrases, especially cross-language paraphrases without shared words. Its relevance score is **not** a probability of truth. Extracted text may lack important context. Always check the original source.
|
| 64 |
+
|
| 65 |
+
The original companion **Python/Gradio ResearchRAG** prototype implements an optional multilingual embedding retriever and experimental generation; this browser edition intentionally does not claim those features because it is hosted without paid server compute. This is a distinct deployable variant, not a replacement for the original source code.
|
| 66 |
+
|
| 67 |
+
## Project files
|
| 68 |
+
|
| 69 |
+
- `index.html`: interface and inline responsive CSS.
|
| 70 |
+
- `engine.js`: local indexing, hybrid lexical scoring and extractive evidence synthesis.
|
| 71 |
+
- `app.js`: user actions, uploads, display and report download.
|
| 72 |
+
- `LICENSE`: MIT license.
|
| 73 |
+
- `README.md`: project card and reproducibility notes.
|
| 74 |
+
|
| 75 |
+
## License and attribution
|
| 76 |
+
|
| 77 |
+
MIT License. Authored for Rodrigo Pantoja's professional open-source portfolio. Demo documents are original illustrative text, not a research corpus. No claims about an external benchmark or real-world model accuracy are made for this browser version.
|
app.js
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/* Browser UI for ResearchRAG Static */
|
| 2 |
+
(function(){
|
| 3 |
+
'use strict';
|
| 4 |
+
const $=id=>document.getElementById(id);
|
| 5 |
+
const state={documents:[],index:null,last:null,question:''};
|
| 6 |
+
const demo=ResearchRAG.DEMO;
|
| 7 |
+
function setStatus(msg,type='info'){const el=$('status');el.textContent=msg;el.dataset.type=type;}
|
| 8 |
+
function setText(id,text){$(id).textContent=String(text);}
|
| 9 |
+
function updateIndex(){state.index=ResearchRAG.buildIndex(state.documents);setText('metric-docs',state.index.docs.length);setText('metric-chunks',state.index.chunks.length);setText('metric-characters',state.index.docs.reduce((n,d)=>n+d.characters,0).toLocaleString());renderDocList();$('run').disabled=!state.index.N;}
|
| 10 |
+
function renderDocList(){const root=$('documents');root.replaceChildren();if(!state.documents.length){const p=document.createElement('p');p.className='muted empty';p.textContent='No documents loaded. Add text, upload .txt/.md/.csv or try the examples.';root.append(p);return;}
|
| 11 |
+
state.documents.forEach((d,i)=>{const row=document.createElement('div');row.className='doc-row';const name=document.createElement('span');name.className='doc-name';name.textContent=d.name;name.title=d.name;const len=document.createElement('span');len.className='doc-len';len.textContent=`${d.text.length.toLocaleString()} chars`;const btn=document.createElement('button');btn.type='button';btn.className='delete';btn.textContent='Remove';btn.setAttribute('aria-label',`Remove ${d.name}`);btn.onclick=()=>{state.documents.splice(i,1);updateIndex();clearResults();setStatus('Document removed. Index rebuilt.');};row.append(name,len,btn);root.append(row);});}
|
| 12 |
+
function clearResults(){state.last=null;$('results').hidden=true;$('results-empty').hidden=false;$('download').disabled=true;}
|
| 13 |
+
function addDoc(name,text){let s=String(text||'').trim();if(s.length<30)throw new Error(`${name}: add at least 30 characters of text.`);if(s.length>350000)throw new Error(`${name}: file is too large; maximum 350,000 characters per document.`);if(state.documents.length>=15)throw new Error('Maximum 15 documents at a time. Remove one before adding more.');const n=String(name||'Untitled document').slice(0,130);let newname=n,number=2;while(state.documents.some(d=>d.name===newname)){newname=`${n} (${number++})`;}state.documents.push({name:newname,text:s});}
|
| 14 |
+
function loadDemo(){state.documents=demo.map(d=>({...d}));updateIndex();$('question').value='Why is source verification important in academic research?';clearResults();setStatus('Three example documents loaded. Press “Ask the documents”.','ok');$('question').focus();}
|
| 15 |
+
function pasteDoc(){try{const name=$('doc-title').value.trim()||'Pasted notes';const text=$('doc-text').value;addDoc(name,text);$('doc-text').value='';$('doc-title').value='';updateIndex();clearResults();setStatus('Document added. You can ask a question now.','ok');}catch(e){setStatus(e.message,'error');}}
|
| 16 |
+
async function readFiles(files){if(!files.length)return;let count=0;const failed=[];
|
| 17 |
+
for(const file of [...files]){try{if(!/\.(txt|md|csv)$/i.test(file.name)){throw new Error('Only .txt, .md and .csv files are supported in this browser edition.');}if(file.size>1048576){throw new Error('File is larger than 1 MB.');}const txt=await file.text();addDoc(file.name,txt);count++;}catch(e){failed.push(`${file.name}: ${e.message}`);}}
|
| 18 |
+
updateIndex();clearResults();setStatus(`${count} file(s) imported.`+(failed.length?' '+failed.join(' '):''),failed.length?'error':'ok');$('files').value='';}
|
| 19 |
+
function textEl(tag,cl,value){const e=document.createElement(tag);if(cl)e.className=cl;e.textContent=value;return e;}
|
| 20 |
+
function renderResults(output){const p=$('results');p.hidden=false;$('results-empty').hidden=true;$('evidence').replaceChildren();$('answer').replaceChildren();setText('hit-count',output.hits.length);setText('result-status',output.status==='ok'?'Evidence found':'No supported answer');
|
| 21 |
+
if(output.status!=='ok'){$('answer').append(textEl('p','muted',output.message));$('download').disabled=true;return;}
|
| 22 |
+
$('answer').append(textEl('p','advisory','Extractive evidence, not a generated answer. Check the cited source passages.'));
|
| 23 |
+
for(const a of output.answer){const box=document.createElement('p');box.className='answer-line';box.append(document.createTextNode(a.text+' '));const cite=textEl('span','cite',`[${a.chunkId}]`);box.append(cite);$('answer').append(box);}
|
| 24 |
+
for(const h of output.hits){const card=document.createElement('article');card.className='evidence-card';const top=document.createElement('div');top.className='evidence-top';top.append(textEl('strong','',`[${h.id}] ${h.doc}`),textEl('span','score',`Relevance ${(h.score*100).toFixed(0)}%`));card.append(top,textEl('p','',h.text));const meta=textEl('p','footnote','Lexical overlap: '+h.matchCount+' query term(s); rank is not a truth score.');card.append(meta);$('evidence').append(card);}
|
| 25 |
+
$('download').disabled=false;
|
| 26 |
+
}
|
| 27 |
+
function ask(){const q=$('question').value.trim();if(!q){setStatus('Write a question before searching.','error');$('question').focus();return;}
|
| 28 |
+
const out=ResearchRAG.query(state.index,q,5);state.last=out;state.question=q;renderResults(out);setStatus(out.status==='ok'?`${out.hits.length} cited source passages found.`:out.message,out.status==='ok'?'ok':'error');$('results').scrollIntoView({behavior:'smooth',block:'start'});
|
| 29 |
+
}
|
| 30 |
+
function exportMd(){if(!state.last||state.last.status!=='ok')return;const q=state.question;const arr=['# ResearchRAG Evidence Report','','**Question:** '+q,'','**Important:** This is extractive, lexical evidence retrieval. It is not a generative answer and does not verify facts.','','## Evidence-based excerpts',''];for(const a of state.last.answer){arr.push('- '+a.text.replace(/\n/g,' ')+' ['+a.chunkId+']');}arr.push('','## Retrieved passages','');for(const h of state.last.hits){arr.push(`### [${h.id}] ${h.doc}`,'',h.text,'',`Lexical relevance: ${(h.score*100).toFixed(0)}%`,'');}
|
| 31 |
+
const blob=new Blob([arr.join('\n')],{type:'text/markdown;charset=utf-8'}),url=URL.createObjectURL(blob),link=document.createElement('a');link.href=url;link.download='researchrag-evidence-report.md';link.click();setTimeout(()=>URL.revokeObjectURL(url),2000);}
|
| 32 |
+
function init(){ $('demo').onclick=loadDemo;$('add-paste').onclick=pasteDoc;$('run').onclick=ask;$('download').onclick=exportMd;$('reset').onclick=()=>{state.documents=[];updateIndex();clearResults();setStatus('Documents cleared from this tab.');};$('files').onchange=e=>readFiles(e.target.files);
|
| 33 |
+
const drop=$('drop');drop.ondragover=e=>{e.preventDefault();drop.classList.add('over');};drop.ondragleave=()=>drop.classList.remove('over');drop.ondrop=e=>{e.preventDefault();drop.classList.remove('over');readFiles(e.dataTransfer.files);};$('question').onkeydown=e=>{if((e.ctrlKey||e.metaKey)&&e.key==='Enter')ask();};updateIndex();setStatus('Ready. Load the sample documents or add your own.');}
|
| 34 |
+
if(document.readyState==='loading')document.addEventListener('DOMContentLoaded',init);else init();
|
| 35 |
+
})();
|
engine.js
ADDED
|
@@ -0,0 +1,125 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/* ResearchRAG Browser Edition | Deterministic local retrieval, no model API. */
|
| 2 |
+
(function (root) {
|
| 3 |
+
'use strict';
|
| 4 |
+
const STOP = new Set(('a al algo algunas algunos ante como con contra cual cuando de del desde donde dos el ella ellos en entre era es esa ese eso esta estar este esto estos fue ha haber hacia hasta hay la las le les lo los mas mi mis muy no nos o para pero por porque que quien se ser si sin sobre son su sus tambien te tiene todo todos tu un una unas uno unos y ya ' +
|
| 5 |
+
'the this that these those with from there which what when where why how does do are was were have has had can could should would for and but not yes into over under between after before both each all any some than then your you our their its out per via into about ' +
|
| 6 |
+
'um uma uns umas do da dos das em para pela pelo pelos pelas aos nas nos por com nao no na e ou mas sao foi esta este isso esses como que quais onde quando ser ter sua suas seus seus ao').split(/\s+/));
|
| 7 |
+
const ACCENT = /[\u0300-\u036f]/g;
|
| 8 |
+
const norm = s => String(s || '').normalize('NFD').replace(ACCENT,'').toLowerCase().replace(/[^a-z0-9\s]/g,' ').replace(/\s+/g,' ').trim();
|
| 9 |
+
const tokens = text => norm(text).split(' ').filter(x => x.length >= 3 && !STOP.has(x)).slice(0,50000);
|
| 10 |
+
const trigrams = text => {
|
| 11 |
+
const s = norm(text).replace(/\s+/g,' '), out = new Set();
|
| 12 |
+
for(let i=0;i<s.length-2;i++) { const tri=s.slice(i,i+3); if(tri.trim().length>=2) out.add(tri); }
|
| 13 |
+
return out;
|
| 14 |
+
};
|
| 15 |
+
function segments(text, limit=650, overlap=1) {
|
| 16 |
+
const cleaned=String(text||'').replace(/\r/g,'').trim();
|
| 17 |
+
if(!cleaned) return [];
|
| 18 |
+
const paragraphs=cleaned.split(/\n\s*\n/).flatMap(p=> {
|
| 19 |
+
const p2=p.replace(/[ \t]+/g,' ').trim();
|
| 20 |
+
if(p2.length<=limit) return p2?[p2]:[];
|
| 21 |
+
// Sentence boundary first; split long sentences only at words.
|
| 22 |
+
const ss=p2.split(/(?<=[.!?])\s+(?=[^\s])/u), units=[];
|
| 23 |
+
for(const sentence of ss) {
|
| 24 |
+
if(sentence.length<=limit) { units.push(sentence); continue; }
|
| 25 |
+
let part='';
|
| 26 |
+
for(const word of sentence.split(/\s+/)) {
|
| 27 |
+
if(part.length+word.length+1>limit && part) {units.push(part);part='';}
|
| 28 |
+
if(word.length>limit) { if(part) {units.push(part);part='';} for(let k=0;k<word.length;k+=limit) units.push(word.slice(k,k+limit)); }
|
| 29 |
+
else part+=(part?' ':'')+word;
|
| 30 |
+
}
|
| 31 |
+
if(part)units.push(part);
|
| 32 |
+
}
|
| 33 |
+
const out=[];let curr='';
|
| 34 |
+
for(const unit of units) {
|
| 35 |
+
if(curr.length+unit.length+1>limit && curr){ out.push(curr);curr=''; }
|
| 36 |
+
curr+=(curr?' ':'')+unit;
|
| 37 |
+
}
|
| 38 |
+
if(curr)out.push(curr);
|
| 39 |
+
return out;
|
| 40 |
+
});
|
| 41 |
+
const result=[];
|
| 42 |
+
for(let i=0;i<paragraphs.length;i++) {
|
| 43 |
+
const val=paragraphs[i]; if(!val.trim())continue;
|
| 44 |
+
const prev=(overlap && i>0?paragraphs[i-1].slice(-120)+' ':'');
|
| 45 |
+
result.push((prev+val).slice(0,limit+130));
|
| 46 |
+
}
|
| 47 |
+
return result;
|
| 48 |
+
}
|
| 49 |
+
const DEMO=[
|
| 50 |
+
{name:'01_Research_Methods.txt',text:'A transparent research workflow begins with a clearly defined question, explicit inclusion criteria, and traceable evidence. Reproducibility improves when data sources, transformations, assumptions, and limitations are documented. Researchers should retain methodological notes and verify every citation against the original source.\n\nFor multilingual research, lexical retrieval identifies matching words and phrases across the indexed documents. Semantic similarity may help surface conceptually related evidence, but semantic similarity is not a truth score. Researchers must inspect the retrieved passages, preserve citations, and report uncertainty whenever the evidence is incomplete.\n\nA review should distinguish reported findings from interpretation. When a result cannot be linked to a source passage, it should not be presented as a verified conclusion.'},
|
| 51 |
+
{name:'02_Latin_America_Policy.txt',text:'Public policy research in Latin America often combines economic, social, educational, and institutional indicators. Cross-country comparisons require caution because definitions, coverage, and reporting years may differ. A responsible data dashboard must expose missing data, sources, and methodological limitations.\n\nEconomic indicators such as GDP per capita are useful for longitudinal comparisons, but they cannot on their own measure social wellbeing. Education indicators may reveal disparities in schooling access and completion. Reliable analyses compare like-for-like data and disclose uncertainty.\n\nOpen public data can improve reproducibility when the extraction date, indicator code, and source are recorded. Data revisions mean that historical series can change over time.'},
|
| 52 |
+
{name:'03_AI_in_Education_ES.txt',text:'La inteligencia artificial en educación puede facilitar la revisión de literatura y el análisis de documentos académicos. Sin embargo, las respuestas automáticas no reemplazan la validación humana de las fuentes.\n\nLa privacidad es especialmente importante cuando se procesan trabajos estudiantiles o investigaciones inéditas. Los documentos sensibles deben mantenerse bajo control de sus propietarios. Un sistema de recuperación documental debe mostrar fragmentos verificables, identificar cada documento y reconocer cuando no encuentra evidencia suficiente.\n\nLa transparencia metodológica exige explicar cómo se seleccionan los fragmentos relevantes y evitar presentar coincidencias de palabras como pruebas definitivas.'}
|
| 53 |
+
];
|
| 54 |
+
function buildIndex(documents) {
|
| 55 |
+
const chunks=[], docs=[];
|
| 56 |
+
documents.forEach((doc,d)=>{
|
| 57 |
+
const name=String(doc.name||`Document ${d+1}`).slice(0,130), text=String(doc.text||'').trim();
|
| 58 |
+
if(!text) return;
|
| 59 |
+
const slices=segments(text); let idx=0;
|
| 60 |
+
for(const slice of slices){idx++;chunks.push({id:`D${docs.length+1}:C${idx}`,docIndex:docs.length,doc:name,order:idx,text:slice,terms:tokens(slice),grams:trigrams(slice)});}
|
| 61 |
+
docs.push({name,characters:text.length,chunks:idx});
|
| 62 |
+
});
|
| 63 |
+
const N=chunks.length, dfs=new Map();let total=0;
|
| 64 |
+
for(const c of chunks){total+=c.terms.length;for(const t of new Set(c.terms))dfs.set(t,(dfs.get(t)||0)+1);}
|
| 65 |
+
return {docs,chunks,dfs,N,avgdl:N?total/N:1};
|
| 66 |
+
}
|
| 67 |
+
function scoreQuery(index, query) {
|
| 68 |
+
const qTerms=[...new Set(tokens(query))], qTrigrams=trigrams(query); const {N,dfs,avgdl}=index;
|
| 69 |
+
return index.chunks.map(c=>{
|
| 70 |
+
const counts=new Map();for(const t of c.terms)counts.set(t,(counts.get(t)||0)+1);
|
| 71 |
+
let bm=0;
|
| 72 |
+
for(const word of qTerms){ const tf=counts.get(word)||0;if(tf<=0)continue;
|
| 73 |
+
const idf=Math.log(1+(N-(dfs.get(word)||0)+.5)/((dfs.get(word)||0)+.5));
|
| 74 |
+
const k1=1.2,b=.75;
|
| 75 |
+
bm+=idf*(tf*(k1+1))/(tf+k1*(1-b+b*c.terms.length/Math.max(1,avgdl)));
|
| 76 |
+
}
|
| 77 |
+
let intersect=0; for(const tri of qTrigrams)if(c.grams.has(tri))intersect++;
|
| 78 |
+
const jac=qTrigrams.size?intersect/(qTrigrams.size+c.grams.size-intersect):0;
|
| 79 |
+
const wordCoverage=qTerms.length?qTerms.filter(t=>counts.has(t)).length/qTerms.length:0;
|
| 80 |
+
const lexical=bm/(bm+2.2);
|
| 81 |
+
const score=0.63*lexical+0.27*wordCoverage+0.10*jac;
|
| 82 |
+
return {...c,score,bm25:bm,wordCoverage,matchCount:qTerms.filter(t=>counts.has(t)).length};
|
| 83 |
+
}).filter(x=>x.score>=0.075 && (x.bm25>0 || x.wordCoverage>0 || x.score>=0.095)).sort((a,b)=>b.score-a.score);
|
| 84 |
+
}
|
| 85 |
+
function snippetSentences(text) {
|
| 86 |
+
const ss=String(text).split(/(?<=[.!?])\s+|\n+/u).map(x=>x.trim()).filter(x=>x.length>=25);
|
| 87 |
+
return ss.length?ss:[String(text).slice(0,400)];
|
| 88 |
+
}
|
| 89 |
+
function query(index,q,k=5) {
|
| 90 |
+
if(!index || !index.N)return {status:'no_documents',hits:[],answer:[],message:'Add a document before searching.'};
|
| 91 |
+
if(!tokens(q).length)return {status:'empty_question',hits:[],answer:[],message:'Enter a question with meaningful words.'};
|
| 92 |
+
const candidates=scoreQuery(index,q), hits=[], selectedIds=new Set();
|
| 93 |
+
// Diversity: prefer new documents or nonadjacent chunks for additional sources.
|
| 94 |
+
for(let i=0;i<Math.min(k,candidates.length);i++){
|
| 95 |
+
let best,bs=-100;
|
| 96 |
+
for(const c of candidates){if(selectedIds.has(c.id))continue;
|
| 97 |
+
const redundancy=hits.some(h=>h.docIndex===c.docIndex)?0.035:0;
|
| 98 |
+
const near=hits.some(h=>h.docIndex===c.docIndex && Math.abs(h.order-c.order)<=1)?0.025:0;
|
| 99 |
+
const v=c.score-redundancy-near;
|
| 100 |
+
if(v>bs){bs=v;best=c;}
|
| 101 |
+
}
|
| 102 |
+
if(!best)break;selectedIds.add(best.id);hits.push(best);
|
| 103 |
+
}
|
| 104 |
+
if(!hits.length)return {status:'no_evidence',hits:[],answer:[],message:'No sufficiently relevant passages found. Try different terms or add more source text.'};
|
| 105 |
+
const qTerms=new Set(tokens(q)); const answerCandidates=[];
|
| 106 |
+
hits.forEach(h=>{
|
| 107 |
+
for(const s of snippetSentences(h.text)){
|
| 108 |
+
const st=new Set(tokens(s)); const m=[...qTerms].filter(t=>st.has(t)).length;
|
| 109 |
+
const density=m/Math.max(1,qTerms.size);
|
| 110 |
+
const importance=0.65*h.score+0.35*density;
|
| 111 |
+
answerCandidates.push({text:s,chunkId:h.id,doc:h.doc,score:importance,match:m});
|
| 112 |
+
}
|
| 113 |
+
});
|
| 114 |
+
answerCandidates.sort((a,b)=>b.score-a.score);
|
| 115 |
+
const answer=[], seen=new Set();
|
| 116 |
+
for(const a of answerCandidates){
|
| 117 |
+
const key=norm(a.text).slice(0,120);if(seen.has(key))continue;
|
| 118 |
+
if(a.match===0 && answer.length) continue;
|
| 119 |
+
answer.push(a);seen.add(key);
|
| 120 |
+
if(answer.length>=3)break;
|
| 121 |
+
}
|
| 122 |
+
return {status:'ok',hits,answer,message:'Evidence-based excerpt selection. Verify against the original documents; this is not a generative answer.'};
|
| 123 |
+
}
|
| 124 |
+
root.ResearchRAG={buildIndex,query,segments,tokens,norm,DEMO};
|
| 125 |
+
})(typeof window!=='undefined'?window:globalThis);
|
index.html
CHANGED
|
@@ -1,19 +1,65 @@
|
|
| 1 |
-
<!
|
| 2 |
-
<html>
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
</html>
|
|
|
|
| 1 |
+
<!DOCTYPE html>
|
| 2 |
+
<html lang="en">
|
| 3 |
+
<head>
|
| 4 |
+
<meta charset="utf-8">
|
| 5 |
+
<meta name="viewport" content="width=device-width,initial-scale=1">
|
| 6 |
+
<meta name="description" content="ResearchRAG Browser Edition: evidence-first document search with transparent citations.">
|
| 7 |
+
<title>ResearchRAG | Evidence-First Document Search</title>
|
| 8 |
+
<style>
|
| 9 |
+
:root{color-scheme:dark;--bg:#07111f;--panel:#0e2033;--panel2:#10283c;--stroke:#284056;--muted:#a6bfd0;--text:#f2f7ff;--blue:#54a6ff;--turq:#4fe0c1;--yellow:#ffc774;--red:#ffb1af}
|
| 10 |
+
*{box-sizing:border-box}html{scroll-behavior:smooth}body{margin:0;background:radial-gradient(ellipse 60% 30% at 86% 0,rgba(35,96,138,.24),transparent),var(--bg);color:var(--text);font:15px/1.55 ui-sans-serif,system-ui,-apple-system,"Segoe UI",Roboto,Arial,sans-serif}
|
| 11 |
+
main{max-width:1170px;padding:44px 22px 90px;margin:auto}h1,h2,h3,p{margin-top:0}header{padding:0 0 26px}header .eyebrow{color:#80c9ed;text-transform:uppercase;letter-spacing:.19em;font-size:11px;font-weight:800;margin-bottom:9px}.hero h1{font-size:clamp(36px,6.5vw,72px);letter-spacing:-.05em;line-height:1.05;margin:0 0 18px}.hero p{color:#c1d5e2;max-width:770px;font-size:17px}.badges{display:flex;flex-wrap:wrap;gap:8px;margin-top:18px}.badges span{border:1px solid #385b74;border-radius:40px;padding:4px 10px;color:#d3e9f4;font-size:12px}
|
| 12 |
+
.layout{display:grid;grid-template-columns:390px minmax(0,1fr);gap:20px;align-items:start}.column{display:flex;flex-direction:column;gap:18px}.card{background:linear-gradient(150deg,rgba(18,43,63,.95),rgba(9,28,45,.97));border:1px solid var(--stroke);border-radius:18px;padding:22px;box-shadow:0 14px 45px rgba(0,0,0,.09)}.card h2{font-size:20px;letter-spacing:-.02em;margin:0 0 6px}.subtitle,.muted{color:var(--muted)}.subtitle{font-size:13px;margin-bottom:20px}.field{display:flex;flex-direction:column;gap:7px;margin:16px 0}.field label{font-weight:700;font-size:13px}.field small{color:var(--muted)}input[type=text],textarea{color:var(--text);width:100%;background:#081829;border:1px solid #365269;padding:12px 13px;border-radius:11px;font:inherit;resize:vertical;outline:none}input[type=text]:focus,textarea:focus{border-color:var(--turq);box-shadow:0 0 0 3px rgba(79,224,193,.12)}textarea{min-height:104px}#question{min-height:110px}button{cursor:pointer;font:inherit;font-weight:750;border-radius:10px;border:1px solid #3d6078;padding:10px 13px;background:#19344b;color:#e4f4fc;transition:filter .15s,transform .15s}button:hover:not(:disabled){filter:brightness(1.15)}button:focus-visible{outline:3px solid #83d8ec;outline-offset:2px}button:disabled{opacity:.4;cursor:not-allowed}.primary{border-color:#3998ff;background:#287ae6;color:white}.secondary{background:#122a3d}.button-row{display:flex;flex-wrap:wrap;gap:9px}.button-row>*{flex:1 1 auto}.drop{border:2px dashed #42657a;padding:18px;border-radius:12px;text-align:center;display:block;cursor:pointer;background:#0c1f31}.drop:hover,.drop.over{border-color:var(--turq);background:#103147}.drop input{position:absolute;opacity:0;width:1px;height:1px}.drop span{display:block;color:#dbeef7;font-weight:700}.drop small{color:var(--muted)}.doc-row{display:flex;align-items:center;gap:8px;padding:9px 0;border-top:1px solid #2a4557}.doc-name{font-size:12px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;min-width:0;flex:1}.doc-len{font-size:11px;color:var(--muted);white-space:nowrap}.delete{background:none;padding:3px 7px;font-size:11px;border-color:#536a76}.empty{font-size:13px}#status{font-size:13px;padding:11px 13px;border-radius:10px;background:#10283c;color:#c8dfe8;border:1px solid #31536a}#status[data-type=error]{color:#ffd7d4;border-color:#a95c5c;background:#3a212b}#status[data-type=ok]{color:#d1ffeb;border-color:#357a6d;background:#11392f}
|
| 13 |
+
.metrics{display:grid;grid-template-columns:repeat(3,1fr);gap:10px;margin-bottom:17px}.metric{border:1px solid #315069;border-radius:12px;background:#0b1c2e;padding:12px}.metric strong{display:block;font-size:25px;line-height:1.2}.metric span{font-size:11px;color:var(--muted)}.question{background:linear-gradient(160deg,#102b42,#112137);border:1px solid #356180;padding:22px;border-radius:15px}.question h2{font-size:20px;margin-bottom:8px}.tip{font-size:12px;color:#b5c9d7;margin-top:10px}.result-banner{display:flex;justify-content:space-between;align-items:center;gap:8px;flex-wrap:wrap;margin-bottom:12px}.result-pill{border-radius:40px;background:#133a40;border:1px solid #326a71;color:#bde8e0;padding:4px 10px;font-size:12px}.advisory{font-size:13px;color:#dfd8ae}.answer-line{line-height:1.65;color:#e4f4fb;border-left:3px solid #3689d1;padding:10px 12px;margin:8px 0 12px;background:rgba(16,49,75,.6);border-radius:0 8px 8px 0}.cite{white-space:nowrap;font-weight:800;color:#8fe6e0}.evidence-card{padding:15px 0;border-top:1px solid #35516b}.evidence-top{display:flex;justify-content:space-between;gap:10px;flex-wrap:wrap;margin-bottom:10px}.evidence-top strong{font-size:14px;color:#e0f7ff;word-break:break-word}.evidence-card p{white-space:pre-wrap;overflow-wrap:anywhere;font-size:14px;color:#cfdfed;margin-bottom:8px}.evidence-card .footnote{font-size:11px;color:#90acbf;margin:0}.score{font-size:12px;color:var(--turq);white-space:nowrap}.warning{margin-top:15px;font-size:12px;color:var(--muted);border-top:1px solid #2c495f;padding-top:16px}footer{margin-top:30px;color:#9fb7ca;font-size:12px}footer a{color:#9cdbfc}code{color:#bdeaff}.privacy{color:#d1e7f1}button.large{font-size:15px;padding:12px 17px}
|
| 14 |
+
@media (max-width:900px){main{padding:25px 14px 60px}.layout{grid-template-columns:1fr}.card{padding:18px}.hero h1{font-size:44px}.hero p{font-size:15px}}@media(max-width:520px){.metrics{grid-template-columns:repeat(3,1fr)}.metric{padding:9px}.metric strong{font-size:19px}}
|
| 15 |
+
</style>
|
| 16 |
+
</head>
|
| 17 |
+
<body>
|
| 18 |
+
<main>
|
| 19 |
+
<header class="hero">
|
| 20 |
+
<div class="eyebrow">RODRIGO PANTOJA · OPEN SOURCE RESEARCH TOOLS</div>
|
| 21 |
+
<h1>📚 ResearchRAG</h1>
|
| 22 |
+
<p>Find evidence across your documents. Ask a question, inspect matched passages, and export source-linked excerpts without sending document text to an AI service.</p>
|
| 23 |
+
<div class="badges"><span>English · Español · Português</span><span>Source-linked excerpts</span><span>Runs in your browser</span><span>Free Static Space</span><span>No generative AI API</span></div>
|
| 24 |
+
</header>
|
| 25 |
+
<div class="layout">
|
| 26 |
+
<div class="column">
|
| 27 |
+
<section class="card" aria-labelledby="library-title">
|
| 28 |
+
<h2 id="library-title">1. Build your document library</h2>
|
| 29 |
+
<p class="subtitle">Load an example set or add your own text. Nothing is transmitted to a server by this application.</p>
|
| 30 |
+
<div class="button-row"><button id="demo" type="button" class="primary">Load 3 example documents</button><button id="reset" type="button">Clear library</button></div>
|
| 31 |
+
<div class="field"><label for="files">Upload text-based files</label><label for="files" id="drop" class="drop"><span>⬆ Drag files here or click to browse</span><small>.txt, .md, .csv · up to 1 MB each</small><input id="files" type="file" accept=".txt,.md,.csv,text/plain,text/markdown,text/csv" multiple></label></div>
|
| 32 |
+
<div class="field"><label for="doc-title">Or paste a document</label><input id="doc-title" type="text" maxlength="130" placeholder="Document title (optional)" autocomplete="off"><textarea id="doc-text" placeholder="Paste research notes, articles or other source text here..." aria-label="Document text"></textarea><button id="add-paste" type="button">Add pasted document</button></div>
|
| 33 |
+
<h3 style="font-size:14px;margin:18px 0 8px">Indexed source documents</h3>
|
| 34 |
+
<div id="documents" aria-live="polite"></div>
|
| 35 |
+
<p class="warning"><strong>Formats:</strong> Paste text or upload TXT, Markdown or CSV. To use PDF/DOCX, first copy or export its selectable text. Scanned PDFs require OCR and are not supported by this static edition.</p>
|
| 36 |
+
</section>
|
| 37 |
+
<section class="card" aria-label="Project methodology">
|
| 38 |
+
<h2>How it works</h2>
|
| 39 |
+
<p class="subtitle" style="margin-bottom:8px">Transparent document retrieval, not an LLM chatbot</p>
|
| 40 |
+
<p class="muted">The app breaks source text into passages, ranks relevant passages using word overlap and character matching, then selects source sentences with document identifiers such as <code>[D1:C2]</code>.</p>
|
| 41 |
+
<p class="muted">It does <strong class="privacy">not</strong> generate new claims, run semantic embeddings, check facts or provide conclusions beyond the uploaded text.</p>
|
| 42 |
+
</section>
|
| 43 |
+
</div>
|
| 44 |
+
<div class="column">
|
| 45 |
+
<section class="card" aria-label="Search documents">
|
| 46 |
+
<div class="metrics" aria-label="Document statistics"><div class="metric"><strong id="metric-docs">0</strong><span>documents</span></div><div class="metric"><strong id="metric-chunks">0</strong><span>source passages</span></div><div class="metric"><strong id="metric-characters">0</strong><span>characters</span></div></div>
|
| 47 |
+
<div class="question"><h2>2. Ask the documents</h2><p class="subtitle">Ask a question that the indexed documents might answer.</p><label class="field" for="question"><span style="font-weight:700">Your research question</span><textarea id="question" placeholder="Example: Why is source verification important in research?"></textarea></label><div class="button-row"><button id="run" type="button" class="primary large" disabled>🔎 Find evidence</button><button id="download" type="button" disabled>↓ Export report (.md)</button></div><p class="tip">Tip: Ctrl/⌘ + Enter runs your question.</p></div>
|
| 48 |
+
<p id="status" role="status" aria-live="polite" style="margin:14px 0 0">Ready.</p>
|
| 49 |
+
</section>
|
| 50 |
+
<section id="results-empty" class="card"><h2>3. Review the findings</h2><p class="muted" style="margin:0">Your cited excerpts will appear here. Start by loading the three example documents, then click “Find evidence”.</p></section>
|
| 51 |
+
<section id="results" class="card" hidden aria-live="polite">
|
| 52 |
+
<div class="result-banner"><h2 style="margin:0">3. Review the findings</h2><span id="result-status" class="result-pill">Evidence found</span></div>
|
| 53 |
+
<p class="subtitle">Found <strong id="hit-count">0</strong> relevant source passages.</p>
|
| 54 |
+
<div id="answer"></div>
|
| 55 |
+
<h3 style="font-size:18px;margin:22px 0 12px">Cited source passages</h3><div id="evidence"></div>
|
| 56 |
+
<p class="warning">A relevance percentage estimates lexical similarity, not credibility, truth or factual certainty. Verify each claim with the original document.</p>
|
| 57 |
+
</section>
|
| 58 |
+
</div>
|
| 59 |
+
</div>
|
| 60 |
+
<footer>ResearchRAG Browser Edition · Rodrigo Pantoja · MIT License · Static Hugging Face Space · Document text is processed locally in this browser tab; do not treat extractive retrieval as verified fact.</footer>
|
| 61 |
+
</main>
|
| 62 |
+
<script src="engine.js"></script>
|
| 63 |
+
<script src="app.js"></script>
|
| 64 |
+
</body>
|
| 65 |
</html>
|