Spaces:
Running
Running
|
Download README.md from thelamapi/Next-Trace: direct link, hf CLI and curl.
- Browser
- Download file 3.48 kB
-
https://huggingface.co/spaces/thelamapi/Next-Trace/resolve/main/README.md
- Command line
-
hf download hf://spaces/thelamapi/Next-Trace/README.md
-
curl -L -o README.md https://huggingface.co/spaces/thelamapi/Next-Trace/resolve/main/README.md
3.48 kB
| title: Next Trace | |
| emoji: 🔎 | |
| colorFrom: green | |
| colorTo: gray | |
| sdk: static | |
| app_file: index.html | |
| pinned: false | |
| license: cc-by-sa-4.0 | |
| short_description: Document answers with highlighted evidence | |
| # Next Trace | |
| Next Trace answers *where* in a document the answer is. Give it a Turkish or English document and a question; it | |
| marks the sentence(s) the answer rests on, or says that **the document does not contain enough evidence**. It is a | |
| small learned reranker (18 language-light features → MLP) plus an answerability head, compared with a BM25 baseline. | |
| **Model:** https://huggingface.co/thelamapi/Next-Trace — paste any document, ask a question, and compare with BM25; it runs entirely in the browser. | |
| ## Results (BM25 + threshold → Next Trace) | |
| | Test set | Questions | Trace accuracy | Evidence@1 on answerable | Correct abstention on unanswerable | Answered (answerable) | | |
| |---|---:|---:|---:|---:|---:| | |
| | SQuAD2-dev (EN, adversarial unanswerables) | 11873 | 0.492 → **0.510** | 0.825 → 0.842 | 0.415 → 0.318 | 0.645 → 0.801 | | |
| | XQuAD-tr (TR, zero-shot) | 2327 | 0.719 → **0.788** | 0.770 → 0.803 | 0.934 → 0.972 | 0.612 → 0.721 | | |
| | XQuAD-en (EN) | 2319 | 0.732 → **0.820** | 0.803 → 0.824 | 0.953 → 0.965 | 0.606 → 0.790 | | |
| *Trace accuracy* counts an answerable question as correct only if the system answers **and** cites the gold sentence first, | |
| and an unanswerable question as correct only if it abstains. On SQuAD 2.0's adversarial unanswerables, the cited sentence | |
| contained the annotators' *plausible but wrong* answer 44.6% (BM25) vs | |
| 51.6% (Next Trace) of the time. | |
| Turkish is **zero-shot**: no Turkish text was used for training or thresholds. | |
|  | |
| ## Method | |
| * Sentence splitting and tokenisation are language-light: Turkish-aware lower-casing, 5-letter stems, TR+EN stop words. | |
| * Features: BM25 (normalised and rank), matched / missing IDF mass, best missing key term, character 4-gram Jaccard, bigram overlap, | |
| question-type cues (when/kaç ↔ digits, who/kim ↔ capitalised names), length, position, neighbour scores, document-level BM25 strength, | |
| share of question terms absent from the whole document, and numbers in the question missing from the sentence (2019 vs 2020). | |
| * Sentence model: MLP 18→32→32→1 (BCE) on SQuAD 2.0 train articles (split A, 126503 questions). | |
| Answerability head: logistic regression on 9 document features (split B). Training splits add unanswerables made by asking questions against another paragraph of the same article. Thresholds: split V (answerable ≥ 0.60; BM25 ≥ 2.79). | |
| * Splits are by article title; XQuAD and SQuAD dev are test-only. | |
| * Browser: line-by-line JavaScript port; Node parity on 9 documents, max abs difference 1.6653345369377348e-16. | |
| ## Usage | |
| ```bash | |
| python inference.py --doc "Kasaba kütüphanesi 1962 yılında açıldı. ..." --question "Kütüphane ne zaman açıldı?" | |
| python inference.py --selftest | |
| ``` | |
| ## Limitations | |
| * Selects evidence sentences; it does not write an answer. | |
| * Lexical features: paraphrases with no word overlap and multi-hop questions are hard; long documents dilute BM25 statistics. | |
| * Turkish performance is measured on translated XQuAD only. | |
| * Failure examples for every test set are in `evaluation.json`. Hardware: NVIDIA L4; profile full. | |
| Data: SQuAD 2.0 (Rajpurkar et al., 2018) and XQuAD (Artetxe et al., 2020), both CC BY-SA 4.0. The Space's examples file contains short excerpts under the same license. | |