samwjya commited on
Commit
e71840e
·
1 Parent(s): 4bbdc15

gitignore

Browse files
Files changed (1) hide show
  1. CLAUDE.md +0 -128
CLAUDE.md DELETED
@@ -1,128 +0,0 @@
1
- # PaperRank
2
-
3
- An academic paper reranking system built for the Forward Data Lab research task at UIUC.
4
- Adapts the Multi-Reranker approach from the ACM-ICAIF '24 paper
5
- "Multi-Reranker: Maximizing performance of retrieval-augmented generation in the FinanceRAG challenge"
6
- and applies it to academic literature search using the Semantic Scholar API.
7
-
8
- ## Project Goal
9
-
10
- Compare BM25 baseline retrieval vs. multi-model cross-encoder reranking on academic papers.
11
- The system fetches live papers from Semantic Scholar, ranks them with BM25, then reranks
12
- using cross-encoder models inspired by the FinanceRAG paper. Evaluate using NDCG@10 and MRR.
13
-
14
- ## Stack
15
-
16
- - **Language**: Python 3.10+
17
- - **Data Source**: Semantic Scholar API (free, no key required)
18
- - **Baseline Retrieval**: rank_bm25
19
- - **Reranking**: sentence-transformers CrossEncoder (local, no API key)
20
- - Primary model: BAAI/bge-reranker-v2-m3
21
- - **Evaluation**: NDCG@10, MRR (manual relevance judgments)
22
- - **CLI**: argparse
23
- ## Project Structure
24
-
25
- ```
26
- paperrank/
27
- ├── CLAUDE.md # This file
28
- ├── main.py # CLI entry point
29
- ├── retriever.py # Semantic Scholar API fetching + BM25 ranking
30
- ├── reranker.py # CrossEncoder reranking logic (adapted from FinanceRAG)
31
- ├── evaluate.py # NDCG@10 and MRR metrics + side-by-side comparison
32
- ├── requirements.txt # Dependencies
33
- └── README.md # Project documentation
34
- ```
35
-
36
- ## Key Files
37
-
38
- - `retriever.py` — fetches top 50 candidate papers from Semantic Scholar API, ranks with BM25
39
- - `reranker.py` — scores each (query, title + abstract) pair with CrossEncoder, returns reranked list
40
- - `evaluate.py` — computes NDCG@10 and MRR, prints side-by-side BM25 vs reranked comparison table
41
- - `main.py` — CLI entry: takes a query string, runs full pipeline, pretty prints results
42
- ## Architecture
43
-
44
- ```
45
- User Query
46
- │
47
- ▼
48
- Semantic Scholar API (top 50 candidates)
49
- │
50
- ▼
51
- BM25 Baseline Ranking
52
- │
53
- ▼
54
- CrossEncoder Reranker (BAAI/bge-reranker-v2-m3, runs locally)
55
- │
56
- ▼
57
- Side-by-side Comparison (BM25 rank vs Reranked rank, with scores)
58
- │
59
- ▼
60
- NDCG@10 / MRR Evaluation
61
- ```
62
-
63
- ## Implementation Notes
64
-
65
- - All models run **locally** — no OpenAI, Anthropic, or any paid API calls
66
- - Reranker input: concatenate paper title + abstract as the document text
67
- - BM25 tokenization: lowercase + whitespace split on title + abstract
68
- - Fetch 50 candidates from Semantic Scholar, rerank to top 10 for evaluation
69
- - Always show both BM25 and reranked results side-by-side for demo clarity
70
- - Use `pandas` DataFrame for pretty printing comparison tables
71
- ## Reference Paper
72
-
73
- - **Paper**: Multi-Reranker: Maximizing performance of RAG in the FinanceRAG challenge (ACM-ICAIF '24)
74
- - **Authors**: Joohyun Lee, Minji Roh
75
- - **ArXiv**: https://arxiv.org/abs/2411.16732
76
- - **Original repo**: https://github.com/cv-lee/FinanceRAG
77
- - **Key adaptation**: We replace their static finance corpus with live Semantic Scholar API ingestion,
78
- and apply their multi-model reranking strategy to academic literature search instead of finance docs.
79
- ## Demo Queries (for testing + report)
80
-
81
- 1. "reducing cloud infrastructure costs using Kubernetes autoscaling"
82
- 2. "large language models for academic paper retrieval"
83
- 3. "graph neural networks for fraud detection"
84
- 4. "transformer models for information retrieval reranking"
85
- ## Claude Code Instructions
86
-
87
- After writing each file, you MUST:
88
- 1. Print a plain English explanation of what the file does and why
89
- 2. Explain each function — what it takes as input, what it does, what it returns
90
- 3. Explain any non-obvious technical choices (e.g. why BM25, why concatenate title+abstract)
91
- 4. Show the exact command to run that file or test it
92
- 5. Show expected output so I know it's working correctly
93
- Format each explanation like this:
94
-
95
- ---
96
- ### What this file does
97
- ...
98
-
99
- ### Function breakdown
100
- - `function_name(args)` — what it does in plain English
101
- ### Why we made these choices
102
- ...
103
-
104
- ### How to run it
105
- ```bash
106
- python filename.py --arg value
107
- ```
108
-
109
- ### Expected output
110
- ...
111
- ---
112
-
113
- Do this for every file you create, no exceptions.
114
-
115
- ## Dependencies
116
-
117
- ```
118
- requests
119
- rank_bm25
120
- sentence-transformers
121
- pandas
122
- torch
123
- ```
124
-
125
- Install with:
126
- ```bash
127
- pip install requests rank_bm25 sentence-transformers pandas torch
128
- ```