Spaces:
Sleeping
Sleeping
File size: 19,701 Bytes
81e0253 fc5156b effcd1e fc5156b b2f4ef1 fc5156b b2f4ef1 fc5156b b2f4ef1 fc5156b b2f4ef1 fc5156b fc5075d fc5156b 49eecdc fc5156b 49eecdc fc5156b fc5075d eb7f266 fc5075d eb7f266 fc5075d fc5156b b2f4ef1 fc5156b 375d687 fc5156b 375d687 fc5156b b2f4ef1 fc5156b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 | ---
title: FinanceFlow AI
emoji: π
colorFrom: blue
colorTo: indigo
sdk: docker
app_port: 7860
pinned: false
license: mit
---
<!-- ^ The YAML block above configures Hugging Face Spaces (Docker deploy). It must
stay at the very top for HF to read it; GitHub just renders it as a small block.
Remove it if you don't deploy to HF Spaces. -->
# FinanceFlow AI



> Upload business finance documents (invoices, contracts, payment notes), ask
> questions in plain English, and get **source-backed answers with citations** β
> or a clear "Not enough information" when the document doesn't support an answer.
> **30-second tour:** start the app β **Dashboard β Load sample documents** β
> **Ask β click a sample question**. You'll see a grounded answer with a
> highlighted citation, and an unanswerable question correctly refused.
FinanceFlow AI is a compact, production-shaped **RAG (Retrieval-Augmented
Generation)** MVP built for fintech / AI-workflow / B2B-SaaS portfolios. It runs
**fully offline with no paid API key** using an extractive answer engine, and can
optionally call OpenAI or Anthropic for nicer phrasing.
---
## Problem statement
Finance and operations teams drown in documents: invoices, vendor contracts,
dunning notes, payment terms. Answering a simple question ("What's the total due?
When is it overdue? What's the contract's payment term?") means hunting through
PDFs by hand. Generic chatbots *hallucinate* numbers and dates β unacceptable in
finance, where a wrong figure has real consequences.
## Target users
- **Finance / AP / AR analysts** who answer recurring questions from invoices,
contracts, and dunning notes.
- **Operations & RevOps teams** triaging vendor terms, due dates, and late fees.
- **Product / AI teams** that need a grounded, auditable Q&A pattern they can
trust in front of customers.
## Why this project matters
- **Document automation** is one of the highest-value, most concrete uses of AI
in fintech and B2B SaaS.
- It demonstrates **trustworthy AI**: every answer is grounded in retrieved
evidence, and the system **abstains** instead of guessing.
- It mirrors real **AI product-operations** concerns: retrieval quality,
citations, evaluation metrics, and graceful degradation without a paid model.
## Documentation
- [CASE_STUDY.md](CASE_STUDY.md) β product case study (problem β results)
- [ARCHITECTURE.md](ARCHITECTURE.md) β system design + diagrams
- [INTERVIEW_NOTES.md](INTERVIEW_NOTES.md) β direct answers to common questions
- [DEMO_SCRIPT.md](DEMO_SCRIPT.md) β 60-second demo walkthrough
---
## Features
- π€ **Upload** PDF or TXT documents (validated, size-limited).
- π§© **Automatic parsing + chunking** with page numbers preserved for PDFs.
- π **Retrieval** via a TF-IDF-style keyword/semantic-like ranker (no model
required).
- π¬ **Ask questions** and get an answer built **only** from retrieved context.
- π **Citations** on every answer: document name, page/chunk, source excerpt,
and relevance score.
- π **Anti-hallucination**: abstains ("Not enough information in the uploaded
document.") when evidence is weak; never invents numbers/dates/names.
- π **Dashboard**: document list, processed/failed status, chunk counts,
Q&A history, delete, and a one-click **"Load sample documents"** demo mode.
- β
**Evaluation**: a **golden dataset** scored in the UI *and* from the CLI,
reporting citation coverage, correct abstentions, and evidence-match rate.
- πͺ΅ **Logging** across ingestion, retrieval, ask, abstention, and evaluation,
plus a global exception handler so a bad request never crashes the server.
- π§ͺ **Tested**: 20 backend tests (pytest + FastAPI TestClient) and **CI** via
GitHub Actions.
- π³ **Dockerized**: `docker compose up --build` runs backend + frontend.
- π€ **Optional LLM mode** (OpenAI/Anthropic) behind env vars, with automatic
fallback to extractive mode.
---
## Tech stack
| Layer | Choice |
| ------------ | --------------------------------------- |
| Backend | Python, FastAPI, Uvicorn |
| Database | SQLite via SQLAlchemy |
| Parsing | `pypdf` (PDF), built-in reader (TXT) |
| Retrieval | Custom TF-IDF keyword ranker (no ML dep)|
| Frontend | React + Vite |
| Styling | Plain modern CSS |
| LLM (opt-in) | OpenAI or Anthropic via env vars |
---
## Architecture overview
```
ββββββββββββββββββββββββββββ
Browser ββββΊ β React + Vite frontend β
β Dashboard / Ask / Eval β
βββββββββββββββ¬ββββββββββββββ
/api proxy β REST/JSON
βββββββββββββββΌββββββββββββββ
β FastAPI backend β
β upload ask history ... β
βββββ¬ββββββββββββ¬βββββββββββββ
parse+chunk β β retrieve + answer
βββββΌββββ βββββΌβββββββββββββββ
β pypdf β β TF-IDF ranker + β
β /txt β β extractive/LLM β
βββββ¬ββββ βββββ¬βββββββββββββββ
β β
βββββΌββββββββββββΌββββ
β SQLite (SQLAlchemy)
β documents / chunks / qa_history
ββββββββββββββββββββββ
```
**Backend modules** (`backend/app/`):
- `config.py` β env-driven settings, with safe defaults.
- `database.py` / `models.py` β SQLite + SQLAlchemy models
(`Document`, `Chunk`, `QA`).
- `parsing.py` β extract text per page (PDF) or whole file (TXT).
- `chunking.py` β overlapping word-window chunking.
- `retrieval.py` β TF-IDF keyword ranking with stopwords + length normalization.
- `answering.py` β abstention logic + extractive answer + citation builder.
- `llm.py` β optional OpenAI/Anthropic synthesis (graceful fallback).
- `ingest.py` β shared parseβchunkβstore pipeline.
- `evaluation.py` β the 5-question evaluation harness.
- `main.py` β FastAPI app and all endpoints.
---
## How RAG / retrieval works
1. **Ingest**: extracted text is normalized and split into overlapping
~120-word chunks (25-word overlap) so a fact on a boundary stays intact.
Each chunk stores `document_id`, `chunk_index`, `page`, and `text`.
2. **Index at query time**: for the selected document, we compute an **IDF**
weight per term across its chunks (rarer words matter more).
3. **Score**: each chunk is scored as `Ξ£ tf(term)Β·idf(term)` over the question's
terms, normalized by `β(chunk length)` so long chunks don't dominate.
4. **Select**: the top-K chunks become the context; below-threshold scores are
dropped.
5. **Answer**: in extractive mode we return the densest verbatim word-window
from the best chunk (re-centered on the matched terms so the value next to a
label like "Total Amount Due $2,413.98" is kept); in LLM mode we pass the
same context to the model with a strict, grounding-only prompt.
## How hallucination is reduced
- **Two abstention gates** that return *"Not enough information in the uploaded
document."* and show no answer:
1. **Score floor** β the best chunk must clear `SCORE_THRESHOLD`.
2. **Term coverage** β the evidence must share at least two distinct content
words with the question, so a single incidental overlap (e.g. asking for a
"social security number" and matching only the word "number") can't produce
a confident answer.
- **Extractive by default**: answers are copied verbatim from the document β the
engine literally cannot invent a number that isn't there.
- **Grounded LLM prompt** (when enabled): system prompt forbids inventing facts
and requires the exact abstain string when context is insufficient;
temperature is 0.
- **Evidence always shown**: every answer displays its citations so a human can
verify instantly.
- **Evaluation metrics** track `unsupported_answer_count` to catch regressions.
> **Privacy note:** raw uploads are parsed into chunks and then discarded β the
> app stores extracted text, never the original file. There are no orphaned
> documents on disk to manage or leak.
---
## Project structure
```
financeflow-ai/
ββ backend/
β ββ app/
β β ββ main.py # FastAPI app + endpoints (tagged Swagger docs)
β β ββ config.py # env-driven settings, safe defaults
β β ββ database.py # SQLAlchemy engine + SQLite FK pragma
β β ββ models.py # Document, Chunk, QA tables
β β ββ schemas.py # Pydantic request/response models
β β ββ parsing.py # PDF (pypdf) + TXT extraction
β β ββ chunking.py # overlapping word-window chunking
β β ββ retrieval.py # TF-IDF keyword ranker (no ML dep)
β β ββ answering.py # abstention gates + focused extractive answer
β β ββ llm.py # optional OpenAI/Anthropic synthesis
β β ββ ingest.py # shared parseβchunkβstore pipeline
β β ββ evaluation.py # golden-dataset eval + metrics
β β ββ eval_cli.py # `python -m app.eval_cli`
β β ββ golden_dataset.json# questions + expected evidence/abstentions
β β ββ logging_config.py # one-line logging setup
β ββ tests/ # pytest unit + API tests (20 cases)
β ββ Dockerfile
β ββ requirements.txt / requirements-dev.txt
β ββ .env.example
ββ frontend/ # React + Vite (Dashboard / Ask / Evaluation)
β ββ src/components/ # Dashboard, AskPanel, Evaluation, Citations
β ββ Dockerfile + nginx.conf
β ββ vite.config.js
ββ sample_docs/ # fictional invoice, contract, payment note
ββ docker-compose.yml
ββ Makefile
ββ .github/workflows/ci.yml
```
---
## Local setup
### Prerequisites
- Python 3.10+ (3.12 recommended)
- Node.js 18+
### Option A β Docker (one command)
```bash
docker compose up --build
```
Open **http://localhost:5173**. The frontend (nginx) proxies `/api` to the
backend container. Stop with `docker compose down`.
### Option B β Run locally
**1) Backend**
```bash
cd backend
python -m venv .venv
# Windows (PowerShell): .venv\Scripts\Activate.ps1
# macOS/Linux: source .venv/bin/activate
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8000
```
Backend runs at **http://127.0.0.1:8000** (interactive Swagger docs at `/docs`).
> Optional: copy `.env.example` to `.env` and set `ANSWER_MODE=llm` plus a key
> to enable LLM answers. Leave it as-is to run fully offline.
**2) Frontend** (second terminal)
```bash
cd frontend
npm install
npm run dev
```
Frontend runs at **http://localhost:5173** and proxies `/api` to the backend.
Then open the app and click **Try Demo** on the Overview screen β it loads the
sample documents and answers a question with citations.
### Make shortcuts
```bash
make install # install backend (dev) + frontend deps
make backend # run the API
make frontend # run the UI
make test # run the backend tests
make eval # run the anti-hallucination evaluation (CLI)
make reset-db # drop + recreate the local database (clean slate)
make docker-up # build + run everything in Docker
```
> No `make`? Run the underlying command directly, e.g. reset the DB with
> `cd backend && python -m app.reset_db`.
---
## How to run locally (quick reference)
| What | Command | URL |
| ---- | ------- | --- |
| Backend | `cd backend && uvicorn app.main:app --reload --port 8000` | http://127.0.0.1:8000 Β· docs at `/docs` |
| Frontend | `cd frontend && npm run dev` | http://localhost:5173 |
| Tests | `cd backend && pytest` | β |
| Evaluation (CLI) | `cd backend && python -m app.eval_cli` | β |
The app you open is **http://localhost:5173**. The backend must be running for it
to work.
---
## How to upload to GitHub
First time (the repo is already initialized with commits):
```bash
# 1. Create an EMPTY repo on github.com (no README/license) named e.g. financeflow-ai
# 2. From the project root, point your local repo at it and push:
git remote add origin https://github.com/aissugur/financeflow-ai.git
git branch -M main
git push -u origin main
```
After that, day-to-day:
```bash
git add -A
git commit -m "Describe what changed"
git push
```
> If `git remote add origin` says it already exists, update it instead:
> `git remote set-url origin https://github.com/aissugur/financeflow-ai.git`
**Files that must NOT be pushed** (already handled by `.gitignore` β never force-add them):
- `backend/.env` β your real API keys
- `backend/data/` and any `*.db` / `*.sqlite` β the local database
- `backend/.venv/`, `venv/` β Python virtual environments
- `frontend/node_modules/`, `frontend/dist/` β installed deps & build output
- `__pycache__/`, `.pytest_cache/`, `*.log` β caches and logs
- `.claude/` β local tooling settings
- `ΠΎΡΠΈΠ³.prj` / `ΠΎΡΠΈΠ³.sav` / `ΠΎΡΠΈΠ³.xml` β unrelated files that happen to be in this folder
Only `.env.example` (with empty key fields) is committed β never the real `.env`.
---
## API documentation summary
| Method | Path | Description |
| ------ | ------------------------- | ---------------------------------------- |
| GET | `/health` | Health + current answer mode |
| POST | `/documents/upload` | Upload PDF/TXT (multipart `file`) |
| GET | `/documents` | List all documents |
| GET | `/documents/{id}` | Get one document's metadata |
| DELETE | `/documents/{id}` | Delete a document (+ chunks & history) |
| POST | `/ask` | `{document_id, question}` β answer+cites |
| GET | `/history?document_id=` | Past Q&A (optionally filtered) |
| POST | `/evaluate` | Run the golden-dataset evaluation |
| POST | `/demo/seed` | Load the bundled sample documents |
Full interactive docs (with request/response schemas) are auto-generated by
FastAPI at **`/docs`**.
Example:
```bash
curl -X POST http://127.0.0.1:8000/ask \
-H "Content-Type: application/json" \
-d '{"document_id":1,"question":"What is the total amount due?"}'
```
---
## Testing & evaluation
**Unit + API tests** (pytest + FastAPI TestClient, no running server needed):
```bash
cd backend
pip install -r requirements-dev.txt
pytest # 20 tests: chunking, retrieval, abstention, API, eval
```
Coverage includes bad file types, empty files, missing documents, weak context,
unsupported questions, the full uploadβaskβdelete flow, and the evaluation suite.
CI runs these on every push (`.github/workflows/ci.yml`).
**Anti-hallucination evaluation** against the golden dataset
(`backend/app/golden_dataset.json` β questions paired with expected evidence and
expected abstentions). Run it from the UI (**Evaluation β Run evaluation**) or the
CLI:
```bash
cd backend
python -m app.eval_cli
```
Reported metrics:
| Metric | Meaning |
| ------ | ------- |
| `total_questions` | size of the golden set |
| `answered_with_citation` | answerable questions answered *with* a citation |
| `correct_abstentions` | unanswerable questions correctly refused |
| `unsupported_answer_count` | answers without evidence (should be **0**) |
| `citation_coverage` | cited answers / answerable questions |
| `evidence_match_rate` | expected evidence actually surfaced in the citation |
Current result: **7/7 passed, citation_coverage 1.0, evidence_match_rate 1.0,
0 unsupported answers.**
> A dependency-free `backend/e2e_test.py` also exercises the live HTTP API end to
> end if you prefer a black-box smoke test.
---
## Screenshots
> Premium dark-first UI. Create a `docs/` folder and drop in real screenshots
> before sharing, then the links below will render.
- `docs/screenshot-overview.png` β landing/overview: hero pitch, trust badges, documents
- `docs/screenshot-ask.png` β Ask workspace: answer card + right-hand evidence panel
- `docs/screenshot-evaluation.png` β evaluation metric cards + pass/fail table
---
## Demo script (1-minute Loom)
1. **(0:00β0:10)** "This is FinanceFlow AI β it answers questions about finance
documents with citations, and refuses to guess." Show the dashboard.
2. **(0:10β0:20)** Click **Load sample documents** β three docs appear as
*processed* with chunk counts.
3. **(0:20β0:40)** Go to **Ask**, click the *"What is the total amount due?"*
chip β show the answer **and the highlighted citation** underneath.
4. **(0:40β0:50)** Ask *"What is the customer's social security number?"* β show
it **abstains** with "Not enough informationβ¦". This is the anti-hallucination
story.
5. **(0:50β1:00)** Open **Evaluation**, click **Run evaluation**, show
citation_coverage 100%, correct abstentions, and 0 unsupported answers. Done.
---
## What this project demonstrates (for reviewers)
| Skill area | Where to look |
| ---------- | ------------- |
| **Backend / API design** | FastAPI with validation, tagged Swagger docs, typed Pydantic I/O, SQLAlchemy models, a global exception handler β `backend/app/main.py` |
| **AI / RAG engineering** | Chunking, TF-IDF retrieval, focused extractive answers, optional LLM layer β `retrieval.py`, `answering.py` |
| **Trustworthy AI / AI product ops** | Two abstention gates, citation-backed answers, and a **golden-dataset evaluation** with coverage/abstention/evidence metrics β `answering.py`, `evaluation.py` |
| **Testing & CI** | 20 pytest cases (unit + API) + GitHub Actions β `backend/tests/`, `.github/workflows/ci.yml` |
| **DevOps / DX** | Dockerfiles, `docker-compose.yml`, `Makefile`, `.env.example`, logging |
| **Product / UX** | Clean B2B dashboard, demo mode, sample-question chips, empty/loading/error/success states β `frontend/src/` |
**Role relevance:** the evaluation harness and abstention behaviour map directly
to **AI Product / Product Operations** work (defining quality, measuring grounded
vs. refused answers); the retrieval + API + tests map to **Backend/ML**; the
uploadβretrieveβanswerβcite loop is the core of **AI workflow / document
automation**.
---
## Future improvements
- Swap the keyword ranker for vector embeddings (e.g. `sentence-transformers`)
with a FAISS/SQLite-vec index for semantic recall.
- Cross-document Q&A and multi-document collections.
- Highlight the exact answer span inside the cited excerpt.
- User accounts + per-tenant isolation for real B2B SaaS.
- Background processing queue for large PDFs; OCR for scanned documents.
- Confidence calibration and a human-in-the-loop review queue.
---
## License
MIT β sample documents are fictional and safe to publish.
|