# Architecture Deep Dive — GraphRAG Inference System

## System Overview

```
┌────────────────────────────────────────────────────────────────────────────┐
│                         GraphRAG Inference System                          │
│                                                                            │
│  ┌──────────────────────────────────────────────────────────────────────┐  │
│  │                    React Dashboard (Port 5173)                       │  │
│  │  ┌────────────────┐  ┌────────────────┐  ┌────────────────────────┐ │  │
│  │  │  Query Input   │  │ Token BarChart  │  │   Accuracy RadarChart  │ │  │
│  │  │  Sample Q's    │  │ (Recharts)      │  │   (LLM-as-Judge)       │ │  │
│  │  └───────┬────────┘  └────────────────┘  └────────────────────────┘ │  │
│  │          │ POST /query/compare                                        │  │
│  └──────────┼───────────────────────────────────────────────────────────┘  │
│             │                                                              │
│  ┌──────────▼───────────────────────────────────────────────────────────┐  │
│  │                    FastAPI Server (Port 8000)                         │  │
│  │                                                                      │  │
│  │  ┌──────────────┐  ┌──────────────────┐  ┌──────────────────────┐  │  │
│  │  │  Pipeline 1  │  │   Pipeline 2      │  │     Pipeline 3       │  │  │
│  │  │  LLM Only    │  │   Basic RAG       │  │     GraphRAG         │  │  │
│  │  │              │  │                   │  │                      │  │  │
│  │  │  • sys prompt│  │  • embed query    │  │  • extract entities  │  │  │
│  │  │  • question  │  │  • FAISS search   │  │  • TG traversal      │  │  │
│  │  │  • LLM call  │  │  • top-K chunks   │  │  • serialize graph   │  │  │
│  │  │              │  │  • LLM call       │  │  • LLM call          │  │  │
│  │  └──────┬───────┘  └───────┬──────────┘  └──────────┬───────────┘  │  │
│  │         │                  │                          │              │  │
│  │         └──────────────────┴──────────────────────────┘              │  │
│  │                            │                                          │  │
│  │                   ┌────────▼────────┐                                │  │
│  │                   │  LLM-as-Judge   │                                │  │
│  │                   │  + Metrics Agg  │                                │  │
│  │                   └────────┬────────┘                                │  │
│  └────────────────────────────┼────────────────────────────────────────┘  │
│                               │                                            │
└───────────────────────────────┼────────────────────────────────────────────┘
                                │
              ┌─────────────────┴─────────────────┐
              │                                   │
   ┌──────────▼──────────┐             ┌──────────▼──────────┐
   │   FAISS Vector DB   │             │   TigerGraph Savanna │
   │   (in-memory/pkl)   │             │   Knowledge Graph    │
   │                     │             │                      │
   │  chunks[]           │             │  Entity vertices     │
   │  embeddings[]       │             │  RELATED_TO edges    │
   │  IndexFlatIP        │             │  MENTIONED_IN edges  │
   └─────────────────────┘             └──────────────────────┘
                                                │
                                     ┌──────────▼──────────┐
                                     │   OpenAI API         │
                                     │   (LLM + Embeddings) │
                                     └─────────────────────┘
```

---

## Component Details

### 1. Ingestion Pipeline (`backend/graph/ingestion.py`)

Converts raw documents into the knowledge graph:

```
Raw Docs (.txt/.md)
    │
    ▼  chunk_text() — token-aware, 512 tokens + 64 overlap
Document Chunks
    │
    ▼  extract_entities_and_relations() — GPT-4o-mini JSON extraction
Entities + Relationships
    │
    ├─▶ get_embeddings_batch() — text-embedding-3-small
    │       │
    │       ▼
    │   Entity embeddings (for future vector-on-graph hybrid)
    │
    ├─▶ tg.upsert_entity() — Entity vertex in TigerGraph
    ├─▶ tg.upsert_document() — Document vertex in TigerGraph
    ├─▶ tg.upsert_relationship() — RELATED_TO edge
    └─▶ tg.link_entity_to_document() — MENTIONED_IN edge
```

Simultaneously, all raw chunks are added to FAISS for Pipeline 2.

### 2. Basic RAG Pipeline (`backend/rag/basic_rag.py`)

```
User Query
    │
    ▼  openai.embeddings.create() → 1536-dim vector
Query Vector
    │
    ▼  faiss.IndexFlatIP.search() — cosine similarity
Top-K Chunk Indices (K=5)
    │
    ▼  Concatenate chunk texts
Context String (~3,000 tokens)
    │
    ▼  openai.chat.completions.create()
Answer + Token Usage
```

Token overhead: system prompt (~50) + context (~3,000) + question (~50) = ~3,100 prompt tokens

### 3. GraphRAG Pipeline (`backend/rag/graph_rag.py`)

```
User Query
    │
    ▼  extract_query_entities() — LLM → ["Entity A", "Entity B"]
Seed Entities
    │
    ▼  tg.get_entity_subgraph() — GSQL multi-hop traversal
Subgraph {entities[], relationships[], documents[]}
    │
    ▼  serialize_subgraph() — compact structured text
Graph Context (~400-800 tokens)
    │
    ▼  openai.chat.completions.create()
Answer + Token Usage
```

Token overhead: system prompt (~60) + graph context (~600) + question (~50) = ~710 prompt tokens

**Token reduction: ~77%**

### 4. LLM-as-a-Judge (`backend/llm/judge.py`)

```
Question + Answer + Ground Truth
    │
    ▼  GPT-4o-mini with JSON response format
{
  "score": 8,
  "correctness": 9,
  "completeness": 7,
  "clarity": 8,
  "relevance": 8,
  "reasoning": "..."
}
```

Run for both Basic RAG and GraphRAG answers. Used for per-query comparison and aggregate pass rate.

### 5. BERTScore Evaluation (`evaluation/benchmark.py`)

```python
from bert_score import score as bert_score_fn
P, R, F1 = bert_score_fn(
    predictions=[graph_answers],
    references=[ground_truths],
    lang="en",
    model_type="distilbert-base-uncased"
)
```

Target: F1 rescaled ≥ 0.55 (raw ≥ 0.88) for bonus points.

---

## Knowledge Graph Schema

### Vertices

| Type | Key Attributes | Purpose |
|------|---------------|---------|
| Entity | name, entity_type, description, embedding | Core knowledge nodes |
| Document | title, content, chunk_index, token_count | Source text chunks |
| Concept | name, category, importance_score | High-level topics |

### Edges

| Type | Direction | Attributes | Purpose |
|------|-----------|-----------|---------|
| RELATED_TO | Entity ↔ Entity | relation_type, confidence, context | Core relationships |
| MENTIONED_IN | Entity → Document | frequency, relevance_score | Entity-source links |
| HAS_CONCEPT | Document → Concept | weight | Topic tagging |
| CO_OCCURS_WITH | Entity ↔ Entity | co_occurrence_count | Statistical co-occurrence |

---

## Subgraph Serialization Format

The key to token efficiency is our compact serialization format:

```
## ENTITIES
• [Entity Name] [TYPE]: description text

## RELATIONSHIPS
• Entity A —[RELATION_TYPE]→ Entity B (conf: 0.95) | context snippet

## SUPPORTING TEXT (top-1 doc, max 300 chars)
[Document Title]: first 300 characters of relevant chunk...
```

This format is:
- Hierarchically structured (LLM can parse with zero instruction)
- Non-redundant (each fact stated once)
- Confidence-weighted (LLM can trust high-confidence facts more)
- Bounded (caps prevent runaway token usage)

---

## API Endpoints

| Method | Path | Description |
|--------|------|-------------|
| GET | /health | Server health + TG connection status |
| POST | /query/compare | Run all 3 pipelines, return full metrics |
| GET | /stats/session | Cumulative session token savings |
| GET | /graph/stats | TigerGraph vertex/edge counts |
| POST | /ingest/text | Quick-ingest a document |

---

## Performance Characteristics

| Operation | Typical Latency |
|-----------|----------------|
| Query embedding (OpenAI) | 150–300ms |
| FAISS search (100K chunks) | <10ms |
| Entity extraction (LLM) | 300–600ms |
| TigerGraph GSQL traversal | 50–200ms |
| LLM generation (gpt-4o-mini) | 800–2000ms |
| LLM Judge evaluation | 300–600ms |
| **Total: Basic RAG** | **~1,500–2,500ms** |
| **Total: GraphRAG** | **~1,000–1,800ms** |

GraphRAG's latency advantage comes from:
1. No FAISS search needed (graph traversal replaces it)
2. Smaller context = faster LLM generation
3. No chunk deduplication step needed

---

## Scaling Considerations (Round 2)

For 50-100M token datasets:
- TigerGraph Savanna handles billions of edges natively — no schema changes needed
- FAISS upgrades to `IndexIVFFlat` or `HNSW` for ANN search at scale
- Ingestion parallelized with async batch processing
- Entity extraction cached — same entities aren't re-extracted across document runs
- Community detection (TigerGraph built-in) adds global context layer for broad queries