Blame for docs/DESIGN.md

# Cortex: Why FalkorDB for AI Engagements, and More (Design v0.2)

> Status: **DRAFT v0.3 for review** · 2026-09-30 · Name: **Cortex** (confirmed)
> v0.2: Neo4j comparison dropped, PDF ingestion added, free LLM by default, PDF upload feature added.
> v0.3: role impersonation, tokenomics, PIN-protected keys, a consolidated "Limits & Restrictions" section, 4h idle reset, branding.

## 1. Goal
Show, with evidence people can test themselves, why **FalkorDB** is the right foundation for AI
engagements, and what it adds beyond RAG.

- **Main comparison:** **GraphRAG on FalkorDB** vs **plain vector RAG**, the approach our environment already uses.
  Both run on the same PDFs, with the same questions and the same LLM. The only thing that differs is the graph.
- **"And more":** the same graph also supports things RAG can't do: fraud-ring detection, lineage and versioning,
  role-based guardrails, and many isolated graphs in one instance.
- **The data:** "Cortex Bank", a synthetic bank founded in 1926, with 12 departments.
- **Audience:** executives (the story and the impact) and engineers (how it works and whether it reproduces).
- **Access:** a public URL that people use on their own. No presenter or laptop needs to be available.
- **Format:** hybrid, a live walkthrough plus a recorded backup video. The video will be made once the build is nearly finished.
- **Timeline:** 2 days.

**Out of scope for now:** the Neo4j comparison (it can go in a later write-up if needed) and hosting on AWS (see §13).

## 2. Principles
1. **Measure, don't claim.** Every advantage shown comes with a result people can reproduce.
2. **Fair comparison.** Same PDFs, chunks, embeddings, LLM and prompts. Only the retrieval strategy differs.
3. **Honest.** Where plain RAG is just as good (for example, simple lookups of a single fact), we show that.
4. **Free to run.** A free LLM by default, free hosting, and local embeddings.
5. **Reproducible.** `docker compose up` on any laptop gives the same results.

## 3. How Content Gets In (hybrid)

| Layer | Source | How it gets in | Size |
|---|---|---|---|
| **Knowledge**: policies, SOPs, guardrails, KPIs, audits, regulations | **~60 synthetic PDFs** (written by Claude, rendered as realistic bank documents) | **LangChain pipeline**: load → chunk → embed, and have the LLM extract entities and relationships → FalkorDB | ~10–30k nodes |
| **Operations**: customers, accounts, transactions, devices, fraud rings | Structured data (banks keep this in databases, not PDFs) | Seeded Python generator → bulk load | ~1M nodes / ~5M relationships |
| **User uploads** | Visitor's own PDF (≤5 MB, ≤20 pages) | Same pipeline, into a **private sandbox graph** for that session that is deleted after 24h | Small |

```
PDF ─► LangChain loader ─► chunks ─┬─► embeddings (local model) ─► FalkorDB vector index
                                   └─► LLM entity/relation extraction (schema-guided) ─► FalkorDB graph
                                        (chunks linked to the entities they mention = evidence)
```

- **Extraction is guided by a schema.** The LLM may only produce the node and relationship types allowed in §4,
  and its output is validated before loading.
- **The knowledge layer is pre-built.** The 60 PDFs are ingested once at build time and the graph snapshot is
  bundled into the image, so start-up is fast and doesn't use any LLM quota. The pipeline can be re-run at any time.

## 4. Graph Model

**Knowledge layer**
| Node | Examples / key properties |
|---|---|
| `Document`, `Chunk` | Source PDF, page, text, embedding (the evidence) |
| `Department` | Retail, Corporate & Commercial, Wealth, Treasury & Markets, Credit, Risk, Compliance (AML/KYC), Operations, IT & Cyber, HR, Legal, Internal Audit |
| `Role`, `Person` | Title, clearance level |
| `SOP`, `Policy`, `Control` | Version, effective dates, owner |
| `Risk`, `Regulation` | Basel III/IV, AML/KYC, GDPR, DORA, PCI-DSS, SOX, BCBS 239, EU AI Act |
| `Guardrail` | Separation of duties, maker-checker, AI-usage limits, data-access rules |
| `KPI`, `KRI` | Targets, thresholds, current values |
| `System`, `AuditFinding`, `Entity` (M&A history) | Legacy mainframe, acquired banks |

Relationships: `OWNS`, `IMPLEMENTS`, `MITIGATES`, `ENFORCES`, `SUPERSEDES`, `APPLIES_TO`, `CAN_ACCESS`,
`MEASURES`, `RAISED_ON`, `MERGED_INTO`, `MENTIONS` (Chunk→entity).

**Operations layer:** `Customer`, `Account`, `Transaction`, `Device`, `IP`, `Address`, `Branch`, `Merchant`,
with fraud rings planted in the data and labelled with the right answers.

## 5. Synthetic Content (generated by Claude)
- **~60 PDFs** across the 12 departments: policies, SOPs, guardrail standards, KPI frameworks, audit reports and
  regulatory summaries. They look realistic: letterhead, document IDs, version history tables, numbered
  sections and approval sign-offs.
- **Deliberate cross-references** between documents (for example, an SOP cites a policy, which cites a
  regulation). This is what makes questions that need several hops possible.
- **Some legacy versions** (such as a 1998 SOP replaced in 2019), so versioning and lineage can be shown.
- Source text is kept as Markdown in the repo, and the PDFs are rendered from it deterministically.

## 6. LLM & Embeddings
| Use | Default (free) | Optional |
|---|---|---|
| Q&A and upload extraction | **Google Gemini free tier** (a Flash-class model), using the owner's key | Claude Sonnet 5.5 or Gemini, using the **tester's own key** |
| Embeddings | Small open-source model running locally on CPU | — |

- Switching providers is a single setting (LangChain chat-model interface).
- **Keys are protected by PINs** (details in §11):
  - **Owner's Gemini key:** LLM features only unlock after entering an **Access PIN** that the owner shares with testers.
  - **Tester's own key (BYOK):** protected by a **personal PIN** the tester chooses.

## 7. Demo Scenarios
| # | Scenario | What it proves |
|---|---|---|
| 1 | **Regulatory impact:** "An EU AI Act amendment lands. What's affected?" | Following 4–6 links through the graph: regulation → policies → controls → SOPs → teams → KPIs |
| 2 | **Side by side:** the same question answered by plain RAG and by GraphRAG, each showing its sources | Better answers on questions that need several hops or several documents |
| 3 | **Role impersonation:** the tester picks a role (see §7.1) and asks the same question; a **compare mode** shows two roles side by side | Access control and guardrails built into the graph |
| 4 | **Upload your own PDF:** see it turn into a graph and ask questions about it | Real ingestion, plus isolated graphs per session |
| 5 | **Fraud rings** across ~1M records | The "and more": graph analytics beyond RAG |
| 6 | **Risk roll-up:** one control → department KRI → a bank-wide score | Aggregating up a hierarchy |
| 7 | **Version history:** which SOP replaced which, and why | Lineage |
| 8 | **Performance panel:** response times for these queries on the 1M-node graph | Speed at a realistic scale |

### 7.1 Role Impersonation
Testers choose who they are, and the same question gets different answers depending on what that role is allowed to see.
Access is decided by graph paths (`Role -[:CAN_ACCESS]-> Policy/System`, `Guardrail -[:APPLIES_TO]-> Role`) and
enforced **before** retrieval, so the LLM never sees content the role isn't allowed to see.

| Role | Clearance | Sees |
|---|---|---|
| Branch Teller | L1 | Retail SOPs, customer-facing policies |
| Relationship Manager (Wealth) | L2 | Plus client suitability, wealth products |
| Credit Analyst | L2 | Plus credit policy, limits, risk ratings |
| Compliance Officer (AML/KYC) | L3 | Plus AML typologies, fraud-ring alerts, SAR procedures |
| Chief Risk Officer | L4 | Everything, including bank-wide KRIs and board risk reports |
| Internal Auditor | L4 (read-only) | Everything, plus audit findings and control test results |
| External Contractor (IT) | L0 | Only IT runbooks for their assigned systems |

- A role badge is always visible. Every answer shows **"Answered as: <role>"** and, where relevant, notes that
  N sources were withheld for this role (without revealing what they contain).
- **Compare mode:** one question, two roles, answers side by side, with the difference highlighted.
- This is a simulation for the demo: choosing a role is not authentication.

## 8. Evaluation (plain RAG vs GraphRAG)
- **~30 questions with known correct answers** in 4 types: single fact, multi-hop, cross-document, aggregation/compliance.
- **Two setups:** (a) vector-only RAG (top-k chunks), (b) GraphRAG (vector entry points + graph traversal).
  Both use FalkorDB, so the only difference is whether the graph is used.
- **Metrics:**
  - answer accuracy (graded against the known answers)
  - completeness (were all affected items found?)
  - whether citations are correct
  - response time
  - **tokenomics** (see §8.1)
- **Expected results, stated up front:** similar on single-fact questions, and GraphRAG ahead on the other types.
  Results are published in the UI as they come out.

### 8.1 Tokenomics
Every answer shows its token economics, and the evaluation reports them in aggregate:
| Metric | Meaning |
|---|---|
| Input / output tokens | What went into and came out of the LLM |
| Context efficiency | How many of the tokens sent were actually cited in the answer |
| Cost per answer | Tokens × list price of the selected model (shown as $0 on the free tier, with the equivalent paid cost next to it) |
| **Cost per correct answer** | Total cost ÷ correct answers. This is the headline number for executives |
| Tokens saved | GraphRAG vs vector RAG. Precise graph context typically means fewer, more relevant tokens (to be measured) |

A running session total ("You've used X tokens ≈ $Y") appears in the header.

## 9. Architecture & Hosting

> **Build notes (v0.3 implementation):**
> - **Hosting changed:** Docker Spaces are paid on this account, so Cortex runs as a **Gradio-SDK Space on ZeroGPU (free)**.
>   `app.py` serves the FastAPI app on port 7860. FalkorDB runs **embedded**, using the redis-server and FalkorDB module
>   shipped in the `falkordblite` package. The fallback, if the Space image can't run it, is the FalkorDB Cloud free tier.
> - **Deploy:** a GitHub Action (`.github/workflows/deploy-cortex-space.yml`) pushes `cortex/` to `sk-aiu/cortex`.
> - **Extraction:** bank PDFs use pattern extraction (0 tokens; 295/295 relationships correct against the answer key).
>   Uploads use LLM structured extraction.
> - **No LLM-written Cypher in v1:** all Cypher is fixed and read-only, which is stricter than §11's rule.
> - **Operational graph size** is chosen automatically from available RAM (≥12 GB: ~1M nodes; ≥6 GB: 50%; otherwise 20%).

```
      Hugging Face Docker Space (free, 16 GB RAM, 2 vCPU, port 7860)
 ┌─────────────────────────────────────────────────────────────┐
 │ supervisord                                                  │
 │  ├─ FalkorDB (knowledge graph + ops graph + sandbox graphs)  │
 │  └─ App: FastAPI + static web UI                             │
 │        ├─ LangChain pipelines (ingest, RAG, GraphRAG)        │
 │        ├─ Local embeddings (CPU)                             │
 │        └─ LLM gateway → Gemini (default) / Claude (BYO key)  │
 └─────────────────────────────────────────────────────────────┘
```
- Everything runs in one container, and the same image runs locally with `docker compose up`.
- **Idle reset after 4 hours:** if nobody uses the app for 4 hours, it resets itself. Sandbox graphs and chat
  sessions are cleared, memory is freed, and PIN sessions expire.
  - Note: on the free tier, Hugging Face decides when the container itself goes to sleep (about 48 hours idle),
    and that can't be changed without paid hardware. The 4-hour rule is enforced by our app.
  - Waking the container after it sleeps takes 1–3 minutes, and a "warming up" page is shown meanwhile.
- **Removing Neo4j frees about 6 GB**, which leaves plenty of room on 16 GB.

## 10. UI
- **Executive view:** landing page with key figures, one-click buttons for scenarios 1–8, a graph visualisation,
  and the "Ask Cortex" chat with side-by-side answers, source paths and a tokenomics strip for each answer.
- **Role switcher** in the header (§7.1), with compare mode.
- **Engineer view:** the raw Cypher behind every answer, the pipeline steps, evaluation results (CSV download),
  a link to the repo, and a settings panel for choosing the LLM or entering your own key.
- **Stack:** FastAPI backend and a lightweight static front end (Cytoscape.js from a CDN for the graph view).

## 11. Security, Cost, Limits & Restrictions
**Every limit is shown to users where it applies**: on the upload box, in the chat header, in the settings panel,
and in a "Limits & fair use" page linked from the footer. Nothing is limited silently.

### 11.1 Key protection (PINs)
| Key | Protection | Where the user sees it |
|---|---|---|
| Owner's Gemini key | Stored as a Space secret and never sent to the browser. LLM features need an **Access PIN** (6+ digits, shared by the owner, can be changed at any time). A valid PIN gives an 8-hour session | PIN prompt when the tester first asks a question |
| Tester's own key (BYOK) | Encrypted **in the tester's browser** with a personal PIN (WebCrypto AES-GCM, key derived with PBKDF2). Decrypted only when a request is sent, passed to the provider over HTTPS, never stored or logged on the server | Settings panel: "Your key is encrypted on this device. Use a key with a spending limit." |
| Failed PIN attempts | 5 failures lead to a 15-minute lockout for that session or IP | Lockout message with a countdown |

### 11.2 Limits & restrictions (the user-facing list)
| Area | Limit / rule | Shown where |
|---|---|---|
| Sandbox graphs | **Deleted 24 hours after upload**, or at the 4-hour idle reset if that comes first | Countdown timer on the sandbox ("Wipes in 23h 12m") |
| Uploads | ≤ 5 MB, ≤ 20 pages, PDF only, text-only extraction, max 3 uploads per session | On the upload box |
| Shared free LLM | Questions per session and per day (e.g. 20 / 200). BYOK has no app limit | Counter in the chat header ("12 of 20 left today") |
| Questions | Max 500 characters. Harmful or off-topic prompts are refused | Under the input box |
| LLM-generated queries | **Read-only** Cypher (write clauses blocked), 5-second timeout, 1,000-row limit | Engineer view, with the query |
| Role impersonation | A simulation, not authentication | Role badge tooltip |
| Data | 100% synthetic. Never upload real customer or confidential data | Banner on the landing page and the upload box |
| Idle reset | After 4 hours without activity, sessions, sandboxes and PIN sessions are cleared | Footer note |
| Scripted scenarios | Answers are **pre-computed**: they cost nothing and work even if the LLM is down | "Cached answer" tag |
| Logging | Only questions and response times are logged. No keys, PINs or uploaded content | Limits & fair use page |

## 11A. Name & Branding
- **Name: Cortex.** The name suggests the "intelligence layer" of the enterprise, and follows the IT convention of
  "Cortex" for intelligence, security and engineering platforms.
  - **Caution:** several commercial products already use "Cortex". That's fine for an internal demo, but check
    trademarks before any external or commercial use. The fictional bank is **"Cortex Bank"**.
- **Logo:** an original mark. A "C" formed by an arc of connected graph nodes, with a bright node at its opening
  (knowledge → insight). Navy-to-orange gradient. Files: `docs/brand/cortex-logo.svg` / `.png`.
- **Palette:** navy `#0B1B3A`, blue `#1E3A8A`, orange `#F97316`, teal `#0EA5A4` (for "success"), slate for text.

## 12. Repository Layout
```
cortex/
  docs/        DESIGN.md, DEMO_SCRIPT.md, EVAL.md, one-pager
  content/     source Markdown for the ~60 bank documents
  pdfs/        rendered PDFs (committed)
  generator/   pdf_render.py, ops_data.py (seeded, fraud injection)
  ingest/      LangChain pipeline: load, chunk, embed, extract, load to FalkorDB
  app/         FastAPI, RAG + GraphRAG chains, LLM gateway, static UI
  eval/        questions + known answers, runner, results
  deploy/      Dockerfile (HF Space), supervisord.conf, docker-compose.yml
  tests/
```

## 13. Future State (not now)
- **AWS deployment** (for example ECS, or EC2 with FalkorDB Cloud) once there's an AWS account and budget.
- **Neo4j comparison** as an optional write-up or benchmark appendix.
- **Live external feed:** daily AI and regulatory news ingested as `Signal` nodes that link to the policies they
  affect. This was the original "AI News" idea.

## 14. Two-Day Plan
| When | Deliverable |
|---|---|
| Day 1 AM | Bank content (Markdown) → ~60 PDFs; ops data generator |
| Day 1 PM | LangChain ingestion into FalkorDB; graph snapshot; RAG and GraphRAG chains working |
| Day 2 AM | UI (executive and engineer views, role switcher, tokenomics), scenarios, uploads, PIN-protected keys, evaluation run |
| Day 2 PM | Hugging Face Space deploy, limits page, demo script, updated one-pager |
| After build | Recorded backup video, once everything is built and nearly ready |

## 15. Risks
| Risk | Mitigation |
|---|---|
| Free-tier LLM rate limits | Pre-built graph, cached scenario answers, BYO key |
| Imperfect LLM extraction | Schema-guided extraction, validation, spot checks against known answers |
| Space cold start | Warm-up page, fast snapshot load |
| Trust around BYO keys | Encrypted in the browser with a PIN, not logged, disclosure on the page, suggest limited keys |
| Access PIN leaked | Owner changes it in Space secrets; rate limits cap the damage |
| Hugging Face sleep can't be set to 4h on the free tier | App-level 4-hour idle reset (§9) |
| Licensing (FalkorDB is SSPL) | Flag to Legal before any production use |

**Open items:** owner creates a free Gemini API key and a Hugging Face Space, and chooses the Access PIN (instructions will be provided).

## 16. Cradle of the Learning Portal
Cortex is where the owner's **Learning Portal** started (1 Oct 2026). After Cortex went live, the owner wanted to be able to
explain it from scratch, at any depth, without reading long text. That became a separate product:
a private, phone-first portal that turns every project into learning, in a loop: **Build → Capture → Learn → Explore → Decide → Build**.

**Agreed brief (for the new repo; the portal's name = the repo's name, chosen by the owner):**
| Topic | Decision |
|---|---|
| Proof of concept | Cortex only; other projects join later |
| Device and privacy | iPhone first; private (private Hugging Face Space) |
| Formats | Audio episodes, 60-90 s narrated reels, swipe cards, 3-question checks, "explain it back"; text is the fallback |
| Voice | Kokoro-82M `af_heart`, same as the X-Ray film (`docs/film/make_audio.py`) |
| Structure | Concepts ↔ Projects ↔ Decisions, linked as a graph |
| Retention | Daily 5-minute spaced-repetition drill |
| Updates | Weekly scheduled run reads the repos (commits + `DECISIONS.md`) and refreshes learning packs; on demand too |
| Explore | Weekly tech radar (Adopt / Trial / Watch / Hold) for the technologies and techniques each project uses, with dated sources; the owner decides |
| Capture habit | Every project keeps a `DECISIONS.md` (decision, options, reason, lesson); see [DECISIONS.md](DECISIONS.md) |