Spaces:

OGrohit
/

ringguard

Sleeping

Rohit Patil

readme: full rewrite — clean UTF-8, accurate content, no encoding artifacts

279fada 2 months ago

19.1 kB

---
title: RingGuard
colorFrom: red
colorTo: yellow
sdk: docker
pinned: true
license: mit
short_description: Graph-based fraud ring detection for Razorpay merchants
---
# RingGuard

![Track](https://img.shields.io/badge/Track-AI%20Risk%20Manager-blue) ![Event](https://img.shields.io/badge/Event-Razorpay%20AI%20Buildathon%202026-orange) ![Status](https://img.shields.io/badge/Status-Phase%203%20Complete-brightgreen) ![Python](https://img.shields.io/badge/Python-3.11-blue) ![License](https://img.shields.io/badge/License-MIT-green)

**Coordinated fraud detection for Razorpay merchants — finding the rings per-order scoring can't see.**

RingGuard analyzes a merchant's order and account data to detect coordinated abuse — accounts secretly linked by shared devices, addresses, or payment instruments, working together to exploit returns, promo codes, or chargebacks. Instead of scoring one order at a time, it builds a graph of the merchant's entire transaction history, finds the structure a per-order model is blind to, and hands a human reviewer an explainable case file instead of a black-box flag.

Built for **Track 02 — AI Risk Manager** at the Razorpay AI Buildathon 2026.

---

## Table of Contents

1. [The Problem](#the-problem)
2. [The Solution](#the-solution)
3. [Why This, Not Thirdwatch](#why-this-not-thirdwatch)
4. [How It Works](#how-it-works)
5. [AI Judgment — Where Each Tool Is Used](#ai-judgment--where-each-tool-is-used)
6. [System Architecture](#system-architecture)
7. [Tech Stack](#tech-stack)
8. [Repository Structure](#repository-structure)
9. [Getting Started](#getting-started)
10. [Usage](#usage)
11. [Evaluation and Metrics](#evaluation-and-metrics)
12. [Known Limitations](#known-limitations)
13. [Policy and Safety Boundaries](#policy-and-safety-boundaries)
14. [Audit Trail](#audit-trail)
15. [Demo](#demo)
16. [Roadmap](#roadmap)
17. [Acknowledgments](#acknowledgments)
18. [License](#license)

---

## The Problem

Cash-on-delivery return-to-origin rates in Indian e-commerce run as high as 30-40%, and coordinated abuse — not isolated bad actors — is a meaningful share of that loss. Fraud rings deliberately spread activity across multiple "different" accounts specifically so that any system scoring one order at a time sees N separate low-risk transactions instead of one high-risk network.

Per-order risk engines, however sophisticated, are structurally blind to this by design — they were never built to look across accounts.

## The Solution

RingGuard is a defensive, AI-assisted investigation agent that:

- **Ingests** a merchant's order, customer, device, address, and payment data (Razorpay Orders/Payments-shaped).
- **Builds a graph** connecting accounts through shared devices, payment instruments, addresses, and contact details.
- **Detects communities** in that graph — clusters of accounts more connected to each other than chance would suggest.
- **Scores each cluster** with a trained LightGBM model using graph structure and behavioral features.
- **Explains the finding** in plain language via an LLM that reads only the structured evidence it is given — never free-reasoning, never inventing facts.
- **Recommends a bounded action** — flag, hold, or escalate — gated by deterministic policy rules, with a human always in the loop for anything above the auto-hold threshold.

It never bans an account or seizes funds. It flags, it explains, and it waits for a human on anything that matters.

## Why This, Not Thirdwatch

Razorpay has owned Thirdwatch since 2019 — a mature fraud/RTO engine scoring individual orders on roughly 300 device and behavioral signals. Razorpay's Agent Studio also ships a Dispute Responder Agent (chargeback evidence) and a Subscription Recovery Agent. None of these look *across* orders and accounts for structure — they operate at the level of one order or one case at a time.

RingGuard is not a better per-order scorer. It is a different problem shape: network-level detection of coordinated behavior that is invisible when you only ever look at one order at a time. That is the entire differentiation argument, and it is stated here first rather than left for a judge to have to ask about.

## How It Works

```
Synthetic order/account stream (Razorpay Orders/Payments-shaped)
        |
        v
Entity resolution — normalize phone/email/address, hash device + payment fields
        |
        v
Graph construction — Customer / Device / Address / PaymentInstrument nodes
        |
        v
Community detection — connected components + Louvain (deterministic, seed=42)
  Two-pass: strong subgraph first (shared device/instrument >= 2 orders),
            weak-only subgraph second (shared address, no strong link)
        |
        v
Feature extraction — cluster size, strong/weak link ratio, order velocity,
                      return rate, chargeback rate, promo usage, exposure (14 features)
        |
        v
LightGBM risk model — single learned probability, no hand-blended scoring
        |
        v
Policy gate (deterministic, v3.0)
   |-- risk >= 0.85 and exposure <= Rs.25,000 ->  bounded auto-hold (Razorpay API, test mode)
   |-- risk >= 0.85 and exposure >  Rs.25,000 ->  escalate, no auto-action
   |-- 0.5 <= risk < 0.85                    ->  escalate to human review queue
   +-- risk < 0.5                            ->  log only
        |
        v
Investigator agent (LangGraph + Groq LLM) — evidence bundle in, case narrative out
        |
        v
Razorpay test-mode action (order.create + order.edit notes + order.fetch verify)
        |
        v
Audit logger — every stage, every decision, timestamped (audit_log.jsonl)
```

A cluster built entirely from **weak links** (e.g., three accounts sharing only an address — plausible for a household) is designed to score low. Distinguishing that from a **strong-link** cluster (shared device fingerprint used in 2+ orders) is what keeps false positives down, and it is built into the feature set from the start.

## AI Judgment — Where Each Tool Is Used

| Stage | Tool | Why |
|-------|------|-----|
| Graph construction, edge rules, policy gate | Deterministic code | Money-adjacent decisions need to be boring and auditable — no model judgment here |
| Community detection, clustering | Classical graph algorithms (Louvain + connected components) | Structural pattern detection, not a language problem |
| Ring risk scoring | LightGBM (supervised, tabular) | Faster, cheaper, and more auditable than an LLM for a numeric classification task |
| Case narrative and explanation | LLM (Groq — `openai/gpt-oss-120b`) | The one place where synthesizing evidence into something a reviewer can act on quickly genuinely benefits from language generation |
| Final action | Deterministic policy gate, human-approved above threshold | The LLM never has authority to move money or take action on its own |

## System Architecture

```mermaid
flowchart TD
    A[Synthetic Order Stream] --> B[Entity Resolution]
    B --> C[Graph Construction - NetworkX]
    C --> D[Two-Pass Community Detection]
    D --> E[Feature Extraction - 14 features]
    E --> F[LightGBM Risk Model]
    F --> G{Deterministic Policy Gate}
    G -->|risk >= 0.85, exposure <= 25k INR| H[Auto-Hold via Razorpay API]
    G -->|risk >= 0.85, exposure > 25k INR| I[Escalate to Human Review]
    G -->|0.5 <= risk < 0.85| I
    G -->|risk < 0.5| J[Log Only]
    H --> K[Investigator Agent - LangGraph + Groq LLM]
    I --> K
    K --> L[Case Narrative + Confidence Statement]
    L --> M[Audit Log + FastAPI Dashboard]
    J --> M
```

## Tech Stack

| Layer | Choice |
|-------|--------|
| Language / API | Python 3.11, FastAPI |
| Graph | NetworkX + python-louvain |
| Scoring | LightGBM |
| Agent orchestration | LangGraph |
| LLM | Groq (`openai/gpt-oss-120b`, plain chat completion — no tool calls) |
| Razorpay integration | Official Razorpay Python SDK v2.0.1 (test-mode, verified) |
| Synthetic data | Faker |
| Dashboard | Server-side HTML rendered by FastAPI |
| Deployment | HuggingFace Spaces (Docker) |

No paid dependencies anywhere in this stack.

## Repository Structure

```
ringguard/
|-- README.md
|-- requirements.txt
|-- .env.example
|-- Dockerfile
|-- data/
|   |-- generate_synthetic_data.py     <- 500 customers, 10 rings, seed=42
|   |-- augment_demo_dataset.py        <- adds RING11 + BOUNDARY01
|   +-- synthetic/
|       |-- customers.json             <- 506 customers
|       |-- orders.json                <- 1,215 orders
|       |-- ground_truth.json          <- 11 rings
|       |-- candidate_clusters.json    <- 28 candidate clusters
|       |-- scored_clusters.json       <- 28 clusters with risk_probability
|       +-- cases_demo.json            <- 28 cases with LLM narratives + Razorpay action_taken
|-- graph/
|   |-- build_graph.py                 <- entity resolution + heterogeneous graph
|   +-- detect_communities.py          <- two-pass community detection
|-- scoring/
|   |-- features.py                    <- 14-feature extraction
|   |-- train_model.py                 <- LightGBM training (training dataset only)
|   |-- score_demo.py                  <- scores demo dataset
|   +-- model.lgb                      <- trained model (do not retrain without reason)
|-- agent/
|   |-- policy.py                      <- deterministic policy gate (v3.0)
|   |-- investigator.py                <- Groq LLM investigator node
|   |-- graph.py                       <- LangGraph StateGraph orchestration
|   +-- run_demo.py                    <- end-to-end demo runner
|-- integration/
|   +-- razorpay_actions.py            <- Razorpay test-mode: create + edit + fetch orders
|-- api/
|   +-- main.py                        <- FastAPI dashboard (/, /cases, /health)
|-- eval/
|   |-- evaluate.py                    <- holdout evaluation script
|   +-- eval_results.json              <- real evaluation numbers (do not edit manually)
+-- audit/
    |-- logger.py                      <- append-only structured audit logger
    |-- generate_audit_log.py          <- bootstraps audit_log.jsonl from cases_demo.json
    +-- audit_log.jsonl                <- full audit trail, 28 entries
```

## Getting Started

### Prerequisites

- Python 3.11+
- A free [Groq API key](https://console.groq.com)
- Razorpay test-mode API keys (`rzp_test_...`) from the [Razorpay Dashboard](https://dashboard.razorpay.com)

### Installation

```bash
git clone https://github.com/rohitdecodes/ringguard.git
cd ringguard
python -m venv venv
source venv/bin/activate       # Windows: venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env            # fill in GROQ_API_KEY, RAZORPAY_KEY_ID, RAZORPAY_KEY_SECRET
```

### Environment variables

```
GROQ_API_KEY=your_groq_api_key
RAZORPAY_KEY_ID=rzp_test_...
RAZORPAY_KEY_SECRET=...
```

Keys are read from `.env` at runtime. `.env` is gitignored — never commit credentials.

## Usage

The pipeline runs in order. Each step can be re-run independently.

```bash
# 1. Generate base synthetic data (seed=42, deterministic)
python data/generate_synthetic_data.py
# 2. Augment with RING11 (auto_hold case) + BOUNDARY01 (hard ambiguous case)
python data/augment_demo_dataset.py
# 3. Build the heterogeneous graph
python graph/build_graph.py
# 4. Detect candidate clusters (two-pass: strong + weak-only)
python graph/detect_communities.py
# 5. Score every cluster with the trained LightGBM model
python scoring/score_demo.py
# 6. Run the full LangGraph agent pipeline (LLM calls for escalate/auto_hold cases)
python agent/run_demo.py
# 7. Execute Razorpay test-mode actions for all actioned cases
python integration/razorpay_actions.py
# 8. Regenerate audit log from finalized cases
python audit/generate_audit_log.py
# 9. Run held-out evaluation
python eval/evaluate.py
# 10. Launch the dashboard
uvicorn api.main:app --port 7860
# Open http://localhost:7860
```

**Deployed dashboard:** [https://ogrohit-ringguard.hf.space](https://ogrohit-ringguard.hf.space)  
Health check: [https://ogrohit-ringguard.hf.space/health](https://ogrohit-ringguard.hf.space/health)

## Evaluation and Metrics

Evaluated on a held-out cluster-level split (no ring's accounts appear in both train and test). Fixed seed=42, stratified 80/20 split, 22 held-out clusters.

| Metric | Value |
|--------|-------|
| **Precision** | **1.0000** |
| **Recall** | **1.0000** |
| **F1** | **1.0000** |
| False-positive cost (held orders wrongly flagged) | **Rs. 0.00** |
| Inference latency (feature extraction + model) | **< 0.1 ms/cluster** |

Holdout breakdown: 12 positive (fraud rings), 10 negative (benign clusters).

### Policy action breakdown — holdout set

| Action | Count |
|--------|-------|
| auto_hold | 0 |
| escalate | 12 |
| log_only | 10 |

All 12 positive clusters score >= 0.85 but have total exposure above Rs. 25,000 (realistic ring sizes accumulate large amounts), so they route to `escalate` rather than `auto_hold`. This is correct behavior.

### Demo dataset — 28 clusters (post-augmentation)

| Action | Count | Notes |
|--------|-------|-------|
| auto_hold | 1 | RING11 — risk 98.67%, exposure Rs. 19,800 |
| escalate | 10 | Original 10 rings — risk 99.1-99.7%, large exposure |
| log_only | 17 | Includes BOUNDARY01 (see Known Limitations) |

### Feature importances (top 5)

| Feature | Importance |
|---------|-----------|
| avg_order_amount | 145 |
| chargeback_rate | 120 |
| strong_link_count | 90 |
| return_rate | 73 |
| order_velocity | 37 |
## Known Limitations
**Read this before drawing conclusions from the precision/recall numbers above.**
The current P=R=F1=1.0 figures reflect that the synthetic training data does not yet contain genuinely ambiguous examples. The model has not been tested on a hard case — one where real human experts would disagree.
**1. Training distribution boundary.**
The model's top feature is `avg_order_amount` (importance=145). Positive training examples have avg_order_amount >= Rs. 3,249; all negatives are below Rs. 3,171. This creates a near-hard threshold in a learned model. Real rings could fall on either side of it.
**2. BOUNDARY01 — the designed hard case.**
BOUNDARY01 (CUST0504-CUST0506) was rebuilt in Phase 3 as a genuine structural challenge: 3 accounts sharing device DEV0402 with 2 orders each (count >= 2, forming a `strong_anchored` cluster with strong_link_count=3 — the same structural signature as a confirmed fraud ring). Yet: zero chargebacks, low return rate (1/9), modest avg_order_amount (~Rs. 1,183). Model output: **risk=2.08%, log_only**.
The model is not wrong. Zero chargebacks and low order values legitimately suppress the score — that is precisely what the feature weights say to do. But this also means: if a real low-value shared-device cluster were fraudulent, the current model would miss it. This is an honest limitation of the training distribution.
**3. Synthetic data only.**
All evaluation is against Faker-generated data. Real merchants have messier entity resolution, more diverse ring structures, and ambiguous cases the current training data cannot represent. Real-world P/R/F1 should be expected to be materially lower.
**4. Auto-hold threshold calibration.**
The original Rs. 5,000 auto-hold cap was incompatible with the training distribution (a minimal ring totals ~Rs. 19,800). Updated to Rs. 25,000 in Phase 3 (v3.0). The threshold was never validated against real production exposure data.
**5. LLM model substitution.**
`llama-3.3-70b-versatile` (planned) was unavailable on the Groq account. `groq/compound-mini` was used first; the final pipeline uses `openai/gpt-oss-120b`. Both produce valid narratives with zero grounding failures across all 11 LLM calls. Model is configurable via `GROQ_MODEL` environment variable.
Real-world performance validation requires labeled production data. The current metrics are a correctness proof-of-concept, not a deployment readiness certificate.
## Policy and Safety Boundaries
RingGuard is **defense-only**, per Track 02's rules:
- It can flag, hold, and escalate. It cannot ban accounts, seize funds, or take any irreversible action.
- Auto-hold only fires at risk >= 0.85 AND total cluster exposure <= Rs. 25,000. Everything above that threshold, or in the mid-confidence band (0.5-0.85), goes to a human review queue — never auto-actioned.
- The LLM has no authority to trigger an action. It explains a decision the deterministic policy gate already made.
- The LLM (`openai/gpt-oss-120b`) is called as a plain chat completion with no `tools` parameter — it has no web-search or code-execution access. It reads only the structured evidence bundle.
- All Razorpay actions are test-mode only (`rzp_test_...` keys enforced in code). No real money moves.
## Audit Trail
Every case writes one JSON record to `audit/audit_log.jsonl` containing:
```
case_id, timestamp, cluster_members, evidence_bundle, graph_score_inputs (all 14 features),
lightgbm_probability, llm_narrative, llm_confidence_statement, policy_decision,
action_taken (Razorpay order data or null), human_reviewer, outcome,
reason_if_not_auto_actioned
```
The log is append-only. Every case in `cases_demo.json` has a matching audit entry, including `log_only` cases. The dashboard at `/cases/{case_id}/detail` shows the full audit chain for each case.
## Demo
**Live deployment:** [https://ogrohit-ringguard.hf.space](https://ogrohit-ringguard.hf.space)

The dashboard shows all 28 detected cases, risk scores, policy decisions, and Razorpay order IDs. Click any case row to see the full evidence bundle, LLM narrative, and audit chain.

**To run the pipeline locally:**

```bash
# Run the full detection pipeline (LLM calls included)
python agent/run_demo.py
# Run live evaluation (produces real numbers, not pre-computed)
python eval/evaluate.py
# Start the dashboard
uvicorn api.main:app --port 7860
```

## Roadmap

- [x] **Phase 1 — Detection Core:** synthetic data generation, entity resolution, graph construction, two-pass community detection (strong + weak subgraphs). 10/10 rings recovered, 0/5 false-positive stress clusters.
- [x] **Phase 2 — Intelligence Layer:** 14-feature extraction, LightGBM risk model (P=R=F1=1.0 on holdout), LangGraph agent orchestration, Groq LLM investigator node, deterministic policy gate, grounding check.
- [x] **Phase 3 — Integration and Submission:** RING11 (auto_hold demo case) + BOUNDARY01 (hard ambiguous case), Razorpay Python SDK test-mode integration (create/edit/fetch, notes verified), append-only audit logger, FastAPI dashboard, HuggingFace Spaces deployment (Docker), final evaluation.
## Acknowledgments
Built with [LangGraph](https://langchain-ai.github.io/langgraph/), [Groq](https://groq.com) (`openai/gpt-oss-120b`), [LightGBM](https://lightgbm.readthedocs.io/), [NetworkX](https://networkx.org/), [python-louvain](https://python-louvain.readthedocs.io/), and the [Razorpay Python SDK](https://github.com/razorpay/razorpay-python).
## License
MIT — see [LICENSE](LICENSE).

	---
	title: RingGuard
	colorFrom: red
	colorTo: yellow
	sdk: docker
	pinned: true
	license: mit
	short_description: Graph-based fraud ring detection for Razorpay merchants
	---

	# RingGuard

	![Track](https://img.shields.io/badge/Track-AI%20Risk%20Manager-blue) ![Event](https://img.shields.io/badge/Event-Razorpay%20AI%20Buildathon%202026-orange) ![Status](https://img.shields.io/badge/Status-Phase%203%20Complete-brightgreen) ![Python](https://img.shields.io/badge/Python-3.11-blue) ![License](https://img.shields.io/badge/License-MIT-green)

	Coordinated fraud detection for Razorpay merchants — finding the rings per-order scoring can't see.

	RingGuard analyzes a merchant's order and account data to detect coordinated abuse — accounts secretly linked by shared devices, addresses, or payment instruments, working together to exploit returns, promo codes, or chargebacks. Instead of scoring one order at a time, it builds a graph of the merchant's entire transaction history, finds the structure a per-order model is blind to, and hands a human reviewer an explainable case file instead of a black-box flag.

	Built for Track 02 — AI Risk Manager at the Razorpay AI Buildathon 2026.

	---

	## Table of Contents

	1. [The Problem](#the-problem)
	2. [The Solution](#the-solution)
	3. [Why This, Not Thirdwatch](#why-this-not-thirdwatch)
	4. [How It Works](#how-it-works)
	5. [AI Judgment — Where Each Tool Is Used](#ai-judgment--where-each-tool-is-used)
	6. [System Architecture](#system-architecture)
	7. [Tech Stack](#tech-stack)
	8. [Repository Structure](#repository-structure)
	9. [Getting Started](#getting-started)
	10. [Usage](#usage)
	11. [Evaluation and Metrics](#evaluation-and-metrics)
	12. [Known Limitations](#known-limitations)
	13. [Policy and Safety Boundaries](#policy-and-safety-boundaries)
	14. [Audit Trail](#audit-trail)
	15. [Demo](#demo)
	16. [Roadmap](#roadmap)
	17. [Acknowledgments](#acknowledgments)
	18. [License](#license)

	---

	## The Problem

	Cash-on-delivery return-to-origin rates in Indian e-commerce run as high as 30-40%, and coordinated abuse — not isolated bad actors — is a meaningful share of that loss. Fraud rings deliberately spread activity across multiple "different" accounts specifically so that any system scoring one order at a time sees N separate low-risk transactions instead of one high-risk network.

	Per-order risk engines, however sophisticated, are structurally blind to this by design — they were never built to look across accounts.

	## The Solution

	RingGuard is a defensive, AI-assisted investigation agent that:

	- Ingests a merchant's order, customer, device, address, and payment data (Razorpay Orders/Payments-shaped).
	- Builds a graph connecting accounts through shared devices, payment instruments, addresses, and contact details.
	- Detects communities in that graph — clusters of accounts more connected to each other than chance would suggest.
	- Scores each cluster with a trained LightGBM model using graph structure and behavioral features.
	- Explains the finding in plain language via an LLM that reads only the structured evidence it is given — never free-reasoning, never inventing facts.
	- Recommends a bounded action — flag, hold, or escalate — gated by deterministic policy rules, with a human always in the loop for anything above the auto-hold threshold.

	It never bans an account or seizes funds. It flags, it explains, and it waits for a human on anything that matters.

	## Why This, Not Thirdwatch

	Razorpay has owned Thirdwatch since 2019 — a mature fraud/RTO engine scoring individual orders on roughly 300 device and behavioral signals. Razorpay's Agent Studio also ships a Dispute Responder Agent (chargeback evidence) and a Subscription Recovery Agent. None of these look across orders and accounts for structure — they operate at the level of one order or one case at a time.

	RingGuard is not a better per-order scorer. It is a different problem shape: network-level detection of coordinated behavior that is invisible when you only ever look at one order at a time. That is the entire differentiation argument, and it is stated here first rather than left for a judge to have to ask about.

	## How It Works

	```
	Synthetic order/account stream (Razorpay Orders/Payments-shaped)
	\|
	v
	Entity resolution — normalize phone/email/address, hash device + payment fields
	\|
	v
	Graph construction — Customer / Device / Address / PaymentInstrument nodes
	\|
	v
	Community detection — connected components + Louvain (deterministic, seed=42)
	Two-pass: strong subgraph first (shared device/instrument >= 2 orders),
	weak-only subgraph second (shared address, no strong link)
	\|
	v
	Feature extraction — cluster size, strong/weak link ratio, order velocity,
	return rate, chargeback rate, promo usage, exposure (14 features)
	\|
	v
	LightGBM risk model — single learned probability, no hand-blended scoring
	\|
	v
	Policy gate (deterministic, v3.0)
	\|-- risk >= 0.85 and exposure <= Rs.25,000 -> bounded auto-hold (Razorpay API, test mode)
	\|-- risk >= 0.85 and exposure > Rs.25,000 -> escalate, no auto-action
	\|-- 0.5 <= risk < 0.85 -> escalate to human review queue
	+-- risk < 0.5 -> log only
	\|
	v
	Investigator agent (LangGraph + Groq LLM) — evidence bundle in, case narrative out
	\|
	v
	Razorpay test-mode action (order.create + order.edit notes + order.fetch verify)
	\|
	v
	Audit logger — every stage, every decision, timestamped (audit_log.jsonl)
	```

	A cluster built entirely from weak links (e.g., three accounts sharing only an address — plausible for a household) is designed to score low. Distinguishing that from a strong-link cluster (shared device fingerprint used in 2+ orders) is what keeps false positives down, and it is built into the feature set from the start.

	## AI Judgment — Where Each Tool Is Used

	\| Stage \| Tool \| Why \|
	\|-------\|------\|-----\|
	\| Graph construction, edge rules, policy gate \| Deterministic code \| Money-adjacent decisions need to be boring and auditable — no model judgment here \|
	\| Community detection, clustering \| Classical graph algorithms (Louvain + connected components) \| Structural pattern detection, not a language problem \|
	\| Ring risk scoring \| LightGBM (supervised, tabular) \| Faster, cheaper, and more auditable than an LLM for a numeric classification task \|
	\| Case narrative and explanation \| LLM (Groq — `openai/gpt-oss-120b`) \| The one place where synthesizing evidence into something a reviewer can act on quickly genuinely benefits from language generation \|
	\| Final action \| Deterministic policy gate, human-approved above threshold \| The LLM never has authority to move money or take action on its own \|

	## System Architecture

	```mermaid
	flowchart TD
	A[Synthetic Order Stream] --> B[Entity Resolution]
	B --> C[Graph Construction - NetworkX]
	C --> D[Two-Pass Community Detection]
	D --> E[Feature Extraction - 14 features]
	E --> F[LightGBM Risk Model]
	F --> G{Deterministic Policy Gate}
	G -->\|risk >= 0.85, exposure <= 25k INR\| H[Auto-Hold via Razorpay API]
	G -->\|risk >= 0.85, exposure > 25k INR\| I[Escalate to Human Review]
	G -->\|0.5 <= risk < 0.85\| I
	G -->\|risk < 0.5\| J[Log Only]
	H --> K[Investigator Agent - LangGraph + Groq LLM]
	I --> K
	K --> L[Case Narrative + Confidence Statement]
	L --> M[Audit Log + FastAPI Dashboard]
	J --> M
	```

	## Tech Stack

	\| Layer \| Choice \|
	\|-------\|--------\|
	\| Language / API \| Python 3.11, FastAPI \|
	\| Graph \| NetworkX + python-louvain \|
	\| Scoring \| LightGBM \|
	\| Agent orchestration \| LangGraph \|
	\| LLM \| Groq (`openai/gpt-oss-120b`, plain chat completion — no tool calls) \|
	\| Razorpay integration \| Official Razorpay Python SDK v2.0.1 (test-mode, verified) \|
	\| Synthetic data \| Faker \|
	\| Dashboard \| Server-side HTML rendered by FastAPI \|
	\| Deployment \| HuggingFace Spaces (Docker) \|

	No paid dependencies anywhere in this stack.

	## Repository Structure

	```
	ringguard/
	\|-- README.md
	\|-- requirements.txt
	\|-- .env.example
	\|-- Dockerfile
	\|-- data/
	\| \|-- generate_synthetic_data.py <- 500 customers, 10 rings, seed=42
	\| \|-- augment_demo_dataset.py <- adds RING11 + BOUNDARY01
	\| +-- synthetic/
	\| \|-- customers.json <- 506 customers
	\| \|-- orders.json <- 1,215 orders
	\| \|-- ground_truth.json <- 11 rings
	\| \|-- candidate_clusters.json <- 28 candidate clusters
	\| \|-- scored_clusters.json <- 28 clusters with risk_probability
	\| +-- cases_demo.json <- 28 cases with LLM narratives + Razorpay action_taken
	\|-- graph/
	\| \|-- build_graph.py <- entity resolution + heterogeneous graph
	\| +-- detect_communities.py <- two-pass community detection
	\|-- scoring/
	\| \|-- features.py <- 14-feature extraction
	\| \|-- train_model.py <- LightGBM training (training dataset only)
	\| \|-- score_demo.py <- scores demo dataset
	\| +-- model.lgb <- trained model (do not retrain without reason)
	\|-- agent/
	\| \|-- policy.py <- deterministic policy gate (v3.0)
	\| \|-- investigator.py <- Groq LLM investigator node
	\| \|-- graph.py <- LangGraph StateGraph orchestration
	\| +-- run_demo.py <- end-to-end demo runner
	\|-- integration/
	\| +-- razorpay_actions.py <- Razorpay test-mode: create + edit + fetch orders
	\|-- api/
	\| +-- main.py <- FastAPI dashboard (/, /cases, /health)
	\|-- eval/
	\| \|-- evaluate.py <- holdout evaluation script
	\| +-- eval_results.json <- real evaluation numbers (do not edit manually)
	+-- audit/
	\|-- logger.py <- append-only structured audit logger
	\|-- generate_audit_log.py <- bootstraps audit_log.jsonl from cases_demo.json
	+-- audit_log.jsonl <- full audit trail, 28 entries
	```

	## Getting Started

	### Prerequisites

	- Python 3.11+
	- A free [Groq API key](https://console.groq.com)
	- Razorpay test-mode API keys (`rzp_test_...`) from the [Razorpay Dashboard](https://dashboard.razorpay.com)

	### Installation

	```bash
	git clone https://github.com/rohitdecodes/ringguard.git
	cd ringguard
	python -m venv venv
	source venv/bin/activate # Windows: venv\Scripts\activate
	pip install -r requirements.txt
	cp .env.example .env # fill in GROQ_API_KEY, RAZORPAY_KEY_ID, RAZORPAY_KEY_SECRET
	```

	### Environment variables

	```
	GROQ_API_KEY=your_groq_api_key
	RAZORPAY_KEY_ID=rzp_test_...
	RAZORPAY_KEY_SECRET=...
	```

	Keys are read from `.env` at runtime. `.env` is gitignored — never commit credentials.

	## Usage

	The pipeline runs in order. Each step can be re-run independently.

	```bash
	# 1. Generate base synthetic data (seed=42, deterministic)
	python data/generate_synthetic_data.py

	# 2. Augment with RING11 (auto_hold case) + BOUNDARY01 (hard ambiguous case)
	python data/augment_demo_dataset.py

	# 3. Build the heterogeneous graph
	python graph/build_graph.py

	# 4. Detect candidate clusters (two-pass: strong + weak-only)
	python graph/detect_communities.py

	# 5. Score every cluster with the trained LightGBM model
	python scoring/score_demo.py

	# 6. Run the full LangGraph agent pipeline (LLM calls for escalate/auto_hold cases)
	python agent/run_demo.py

	# 7. Execute Razorpay test-mode actions for all actioned cases
	python integration/razorpay_actions.py

	# 8. Regenerate audit log from finalized cases
	python audit/generate_audit_log.py

	# 9. Run held-out evaluation
	python eval/evaluate.py

	# 10. Launch the dashboard
	uvicorn api.main:app --port 7860
	# Open http://localhost:7860
	```

	Deployed dashboard: [https://ogrohit-ringguard.hf.space](https://ogrohit-ringguard.hf.space)
	Health check: [https://ogrohit-ringguard.hf.space/health](https://ogrohit-ringguard.hf.space/health)

	## Evaluation and Metrics

	Evaluated on a held-out cluster-level split (no ring's accounts appear in both train and test). Fixed seed=42, stratified 80/20 split, 22 held-out clusters.

	\| Metric \| Value \|
	\|--------\|-------\|
	\| Precision \| 1.0000 \|
	\| Recall \| 1.0000 \|
	\| F1 \| 1.0000 \|
	\| False-positive cost (held orders wrongly flagged) \| Rs. 0.00 \|
	\| Inference latency (feature extraction + model) \| < 0.1 ms/cluster \|

	Holdout breakdown: 12 positive (fraud rings), 10 negative (benign clusters).

	### Policy action breakdown — holdout set

	\| Action \| Count \|
	\|--------\|-------\|
	\| auto_hold \| 0 \|
	\| escalate \| 12 \|
	\| log_only \| 10 \|

	All 12 positive clusters score >= 0.85 but have total exposure above Rs. 25,000 (realistic ring sizes accumulate large amounts), so they route to `escalate` rather than `auto_hold`. This is correct behavior.

	### Demo dataset — 28 clusters (post-augmentation)

	\| Action \| Count \| Notes \|
	\|--------\|-------\|-------\|
	\| auto_hold \| 1 \| RING11 — risk 98.67%, exposure Rs. 19,800 \|
	\| escalate \| 10 \| Original 10 rings — risk 99.1-99.7%, large exposure \|
	\| log_only \| 17 \| Includes BOUNDARY01 (see Known Limitations) \|

	### Feature importances (top 5)

	\| Feature \| Importance \|
	\|---------\|-----------\|
	\| avg_order_amount \| 145 \|
	\| chargeback_rate \| 120 \|
	\| strong_link_count \| 90 \|
	\| return_rate \| 73 \|
	\| order_velocity \| 37 \|

	## Known Limitations

	Read this before drawing conclusions from the precision/recall numbers above.

	The current P=R=F1=1.0 figures reflect that the synthetic training data does not yet contain genuinely ambiguous examples. The model has not been tested on a hard case — one where real human experts would disagree.

	1. Training distribution boundary.
	The model's top feature is `avg_order_amount` (importance=145). Positive training examples have avg_order_amount >= Rs. 3,249; all negatives are below Rs. 3,171. This creates a near-hard threshold in a learned model. Real rings could fall on either side of it.

	2. BOUNDARY01 — the designed hard case.
	BOUNDARY01 (CUST0504-CUST0506) was rebuilt in Phase 3 as a genuine structural challenge: 3 accounts sharing device DEV0402 with 2 orders each (count >= 2, forming a `strong_anchored` cluster with strong_link_count=3 — the same structural signature as a confirmed fraud ring). Yet: zero chargebacks, low return rate (1/9), modest avg_order_amount (~Rs. 1,183). Model output: risk=2.08%, log_only.

	The model is not wrong. Zero chargebacks and low order values legitimately suppress the score — that is precisely what the feature weights say to do. But this also means: if a real low-value shared-device cluster were fraudulent, the current model would miss it. This is an honest limitation of the training distribution.

	3. Synthetic data only.
	All evaluation is against Faker-generated data. Real merchants have messier entity resolution, more diverse ring structures, and ambiguous cases the current training data cannot represent. Real-world P/R/F1 should be expected to be materially lower.

	4. Auto-hold threshold calibration.
	The original Rs. 5,000 auto-hold cap was incompatible with the training distribution (a minimal ring totals ~Rs. 19,800). Updated to Rs. 25,000 in Phase 3 (v3.0). The threshold was never validated against real production exposure data.

	5. LLM model substitution.
	`llama-3.3-70b-versatile` (planned) was unavailable on the Groq account. `groq/compound-mini` was used first; the final pipeline uses `openai/gpt-oss-120b`. Both produce valid narratives with zero grounding failures across all 11 LLM calls. Model is configurable via `GROQ_MODEL` environment variable.

	Real-world performance validation requires labeled production data. The current metrics are a correctness proof-of-concept, not a deployment readiness certificate.

	## Policy and Safety Boundaries

	RingGuard is defense-only, per Track 02's rules:

	- It can flag, hold, and escalate. It cannot ban accounts, seize funds, or take any irreversible action.
	- Auto-hold only fires at risk >= 0.85 AND total cluster exposure <= Rs. 25,000. Everything above that threshold, or in the mid-confidence band (0.5-0.85), goes to a human review queue — never auto-actioned.
	- The LLM has no authority to trigger an action. It explains a decision the deterministic policy gate already made.
	- The LLM (`openai/gpt-oss-120b`) is called as a plain chat completion with no `tools` parameter — it has no web-search or code-execution access. It reads only the structured evidence bundle.
	- All Razorpay actions are test-mode only (`rzp_test_...` keys enforced in code). No real money moves.

	## Audit Trail

	Every case writes one JSON record to `audit/audit_log.jsonl` containing:

	```
	case_id, timestamp, cluster_members, evidence_bundle, graph_score_inputs (all 14 features),
	lightgbm_probability, llm_narrative, llm_confidence_statement, policy_decision,
	action_taken (Razorpay order data or null), human_reviewer, outcome,
	reason_if_not_auto_actioned
	```

	The log is append-only. Every case in `cases_demo.json` has a matching audit entry, including `log_only` cases. The dashboard at `/cases/{case_id}/detail` shows the full audit chain for each case.

	## Demo

	Live deployment: [https://ogrohit-ringguard.hf.space](https://ogrohit-ringguard.hf.space)

	The dashboard shows all 28 detected cases, risk scores, policy decisions, and Razorpay order IDs. Click any case row to see the full evidence bundle, LLM narrative, and audit chain.

	To run the pipeline locally:

	```bash
	# Run the full detection pipeline (LLM calls included)
	python agent/run_demo.py

	# Run live evaluation (produces real numbers, not pre-computed)
	python eval/evaluate.py

	# Start the dashboard
	uvicorn api.main:app --port 7860
	```

	## Roadmap

	- [x] Phase 1 — Detection Core: synthetic data generation, entity resolution, graph construction, two-pass community detection (strong + weak subgraphs). 10/10 rings recovered, 0/5 false-positive stress clusters.
	- [x] Phase 2 — Intelligence Layer: 14-feature extraction, LightGBM risk model (P=R=F1=1.0 on holdout), LangGraph agent orchestration, Groq LLM investigator node, deterministic policy gate, grounding check.
	- [x] Phase 3 — Integration and Submission: RING11 (auto_hold demo case) + BOUNDARY01 (hard ambiguous case), Razorpay Python SDK test-mode integration (create/edit/fetch, notes verified), append-only audit logger, FastAPI dashboard, HuggingFace Spaces deployment (Docker), final evaluation.

	## Acknowledgments

	Built with [LangGraph](https://langchain-ai.github.io/langgraph/), [Groq](https://groq.com) (`openai/gpt-oss-120b`), [LightGBM](https://lightgbm.readthedocs.io/), [NetworkX](https://networkx.org/), [python-louvain](https://python-louvain.readthedocs.io/), and the [Razorpay Python SDK](https://github.com/razorpay/razorpay-python).

	## License

	MIT — see [LICENSE](LICENSE).