---
title: RingGuard
colorFrom: red
colorTo: yellow
sdk: docker
pinned: true
license: mit
short_description: Graph-based fraud ring detection for Razorpay merchants
---

# RingGuard

![Track](https://img.shields.io/badge/Track-AI%20Risk%20Manager-blue) ![Event](https://img.shields.io/badge/Event-Razorpay%20AI%20Buildathon%202026-orange) ![Status](https://img.shields.io/badge/Status-Phase%203%20Complete-brightgreen) ![Python](https://img.shields.io/badge/Python-3.11-blue) ![License](https://img.shields.io/badge/License-MIT-green)

**Coordinated fraud detection for Razorpay merchants — finding the rings per-order scoring can't see.**

RingGuard analyzes a merchant's order and account data to detect coordinated abuse — accounts secretly linked by shared devices, addresses, or payment instruments, working together to exploit returns, promo codes, or chargebacks. Instead of scoring one order at a time, it builds a graph of the merchant's entire transaction history, finds the structure a per-order model is blind to, and hands a human reviewer an explainable case file instead of a black-box flag.

Built for **Track 02 — AI Risk Manager** at the Razorpay AI Buildathon 2026.

---

## Table of Contents

1. [The Problem](#the-problem)
2. [The Solution](#the-solution)
3. [Why This, Not Thirdwatch](#why-this-not-thirdwatch)
4. [How It Works](#how-it-works)
5. [AI Judgment — Where Each Tool Is Used](#ai-judgment--where-each-tool-is-used)
6. [System Architecture](#system-architecture)
7. [Tech Stack](#tech-stack)
8. [Repository Structure](#repository-structure)
9. [Getting Started](#getting-started)
10. [Usage](#usage)
11. [Evaluation and Metrics](#evaluation-and-metrics)
12. [Known Limitations](#known-limitations)
13. [Policy and Safety Boundaries](#policy-and-safety-boundaries)
14. [Audit Trail](#audit-trail)
15. [Demo](#demo)
16. [Roadmap](#roadmap)
17. [Acknowledgments](#acknowledgments)
18. [License](#license)

---

## The Problem

Cash-on-delivery return-to-origin rates in Indian e-commerce run as high as 30-40%, and coordinated abuse — not isolated bad actors — is a meaningful share of that loss. Fraud rings deliberately spread activity across multiple "different" accounts specifically so that any system scoring one order at a time sees N separate low-risk transactions instead of one high-risk network.

Per-order risk engines, however sophisticated, are structurally blind to this by design — they were never built to look across accounts.

## The Solution

RingGuard is a defensive, AI-assisted investigation agent that:

- **Ingests** a merchant's order, customer, device, address, and payment data (Razorpay Orders/Payments-shaped).
- **Builds a graph** connecting accounts through shared devices, payment instruments, addresses, and contact details.
- **Detects communities** in that graph — clusters of accounts more connected to each other than chance would suggest.
- **Scores each cluster** with a trained LightGBM model using graph structure and behavioral features.
- **Explains the finding** in plain language via an LLM that reads only the structured evidence it is given — never free-reasoning, never inventing facts.
- **Recommends a bounded action** — flag, hold, or escalate — gated by deterministic policy rules, with a human always in the loop for anything above the auto-hold threshold.

It never bans an account or seizes funds. It flags, it explains, and it waits for a human on anything that matters.

## Why This, Not Thirdwatch

Razorpay has owned Thirdwatch since 2019 — a mature fraud/RTO engine scoring individual orders on roughly 300 device and behavioral signals. Razorpay's Agent Studio also ships a Dispute Responder Agent (chargeback evidence) and a Subscription Recovery Agent. None of these look *across* orders and accounts for structure — they operate at the level of one order or one case at a time.

RingGuard is not a better per-order scorer. It is a different problem shape: network-level detection of coordinated behavior that is invisible when you only ever look at one order at a time. That is the entire differentiation argument, and it is stated here first rather than left for a judge to have to ask about.

## How It Works

```
Synthetic order/account stream (Razorpay Orders/Payments-shaped)
        |
        v
Entity resolution — normalize phone/email/address, hash device + payment fields
        |
        v
Graph construction — Customer / Device / Address / PaymentInstrument nodes
        |
        v
Community detection — connected components + Louvain (deterministic, seed=42)
  Two-pass: strong subgraph first (shared device/instrument >= 2 orders),
            weak-only subgraph second (shared address, no strong link)
        |
        v
Feature extraction — cluster size, strong/weak link ratio, order velocity,
                      return rate, chargeback rate, promo usage, exposure (14 features)
        |
        v
LightGBM risk model — single learned probability, no hand-blended scoring
        |
        v
Policy gate (deterministic, v3.0)
   |-- risk >= 0.85 and exposure <= Rs.25,000 ->  bounded auto-hold (Razorpay API, test mode)
   |-- risk >= 0.85 and exposure >  Rs.25,000 ->  escalate, no auto-action
   |-- 0.5 <= risk < 0.85                    ->  escalate to human review queue
   +-- risk < 0.5                            ->  log only
        |
        v
Investigator agent (LangGraph + Groq LLM) — evidence bundle in, case narrative out
        |
        v
Razorpay test-mode action (order.create + order.edit notes + order.fetch verify)
        |
        v
Audit logger — every stage, every decision, timestamped (audit_log.jsonl)
```

A cluster built entirely from **weak links** (e.g., three accounts sharing only an address — plausible for a household) is designed to score low. Distinguishing that from a **strong-link** cluster (shared device fingerprint used in 2+ orders) is what keeps false positives down, and it is built into the feature set from the start.

## AI Judgment — Where Each Tool Is Used

| Stage | Tool | Why |
|-------|------|-----|
| Graph construction, edge rules, policy gate | Deterministic code | Money-adjacent decisions need to be boring and auditable — no model judgment here |
| Community detection, clustering | Classical graph algorithms (Louvain + connected components) | Structural pattern detection, not a language problem |
| Ring risk scoring | LightGBM (supervised, tabular) | Faster, cheaper, and more auditable than an LLM for a numeric classification task |
| Case narrative and explanation | LLM (Groq — `openai/gpt-oss-120b`) | The one place where synthesizing evidence into something a reviewer can act on quickly genuinely benefits from language generation |
| Final action | Deterministic policy gate, human-approved above threshold | The LLM never has authority to move money or take action on its own |

## System Architecture

```mermaid
flowchart TD
    A[Synthetic Order Stream] --> B[Entity Resolution]
    B --> C[Graph Construction - NetworkX]
    C --> D[Two-Pass Community Detection]
    D --> E[Feature Extraction - 14 features]
    E --> F[LightGBM Risk Model]
    F --> G{Deterministic Policy Gate}
    G -->|risk >= 0.85, exposure <= 25k INR| H[Auto-Hold via Razorpay API]
    G -->|risk >= 0.85, exposure > 25k INR| I[Escalate to Human Review]
    G -->|0.5 <= risk < 0.85| I
    G -->|risk < 0.5| J[Log Only]
    H --> K[Investigator Agent - LangGraph + Groq LLM]
    I --> K
    K --> L[Case Narrative + Confidence Statement]
    L --> M[Audit Log + FastAPI Dashboard]
    J --> M
```

## Tech Stack

| Layer | Choice |
|-------|--------|
| Language / API | Python 3.11, FastAPI |
| Graph | NetworkX + python-louvain |
| Scoring | LightGBM |
| Agent orchestration | LangGraph |
| LLM | Groq (`openai/gpt-oss-120b`, plain chat completion — no tool calls) |
| Razorpay integration | Official Razorpay Python SDK v2.0.1 (test-mode, verified) |
| Synthetic data | Faker |
| Dashboard | Server-side HTML rendered by FastAPI |
| Deployment | HuggingFace Spaces (Docker) |

No paid dependencies anywhere in this stack.

## Repository Structure

```
ringguard/
|-- README.md
|-- requirements.txt
|-- .env.example
|-- Dockerfile
|-- data/
|   |-- generate_synthetic_data.py     <- 500 customers, 10 rings, seed=42
|   |-- augment_demo_dataset.py        <- adds RING11 + BOUNDARY01
|   +-- synthetic/
|       |-- customers.json             <- 506 customers
|       |-- orders.json                <- 1,215 orders
|       |-- ground_truth.json          <- 11 rings
|       |-- candidate_clusters.json    <- 28 candidate clusters
|       |-- scored_clusters.json       <- 28 clusters with risk_probability
|       +-- cases_demo.json            <- 28 cases with LLM narratives + Razorpay action_taken
|-- graph/
|   |-- build_graph.py                 <- entity resolution + heterogeneous graph
|   +-- detect_communities.py          <- two-pass community detection
|-- scoring/
|   |-- features.py                    <- 14-feature extraction
|   |-- train_model.py                 <- LightGBM training (training dataset only)
|   |-- score_demo.py                  <- scores demo dataset
|   +-- model.lgb                      <- trained model (do not retrain without reason)
|-- agent/
|   |-- policy.py                      <- deterministic policy gate (v3.0)
|   |-- investigator.py                <- Groq LLM investigator node
|   |-- graph.py                       <- LangGraph StateGraph orchestration
|   +-- run_demo.py                    <- end-to-end demo runner
|-- integration/
|   +-- razorpay_actions.py            <- Razorpay test-mode: create + edit + fetch orders
|-- api/
|   +-- main.py                        <- FastAPI dashboard (/, /cases, /health)
|-- eval/
|   |-- evaluate.py                    <- holdout evaluation script
|   +-- eval_results.json              <- real evaluation numbers (do not edit manually)
+-- audit/
    |-- logger.py                      <- append-only structured audit logger
    |-- generate_audit_log.py          <- bootstraps audit_log.jsonl from cases_demo.json
    +-- audit_log.jsonl                <- full audit trail, 28 entries
```

## Getting Started

### Prerequisites

- Python 3.11+
- A free [Groq API key](https://console.groq.com)
- Razorpay test-mode API keys (`rzp_test_...`) from the [Razorpay Dashboard](https://dashboard.razorpay.com)

### Installation

```bash
git clone https://github.com/rohitdecodes/ringguard.git
cd ringguard
python -m venv venv
source venv/bin/activate       # Windows: venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env            # fill in GROQ_API_KEY, RAZORPAY_KEY_ID, RAZORPAY_KEY_SECRET
```

### Environment variables

```
GROQ_API_KEY=your_groq_api_key
RAZORPAY_KEY_ID=rzp_test_...
RAZORPAY_KEY_SECRET=...
```

Keys are read from `.env` at runtime. `.env` is gitignored — never commit credentials.

## Usage

The pipeline runs in order. Each step can be re-run independently.

```bash
# 1. Generate base synthetic data (seed=42, deterministic)
python data/generate_synthetic_data.py

# 2. Augment with RING11 (auto_hold case) + BOUNDARY01 (hard ambiguous case)
python data/augment_demo_dataset.py

# 3. Build the heterogeneous graph
python graph/build_graph.py

# 4. Detect candidate clusters (two-pass: strong + weak-only)
python graph/detect_communities.py

# 5. Score every cluster with the trained LightGBM model
python scoring/score_demo.py

# 6. Run the full LangGraph agent pipeline (LLM calls for escalate/auto_hold cases)
python agent/run_demo.py

# 7. Execute Razorpay test-mode actions for all actioned cases
python integration/razorpay_actions.py

# 8. Regenerate audit log from finalized cases
python audit/generate_audit_log.py

# 9. Run held-out evaluation
python eval/evaluate.py

# 10. Launch the dashboard
uvicorn api.main:app --port 7860
# Open http://localhost:7860
```

**Deployed dashboard:** [https://ogrohit-ringguard.hf.space](https://ogrohit-ringguard.hf.space)  
Health check: [https://ogrohit-ringguard.hf.space/health](https://ogrohit-ringguard.hf.space/health)

## Evaluation and Metrics

Evaluated on a held-out cluster-level split (no ring's accounts appear in both train and test). Fixed seed=42, stratified 80/20 split, 22 held-out clusters.

| Metric | Value |
|--------|-------|
| **Precision** | **1.0000** |
| **Recall** | **1.0000** |
| **F1** | **1.0000** |
| False-positive cost (held orders wrongly flagged) | **Rs. 0.00** |
| Inference latency (feature extraction + model) | **< 0.1 ms/cluster** |

Holdout breakdown: 12 positive (fraud rings), 10 negative (benign clusters).

### Policy action breakdown — holdout set

| Action | Count |
|--------|-------|
| auto_hold | 0 |
| escalate | 12 |
| log_only | 10 |

All 12 positive clusters score >= 0.85 but have total exposure above Rs. 25,000 (realistic ring sizes accumulate large amounts), so they route to `escalate` rather than `auto_hold`. This is correct behavior.

### Demo dataset — 28 clusters (post-augmentation)

| Action | Count | Notes |
|--------|-------|-------|
| auto_hold | 1 | RING11 — risk 98.67%, exposure Rs. 19,800 |
| escalate | 10 | Original 10 rings — risk 99.1-99.7%, large exposure |
| log_only | 17 | Includes BOUNDARY01 (see Known Limitations) |

### Feature importances (top 5)

| Feature | Importance |
|---------|-----------|
| avg_order_amount | 145 |
| chargeback_rate | 120 |
| strong_link_count | 90 |
| return_rate | 73 |
| order_velocity | 37 |

## Known Limitations

**Read this before drawing conclusions from the precision/recall numbers above.**

The current P=R=F1=1.0 figures reflect that the synthetic training data does not yet contain genuinely ambiguous examples. The model has not been tested on a hard case — one where real human experts would disagree.

**1. Training distribution boundary.**
The model's top feature is `avg_order_amount` (importance=145). Positive training examples have avg_order_amount >= Rs. 3,249; all negatives are below Rs. 3,171. This creates a near-hard threshold in a learned model. Real rings could fall on either side of it.

**2. BOUNDARY01 — the designed hard case.**
BOUNDARY01 (CUST0504-CUST0506) was rebuilt in Phase 3 as a genuine structural challenge: 3 accounts sharing device DEV0402 with 2 orders each (count >= 2, forming a `strong_anchored` cluster with strong_link_count=3 — the same structural signature as a confirmed fraud ring). Yet: zero chargebacks, low return rate (1/9), modest avg_order_amount (~Rs. 1,183). Model output: **risk=2.08%, log_only**.

The model is not wrong. Zero chargebacks and low order values legitimately suppress the score — that is precisely what the feature weights say to do. But this also means: if a real low-value shared-device cluster were fraudulent, the current model would miss it. This is an honest limitation of the training distribution.

**3. Synthetic data only.**
All evaluation is against Faker-generated data. Real merchants have messier entity resolution, more diverse ring structures, and ambiguous cases the current training data cannot represent. Real-world P/R/F1 should be expected to be materially lower.

**4. Auto-hold threshold calibration.**
The original Rs. 5,000 auto-hold cap was incompatible with the training distribution (a minimal ring totals ~Rs. 19,800). Updated to Rs. 25,000 in Phase 3 (v3.0). The threshold was never validated against real production exposure data.

**5. LLM model substitution.**
`llama-3.3-70b-versatile` (planned) was unavailable on the Groq account. `groq/compound-mini` was used first; the final pipeline uses `openai/gpt-oss-120b`. Both produce valid narratives with zero grounding failures across all 11 LLM calls. Model is configurable via `GROQ_MODEL` environment variable.

Real-world performance validation requires labeled production data. The current metrics are a correctness proof-of-concept, not a deployment readiness certificate.

## Policy and Safety Boundaries

RingGuard is **defense-only**, per Track 02's rules:

- It can flag, hold, and escalate. It cannot ban accounts, seize funds, or take any irreversible action.
- Auto-hold only fires at risk >= 0.85 AND total cluster exposure <= Rs. 25,000. Everything above that threshold, or in the mid-confidence band (0.5-0.85), goes to a human review queue — never auto-actioned.
- The LLM has no authority to trigger an action. It explains a decision the deterministic policy gate already made.
- The LLM (`openai/gpt-oss-120b`) is called as a plain chat completion with no `tools` parameter — it has no web-search or code-execution access. It reads only the structured evidence bundle.
- All Razorpay actions are test-mode only (`rzp_test_...` keys enforced in code). No real money moves.

## Audit Trail

Every case writes one JSON record to `audit/audit_log.jsonl` containing:

```
case_id, timestamp, cluster_members, evidence_bundle, graph_score_inputs (all 14 features),
lightgbm_probability, llm_narrative, llm_confidence_statement, policy_decision,
action_taken (Razorpay order data or null), human_reviewer, outcome,
reason_if_not_auto_actioned
```

The log is append-only. Every case in `cases_demo.json` has a matching audit entry, including `log_only` cases. The dashboard at `/cases/{case_id}/detail` shows the full audit chain for each case.

## Demo

**Live deployment:** [https://ogrohit-ringguard.hf.space](https://ogrohit-ringguard.hf.space)

The dashboard shows all 28 detected cases, risk scores, policy decisions, and Razorpay order IDs. Click any case row to see the full evidence bundle, LLM narrative, and audit chain.

**To run the pipeline locally:**

```bash
# Run the full detection pipeline (LLM calls included)
python agent/run_demo.py

# Run live evaluation (produces real numbers, not pre-computed)
python eval/evaluate.py

# Start the dashboard
uvicorn api.main:app --port 7860
```

## Roadmap

- [x] **Phase 1 — Detection Core:** synthetic data generation, entity resolution, graph construction, two-pass community detection (strong + weak subgraphs). 10/10 rings recovered, 0/5 false-positive stress clusters.
- [x] **Phase 2 — Intelligence Layer:** 14-feature extraction, LightGBM risk model (P=R=F1=1.0 on holdout), LangGraph agent orchestration, Groq LLM investigator node, deterministic policy gate, grounding check.
- [x] **Phase 3 — Integration and Submission:** RING11 (auto_hold demo case) + BOUNDARY01 (hard ambiguous case), Razorpay Python SDK test-mode integration (create/edit/fetch, notes verified), append-only audit logger, FastAPI dashboard, HuggingFace Spaces deployment (Docker), final evaluation.

## Acknowledgments

Built with [LangGraph](https://langchain-ai.github.io/langgraph/), [Groq](https://groq.com) (`openai/gpt-oss-120b`), [LightGBM](https://lightgbm.readthedocs.io/), [NetworkX](https://networkx.org/), [python-louvain](https://python-louvain.readthedocs.io/), and the [Razorpay Python SDK](https://github.com/razorpay/razorpay-python).

## License

MIT — see [LICENSE](LICENSE).