File size: 2,759 Bytes
51b3964
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
318732d
 
 
51b3964
 
 
 
 
 
 
 
 
2eb6704
 
 
2f024ae
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2eb6704
51b3964
 
 
318732d
 
51b3964
318732d
 
51b3964
318732d
51b3964
 
 
 
 
 
 
 
318732d
 
 
51b3964
 
 
 
 
 
 
 
 
78c1c50
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
---
title: RFR Knowledge Platform
emoji: 🌊
colorFrom: blue
colorTo: green
sdk: streamlit
app_file: app/app.py
pinned: false
---

# RFR Co-op: LLM-Assisted Managed Retreat RAG Pipeline

A RAG (Retrieval-Augmented Generation) pipeline for extracting and synthesizing insights from managed retreat case study literature, built for the **Retreat From Risk (RFR)** research initiative.

## Setup

```bash
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```

## Usage

```bash
# 1. Set API keys
export ANTHROPIC_API_KEY=your_key_here
export OPENAI_API_KEY=your_key_here

# 2. Convert canonical Ver2 memos to JSON chunks, then build embeddings
python ingest/ingest_pipeline_outputs.py
python ingest/create_summary_chunks.py
python ingest/embed_and_index.py

# 3. Run CLI pipeline
python rag/pipeline.py

# 4. Launch Streamlit web app
streamlit run app/app.py
```

## Hugging Face request intake

The **New Case Study Requests** tab saves each submitted request to Google Drive,
under folder `10n_-uOCT2GXu_G3r1qi5qJr8hHoy-E_y`. In Hugging Face Spaces, use
OAuth credentials for the Google account that should own submitted files, usually
`rfr.rag.uwaterloo@gmail.com`. Set these secrets:

```text
GOOGLE_OAUTH_CLIENT_ID
GOOGLE_OAUTH_CLIENT_SECRET
GOOGLE_OAUTH_REFRESH_TOKEN
```

To generate those values, run:

```bash
python3 -m case_study_pipeline.create_drive_oauth_secrets \
  --credentials-file /path/to/oauth-client.json \
  --space-id UWRFR/UWaterlooRFRChatBot
```

To use a different parent folder, set `RFR_REQUESTS_DRIVE_FOLDER_ID`.

## Ingesting documents

```bash
# Convert canonical case-study memos into section chunks
python ingest/ingest_pipeline_outputs.py

# Refresh overview-summary chunks for broad comparison queries
python ingest/create_summary_chunks.py

# Build the precomputed embedding store used by rag/retriever.py
python ingest/embed_and_index.py
```

## Project structure

```
rfr-rag/
β”œβ”€β”€ data/raw/           # Original PDFs and spreadsheets (not committed)
β”œβ”€β”€ data/extracted/     # Ver2-derived JSON chunks + embedding store
β”œβ”€β”€ ingest/             # Ver2 ingestion, summary chunks, embedding script
β”œβ”€β”€ case_study_pipeline/# Source extraction/audit workflow
β”œβ”€β”€ rag/                # Retriever, prompt builder, pipeline
β”œβ”€β”€ persona/            # Decision-maker persona config
β”œβ”€β”€ questionnaire/      # Guided questionnaire structure
β”œβ”€β”€ app/                # Streamlit interface
β”œβ”€β”€ docs/               # Extraction schema, memos
└── tests/
```

See [CLAUDE.md](CLAUDE.md) for full project context.

For day-to-day website upkeep and adding new case studies, see
[Website Maintenance and New Case Study Guide](docs/website_maintenance.md).