UWaterlooRFRChatBot / README.md
Anna Zhou
Use OAuth for Drive request uploads
2f024ae
|
Raw History Blame Contribute Delete
2.76 kB

A newer version of the Streamlit SDK is available: 1.67.0

Upgrade
metadata
title: RFR Knowledge Platform
emoji: 🌊
colorFrom: blue
colorTo: green
sdk: streamlit
app_file: app/app.py
pinned: false

RFR Co-op: LLM-Assisted Managed Retreat RAG Pipeline

A RAG (Retrieval-Augmented Generation) pipeline for extracting and synthesizing insights from managed retreat case study literature, built for the Retreat From Risk (RFR) research initiative.

Setup

python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Usage

# 1. Set API keys
export ANTHROPIC_API_KEY=your_key_here
export OPENAI_API_KEY=your_key_here

# 2. Convert canonical Ver2 memos to JSON chunks, then build embeddings
python ingest/ingest_pipeline_outputs.py
python ingest/create_summary_chunks.py
python ingest/embed_and_index.py

# 3. Run CLI pipeline
python rag/pipeline.py

# 4. Launch Streamlit web app
streamlit run app/app.py

Hugging Face request intake

The New Case Study Requests tab saves each submitted request to Google Drive, under folder 10n_-uOCT2GXu_G3r1qi5qJr8hHoy-E_y. In Hugging Face Spaces, use OAuth credentials for the Google account that should own submitted files, usually rfr.rag.uwaterloo@gmail.com. Set these secrets:

GOOGLE_OAUTH_CLIENT_ID
GOOGLE_OAUTH_CLIENT_SECRET
GOOGLE_OAUTH_REFRESH_TOKEN

To generate those values, run:

python3 -m case_study_pipeline.create_drive_oauth_secrets \
  --credentials-file /path/to/oauth-client.json \
  --space-id UWRFR/UWaterlooRFRChatBot

To use a different parent folder, set RFR_REQUESTS_DRIVE_FOLDER_ID.

Ingesting documents

# Convert canonical case-study memos into section chunks
python ingest/ingest_pipeline_outputs.py

# Refresh overview-summary chunks for broad comparison queries
python ingest/create_summary_chunks.py

# Build the precomputed embedding store used by rag/retriever.py
python ingest/embed_and_index.py

Project structure

rfr-rag/
β”œβ”€β”€ data/raw/           # Original PDFs and spreadsheets (not committed)
β”œβ”€β”€ data/extracted/     # Ver2-derived JSON chunks + embedding store
β”œβ”€β”€ ingest/             # Ver2 ingestion, summary chunks, embedding script
β”œβ”€β”€ case_study_pipeline/# Source extraction/audit workflow
β”œβ”€β”€ rag/                # Retriever, prompt builder, pipeline
β”œβ”€β”€ persona/            # Decision-maker persona config
β”œβ”€β”€ questionnaire/      # Guided questionnaire structure
β”œβ”€β”€ app/                # Streamlit interface
β”œβ”€β”€ docs/               # Extraction schema, memos
└── tests/

See CLAUDE.md for full project context.

For day-to-day website upkeep and adding new case studies, see Website Maintenance and New Case Study Guide.