Spaces:
Sleeping
Download case_study_pipeline/README.md from UWRFR/UWaterlooRFRChatBot: direct link, hf CLI and curl.
- Browser
- Download file 12 kB
-
https://huggingface.co/spaces/UWRFR/UWaterlooRFRChatBot/resolve/main/case_study_pipeline/README.md
- Command line
-
hf download hf://spaces/UWRFR/UWaterlooRFRChatBot/case_study_pipeline/README.md
-
curl -L -o README.md https://huggingface.co/spaces/UWRFR/UWaterlooRFRChatBot/resolve/main/case_study_pipeline/README.md
A newer version of the Streamlit SDK is available: 1.67.0
Case Study Processing Pipeline
Automates the 4-step managed retreat case study workflow:
- Claude generates a narrative case-study questionnaire document (Version 1) from the RFR questionnaire, codebook, cross-cutting themes guide, note-taking guidelines, and a folder of source documents for one case study.
- ChatGPT audits Version 1 against the sources and codebook (Check 1), producing a scored report (0-100).
- Claude revises Version 1 into Version 2 using the Check 1 feedback.
- ChatGPT audits Version 2 in a fresh, standalone call with no memory of Check 1 (Check 2), producing another scored report so you can compare Check 1 vs. Check 2 to measure improvement.
The default workflow still uses Claude for drafting/revision and ChatGPT for
audits. For shared-account use, pass --llm-provider claude or
--llm-provider openai to run all four steps with one provider/key.
Setup
- Make sure
rfr-rag/.envhas the key or keys needed for your workflow:
ForANTHROPIC_API_KEY=... OPENAI_API_KEY=...--llm-provider claude, onlyANTHROPIC_API_KEYis required. For--llm-provider openai, onlyOPENAI_API_KEYis required. The default mixed workflow requires both. - Install dependencies (adds
fpdf2to the existing requirements):pip install -r requirements.txt
Context documents
The context/ folder contains the shared reference material used in every run:
ai_questionnaire.mdβ the RFR coding questionscodebook.mdβ the RFR codebook (categories, codes, descriptions)cross_cutting_guide.mdβ the RFR cross-cutting themes guidenote_taking_guidelines.mdβ the RFR note-taking guidelines for LLMs
These were generated from the RFR Drive docs. If the team updates those docs, re-export them (File > Download > Plain text for Google Docs, or re-export the codebook spreadsheet) and overwrite the corresponding file here.
Usage
For any case study, put its source documents (PDFs of papers/reports; .txt/.md
also supported) in their own folder, then run:
python -m case_study_pipeline.run_case_study --case-folder data/raw/CS6
This writes to data/raw/CS6/pipeline_output/ by default:
CS6_Ver1.md/CS6_Ver1.pdfβ initial questionnaire documentCS6_Check1_report.md/.pdfβ Check 1 audit report + gradeCS6_Ver2.md/.pdfβ revised questionnaire documentCS6_Check2_report.md/.pdfβ Check 2 audit report + grade
A summary with both grades and the change is printed at the end.
Useful options
python -m case_study_pipeline.run_case_study \
--case-folder data/raw/CS6 \
--case-id CS6 \ # defaults to the folder name
--output-dir outputs/CS6 \ # defaults to <case-folder>/pipeline_output/
--llm-provider claude \ # mixed, claude, or openai
--claude-model claude-sonnet-4-6 \
--openai-model gpt-5.4-mini \
--max-tokens 8000 \ # max output tokens per LLM call
--max-chars-per-source 120000 \ # truncate very long source files (0 = no limit)
--force # overwrite existing output files without asking
By default, if an output PDF already exists you'll be asked to confirm before it's
overwritten (the .md files are always (re)written so you don't lose the run, but
existing PDFs are kept unless you confirm or pass --force).
Shared one-provider workflow
Use this when the team wants a simple process that only needs one LLM account. The Hugging Face app can collect requests and uploaded papers without any API calls; a project member then runs the extraction locally:
python3 -m case_study_pipeline.run_case_study \
--case-folder data/raw/CSxx-ShortName \
--case-id CSxx \
--llm-provider claude \
--max-chars-per-source 150000 \
--force
python3 -m case_study_pipeline.select_best_run --case-id CSxx
python3 ingest/ingest_pipeline_outputs.py --case-id CSxx
python3 ingest/create_summary_chunks.py
python3 ingest/embed_and_index.py
Running just the audit step (Check 1 / Check 2 only)
Sometimes Version 1 or Version 2 of a case study's questionnaire document is written by hand (e.g. Claude generated it directly in a chat, without calling the Anthropic API). In that situation you don't need to re-run the whole pipeline -- you just need ChatGPT/OpenAI to audit the existing document and produce a scored report.
# Audit data/raw/CS11/pipeline_output/CS11_Ver1.md
python -m case_study_pipeline.run_check1 --case-id CS11
# Audit data/raw/CS11/pipeline_output/CS11_Ver2.md (also compares against the
# Check 1 grade, if a CS11_Check1_report.md is already there)
python -m case_study_pipeline.run_check2 --case-id CS11
--case-id is the only required option -- it's how you tell the script which case
study to audit (e.g. CS6, CS11, ...). Both scripts accept the same --case-folder,
--output-dir, --openai-model, --max-tokens, and --max-chars-per-source options
as run_case_study.py. Each writes <CASE_ID>_Check1_report.{md,pdf} or
<CASE_ID>_Check2_report.{md,pdf} into the case study's pipeline_output/ folder.
Requires OPENAI_API_KEY (in rfr-rag/.env or the environment) -- no Anthropic key
needed for these two scripts.
High-precision paper discovery
Use discover_sources.py before running the extraction pipeline when a case
needs source cleanup, DOI lookup, or supplemental paper discovery.
python3 -m case_study_pipeline.discover_sources --case-id CS36
python3 -m case_study_pipeline.discover_sources \
--case-id CS36 \
--alias "Pitt County" \
--alias "Hurricane Floyd"
The discovery command is deliberately conservative:
- Existing
sources.txt/dois.txtentries are treated as accepted seed sources, but still resolved through scholarly metadata APIs where possible. - New search results are not silently accepted. They are classified as
review_requiredorrejected_by_identity_filter. - Direct case evidence must match the core place identity, such as the village,
city, resettlement name, province/state, and country from the spreadsheet or
case_meta.json. - National/provincial/local legal and policy papers can be surfaced as
candidate_policy_context, but they stay separate from direct case evidence.
The command writes:
data/raw/<CASE>/
βββ sources.json # structured accepted seed sources
βββ paper_candidates.json # all candidates with evidence/scoring
βββ paper_discovery_report.md # human-readable review report
Recommended review rule: only use accepted_seed sources and manually approved
review_required candidates in the LLM extraction. Do not use rejected/background
papers unless a human reclassifies them after reading the full text.
Automated source acquisition
After discovery, run acquire_sources.py to populate the case folder with source
files the extraction pipeline can read:
python3 -m case_study_pipeline.acquire_sources --case-id CS39
python3 -m case_study_pipeline.acquire_sources \
--case-id CS38 \
--extra-url "https://www.academia.edu/37237146/Heritage_and_Postdisaster_Recovery_Indigenous_Community_Resilience"
This command prefers publisher/open-access PDFs, then falls back to a
provenance-labelled .md capture when the PDF is blocked but the article/report
text is publicly reachable. That fallback is intentional: the extraction
pipeline reads .pdf, .txt, and .md, so a full-text markdown capture is
better than stopping the pipeline on a missing PDF. Use the generated
source_acquisition_report.json to see what was acquired.
If a publisher PDF is paywalled and no public full-text route is available, the
script will not bypass access controls. In that case, the non-manual route is to
add an authorized/open URL with --extra-url, or use the EZproxy/browser flow
below for institutional access.
Session notes
session_notes/ holds one-off "continue this work" notes written for specific case
studies (for example, a note describing how CS11's Version 1 document was produced
by hand). These are historical records, not part of the reusable pipeline -- the
scripts above work for any case study ID.
Notes
- Source PDFs are read with
pdfplumber(already a project dependency). - Output PDFs are generated with
fpdf2from the LLM's narrative text. Smart quotes/dashes are converted to plain ASCII for compatibility with the built-in PDF fonts. - "No memory" for Check 2: each ChatGPT call in this pipeline is a single
standalone request (system + user message only, no prior turns or
previous_response_id). The Chat Completions API has no per-conversation memory by default, so Check 2 is independent of Check 1 by construction β there's no separate toggle to flip.
Fetching paywalled papers via UWaterloo EZproxy (Chrome-assisted)
For papers that fetch_papers.py can't get through Unpaywall / Sci-Hub /
Semantic Scholar (i.e. paywalled, behind a major publisher), use UWaterloo's
institutional library access. This cannot be fully scripted β WatIAM
requires Duo two-factor authentication, which only a human can complete. The
workflow below uses the Claude-in-Chrome browser extension to drive an
already-logged-in browser session, with the actual login/Duo step done by
the user once per browser session (sessions last several hours).
One-time setup (per machine)
In Chrome, go to chrome://settings/downloads and set:
- Location:
/Users/AnnaZhou/rfr-rag/data/raw/_inbox - "Ask where to save each file before downloading" β turn this OFF
This makes every PDF download save automatically into the repo's inbox
folder with no save dialog, so an agent driving the browser can trigger a
download and immediately find the file on disk (this folder is gitignored
via data/raw/ in .gitignore).
Recurring procedure (every session, every case study)
- Run
fetch_papers.pyfirst β it handles every paper it can without needing institutional access:python3 -m case_study_pipeline.fetch_papers --case-id CS13 - For DOIs that still fail, use UWaterloo EZproxy through Chrome:
- Navigate to
https://login.proxy.lib.uwaterloo.ca/login?qurl=<any target URL>and click "Using WatIAM". Ask the user to enter credentials and approve Duo themselves β never enter passwords for them. - Once logged in, for each remaining DOI:
a. Navigate to
https://doi.org/<DOI>β this redirects to the publisher's real article URL. b. Convert that URL to its EZproxy form using the dash-domain rule (seecase_study_pipeline/ezproxy.py):link.springer.comβlink-springer-com.proxy.lib.uwaterloo.caagupubs.onlinelibrary.wiley.comβagupubs-onlinelibrary-wiley-com.proxy.lib.uwaterloo.ca(any publisher domain follows this same pattern) c. Navigate to the EZproxy URL, usefindto locate the "Download PDF" link, and click it. The PDF lands indata/raw/_inbox/automatically (no dialog, because of the one-time Chrome setting above). d. Check the inbox and file it away:
This moves the PDF intopython3 -m case_study_pipeline.sort_inbox --list python3 -m case_study_pipeline.sort_inbox \ --filename "<exact name in inbox>" \ --case-id CS13 \ --rename "Author_Year_Journal_FULLTEXT.pdf"data/raw/CS13/and uploads it to the correct Drive folder automatically (viadrive_folders.json+drive_upload.pyβ same OAuth token already set up for the pipeline's own output uploads).
- Navigate to
- The EZproxy session stays alive in the browser for the rest of the working session β no need to re-login for subsequent papers/case studies unless the session expires (multi-hour timeout) or Chrome is restarted.