Spaces:
Runtime error
Runtime error
Upload README (1).md
Browse files- README (1).md +243 -0
README (1).md
ADDED
|
@@ -0,0 +1,243 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: PsycheOS
|
| 3 |
+
emoji: π§
|
| 4 |
+
colorFrom: purple
|
| 5 |
+
colorTo: pink
|
| 6 |
+
sdk: streamlit
|
| 7 |
+
sdk_version: 1.32.0
|
| 8 |
+
app_file: app/streamlit_app.py
|
| 9 |
+
pinned: true
|
| 10 |
+
license: mit
|
| 11 |
+
tags:
|
| 12 |
+
- reinforcement-learning
|
| 13 |
+
- mental-health
|
| 14 |
+
- multi-agent
|
| 15 |
+
- openenv
|
| 16 |
+
- grpo
|
| 17 |
+
- hackathon
|
| 18 |
+
---
|
| 19 |
+
|
| 20 |
+
# π§ PsycheOS β Mental Health Crisis Operating System
|
| 21 |
+
|
| 22 |
+
> **Meta PyTorch OpenEnv Hackathon Γ Scaler School of Technology β Grand Finale**
|
| 23 |
+
> Solo submission Β· Mesa Sarah Vasantha Zephyr Β· April 25β26, 2026 Β· Bangalore
|
| 24 |
+
|
| 25 |
+
---
|
| 26 |
+
|
| 27 |
+
## π Links
|
| 28 |
+
|
| 29 |
+
| Resource | URL |
|
| 30 |
+
|----------|-----|
|
| 31 |
+
| π Live HF Space | https://huggingface.co/spaces/mesazephyr/psycheos |
|
| 32 |
+
| π Mini-blog (HuggingFace) | _[Add after training β April 25]_ |
|
| 33 |
+
| π₯ Demo video (YouTube, <2 min) | _[Add after training β April 25]_ |
|
| 34 |
+
| π» Training notebook (Colab) | [](https://colab.research.google.com/github/mesazephyr/psycheos/blob/main/training/colab_training.ipynb) |
|
| 35 |
+
| π€ LoRA weights | _[Add after training β mesazephyr/psycheos-lora]_ |
|
| 36 |
+
|
| 37 |
+
---
|
| 38 |
+
|
| 39 |
+
## The Problem
|
| 40 |
+
|
| 41 |
+
**197 million** Indians live with a mental disorder.
|
| 42 |
+
**9,000** psychiatrists exist for a population of 1.4 billion.
|
| 43 |
+
That is **one doctor for every 155,000 people** who need one.
|
| 44 |
+
**83%** receive zero care.
|
| 45 |
+
|
| 46 |
+
The gap is not just statistical β it is deadly. Delayed crisis identification and zero continuity between care interactions cost lives. PsycheOS is built to close that gap.
|
| 47 |
+
|
| 48 |
+
---
|
| 49 |
+
|
| 50 |
+
## What is PsycheOS?
|
| 51 |
+
|
| 52 |
+
PsycheOS is simultaneously:
|
| 53 |
+
|
| 54 |
+
1. **A real deployable app** β open it in your browser right now, talk to it
|
| 55 |
+
2. **An OpenEnv RL training environment** β Gymnasium-compatible POMDP with measurable reward signals
|
| 56 |
+
3. **A self-improving multi-agent system** β gets measurably better at crisis detection through GRPO + curriculum self-play
|
| 57 |
+
|
| 58 |
+
---
|
| 59 |
+
|
| 60 |
+
## How the Environment Works
|
| 61 |
+
|
| 62 |
+
### What the agent sees (observation space)
|
| 63 |
+
|
| 64 |
+
```
|
| 65 |
+
{
|
| 66 |
+
"message": "<patient's raw text>",
|
| 67 |
+
"session_history": [last 3 turns with distress labels],
|
| 68 |
+
"turn": <int>,
|
| 69 |
+
"session": <int>
|
| 70 |
+
}
|
| 71 |
+
```
|
| 72 |
+
|
| 73 |
+
### What the agent does (action space)
|
| 74 |
+
|
| 75 |
+
```
|
| 76 |
+
{
|
| 77 |
+
"distress_level": <1β5>, # classify severity
|
| 78 |
+
"response": "<text>", # generate empathetic reply
|
| 79 |
+
"escalate": <true/false> # route to human counselor?
|
| 80 |
+
}
|
| 81 |
+
```
|
| 82 |
+
|
| 83 |
+
### What the agent gets rewarded for
|
| 84 |
+
|
| 85 |
+
| Signal | Weight | Description |
|
| 86 |
+
|--------|--------|-------------|
|
| 87 |
+
| Detection F1 | 40% | Accuracy of distress severity classification |
|
| 88 |
+
| Empathy DPO | 40% | Psychiatrist preference margin on responses |
|
| 89 |
+
| Escalation P/R | 20% | Crisis routing precision β **10x penalty on false negatives** |
|
| 90 |
+
|
| 91 |
+
The reward is hard to game. An agent that always escalates scores β1 on false positives. An agent that never escalates scores β10 on every missed crisis. The only way to score well is to actually solve the task.
|
| 92 |
+
|
| 93 |
+
---
|
| 94 |
+
|
| 95 |
+
## The 5 Agents
|
| 96 |
+
|
| 97 |
+
```
|
| 98 |
+
User message
|
| 99 |
+
β
|
| 100 |
+
βΌ
|
| 101 |
+
βββββββββββββββ ββββββββββββββββ βββββββββββββββββββββ
|
| 102 |
+
β Memory ββββββΆβ Detector ββββββΆβ Empathy Agent β
|
| 103 |
+
β Retrieval β β Agent (1β5) β β (DPO fine-tuned) β
|
| 104 |
+
βββββββββββββββ ββββββββββββββββ βββββββββββββββββββββ
|
| 105 |
+
β
|
| 106 |
+
βΌ
|
| 107 |
+
ββββββββββββββββ βββββββββββββββββββββ
|
| 108 |
+
β Oversight βββββββ Crisis Escalator β
|
| 109 |
+
β Agent β β Agent β
|
| 110 |
+
ββββββββββββββββ βββββββββββββββββββββ
|
| 111 |
+
β
|
| 112 |
+
βΌ
|
| 113 |
+
Final response
|
| 114 |
+
(or crisis escalation)
|
| 115 |
+
```
|
| 116 |
+
|
| 117 |
+
| Agent | Role | Bonus prize target |
|
| 118 |
+
|-------|------|--------------------|
|
| 119 |
+
| π **Detector** | Classifies distress 1β5 from free text | β |
|
| 120 |
+
| π¬ **Empathy** | Generates responses (DPO fine-tuned on psychiatrist preferences) | β |
|
| 121 |
+
| π¨ **Crisis Escalator** | Routes to human counselor (10x FN penalty) | β |
|
| 122 |
+
| π **Oversight** | Monitors all agents for contradictions in real time | Fleet AI |
|
| 123 |
+
| π **Curriculum** | Auto-generates harder episodes when agents plateau | Snorkel AI |
|
| 124 |
+
|
| 125 |
+
---
|
| 126 |
+
|
| 127 |
+
## Self-Improvement Loop
|
| 128 |
+
|
| 129 |
+
```
|
| 130 |
+
Episode β Reward β Curriculum tracks rolling avg
|
| 131 |
+
β
|
| 132 |
+
improvement < 2% over 50 episodes?
|
| 133 |
+
YES βββββββββββββββββββββββββββΆ harder episodes generated
|
| 134 |
+
NO βββββββββββββββββββοΏ½οΏ½βββββββΆ continue current difficulty
|
| 135 |
+
β
|
| 136 |
+
DPO fine-tune empathy agent
|
| 137 |
+
on psychiatrist preference pairs
|
| 138 |
+
(preferences shift each epoch β
|
| 139 |
+
satisfies Snorkel AI requirement)
|
| 140 |
+
β
|
| 141 |
+
LoRA adapters updated via GRPO
|
| 142 |
+
```
|
| 143 |
+
|
| 144 |
+
---
|
| 145 |
+
|
| 146 |
+
## Training Results
|
| 147 |
+
|
| 148 |
+
> β οΈ _Placeholder β replace with real numbers and plots after onsite training April 25._
|
| 149 |
+
|
| 150 |
+

|
| 151 |
+
_Fig 1. Composite reward, detection F1, and DPO win-rate over 500 training episodes (baseline vs trained on same axes)._
|
| 152 |
+
|
| 153 |
+
| Metric | Baseline (untrained) | After training |
|
| 154 |
+
|--------|---------------------|----------------|
|
| 155 |
+
| Detection F1 | 0.61 | β |
|
| 156 |
+
| Escalation Precision | 0.71 | β |
|
| 157 |
+
| DPO Win-Rate vs baseline | 51% | β |
|
| 158 |
+
| Avg Composite Reward | 0.35 | β |
|
| 159 |
+
|
| 160 |
+
---
|
| 161 |
+
|
| 162 |
+
## Hackathon Theme Coverage
|
| 163 |
+
|
| 164 |
+
| Theme | Coverage |
|
| 165 |
+
|-------|----------|
|
| 166 |
+
| Theme 1 β Multi-Agent Interactions | β
5 agents cooperating, competing, overriding each other |
|
| 167 |
+
| Theme 3 β World Modeling (Personalized) | β
Dynamic patient world across multi-session POMDP |
|
| 168 |
+
| Theme 4 β Self-Improvement | β
Curriculum self-play + DPO adaptation per epoch |
|
| 169 |
+
| Theme 5 β Wild Card | β
First mental health crisis RL environment |
|
| 170 |
+
|
| 171 |
+
**Bonus prizes targeted:** Fleet AI Β· Snorkel AI Β· Halluminate
|
| 172 |
+
|
| 173 |
+
---
|
| 174 |
+
|
| 175 |
+
## Run Locally
|
| 176 |
+
|
| 177 |
+
```bash
|
| 178 |
+
git clone https://huggingface.co/spaces/mesazephyr/psycheos
|
| 179 |
+
cd psycheos
|
| 180 |
+
pip install -r requirements.txt
|
| 181 |
+
streamlit run app/streamlit_app.py
|
| 182 |
+
```
|
| 183 |
+
|
| 184 |
+
## Train (onsite with HF compute credits)
|
| 185 |
+
|
| 186 |
+
```bash
|
| 187 |
+
# GRPO training loop
|
| 188 |
+
python training/grpo_train.py --episodes 1000 --epochs 3
|
| 189 |
+
|
| 190 |
+
# DPO fine-tuning β empathy agent
|
| 191 |
+
python training/dpo_finetune.py
|
| 192 |
+
|
| 193 |
+
# Evaluate + generate reward curve plots
|
| 194 |
+
python eval/reward_curves.py
|
| 195 |
+
```
|
| 196 |
+
|
| 197 |
+
---
|
| 198 |
+
|
| 199 |
+
## Repository Structure
|
| 200 |
+
|
| 201 |
+
```
|
| 202 |
+
psycheos/
|
| 203 |
+
βββ openenv.yaml β OpenEnv manifest (required by judges)
|
| 204 |
+
βββ README.md β This file
|
| 205 |
+
βββ requirements.txt
|
| 206 |
+
βββ app/
|
| 207 |
+
β βββ streamlit_app.py β Live browser UI
|
| 208 |
+
βββ environment/
|
| 209 |
+
β βββ psycheos_env.py β OpenEnv POMDP environment
|
| 210 |
+
βββ agents/
|
| 211 |
+
β βββ agents.py β All 5 agent implementations
|
| 212 |
+
β βββ graph.py β LangGraph state machine
|
| 213 |
+
βββ memory/
|
| 214 |
+
β βββ faiss_store.py β FAISS + RAG cross-session memory
|
| 215 |
+
βββ reward/
|
| 216 |
+
β βββ reward_functions.py β 5 composable reward signals
|
| 217 |
+
βββ training/
|
| 218 |
+
β βββ grpo_train.py β GRPO training loop
|
| 219 |
+
β βββ dpo_finetune.py β DPO fine-tuning script
|
| 220 |
+
β βββ colab_training.ipynb β Runnable Colab notebook
|
| 221 |
+
βββ eval/
|
| 222 |
+
β βββ reward_curves.py β Evaluation + matplotlib plots
|
| 223 |
+
βββ plots/
|
| 224 |
+
βββ reward_curves.png β Added after training
|
| 225 |
+
```
|
| 226 |
+
|
| 227 |
+
---
|
| 228 |
+
|
| 229 |
+
## Technical Stack
|
| 230 |
+
|
| 231 |
+
| Layer | Tech |
|
| 232 |
+
|-------|------|
|
| 233 |
+
| Environment | OpenEnv + Gymnasium POMDP |
|
| 234 |
+
| Agents | LangGraph state machine |
|
| 235 |
+
| Memory | FAISS + sentence-transformers RAG |
|
| 236 |
+
| Training | GRPO Β· HuggingFace TRL Β· Unsloth |
|
| 237 |
+
| Adapters | LoRA (r=16) Β· 4-bit quantization |
|
| 238 |
+
| Self-improvement | DPO on psychiatrist preferences + curriculum self-play |
|
| 239 |
+
| UI | Streamlit Β· live agent activations |
|
| 240 |
+
|
| 241 |
+
---
|
| 242 |
+
|
| 243 |
+
*PsycheOS β Mesa Sarah Vasantha Zephyr β mesazephyr1516@gmail.com*
|