sarahzephyr commited on
Commit
4d596f8
Β·
verified Β·
1 Parent(s): 45d5fbd

Upload README (1).md

Browse files
Files changed (1) hide show
  1. README (1).md +243 -0
README (1).md ADDED
@@ -0,0 +1,243 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: PsycheOS
3
+ emoji: 🧠
4
+ colorFrom: purple
5
+ colorTo: pink
6
+ sdk: streamlit
7
+ sdk_version: 1.32.0
8
+ app_file: app/streamlit_app.py
9
+ pinned: true
10
+ license: mit
11
+ tags:
12
+ - reinforcement-learning
13
+ - mental-health
14
+ - multi-agent
15
+ - openenv
16
+ - grpo
17
+ - hackathon
18
+ ---
19
+
20
+ # 🧠 PsycheOS β€” Mental Health Crisis Operating System
21
+
22
+ > **Meta PyTorch OpenEnv Hackathon Γ— Scaler School of Technology β€” Grand Finale**
23
+ > Solo submission Β· Mesa Sarah Vasantha Zephyr Β· April 25–26, 2026 Β· Bangalore
24
+
25
+ ---
26
+
27
+ ## πŸ”— Links
28
+
29
+ | Resource | URL |
30
+ |----------|-----|
31
+ | πŸš€ Live HF Space | https://huggingface.co/spaces/mesazephyr/psycheos |
32
+ | πŸ“ Mini-blog (HuggingFace) | _[Add after training β€” April 25]_ |
33
+ | πŸŽ₯ Demo video (YouTube, <2 min) | _[Add after training β€” April 25]_ |
34
+ | πŸ’» Training notebook (Colab) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mesazephyr/psycheos/blob/main/training/colab_training.ipynb) |
35
+ | πŸ€— LoRA weights | _[Add after training β€” mesazephyr/psycheos-lora]_ |
36
+
37
+ ---
38
+
39
+ ## The Problem
40
+
41
+ **197 million** Indians live with a mental disorder.
42
+ **9,000** psychiatrists exist for a population of 1.4 billion.
43
+ That is **one doctor for every 155,000 people** who need one.
44
+ **83%** receive zero care.
45
+
46
+ The gap is not just statistical β€” it is deadly. Delayed crisis identification and zero continuity between care interactions cost lives. PsycheOS is built to close that gap.
47
+
48
+ ---
49
+
50
+ ## What is PsycheOS?
51
+
52
+ PsycheOS is simultaneously:
53
+
54
+ 1. **A real deployable app** β€” open it in your browser right now, talk to it
55
+ 2. **An OpenEnv RL training environment** β€” Gymnasium-compatible POMDP with measurable reward signals
56
+ 3. **A self-improving multi-agent system** β€” gets measurably better at crisis detection through GRPO + curriculum self-play
57
+
58
+ ---
59
+
60
+ ## How the Environment Works
61
+
62
+ ### What the agent sees (observation space)
63
+
64
+ ```
65
+ {
66
+ "message": "<patient's raw text>",
67
+ "session_history": [last 3 turns with distress labels],
68
+ "turn": <int>,
69
+ "session": <int>
70
+ }
71
+ ```
72
+
73
+ ### What the agent does (action space)
74
+
75
+ ```
76
+ {
77
+ "distress_level": <1–5>, # classify severity
78
+ "response": "<text>", # generate empathetic reply
79
+ "escalate": <true/false> # route to human counselor?
80
+ }
81
+ ```
82
+
83
+ ### What the agent gets rewarded for
84
+
85
+ | Signal | Weight | Description |
86
+ |--------|--------|-------------|
87
+ | Detection F1 | 40% | Accuracy of distress severity classification |
88
+ | Empathy DPO | 40% | Psychiatrist preference margin on responses |
89
+ | Escalation P/R | 20% | Crisis routing precision β€” **10x penalty on false negatives** |
90
+
91
+ The reward is hard to game. An agent that always escalates scores βˆ’1 on false positives. An agent that never escalates scores βˆ’10 on every missed crisis. The only way to score well is to actually solve the task.
92
+
93
+ ---
94
+
95
+ ## The 5 Agents
96
+
97
+ ```
98
+ User message
99
+ β”‚
100
+ β–Ό
101
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
102
+ β”‚ Memory │────▢│ Detector │────▢│ Empathy Agent β”‚
103
+ β”‚ Retrieval β”‚ β”‚ Agent (1–5) β”‚ β”‚ (DPO fine-tuned) β”‚
104
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
105
+ β”‚
106
+ β–Ό
107
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
108
+ β”‚ Oversight │◀────│ Crisis Escalator β”‚
109
+ β”‚ Agent β”‚ β”‚ Agent β”‚
110
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
111
+ β”‚
112
+ β–Ό
113
+ Final response
114
+ (or crisis escalation)
115
+ ```
116
+
117
+ | Agent | Role | Bonus prize target |
118
+ |-------|------|--------------------|
119
+ | πŸ” **Detector** | Classifies distress 1–5 from free text | β€” |
120
+ | πŸ’¬ **Empathy** | Generates responses (DPO fine-tuned on psychiatrist preferences) | β€” |
121
+ | 🚨 **Crisis Escalator** | Routes to human counselor (10x FN penalty) | β€” |
122
+ | πŸ‘ **Oversight** | Monitors all agents for contradictions in real time | Fleet AI |
123
+ | πŸŽ“ **Curriculum** | Auto-generates harder episodes when agents plateau | Snorkel AI |
124
+
125
+ ---
126
+
127
+ ## Self-Improvement Loop
128
+
129
+ ```
130
+ Episode β†’ Reward β†’ Curriculum tracks rolling avg
131
+ β”‚
132
+ improvement < 2% over 50 episodes?
133
+ YES ──────────────────────────▢ harder episodes generated
134
+ NO ───────────────────��──────▢ continue current difficulty
135
+ β”‚
136
+ DPO fine-tune empathy agent
137
+ on psychiatrist preference pairs
138
+ (preferences shift each epoch β€”
139
+ satisfies Snorkel AI requirement)
140
+ β”‚
141
+ LoRA adapters updated via GRPO
142
+ ```
143
+
144
+ ---
145
+
146
+ ## Training Results
147
+
148
+ > ⚠️ _Placeholder β€” replace with real numbers and plots after onsite training April 25._
149
+
150
+ ![Reward Curves](plots/reward_curves.png)
151
+ _Fig 1. Composite reward, detection F1, and DPO win-rate over 500 training episodes (baseline vs trained on same axes)._
152
+
153
+ | Metric | Baseline (untrained) | After training |
154
+ |--------|---------------------|----------------|
155
+ | Detection F1 | 0.61 | β€” |
156
+ | Escalation Precision | 0.71 | β€” |
157
+ | DPO Win-Rate vs baseline | 51% | β€” |
158
+ | Avg Composite Reward | 0.35 | β€” |
159
+
160
+ ---
161
+
162
+ ## Hackathon Theme Coverage
163
+
164
+ | Theme | Coverage |
165
+ |-------|----------|
166
+ | Theme 1 β€” Multi-Agent Interactions | βœ… 5 agents cooperating, competing, overriding each other |
167
+ | Theme 3 β€” World Modeling (Personalized) | βœ… Dynamic patient world across multi-session POMDP |
168
+ | Theme 4 β€” Self-Improvement | βœ… Curriculum self-play + DPO adaptation per epoch |
169
+ | Theme 5 β€” Wild Card | βœ… First mental health crisis RL environment |
170
+
171
+ **Bonus prizes targeted:** Fleet AI Β· Snorkel AI Β· Halluminate
172
+
173
+ ---
174
+
175
+ ## Run Locally
176
+
177
+ ```bash
178
+ git clone https://huggingface.co/spaces/mesazephyr/psycheos
179
+ cd psycheos
180
+ pip install -r requirements.txt
181
+ streamlit run app/streamlit_app.py
182
+ ```
183
+
184
+ ## Train (onsite with HF compute credits)
185
+
186
+ ```bash
187
+ # GRPO training loop
188
+ python training/grpo_train.py --episodes 1000 --epochs 3
189
+
190
+ # DPO fine-tuning β€” empathy agent
191
+ python training/dpo_finetune.py
192
+
193
+ # Evaluate + generate reward curve plots
194
+ python eval/reward_curves.py
195
+ ```
196
+
197
+ ---
198
+
199
+ ## Repository Structure
200
+
201
+ ```
202
+ psycheos/
203
+ β”œβ”€β”€ openenv.yaml ← OpenEnv manifest (required by judges)
204
+ β”œβ”€β”€ README.md ← This file
205
+ β”œβ”€β”€ requirements.txt
206
+ β”œβ”€β”€ app/
207
+ β”‚ └── streamlit_app.py ← Live browser UI
208
+ β”œβ”€β”€ environment/
209
+ β”‚ └── psycheos_env.py ← OpenEnv POMDP environment
210
+ β”œβ”€β”€ agents/
211
+ β”‚ β”œβ”€β”€ agents.py ← All 5 agent implementations
212
+ β”‚ └── graph.py ← LangGraph state machine
213
+ β”œβ”€β”€ memory/
214
+ β”‚ └── faiss_store.py ← FAISS + RAG cross-session memory
215
+ β”œβ”€β”€ reward/
216
+ β”‚ └── reward_functions.py ← 5 composable reward signals
217
+ β”œβ”€β”€ training/
218
+ β”‚ β”œβ”€β”€ grpo_train.py ← GRPO training loop
219
+ β”‚ β”œβ”€β”€ dpo_finetune.py ← DPO fine-tuning script
220
+ β”‚ └── colab_training.ipynb ← Runnable Colab notebook
221
+ β”œβ”€β”€ eval/
222
+ β”‚ └── reward_curves.py ← Evaluation + matplotlib plots
223
+ └── plots/
224
+ └── reward_curves.png ← Added after training
225
+ ```
226
+
227
+ ---
228
+
229
+ ## Technical Stack
230
+
231
+ | Layer | Tech |
232
+ |-------|------|
233
+ | Environment | OpenEnv + Gymnasium POMDP |
234
+ | Agents | LangGraph state machine |
235
+ | Memory | FAISS + sentence-transformers RAG |
236
+ | Training | GRPO Β· HuggingFace TRL Β· Unsloth |
237
+ | Adapters | LoRA (r=16) Β· 4-bit quantization |
238
+ | Self-improvement | DPO on psychiatrist preferences + curriculum self-play |
239
+ | UI | Streamlit Β· live agent activations |
240
+
241
+ ---
242
+
243
+ *PsycheOS β€” Mesa Sarah Vasantha Zephyr β€” mesazephyr1516@gmail.com*