jonathan4e commited on
Commit
ca5137c
·
verified ·
1 Parent(s): e222138

Deploy AIRA v2 - Qwen vision + voice + heat safety

Browse files
Files changed (3) hide show
  1. README.md +91 -7
  2. app.py +544 -0
  3. requirements.txt +12 -0
README.md CHANGED
@@ -1,13 +1,97 @@
1
  ---
2
- title: Aira V2
3
- emoji: 🐠
4
- colorFrom: pink
5
- colorTo: purple
6
  sdk: gradio
7
- sdk_version: 6.31.0
8
- python_version: '3.13'
9
  app_file: app.py
10
  pinned: false
 
11
  ---
12
 
13
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: AIRA v2 — Arid-climate Intelligent Risk Assistant
3
+ emoji: 🌡️
4
+ colorFrom: yellow
5
+ colorTo: red
6
  sdk: gradio
7
+ sdk_version: 4.44
 
8
  app_file: app.py
9
  pinned: false
10
+ license: mit
11
  ---
12
 
13
+ # 🌡️ AIRA v2 — Arid-climate Intelligent Risk Assistant
14
+
15
+ **AI-powered voice & vision safety copilot for heat-exposed workers in Qatar.**
16
+ Built for the **QAI × Hugging Face Hackathon**. Aligned with **Qatar National Vision 2030 — Quality of Life**.
17
+
18
+ ## 🏗️ Architecture — Full Qwen Stack
19
+
20
+ | Layer | Model | Role |
21
+ |-------|-------|------|
22
+ | 👁️ **Vision** | `Qwen/Qwen2-VL-2B-Instruct` | Image analysis of job-site heat-stress indicators |
23
+ | 🧠 **Reasoning** | `Qwen/Qwen3-4B-Instruct` | WBGT calculation, hydration & work-rest planning (Qatar MoPH aligned) |
24
+ | 🎤 **STT** | `openai/whisper-tiny` | Speech-to-text in Arabic / Hindi / Urdu / English |
25
+ | 🔊 **TTS** | `gTTS` | Spoken safety advice read back to worker |
26
+ | 🌤️ **Weather** | Open-Meteo API | Live Qatar temperature & humidity |
27
+
28
+ ## 📁 Repository Structure
29
+
30
+ ```
31
+ AIRA/
32
+ ├── app.py # Gradio Space: Vision + Qwen3 Reasoning + Voice + WBGT Safety
33
+ ├── train_qwen_vision.py # LoRA fine-tuning script for Qwen2-VL on heat-stress imagery
34
+ ├── requirements.txt # Python dependencies
35
+ └── README.md # This file (HF Space metadata + docs)
36
+ ```
37
+
38
+ ## 🚀 Deploy to Hugging Face Spaces
39
+
40
+ ### Prerequisites
41
+ > ⚠️ Gradio Spaces require a **HF Pro subscription** on cpu-basic hardware.
42
+ > Subscribe at https://huggingface.co/pro (your $100 credits can cover this).
43
+
44
+ ### One-command deploy
45
+
46
+ ```bash
47
+ # Install HF CLI
48
+ curl -LsSf https://hf.co/cli/install.sh | bash -s
49
+
50
+ # Login (you must do this yourself)
51
+ hf auth login
52
+
53
+ # Create + upload the Space (run once Pro is active)
54
+ hf repos create jonathan4e/aira-v2 --type space --sdk gradio --exist-ok
55
+ hf upload jonathan4e/aira-v2 --type space app.py requirements.txt README.md --commit-message "Deploy AIRA v2"
56
+ hf spaces settings jonathan4e/aira-v2 --hardware t4-small
57
+ hf spaces wait jonathan4e/aira-v2 --timeout 600
58
+ ```
59
+
60
+ **Your Space URL:** `https://huggingface.co/spaces/jonathan4e/aira-v2`
61
+
62
+ ## 🎯 Fine-tune Qwen2-VL on Your Own Dataset
63
+
64
+ ```bash
65
+ # 1. Create/push a heat-stress image dataset to HF
66
+ python create_dataset.py --push jonathan4e/aira-heat-stress
67
+
68
+ # 2. Fine-tune Qwen2-VL-2B with LoRA (PEFT)
69
+ python train_qwen_vision.py \
70
+ --dataset jonathan4e/aira-heat-stress \
71
+ --output jonathan4e/aira-qwen2vl-heat \
72
+ --epochs 1 --batch 2
73
+
74
+ # 3. Update app.py VISION_MODEL_ID to your fine-tuned model
75
+ ```
76
+
77
+ ## 🧠 WBGT Safety Logic (Qatar MoPH Aligned)
78
+
79
+ - **WBGT estimate** from temperature + humidity + solar load
80
+ - **Risk bands:** Low (<29°C), Moderate (29–31°C), High (31–33°C), Extreme (>33°C)
81
+ - **Output:** hydration (L/hr), work/rest cycle (min work / min rest shade), immediate actions
82
+ - Multilingual: English, Arabic, Hindi, Urdu
83
+
84
+ ## 📱 Demo Features
85
+
86
+ - `gr.Examples` pre-loaded for instant hackathon judging
87
+ - Mobile-responsive Gradio UI with RTL support for Arabic/Urdu
88
+ - Voice input → transcription → Qwen3 reasoning → TTS audio output
89
+ - Image upload → Qwen2-VL heat-stress classification
90
+
91
+ ## 🇶🇦 Alignment
92
+
93
+ Supports **Qatar National Vision 2030 — Quality of Life** by protecting outdoor, logistics, and construction workers from extreme summer heat with free, open-source, mobile-accessible AI.
94
+
95
+ ## 📄 License
96
+
97
+ MIT
app.py ADDED
@@ -0,0 +1,544 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ app.py — AIRA: Arid-climate Intelligent Risk Assistant
3
+ Complete Gradio app for Hugging Face Spaces.
4
+ Qwen stack: Qwen2-VL-2B (vision) + Qwen3-8B-Instruct (reasoning) + Whisper-Tiny (STT) + gTTS (TTS)
5
+ """
6
+ from __future__ import annotations
7
+
8
+ import math
9
+ import threading
10
+ import requests
11
+ from datetime import datetime, timezone
12
+ from typing import Optional, Dict, Any, List, Tuple
13
+
14
+ import gradio as gr
15
+ from PIL import Image, ImageDraw
16
+
17
+ # ---------------- Optional ML deps (lazy load) ----------------
18
+ try:
19
+ import torch
20
+ from transformers import (
21
+ Qwen2VLForConditionalGeneration,
22
+ AutoProcessor,
23
+ AutoModelForCausalLM,
24
+ AutoTokenizer,
25
+ pipeline,
26
+ )
27
+ _TORCH = True
28
+ except Exception:
29
+ _TORCH = False
30
+
31
+ try:
32
+ from gtts import gTTS
33
+ _GTTS = True
34
+ except Exception:
35
+ _GTTS = False
36
+
37
+ # ---------------- Config ----------------
38
+ QATAR_CITIES: Dict[str, Dict[str, float]] = {
39
+ "Doha": {"lat": 25.2867, "lon": 51.5343},
40
+ "Al Wakrah": {"lat": 25.1715, "lon": 51.6047},
41
+ "Al Khor": {"lat": 25.6819, "lon": 51.5010},
42
+ "Mesaieed": {"lat": 24.7728, "lon": 51.5045},
43
+ "Dukhan": {"lat": 25.4219, "lon": 50.7531},
44
+ "Al Rayyan": {"lat": 25.2333, "lon": 51.4333},
45
+ "Lusail": {"lat": 25.3628, "lon": 51.5010},
46
+ }
47
+
48
+ LANGS: Dict[str, Dict[str, Any]] = {
49
+ "en": {"name": "English", "tts": "en", "rtl": False},
50
+ "ar": {"name": "\u0627\u0644\u0639\u0631\u0628\u064a\u0629", "tts": "ar", "rtl": True},
51
+ "hi": {"name": "\u0939\u093f\u0928\u094d\u0926\u093f", "tts": "hi", "rtl": False},
52
+ "ur": {"name": "\u0627\u0631\u062f\u0648", "tts": "ur", "rtl": False},
53
+ }
54
+
55
+ VISION_MODEL_ID = "Qwen/Qwen2-VL-2B-Instruct"
56
+ REASON_MODEL_ID = "Qwen/Qwen3-4B-Instruct" # Qwen3 reasoning engine
57
+ ASR_MODEL_ID = "openai/whisper-tiny"
58
+
59
+ VISION_PROMPT = (
60
+ "You are AIRA, a workplace heat-safety assistant for Qatar outdoor workers. "
61
+ "Inspect the image for heat-stress indicators: sweat, flushed/pale skin, dizziness posture, "
62
+ "lack of shade, direct sun on workers, missing water, heavy PPE, hot surfaces, unsafe conditions. "
63
+ "Classify into: Safe_Shaded, Direct_Sun_Exposure, High_Physical_Exertion, Visible_Heat_Distress. "
64
+ "Be concise, practical, safety-first. Reply in the user's language."
65
+ )
66
+
67
+ QATAR_SYSTEM_PROMPT = (
68
+ "You are AIRA, a heat-safety reasoning engine for Qatar outdoor workers, aligned with Qatar "
69
+ "Ministry of Public Health (MoPH) Wet Bulb Globe Temperature (WBGT) guidelines. "
70
+ "Given temperature, humidity, work intensity, and vision findings, output: "
71
+ "(1) WBGT estimate, (2) Heat risk level (Low/Moderate/High/Extreme), "
72
+ "(3) Hydration plan (Litres per hour), (4) Work-rest cycle (minutes work / minutes rest in shade), "
73
+ "(5) Immediate safety actions in Arabic, Hindi, Urdu, or English as requested. "
74
+ "Be concise, actionable, and safety-first."
75
+ )
76
+
77
+ # Model singletons (lazy, thread-safe)
78
+ _lock = threading.Lock()
79
+ _vm = None
80
+ _vp = None
81
+ _rm = None
82
+ _rt = None
83
+ _asr = None
84
+
85
+
86
+ def _load_vision():
87
+ global _vm, _vp
88
+ with _lock:
89
+ if _vm is None and _TORCH:
90
+ _vm = Qwen2VLForConditionalGeneration.from_pretrained(
91
+ VISION_MODEL_ID,
92
+ torch_dtype=torch.bfloat16,
93
+ device_map="auto",
94
+ trust_remote_code=True,
95
+ )
96
+ _vp = AutoProcessor.from_pretrained(VISION_MODEL_ID, trust_remote_code=True)
97
+ return _vm, _vp
98
+
99
+
100
+ def _load_reason():
101
+ global _rm, _rt
102
+ with _lock:
103
+ if _rm is None and _TORCH:
104
+ _rm = AutoModelForCausalLM.from_pretrained(
105
+ REASON_MODEL_ID,
106
+ torch_dtype=torch.bfloat16,
107
+ device_map="auto",
108
+ trust_remote_code=True,
109
+ )
110
+ _rt = AutoTokenizer.from_pretrained(REASON_MODEL_ID, trust_remote_code=True)
111
+ return _rm, _rt
112
+
113
+
114
+ def _load_asr():
115
+ global _asr
116
+ with _lock:
117
+ if _asr is None and _TORCH:
118
+ _asr = pipeline(
119
+ "automatic-speech-recognition",
120
+ model=ASR_MODEL_ID,
121
+ device=0 if torch.cuda.is_available() else -1,
122
+ )
123
+ return _asr
124
+ # ---------------- Weather (Open-Meteo, free, no key) ----------------
125
+ def get_qatar_weather(city: str = "Doha") -> Dict[str, Any]:
126
+ c = QATAR_CITIES.get(city, QATAR_CITIES["Doha"])
127
+ try:
128
+ r = requests.get(
129
+ "https://api.open-meteo.com/v1/forecast",
130
+ params={
131
+ "latitude": c["lat"],
132
+ "longitude": c["lon"],
133
+ "current": "temperature_2m,relative_humidity_2m,apparent_temperature,wind_speed_10m",
134
+ "timezone": "Asia/Qatar",
135
+ },
136
+ timeout=15,
137
+ )
138
+ r.raise_for_status()
139
+ d = r.json()["current"]
140
+ return {
141
+ "city": city,
142
+ "temp": d.get("temperature_2m"),
143
+ "feels_like": d.get("apparent_temperature"),
144
+ "humidity": d.get("relative_humidity_2m"),
145
+ "wind": d.get("wind_speed_10m"),
146
+ "ok": True,
147
+ }
148
+ except Exception as e:
149
+ return {"city": city, "temp": 44, "feels_like": 52, "humidity": 45, "wind": 12, "ok": False, "e": str(e)}
150
+
151
+
152
+ # ---------------- WBGT / Safety Engine (Qatar MoPH aligned) ----------------
153
+ def _wet_bulb(temp_c: float, rh: float) -> float:
154
+ """Stull approximation of wet-bulb temperature (°C)."""
155
+ t = temp_c
156
+ h = rh
157
+ tw = (
158
+ t * math.atan(0.151977 * math.sqrt(h + 8.313659))
159
+ + math.atan(t + h)
160
+ - math.atan(h - 1.676331)
161
+ + 0.00391838 * h ** (3 / 2) * math.atan(0.023101 * h)
162
+ - 4.686035
163
+ )
164
+ return tw
165
+
166
+
167
+ def wbgt_estimate(temp_c: float, rh: float, sun_exposed: bool = True) -> float:
168
+ """Estimate WBGT (°C). If sun-exposed, add solar radiation load (~+2 to +4°C globe)."""
169
+ tw = _wet_bulb(temp_c, rh)
170
+ # Simplified WBGT = 0.7*Tw + 0.2*Tg + 0.1*T ; approximate Tg ≈ T + solar load
171
+ tg = temp_c + (3.0 if sun_exposed else 1.0)
172
+ wbgt = 0.7 * tw + 0.2 * tg + 0.1 * temp_c
173
+ return round(wbgt, 1)
174
+
175
+
176
+ def wbgt_guidance(wbgt: float, intensity: str = "moderate") -> Dict[str, Any]:
177
+ """Qatar MoPH-style work/rest & hydration guidance from WBGT."""
178
+ if wbgt < 29:
179
+ level, label, color = "low", "Low", "#22c55e"
180
+ work, rest = 60, 0
181
+ water = 0.5
182
+ elif wbgt < 31:
183
+ level, label, color = "moderate", "Moderate", "#eab308"
184
+ work, rest = 45, 15
185
+ water = 0.75
186
+ elif wbgt < 33:
187
+ level, label, color = "high", "High", "#f97316"
188
+ work, rest = 30, 30
189
+ water = 1.0
190
+ else:
191
+ level, label, color = "extreme", "Extreme", "#ef4444"
192
+ work, rest = 0, 60
193
+ water = 1.25
194
+
195
+ # Adjust for intensity
196
+ mult = {"light": 0.9, "moderate": 1.0, "heavy": 1.2}.get(intensity, 1.0)
197
+ water = round(water * mult, 2)
198
+
199
+ return {
200
+ "wbgt": wbgt,
201
+ "level": level,
202
+ "label": label,
203
+ "color": color,
204
+ "work_min": work,
205
+ "rest_min": rest,
206
+ "water_l_hr": water,
207
+ "intensity": intensity,
208
+ }
209
+
210
+
211
+ def safety_report(temp: float, hum: float, intensity: str, vision_text: str, lang: str) -> str:
212
+ """Generate localized safety report using Qwen3 reasoning (fallback to template)."""
213
+ wbgt = wbgt_estimate(temp, hum, sun_exposed="direct sun" in vision_text.lower())
214
+ guidance = wbgt_guidance(wbgt, intensity)
215
+
216
+ rm, rt = _load_reason()
217
+ if rm is not None and rt is not None:
218
+ try:
219
+ user_msg = (
220
+ f"Temperature: {temp}°C, Humidity: {hum}%, WBGT: {wbgt}°C, "
221
+ f"Intensity: {intensity}, Vision: {vision_text}. "
222
+ f"Give a concise heat-safety report in {lang}."
223
+ )
224
+ messages = [
225
+ {"role": "system", "content": QATAR_SYSTEM_PROMPT},
226
+ {"role": "user", "content": user_msg},
227
+ ]
228
+ text = rt.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
229
+ inputs = rt(text=[text], return_tensors="pt").to(rm.device)
230
+ out = rm.generate(**inputs, max_new_tokens=300, do_sample=False)
231
+ result = rt.batch_decode(out, skip_special_tokens=True)[0]
232
+ if "assistant" in result:
233
+ result = result.split("assistant")[-1].strip()
234
+ return result or _template_report(guidance, vision_text, lang)
235
+ except Exception as e:
236
+ return f"(Qwen3 reasoning unavailable: {e})\n\n" + _template_report(guidance, vision_text, lang)
237
+ return _template_report(guidance, vision_text, lang)
238
+
239
+
240
+ def _template_report(guidance: Dict, vision_text: str, lang: str) -> str:
241
+ templates = {
242
+ "en": (
243
+ f"**WBGT:** {guidance['wbgt']}°C | **Risk:** {guidance['label']} \n"
244
+ f"**Hydration:** {guidance['water_l_hr']} L/hr | **Work:** {guidance['work_min']} min / "
245
+ f"**Rest:** {guidance['rest_min']} min in shade \n"
246
+ f"**Vision findings:** {vision_text} \n"
247
+ f"**Action:** {guidance['note'] if 'note' in guidance else 'Monitor worker closely.'}"
248
+ ),
249
+ "ar": (
250
+ f"**WBGT:** {guidance['wbgt']}°C | **المخاطر:** {guidance['label']} \n"
251
+ f"**الترطيب:** {guidance['water_l_hr']} لتر/ساعة | **العمل:** {guidance['work_min']} دقيقة / "
252
+ f"**الراحة:** {guidance['rest_min']} دقيقة في الظل \n"
253
+ f"**ملاحظات الرؤية:** {vision_text}"
254
+ ),
255
+ "hi": (
256
+ f"**WBGT:** {guidance['wbgt']}°C | **जोखिम:** {guidance['label']} \n"
257
+ f"**हाइड्रेशन:** {guidance['water_l_hr']} L/घंटा | **काम:** {guidance['work_min']} मिनट / "
258
+ f"**आराम:** {guidance['rest_min']} मिनट छांव में \n"
259
+ f"**विज़न निष्कर्ष:** {vision_text}"
260
+ ),
261
+ "ur": (
262
+ f"**WBGT:** {guidance['wbgt']}°C | **خطرہ:** {guidance['label']} \n"
263
+ f"**پانی:** {guidance['water_l_hr']} L/گھنٹہ | **کام:** {guidance['work_min']} منٹ / "
264
+ f"** आरام:** {guidance['rest_min']} منٹ سائے میں \n"
265
+ f"**بینائی کے نتائج:** {vision_text}"
266
+ ),
267
+ }
268
+ return templates.get(lang, templates["en"])
269
+ # ---------------- Vision analysis (Qwen2-VL) ----------------
270
+ def analyze_image(image: Image.Image, lang: str = "en") -> str:
271
+ vm, vp = _load_vision()
272
+ if vm is None or vp is None:
273
+ return _vision_fallback(image, lang)
274
+ try:
275
+ messages = [
276
+ {
277
+ "role": "user",
278
+ "content": [
279
+ {"type": "image", "image": image},
280
+ {"type": "text", "text": VISION_PROMPT},
281
+ ],
282
+ }
283
+ ]
284
+ text = vp.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
285
+ inputs = vp(text=[text], images=[image], padding=True, return_tensors="pt").to(vm.device)
286
+ out = vm.generate(**inputs, max_new_tokens=256, do_sample=False)
287
+ result = vp.batch_decode(out, skip_special_tokens=True)[0]
288
+ if "assistant" in result:
289
+ result = result.split("assistant")[-1].strip()
290
+ return result or _vision_fallback(image, lang)
291
+ except Exception as e:
292
+ return f"(Vision error: {e})\n\n" + _vision_fallback(image, lang)
293
+
294
+
295
+ def _vision_fallback(image: Image.Image, lang: str) -> str:
296
+ wd, ht = image.size
297
+ msgs = {
298
+ "en": f"Vision fallback ({wd}x{ht}). Check sweat, skin, shade, water. If unwell, stop & cool down.",
299
+ "ar": f"\u0628\u062f\u064a\u0644 \u0627\u0644\u0631\u0624\u064a\u0629 ({wd}\u00d7{ht}). \u062a\u062d\u0642\u0642 \u0645\u0646 \u0627\u0644\u0639\u0631\u0642, \u0627\u0644\u0628\u0634\u0627\u0631\u0629, \u0627\u0644\u0638\u0644, \u0627\u0644\u0645\u0627\u0621.",
300
+ "hi": f"\u0935\u093f\u091c\u094d\u091e\u093e \u092c\u093e\u0902\u0921 ({wd}\u00d7{ht}). \u092a\u0938\u094d\u0925\u093e\u0928, \u0924\u094d\u092f\u093e\u0916, \u091b\u092a\u093e\u0930, \u092a\u093e\u0928\u093e \u091c\u093e\u0924\u094d\u092e\u093e.",
301
+ "ur": f"\u0628\u06cc\u0646\u0627\u0626\u06cc \u0645\u062a\u0628\u0627\u062f\u0644 ({wd}\u00d7{ht}). \u067e\u0633\u06cc\u0646\u0627, \u062c\u0644\u062f, \u0633\u0627\u064a\u0647, \u067e\u0627\u0646\u06cc \u062a\u0627\u0645 \u06a9\u06cc\u0646.",
302
+ }
303
+ return msgs.get(lang, msgs["en"])
304
+
305
+
306
+ # ---------------- Voice: STT + TTS ----------------
307
+ def transcribe(audio_path: Optional[str], lang: str) -> str:
308
+ if not audio_path:
309
+ return ""
310
+ asr = _load_asr()
311
+ if asr is None:
312
+ return "(Whisper unavailable — type instead.)"
313
+ try:
314
+ res = asr(audio_path, generate_kwargs={"language": lang if lang == "en" else None})
315
+ return res.get("text", "").strip()
316
+ except Exception as e:
317
+ return f"(STT error: {e})"
318
+
319
+
320
+ def tts_to_file(text: str, lang: str) -> Optional[str]:
321
+ if not text or not _GTTS:
322
+ return None
323
+ try:
324
+ path = "/tmp/aira_tts.mp3"
325
+ gTTS(text=text, lang=LANGS.get(lang, {}).get("tts", "en")).save(path)
326
+ return path
327
+ except Exception:
328
+ return None
329
+
330
+ # ---------------- UI strings ----------------
331
+ UI: Dict[str, Dict[str, str]] = {
332
+ "en": {
333
+ "t": "AIRA v2", "s": "Voice • Vision • Safety Copilot — Qatar Heat Workers",
334
+ "tag": "Qwen2-VL • Qwen3 Reasoning • Whisper STT • gTTS • HF Spaces",
335
+ "w": "Live Qatar Weather", "c": "City", "rf": "Refresh",
336
+ "v": "Vision Scan", "vo": "Voice Copilot", "sf": "Safety Planner",
337
+ "vp": "Concern? (optional)", "vb": "Analyze Image",
338
+ "vb2": "Record & Translate", "tts": "Speak Safety Advice",
339
+ "sn": "Hydration & Work-Rest Plan", "wt": "Weight (kg)",
340
+ "wi": "Work intensity", "l": "Light", "m": "Moderate", "h": "Heavy",
341
+ "lg": "Language", "st": "Temp (°C)", "hu": "Humidity (%)",
342
+ "wbgt": "WBGT & Safety", "f": "Qatar Vision 2030 • Quality of Life • Open Source",
343
+ },
344
+ "ar": {
345
+ "t": "أيرا v2", "s": "مساعد صوت وبصر وأمان — عمال حرارة قطر",
346
+ "tag": "كيوين2-في إل • كيوين3 reasoning • ويسبر • جي تي تي إس • هجنج فيس سبيسز",
347
+ "w": "\u0627\u0644\u0637\u0642\u0633", "c": "\u0627\u0644\u0645\u062f\u064a\u0646\u0629", "rf": "\u062a\u062d\u062f\u064a\u062f",
348
+ "v": "\u062a\u062d\u0644\u064a\u0644 \u0628\u0635\u0631\u064a", "vo": "\u0627\u0644\u0635\u0648\u062a", "sf": "\u0627\u0644\u0623\u0645\u0646",
349
+ "vp": "\u0645\u0627\u0630\u0627 \u064a\u0642\u0644\u0642\u0643? ", "vb": "\u062a\u062d\u0644\u064a\u0644 \u0627\u0644\u0635\u0648\u0627\u0631",
350
+ "vb2": "\u062a\u0633\u062c\u064a\u0644 \u0648\u062a\u0631\u062c\u0645\u0629", "tts": "\u0627\u0633\u062a\u0645\u0639 \u0644\u0644\u0646\u0635\u064a\u062d\u0629",
351
+ "sn": "\u062e\u0637\u0629 \u0627\u0644\u062a\u0631\u0637\u064a\u0628 \u0648\u0627\u0644\u0648\u0642\u0641", "wt": "\u0627\u0644\u0648\u0632\u0646 (\u0643\u063a)",
352
+ "wi": "\u0634\u062f\u0629 \u0627\u0644\u0639\u0645\u0644", "l": "\u062e\u0641\u064a\u0641", "m": "\u0645\u062a\u0648\u0633\u0637", "h": "\u062b\u0642\u064a\u0644",
353
+ "lg": "\u0627\u0644\u0644\u063a\u0629", "st": "\u062f\u0631\u062c\u0629 \u0627\u0644\ud83e\udd21\ufe0f", "hu": "\u0646\u0633\u0628\u0629 \u0627\u0644\u0647\u0648\u0627\u0621 (%)",
354
+ "wbgt": "\u0648\u0628\u062c\u064a\u0634\u064a \u0648\u0627\u0644\u0623\u0645\u0646", "f": "\u0631\u0624\u064a\u0629 \u0642\u0637\u0631 2030 \u2022 \u0645\u062d\u062a\u0648\u0649 \u0627\u0644\u062d\u064a\u0627\u0629 \u2022 \u0645\u0641\u062a\u0648\u062d \u0627\u0644\u0645\u062d\u062f\u0648",
355
+ },
356
+ "hi": {
357
+ "t": "आईरा v2", "s": "आवाज़ • दृष्टि • सुरक्षा सहायक — कतर के गर्मी श्रमिक",
358
+ "tag": "क्यूइन2-वीएल • क्यूइन3 तर्क • व्हिस्पर • जीटीटीएस • एचएफ स्पेस",
359
+ "w": "\u092e\u094c\u092e\u093e", "c": "\u0936\u0939\u0930", "rf": "\u0924\u093e\u091a\u093e\u0921\u093c",
360
+ "v": "\u0935\u093f\u091c\u094d\u091e\u093e \u0938\u0915\u092c\u0942", "vo": "\u0906\u0935\u094d\u092f\u093e", "sf": "\u092f\u0947\u0928\u094d\u0927\u0915\u093e\u0930\u094d\u091c",
361
+ "vp": "\u091a\u094d\u092f\u093e\u0924\u093f? ", "vb": "\u091b\u093e\u0935\u093f \u092a\u094d\u0930\u0935\u093e\u0928",
362
+ "vb2": "\u0930\u093f\u0902\u0915\u094d\u0917 \u640 \u0905\u0928\u0941\u0926\u093e\u0921", "tts": "\u0938\u092e\u093e \u0938\u0942\u0928\u093e",
363
+ "sn": "\u091c\u0932\u094d\u092f\u094b\u092b\u094c\u0928 \u2022 \u0935\u093e\u0930\u094d\u092f\u093e-\u0935\u094d\u0930\u092e\u093f\u0930\u094d\u092e\u093e", "wt": "\u0935\u0928\u0928 (\u0915\u093f\u0902\u0917)",
364
+ "wi": "\u0915\u093e\u0930\u094d\u0924 \u0924\u093f\u0935\u093e\u091f", "l": "\u0939\u092e\u0932", "m": "\u092e\u0902\u0921\u0935\u094d\u092e\u094d\u092e", "h": "\u092d\u093e\u0930\u0940",
365
+ "lg": "\u092d\u093e\u0907\u091c", "st": "\u0926\u094d\u0935\u093e\u091f (\u092e\u094d\u092e\u093e\u0902)", "hu": "\u0928\u094d\u092f\u093e\u0930\u094d\u0925\u093e\u092f (%)",
366
+ "wbgt": "डब्ल्यूबीजीटी और सुरक्षा", "f": "\u0915\u093e\u0924\u0930 \u0935\u093f\u0936\u094d\u0935\u0935\u093e\u0928 2030 \u2022 \u091c\u0940\u092e\u093e\u0928 \u0917\u0941\u0928\u093f\u092e \u2022 \u0914\u092a\u0923 \u0938\u094c\u092e\u0938",
367
+ },
368
+ "ur": {
369
+ "t": "ایرا v2", "s": "آواز • بینائی • حفاظتی معاون — گرمی کے مزدور",
370
+ "tag": "کوئیں2-وی ایل • کوئیں3 استدلال • ويسپر • جی ٹی ٹی ایس • ایچ ایف سلیٹس",
371
+ "w": "\u0645\u0648\u0633\u0645", "c": "\u0634\u0647\u0631", "rf": "\u062a\u0627\u0632\u0647 \u0643\u06cc\u064a\u0627",
372
+ "v": "\u0628\u06cc\u0646\u0627\u0626\u06cc", "vo": "\u0622\u0627\u0632", "sf": "\u067e\u0644\u0627\u0646 \u0627\u0645\u0646",
373
+ "vp": "\u06a9\u06cc\u0627 \u0646\u06cc\u06ba\u0634? ", "vb": "\u062a\u0633\u0648\u06cc\u0631 \u0635\u0648\u06cc\u0631",
374
+ "vb2": "\u0631\u06cc\u06a9\u0631\u0627\u062f \u0648 \u062a\u0631\u062c\u0645\u0629", "tts": "\u0645\u0634\u0648\u0631\u0647 \u0633\u0646\u06cc\u0646",
375
+ "sn": "\u067e\u0627\u0646\u06cc \u0648 \u0648\u0642\u0641\u0647", "wt": "\u0648\u0632\u0646 (\u06a9\u06cc\u0644\u0648\u063a\u0631\u0627\u0645)",
376
+ "wi": "\u06a9\u0627\u0645 \u0643\u06cc \u0634\u062f\u062a", "l": "\u0647\u0644\u0643\u0627", "m": "\u0645\u062a\u0648\u0633\u0648\u062a", "h": "\u0628\u062d\u0627\u0631\u06cc",
377
+ "lg": "\u0632\u0628\u0627\u0646", "st": "درجہ حرارت (\u0628\u0648)", "hu": "نمی (\u0641\u06ccصد)",
378
+ "wbgt": "وی بی جی ٹی اور حفاظت", "f": "\u0642\u0637\u0631 \u0648\u06cc\u0632\u06cc\u0646 2030 \u2022 \u0632\u06cc\u0646\u06af\u06cc \u0643\u0627 \u0645\u06cc\u0627\u0631 \u2022 \u0627\u0648\u067e\u0646 \u0633\u0648\u0631\u0633",
379
+ },
380
+ }
381
+
382
+
383
+ def tr(lang: str, k: str) -> str:
384
+ return UI.get(lang, UI["en"]).get(k, UI["en"][k])
385
+
386
+ # ---------------- CSS ----------------
387
+ CSS = """
388
+ :root{--deep:#78350f;--desert:#d97706;--maroon:#8d2f3a;}
389
+ .gradio-container{font-family:system-ui,sans-serif;background:linear-gradient(135deg,#fdf3e3 0%,#e8d4b3 60%,#d97706 100%);min-height:100vh}
390
+ body[dir=rtl],.gradio-container[dir=rtl]{direction:rtl}
391
+ .aira-header{text-align:center;padding:2rem 1rem 1rem;border-bottom:3px solid var(--maroon);margin-bottom:1rem}
392
+ .aira-title{font-size:2.5rem;font-weight:800;color:var(--deep);margin:0}
393
+ .aira-subtitle{font-size:1.1rem;color:#374151;margin:.25rem 0 .5rem}
394
+ .aira-tagline{font-size:.9rem;color:var(--desert);font-weight:600;text-transform:uppercase;letter-spacing:1px}
395
+ .weather-card{background:rgba(255,255,255,.85);border-radius:16px;padding:1.25rem;box-shadow:0 10px 25px -5px rgba(0,0,0,.1);margin-bottom:1rem;border-left:5px solid var(--maroon)}
396
+ .weather-temp{font-size:3rem;font-weight:900;color:var(--deep)}
397
+ .risk-badge{display:inline-block;padding:.35rem 1rem;border-radius:999px;font-weight:800;font-size:.95rem;color:#fff}
398
+ .plan-card{background:rgba(255,255,255,.85);border-radius:16px;padding:1.25rem;box-shadow:0 10px 25px -5px rgba(0,0,0,.1);margin-bottom:1rem}
399
+ footer{text-align:center;padding:1.5rem;color:#6b7280;font-size:.85rem;border-top:1px solid rgba(255,255,255,.3);margin-top:2rem}
400
+ @media(max-width:640px){.aira-title{font-size:1.75rem}.weather-temp{font-size:2.25rem}}
401
+ # ---------------- Gradio App ----------------
402
+ def build_app() -> gr.Blocks:
403
+ with gr.Blocks(title="AIRA v2", css=CSS, theme=gr.themes.Soft(primary_hue="orange")) as demo:
404
+ # State
405
+ lang = gr.State("en")
406
+ city = gr.State("Doha")
407
+ ws = gr.State({})
408
+ guidance = gr.State({})
409
+
410
+ # Header
411
+ gr.HTML(
412
+ f'<div class="aira-header">'
413
+ f'<div class="aira-title">\U0001f321\ufe0f {tr("en","t")}</div>'
414
+ f'<div class="aira-subtitle">{tr("en","s")}</div>'
415
+ f'<div class="aira-tagline">{tr("en","tag")}</div>'
416
+ f'</div>'
417
+ )
418
+
419
+ # Weather row
420
+ with gr.Row(equal_height=True):
421
+ cd = gr.Dropdown(choices=list(QATAR_CITIES), value="Doha", label=tr("en","c"), scale=1)
422
+ rb = gr.Button(tr("en","rf"), variant="primary", scale=0)
423
+ wd = gr.HTML(value=_weather_html("Doha", get_qatar_weather("Doha")))
424
+
425
+ # Tabs
426
+ with gr.Tabs():
427
+ # --- Vision Tab ---
428
+ with gr.TabItem(tr("en","v"), id="vision"):
429
+ with gr.Row():
430
+ with gr.Column(scale=1):
431
+ vi = gr.Image(type="pil", label="Worker / job-site photo")
432
+ vp2 = gr.Textbox(label=tr("en","vp"), placeholder="e.g. sweating, no shade...", lines=2)
433
+ vb = gr.Button(tr("en","vb"), variant="primary")
434
+ with gr.Column(scale=1):
435
+ vo = gr.Markdown(label="Aira Vision Assessment")
436
+
437
+ # --- Voice Tab ---
438
+ with gr.TabItem(tr("en","vo"), id="voice"):
439
+ with gr.Row():
440
+ with gr.Column(scale=1):
441
+ va = gr.Audio(sources=["microphone"], type="filepath", label="Speak AR/HI/UR/EN")
442
+ vb2 = gr.Button(tr("en","vb2"), variant="primary")
443
+ vt = gr.Textbox(label="Transcribed", lines=3, interactive=False)
444
+ with gr.Column(scale=1):
445
+ tb = gr.Button(tr("en","tts"), variant="secondary")
446
+ ta = gr.Audio(label="Safety audio (auto-play)", autoplay=True)
447
+
448
+ # --- Safety Tab ---
449
+ with gr.TabItem(tr("en","sf"), id="safety"):
450
+ with gr.Row():
451
+ with gr.Column(scale=1):
452
+ st_in = gr.Number(label=tr("en","st"), value=40, minimum=0, maximum=60, step=1)
453
+ hu_in = gr.Number(label=tr("en","hu"), value=50, minimum=0, maximum=100, step=1)
454
+ wi_in = gr.Radio([tr("en","l"), tr("en","m"), tr("en","h")], value=tr("en","m"), label=tr("en","wi"))
455
+ sl_in = gr.Dropdown([f"{x['name']} ({k})" for k, x in LANGS.items()], value="English (en)", label=tr("en","lg"))
456
+ with gr.Column(scale=1):
457
+ rh_out = gr.HTML(label=tr("en","wbgt"))
458
+ ph_out = gr.HTML(label=tr("en","sn"))
459
+
460
+ gr.HTML(f"<footer>{tr('en','f')}</footer>")
461
+
462
+ # ---------------- Helpers ----------------
463
+ def _weather_html(ct: str, w: Dict) -> str:
464
+ if not w.get("ok"):
465
+ return f'<div class="weather-card"><div class="weather-temp">{w.get("temp","—")}\u00b0C</div><div style="color:#6b7280">{ct} — weather unavailable</div></div>'
466
+ r = wbgt_guidance(wbgt_estimate(w.get("temp"), w.get("humidity"), True), "moderate")
467
+ return (
468
+ f'<div class="weather-card">'
469
+ f'<div style="display:flex;justify-content:space-between;align-items:start;">'
470
+ f'<div><div class="weather-temp">{w.get("temp")}\u00b0C</div>'
471
+ f'<div style="font-size:.9rem;color:#4b5563">Feels {w.get("feels_like")}\u00b0C • Hum {w.get("humidity")}% • Wind {w.get("wind")} km/h • {ct}</div></div>'
472
+ f'<div class="risk-badge" style="background:{r["color"]}">{r["label"]} risk</div>'
473
+ f'</div></div>'
474
+ )
475
+
476
+ def _safety_html(temp, hum, intensity, lang_code, vision_text=""):
477
+ wbgt = wbgt_estimate(temp, hum, sun_exposed="direct sun" in vision_text.lower())
478
+ g = wbgt_guidance(wbgt, intensity)
479
+ rep = safety_report(temp, hum, intensity, vision_text, lang_code)
480
+ html = (
481
+ f'<div class="plan-card" style="border-left:5px solid {g["color"]}">'
482
+ f'<div style="display:flex;justify-content:space-between;margin-bottom:.5rem;">'
483
+ f'<strong>WBGT:</strong> {g["wbgt"]}°C <strong>Risk:</strong> <span class="risk-badge" style="background:{g["color"]}">{g["label"]}</span>'
484
+ f'</div>'
485
+ f'<div style="margin-top:.5rem;line-height:1.6;">{rep}</div>'
486
+ f'</div>'
487
+ )
488
+ return html, g
489
+
490
+ # ---------------- Event handlers ----------------
491
+ def oref(c):
492
+ w = get_qatar_weather(c)
493
+ ws.value = w
494
+ return _weather_html(c, w)
495
+
496
+ def olg(ch):
497
+ lang.value = ch.split("(")[-1].strip(")") if "(" in ch else "en"
498
+
499
+ def ov(img, pr, l_code, demo_flag):
500
+ if demo_flag or img is None:
501
+ return _vision_fallback(Image.new("RGB", (400, 300), (200, 200, 200)), l_code)
502
+ return analyze_image(img, l_code)
503
+
504
+ def ovc(audio, l_code):
505
+ tx = transcribe(audio, l_code)
506
+ w = ws.value or get_qatar_weather(city.value or "Doha")
507
+ g = guidance.value or wbgt_guidance(wbgt_estimate(w.get("temp"), w.get("humidity"), True), "moderate")
508
+ rep = safety_report(w.get("temp"), w.get("humidity"), g["intensity"], tx or "no voice input", l_code)
509
+ if tx:
510
+ rep = f'You said: "{tx}"\n\n{rep}'
511
+ tts_path = tts_to_file(rep, l_code)
512
+ return rep, tts_path
513
+
514
+ def osafety(temp, hum, wi_label, lang_choice):
515
+ l_code = lang_choice.split("(")[-1].strip(")") if "(" in lang_choice else "en"
516
+ lang.value = l_code
517
+ intensity = {"light": "light", "moderate": "moderate", "heavy": "heavy"}.get(
518
+ {tr("en","l"):"light", tr("en","m"):"moderate", tr("en","h"):"heavy"}.get(wi_label, "moderate"), "moderate"
519
+ )
520
+ html, g = _safety_html(temp, hum, intensity, l_code)
521
+ guidance.value = g
522
+ return html
523
+
524
+ def ovoice_tts(txt, l_code):
525
+ return tts_to_file(txt, l_code)
526
+
527
+ # Wiring
528
+ rb.click(oref, [cd], [wd])
529
+ cd.change(oref, [cd], [wd])
530
+ sl_in.change(olg, [sl_in], [])
531
+ vb.click(ov, [vi, vp2, lang, gr.Checkbox(value=False, visible=False)], [vo])
532
+ vb2.click(ovc, [va, lang], [vt, ta])
533
+ tb.click(ovoice_tts, [vt, lang], [ta])
534
+ st_in.change(osafety, [st_in, hu_in, wi_in, sl_in], [rh_out])
535
+ hu_in.change(osafety, [st_in, hu_in, wi_in, sl_in], [rh_out])
536
+ wi_in.change(osafety, [st_in, hu_in, wi_in, sl_in], [rh_out])
537
+ sl_in.change(osafety, [st_in, hu_in, wi_in, sl_in], [rh_out])
538
+ demo.load(osafety, [st_in, hu_in, wi_in, sl_in], [rh_out])
539
+
540
+ return demo
541
+
542
+
543
+ if __name__ == "__main__":
544
+ build_app().launch(server_name="0.0.0.0", server_port=7860)
requirements.txt ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ gradio>=4.44,<6
2
+ transformers>=4.46
3
+ torch>=2.3
4
+ accelerate>=0.33
5
+ qwen-vl-utils>=0.0.8
6
+ datasets>=2.19
7
+ peft>=0.11
8
+ gtts>=2.3
9
+ Pillow>=10
10
+ requests>=2.31
11
+ numpy
12
+ huggingface_hub>=0.24