bChaukar commited on
Commit
ffe8624
Β·
verified Β·
1 Parent(s): c365c26

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -139
README.md CHANGED
@@ -1,142 +1,15 @@
1
- # RISE β€” integrated Colab backend + Hugging Face frontend
 
 
 
 
 
 
 
 
2
 
3
- This package replaces the earlier fragile adapter that depended on variables already loaded in a notebook. The API is self-contained: it loads the models in the API process, and it does **not** call `output.eval_js`, `IPython.display.Audio`, or `run_reading_session()`.
4
 
5
- ## What is integrated
6
 
7
- - **Text generation:** `Qwen/Qwen2.5-1.5B-Instruct`; if it cannot load, the backend tries `google/flan-t5-base`. A small reviewed fallback passage is used if generation fails for a particular passage.
8
- - **Speech recognition:** `openai/whisper-small`, receiving browser-recorded audio converted by `ffmpeg` to mono 16 kHz audio.
9
- - **Pronunciation feedback:** fuzzy word matching and gTTS slow-English pronunciation audio for hesitated, mismatched, or skipped words.
10
- - **Progression:** 5 levels Γ— sentence β†’ paragraph β†’ story; score is 50% word accuracy + 30% pronunciation similarity + 20% completion. Pass threshold is 80%, matching the first code in `Prompt.txt`. A failed attempt retries the same generated passage; passing advances the curriculum.
11
- - **Hand gestures:** 1–5 fingers jump to levels 1–5; thumbs up restarts at Level 1 / sentence; thumbs down ends the session. Browser camera frames are sent to the API for MediaPipe processing.
12
- - **Generation evaluation:** passage word/sentence/paragraph counts, length/structure checks, and fallback rate are included in the end-of-session summary. These heuristic checks do not prove language quality.
13
-
14
- ## Part A β€” Start from a clean Google Colab runtime
15
-
16
- 1. Open a new Colab notebook: https://colab.research.google.com/ . Choose **Runtime β†’ Change runtime type β†’ T4 GPU** if available. Do not run the old notebook's `all_metrics = run_reading_session()` cell.
17
- 2. Upload this ZIP file to the Colab Files panel, or upload `rise_colab_api.py` and `requirements-api.txt` directly.
18
- 3. If you uploaded the ZIP, run:
19
-
20
- ```python
21
- import zipfile
22
- with zipfile.ZipFile('/content/RISE_Complete_Integration.zip') as z:
23
- z.extractall('/content/RISE_Complete_Integration')
24
- ```
25
-
26
- If you uploaded the two files separately, ensure they are in `/content/` and adjust the paths below.
27
-
28
- 4. Install OS and Python dependencies. This may take several minutes:
29
-
30
- ```python
31
- !apt-get update -qq
32
- !apt-get install -y -qq ffmpeg
33
- !pip -q install -r /content/RISE_Complete_Integration/requirements-api.txt
34
- ```
35
-
36
- 5. Start the API as a **separate background process**. This avoids the `asyncio.run() cannot be called from a running event loop` problem in Colab:
37
-
38
- ```python
39
- %cd /content/RISE_Complete_Integration
40
- !nohup uvicorn rise_colab_api:app --host 0.0.0.0 --port 8000 > /content/rise_api.log 2>&1 &
41
- ```
42
-
43
- 6. Give the first model downloads a few minutes, then check the log:
44
-
45
- ```python
46
- !tail -n 60 /content/rise_api.log
47
- ```
48
-
49
- Look for `Application startup complete`. The first run downloads Whisper and the text-generation model; it can take time and uses substantial RAM/GPU memory.
50
-
51
- 7. Test the health endpoint from Python:
52
-
53
- ```python
54
- import requests, time
55
- for i in range(30):
56
- try:
57
- r = requests.get('http://127.0.0.1:8000/api/health', timeout=5)
58
- print(r.status_code, r.json())
59
- break
60
- except requests.RequestException:
61
- print('Waiting for API/model startup…')
62
- time.sleep(10)
63
- ```
64
-
65
- You want HTTP `200` and `"ok": true`. Confirm `models.whisper`, `models.text_generation`, and `models.gesture` are true. If a model is false, inspect `/content/rise_api.log` before proceeding.
66
-
67
- ## Part B β€” Expose the Colab API over HTTPS
68
-
69
- A browser on Hugging Face cannot call Colab's `127.0.0.1`. Use a temporary HTTPS tunnel for development/testing.
70
-
71
- 1. In a **second Colab cell**, start Cloudflare Tunnel:
72
-
73
- ```python
74
- !wget -q https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64 -O /usr/local/bin/cloudflared
75
- !chmod +x /usr/local/bin/cloudflared
76
- !cloudflared tunnel --url http://127.0.0.1:8000 --no-autoupdate
77
- ```
78
-
79
- 2. Wait for a URL ending in `.trycloudflare.com`. Keep this cell running.
80
- 3. Open `https://YOUR-URL.trycloudflare.com/api/health` in a browser. It should show the same health JSON. Copy the URL exactly. Do not add `/api/health` to the frontend base URL.
81
-
82
- **Important:** Colab runtimes stop, and quick-tunnel URLs change. This is a development bridge, not a permanent production backend. Never put secrets in frontend JavaScript.
83
-
84
- ## Part C β€” Connect the widget to the existing Hugging Face Space
85
-
86
- Space: https://huggingface.co/spaces/bChaukar/RISE/tree/main
87
-
88
- **First check the Space's `README.md` front matter.** The uploaded project has custom pages (`index.html`, `dashboard.html`, `assessment.html`, `speaking.html`) as well as an `app.py` described as a basic Gradio welcome UI. The Space SDK decides which entrypoint actually runs:
89
-
90
- - If the Space is intended to serve the existing custom HTML website, set the Space metadata to the Static SDK, with `sdk: static` and `app_file: index.html` (or omit `app_file` if the Space's current static configuration supports auto-detection). Keep all HTML/CSS/JS files at the repository root. A static Space can host the frontend, but it cannot execute the Colab Python models.
91
- - If you keep `sdk: gradio`, uploading `speaking.html` alone does **not** automatically make it a routed page. You must integrate the widget into the actual Gradio `app.py` UI, or switch to the static frontend approach above. Do not change the SDK until you have checked the current `README.md` metadata and know which UI you want to keep.
92
-
93
- Then connect the widget:
94
-
95
- 1. Download `rise-frontend.js` from this package.
96
- 2. In the Space's **Files** tab, upload `rise-frontend.js` at the repository root, next to `speaking.html` and `style.css`.
97
- 3. Edit `speaking.html`. Add this where you want the reading practice widget to appear (usually inside the main content):
98
-
99
- ```html
100
- <div id="rise-reading-app"></div>
101
- ```
102
-
103
- 4. Immediately before `</body>`, add these scripts. Replace the URL with your actual current Cloudflare HTTPS URL:
104
-
105
- ```html
106
- <script>
107
- window.RISE_API_BASE = "https://YOUR-URL.trycloudflare.com";
108
- </script>
109
- <script src="rise-frontend.js"></script>
110
- ```
111
-
112
- 5. Commit the changes in the Space. Wait for the static Space to rebuild/reload, then open its `speaking` page. Grant microphone and camera permission when the browser asks.
113
- 6. Click **Start practice**, then **Start recording**, read the passage, and click **Stop & evaluate**. The result panel should show Whisper's transcript and the metrics. Click **Hear word** next to an error to play gTTS audio. Click **Detect gesture** and show a gesture to the camera.
114
- 7. If the page says the backend cannot be reached, check (a) the Colab API cell/process is still running, (b) the Cloudflare tunnel is still running, (c) `RISE_API_BASE` exactly matches the current tunnel URL, and (d) the `/api/health` URL works in your browser.
115
-
116
- ### If the Space is static HTML
117
-
118
- The browser code only calls the temporary API; the Python API does not run inside a static Space. Keep the Space's existing frontend files. The added `rise-frontend.js` is a widget for `speaking.html`, not a replacement for your entire design.
119
-
120
- ## API endpoints
121
-
122
- - `GET /api/health` β€” server and model status.
123
- - `POST /api/session/start` β€” generate the first passage.
124
- - `GET /api/session/{session_id}` β€” current session state.
125
- - `POST /api/session/{session_id}/evaluate` β€” upload an audio recording and receive transcript, word events, scores, and next passage.
126
- - `GET /api/pronounce/{word}` β€” slow-English pronunciation MP3.
127
- - `POST /api/session/{session_id}/gesture` β€” upload one JPEG camera frame and apply the recognised gesture.
128
- - `POST /api/session/{session_id}/finish` β€” end session and receive summary/generation checks.
129
-
130
- ## Troubleshooting
131
-
132
- - **Connection refused:** read `/content/rise_api.log`; verify `uvicorn` is running and `/api/health` returns 200.
133
- - **Models false in health JSON:** read the error list in health JSON and `/content/rise_api.log`. Model download or RAM/GPU limits are common causes.
134
- - **ffmpeg error / audio upload fails:** confirm `!ffmpeg -version`; use Chrome/Edge and grant microphone permission.
135
- - **Camera gesture unavailable:** allow camera access; verify MediaPipe is true in health JSON and that `hand_landmarker.task` was downloaded.
136
- - **TTS fails:** gTTS requires internet access at the time a pronunciation is requested.
137
- - **Space reloads but the widget is blank:** confirm `rise-frontend.js` is at the same root path and that the HTML contains both the widget `<div>` and script tags.
138
- - **Do not run the old notebook auto-start line:** remove/disable `all_metrics = run_reading_session()` when using this API package. This API owns its own session lifecycle.
139
-
140
- ## Scope and limitations
141
-
142
- This is a development integration. In-memory sessions reset when the API process restarts; a temporary tunnel is not reliable for public production. Speech scores are heuristic estimates from ASR and fuzzy matching, not a clinical or certified pronunciation assessment. The model and generation evaluation code should be tested with real recordings and browser permissions before a live demonstration.
 
1
+ ---
2
+ title: RISE
3
+ emoji: πŸ“š
4
+ colorFrom: blue
5
+ colorTo: purple
6
+ sdk: static
7
+ app_file: index.html
8
+ pinned: false
9
+ ---
10
 
11
+ # RISE
12
 
13
+ AI-powered reading and communication assistance.
14
 
15
+ RISE provides a website for guided speaking practice, reading activities, assessments, and progress tracking.