Spaces:
Running
Running
feat: deploy SignLens web app to Hugging Face Spaces
Browse filesThis view is limited to 50 files because it contains too many changes. ย See raw diff
- .gitattributes +2 -0
- ASL_Alphabet_Chart.png +3 -0
- ISL_Hand_Signs_Official.png +3 -0
- README.md +36 -3
- app.js +2384 -0
- asl_landmarks_weights.json +0 -0
- asl_signs/A.png +0 -0
- asl_signs/B.png +0 -0
- asl_signs/C.png +0 -0
- asl_signs/D.png +0 -0
- asl_signs/E.png +0 -0
- asl_signs/F.png +0 -0
- asl_signs/G.png +0 -0
- asl_signs/H.png +0 -0
- asl_signs/I.png +0 -0
- asl_signs/J.png +0 -0
- asl_signs/K.png +0 -0
- asl_signs/L.png +0 -0
- asl_signs/M.png +0 -0
- asl_signs/N.png +0 -0
- asl_signs/O.png +0 -0
- asl_signs/P.png +0 -0
- asl_signs/Q.png +0 -0
- asl_signs/R.png +0 -0
- asl_signs/S.png +0 -0
- asl_signs/T.png +0 -0
- asl_signs/U.png +0 -0
- asl_signs/V.png +0 -0
- asl_signs/W.png +0 -0
- asl_signs/X.png +0 -0
- asl_signs/Y.png +0 -0
- asl_signs/Z.png +0 -0
- index.html +393 -19
- isl_landmarks_weights.json +0 -0
- isl_signs/1.png +0 -0
- isl_signs/2.png +0 -0
- isl_signs/3.png +0 -0
- isl_signs/4.png +0 -0
- isl_signs/5.png +0 -0
- isl_signs/6.png +0 -0
- isl_signs/7.png +0 -0
- isl_signs/8.png +0 -0
- isl_signs/9.png +0 -0
- isl_signs/A.png +0 -0
- isl_signs/B.png +0 -0
- isl_signs/C.png +0 -0
- isl_signs/D.png +0 -0
- isl_signs/E.png +0 -0
- isl_signs/F.png +0 -0
- isl_signs/G.png +0 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
ASL_Alphabet_Chart.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
ISL_Hand_Signs_Official.png filter=lfs diff=lfs merge=lfs -text
|
ASL_Alphabet_Chart.png
ADDED
|
Git LFS Details
|
ISL_Hand_Signs_Official.png
ADDED
|
Git LFS Details
|
README.md
CHANGED
|
@@ -1,10 +1,43 @@
|
|
| 1 |
---
|
| 2 |
title: SignLens
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
colorTo: indigo
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
---
|
| 9 |
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
title: SignLens
|
| 3 |
+
emoji: ๐ค
|
| 4 |
+
colorFrom: yellow
|
| 5 |
colorTo: indigo
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
| 8 |
+
license: apache-2.0
|
| 9 |
+
tags:
|
| 10 |
+
- sign-language-detection
|
| 11 |
+
- computer-vision
|
| 12 |
+
- mediapipe
|
| 13 |
+
- indian-sign-language
|
| 14 |
+
- american-sign-language
|
| 15 |
+
- accessibility
|
| 16 |
+
- neural-network
|
| 17 |
---
|
| 18 |
|
| 19 |
+
# ๐ค SignLens: Real-Time Sign Language Translation & Avatar Practice Studio
|
| 20 |
+
|
| 21 |
+
Welcome to **SignLens** running live on Hugging Face Spaces!
|
| 22 |
+
|
| 23 |
+
SignLens is a high-speed, client-side neural landmark translation engine that translates **Indian Sign Language (ISL)** and **American Sign Language (ASL)** directly in your browser with zero server latency.
|
| 24 |
+
|
| 25 |
+
## โจ Features
|
| 26 |
+
1. **Live Camera Recognition**:
|
| 27 |
+
- High-speed 3D landmark tracking powered by MediaPipe WebVision.
|
| 28 |
+
- Dual-mode support for both **ISL** (Dual-Hand 35 classes) and **ASL** (Single-Hand 28 classes).
|
| 29 |
+
- In-browser neural forward pass with confidence thresholds, hold gauge, and streak confirmation.
|
| 30 |
+
2. **Auto-Framing Word & Sentence Builder**:
|
| 31 |
+
- Eliminates gesture spam and flickering.
|
| 32 |
+
- Dynamic word boundary detection and Text-to-Speech (TTS) voice readout.
|
| 33 |
+
3. **VISUALLY โ 3D Skeletal Avatar Studio**:
|
| 34 |
+
- Type any phrase or click sample prompts to watch the interactive avatar demonstrate correct hand and arm motions step-by-step.
|
| 35 |
+
- Interactive scrub bar, playback speed control (0.5x, 1.0x, 1.5x), and live letter timeline.
|
| 36 |
+
- Top-right real-human sign reference card synchronized with the skeleton.
|
| 37 |
+
4. **Official Sign Reference Guide**:
|
| 38 |
+
- Full dictionary of official poses, hand counts (1-Hand vs 2-Hand rules), and high-resolution photo popups.
|
| 39 |
+
|
| 40 |
+
## ๐ Project Links
|
| 41 |
+
- **Model Card & Evaluation**: [Hugging Face: SignLens-Landmark-Model](https://huggingface.co/venmugilrajan/SignLens-Landmark-Model)
|
| 42 |
+
- **Source Code**: [GitHub: Sign_lang](https://github.com/venmugilrajan/Sign_lang)
|
| 43 |
+
- **Author**: Venmugil Rajan
|
app.js
ADDED
|
@@ -0,0 +1,2384 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**
|
| 2 |
+
* SignLens โ MediaPipe Landmark Neural Network Edition
|
| 3 |
+
*
|
| 4 |
+
* Runs classification directly in the browser using the weights
|
| 5 |
+
* of our custom MLP Neural Network trained on the 21 hand landmarks.
|
| 6 |
+
*/
|
| 7 |
+
|
| 8 |
+
// โโ Config โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 9 |
+
let STREAK_NEEDED = 5; // frames of same letter to capture (settings sheet, matches native)
|
| 10 |
+
const COOLDOWN_MS = 350; // ms after capture before detecting again (fluid sign transitions)
|
| 11 |
+
let SPACE_FRAMES = 90; // no-hand frames before auto word push (~3.0s at 30fps, plenty of transition time)
|
| 12 |
+
let MIN_CONFIDENCE = 0.50; // minimum classifier confidence (settings sheet, matches native 0.40-0.50)
|
| 13 |
+
const MIN_HAND_SPAN = 35; // minimum pixel distance between wrist and middle MCP to reject ghost hands
|
| 14 |
+
|
| 15 |
+
// โโ MediaPipe Hand Connections โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 16 |
+
const CONNECTIONS = [
|
| 17 |
+
[0,1],[1,2],[2,3],[3,4],
|
| 18 |
+
[0,5],[5,6],[6,7],[7,8],
|
| 19 |
+
[0,9],[9,10],[10,11],[11,12],
|
| 20 |
+
[0,13],[13,14],[14,15],[15,16],
|
| 21 |
+
[0,17],[17,18],[18,19],[19,20],
|
| 22 |
+
[5,9],[9,13],[13,17],
|
| 23 |
+
];
|
| 24 |
+
|
| 25 |
+
const NODE_COLORS = {
|
| 26 |
+
0:'#ffffff',
|
| 27 |
+
1:'#f2856a',2:'#f2856a',3:'#f2856a',4:'#f2856a',
|
| 28 |
+
5:'#f0a53f',6:'#f0a53f',7:'#f0a53f',8:'#f0a53f',
|
| 29 |
+
9:'#6ec9a0',10:'#6ec9a0',11:'#6ec9a0',12:'#6ec9a0',
|
| 30 |
+
13:'#5aa9e6',14:'#5aa9e6',15:'#5aa9e6',16:'#5aa9e6',
|
| 31 |
+
17:'#b98ae0',18:'#b98ae0',19:'#b98ae0',20:'#b98ae0',
|
| 32 |
+
};
|
| 33 |
+
|
| 34 |
+
const EDGE_COLORS = [
|
| 35 |
+
'#f2856a','#f2856a','#f2856a','#f2856a', // thumb 0-3
|
| 36 |
+
'#f0a53f','#f0a53f','#f0a53f','#f0a53f', // index 4-7
|
| 37 |
+
'#6ec9a0','#6ec9a0','#6ec9a0','#6ec9a0', // middle 8-11
|
| 38 |
+
'#5aa9e6','#5aa9e6','#5aa9e6','#5aa9e6', // ring 12-15
|
| 39 |
+
'#b98ae0','#b98ae0','#b98ae0','#b98ae0', // pinky 16-19
|
| 40 |
+
'#6a6a76','#6a6a76','#6a6a76', // palm 20-22
|
| 41 |
+
];
|
| 42 |
+
|
| 43 |
+
// โโ State โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 44 |
+
let currentMode = 'asl';
|
| 45 |
+
let paused = false;
|
| 46 |
+
let inCooldown = false;
|
| 47 |
+
let cameraActive = false;
|
| 48 |
+
|
| 49 |
+
let currentWord = '';
|
| 50 |
+
let letterConfidences = [];
|
| 51 |
+
let sentenceWords = [];
|
| 52 |
+
|
| 53 |
+
let streakLetter = null;
|
| 54 |
+
let streakCount = 0;
|
| 55 |
+
let noHandCount = 0;
|
| 56 |
+
|
| 57 |
+
// State tracking to prevent runaway letter repeat without releasing hand
|
| 58 |
+
let lastCapturedLetter = '';
|
| 59 |
+
let releasedSinceLastCapture = true;
|
| 60 |
+
let releaseFrames = 0;
|
| 61 |
+
const RELEASE_FRAMES_NEEDED = 3;
|
| 62 |
+
|
| 63 |
+
// State tracking to prevent runaway word repeat without clear user intent
|
| 64 |
+
let lastCommittedWord = '';
|
| 65 |
+
|
| 66 |
+
// Trained Neural Network model weights (kept for OFFLINE fallback only)
|
| 67 |
+
let aslWeights = null;
|
| 68 |
+
let islWeights = null;
|
| 69 |
+
let currentWeights = null;
|
| 70 |
+
|
| 71 |
+
// โโ Server-side inference state โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 72 |
+
// When Flask is running, all classification is done server-side (predict_proba
|
| 73 |
+
// on the sklearn model) for feature parity with run_native.py.
|
| 74 |
+
// If the server is unreachable, we fall back to the JS MLP.
|
| 75 |
+
let serverAvailable = false; // set true after first successful /health probe
|
| 76 |
+
let serverCheckPending = false;
|
| 77 |
+
|
| 78 |
+
// โโ DOM โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 79 |
+
const video = document.getElementById('webcam-video');
|
| 80 |
+
const displayCanvas = document.getElementById('display-canvas');
|
| 81 |
+
const dctx = displayCanvas.getContext('2d');
|
| 82 |
+
const noHandOverlay = document.getElementById('no-hand-overlay');
|
| 83 |
+
const detectedLetter = document.getElementById('detected-letter');
|
| 84 |
+
const detectedConf = document.getElementById('detected-conf');
|
| 85 |
+
const streakBar = document.getElementById('streak-bar');
|
| 86 |
+
const streakLabel = document.getElementById('streak-label');
|
| 87 |
+
const handBadge = document.getElementById('hand-badge');
|
| 88 |
+
const statusPill = document.getElementById('status-pill');
|
| 89 |
+
const statusText = document.getElementById('status-text');
|
| 90 |
+
const wordDisplay = document.getElementById('word-display');
|
| 91 |
+
const sentenceDisplay = document.getElementById('sentence-display');
|
| 92 |
+
const wordChips = document.getElementById('word-chips');
|
| 93 |
+
const modeToggle = document.getElementById('mode-toggle');
|
| 94 |
+
const toast = document.getElementById('toast');
|
| 95 |
+
|
| 96 |
+
// โโ Utilities โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ๏ฟฝ๏ฟฝ๏ฟฝโโโโโโโโโโโโโโโ
|
| 97 |
+
function showToast(msg, ms = 2000) {
|
| 98 |
+
toast.textContent = msg;
|
| 99 |
+
toast.classList.add('show');
|
| 100 |
+
setTimeout(() => toast.classList.remove('show'), ms);
|
| 101 |
+
}
|
| 102 |
+
function setStatus(cls, msg) {
|
| 103 |
+
statusPill.className = `status-pill ${cls}`;
|
| 104 |
+
statusText.textContent = msg;
|
| 105 |
+
}
|
| 106 |
+
function setStreakVisual(pct) {
|
| 107 |
+
streakBar.style.width = `${pct}%`;
|
| 108 |
+
// .dial-ring reads --streak as a registered <number> and interpolates it.
|
| 109 |
+
document.documentElement.style.setProperty('--streak', pct);
|
| 110 |
+
}
|
| 111 |
+
function resetStreak() {
|
| 112 |
+
streakLetter = null; streakCount = 0;
|
| 113 |
+
setStreakVisual(0);
|
| 114 |
+
streakLabel.textContent = `0 / ${STREAK_NEEDED}`;
|
| 115 |
+
}
|
| 116 |
+
|
| 117 |
+
// โโ Word / Sentence โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 118 |
+
function renderWord() {
|
| 119 |
+
wordDisplay.innerHTML = currentWord
|
| 120 |
+
? currentWord
|
| 121 |
+
: '<span class="ph">Start signingโฆ</span>';
|
| 122 |
+
}
|
| 123 |
+
function renderSentence() {
|
| 124 |
+
sentenceDisplay.innerHTML = sentenceWords.length
|
| 125 |
+
? sentenceWords.join(' ')
|
| 126 |
+
: '<em class="ph">Your sentence will appear hereโฆ</em>';
|
| 127 |
+
wordChips.innerHTML = '';
|
| 128 |
+
sentenceWords.forEach(w => {
|
| 129 |
+
const c = document.createElement('span');
|
| 130 |
+
c.className = 'word-chip'; c.textContent = w;
|
| 131 |
+
wordChips.appendChild(c);
|
| 132 |
+
});
|
| 133 |
+
}
|
| 134 |
+
async function pushWord() {
|
| 135 |
+
let w = currentWord.trim();
|
| 136 |
+
if (w) {
|
| 137 |
+
// Attempt spellcheck via backend API if available
|
| 138 |
+
try {
|
| 139 |
+
const res = await fetch('/spellcheck', {
|
| 140 |
+
method: 'POST',
|
| 141 |
+
headers: {'Content-Type': 'application/json'},
|
| 142 |
+
body: JSON.stringify({ word: w, confidences: letterConfidences })
|
| 143 |
+
});
|
| 144 |
+
if (res.ok) {
|
| 145 |
+
const data = await res.json();
|
| 146 |
+
if (data.corrected) {
|
| 147 |
+
w = data.corrected.toUpperCase();
|
| 148 |
+
}
|
| 149 |
+
}
|
| 150 |
+
} catch (e) {
|
| 151 |
+
// Backend not present or offline, keep raw word
|
| 152 |
+
}
|
| 153 |
+
|
| 154 |
+
// Gating: Don't repeat the exact same word back-to-back if triggered by background jitter
|
| 155 |
+
const isDuplicateAutoPush = (sentenceWords.length > 0 && sentenceWords[sentenceWords.length - 1] === w && lastCommittedWord === w);
|
| 156 |
+
if (!isDuplicateAutoPush) {
|
| 157 |
+
sentenceWords.push(w);
|
| 158 |
+
lastCommittedWord = w;
|
| 159 |
+
renderSentence();
|
| 160 |
+
showToast(`โ "${w}"`);
|
| 161 |
+
}
|
| 162 |
+
}
|
| 163 |
+
currentWord = '';
|
| 164 |
+
letterConfidences = [];
|
| 165 |
+
renderWord();
|
| 166 |
+
resetStreak();
|
| 167 |
+
noHandCount = 0;
|
| 168 |
+
lastCapturedLetter = '';
|
| 169 |
+
releasedSinceLastCapture = true;
|
| 170 |
+
releaseFrames = 0;
|
| 171 |
+
}
|
| 172 |
+
|
| 173 |
+
// โโ Letter capture โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 174 |
+
function captureLetter(letter, confidence = 0.85) {
|
| 175 |
+
if (inCooldown) return;
|
| 176 |
+
const l = letter.toLowerCase();
|
| 177 |
+
if (l === 'space' || l === '_') { pushWord(); return; }
|
| 178 |
+
if (l === 'del' || l === 'delete') {
|
| 179 |
+
currentWord = currentWord.slice(0, -1);
|
| 180 |
+
letterConfidences.pop();
|
| 181 |
+
renderWord();
|
| 182 |
+
resetStreak();
|
| 183 |
+
lastCapturedLetter = '';
|
| 184 |
+
releasedSinceLastCapture = true;
|
| 185 |
+
return;
|
| 186 |
+
}
|
| 187 |
+
if (l === 'nothing') return;
|
| 188 |
+
|
| 189 |
+
currentWord += letter.toUpperCase();
|
| 190 |
+
letterConfidences.push(confidence);
|
| 191 |
+
renderWord();
|
| 192 |
+
|
| 193 |
+
lastCapturedLetter = letter.toUpperCase();
|
| 194 |
+
releasedSinceLastCapture = false;
|
| 195 |
+
releaseFrames = 0;
|
| 196 |
+
|
| 197 |
+
detectedLetter.classList.remove('flash');
|
| 198 |
+
void detectedLetter.offsetWidth;
|
| 199 |
+
detectedLetter.classList.add('flash');
|
| 200 |
+
|
| 201 |
+
resetStreak();
|
| 202 |
+
noHandCount = 0;
|
| 203 |
+
inCooldown = true;
|
| 204 |
+
setTimeout(() => { inCooldown = false; }, COOLDOWN_MS);
|
| 205 |
+
}
|
| 206 |
+
|
| 207 |
+
// โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 208 |
+
// โโ Landmark Neural Network Classifier โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 209 |
+
// โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 210 |
+
|
| 211 |
+
/** 15 finger flexion triplets (vertex is the middle index) */
|
| 212 |
+
const ANGLE_TRIPLETS = [
|
| 213 |
+
[0,1,2], [1,2,3], [2,3,4],
|
| 214 |
+
[0,5,6], [5,6,7], [6,7,8],
|
| 215 |
+
[0,9,10], [9,10,11], [10,11,12],
|
| 216 |
+
[0,13,14], [13,14,15], [14,15,16],
|
| 217 |
+
[0,17,18], [17,18,19], [18,19,20],
|
| 218 |
+
];
|
| 219 |
+
|
| 220 |
+
/** Angle at vertex b formed by a-b-c, in radians normalized to [0,1] */
|
| 221 |
+
function calculateAngle3D(a, b, c) {
|
| 222 |
+
const bax = a[0]-b[0], bay = a[1]-b[1], baz = a[2]-b[2];
|
| 223 |
+
const bcx = c[0]-b[0], bcy = c[1]-b[1], bcz = c[2]-b[2];
|
| 224 |
+
const dot = bax*bcx + bay*bcy + baz*bcz;
|
| 225 |
+
const nba = Math.sqrt(bax*bax + bay*bay + baz*baz);
|
| 226 |
+
const nbc = Math.sqrt(bcx*bcx + bcy*bcy + bcz*bcz);
|
| 227 |
+
const cos = Math.min(1, Math.max(-1, dot / (nba*nbc + 1e-6)));
|
| 228 |
+
return Math.acos(cos) / Math.PI;
|
| 229 |
+
}
|
| 230 |
+
|
| 231 |
+
/**
|
| 232 |
+
* Builds the 78-D feature vector the MLP was trained on:
|
| 233 |
+
* 63 aspect-corrected, wrist-centred, palm-scaled, rotation-aligned coords
|
| 234 |
+
* + 15 finger flexion angles.
|
| 235 |
+
*
|
| 236 |
+
* Mirrors train_landmark_classifier.extract_features_from_image() exactly,
|
| 237 |
+
* including its rotation convention. Do not "fix" the rotation here alone --
|
| 238 |
+
* the shipped weights were trained with it (see notes).
|
| 239 |
+
*/
|
| 240 |
+
function normalizeLandmarks(lm, w, h, handedness = "Right", applyHandednessMirror = false) {
|
| 241 |
+
w = w || video.videoWidth || 640;
|
| 242 |
+
h = h || video.videoHeight || 480;
|
| 243 |
+
|
| 244 |
+
// Aspect-ratio corrected pixel space: (x*w, y*h, z*w)
|
| 245 |
+
const pts = lm.map(p => [p.x * w, p.y * h, p.z * w]);
|
| 246 |
+
|
| 247 |
+
// Mirror Left hand to match Right hand dataset distribution (ASL single hand)
|
| 248 |
+
if (applyHandednessMirror && handedness === "Left") {
|
| 249 |
+
const wristX = pts[0][0];
|
| 250 |
+
pts.forEach(p => {
|
| 251 |
+
p[0] = wristX - (p[0] - wristX);
|
| 252 |
+
});
|
| 253 |
+
}
|
| 254 |
+
|
| 255 |
+
// Centre on the wrist
|
| 256 |
+
const wrist = pts[0];
|
| 257 |
+
const cen = pts.map(p => [p[0]-wrist[0], p[1]-wrist[1], p[2]-wrist[2]]);
|
| 258 |
+
|
| 259 |
+
// Palm scale = 2D wrist -> middle MCP distance
|
| 260 |
+
let scale = Math.sqrt(cen[9][0]*cen[9][0] + cen[9][1]*cen[9][1]);
|
| 261 |
+
if (scale < 1e-4) scale = 1;
|
| 262 |
+
|
| 263 |
+
// 2D rotation keyed on landmark 9 against the negative Y axis
|
| 264 |
+
const angle = Math.atan2(cen[9][0]/scale, -cen[9][1]/scale);
|
| 265 |
+
const ca = Math.cos(angle), sa = Math.sin(angle);
|
| 266 |
+
|
| 267 |
+
const features = [];
|
| 268 |
+
// ISL uses the mathematically correct rotation matrix [[cos, -sin], [sin, cos]]
|
| 269 |
+
// that aligns landmark 9 to vertical 12 o'clock, cancelling any hand tilt.
|
| 270 |
+
const isIslMode = (typeof currentMode !== 'undefined' && currentMode === 'isl');
|
| 271 |
+
cen.forEach(p => {
|
| 272 |
+
const x = p[0]/scale, y = p[1]/scale;
|
| 273 |
+
if (isIslMode) {
|
| 274 |
+
features.push(x*ca + y*sa);
|
| 275 |
+
features.push(-x*sa + y*ca);
|
| 276 |
+
} else {
|
| 277 |
+
// ASL legacy rotation
|
| 278 |
+
features.push(x*ca - y*sa);
|
| 279 |
+
features.push(x*sa + y*ca);
|
| 280 |
+
}
|
| 281 |
+
features.push(p[2]/scale); // z is scaled but not rotated
|
| 282 |
+
});
|
| 283 |
+
|
| 284 |
+
// Angles come from the raw pixel points, not the normalized ones
|
| 285 |
+
ANGLE_TRIPLETS.forEach(([i, j, k]) =>
|
| 286 |
+
features.push(calculateAngle3D(pts[i], pts[j], pts[k]))
|
| 287 |
+
);
|
| 288 |
+
|
| 289 |
+
return features; // 63 + 15 = 78
|
| 290 |
+
}
|
| 291 |
+
|
| 292 |
+
/** Matrix multiplication helper */
|
| 293 |
+
function matMul(input, weights, bias) {
|
| 294 |
+
const output = [];
|
| 295 |
+
const numOutputs = bias.length;
|
| 296 |
+
const numInputs = input.length;
|
| 297 |
+
for (let j = 0; j < numOutputs; j++) {
|
| 298 |
+
let sum = bias[j];
|
| 299 |
+
for (let i = 0; i < numInputs; i++) {
|
| 300 |
+
sum += input[i] * weights[i][j];
|
| 301 |
+
}
|
| 302 |
+
output.push(sum);
|
| 303 |
+
}
|
| 304 |
+
return output;
|
| 305 |
+
}
|
| 306 |
+
|
| 307 |
+
/** ReLU activation function */
|
| 308 |
+
function relu(arr) {
|
| 309 |
+
return arr.map(x => Math.max(0, x));
|
| 310 |
+
}
|
| 311 |
+
|
| 312 |
+
/** Softmax activation function */
|
| 313 |
+
function softmax(arr) {
|
| 314 |
+
const max = Math.max(...arr);
|
| 315 |
+
const exps = arr.map(x => Math.exp(x - max));
|
| 316 |
+
const sum = exps.reduce((a, b) => a + b, 0);
|
| 317 |
+
return exps.map(x => x / sum);
|
| 318 |
+
}
|
| 319 |
+
|
| 320 |
+
/**
|
| 321 |
+
* Probes /health to find out whether the Flask backend is up.
|
| 322 |
+
* Called once at startup and retried whenever a classify() call fails.
|
| 323 |
+
*/
|
| 324 |
+
async function probeServer() {
|
| 325 |
+
if (serverCheckPending) return;
|
| 326 |
+
serverCheckPending = true;
|
| 327 |
+
try {
|
| 328 |
+
const r = await fetch('/health', { signal: AbortSignal.timeout(1500) });
|
| 329 |
+
if (r.ok) {
|
| 330 |
+
if (!serverAvailable) {
|
| 331 |
+
serverAvailable = true;
|
| 332 |
+
console.log('โ
Flask backend detected โ using server-side inference');
|
| 333 |
+
setStatus('ready', 'Connected');
|
| 334 |
+
}
|
| 335 |
+
} else {
|
| 336 |
+
serverAvailable = false;
|
| 337 |
+
}
|
| 338 |
+
} catch {
|
| 339 |
+
serverAvailable = false;
|
| 340 |
+
} finally {
|
| 341 |
+
serverCheckPending = false;
|
| 342 |
+
// If server is currently offline, automatically retry in 2.5s
|
| 343 |
+
if (!serverAvailable && cameraActive) {
|
| 344 |
+
setTimeout(probeServer, 2500);
|
| 345 |
+
}
|
| 346 |
+
}
|
| 347 |
+
}
|
| 348 |
+
|
| 349 |
+
/** Load MLP weights from JSON files (used for OFFLINE fallback only) */
|
| 350 |
+
async function loadLandmarkModel() {
|
| 351 |
+
try {
|
| 352 |
+
const resAsl = await fetch('asl_landmarks_weights.json');
|
| 353 |
+
if (resAsl.ok) {
|
| 354 |
+
aslWeights = await resAsl.json();
|
| 355 |
+
console.log('๐ง ASL Neural Network loaded (offline fallback)');
|
| 356 |
+
}
|
| 357 |
+
const resIsl = await fetch('isl_landmarks_weights.json');
|
| 358 |
+
if (resIsl.ok) {
|
| 359 |
+
islWeights = await resIsl.json();
|
| 360 |
+
console.log('๐ง ISL Neural Network loaded (offline fallback)');
|
| 361 |
+
}
|
| 362 |
+
currentWeights = currentMode === 'asl' ? aslWeights : islWeights;
|
| 363 |
+
} catch (err) {
|
| 364 |
+
console.error('Failed to load MLP weights:', err);
|
| 365 |
+
}
|
| 366 |
+
}
|
| 367 |
+
|
| 368 |
+
/**
|
| 369 |
+
* Assembles the 78-D (single-hand) or 156-D (dual-hand) feature vector.
|
| 370 |
+
*
|
| 371 |
+
* Sorting is on the RAW wrist x of the unflipped frame, identical to
|
| 372 |
+
* run_native.extract_landmarks(). If these two ever disagree, every
|
| 373 |
+
* two-handed sign silently decodes to the wrong letter.
|
| 374 |
+
*/
|
| 375 |
+
function buildFeatureVector(multiHandLandmarks, isDualMode, multiHandedness, w, h) {
|
| 376 |
+
if (!isDualMode) {
|
| 377 |
+
const handedness = (multiHandedness && multiHandedness[0] && multiHandedness[0].label)
|
| 378 |
+
? multiHandedness[0].label
|
| 379 |
+
: 'Right';
|
| 380 |
+
// In ASL single hand mode, mirror Left hands to match Right hand dataset
|
| 381 |
+
return normalizeLandmarks(multiHandLandmarks[0], w, h, handedness, true);
|
| 382 |
+
}
|
| 383 |
+
|
| 384 |
+
// Dual mode (ISL):
|
| 385 |
+
// Single-hand ISL model is trained on BOTH Left & Right hands natively,
|
| 386 |
+
// and two-handed classes strictly require both hands.
|
| 387 |
+
// No fragile handedness guessing/mirroring is needed.
|
| 388 |
+
const hands = multiHandLandmarks
|
| 389 |
+
.map(lm => ({ x: lm[0].x, feat: normalizeLandmarks(lm, w, h, 'Right', false) }))
|
| 390 |
+
.sort((a, b) => a.x - b.x);
|
| 391 |
+
|
| 392 |
+
const slot1 = hands[0].feat;
|
| 393 |
+
const slot2 = hands.length > 1 ? hands[1].feat : new Array(78).fill(0);
|
| 394 |
+
return slot1.concat(slot2);
|
| 395 |
+
}
|
| 396 |
+
|
| 397 |
+
// Helper to check if hand landmarks represent a valid, non-ghost hand
|
| 398 |
+
function isValidHand(lm) {
|
| 399 |
+
if (!lm || lm.length < 21) return false;
|
| 400 |
+
const w = displayCanvas.width || 640;
|
| 401 |
+
const h = displayCanvas.height || 480;
|
| 402 |
+
|
| 403 |
+
// Wrist (0) to Middle MCP (9) distance in pixels
|
| 404 |
+
const dx = (lm[9].x - lm[0].x) * w;
|
| 405 |
+
const dy = (lm[9].y - lm[0].y) * h;
|
| 406 |
+
const span = Math.sqrt(dx * dx + dy * dy);
|
| 407 |
+
|
| 408 |
+
// Reject tiny noise clusters / background artifacts
|
| 409 |
+
if (span < MIN_HAND_SPAN) return false;
|
| 410 |
+
|
| 411 |
+
// Check landmark coordinates are reasonably within frame [-0.2, 1.2]
|
| 412 |
+
for (let i = 0; i < lm.length; i++) {
|
| 413 |
+
if (isNaN(lm[i].x) || isNaN(lm[i].y)) return false;
|
| 414 |
+
if (lm[i].x < -0.2 || lm[i].x > 1.2 || lm[i].y < -0.2 || lm[i].y > 1.2) return false;
|
| 415 |
+
}
|
| 416 |
+
return true;
|
| 417 |
+
}
|
| 418 |
+
|
| 419 |
+
/**
|
| 420 |
+
* JS MLP fallback โ used ONLY when Flask server is unreachable.
|
| 421 |
+
* Runs the MLP forward pass in-browser using the cached weight JSON.
|
| 422 |
+
*/
|
| 423 |
+
function classifyJS(featureVector) {
|
| 424 |
+
if (!currentWeights) return { letter: 'nothing', confidence: 0.0 };
|
| 425 |
+
|
| 426 |
+
const x = featureVector;
|
| 427 |
+
if (x.length !== currentWeights.w0.length) {
|
| 428 |
+
return { letter: 'nothing', confidence: 0.0 };
|
| 429 |
+
}
|
| 430 |
+
|
| 431 |
+
let h = relu(matMul(x, currentWeights.w0, currentWeights.b0));
|
| 432 |
+
h = relu(matMul(h, currentWeights.w1, currentWeights.b1));
|
| 433 |
+
if (currentWeights.w2 && currentWeights.b2) {
|
| 434 |
+
if (currentWeights.w3 && currentWeights.b3) {
|
| 435 |
+
h = relu(matMul(h, currentWeights.w2, currentWeights.b2));
|
| 436 |
+
h = matMul(h, currentWeights.w3, currentWeights.b3);
|
| 437 |
+
} else {
|
| 438 |
+
h = matMul(h, currentWeights.w2, currentWeights.b2);
|
| 439 |
+
}
|
| 440 |
+
}
|
| 441 |
+
|
| 442 |
+
let probs = softmax(h);
|
| 443 |
+
|
| 444 |
+
// โโ ISL Hand-Count Constrained Gating โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 445 |
+
if (currentMode === 'isl' && x.length === 156) {
|
| 446 |
+
const isOneHand = x.slice(78).every(v => Math.abs(v) < 1e-4);
|
| 447 |
+
const islSingleHand = new Set(['1', '2', '3', '4', '5', '6', '7', '8', '9', 'C', 'I', 'J', 'L', 'O', 'Q', 'U', 'V']);
|
| 448 |
+
for (let i = 0; i < probs.length; i++) {
|
| 449 |
+
const cls = currentWeights.classes[i];
|
| 450 |
+
if (isOneHand ? !islSingleHand.has(cls) : islSingleHand.has(cls)) {
|
| 451 |
+
probs[i] = 0;
|
| 452 |
+
}
|
| 453 |
+
}
|
| 454 |
+
const pSum = probs.reduce((a, b) => a + b, 0);
|
| 455 |
+
if (pSum > 0) {
|
| 456 |
+
probs = probs.map(p => p / pSum);
|
| 457 |
+
}
|
| 458 |
+
}
|
| 459 |
+
|
| 460 |
+
let maxIdx = 0, maxProb = 0, secondProb = 0;
|
| 461 |
+
for (let i = 0; i < probs.length; i++) {
|
| 462 |
+
if (probs[i] > maxProb) { secondProb = maxProb; maxProb = probs[i]; maxIdx = i; }
|
| 463 |
+
else if (probs[i] > secondProb) { secondProb = probs[i]; }
|
| 464 |
+
}
|
| 465 |
+
if ((maxProb - secondProb) < 0.08 && maxProb < 0.85) {
|
| 466 |
+
return { letter: 'nothing', confidence: maxProb };
|
| 467 |
+
}
|
| 468 |
+
return { letter: currentWeights.classes[maxIdx], confidence: maxProb };
|
| 469 |
+
}
|
| 470 |
+
|
| 471 |
+
/**
|
| 472 |
+
* PRIMARY classify() โ async.
|
| 473 |
+
*
|
| 474 |
+
* Strategy:
|
| 475 |
+
* 1. Build the feature vector in JS (normalization is correct and fast).
|
| 476 |
+
* 2. POST it to Flask /predict_landmarks โ uses sklearn's calibrated
|
| 477 |
+
* predict_proba(), identical to run_native.py. This is why signs that
|
| 478 |
+
* work in native Python also work in the browser.
|
| 479 |
+
* 3. If the server is unavailable, fall back to the JS MLP.
|
| 480 |
+
*/
|
| 481 |
+
async function classify(multiHandLandmarks, multiHandedness) {
|
| 482 |
+
if (!multiHandLandmarks || multiHandLandmarks.length === 0) {
|
| 483 |
+
return { letter: 'nothing', confidence: 0.0 };
|
| 484 |
+
}
|
| 485 |
+
|
| 486 |
+
// Determine if active model is dual-hand (156-D) or single-hand (78-D)
|
| 487 |
+
const isDualMode = currentWeights ? currentWeights.w0.length === 156 : false;
|
| 488 |
+
|
| 489 |
+
// Build feature vector with true display/camera aspect ratio
|
| 490 |
+
const cw = (displayCanvas && displayCanvas.width) || (video && video.videoWidth) || 640;
|
| 491 |
+
const ch = (displayCanvas && displayCanvas.height) || (video && video.videoHeight) || 480;
|
| 492 |
+
const featureVector = buildFeatureVector(multiHandLandmarks, isDualMode, multiHandedness, cw, ch);
|
| 493 |
+
|
| 494 |
+
// โโ PATH A: Server-side inference (preferred) โโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 495 |
+
if (serverAvailable) {
|
| 496 |
+
try {
|
| 497 |
+
const response = await fetch('/predict_landmarks', {
|
| 498 |
+
method: 'POST',
|
| 499 |
+
headers: { 'Content-Type': 'application/json' },
|
| 500 |
+
body: JSON.stringify({ mode: currentMode, landmarks: featureVector }),
|
| 501 |
+
signal: AbortSignal.timeout(1500), // 1500ms resilient timeout per frame
|
| 502 |
+
});
|
| 503 |
+
|
| 504 |
+
if (response.ok) {
|
| 505 |
+
const data = await response.json();
|
| 506 |
+
return {
|
| 507 |
+
letter: data.letter || 'nothing',
|
| 508 |
+
confidence: data.confidence || 0.0,
|
| 509 |
+
top3: data.top3 || [],
|
| 510 |
+
};
|
| 511 |
+
}
|
| 512 |
+
|
| 513 |
+
// Non-200 response โ server up but something is wrong
|
| 514 |
+
console.warn('[classify] Server error:', response.status);
|
| 515 |
+
return { letter: 'nothing', confidence: 0.0 };
|
| 516 |
+
|
| 517 |
+
} catch (err) {
|
| 518 |
+
// Timeout or network error โ mark server down and re-probe
|
| 519 |
+
console.warn('[classify] Server unreachable, switching to JS fallback:', err.name);
|
| 520 |
+
serverAvailable = false;
|
| 521 |
+
showToast('โ ๏ธ Server offline โ using JS fallback', 3000);
|
| 522 |
+
setStatus('loading', 'JS fallback');
|
| 523 |
+
probeServer(); // async retry in background
|
| 524 |
+
}
|
| 525 |
+
}
|
| 526 |
+
|
| 527 |
+
// โโ PATH B: JS MLP fallback โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 528 |
+
if (!currentWeights) return { letter: 'nothing', confidence: 0.0 };
|
| 529 |
+
return classifyJS(featureVector);
|
| 530 |
+
}
|
| 531 |
+
|
| 532 |
+
// โโ Process classification result โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 533 |
+
function processResult(letter, confidence) {
|
| 534 |
+
const conf = Math.round(confidence * 100);
|
| 535 |
+
|
| 536 |
+
detectedLetter.textContent = (letter === 'nothing' || confidence < MIN_CONFIDENCE)
|
| 537 |
+
? ''
|
| 538 |
+
: letter.toUpperCase();
|
| 539 |
+
|
| 540 |
+
detectedConf.textContent = letter === 'nothing'
|
| 541 |
+
? 'Unclear gesture'
|
| 542 |
+
: `${conf}% ยท ${letter.toUpperCase()}`;
|
| 543 |
+
|
| 544 |
+
const effective = (confidence >= MIN_CONFIDENCE && letter !== 'nothing') ? letter : null;
|
| 545 |
+
|
| 546 |
+
// Track release for repeating the same letter
|
| 547 |
+
if (!effective || effective !== lastCapturedLetter) {
|
| 548 |
+
releaseFrames++;
|
| 549 |
+
if (releaseFrames >= RELEASE_FRAMES_NEEDED) {
|
| 550 |
+
releasedSinceLastCapture = true;
|
| 551 |
+
}
|
| 552 |
+
} else {
|
| 553 |
+
releaseFrames = 0;
|
| 554 |
+
}
|
| 555 |
+
|
| 556 |
+
if (paused || inCooldown || !effective) {
|
| 557 |
+
if (!effective) resetStreak();
|
| 558 |
+
return;
|
| 559 |
+
}
|
| 560 |
+
|
| 561 |
+
// Prevent spamming the exact same letter repeatedly without releasing hand
|
| 562 |
+
if (effective.toUpperCase() === lastCapturedLetter && !releasedSinceLastCapture) {
|
| 563 |
+
resetStreak();
|
| 564 |
+
return;
|
| 565 |
+
}
|
| 566 |
+
|
| 567 |
+
noHandCount = 0;
|
| 568 |
+
|
| 569 |
+
if (effective === streakLetter) {
|
| 570 |
+
streakCount++;
|
| 571 |
+
} else {
|
| 572 |
+
streakLetter = effective;
|
| 573 |
+
streakCount = 1;
|
| 574 |
+
}
|
| 575 |
+
|
| 576 |
+
const pct = Math.min(100, Math.round((streakCount / STREAK_NEEDED) * 100));
|
| 577 |
+
setStreakVisual(pct);
|
| 578 |
+
streakLabel.textContent = `${Math.min(streakCount, STREAK_NEEDED)} / ${STREAK_NEEDED}`;
|
| 579 |
+
|
| 580 |
+
if (streakCount >= STREAK_NEEDED) {
|
| 581 |
+
captureLetter(streakLetter, confidence);
|
| 582 |
+
}
|
| 583 |
+
}
|
| 584 |
+
|
| 585 |
+
// (handLandmarker and animFrameId declared near initMediaPipe below)
|
| 586 |
+
|
| 587 |
+
// Prevent concurrent classify calls from piling up while a fetch is in flight
|
| 588 |
+
let classifyInFlight = false;
|
| 589 |
+
|
| 590 |
+
async function onResults(results) {
|
| 591 |
+
dctx.save();
|
| 592 |
+
dctx.clearRect(0, 0, displayCanvas.width, displayCanvas.height);
|
| 593 |
+
dctx.drawImage(results.image, 0, 0, displayCanvas.width, displayCanvas.height);
|
| 594 |
+
|
| 595 |
+
// Filter for genuine hands with valid size and geometry
|
| 596 |
+
const validHands = (results.multiHandLandmarks || []).filter(isValidHand);
|
| 597 |
+
|
| 598 |
+
if (validHands.length > 0) {
|
| 599 |
+
noHandOverlay.classList.remove('visible');
|
| 600 |
+
handBadge.className = 'hand-badge live';
|
| 601 |
+
if (currentMode === 'isl') {
|
| 602 |
+
handBadge.textContent = validHands.length > 1 ? '2 HANDS ยท ISL (A-Z)' : '1 HAND ยท ISL (1-9, C, I, L, O, U, V)';
|
| 603 |
+
} else {
|
| 604 |
+
handBadge.textContent = validHands.length > 1 ? '2 HANDS DETECTED' : 'HAND DETECTED';
|
| 605 |
+
}
|
| 606 |
+
validHands.forEach(drawSkeleton);
|
| 607 |
+
|
| 608 |
+
// Skip this frame if the previous classify() hasn't returned yet
|
| 609 |
+
// (avoids a queue of stale requests building up on slow networks)
|
| 610 |
+
if (!classifyInFlight) {
|
| 611 |
+
classifyInFlight = true;
|
| 612 |
+
try {
|
| 613 |
+
const res = await classify(validHands, results.multiHandedness);
|
| 614 |
+
processResult(res.letter, res.confidence);
|
| 615 |
+
} finally {
|
| 616 |
+
classifyInFlight = false;
|
| 617 |
+
}
|
| 618 |
+
}
|
| 619 |
+
|
| 620 |
+
} else {
|
| 621 |
+
noHandOverlay.classList.add('visible');
|
| 622 |
+
handBadge.className = 'hand-badge';
|
| 623 |
+
handBadge.textContent = 'NO HAND';
|
| 624 |
+
detectedLetter.textContent = '';
|
| 625 |
+
detectedConf.textContent = 'Waitingโฆ';
|
| 626 |
+
resetStreak();
|
| 627 |
+
|
| 628 |
+
// Hand was dropped/removed -> immediately grant release permission for next letter
|
| 629 |
+
releaseFrames++;
|
| 630 |
+
if (releaseFrames >= RELEASE_FRAMES_NEEDED) {
|
| 631 |
+
releasedSinceLastCapture = true;
|
| 632 |
+
}
|
| 633 |
+
|
| 634 |
+
if (currentWord) {
|
| 635 |
+
noHandCount++;
|
| 636 |
+
if (noHandCount >= SPACE_FRAMES) {
|
| 637 |
+
pushWord();
|
| 638 |
+
}
|
| 639 |
+
}
|
| 640 |
+
}
|
| 641 |
+
dctx.restore();
|
| 642 |
+
}
|
| 643 |
+
|
| 644 |
+
function drawSkeleton(lm) {
|
| 645 |
+
const w = displayCanvas.width;
|
| 646 |
+
const h = displayCanvas.height;
|
| 647 |
+
|
| 648 |
+
// Draw lines
|
| 649 |
+
CONNECTIONS.forEach(([i, j], idx) => {
|
| 650 |
+
const p1 = lm[i];
|
| 651 |
+
const p2 = lm[j];
|
| 652 |
+
dctx.beginPath();
|
| 653 |
+
dctx.moveTo(p1.x * w, p1.y * h);
|
| 654 |
+
dctx.lineTo(p2.x * w, p2.y * h);
|
| 655 |
+
dctx.strokeStyle = EDGE_COLORS[idx] || '#6a6a76'; // arrays are aligned; neutral if ever not
|
| 656 |
+
dctx.lineWidth = 3;
|
| 657 |
+
dctx.lineCap = 'round';
|
| 658 |
+
dctx.stroke();
|
| 659 |
+
});
|
| 660 |
+
|
| 661 |
+
// Draw points
|
| 662 |
+
lm.forEach((p, i) => {
|
| 663 |
+
const x = p.x * w;
|
| 664 |
+
const y = p.y * h;
|
| 665 |
+
dctx.beginPath();
|
| 666 |
+
dctx.arc(x, y, i === 0 ? 7 : 5, 0, 2 * Math.PI);
|
| 667 |
+
dctx.fillStyle = NODE_COLORS[i] || '#ffffff';
|
| 668 |
+
dctx.fill();
|
| 669 |
+
dctx.strokeStyle = '#ffffff';
|
| 670 |
+
dctx.lineWidth = 1.5;
|
| 671 |
+
dctx.stroke();
|
| 672 |
+
});
|
| 673 |
+
}
|
| 674 |
+
|
| 675 |
+
/**
|
| 676 |
+
* Renders a styled, in-theme camera failure inside the camera panel.
|
| 677 |
+
* Replaces the native alert() MediaPipe would otherwise raise.
|
| 678 |
+
*/
|
| 679 |
+
function showCameraError(err) {
|
| 680 |
+
const name = (err && err.name) ? err.name : 'Error';
|
| 681 |
+
const HINTS = {
|
| 682 |
+
NotReadableError: 'Another application is already using the camera. Close it (video call, another browser tab, your editor\u2019s preview pane) and reload.',
|
| 683 |
+
NotAllowedError: 'Camera permission was denied. Allow camera access for this site, then reload.',
|
| 684 |
+
NotFoundError: 'No camera device was found on this machine.',
|
| 685 |
+
NotSupportedError: 'This browser cannot access a camera on this page. A secure origin (https:// or localhost) is required.',
|
| 686 |
+
OverconstrainedError: 'No camera matches the requested 640\u00d7480 video mode.',
|
| 687 |
+
AbortError: 'The camera was released before it could start. Reload to retry.',
|
| 688 |
+
};
|
| 689 |
+
const msg = HINTS[name] || (err && err.message) || 'The camera could not be started.';
|
| 690 |
+
|
| 691 |
+
noHandOverlay.innerHTML =
|
| 692 |
+
'<div class="cam-error-icon" aria-hidden="true">\u26A0</div>' +
|
| 693 |
+
'<div class="cam-error-title">Camera unavailable</div>' +
|
| 694 |
+
'<div class="cam-error-msg"></div>' +
|
| 695 |
+
'<div class="cam-error-code"></div>';
|
| 696 |
+
// textContent, not innerHTML -- the error string is not ours to trust as markup
|
| 697 |
+
noHandOverlay.querySelector('.cam-error-msg').textContent = msg;
|
| 698 |
+
noHandOverlay.querySelector('.cam-error-code').textContent = name;
|
| 699 |
+
noHandOverlay.classList.add('error');
|
| 700 |
+
noHandOverlay.classList.remove('hidden');
|
| 701 |
+
noHandOverlay.setAttribute('role', 'alert');
|
| 702 |
+
|
| 703 |
+
setStatus('err', 'Camera unavailable');
|
| 704 |
+
detectedLetter.textContent = '';
|
| 705 |
+
detectedConf.textContent = 'No camera';
|
| 706 |
+
console.error('[SignLens] camera init failed:', err);
|
| 707 |
+
}
|
| 708 |
+
|
| 709 |
+
// โโ MediaPipe Tasks Vision Camera Loop โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 710 |
+
let handLandmarker = null;
|
| 711 |
+
let animFrameId = null;
|
| 712 |
+
|
| 713 |
+
/**
|
| 714 |
+
* Converts new Tasks Vision HandLandmarker results into the same shape
|
| 715 |
+
* the rest of app.js expects:
|
| 716 |
+
* multiHandLandmarks: Array of [{x,y,z}, ...21] (same as old API)
|
| 717 |
+
* multiHandedness: Array of {label: 'Right'|'Left'} (same field name)
|
| 718 |
+
*/
|
| 719 |
+
function adaptResults(result) {
|
| 720 |
+
const rawLandmarks = result.landmarks || [];
|
| 721 |
+
const rawHandedness = result.handedness || [];
|
| 722 |
+
|
| 723 |
+
const multiHandLandmarks = [];
|
| 724 |
+
const multiHandedness = [];
|
| 725 |
+
|
| 726 |
+
for (let i = 0; i < rawLandmarks.length; i++) {
|
| 727 |
+
const lms = rawLandmarks[i];
|
| 728 |
+
if (!lms || lms.length < 21) continue;
|
| 729 |
+
|
| 730 |
+
// Anatomical validation:
|
| 731 |
+
// Rejects facial false positives while fully supporting profile/curled hand signs like 'O', 'C', 'E', 'S'
|
| 732 |
+
const palmLen = Math.hypot(lms[9].x - lms[0].x, lms[9].y - lms[0].y);
|
| 733 |
+
const totalSpan = Math.max(...lms.map(p => Math.hypot(p.x - lms[0].x, p.y - lms[0].y)));
|
| 734 |
+
if (palmLen < 0.03 || totalSpan < 0.06) {
|
| 735 |
+
continue;
|
| 736 |
+
}
|
| 737 |
+
|
| 738 |
+
multiHandLandmarks.push(lms);
|
| 739 |
+
const h = rawHandedness[i];
|
| 740 |
+
multiHandedness.push({
|
| 741 |
+
label: (h && h[0] && h[0].categoryName) ? h[0].categoryName : 'Right'
|
| 742 |
+
});
|
| 743 |
+
}
|
| 744 |
+
|
| 745 |
+
return { multiHandLandmarks, multiHandedness };
|
| 746 |
+
}
|
| 747 |
+
|
| 748 |
+
async function initMediaPipe() {
|
| 749 |
+
// Probe getUserMedia first so we can show a styled error instead of a
|
| 750 |
+
// native alert() on failure.
|
| 751 |
+
try {
|
| 752 |
+
const probe = await navigator.mediaDevices.getUserMedia({
|
| 753 |
+
video: { width: 640, height: 480 }
|
| 754 |
+
});
|
| 755 |
+
probe.getTracks().forEach(t => t.stop());
|
| 756 |
+
} catch (err) {
|
| 757 |
+
showCameraError(err);
|
| 758 |
+
return false;
|
| 759 |
+
}
|
| 760 |
+
|
| 761 |
+
// Dynamically import the Tasks Vision ES-module bundle.
|
| 762 |
+
// More reliable than a <script> UMD tag: exports land directly in scope,
|
| 763 |
+
// no window.* namespace hunting required.
|
| 764 |
+
const { FilesetResolver, HandLandmarker } = await import(
|
| 765 |
+
'https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@0.10.14/vision_bundle.mjs'
|
| 766 |
+
);
|
| 767 |
+
|
| 768 |
+
// Resolve WASM files from the same CDN version
|
| 769 |
+
const vision = await FilesetResolver.forVisionTasks(
|
| 770 |
+
'https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@0.10.14/wasm'
|
| 771 |
+
);
|
| 772 |
+
|
| 773 |
+
handLandmarker = await HandLandmarker.createFromOptions(vision, {
|
| 774 |
+
baseOptions: {
|
| 775 |
+
// Same hand_landmarker model as Python's hand_landmarker.task โ identical coordinate space
|
| 776 |
+
modelAssetPath: 'https://storage.googleapis.com/mediapipe-models/hand_landmarker/hand_landmarker/float16/1/hand_landmarker.task',
|
| 777 |
+
delegate: 'GPU',
|
| 778 |
+
},
|
| 779 |
+
runningMode: 'VIDEO',
|
| 780 |
+
numHands: 2,
|
| 781 |
+
minHandDetectionConfidence: 0.25,
|
| 782 |
+
minHandPresenceConfidence: 0.25,
|
| 783 |
+
minTrackingConfidence: 0.25,
|
| 784 |
+
});
|
| 785 |
+
|
| 786 |
+
// Start camera stream
|
| 787 |
+
let stream;
|
| 788 |
+
try {
|
| 789 |
+
stream = await navigator.mediaDevices.getUserMedia({
|
| 790 |
+
video: { width: 640, height: 480, facingMode: 'user' }
|
| 791 |
+
});
|
| 792 |
+
} catch (err) {
|
| 793 |
+
showCameraError(err);
|
| 794 |
+
return false;
|
| 795 |
+
}
|
| 796 |
+
|
| 797 |
+
video.srcObject = stream;
|
| 798 |
+
await new Promise(resolve => { video.onloadedmetadata = resolve; });
|
| 799 |
+
video.play();
|
| 800 |
+
|
| 801 |
+
cameraActive = true;
|
| 802 |
+
setStatus('ready', 'Connected');
|
| 803 |
+
|
| 804 |
+
// Detection loop using requestAnimationFrame for smooth 30fps
|
| 805 |
+
let lastVideoTime = -1;
|
| 806 |
+
function detect() {
|
| 807 |
+
if (!cameraActive) { animFrameId = null; return; }
|
| 808 |
+
animFrameId = requestAnimationFrame(detect);
|
| 809 |
+
|
| 810 |
+
if (video.currentTime === lastVideoTime) return; // no new frame yet
|
| 811 |
+
lastVideoTime = video.currentTime;
|
| 812 |
+
|
| 813 |
+
displayCanvas.width = video.videoWidth || 640;
|
| 814 |
+
displayCanvas.height = video.videoHeight || 480;
|
| 815 |
+
|
| 816 |
+
// detectForVideo uses the video timestamp in ms for temporal smoothing
|
| 817 |
+
const result = handLandmarker.detectForVideo(video, performance.now());
|
| 818 |
+
const adapted = adaptResults(result);
|
| 819 |
+
|
| 820 |
+
// Synthesise a results object matching what onResults() already expects
|
| 821 |
+
onResults({
|
| 822 |
+
image: video,
|
| 823 |
+
multiHandLandmarks: adapted.multiHandLandmarks,
|
| 824 |
+
multiHandedness: adapted.multiHandedness,
|
| 825 |
+
});
|
| 826 |
+
}
|
| 827 |
+
detect();
|
| 828 |
+
return true;
|
| 829 |
+
}
|
| 830 |
+
|
| 831 |
+
// โโ Controls Setup โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 832 |
+
function setupControls() {
|
| 833 |
+
document.getElementById('btn-backspace').addEventListener('click', () => {
|
| 834 |
+
currentWord = currentWord.slice(0, -1);
|
| 835 |
+
letterConfidences.pop();
|
| 836 |
+
renderWord();
|
| 837 |
+
resetStreak();
|
| 838 |
+
});
|
| 839 |
+
|
| 840 |
+
document.getElementById('btn-add-space').addEventListener('click', () => {
|
| 841 |
+
pushWord();
|
| 842 |
+
});
|
| 843 |
+
|
| 844 |
+
document.getElementById('btn-clear-word').addEventListener('click', () => {
|
| 845 |
+
currentWord = '';
|
| 846 |
+
letterConfidences = [];
|
| 847 |
+
renderWord();
|
| 848 |
+
resetStreak();
|
| 849 |
+
});
|
| 850 |
+
|
| 851 |
+
document.getElementById('btn-clear-sentence').addEventListener('click', () => {
|
| 852 |
+
sentenceWords = [];
|
| 853 |
+
currentWord = '';
|
| 854 |
+
letterConfidences = [];
|
| 855 |
+
renderWord();
|
| 856 |
+
renderSentence();
|
| 857 |
+
resetStreak();
|
| 858 |
+
});
|
| 859 |
+
|
| 860 |
+
document.getElementById('btn-pause-detection').addEventListener('click', function() {
|
| 861 |
+
paused = !paused;
|
| 862 |
+
this.textContent = paused ? 'Resume' : 'Pause';
|
| 863 |
+
this.classList.toggle('active-btn', paused);
|
| 864 |
+
});
|
| 865 |
+
|
| 866 |
+
document.getElementById('btn-camera-toggle').addEventListener('click', function() {
|
| 867 |
+
cameraActive = !cameraActive;
|
| 868 |
+
this.textContent = cameraActive ? 'Stop Camera' : 'Start Camera';
|
| 869 |
+
this.classList.toggle('active-btn', !cameraActive);
|
| 870 |
+
if (!cameraActive && animFrameId) {
|
| 871 |
+
cancelAnimationFrame(animFrameId);
|
| 872 |
+
animFrameId = null;
|
| 873 |
+
} else if (cameraActive && !animFrameId && handLandmarker) {
|
| 874 |
+
// restart loop
|
| 875 |
+
let lastVideoTime = -1;
|
| 876 |
+
function detect() {
|
| 877 |
+
if (!cameraActive) { animFrameId = null; return; }
|
| 878 |
+
animFrameId = requestAnimationFrame(detect);
|
| 879 |
+
if (video.currentTime === lastVideoTime) return;
|
| 880 |
+
lastVideoTime = video.currentTime;
|
| 881 |
+
displayCanvas.width = video.videoWidth || 640;
|
| 882 |
+
displayCanvas.height = video.videoHeight || 480;
|
| 883 |
+
const result = handLandmarker.detectForVideo(video, performance.now());
|
| 884 |
+
const adapted = adaptResults(result);
|
| 885 |
+
onResults({ image: video, multiHandLandmarks: adapted.multiHandLandmarks, multiHandedness: adapted.multiHandedness });
|
| 886 |
+
}
|
| 887 |
+
detect();
|
| 888 |
+
}
|
| 889 |
+
});
|
| 890 |
+
|
| 891 |
+
document.getElementById('btn-copy').addEventListener('click', () => {
|
| 892 |
+
if (sentenceWords.length) {
|
| 893 |
+
navigator.clipboard.writeText(sentenceWords.join(' '));
|
| 894 |
+
showToast('๐ Copied to clipboard!');
|
| 895 |
+
}
|
| 896 |
+
});
|
| 897 |
+
|
| 898 |
+
// Global reference to prevent Chrome garbage-collection bug on SpeechSynthesisUtterance
|
| 899 |
+
window._activeUtterance = null;
|
| 900 |
+
|
| 901 |
+
function fallbackServerSpeak(text, btn) {
|
| 902 |
+
fetch('/speak', {
|
| 903 |
+
method: 'POST',
|
| 904 |
+
headers: { 'Content-Type': 'application/json' },
|
| 905 |
+
body: JSON.stringify({ text })
|
| 906 |
+
}).then(res => res.json())
|
| 907 |
+
.then(data => {
|
| 908 |
+
console.log('[SignLens] Server TTS spoke:', data);
|
| 909 |
+
})
|
| 910 |
+
.catch(err => console.error('[SignLens] Server TTS failed:', err))
|
| 911 |
+
.finally(() => {
|
| 912 |
+
if (btn) {
|
| 913 |
+
btn.classList.remove('active');
|
| 914 |
+
btn.textContent = 'Speak';
|
| 915 |
+
}
|
| 916 |
+
});
|
| 917 |
+
}
|
| 918 |
+
|
| 919 |
+
function speakSentence() {
|
| 920 |
+
const text = (sentenceWords.length
|
| 921 |
+
? sentenceWords.join(' ')
|
| 922 |
+
: (sentenceDisplay.innerText || '').replace(/Your sentence will appear here[โฆ\.]*/i, '')
|
| 923 |
+
).trim();
|
| 924 |
+
|
| 925 |
+
if (!text) {
|
| 926 |
+
showToast('โ ๏ธ No sentence to speak yet');
|
| 927 |
+
return;
|
| 928 |
+
}
|
| 929 |
+
|
| 930 |
+
const btnSpeak = document.getElementById('btn-speak');
|
| 931 |
+
if (btnSpeak) {
|
| 932 |
+
btnSpeak.classList.add('active');
|
| 933 |
+
btnSpeak.textContent = 'Speakingโฆ';
|
| 934 |
+
}
|
| 935 |
+
showToast(`๐ Speaking: "${text}"`);
|
| 936 |
+
|
| 937 |
+
if ('speechSynthesis' in window) {
|
| 938 |
+
try {
|
| 939 |
+
window.speechSynthesis.cancel(); // Reset stuck Chromium speech queue
|
| 940 |
+
|
| 941 |
+
const u = new SpeechSynthesisUtterance(text);
|
| 942 |
+
window._activeUtterance = u; // Keep reference to prevent V8 GC drop
|
| 943 |
+
|
| 944 |
+
u.lang = 'en-US';
|
| 945 |
+
u.rate = 0.95;
|
| 946 |
+
u.pitch = 1.0;
|
| 947 |
+
u.volume = 1.0;
|
| 948 |
+
|
| 949 |
+
const voices = window.speechSynthesis.getVoices();
|
| 950 |
+
if (voices && voices.length > 0) {
|
| 951 |
+
const enVoice = voices.find(v => v.lang.startsWith('en') && !v.localService) ||
|
| 952 |
+
voices.find(v => v.lang.startsWith('en')) ||
|
| 953 |
+
voices[0];
|
| 954 |
+
if (enVoice) u.voice = enVoice;
|
| 955 |
+
}
|
| 956 |
+
|
| 957 |
+
u.onend = () => {
|
| 958 |
+
window._activeUtterance = null;
|
| 959 |
+
if (btnSpeak) {
|
| 960 |
+
btnSpeak.classList.remove('active');
|
| 961 |
+
btnSpeak.textContent = 'Speak';
|
| 962 |
+
}
|
| 963 |
+
};
|
| 964 |
+
|
| 965 |
+
u.onerror = (e) => {
|
| 966 |
+
console.warn('[SignLens] Browser speech error, falling back to server TTS:', e);
|
| 967 |
+
window._activeUtterance = null;
|
| 968 |
+
fallbackServerSpeak(text, btnSpeak);
|
| 969 |
+
};
|
| 970 |
+
|
| 971 |
+
window.speechSynthesis.speak(u);
|
| 972 |
+
|
| 973 |
+
if (window.speechSynthesis.paused) {
|
| 974 |
+
window.speechSynthesis.resume();
|
| 975 |
+
}
|
| 976 |
+
|
| 977 |
+
// Safety timeout to reset button state after utterance
|
| 978 |
+
setTimeout(() => {
|
| 979 |
+
if (btnSpeak && btnSpeak.textContent === 'Speakingโฆ') {
|
| 980 |
+
btnSpeak.classList.remove('active');
|
| 981 |
+
btnSpeak.textContent = 'Speak';
|
| 982 |
+
}
|
| 983 |
+
}, Math.max(3000, text.split(' ').length * 1200));
|
| 984 |
+
|
| 985 |
+
} catch (err) {
|
| 986 |
+
console.warn('[SignLens] SpeechSynthesis exception:', err);
|
| 987 |
+
fallbackServerSpeak(text, btnSpeak);
|
| 988 |
+
}
|
| 989 |
+
} else {
|
| 990 |
+
fallbackServerSpeak(text, btnSpeak);
|
| 991 |
+
}
|
| 992 |
+
}
|
| 993 |
+
|
| 994 |
+
document.getElementById('btn-speak').addEventListener('click', speakSentence);
|
| 995 |
+
|
| 996 |
+
// Mode buttons
|
| 997 |
+
const btnAsl = document.getElementById('btn-asl');
|
| 998 |
+
const btnIsl = document.getElementById('btn-isl');
|
| 999 |
+
const btnVisually = document.getElementById('btn-visually');
|
| 1000 |
+
const liveWorkspace = document.getElementById('live-workspace');
|
| 1001 |
+
const visuallyWorkspace = document.getElementById('visually-workspace');
|
| 1002 |
+
|
| 1003 |
+
function isInVisuallyMode() {
|
| 1004 |
+
return visuallyWorkspace && !visuallyWorkspace.classList.contains('hidden');
|
| 1005 |
+
}
|
| 1006 |
+
|
| 1007 |
+
btnAsl.addEventListener('click', () => {
|
| 1008 |
+
currentMode = 'asl';
|
| 1009 |
+
currentWeights = aslWeights;
|
| 1010 |
+
btnAsl.classList.add('active');
|
| 1011 |
+
btnIsl.classList.remove('active');
|
| 1012 |
+
if (btnVisually) btnVisually.classList.remove('active');
|
| 1013 |
+
modeToggle.classList.remove('isl-mode', 'visually-mode');
|
| 1014 |
+
if (liveWorkspace) liveWorkspace.classList.remove('hidden');
|
| 1015 |
+
if (visuallyWorkspace) visuallyWorkspace.classList.add('hidden');
|
| 1016 |
+
resetStreak();
|
| 1017 |
+
showToast('Switched to ASL (Live Camera)');
|
| 1018 |
+
});
|
| 1019 |
+
|
| 1020 |
+
btnIsl.addEventListener('click', () => {
|
| 1021 |
+
currentMode = 'isl';
|
| 1022 |
+
currentWeights = islWeights;
|
| 1023 |
+
btnIsl.classList.add('active');
|
| 1024 |
+
btnAsl.classList.remove('active');
|
| 1025 |
+
if (btnVisually) btnVisually.classList.remove('active');
|
| 1026 |
+
modeToggle.classList.remove('visually-mode');
|
| 1027 |
+
modeToggle.classList.add('isl-mode');
|
| 1028 |
+
if (liveWorkspace) liveWorkspace.classList.remove('hidden');
|
| 1029 |
+
if (visuallyWorkspace) visuallyWorkspace.classList.add('hidden');
|
| 1030 |
+
resetStreak();
|
| 1031 |
+
showToast('Switched to ISL (Live Camera)');
|
| 1032 |
+
});
|
| 1033 |
+
|
| 1034 |
+
if (btnVisually) {
|
| 1035 |
+
btnVisually.addEventListener('click', () => {
|
| 1036 |
+
btnVisually.classList.add('active');
|
| 1037 |
+
btnAsl.classList.remove('active');
|
| 1038 |
+
btnIsl.classList.remove('active');
|
| 1039 |
+
modeToggle.classList.remove('isl-mode');
|
| 1040 |
+
modeToggle.classList.add('visually-mode');
|
| 1041 |
+
if (liveWorkspace) liveWorkspace.classList.add('hidden');
|
| 1042 |
+
if (visuallyWorkspace) visuallyWorkspace.classList.remove('hidden');
|
| 1043 |
+
activatePracticeStudio();
|
| 1044 |
+
showToast('Practice Studio Active');
|
| 1045 |
+
});
|
| 1046 |
+
}
|
| 1047 |
+
|
| 1048 |
+
// Settings sheet. Both thresholds are read live by processResult(),
|
| 1049 |
+
// so no restart is needed after a change.
|
| 1050 |
+
const setConf = document.getElementById('set-confidence');
|
| 1051 |
+
const outConf = document.getElementById('out-confidence');
|
| 1052 |
+
const setStreak = document.getElementById('set-streak');
|
| 1053 |
+
const outStreak = document.getElementById('out-streak');
|
| 1054 |
+
const setTimeoutInput = document.getElementById('set-timeout');
|
| 1055 |
+
const outTimeout = document.getElementById('out-timeout');
|
| 1056 |
+
|
| 1057 |
+
setConf.value = Math.round(MIN_CONFIDENCE * 100);
|
| 1058 |
+
outConf.value = `${setConf.value}%`;
|
| 1059 |
+
setStreak.value = STREAK_NEEDED;
|
| 1060 |
+
outStreak.value = STREAK_NEEDED;
|
| 1061 |
+
if (setTimeoutInput && outTimeout) {
|
| 1062 |
+
setTimeoutInput.value = (SPACE_FRAMES / 30).toFixed(1);
|
| 1063 |
+
outTimeout.value = `${setTimeoutInput.value}s`;
|
| 1064 |
+
}
|
| 1065 |
+
|
| 1066 |
+
setConf.addEventListener('input', () => {
|
| 1067 |
+
MIN_CONFIDENCE = Number(setConf.value) / 100;
|
| 1068 |
+
outConf.value = `${setConf.value}%`;
|
| 1069 |
+
});
|
| 1070 |
+
|
| 1071 |
+
setStreak.addEventListener('input', () => {
|
| 1072 |
+
STREAK_NEEDED = Number(setStreak.value);
|
| 1073 |
+
outStreak.value = STREAK_NEEDED;
|
| 1074 |
+
resetStreak(); // rescale the dial against the new target
|
| 1075 |
+
});
|
| 1076 |
+
|
| 1077 |
+
if (setTimeoutInput && outTimeout) {
|
| 1078 |
+
setTimeoutInput.addEventListener('input', () => {
|
| 1079 |
+
const sec = Number(setTimeoutInput.value);
|
| 1080 |
+
SPACE_FRAMES = Math.round(sec * 30);
|
| 1081 |
+
outTimeout.value = `${sec.toFixed(1)}s`;
|
| 1082 |
+
});
|
| 1083 |
+
}
|
| 1084 |
+
|
| 1085 |
+
// Theme Toggle (Light / Dark Mode)
|
| 1086 |
+
const btnThemeToggle = document.getElementById('btn-theme-toggle');
|
| 1087 |
+
const iconThemeToggle = document.getElementById('theme-toggle-icon');
|
| 1088 |
+
|
| 1089 |
+
function applyTheme(theme) {
|
| 1090 |
+
const isLight = theme === 'light';
|
| 1091 |
+
document.documentElement.setAttribute('data-theme', isLight ? 'light' : 'dark');
|
| 1092 |
+
localStorage.setItem('signlens_theme', isLight ? 'light' : 'dark');
|
| 1093 |
+
if (iconThemeToggle) {
|
| 1094 |
+
iconThemeToggle.textContent = isLight ? '๐' : 'โ๏ธ';
|
| 1095 |
+
}
|
| 1096 |
+
if (btnThemeToggle) {
|
| 1097 |
+
btnThemeToggle.setAttribute('title', isLight ? 'Switch to Dark Mode' : 'Switch to Light Mode');
|
| 1098 |
+
btnThemeToggle.setAttribute('aria-label', isLight ? 'Switch to Dark Mode' : 'Switch to Light Mode');
|
| 1099 |
+
}
|
| 1100 |
+
// Re-render avatar if in practice mode so canvas background updates immediately
|
| 1101 |
+
if (typeof drawAvatar === 'function') {
|
| 1102 |
+
try { drawAvatar(); } catch (_) {}
|
| 1103 |
+
}
|
| 1104 |
+
}
|
| 1105 |
+
|
| 1106 |
+
// Load saved preference or fallback to dark
|
| 1107 |
+
const savedTheme = localStorage.getItem('signlens_theme') || 'dark';
|
| 1108 |
+
applyTheme(savedTheme);
|
| 1109 |
+
|
| 1110 |
+
if (btnThemeToggle) {
|
| 1111 |
+
btnThemeToggle.addEventListener('click', () => {
|
| 1112 |
+
const currentTheme = document.documentElement.getAttribute('data-theme') || 'dark';
|
| 1113 |
+
const nextTheme = currentTheme === 'light' ? 'dark' : 'light';
|
| 1114 |
+
applyTheme(nextTheme);
|
| 1115 |
+
showToast(nextTheme === 'light' ? 'โ๏ธ Switched to Light Mode' : '๐ Switched to Dark Mode');
|
| 1116 |
+
});
|
| 1117 |
+
}
|
| 1118 |
+
|
| 1119 |
+
// Keyboard shortcuts
|
| 1120 |
+
document.addEventListener('keydown', (e) => {
|
| 1121 |
+
// Ignore shortcuts while a control has focus, or a sheet is open.
|
| 1122 |
+
if (e.target.closest('input, textarea, [popover]')) return;
|
| 1123 |
+
|
| 1124 |
+
if (e.key === ' ' || e.code === 'Space') {
|
| 1125 |
+
e.preventDefault();
|
| 1126 |
+
pushWord();
|
| 1127 |
+
} else if (e.key === 'Backspace') {
|
| 1128 |
+
currentWord = currentWord.slice(0, -1);
|
| 1129 |
+
letterConfidences.pop();
|
| 1130 |
+
renderWord();
|
| 1131 |
+
} else if (e.key === '1') {
|
| 1132 |
+
btnAsl.click();
|
| 1133 |
+
} else if (e.key === '2') {
|
| 1134 |
+
btnIsl.click();
|
| 1135 |
+
} else if (e.key === '3') {
|
| 1136 |
+
if (btnVisually) btnVisually.click();
|
| 1137 |
+
}
|
| 1138 |
+
});
|
| 1139 |
+
}
|
| 1140 |
+
|
| 1141 |
+
// โโ Sign Reference Guide Modal โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 1142 |
+
// โโ Sign Reference Guide Modal โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 1143 |
+
function initSignGuide() {
|
| 1144 |
+
const panelGuide = document.getElementById('panel-guide');
|
| 1145 |
+
const btnOpenGuide = document.getElementById('btn-open-guide');
|
| 1146 |
+
const btnCloseGuide = document.getElementById('btn-close-guide');
|
| 1147 |
+
const tabIsl = document.getElementById('tab-guide-isl');
|
| 1148 |
+
const tabAsl = document.getElementById('tab-guide-asl');
|
| 1149 |
+
const contentIsl = document.getElementById('guide-content-isl');
|
| 1150 |
+
const contentAsl = document.getElementById('guide-content-asl');
|
| 1151 |
+
const singleGrid = document.getElementById('isl-single-cards');
|
| 1152 |
+
const dualGrid = document.getElementById('isl-dual-cards');
|
| 1153 |
+
|
| 1154 |
+
const islDescFallback = {
|
| 1155 |
+
'1': 'Index finger extended upwards',
|
| 1156 |
+
'2': 'Index and middle fingers extended in V-shape',
|
| 1157 |
+
'3': 'Thumb, index, and middle fingers extended',
|
| 1158 |
+
'4': 'Four fingers extended upwards',
|
| 1159 |
+
'5': 'Open hand with all five fingers spread',
|
| 1160 |
+
'6': 'Thumb and little finger extended',
|
| 1161 |
+
'7': 'Thumb, index, and little finger extended',
|
| 1162 |
+
'8': 'Middle, ring, and little fingers extended',
|
| 1163 |
+
'9': 'Thumb touching index finger in circle',
|
| 1164 |
+
'C': 'Curved hand forming C-shape',
|
| 1165 |
+
'I': 'Little finger extended upwards',
|
| 1166 |
+
'L': 'Thumb and index finger forming L-shape',
|
| 1167 |
+
'O': 'All fingers curved touching thumb in O-shape',
|
| 1168 |
+
'U': 'Index and middle fingers held together vertically',
|
| 1169 |
+
'V': 'Index and middle fingers in open V-shape',
|
| 1170 |
+
'A': 'Dominant index finger touching non-dominant thumb tip',
|
| 1171 |
+
'B': 'Both hands forming double circles / glasses pose',
|
| 1172 |
+
'D': 'Dominant index pointing to non-dominant index forming D',
|
| 1173 |
+
'E': 'Left hand curved C/horizontal, right index touching middle knuckle',
|
| 1174 |
+
'F': 'Dominant index and middle crossing non-dominant index and middle',
|
| 1175 |
+
'G': 'Both fists held together vertically with knuckles facing forward',
|
| 1176 |
+
'H': 'Dominant open palm sweeping flat across non-dominant palm',
|
| 1177 |
+
'J': 'Index finger drawing J stroke across non-dominant palm',
|
| 1178 |
+
'K': 'Dominant index finger hooking over non-dominant index finger',
|
| 1179 |
+
'M': 'Dominant three fingers laid across non-dominant palm',
|
| 1180 |
+
'N': 'Dominant two fingers laid across non-dominant palm',
|
| 1181 |
+
'P': 'Dominant index finger forming circle with thumb touching non-dominant tip',
|
| 1182 |
+
'Q': 'Dominant hand forming circle loop with non-dominant index through',
|
| 1183 |
+
'R': 'Dominant index finger hooked on non-dominant palm',
|
| 1184 |
+
'S': 'Dominant pinky hooking non-dominant pinky / fist lock',
|
| 1185 |
+
'T': 'Dominant index finger placed perpendicularly on non-dominant index side',
|
| 1186 |
+
'W': 'Interlocking open fingers of both hands upright',
|
| 1187 |
+
'X': 'Crossing both index fingers in an X',
|
| 1188 |
+
'Y': 'Dominant index pointing between non-dominant thumb and index',
|
| 1189 |
+
'Z': 'Dominant open palm held upright against non-dominant palm'
|
| 1190 |
+
};
|
| 1191 |
+
|
| 1192 |
+
function createGuideCard(letter, isTwoHanded, desc) {
|
| 1193 |
+
const el = document.createElement('div');
|
| 1194 |
+
el.className = 'guide-card';
|
| 1195 |
+
el.title = `${letter}: ${desc} (click to enlarge)`;
|
| 1196 |
+
el.dataset.letter = letter;
|
| 1197 |
+
el.dataset.hands = isTwoHanded ? '2' : '1';
|
| 1198 |
+
el.dataset.desc = desc;
|
| 1199 |
+
el.dataset.lang = 'ISL';
|
| 1200 |
+
el.dataset.imgSrc = `isl_signs/${letter}.png`;
|
| 1201 |
+
el.innerHTML = `
|
| 1202 |
+
<div class="guide-card-img-wrap">
|
| 1203 |
+
<img src="isl_signs/${letter}.png" alt="Sign ${letter}" loading="lazy" />
|
| 1204 |
+
</div>
|
| 1205 |
+
<div class="guide-card-body">
|
| 1206 |
+
<span class="guide-card-char">${letter}</span>
|
| 1207 |
+
<span class="guide-card-hands ${isTwoHanded ? 'pill-blue' : 'pill-amber'} pill-badge">${isTwoHanded ? '2 Hands' : '1 Hand'}</span>
|
| 1208 |
+
</div>
|
| 1209 |
+
<p class="guide-card-desc" style="padding: 6px 10px 10px; font-size: 0.72rem; color: var(--text-dim); line-height: 1.35; margin: 0;">${desc}</p>
|
| 1210 |
+
`;
|
| 1211 |
+
el.addEventListener('click', () => openSignLightbox(el));
|
| 1212 |
+
return el;
|
| 1213 |
+
}
|
| 1214 |
+
|
| 1215 |
+
function renderGuideCards() {
|
| 1216 |
+
if (!singleGrid || !dualGrid) return;
|
| 1217 |
+
singleGrid.innerHTML = '';
|
| 1218 |
+
dualGrid.innerHTML = '';
|
| 1219 |
+
|
| 1220 |
+
const isl = (practiceTemplates && practiceTemplates.isl) ? practiceTemplates.isl : {};
|
| 1221 |
+
const singleKeys = ['1', '2', '3', '4', '5', '6', '7', '8', '9', 'C', 'I', 'L', 'O', 'U', 'V'];
|
| 1222 |
+
const dualKeys = ['A', 'B', 'D', 'E', 'F', 'G', 'H', 'J', 'K', 'M', 'N', 'P', 'Q', 'R', 'S', 'T', 'W', 'X', 'Y', 'Z'];
|
| 1223 |
+
|
| 1224 |
+
singleKeys.forEach(k => {
|
| 1225 |
+
const desc = (isl[k] && isl[k].desc) ? isl[k].desc : (islDescFallback[k] || 'Single hand sign');
|
| 1226 |
+
singleGrid.appendChild(createGuideCard(k, false, desc));
|
| 1227 |
+
});
|
| 1228 |
+
|
| 1229 |
+
dualKeys.forEach(k => {
|
| 1230 |
+
const desc = (isl[k] && isl[k].desc) ? isl[k].desc : (islDescFallback[k] || 'Dual hand sign');
|
| 1231 |
+
dualGrid.appendChild(createGuideCard(k, true, desc));
|
| 1232 |
+
});
|
| 1233 |
+
}
|
| 1234 |
+
|
| 1235 |
+
const aslDescFallback = {
|
| 1236 |
+
'A': 'Fist with thumb on side', 'B': 'All fingers together pointing up',
|
| 1237 |
+
'C': 'Curved hand in C-shape', 'D': 'Index up, fingers curve to thumb',
|
| 1238 |
+
'E': 'Fingers bent, thumb tucked', 'F': 'Index-thumb circle, three fingers up',
|
| 1239 |
+
'G': 'Index and thumb point sideways', 'H': 'Two fingers point sideways',
|
| 1240 |
+
'I': 'Pinky finger up', 'J': 'Pinky draws J in air',
|
| 1241 |
+
'K': 'Index up, middle angled, thumb out', 'L': 'L-shape โ thumb and index',
|
| 1242 |
+
'M': 'Three fingers over thumb', 'N': 'Two fingers over thumb',
|
| 1243 |
+
'O': 'All fingers curve to form O', 'P': 'K shape pointing down',
|
| 1244 |
+
'Q': 'G shape pointing down', 'R': 'Two fingers crossed',
|
| 1245 |
+
'S': 'Fist with thumb over fingers', 'T': 'Thumb between index and middle',
|
| 1246 |
+
'U': 'Two fingers together pointing up', 'V': 'Two fingers spread in V',
|
| 1247 |
+
'W': 'Three fingers spread out', 'X': 'Index finger hooks',
|
| 1248 |
+
'Y': 'Thumb and pinky out', 'Z': 'Index draws Z in air'
|
| 1249 |
+
};
|
| 1250 |
+
|
| 1251 |
+
function renderAslCards() {
|
| 1252 |
+
const aslGrid = document.getElementById('asl-cards');
|
| 1253 |
+
if (!aslGrid || aslGrid.children.length > 0) return; // render once
|
| 1254 |
+
aslGrid.innerHTML = '';
|
| 1255 |
+
const aslLetters = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'.split('');
|
| 1256 |
+
aslLetters.forEach(letter => {
|
| 1257 |
+
const desc = aslDescFallback[letter] || 'ASL hand sign';
|
| 1258 |
+
const el = document.createElement('div');
|
| 1259 |
+
el.className = 'guide-card';
|
| 1260 |
+
el.title = `${letter}: ${desc} (click to enlarge)`;
|
| 1261 |
+
el.dataset.letter = letter;
|
| 1262 |
+
el.dataset.hands = '1';
|
| 1263 |
+
el.dataset.desc = desc;
|
| 1264 |
+
el.dataset.lang = 'ASL';
|
| 1265 |
+
el.dataset.imgSrc = `asl_signs/${letter}.png`;
|
| 1266 |
+
el.innerHTML = `
|
| 1267 |
+
<div class="guide-card-img-wrap">
|
| 1268 |
+
<img src="asl_signs/${letter}.png" alt="ASL Sign ${letter}" loading="lazy" />
|
| 1269 |
+
</div>
|
| 1270 |
+
<div class="guide-card-body">
|
| 1271 |
+
<span class="guide-card-char">${letter}</span>
|
| 1272 |
+
<span class="guide-card-hands pill-amber pill-badge">1 Hand</span>
|
| 1273 |
+
</div>
|
| 1274 |
+
<p class="guide-card-desc" style="padding: 6px 10px 10px; font-size: 0.72rem; color: var(--text-dim); line-height: 1.35; margin: 0;">${desc}</p>
|
| 1275 |
+
`;
|
| 1276 |
+
el.addEventListener('click', () => openSignLightbox(el));
|
| 1277 |
+
aslGrid.appendChild(el);
|
| 1278 |
+
});
|
| 1279 |
+
}
|
| 1280 |
+
|
| 1281 |
+
function openGuide() {
|
| 1282 |
+
renderGuideCards();
|
| 1283 |
+
renderAslCards();
|
| 1284 |
+
if (panelGuide) {
|
| 1285 |
+
try {
|
| 1286 |
+
if (panelGuide.showPopover) {
|
| 1287 |
+
panelGuide.showPopover();
|
| 1288 |
+
} else {
|
| 1289 |
+
panelGuide.style.display = 'flex';
|
| 1290 |
+
panelGuide.classList.add('open');
|
| 1291 |
+
}
|
| 1292 |
+
} catch (err) {
|
| 1293 |
+
panelGuide.style.display = 'flex';
|
| 1294 |
+
panelGuide.classList.add('open');
|
| 1295 |
+
}
|
| 1296 |
+
}
|
| 1297 |
+
}
|
| 1298 |
+
|
| 1299 |
+
function closeGuide() {
|
| 1300 |
+
if (panelGuide) {
|
| 1301 |
+
try {
|
| 1302 |
+
if (panelGuide.hidePopover) {
|
| 1303 |
+
panelGuide.hidePopover();
|
| 1304 |
+
} else {
|
| 1305 |
+
panelGuide.style.display = 'none';
|
| 1306 |
+
panelGuide.classList.remove('open');
|
| 1307 |
+
}
|
| 1308 |
+
} catch (err) {
|
| 1309 |
+
panelGuide.style.display = 'none';
|
| 1310 |
+
panelGuide.classList.remove('open');
|
| 1311 |
+
}
|
| 1312 |
+
}
|
| 1313 |
+
}
|
| 1314 |
+
|
| 1315 |
+
if (panelGuide) {
|
| 1316 |
+
// Synchronize inline display style with popover state
|
| 1317 |
+
panelGuide.addEventListener('toggle', (e) => {
|
| 1318 |
+
if (e.newState === 'open') {
|
| 1319 |
+
panelGuide.style.display = 'flex';
|
| 1320 |
+
panelGuide.classList.add('open');
|
| 1321 |
+
} else {
|
| 1322 |
+
panelGuide.style.display = 'none';
|
| 1323 |
+
panelGuide.classList.remove('open');
|
| 1324 |
+
}
|
| 1325 |
+
});
|
| 1326 |
+
// Ensure it starts closed and strictly hidden
|
| 1327 |
+
panelGuide.style.display = 'none';
|
| 1328 |
+
panelGuide.classList.remove('open');
|
| 1329 |
+
}
|
| 1330 |
+
|
| 1331 |
+
const stageGuideBtn = document.getElementById('stage-guide-btn');
|
| 1332 |
+
if (stageGuideBtn) {
|
| 1333 |
+
stageGuideBtn.addEventListener('click', (e) => {
|
| 1334 |
+
e.preventDefault();
|
| 1335 |
+
openGuide();
|
| 1336 |
+
});
|
| 1337 |
+
}
|
| 1338 |
+
|
| 1339 |
+
if (btnOpenGuide) {
|
| 1340 |
+
btnOpenGuide.addEventListener('click', (e) => {
|
| 1341 |
+
e.preventDefault();
|
| 1342 |
+
openGuide();
|
| 1343 |
+
});
|
| 1344 |
+
}
|
| 1345 |
+
|
| 1346 |
+
if (btnCloseGuide) {
|
| 1347 |
+
btnCloseGuide.addEventListener('click', (e) => {
|
| 1348 |
+
e.preventDefault();
|
| 1349 |
+
closeGuide();
|
| 1350 |
+
});
|
| 1351 |
+
}
|
| 1352 |
+
|
| 1353 |
+
if (tabIsl && tabAsl && contentIsl && contentAsl) {
|
| 1354 |
+
tabIsl.addEventListener('click', () => {
|
| 1355 |
+
tabIsl.classList.add('active');
|
| 1356 |
+
tabAsl.classList.remove('active');
|
| 1357 |
+
tabIsl.setAttribute('aria-selected', 'true');
|
| 1358 |
+
tabAsl.setAttribute('aria-selected', 'false');
|
| 1359 |
+
contentIsl.classList.remove('hidden');
|
| 1360 |
+
contentAsl.classList.add('hidden');
|
| 1361 |
+
});
|
| 1362 |
+
|
| 1363 |
+
tabAsl.addEventListener('click', () => {
|
| 1364 |
+
tabAsl.classList.add('active');
|
| 1365 |
+
tabIsl.classList.remove('active');
|
| 1366 |
+
tabAsl.setAttribute('aria-selected', 'true');
|
| 1367 |
+
tabIsl.setAttribute('aria-selected', 'false');
|
| 1368 |
+
contentAsl.classList.remove('hidden');
|
| 1369 |
+
contentIsl.classList.add('hidden');
|
| 1370 |
+
});
|
| 1371 |
+
}
|
| 1372 |
+
|
| 1373 |
+
// โโ Sign Lightbox popup โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 1374 |
+
const lightboxOverlay = document.getElementById('sign-lightbox-overlay');
|
| 1375 |
+
const lightboxImg = document.getElementById('sign-lightbox-img');
|
| 1376 |
+
const lightboxLetter = document.getElementById('sign-lightbox-letter');
|
| 1377 |
+
const lightboxType = document.getElementById('sign-lightbox-type');
|
| 1378 |
+
const lightboxDesc = document.getElementById('sign-lightbox-desc');
|
| 1379 |
+
const lightboxLang = document.getElementById('sign-lightbox-lang');
|
| 1380 |
+
const lightboxCloseBtn = document.getElementById('sign-lightbox-close');
|
| 1381 |
+
|
| 1382 |
+
function openSignLightbox(dataOrEl) {
|
| 1383 |
+
if (!lightboxOverlay) return;
|
| 1384 |
+
|
| 1385 |
+
let letter = '?';
|
| 1386 |
+
let hands = '1';
|
| 1387 |
+
let desc = '';
|
| 1388 |
+
let lang = 'ISL';
|
| 1389 |
+
let imgSrc = '';
|
| 1390 |
+
|
| 1391 |
+
if (dataOrEl instanceof HTMLElement) {
|
| 1392 |
+
letter = dataOrEl.dataset.letter || '?';
|
| 1393 |
+
hands = dataOrEl.dataset.hands || '1';
|
| 1394 |
+
desc = dataOrEl.dataset.desc || '';
|
| 1395 |
+
lang = dataOrEl.dataset.lang || 'ISL';
|
| 1396 |
+
imgSrc = dataOrEl.dataset.imgSrc || '';
|
| 1397 |
+
} else if (typeof dataOrEl === 'object' && dataOrEl !== null) {
|
| 1398 |
+
letter = dataOrEl.letter || '?';
|
| 1399 |
+
hands = String(dataOrEl.hands || '1');
|
| 1400 |
+
desc = dataOrEl.desc || '';
|
| 1401 |
+
lang = dataOrEl.lang || 'ISL';
|
| 1402 |
+
imgSrc = dataOrEl.imgSrc || '';
|
| 1403 |
+
}
|
| 1404 |
+
|
| 1405 |
+
if (lightboxImg) lightboxImg.src = imgSrc;
|
| 1406 |
+
if (lightboxLetter) lightboxLetter.textContent = `Sign: ${letter}`;
|
| 1407 |
+
if (lightboxType) {
|
| 1408 |
+
const isTwoHand = hands === '2';
|
| 1409 |
+
lightboxType.textContent = isTwoHand ? 'โ๏ธ 2 Hands Required' : '๐ค 1 Hand Only';
|
| 1410 |
+
lightboxType.className = `sign-lightbox-type ${isTwoHand ? 'two-hand' : 'one-hand'}`;
|
| 1411 |
+
}
|
| 1412 |
+
if (lightboxDesc) lightboxDesc.textContent = desc;
|
| 1413 |
+
if (lightboxLang) lightboxLang.textContent = (lang.toUpperCase() === 'ASL') ? '๐บ๐ธ American Sign Language (ASL)' : '๐ฎ๐ณ Indian Sign Language (ISL)';
|
| 1414 |
+
|
| 1415 |
+
// Open natively in the browser Top Layer (above all popovers and dialogs)
|
| 1416 |
+
if (typeof lightboxOverlay.showModal === 'function') {
|
| 1417 |
+
if (!lightboxOverlay.open) {
|
| 1418 |
+
lightboxOverlay.showModal();
|
| 1419 |
+
}
|
| 1420 |
+
} else {
|
| 1421 |
+
lightboxOverlay.setAttribute('open', '');
|
| 1422 |
+
}
|
| 1423 |
+
}
|
| 1424 |
+
|
| 1425 |
+
function closeSignLightbox() {
|
| 1426 |
+
if (!lightboxOverlay) return;
|
| 1427 |
+
if (typeof lightboxOverlay.close === 'function') {
|
| 1428 |
+
if (lightboxOverlay.open) {
|
| 1429 |
+
lightboxOverlay.close();
|
| 1430 |
+
}
|
| 1431 |
+
} else {
|
| 1432 |
+
lightboxOverlay.removeAttribute('open');
|
| 1433 |
+
}
|
| 1434 |
+
}
|
| 1435 |
+
|
| 1436 |
+
// Make openSignLightbox available globally for card click handlers
|
| 1437 |
+
window.openSignLightbox = openSignLightbox;
|
| 1438 |
+
|
| 1439 |
+
// Close button
|
| 1440 |
+
if (lightboxCloseBtn) {
|
| 1441 |
+
lightboxCloseBtn.addEventListener('click', (e) => {
|
| 1442 |
+
e.preventDefault();
|
| 1443 |
+
e.stopPropagation();
|
| 1444 |
+
closeSignLightbox();
|
| 1445 |
+
});
|
| 1446 |
+
}
|
| 1447 |
+
|
| 1448 |
+
// Click outside (on backdrop) to close dialog
|
| 1449 |
+
if (lightboxOverlay) {
|
| 1450 |
+
lightboxOverlay.addEventListener('click', (e) => {
|
| 1451 |
+
if (e.target === lightboxOverlay) {
|
| 1452 |
+
closeSignLightbox();
|
| 1453 |
+
}
|
| 1454 |
+
});
|
| 1455 |
+
}
|
| 1456 |
+
|
| 1457 |
+
// ESC key natively closes <dialog>, but add safety handler
|
| 1458 |
+
document.addEventListener('keydown', (e) => {
|
| 1459 |
+
if (e.key === 'Escape' && lightboxOverlay && lightboxOverlay.open) {
|
| 1460 |
+
closeSignLightbox();
|
| 1461 |
+
}
|
| 1462 |
+
});
|
| 1463 |
+
|
| 1464 |
+
window._renderGuideCards = renderGuideCards;
|
| 1465 |
+
window._openSignGuide = openGuide;
|
| 1466 |
+
}
|
| 1467 |
+
|
| 1468 |
+
// โโ Startup โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 1469 |
+
window.addEventListener('DOMContentLoaded', async () => {
|
| 1470 |
+
displayCanvas.width = 640;
|
| 1471 |
+
displayCanvas.height = 480;
|
| 1472 |
+
|
| 1473 |
+
// Wire controls FIRST: a camera failure must not leave every button dead.
|
| 1474 |
+
setupControls();
|
| 1475 |
+
initPracticeStudio();
|
| 1476 |
+
initSignGuide();
|
| 1477 |
+
|
| 1478 |
+
setStatus('loading', 'Loadingโฆ');
|
| 1479 |
+
try {
|
| 1480 |
+
// 1. Probe for Flask backend (async, result used by classify())
|
| 1481 |
+
probeServer();
|
| 1482 |
+
|
| 1483 |
+
// 2. Always load JS MLP weights for offline fallback
|
| 1484 |
+
await loadLandmarkModel();
|
| 1485 |
+
|
| 1486 |
+
// 3. Start camera
|
| 1487 |
+
await initMediaPipe();
|
| 1488 |
+
} catch (err) {
|
| 1489 |
+
showCameraError(err);
|
| 1490 |
+
}
|
| 1491 |
+
});
|
| 1492 |
+
|
| 1493 |
+
// โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 1494 |
+
// VISUALLY (PRACTICE STUDIO) IMPLEMENTATION
|
| 1495 |
+
// โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 1496 |
+
|
| 1497 |
+
let practiceTemplates = { asl: {}, isl: {} };
|
| 1498 |
+
let practiceDialect = 'asl'; // 'asl' | 'isl'
|
| 1499 |
+
let practiceSpeed = 1.0; // 0.5 | 1.0 | 1.5
|
| 1500 |
+
|
| 1501 |
+
// Sequence and animation state
|
| 1502 |
+
let practiceSequence = []; // e.g. ['H', 'I', ' ', 'B', 'A', 'B', 'Y']
|
| 1503 |
+
let practiceCurrentIndex = 0;
|
| 1504 |
+
let practiceIsPlaying = false;
|
| 1505 |
+
let practiceStepStartTime = 0;
|
| 1506 |
+
let practiceAnimFrameId = null;
|
| 1507 |
+
|
| 1508 |
+
// Poses for interpolation: arrays of 21 landmarks [x, y, z]
|
| 1509 |
+
let practiceFromPose = null;
|
| 1510 |
+
let practiceToPose = null;
|
| 1511 |
+
let practiceCurrentPose = null;
|
| 1512 |
+
|
| 1513 |
+
// Default neutral resting pose (21 landmarks)
|
| 1514 |
+
const NEUTRAL_REST_POSE = Array(21).fill(0).map((_, i) => {
|
| 1515 |
+
if (i === 0) return [0, 0, 0];
|
| 1516 |
+
const fingerIdx = Math.floor((i - 1) / 4);
|
| 1517 |
+
const jointIdx = ((i - 1) % 4) + 1;
|
| 1518 |
+
const xOffset = (fingerIdx - 2) * 0.18;
|
| 1519 |
+
const yOffset = -0.35 * jointIdx;
|
| 1520 |
+
return [xOffset, yOffset, 0];
|
| 1521 |
+
});
|
| 1522 |
+
|
| 1523 |
+
function clone21(pts) {
|
| 1524 |
+
if (!pts || !Array.isArray(pts)) return null;
|
| 1525 |
+
return pts.map(pt => [pt[0], pt[1], pt[2] || 0]);
|
| 1526 |
+
}
|
| 1527 |
+
|
| 1528 |
+
function normalizePose(raw) {
|
| 1529 |
+
if (!raw) {
|
| 1530 |
+
return {
|
| 1531 |
+
twoHanded: false,
|
| 1532 |
+
left: null,
|
| 1533 |
+
right: clone21(NEUTRAL_REST_POSE),
|
| 1534 |
+
wristL: [0.0, 0.0],
|
| 1535 |
+
wristR: [0.0, 0.0],
|
| 1536 |
+
desc: ''
|
| 1537 |
+
};
|
| 1538 |
+
}
|
| 1539 |
+
if (Array.isArray(raw)) {
|
| 1540 |
+
return {
|
| 1541 |
+
twoHanded: false,
|
| 1542 |
+
left: null,
|
| 1543 |
+
right: clone21(raw),
|
| 1544 |
+
wristL: [0.0, 0.0],
|
| 1545 |
+
wristR: [0.0, 0.0],
|
| 1546 |
+
desc: ''
|
| 1547 |
+
};
|
| 1548 |
+
}
|
| 1549 |
+
return {
|
| 1550 |
+
twoHanded: !!raw.twoHanded,
|
| 1551 |
+
left: raw.left ? clone21(raw.left) : null,
|
| 1552 |
+
right: raw.right ? clone21(raw.right) : clone21(NEUTRAL_REST_POSE),
|
| 1553 |
+
wristL: raw.wristL ? [raw.wristL[0], raw.wristL[1]] : [0.0, 0.0],
|
| 1554 |
+
wristR: raw.wristR ? [raw.wristR[0], raw.wristR[1]] : [0.0, 0.0],
|
| 1555 |
+
desc: raw.desc || ''
|
| 1556 |
+
};
|
| 1557 |
+
}
|
| 1558 |
+
|
| 1559 |
+
function clonePose(pose) {
|
| 1560 |
+
const norm = normalizePose(pose);
|
| 1561 |
+
return {
|
| 1562 |
+
twoHanded: norm.twoHanded,
|
| 1563 |
+
left: clone21(norm.left),
|
| 1564 |
+
right: clone21(norm.right),
|
| 1565 |
+
wristL: [...norm.wristL],
|
| 1566 |
+
wristR: [...norm.wristR],
|
| 1567 |
+
desc: norm.desc
|
| 1568 |
+
};
|
| 1569 |
+
}
|
| 1570 |
+
|
| 1571 |
+
function initPracticeStudio() {
|
| 1572 |
+
const canvas = document.getElementById('avatar-canvas');
|
| 1573 |
+
if (!canvas) return;
|
| 1574 |
+
const ctx = canvas.getContext('2d');
|
| 1575 |
+
|
| 1576 |
+
// 1. Fetch templates
|
| 1577 |
+
fetch('/sign_templates.json?v=' + Date.now(), { cache: 'no-store' })
|
| 1578 |
+
.then(res => {
|
| 1579 |
+
if (!res.ok) throw new Error('HTTP ' + res.status);
|
| 1580 |
+
return res.json();
|
| 1581 |
+
})
|
| 1582 |
+
.then(data => {
|
| 1583 |
+
practiceTemplates = data;
|
| 1584 |
+
console.log('[PracticeStudio] Loaded sign templates:', Object.keys(data.asl || {}).length, 'ASL,', Object.keys(data.isl || {}).length, 'ISL');
|
| 1585 |
+
// Set initial pose
|
| 1586 |
+
practiceCurrentPose = clonePose(getTemplatePose('REST'));
|
| 1587 |
+
drawAvatar();
|
| 1588 |
+
const panelGuide = document.getElementById('panel-guide');
|
| 1589 |
+
if (panelGuide && (panelGuide.matches(':popover-open') || panelGuide.classList.contains('open'))) {
|
| 1590 |
+
if (window._renderGuideCards) window._renderGuideCards();
|
| 1591 |
+
}
|
| 1592 |
+
})
|
| 1593 |
+
.catch(err => {
|
| 1594 |
+
console.warn('[PracticeStudio] Failed to load sign_templates.json:', err);
|
| 1595 |
+
});
|
| 1596 |
+
|
| 1597 |
+
// 2. DOM elements
|
| 1598 |
+
const txtInput = document.getElementById('practice-input');
|
| 1599 |
+
const charCount = document.getElementById('practice-char-count');
|
| 1600 |
+
const btnMic = document.getElementById('btn-mic');
|
| 1601 |
+
const micLabel = document.getElementById('mic-label');
|
| 1602 |
+
const btnSpeakInput = document.getElementById('btn-speak-input');
|
| 1603 |
+
const btnClearPractice = document.getElementById('btn-clear-practice');
|
| 1604 |
+
const btnStartSign = document.getElementById('btn-start-sign');
|
| 1605 |
+
const sequenceChipsContainer = document.getElementById('sequence-chips');
|
| 1606 |
+
const seqCounter = document.getElementById('seq-counter');
|
| 1607 |
+
|
| 1608 |
+
const btnPracAsl = document.getElementById('btn-practice-asl');
|
| 1609 |
+
const btnPracIsl = document.getElementById('btn-practice-isl');
|
| 1610 |
+
|
| 1611 |
+
const hudChar = document.getElementById('hud-letter-char');
|
| 1612 |
+
const hudSub = document.getElementById('hud-letter-sub');
|
| 1613 |
+
const hudCue = document.getElementById('hud-letter-cue');
|
| 1614 |
+
const hudStatus = document.getElementById('avatar-hud-status');
|
| 1615 |
+
const avatarStatusText = document.getElementById('avatar-status-text');
|
| 1616 |
+
|
| 1617 |
+
// Top-Right Sign Gesture Image Preview Reference
|
| 1618 |
+
const signPreviewWrap = document.getElementById('avatar-sign-preview');
|
| 1619 |
+
const signPreviewImg = document.getElementById('avatar-sign-img');
|
| 1620 |
+
const signPreviewBadge = document.getElementById('avatar-sign-badge');
|
| 1621 |
+
const signPreviewChar = document.getElementById('avatar-sign-char');
|
| 1622 |
+
const signPreviewHands = document.getElementById('avatar-sign-hands');
|
| 1623 |
+
const signPreviewDesc = document.getElementById('avatar-sign-desc');
|
| 1624 |
+
|
| 1625 |
+
if (signPreviewWrap) {
|
| 1626 |
+
signPreviewWrap.addEventListener('click', () => {
|
| 1627 |
+
if (signPreviewImg && signPreviewImg.dataset.letter) {
|
| 1628 |
+
if (typeof openSignLightbox === 'function') {
|
| 1629 |
+
openSignLightbox({
|
| 1630 |
+
letter: signPreviewImg.dataset.letter,
|
| 1631 |
+
hands: signPreviewImg.dataset.hands || '1',
|
| 1632 |
+
desc: signPreviewImg.dataset.desc || '',
|
| 1633 |
+
lang: signPreviewImg.dataset.lang || 'ISL',
|
| 1634 |
+
imgSrc: signPreviewImg.src
|
| 1635 |
+
});
|
| 1636 |
+
}
|
| 1637 |
+
}
|
| 1638 |
+
});
|
| 1639 |
+
}
|
| 1640 |
+
|
| 1641 |
+
const btnPrev = document.getElementById('btn-avatar-prev');
|
| 1642 |
+
const btnPlayPause = document.getElementById('btn-avatar-playpause');
|
| 1643 |
+
const btnNext = document.getElementById('btn-avatar-next');
|
| 1644 |
+
const btnReplay = document.getElementById('btn-avatar-replay');
|
| 1645 |
+
const scrubber = document.getElementById('avatar-scrubber');
|
| 1646 |
+
|
| 1647 |
+
// Resize canvas handler
|
| 1648 |
+
function resizeCanvas() {
|
| 1649 |
+
if (!canvas.parentElement) return;
|
| 1650 |
+
const rect = canvas.parentElement.getBoundingClientRect();
|
| 1651 |
+
const dpr = window.devicePixelRatio || 1;
|
| 1652 |
+
const w = Math.floor(rect.width);
|
| 1653 |
+
const h = Math.floor(rect.height);
|
| 1654 |
+
if (w > 0 && h > 0 && (canvas.width !== w * dpr || canvas.height !== h * dpr)) {
|
| 1655 |
+
canvas.width = w * dpr;
|
| 1656 |
+
canvas.height = h * dpr;
|
| 1657 |
+
drawAvatar();
|
| 1658 |
+
}
|
| 1659 |
+
}
|
| 1660 |
+
window.addEventListener('resize', resizeCanvas);
|
| 1661 |
+
setTimeout(resizeCanvas, 100);
|
| 1662 |
+
|
| 1663 |
+
// Get template for a character
|
| 1664 |
+
function getTemplatePose(char) {
|
| 1665 |
+
const dict = practiceTemplates[practiceDialect] || practiceTemplates.asl || {};
|
| 1666 |
+
const upper = String(char).toUpperCase();
|
| 1667 |
+
if (dict[upper]) return normalizePose(dict[upper]);
|
| 1668 |
+
if (dict['REST']) return normalizePose(dict['REST']);
|
| 1669 |
+
return normalizePose(null);
|
| 1670 |
+
}
|
| 1671 |
+
|
| 1672 |
+
const ASL_SAMPLES = ['HI', 'HELLO', 'THANK YOU', 'HOW ARE YOU', 'BABY'];
|
| 1673 |
+
const ISL_SAMPLES = ['NAMASTE', 'A B C', '1 2 3', 'ONAM', 'INDIA'];
|
| 1674 |
+
|
| 1675 |
+
function updateSampleChips(samples) {
|
| 1676 |
+
const chipsContainer = document.querySelector('.quick-samples');
|
| 1677 |
+
if (!chipsContainer) return;
|
| 1678 |
+
const chips = chipsContainer.querySelectorAll('.sample-chip');
|
| 1679 |
+
chips.forEach((chip, idx) => {
|
| 1680 |
+
if (samples[idx] !== undefined) {
|
| 1681 |
+
chip.dataset.text = samples[idx];
|
| 1682 |
+
chip.textContent = samples[idx];
|
| 1683 |
+
chip.style.display = '';
|
| 1684 |
+
} else {
|
| 1685 |
+
chip.style.display = 'none';
|
| 1686 |
+
}
|
| 1687 |
+
});
|
| 1688 |
+
}
|
| 1689 |
+
|
| 1690 |
+
// Dialect switch inside Practice
|
| 1691 |
+
if (btnPracAsl && btnPracIsl) {
|
| 1692 |
+
btnPracAsl.addEventListener('click', () => {
|
| 1693 |
+
practiceDialect = 'asl';
|
| 1694 |
+
btnPracAsl.classList.add('active');
|
| 1695 |
+
btnPracIsl.classList.remove('active');
|
| 1696 |
+
updateSampleChips(ASL_SAMPLES);
|
| 1697 |
+
rebuildSequence(false);
|
| 1698 |
+
showToast('Practice dialect: ASL');
|
| 1699 |
+
});
|
| 1700 |
+
btnPracIsl.addEventListener('click', () => {
|
| 1701 |
+
practiceDialect = 'isl';
|
| 1702 |
+
btnPracIsl.classList.add('active');
|
| 1703 |
+
btnPracAsl.classList.remove('active');
|
| 1704 |
+
updateSampleChips(ISL_SAMPLES);
|
| 1705 |
+
rebuildSequence(false);
|
| 1706 |
+
showToast('Practice dialect: ISL');
|
| 1707 |
+
});
|
| 1708 |
+
}
|
| 1709 |
+
|
| 1710 |
+
// Speed controls
|
| 1711 |
+
document.querySelectorAll('.speed-btn').forEach(btn => {
|
| 1712 |
+
btn.addEventListener('click', () => {
|
| 1713 |
+
document.querySelectorAll('.speed-btn').forEach(b => b.classList.remove('active'));
|
| 1714 |
+
btn.classList.add('active');
|
| 1715 |
+
practiceSpeed = parseFloat(btn.dataset.speed) || 1.0;
|
| 1716 |
+
});
|
| 1717 |
+
});
|
| 1718 |
+
|
| 1719 |
+
// Quick sample chips
|
| 1720 |
+
document.querySelectorAll('.sample-chip').forEach(chip => {
|
| 1721 |
+
chip.addEventListener('click', () => {
|
| 1722 |
+
const text = chip.dataset.text || '';
|
| 1723 |
+
if (txtInput) {
|
| 1724 |
+
txtInput.value = text;
|
| 1725 |
+
updateCharCount();
|
| 1726 |
+
}
|
| 1727 |
+
rebuildSequence(true);
|
| 1728 |
+
});
|
| 1729 |
+
});
|
| 1730 |
+
|
| 1731 |
+
function updateCharCount() {
|
| 1732 |
+
if (!txtInput || !charCount) return;
|
| 1733 |
+
charCount.textContent = `${txtInput.value.length}/200`;
|
| 1734 |
+
}
|
| 1735 |
+
|
| 1736 |
+
if (txtInput) {
|
| 1737 |
+
txtInput.addEventListener('input', () => {
|
| 1738 |
+
updateCharCount();
|
| 1739 |
+
rebuildSequence(false);
|
| 1740 |
+
});
|
| 1741 |
+
txtInput.addEventListener('keydown', (e) => {
|
| 1742 |
+
if (e.key === 'Enter' && !e.shiftKey) {
|
| 1743 |
+
e.preventDefault();
|
| 1744 |
+
rebuildSequence(true);
|
| 1745 |
+
}
|
| 1746 |
+
});
|
| 1747 |
+
}
|
| 1748 |
+
|
| 1749 |
+
if (btnClearPractice) {
|
| 1750 |
+
btnClearPractice.addEventListener('click', () => {
|
| 1751 |
+
if (txtInput) txtInput.value = '';
|
| 1752 |
+
updateCharCount();
|
| 1753 |
+
practiceSequence = [];
|
| 1754 |
+
practiceCurrentIndex = 0;
|
| 1755 |
+
pausePlayback();
|
| 1756 |
+
renderSequenceChips();
|
| 1757 |
+
updateHUD();
|
| 1758 |
+
practiceCurrentPose = clonePose(getTemplatePose('REST'));
|
| 1759 |
+
drawAvatar();
|
| 1760 |
+
});
|
| 1761 |
+
}
|
| 1762 |
+
|
| 1763 |
+
// Voice Speech-to-Text
|
| 1764 |
+
if (btnMic) {
|
| 1765 |
+
let recognition = null;
|
| 1766 |
+
const SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition;
|
| 1767 |
+
|
| 1768 |
+
if (SpeechRecognition) {
|
| 1769 |
+
recognition = new SpeechRecognition();
|
| 1770 |
+
recognition.continuous = false;
|
| 1771 |
+
recognition.interimResults = false;
|
| 1772 |
+
recognition.lang = 'en-US';
|
| 1773 |
+
|
| 1774 |
+
recognition.onstart = () => {
|
| 1775 |
+
btnMic.classList.add('listening');
|
| 1776 |
+
if (micLabel) micLabel.textContent = 'Listeningโฆ';
|
| 1777 |
+
showToast('Speak now โ listeningโฆ');
|
| 1778 |
+
};
|
| 1779 |
+
|
| 1780 |
+
recognition.onresult = (event) => {
|
| 1781 |
+
const transcript = event.results[0][0].transcript;
|
| 1782 |
+
if (txtInput) {
|
| 1783 |
+
txtInput.value = transcript.toUpperCase();
|
| 1784 |
+
updateCharCount();
|
| 1785 |
+
rebuildSequence(true);
|
| 1786 |
+
}
|
| 1787 |
+
};
|
| 1788 |
+
|
| 1789 |
+
recognition.onerror = (event) => {
|
| 1790 |
+
console.warn('[PracticeStudio] Speech recognition error:', event.error);
|
| 1791 |
+
btnMic.classList.remove('listening');
|
| 1792 |
+
if (micLabel) micLabel.textContent = 'Speak';
|
| 1793 |
+
showToast('Voice error: ' + event.error);
|
| 1794 |
+
};
|
| 1795 |
+
|
| 1796 |
+
recognition.onend = () => {
|
| 1797 |
+
btnMic.classList.remove('listening');
|
| 1798 |
+
if (micLabel) micLabel.textContent = 'Speak';
|
| 1799 |
+
};
|
| 1800 |
+
|
| 1801 |
+
btnMic.addEventListener('click', () => {
|
| 1802 |
+
try {
|
| 1803 |
+
if (btnMic.classList.contains('listening')) {
|
| 1804 |
+
recognition.stop();
|
| 1805 |
+
} else {
|
| 1806 |
+
recognition.start();
|
| 1807 |
+
}
|
| 1808 |
+
} catch (err) {
|
| 1809 |
+
console.warn('[PracticeStudio] Speech start exception:', err);
|
| 1810 |
+
}
|
| 1811 |
+
});
|
| 1812 |
+
} else {
|
| 1813 |
+
btnMic.addEventListener('click', () => {
|
| 1814 |
+
showToast('Speech recognition not supported in this browser. Please type.');
|
| 1815 |
+
});
|
| 1816 |
+
}
|
| 1817 |
+
}
|
| 1818 |
+
|
| 1819 |
+
// TTS speak aloud
|
| 1820 |
+
if (btnSpeakInput) {
|
| 1821 |
+
btnSpeakInput.addEventListener('click', () => {
|
| 1822 |
+
const text = (txtInput ? txtInput.value.trim() : '');
|
| 1823 |
+
if (!text) {
|
| 1824 |
+
showToast('Type some words to speak aloud.');
|
| 1825 |
+
return;
|
| 1826 |
+
}
|
| 1827 |
+
if ('speechSynthesis' in window) {
|
| 1828 |
+
window.speechSynthesis.cancel();
|
| 1829 |
+
const utter = new SpeechSynthesisUtterance(text);
|
| 1830 |
+
utter.rate = 0.95;
|
| 1831 |
+
window.speechSynthesis.speak(utter);
|
| 1832 |
+
}
|
| 1833 |
+
});
|
| 1834 |
+
}
|
| 1835 |
+
|
| 1836 |
+
// Start Demonstration Button
|
| 1837 |
+
if (btnStartSign) {
|
| 1838 |
+
btnStartSign.addEventListener('click', () => {
|
| 1839 |
+
rebuildSequence(true);
|
| 1840 |
+
});
|
| 1841 |
+
}
|
| 1842 |
+
|
| 1843 |
+
// Parse text into sequence of signs
|
| 1844 |
+
function rebuildSequence(autoPlay = false) {
|
| 1845 |
+
const raw = (txtInput ? txtInput.value : '').toUpperCase();
|
| 1846 |
+
const cleanChars = [];
|
| 1847 |
+
for (let i = 0; i < raw.length; i++) {
|
| 1848 |
+
const ch = raw[i];
|
| 1849 |
+
if ((ch >= 'A' && ch <= 'Z') || (ch >= '0' && ch <= '9')) {
|
| 1850 |
+
cleanChars.push(ch);
|
| 1851 |
+
} else if (ch === ' ' && cleanChars.length > 0 && cleanChars[cleanChars.length - 1] !== ' ') {
|
| 1852 |
+
cleanChars.push(' ');
|
| 1853 |
+
}
|
| 1854 |
+
}
|
| 1855 |
+
// Remove trailing space
|
| 1856 |
+
if (cleanChars.length > 0 && cleanChars[cleanChars.length - 1] === ' ') {
|
| 1857 |
+
cleanChars.pop();
|
| 1858 |
+
}
|
| 1859 |
+
|
| 1860 |
+
practiceSequence = cleanChars;
|
| 1861 |
+
practiceCurrentIndex = 0;
|
| 1862 |
+
renderSequenceChips();
|
| 1863 |
+
updateHUD();
|
| 1864 |
+
|
| 1865 |
+
if (practiceSequence.length === 0) {
|
| 1866 |
+
practiceCurrentPose = clonePose(getTemplatePose('REST'));
|
| 1867 |
+
drawAvatar();
|
| 1868 |
+
return;
|
| 1869 |
+
}
|
| 1870 |
+
|
| 1871 |
+
if (autoPlay) {
|
| 1872 |
+
startPlayback();
|
| 1873 |
+
} else {
|
| 1874 |
+
jumpToStep(0);
|
| 1875 |
+
}
|
| 1876 |
+
}
|
| 1877 |
+
|
| 1878 |
+
function renderSequenceChips() {
|
| 1879 |
+
if (!sequenceChipsContainer) return;
|
| 1880 |
+
sequenceChipsContainer.innerHTML = '';
|
| 1881 |
+
|
| 1882 |
+
if (practiceSequence.length === 0) {
|
| 1883 |
+
sequenceChipsContainer.innerHTML = '<span class="ph">Enter text above to preview gesture timelineโฆ</span>';
|
| 1884 |
+
if (seqCounter) seqCounter.textContent = '0 / 0';
|
| 1885 |
+
if (scrubber) scrubber.value = 0;
|
| 1886 |
+
return;
|
| 1887 |
+
}
|
| 1888 |
+
|
| 1889 |
+
if (seqCounter) {
|
| 1890 |
+
seqCounter.textContent = `${practiceCurrentIndex + 1} / ${practiceSequence.length}`;
|
| 1891 |
+
}
|
| 1892 |
+
|
| 1893 |
+
practiceSequence.forEach((char, idx) => {
|
| 1894 |
+
const chip = document.createElement('button');
|
| 1895 |
+
chip.className = 'sequence-chip' + (char === ' ' ? ' is-space' : '') + (idx === practiceCurrentIndex ? ' active' : '');
|
| 1896 |
+
chip.textContent = (char === ' ' ? 'โฃ' : char);
|
| 1897 |
+
chip.title = (char === ' ' ? 'Space (Rest pause)' : `Letter ${char}`);
|
| 1898 |
+
chip.addEventListener('click', () => {
|
| 1899 |
+
jumpToStep(idx);
|
| 1900 |
+
});
|
| 1901 |
+
sequenceChipsContainer.appendChild(chip);
|
| 1902 |
+
});
|
| 1903 |
+
|
| 1904 |
+
if (scrubber) {
|
| 1905 |
+
const pct = (practiceSequence.length > 1) ? (practiceCurrentIndex / (practiceSequence.length - 1)) * 100 : 0;
|
| 1906 |
+
scrubber.value = pct;
|
| 1907 |
+
}
|
| 1908 |
+
}
|
| 1909 |
+
|
| 1910 |
+
function updateHUD() {
|
| 1911 |
+
if (practiceSequence.length === 0) {
|
| 1912 |
+
if (hudChar) hudChar.textContent = 'โ';
|
| 1913 |
+
if (hudSub) hudSub.textContent = 'Waiting for input';
|
| 1914 |
+
if (hudCue) { hudCue.style.display = 'none'; hudCue.textContent = ''; }
|
| 1915 |
+
if (avatarStatusText) avatarStatusText.textContent = 'Ready';
|
| 1916 |
+
if (hudStatus) hudStatus.classList.remove('playing');
|
| 1917 |
+
if (signPreviewWrap) signPreviewWrap.style.display = 'none';
|
| 1918 |
+
return;
|
| 1919 |
+
}
|
| 1920 |
+
|
| 1921 |
+
const currChar = practiceSequence[practiceCurrentIndex];
|
| 1922 |
+
if (hudChar) {
|
| 1923 |
+
hudChar.textContent = (currChar === ' ' ? 'โฃ' : currChar);
|
| 1924 |
+
}
|
| 1925 |
+
|
| 1926 |
+
const currPose = getTemplatePose(currChar === ' ' ? 'REST' : currChar);
|
| 1927 |
+
const is2H = currPose.twoHanded;
|
| 1928 |
+
const desc = currPose.desc;
|
| 1929 |
+
|
| 1930 |
+
if (hudSub) {
|
| 1931 |
+
if (currChar === ' ') {
|
| 1932 |
+
hudSub.textContent = `Word Boundary Pause (${practiceCurrentIndex + 1}/${practiceSequence.length})`;
|
| 1933 |
+
} else {
|
| 1934 |
+
const handTag = (practiceDialect === 'isl') ? (is2H ? ' โข Dual Hand (Two-Handed)' : ' โข Single Hand') : ' โข Single Hand';
|
| 1935 |
+
hudSub.textContent = `${practiceDialect.toUpperCase()} Sign โข Step ${practiceCurrentIndex + 1} of ${practiceSequence.length}${handTag}`;
|
| 1936 |
+
}
|
| 1937 |
+
}
|
| 1938 |
+
|
| 1939 |
+
if (hudCue) {
|
| 1940 |
+
if (desc && currChar !== ' ') {
|
| 1941 |
+
hudCue.style.display = 'inline-flex';
|
| 1942 |
+
hudCue.textContent = `${is2H ? '๐ Two-Handed: ' : 'โ '}${desc}`;
|
| 1943 |
+
} else {
|
| 1944 |
+
hudCue.style.display = 'none';
|
| 1945 |
+
hudCue.textContent = '';
|
| 1946 |
+
}
|
| 1947 |
+
}
|
| 1948 |
+
|
| 1949 |
+
if (avatarStatusText) {
|
| 1950 |
+
avatarStatusText.textContent = practiceIsPlaying ? 'Demonstratingโฆ' : 'Paused';
|
| 1951 |
+
}
|
| 1952 |
+
if (hudStatus) {
|
| 1953 |
+
if (practiceIsPlaying) hudStatus.classList.add('playing');
|
| 1954 |
+
else hudStatus.classList.remove('playing');
|
| 1955 |
+
}
|
| 1956 |
+
|
| 1957 |
+
// Update Top-Right Sign Gesture Image Preview Reference
|
| 1958 |
+
if (signPreviewWrap) {
|
| 1959 |
+
if (currChar && currChar !== ' ') {
|
| 1960 |
+
const lang = (practiceDialect || 'isl').toLowerCase();
|
| 1961 |
+
const imgSrc = (lang === 'asl') ? `asl_signs/${currChar}.png` : `isl_signs/${currChar}.png`;
|
| 1962 |
+
if (signPreviewImg) {
|
| 1963 |
+
signPreviewImg.src = imgSrc;
|
| 1964 |
+
signPreviewImg.dataset.letter = currChar;
|
| 1965 |
+
signPreviewImg.dataset.hands = is2H ? '2' : '1';
|
| 1966 |
+
signPreviewImg.dataset.desc = desc || '';
|
| 1967 |
+
signPreviewImg.dataset.lang = lang.toUpperCase();
|
| 1968 |
+
}
|
| 1969 |
+
if (signPreviewBadge) signPreviewBadge.textContent = lang.toUpperCase();
|
| 1970 |
+
if (signPreviewChar) signPreviewChar.textContent = `Sign: ${currChar}`;
|
| 1971 |
+
if (signPreviewHands) signPreviewHands.textContent = is2H ? '2 Hands' : '1 Hand';
|
| 1972 |
+
if (signPreviewDesc) signPreviewDesc.textContent = desc || (is2H ? 'Dual-hand gesture' : 'Single-hand gesture');
|
| 1973 |
+
signPreviewWrap.style.display = 'flex';
|
| 1974 |
+
} else {
|
| 1975 |
+
signPreviewWrap.style.display = 'none';
|
| 1976 |
+
}
|
| 1977 |
+
}
|
| 1978 |
+
}
|
| 1979 |
+
|
| 1980 |
+
function jumpToStep(idx) {
|
| 1981 |
+
if (practiceSequence.length === 0) return;
|
| 1982 |
+
practiceCurrentIndex = Math.max(0, Math.min(idx, practiceSequence.length - 1));
|
| 1983 |
+
const char = practiceSequence[practiceCurrentIndex];
|
| 1984 |
+
practiceFromPose = practiceCurrentPose ? clonePose(practiceCurrentPose) : clonePose(getTemplatePose('REST'));
|
| 1985 |
+
practiceToPose = clonePose(getTemplatePose(char === ' ' ? 'REST' : char));
|
| 1986 |
+
practiceStepStartTime = performance.now();
|
| 1987 |
+
renderSequenceChips();
|
| 1988 |
+
updateHUD();
|
| 1989 |
+
|
| 1990 |
+
// Scroll active chip into view
|
| 1991 |
+
const activeChip = sequenceChipsContainer ? sequenceChipsContainer.children[practiceCurrentIndex] : null;
|
| 1992 |
+
if (activeChip && activeChip.scrollIntoView) {
|
| 1993 |
+
activeChip.scrollIntoView({ behavior: 'smooth', block: 'nearest', inline: 'center' });
|
| 1994 |
+
}
|
| 1995 |
+
}
|
| 1996 |
+
|
| 1997 |
+
function startPlayback() {
|
| 1998 |
+
if (practiceSequence.length === 0) return;
|
| 1999 |
+
practiceIsPlaying = true;
|
| 2000 |
+
if (btnPlayPause) btnPlayPause.textContent = 'โธ';
|
| 2001 |
+
jumpToStep(practiceCurrentIndex);
|
| 2002 |
+
updateHUD();
|
| 2003 |
+
}
|
| 2004 |
+
|
| 2005 |
+
function pausePlayback() {
|
| 2006 |
+
practiceIsPlaying = false;
|
| 2007 |
+
if (btnPlayPause) btnPlayPause.textContent = 'โถ';
|
| 2008 |
+
updateHUD();
|
| 2009 |
+
}
|
| 2010 |
+
|
| 2011 |
+
// Playback control buttons
|
| 2012 |
+
if (btnPlayPause) {
|
| 2013 |
+
btnPlayPause.addEventListener('click', () => {
|
| 2014 |
+
if (practiceIsPlaying) {
|
| 2015 |
+
pausePlayback();
|
| 2016 |
+
} else {
|
| 2017 |
+
if (practiceCurrentIndex >= practiceSequence.length - 1) {
|
| 2018 |
+
practiceCurrentIndex = 0;
|
| 2019 |
+
}
|
| 2020 |
+
startPlayback();
|
| 2021 |
+
}
|
| 2022 |
+
});
|
| 2023 |
+
}
|
| 2024 |
+
|
| 2025 |
+
if (btnPrev) {
|
| 2026 |
+
btnPrev.addEventListener('click', () => {
|
| 2027 |
+
pausePlayback();
|
| 2028 |
+
jumpToStep(practiceCurrentIndex - 1);
|
| 2029 |
+
});
|
| 2030 |
+
}
|
| 2031 |
+
|
| 2032 |
+
if (btnNext) {
|
| 2033 |
+
btnNext.addEventListener('click', () => {
|
| 2034 |
+
pausePlayback();
|
| 2035 |
+
jumpToStep(practiceCurrentIndex + 1);
|
| 2036 |
+
});
|
| 2037 |
+
}
|
| 2038 |
+
|
| 2039 |
+
if (btnReplay) {
|
| 2040 |
+
btnReplay.addEventListener('click', () => {
|
| 2041 |
+
practiceCurrentIndex = 0;
|
| 2042 |
+
startPlayback();
|
| 2043 |
+
});
|
| 2044 |
+
}
|
| 2045 |
+
|
| 2046 |
+
if (scrubber) {
|
| 2047 |
+
scrubber.addEventListener('input', () => {
|
| 2048 |
+
if (practiceSequence.length === 0) return;
|
| 2049 |
+
pausePlayback();
|
| 2050 |
+
const val = parseFloat(scrubber.value);
|
| 2051 |
+
const targetIdx = Math.round((val / 100) * (practiceSequence.length - 1));
|
| 2052 |
+
jumpToStep(targetIdx);
|
| 2053 |
+
});
|
| 2054 |
+
}
|
| 2055 |
+
|
| 2056 |
+
// Interpolation & Render Loop
|
| 2057 |
+
function lerp(a, b, t) {
|
| 2058 |
+
return a + (b - a) * t;
|
| 2059 |
+
}
|
| 2060 |
+
|
| 2061 |
+
function easeInOutCubic(x) {
|
| 2062 |
+
return x < 0.5 ? 4 * x * x * x : 1 - Math.pow(-2 * x + 2, 3) / 2;
|
| 2063 |
+
}
|
| 2064 |
+
|
| 2065 |
+
function avatarAnimationLoop(now) {
|
| 2066 |
+
practiceAnimFrameId = requestAnimationFrame(avatarAnimationLoop);
|
| 2067 |
+
|
| 2068 |
+
// Calculate step timing based on speed
|
| 2069 |
+
const transitionMs = 450 / practiceSpeed;
|
| 2070 |
+
const holdMs = 700 / practiceSpeed;
|
| 2071 |
+
const totalStepMs = transitionMs + holdMs;
|
| 2072 |
+
|
| 2073 |
+
if (practiceToPose && practiceFromPose) {
|
| 2074 |
+
const elapsed = now - practiceStepStartTime;
|
| 2075 |
+
const progress = Math.min(1.0, elapsed / transitionMs);
|
| 2076 |
+
const t = easeInOutCubic(progress);
|
| 2077 |
+
|
| 2078 |
+
if (!practiceCurrentPose) {
|
| 2079 |
+
practiceCurrentPose = clonePose(practiceFromPose);
|
| 2080 |
+
}
|
| 2081 |
+
|
| 2082 |
+
// Interpolate right hand
|
| 2083 |
+
const fromR = practiceFromPose.right || NEUTRAL_REST_POSE;
|
| 2084 |
+
const toR = practiceToPose.right || NEUTRAL_REST_POSE;
|
| 2085 |
+
if (!practiceCurrentPose.right) practiceCurrentPose.right = clone21(fromR);
|
| 2086 |
+
for (let i = 0; i < 21; i++) {
|
| 2087 |
+
practiceCurrentPose.right[i][0] = lerp(fromR[i][0], toR[i][0], t);
|
| 2088 |
+
practiceCurrentPose.right[i][1] = lerp(fromR[i][1], toR[i][1], t);
|
| 2089 |
+
practiceCurrentPose.right[i][2] = lerp(fromR[i][2], toR[i][2], t);
|
| 2090 |
+
}
|
| 2091 |
+
|
| 2092 |
+
// Interpolate left hand
|
| 2093 |
+
const fromL = practiceFromPose.left;
|
| 2094 |
+
const toL = practiceToPose.left;
|
| 2095 |
+
if (fromL || toL) {
|
| 2096 |
+
const effFromL = fromL || NEUTRAL_REST_POSE;
|
| 2097 |
+
const effToL = toL || NEUTRAL_REST_POSE;
|
| 2098 |
+
if (!practiceCurrentPose.left) practiceCurrentPose.left = clone21(effFromL);
|
| 2099 |
+
for (let i = 0; i < 21; i++) {
|
| 2100 |
+
practiceCurrentPose.left[i][0] = lerp(effFromL[i][0], effToL[i][0], t);
|
| 2101 |
+
practiceCurrentPose.left[i][1] = lerp(effFromL[i][1], effToL[i][1], t);
|
| 2102 |
+
practiceCurrentPose.left[i][2] = lerp(effFromL[i][2], effToL[i][2], t);
|
| 2103 |
+
}
|
| 2104 |
+
practiceCurrentPose.twoHanded = (t > 0.5) ? practiceToPose.twoHanded : practiceFromPose.twoHanded;
|
| 2105 |
+
} else {
|
| 2106 |
+
practiceCurrentPose.left = null;
|
| 2107 |
+
practiceCurrentPose.twoHanded = false;
|
| 2108 |
+
}
|
| 2109 |
+
|
| 2110 |
+
// Interpolate wrist offsets
|
| 2111 |
+
practiceCurrentPose.wristL = [
|
| 2112 |
+
lerp(practiceFromPose.wristL ? practiceFromPose.wristL[0] : 0, practiceToPose.wristL ? practiceToPose.wristL[0] : 0, t),
|
| 2113 |
+
lerp(practiceFromPose.wristL ? practiceFromPose.wristL[1] : 0, practiceToPose.wristL ? practiceToPose.wristL[1] : 0, t)
|
| 2114 |
+
];
|
| 2115 |
+
practiceCurrentPose.wristR = [
|
| 2116 |
+
lerp(practiceFromPose.wristR ? practiceFromPose.wristR[0] : 0, practiceToPose.wristR ? practiceToPose.wristR[0] : 0, t),
|
| 2117 |
+
lerp(practiceFromPose.wristR ? practiceFromPose.wristR[1] : 0, practiceToPose.wristR ? practiceToPose.wristR[1] : 0, t)
|
| 2118 |
+
];
|
| 2119 |
+
practiceCurrentPose.desc = practiceToPose.desc;
|
| 2120 |
+
|
| 2121 |
+
if (practiceIsPlaying && elapsed >= totalStepMs) {
|
| 2122 |
+
if (practiceCurrentIndex < practiceSequence.length - 1) {
|
| 2123 |
+
jumpToStep(practiceCurrentIndex + 1);
|
| 2124 |
+
} else {
|
| 2125 |
+
// Completed full sequence
|
| 2126 |
+
pausePlayback();
|
| 2127 |
+
if (avatarStatusText) avatarStatusText.textContent = 'Complete';
|
| 2128 |
+
}
|
| 2129 |
+
}
|
| 2130 |
+
}
|
| 2131 |
+
|
| 2132 |
+
drawAvatar();
|
| 2133 |
+
}
|
| 2134 |
+
|
| 2135 |
+
// Draw 21-landmark hand skeleton with 5 distinct finger colors, palm metacarpals, and pearl beads
|
| 2136 |
+
function drawHandSkeleton(ctx, pose21, wristX, wristY, dpr, scale, isLeft) {
|
| 2137 |
+
if (!pose21 || pose21.length !== 21) return;
|
| 2138 |
+
|
| 2139 |
+
// Project landmarks to screen coordinates
|
| 2140 |
+
const pts = pose21.map(pt => {
|
| 2141 |
+
const lx = pt[0];
|
| 2142 |
+
const ly = pt[1];
|
| 2143 |
+
const lz = pt[2] || 0;
|
| 2144 |
+
const depth = 1.0 / (1.0 - lz * 0.18);
|
| 2145 |
+
return [
|
| 2146 |
+
wristX + lx * scale * depth,
|
| 2147 |
+
wristY + ly * scale * depth,
|
| 2148 |
+
lz
|
| 2149 |
+
];
|
| 2150 |
+
});
|
| 2151 |
+
|
| 2152 |
+
// Palm Metacarpal Structure (cool titanium slate connecting wrist to the 4 knuckles)
|
| 2153 |
+
const palmCol = 'rgba(145, 160, 175, 0.85)';
|
| 2154 |
+
ctx.strokeStyle = palmCol;
|
| 2155 |
+
ctx.lineWidth = 3.5 * dpr;
|
| 2156 |
+
|
| 2157 |
+
// Metacarpal rays: wrist to MCPs (5, 9, 13, 17)
|
| 2158 |
+
[5, 9, 13, 17].forEach(mcp => {
|
| 2159 |
+
ctx.beginPath();
|
| 2160 |
+
ctx.moveTo(wristX, wristY);
|
| 2161 |
+
ctx.lineTo(pts[mcp][0], pts[mcp][1]);
|
| 2162 |
+
ctx.stroke();
|
| 2163 |
+
});
|
| 2164 |
+
|
| 2165 |
+
// Knuckle transverse arch: connecting MCP 5 -> 9 -> 13 -> 17
|
| 2166 |
+
ctx.lineWidth = 4.5 * dpr;
|
| 2167 |
+
ctx.beginPath();
|
| 2168 |
+
ctx.moveTo(pts[5][0], pts[5][1]);
|
| 2169 |
+
ctx.lineTo(pts[9][0], pts[9][1]);
|
| 2170 |
+
ctx.lineTo(pts[13][0], pts[13][1]);
|
| 2171 |
+
ctx.lineTo(pts[17][0], pts[17][1]);
|
| 2172 |
+
ctx.stroke();
|
| 2173 |
+
|
| 2174 |
+
// Finger Segments with exact reference colors:
|
| 2175 |
+
// Thumb: Coral / Red-Pink (wrist 0 -> 1 -> 2 -> 3 -> 4)
|
| 2176 |
+
// Index: Deep Electric Blue (5 -> 6 -> 7 -> 8)
|
| 2177 |
+
// Middle: Bright Emerald Green (9 -> 10 -> 11 -> 12)
|
| 2178 |
+
// Ring: Vibrant Cyan (13 -> 14 -> 15 -> 16)
|
| 2179 |
+
// Pinky: Warm Orange (17 -> 18 -> 19 -> 20)
|
| 2180 |
+
const fingerGroups = [
|
| 2181 |
+
{ indices: [0, 1, 2, 3, 4], color: '#ff4d4d', width: 6.0 * dpr }, // Thumb
|
| 2182 |
+
{ indices: [5, 6, 7, 8], color: '#2979ff', width: 6.0 * dpr }, // Index
|
| 2183 |
+
{ indices: [9, 10, 11, 12], color: '#00e676', width: 6.0 * dpr }, // Middle
|
| 2184 |
+
{ indices: [13, 14, 15, 16], color: '#00e5ff', width: 6.0 * dpr }, // Ring
|
| 2185 |
+
{ indices: [17, 18, 19, 20], color: '#ff9800', width: 6.0 * dpr }, // Pinky
|
| 2186 |
+
];
|
| 2187 |
+
|
| 2188 |
+
fingerGroups.forEach(group => {
|
| 2189 |
+
ctx.strokeStyle = group.color;
|
| 2190 |
+
ctx.lineWidth = group.width;
|
| 2191 |
+
for (let k = 0; k < group.indices.length - 1; k++) {
|
| 2192 |
+
const i1 = group.indices[k];
|
| 2193 |
+
const i2 = group.indices[k + 1];
|
| 2194 |
+
ctx.beginPath();
|
| 2195 |
+
ctx.moveTo(pts[i1][0], pts[i1][1]);
|
| 2196 |
+
ctx.lineTo(pts[i2][0], pts[i2][1]);
|
| 2197 |
+
ctx.stroke();
|
| 2198 |
+
}
|
| 2199 |
+
});
|
| 2200 |
+
|
| 2201 |
+
// Joint caps: Clean pearl white dots
|
| 2202 |
+
ctx.fillStyle = '#ffffff';
|
| 2203 |
+
pts.forEach((pt, idx) => {
|
| 2204 |
+
if (idx === 0) return; // Skip wrist, drawn specially below
|
| 2205 |
+
const radius = (idx % 4 === 0) ? 2.5 * dpr : 2.0 * dpr;
|
| 2206 |
+
ctx.beginPath();
|
| 2207 |
+
ctx.arc(pt[0], pt[1], radius, 0, Math.PI * 2);
|
| 2208 |
+
ctx.fill();
|
| 2209 |
+
});
|
| 2210 |
+
|
| 2211 |
+
// Wrist Joint: Clean pearl white bead
|
| 2212 |
+
ctx.fillStyle = '#ffffff';
|
| 2213 |
+
ctx.beginPath();
|
| 2214 |
+
ctx.arc(wristX, wristY, 5 * dpr, 0, Math.PI * 2);
|
| 2215 |
+
ctx.fill();
|
| 2216 |
+
}
|
| 2217 |
+
|
| 2218 |
+
// Draw Avatar Canvas
|
| 2219 |
+
function drawAvatar() {
|
| 2220 |
+
if (!canvas || !ctx) return;
|
| 2221 |
+
const dpr = window.devicePixelRatio || 1;
|
| 2222 |
+
const w = canvas.width;
|
| 2223 |
+
const h = canvas.height;
|
| 2224 |
+
|
| 2225 |
+
ctx.save();
|
| 2226 |
+
ctx.clearRect(0, 0, w, h);
|
| 2227 |
+
|
| 2228 |
+
// 1. Studio background (theme-aware)
|
| 2229 |
+
const isLight = document.documentElement.getAttribute('data-theme') === 'light';
|
| 2230 |
+
ctx.fillStyle = isLight ? '#eaedf4' : '#161619';
|
| 2231 |
+
ctx.fillRect(0, 0, w, h);
|
| 2232 |
+
|
| 2233 |
+
// 2. Stylized Red Silhouette Avatar (Head, Face, Torso, Arms - matching RyloTranslate)
|
| 2234 |
+
const red = '#d82222';
|
| 2235 |
+
ctx.strokeStyle = red;
|
| 2236 |
+
ctx.fillStyle = red;
|
| 2237 |
+
ctx.lineCap = 'round';
|
| 2238 |
+
ctx.lineJoin = 'round';
|
| 2239 |
+
ctx.shadowBlur = 0;
|
| 2240 |
+
|
| 2241 |
+
// Shoulder & torso anchor points
|
| 2242 |
+
const shoulderY = h * 0.48;
|
| 2243 |
+
const leftShoulderX = w * 0.33;
|
| 2244 |
+
const rightShoulderX = w * 0.71;
|
| 2245 |
+
|
| 2246 |
+
// Head dimensions
|
| 2247 |
+
const headCx = w * 0.51;
|
| 2248 |
+
const headCy = h * 0.25;
|
| 2249 |
+
const headRx = w * 0.082;
|
| 2250 |
+
const headRy = h * 0.125;
|
| 2251 |
+
|
| 2252 |
+
// Head contour
|
| 2253 |
+
ctx.lineWidth = 5 * dpr;
|
| 2254 |
+
ctx.beginPath();
|
| 2255 |
+
ctx.ellipse(headCx, headCy, headRx, headRy, 0, 0, Math.PI * 2);
|
| 2256 |
+
ctx.stroke();
|
| 2257 |
+
|
| 2258 |
+
// Eyebrows (bold expressive arches)
|
| 2259 |
+
ctx.lineWidth = 7 * dpr;
|
| 2260 |
+
ctx.beginPath();
|
| 2261 |
+
ctx.arc(headCx - headRx * 0.44, headCy - headRy * 0.35, headRx * 0.34, Math.PI * 1.1, Math.PI * 1.9, false);
|
| 2262 |
+
ctx.stroke();
|
| 2263 |
+
ctx.beginPath();
|
| 2264 |
+
ctx.arc(headCx + headRx * 0.44, headCy - headRy * 0.35, headRx * 0.34, Math.PI * 1.1, Math.PI * 1.9, false);
|
| 2265 |
+
ctx.stroke();
|
| 2266 |
+
|
| 2267 |
+
// Eyes (almond shaped contours)
|
| 2268 |
+
ctx.lineWidth = 3.5 * dpr;
|
| 2269 |
+
ctx.beginPath();
|
| 2270 |
+
ctx.ellipse(headCx - headRx * 0.42, headCy - headRy * 0.12, headRx * 0.24, headRy * 0.12, 0, 0, Math.PI * 2);
|
| 2271 |
+
ctx.stroke();
|
| 2272 |
+
ctx.beginPath();
|
| 2273 |
+
ctx.ellipse(headCx + headRx * 0.42, headCy - headRy * 0.12, headRx * 0.24, headRy * 0.12, 0, 0, Math.PI * 2);
|
| 2274 |
+
ctx.stroke();
|
| 2275 |
+
|
| 2276 |
+
// Mouth (expressive open oval)
|
| 2277 |
+
ctx.lineWidth = 4.5 * dpr;
|
| 2278 |
+
ctx.beginPath();
|
| 2279 |
+
ctx.ellipse(headCx, headCy + headRy * 0.38, headRx * 0.32, headRy * 0.16, 0, 0, Math.PI * 2);
|
| 2280 |
+
ctx.stroke();
|
| 2281 |
+
|
| 2282 |
+
// Torso Frame:
|
| 2283 |
+
ctx.lineWidth = 5 * dpr;
|
| 2284 |
+
|
| 2285 |
+
// Clavicle / shoulder horizontal bar
|
| 2286 |
+
ctx.beginPath();
|
| 2287 |
+
ctx.moveTo(leftShoulderX, shoulderY);
|
| 2288 |
+
ctx.lineTo(rightShoulderX, shoulderY);
|
| 2289 |
+
ctx.stroke();
|
| 2290 |
+
|
| 2291 |
+
// Torso body lines (tapering downwards)
|
| 2292 |
+
ctx.beginPath();
|
| 2293 |
+
ctx.moveTo(leftShoulderX, shoulderY);
|
| 2294 |
+
ctx.lineTo(w * 0.39, h * 0.98);
|
| 2295 |
+
ctx.moveTo(rightShoulderX, shoulderY);
|
| 2296 |
+
ctx.lineTo(w * 0.65, h * 0.98);
|
| 2297 |
+
ctx.stroke();
|
| 2298 |
+
|
| 2299 |
+
// Determine dual-hand vs single-hand posture
|
| 2300 |
+
const is2H = practiceCurrentPose && practiceCurrentPose.twoHanded && practiceCurrentPose.left;
|
| 2301 |
+
const handScale = Math.min(w * 0.20, h * 0.20);
|
| 2302 |
+
|
| 2303 |
+
if (is2H) {
|
| 2304 |
+
// TWO-HANDED MODE: Both arms raised into signing space
|
| 2305 |
+
const offLx = (practiceCurrentPose.wristL ? practiceCurrentPose.wristL[0] : 0) * handScale;
|
| 2306 |
+
const offLy = (practiceCurrentPose.wristL ? practiceCurrentPose.wristL[1] : 0) * handScale;
|
| 2307 |
+
const offRx = (practiceCurrentPose.wristR ? practiceCurrentPose.wristR[0] : 0) * handScale;
|
| 2308 |
+
const offRy = (practiceCurrentPose.wristR ? practiceCurrentPose.wristR[1] : 0) * handScale;
|
| 2309 |
+
|
| 2310 |
+
const wristL_X = w * 0.37 + offLx;
|
| 2311 |
+
const wristL_Y = h * 0.65 + offLy;
|
| 2312 |
+
const wristR_X = w * 0.63 + offRx;
|
| 2313 |
+
const wristR_Y = h * 0.65 + offRy;
|
| 2314 |
+
|
| 2315 |
+
const elbowL_X = leftShoulderX - w * 0.07;
|
| 2316 |
+
const elbowL_Y = (shoulderY + wristL_Y) / 2 + h * 0.08;
|
| 2317 |
+
const elbowR_X = rightShoulderX + w * 0.07;
|
| 2318 |
+
const elbowR_Y = (shoulderY + wristR_Y) / 2 + h * 0.08;
|
| 2319 |
+
|
| 2320 |
+
// Left Arm (viewer-left): Shoulder -> Elbow -> Wrist
|
| 2321 |
+
ctx.beginPath();
|
| 2322 |
+
ctx.moveTo(leftShoulderX, shoulderY);
|
| 2323 |
+
ctx.lineTo(elbowL_X, elbowL_Y);
|
| 2324 |
+
ctx.lineTo(wristL_X, wristL_Y);
|
| 2325 |
+
ctx.stroke();
|
| 2326 |
+
|
| 2327 |
+
// Right Arm (viewer-right): Shoulder -> Elbow -> Wrist
|
| 2328 |
+
ctx.beginPath();
|
| 2329 |
+
ctx.moveTo(rightShoulderX, shoulderY);
|
| 2330 |
+
ctx.lineTo(elbowR_X, elbowR_Y);
|
| 2331 |
+
ctx.lineTo(wristR_X, wristR_Y);
|
| 2332 |
+
ctx.stroke();
|
| 2333 |
+
|
| 2334 |
+
// Render both hand skeletons
|
| 2335 |
+
drawHandSkeleton(ctx, practiceCurrentPose.left, wristL_X, wristL_Y, dpr, handScale, true);
|
| 2336 |
+
drawHandSkeleton(ctx, practiceCurrentPose.right, wristR_X, wristR_Y, dpr, handScale, false);
|
| 2337 |
+
} else {
|
| 2338 |
+
// SINGLE-HANDED MODE: Dominant signing arm active, other arm resting naturally
|
| 2339 |
+
const wristX = w * 0.34;
|
| 2340 |
+
const wristY = h * 0.65;
|
| 2341 |
+
const elbowL_X = w * 0.24;
|
| 2342 |
+
const elbowL_Y = h * 0.84;
|
| 2343 |
+
|
| 2344 |
+
// Signing Arm (viewer-left): Shoulder -> Elbow -> Wrist
|
| 2345 |
+
ctx.beginPath();
|
| 2346 |
+
ctx.moveTo(leftShoulderX, shoulderY);
|
| 2347 |
+
ctx.lineTo(elbowL_X, elbowL_Y);
|
| 2348 |
+
ctx.lineTo(wristX, wristY);
|
| 2349 |
+
ctx.stroke();
|
| 2350 |
+
|
| 2351 |
+
// Resting Arm (viewer-right): Shoulder -> Elbow -> Forearm down
|
| 2352 |
+
const elbowR_X = w * 0.76;
|
| 2353 |
+
const elbowR_Y = h * 0.82;
|
| 2354 |
+
ctx.beginPath();
|
| 2355 |
+
ctx.moveTo(rightShoulderX, shoulderY);
|
| 2356 |
+
ctx.lineTo(elbowR_X, elbowR_Y);
|
| 2357 |
+
ctx.lineTo(w * 0.68, h * 0.98);
|
| 2358 |
+
ctx.stroke();
|
| 2359 |
+
|
| 2360 |
+
// Render single hand skeleton
|
| 2361 |
+
const singlePose = practiceCurrentPose ? (practiceCurrentPose.right || practiceCurrentPose.left) : null;
|
| 2362 |
+
if (singlePose) {
|
| 2363 |
+
drawHandSkeleton(ctx, singlePose, wristX, wristY, dpr, handScale, false);
|
| 2364 |
+
}
|
| 2365 |
+
}
|
| 2366 |
+
|
| 2367 |
+
ctx.restore();
|
| 2368 |
+
}
|
| 2369 |
+
|
| 2370 |
+
// Start animation loop
|
| 2371 |
+
requestAnimationFrame(avatarAnimationLoop);
|
| 2372 |
+
}
|
| 2373 |
+
|
| 2374 |
+
function activatePracticeStudio() {
|
| 2375 |
+
const canvas = document.getElementById('avatar-canvas');
|
| 2376 |
+
if (canvas && canvas.parentElement) {
|
| 2377 |
+
const rect = canvas.parentElement.getBoundingClientRect();
|
| 2378 |
+
const dpr = window.devicePixelRatio || 1;
|
| 2379 |
+
if (rect.width > 0 && rect.height > 0) {
|
| 2380 |
+
canvas.width = Math.floor(rect.width) * dpr;
|
| 2381 |
+
canvas.height = Math.floor(rect.height) * dpr;
|
| 2382 |
+
}
|
| 2383 |
+
}
|
| 2384 |
+
}
|
asl_landmarks_weights.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
asl_signs/A.png
ADDED
|
asl_signs/B.png
ADDED
|
asl_signs/C.png
ADDED
|
asl_signs/D.png
ADDED
|
asl_signs/E.png
ADDED
|
asl_signs/F.png
ADDED
|
asl_signs/G.png
ADDED
|
asl_signs/H.png
ADDED
|
asl_signs/I.png
ADDED
|
asl_signs/J.png
ADDED
|
asl_signs/K.png
ADDED
|
asl_signs/L.png
ADDED
|
asl_signs/M.png
ADDED
|
asl_signs/N.png
ADDED
|
asl_signs/O.png
ADDED
|
asl_signs/P.png
ADDED
|
asl_signs/Q.png
ADDED
|
asl_signs/R.png
ADDED
|
asl_signs/S.png
ADDED
|
asl_signs/T.png
ADDED
|
asl_signs/U.png
ADDED
|
asl_signs/V.png
ADDED
|
asl_signs/W.png
ADDED
|
asl_signs/X.png
ADDED
|
asl_signs/Y.png
ADDED
|
asl_signs/Z.png
ADDED
|
index.html
CHANGED
|
@@ -1,19 +1,393 @@
|
|
| 1 |
-
<!
|
| 2 |
-
<html>
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
<
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
<!DOCTYPE html>
|
| 2 |
+
<html lang="en">
|
| 3 |
+
<head>
|
| 4 |
+
<meta charset="UTF-8" />
|
| 5 |
+
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
| 6 |
+
<meta name="color-scheme" content="dark" />
|
| 7 |
+
<title>SignLens</title>
|
| 8 |
+
|
| 9 |
+
<link rel="preconnect" href="https://fonts.googleapis.com" />
|
| 10 |
+
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
|
| 11 |
+
<link href="https://fonts.googleapis.com/css2?family=Geist:wght@300;400;500;600;700&family=Geist+Mono:wght@400;500;600&display=swap" rel="stylesheet" />
|
| 12 |
+
<link rel="stylesheet" href="style.css?v=14" />
|
| 13 |
+
|
| 14 |
+
<!-- MediaPipe Tasks Vision loaded dynamically inside app.js initMediaPipe() -->
|
| 15 |
+
</head>
|
| 16 |
+
<body>
|
| 17 |
+
|
| 18 |
+
<!-- โโ TOP BAR โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -->
|
| 19 |
+
<header class="topbar">
|
| 20 |
+
|
| 21 |
+
<div class="brand">
|
| 22 |
+
<!-- Single geometric mark: three ascending strokes, a hand reading as signal. -->
|
| 23 |
+
<svg class="brand-mark" viewBox="0 0 24 24" aria-hidden="true" focusable="false">
|
| 24 |
+
<rect x="3" y="13" width="4" height="8" rx="2" />
|
| 25 |
+
<rect x="10" y="7" width="4" height="14" rx="2" />
|
| 26 |
+
<rect x="17" y="3" width="4" height="18" rx="2" />
|
| 27 |
+
</svg>
|
| 28 |
+
<span class="brand-name">SignLens</span>
|
| 29 |
+
</div>
|
| 30 |
+
|
| 31 |
+
<div class="mode-toggle" id="mode-toggle" role="group" aria-label="Sign language">
|
| 32 |
+
<button class="mode-btn active" data-mode="asl" id="btn-asl" aria-pressed="true">ASL</button>
|
| 33 |
+
<button class="mode-btn" data-mode="isl" id="btn-isl" aria-pressed="false">ISL</button>
|
| 34 |
+
<button class="mode-btn" data-mode="visually" id="btn-visually" aria-pressed="false">VISUALLY</button>
|
| 35 |
+
<span class="mode-slider" aria-hidden="true"></span>
|
| 36 |
+
</div>
|
| 37 |
+
|
| 38 |
+
<div class="topbar-right">
|
| 39 |
+
<button class="ghost-btn guide-btn" popovertarget="panel-guide" id="btn-open-guide" aria-label="Open Sign Guide">
|
| 40 |
+
<svg width="15" height="15" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" style="vertical-align: -2px; margin-right: 5px;"><path d="M2 3h6a4 4 0 0 1 4 4v14a3 3 0 0 0-3-3H2z"></path><path d="M22 3h-6a4 4 0 0 0-4 4v14a3 3 0 0 1 3-3h7z"></path></svg><span>Sign Guide</span>
|
| 41 |
+
</button>
|
| 42 |
+
<button class="ghost-btn" popovertarget="panel-settings">Settings</button>
|
| 43 |
+
<button class="ghost-btn" popovertarget="panel-help">Shortcuts</button>
|
| 44 |
+
<button class="ghost-btn theme-toggle-btn" id="btn-theme-toggle" aria-label="Toggle Light or Dark Mode" title="Toggle Light / Dark Mode">
|
| 45 |
+
<span id="theme-toggle-icon" aria-hidden="true">โ๏ธ</span>
|
| 46 |
+
</button>
|
| 47 |
+
<div class="status-pill" id="status-pill">
|
| 48 |
+
<span class="status-dot" aria-hidden="true"></span>
|
| 49 |
+
<span id="status-text" role="status" aria-live="polite">Connecting</span>
|
| 50 |
+
</div>
|
| 51 |
+
</div>
|
| 52 |
+
</header>
|
| 53 |
+
|
| 54 |
+
<!-- โโ WORKSPACE (LIVE CAMERA) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -->
|
| 55 |
+
<main class="workspace" id="live-workspace">
|
| 56 |
+
|
| 57 |
+
<!-- โโ Camera stage โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -->
|
| 58 |
+
<section class="stage" aria-label="Camera">
|
| 59 |
+
|
| 60 |
+
<div class="viewport">
|
| 61 |
+
<video id="webcam-video" autoplay playsinline muted></video>
|
| 62 |
+
<canvas id="display-canvas"></canvas>
|
| 63 |
+
|
| 64 |
+
<div class="no-hand-overlay visible" id="no-hand-overlay">
|
| 65 |
+
<div class="idle-title">Show your hand to the camera</div>
|
| 66 |
+
<div class="idle-sub">Tracking starts as soon as a hand is in frame.</div>
|
| 67 |
+
</div>
|
| 68 |
+
</div>
|
| 69 |
+
|
| 70 |
+
<!-- Live readout: the one thing the user actually watches -->
|
| 71 |
+
<div class="readout">
|
| 72 |
+
|
| 73 |
+
<div class="readout-dial">
|
| 74 |
+
<div class="dial-ring" aria-hidden="true"></div>
|
| 75 |
+
<div class="detected-letter" id="detected-letter" role="status" aria-live="polite" aria-atomic="true"></div>
|
| 76 |
+
</div>
|
| 77 |
+
|
| 78 |
+
<div class="readout-body">
|
| 79 |
+
<div class="readout-line">
|
| 80 |
+
<span class="detected-sub" id="detected-conf">Waiting</span>
|
| 81 |
+
<span class="hand-badge" id="hand-badge" role="status" aria-live="polite">NO HAND</span>
|
| 82 |
+
</div>
|
| 83 |
+
|
| 84 |
+
<div class="streak-row">
|
| 85 |
+
<div class="streak-track"><div class="streak-fill" id="streak-bar"></div></div>
|
| 86 |
+
<span class="streak-label" id="streak-label">0 / 5</span>
|
| 87 |
+
</div>
|
| 88 |
+
</div>
|
| 89 |
+
|
| 90 |
+
<div class="stage-actions">
|
| 91 |
+
<button class="ctrl-btn" popovertarget="panel-guide" id="stage-guide-btn" title="View official sign chart">Guide</button>
|
| 92 |
+
<button class="ctrl-btn" id="btn-pause-detection">Pause</button>
|
| 93 |
+
<button class="ctrl-btn active-btn" id="btn-camera-toggle">Stop Camera</button>
|
| 94 |
+
</div>
|
| 95 |
+
</div>
|
| 96 |
+
</section>
|
| 97 |
+
|
| 98 |
+
<!-- โโ Output rail โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -->
|
| 99 |
+
<aside class="rail">
|
| 100 |
+
|
| 101 |
+
<article class="panel" id="word-panel">
|
| 102 |
+
<header class="panel-head">
|
| 103 |
+
<h2 class="panel-title">Building word</h2>
|
| 104 |
+
</header>
|
| 105 |
+
|
| 106 |
+
<div class="word-display" id="word-display">
|
| 107 |
+
<span class="ph">Start signing</span>
|
| 108 |
+
</div>
|
| 109 |
+
|
| 110 |
+
<div class="panel-actions">
|
| 111 |
+
<button class="btn" id="btn-backspace">Delete</button>
|
| 112 |
+
<button class="btn" id="btn-add-space">Next word</button>
|
| 113 |
+
<button class="btn btn-quiet" id="btn-clear-word">Clear</button>
|
| 114 |
+
</div>
|
| 115 |
+
</article>
|
| 116 |
+
|
| 117 |
+
<article class="panel panel-grow" id="sentence-panel">
|
| 118 |
+
<header class="panel-head">
|
| 119 |
+
<h2 class="panel-title">Sentence</h2>
|
| 120 |
+
<div class="panel-head-actions">
|
| 121 |
+
<button class="btn btn-sm" id="btn-speak">Speak</button>
|
| 122 |
+
<button class="btn btn-sm" id="btn-copy">Copy</button>
|
| 123 |
+
</div>
|
| 124 |
+
</header>
|
| 125 |
+
|
| 126 |
+
<div class="sentence-display" id="sentence-display" role="status" aria-live="polite">
|
| 127 |
+
<em class="ph">Your sentence will appear here</em>
|
| 128 |
+
</div>
|
| 129 |
+
|
| 130 |
+
<div class="word-chips" id="word-chips"></div>
|
| 131 |
+
|
| 132 |
+
<button class="btn btn-danger" id="btn-clear-sentence">Clear all</button>
|
| 133 |
+
</article>
|
| 134 |
+
</aside>
|
| 135 |
+
</main>
|
| 136 |
+
|
| 137 |
+
<!-- โโ VISUALLY (PRACTICE STUDIO) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -->
|
| 138 |
+
<section class="workspace visually-workspace hidden" id="visually-workspace" aria-label="Practice Studio">
|
| 139 |
+
<!-- Left Column: Input, Speech Dictation, and Sequence Breakdown -->
|
| 140 |
+
<aside class="practice-input-rail">
|
| 141 |
+
<article class="panel panel-grow practice-panel">
|
| 142 |
+
<header class="panel-head">
|
| 143 |
+
<div class="practice-head-title">
|
| 144 |
+
<span class="practice-badge">PRACTICE STUDIO</span>
|
| 145 |
+
<h2 class="panel-title">Text to Sign</h2>
|
| 146 |
+
</div>
|
| 147 |
+
<div class="practice-dialect-toggle" id="practice-dialect-toggle">
|
| 148 |
+
<button class="dialect-pill active" id="btn-practice-asl">ASL</button>
|
| 149 |
+
<button class="dialect-pill" id="btn-practice-isl">ISL</button>
|
| 150 |
+
</div>
|
| 151 |
+
</header>
|
| 152 |
+
|
| 153 |
+
<!-- Textarea with integrated voice mic & actions -->
|
| 154 |
+
<div class="practice-textarea-box">
|
| 155 |
+
<textarea id="practice-input" placeholder="Type words or sentence (e.g. 'HI BABY', 'HOW ARE YOU')..." rows="4"></textarea>
|
| 156 |
+
<div class="practice-input-tools">
|
| 157 |
+
<button class="tool-btn mic-btn" id="btn-mic" title="Voice Input (Speech-to-Sign)">
|
| 158 |
+
<svg class="icon" viewBox="0 0 24 24" fill="currentColor">
|
| 159 |
+
<path d="M12 14c1.66 0 3-1.34 3-3V5c0-1.66-1.34-3-3-3S9 3.34 9 5v6c0 1.66 1.34 3 3 3zm5-3c0 2.76-2.24 5-5 5s-5-2.24-5-5H5c0 3.53 2.61 6.43 6 6.92V21h2v-3.08c3.39-.49 6-3.39 6-6.92h-2z"/>
|
| 160 |
+
</svg>
|
| 161 |
+
<span id="mic-label">Speak</span>
|
| 162 |
+
</button>
|
| 163 |
+
<button class="tool-btn" id="btn-speak-input" title="Speak text aloud">
|
| 164 |
+
<svg class="icon" viewBox="0 0 24 24" fill="currentColor">
|
| 165 |
+
<path d="M3 9v6h4l5 5V4L7 9H3zm13.5 3c0-1.77-1.02-3.29-2.5-4.03v8.05c1.48-.73 2.5-2.25 2.5-4.02zM14 3.23v2.06c2.89.86 5 3.54 5 6.71s-2.11 5.85-5 6.71v2.06c4.01-.91 7-4.49 7-8.77s-2.99-7.86-7-8.77z"/>
|
| 166 |
+
</svg>
|
| 167 |
+
</button>
|
| 168 |
+
<button class="tool-btn" id="btn-clear-practice" title="Clear text">Clear</button>
|
| 169 |
+
<span class="char-count" id="practice-char-count">0/200</span>
|
| 170 |
+
</div>
|
| 171 |
+
</div>
|
| 172 |
+
|
| 173 |
+
<!-- Quick sample prompts -->
|
| 174 |
+
<div class="quick-samples">
|
| 175 |
+
<span class="quick-label">Try:</span>
|
| 176 |
+
<button class="sample-chip" data-text="HI">HI</button>
|
| 177 |
+
<button class="sample-chip" data-text="HELLO">HELLO</button>
|
| 178 |
+
<button class="sample-chip" data-text="THANK YOU">THANK YOU</button>
|
| 179 |
+
<button class="sample-chip" data-text="HOW ARE YOU">HOW ARE YOU</button>
|
| 180 |
+
<button class="sample-chip" data-text="BABY">BABY</button>
|
| 181 |
+
</div>
|
| 182 |
+
|
| 183 |
+
<!-- Main Trigger & Speed -->
|
| 184 |
+
<div class="practice-action-row">
|
| 185 |
+
<button class="btn btn-primary btn-demo" id="btn-start-sign">
|
| 186 |
+
<svg class="icon-play" viewBox="0 0 24 24" fill="currentColor"><path d="M8 5v14l11-7z"/></svg>
|
| 187 |
+
<span>Demonstrate Sign</span>
|
| 188 |
+
</button>
|
| 189 |
+
|
| 190 |
+
<div class="speed-control">
|
| 191 |
+
<span class="speed-label">Speed</span>
|
| 192 |
+
<button class="speed-btn" data-speed="0.5">0.5x</button>
|
| 193 |
+
<button class="speed-btn active" data-speed="1.0">1.0x</button>
|
| 194 |
+
<button class="speed-btn" data-speed="1.5">1.5x</button>
|
| 195 |
+
</div>
|
| 196 |
+
</div>
|
| 197 |
+
|
| 198 |
+
<!-- Letter Tape Progression -->
|
| 199 |
+
<div class="sequence-section">
|
| 200 |
+
<div class="sequence-head">
|
| 201 |
+
<span class="seq-title">Sign Sequence Timeline</span>
|
| 202 |
+
<span class="seq-counter" id="seq-counter">0 / 0</span>
|
| 203 |
+
</div>
|
| 204 |
+
<div class="sequence-chips" id="sequence-chips">
|
| 205 |
+
<span class="ph">Enter text above to preview gesture timelineโฆ</span>
|
| 206 |
+
</div>
|
| 207 |
+
</div>
|
| 208 |
+
</article>
|
| 209 |
+
</aside>
|
| 210 |
+
|
| 211 |
+
<!-- Right Column: Interactive 3D Avatar Skeleton Stage -->
|
| 212 |
+
<section class="practice-stage" aria-label="Avatar Stage">
|
| 213 |
+
<div class="avatar-viewport">
|
| 214 |
+
<canvas id="avatar-canvas"></canvas>
|
| 215 |
+
|
| 216 |
+
<!-- Active Letter Glow Badge -->
|
| 217 |
+
<div class="avatar-hud-letter" id="avatar-hud-letter">
|
| 218 |
+
<span class="hud-letter-char" id="hud-letter-char">โ</span>
|
| 219 |
+
<span class="hud-letter-sub" id="hud-letter-sub">Waiting for input</span>
|
| 220 |
+
<span class="hud-letter-cue" id="hud-letter-cue" style="display: none;"></span>
|
| 221 |
+
</div>
|
| 222 |
+
|
| 223 |
+
<!-- Avatar Status Badge -->
|
| 224 |
+
<div class="avatar-hud-status" id="avatar-hud-status">
|
| 225 |
+
<span class="status-badge-dot"></span>
|
| 226 |
+
<span id="avatar-status-text">Ready</span>
|
| 227 |
+
</div>
|
| 228 |
+
|
| 229 |
+
<!-- Real Human Sign Reference Card (Top-Right) -->
|
| 230 |
+
<div class="avatar-sign-preview" id="avatar-sign-preview" title="Official sign gesture photo reference" style="display: none;">
|
| 231 |
+
<div class="avatar-sign-img-wrap">
|
| 232 |
+
<img id="avatar-sign-img" src="" alt="Sign Gesture Reference" />
|
| 233 |
+
<span class="avatar-sign-badge" id="avatar-sign-badge">ISL</span>
|
| 234 |
+
</div>
|
| 235 |
+
<div class="avatar-sign-info">
|
| 236 |
+
<div class="avatar-sign-char-row">
|
| 237 |
+
<span class="avatar-sign-char" id="avatar-sign-char">A</span>
|
| 238 |
+
<span class="avatar-sign-hands" id="avatar-sign-hands">1 Hand</span>
|
| 239 |
+
</div>
|
| 240 |
+
<span class="avatar-sign-desc" id="avatar-sign-desc">Sign gesture reference</span>
|
| 241 |
+
</div>
|
| 242 |
+
</div>
|
| 243 |
+
|
| 244 |
+
|
| 245 |
+
<!-- Playback Controls Floating Bar -->
|
| 246 |
+
<div class="avatar-controls-bar">
|
| 247 |
+
<button class="avatar-ctrl-btn" id="btn-avatar-prev" title="Previous letter">โฎ</button>
|
| 248 |
+
<button class="avatar-ctrl-btn play-pause-btn" id="btn-avatar-playpause" title="Play/Pause">โธ</button>
|
| 249 |
+
<button class="avatar-ctrl-btn" id="btn-avatar-next" title="Next letter">โญ</button>
|
| 250 |
+
<button class="avatar-ctrl-btn" id="btn-avatar-replay" title="Replay all">โฒ</button>
|
| 251 |
+
<div class="avatar-progress-wrap">
|
| 252 |
+
<input type="range" class="avatar-scrubber" id="avatar-scrubber" min="0" max="100" value="0" step="0.1" />
|
| 253 |
+
</div>
|
| 254 |
+
</div>
|
| 255 |
+
</div>
|
| 256 |
+
</section>
|
| 257 |
+
</section>
|
| 258 |
+
|
| 259 |
+
<!-- โโ SETTINGS (top layer) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -->
|
| 260 |
+
<div popover id="panel-settings" class="sheet" style="display: none;">
|
| 261 |
+
<header class="sheet-head">
|
| 262 |
+
<h2 class="sheet-title">Detection</h2>
|
| 263 |
+
<button class="ghost-btn" popovertarget="panel-settings" popovertargetaction="hide">Close</button>
|
| 264 |
+
</header>
|
| 265 |
+
|
| 266 |
+
<div class="field">
|
| 267 |
+
<label for="set-confidence">Minimum confidence</label>
|
| 268 |
+
<div class="field-row">
|
| 269 |
+
<input type="range" id="set-confidence" min="40" max="95" step="5" value="50" />
|
| 270 |
+
<output class="field-value" id="out-confidence">50%</output>
|
| 271 |
+
</div>
|
| 272 |
+
<p class="field-help">Predictions below this are ignored. Raise it if letters are captured too eagerly.</p>
|
| 273 |
+
</div>
|
| 274 |
+
|
| 275 |
+
<div class="field">
|
| 276 |
+
<label for="set-streak">Frames to confirm</label>
|
| 277 |
+
<div class="field-row">
|
| 278 |
+
<input type="range" id="set-streak" min="3" max="15" step="1" value="5" />
|
| 279 |
+
<output class="field-value" id="out-streak">5</output>
|
| 280 |
+
</div>
|
| 281 |
+
<p class="field-help">How long the same sign must hold before the letter is captured.</p>
|
| 282 |
+
</div>
|
| 283 |
+
|
| 284 |
+
<div class="field">
|
| 285 |
+
<label for="set-timeout">Auto-push word delay</label>
|
| 286 |
+
<div class="field-row">
|
| 287 |
+
<input type="range" id="set-timeout" min="1.0" max="5.0" step="0.5" value="3.0" />
|
| 288 |
+
<output class="field-value" id="out-timeout">3.0s</output>
|
| 289 |
+
</div>
|
| 290 |
+
<p class="field-help">How long you must lower your hand before the word commits to sentence.</p>
|
| 291 |
+
</div>
|
| 292 |
+
</div>
|
| 293 |
+
|
| 294 |
+
<!-- โโ SHORTCUTS (top layer) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -->
|
| 295 |
+
<div popover id="panel-help" class="sheet" style="display: none;">
|
| 296 |
+
<header class="sheet-head">
|
| 297 |
+
<h2 class="sheet-title">How it works</h2>
|
| 298 |
+
<button class="ghost-btn" popovertarget="panel-help" popovertargetaction="hide">Close</button>
|
| 299 |
+
</header>
|
| 300 |
+
|
| 301 |
+
<ol class="steps">
|
| 302 |
+
<li><span class="step-n">1</span> Show a hand sign to the camera.</li>
|
| 303 |
+
<li><span class="step-n">2</span> Hold it steady while the ring fills.</li>
|
| 304 |
+
<li><span class="step-n">3</span> The letter is captured and added to the word.</li>
|
| 305 |
+
<li><span class="step-n">4</span> Drop your hand to commit the word to the sentence.</li>
|
| 306 |
+
</ol>
|
| 307 |
+
|
| 308 |
+
<div class="keys">
|
| 309 |
+
<div class="key-row"><kbd>Space</kbd><span>Commit current word</span></div>
|
| 310 |
+
<div class="key-row"><kbd>Backspace</kbd><span>Delete last letter</span></div>
|
| 311 |
+
<div class="key-row"><kbd>1</kbd><span>Switch to ASL</span></div>
|
| 312 |
+
<div class="key-row"><kbd>2</kbd><span>Switch to ISL</span></div>
|
| 313 |
+
<div class="key-row"><kbd>3</kbd><span>Switch to Practice (VISUALLY)</span></div>
|
| 314 |
+
</div>
|
| 315 |
+
</div>
|
| 316 |
+
|
| 317 |
+
<!-- โโ SIGN REFERENCE GUIDE (top layer modal) โโโโโโโโโโโโโโโโโโโโโโโโโโโ -->
|
| 318 |
+
<div popover id="panel-guide" class="sheet sheet-wide" style="display: none;">
|
| 319 |
+
<header class="sheet-head">
|
| 320 |
+
<div class="sheet-title-group">
|
| 321 |
+
<h2 class="sheet-title">Sign Reference Guide</h2>
|
| 322 |
+
<span class="sheet-sub">Official hand gestures, required hand counts & poses</span>
|
| 323 |
+
</div>
|
| 324 |
+
<div class="guide-tabs" role="tablist">
|
| 325 |
+
<button class="guide-tab-btn active" id="tab-guide-isl" role="tab" aria-selected="true">๐ฎ๐ณ ISL (Indian Sign Language)</button>
|
| 326 |
+
<button class="guide-tab-btn" id="tab-guide-asl" role="tab" aria-selected="false">๐บ๐ธ ASL (American Sign Language)</button>
|
| 327 |
+
</div>
|
| 328 |
+
<button class="ghost-btn" id="btn-close-guide" popovertarget="panel-guide" popovertargetaction="hide">Close</button>
|
| 329 |
+
</header>
|
| 330 |
+
|
| 331 |
+
<!-- ISL Tab Content -->
|
| 332 |
+
<div class="guide-tab-content" id="guide-content-isl">
|
| 333 |
+
<div class="guide-alert-banner">
|
| 334 |
+
<span class="banner-icon">โ ๏ธ</span>
|
| 335 |
+
<div>
|
| 336 |
+
<strong>Crucial Rule in ISL:</strong> 20 alphabet letters (<kbd>A, B, D, E, F, G, H, J, K, M, N, P, Q, R, S, T, W, X, Y, Z</kbd>) strictly require <strong>TWO HANDS</strong>. Only numbers <kbd>1-9</kbd> and letters <kbd>C, I, L, O, U, V</kbd> use a single hand!
|
| 337 |
+
</div>
|
| 338 |
+
</div>
|
| 339 |
+
|
| 340 |
+
<div class="guide-section">
|
| 341 |
+
<div class="guide-section-head">
|
| 342 |
+
<h3 class="guide-section-title">Single-Hand Gestures (15 Signs)</h3>
|
| 343 |
+
<span class="pill-badge pill-amber">1-Hand Required</span>
|
| 344 |
+
</div>
|
| 345 |
+
<div class="guide-cards-grid" id="isl-single-cards"></div>
|
| 346 |
+
</div>
|
| 347 |
+
|
| 348 |
+
<div class="guide-section" style="margin-top: 28px;">
|
| 349 |
+
<div class="guide-section-head">
|
| 350 |
+
<h3 class="guide-section-title">Dual-Hand Gestures (20 Signs)</h3>
|
| 351 |
+
<span class="pill-badge pill-blue">2-Hands Required</span>
|
| 352 |
+
</div>
|
| 353 |
+
<div class="guide-cards-grid" id="isl-dual-cards"></div>
|
| 354 |
+
</div>
|
| 355 |
+
</div>
|
| 356 |
+
|
| 357 |
+
<!-- ASL Tab Content -->
|
| 358 |
+
<div class="guide-tab-content hidden" id="guide-content-asl">
|
| 359 |
+
<div class="guide-alert-banner">
|
| 360 |
+
<span class="banner-icon">๐ก</span>
|
| 361 |
+
<div>
|
| 362 |
+
<strong>Rule in ASL:</strong> All 26 alphabet letters (<kbd>A to Z</kbd>) are performed using <strong>ONE dominant hand</strong>.
|
| 363 |
+
</div>
|
| 364 |
+
</div>
|
| 365 |
+
|
| 366 |
+
<div class="guide-section">
|
| 367 |
+
<div class="guide-section-head">
|
| 368 |
+
<h3 class="guide-section-title">ASL Alphabet (26 Signs)</h3>
|
| 369 |
+
<span class="pill-badge pill-amber">1-Hand Only</span>
|
| 370 |
+
</div>
|
| 371 |
+
<div class="guide-cards-grid" id="asl-cards"></div>
|
| 372 |
+
</div>
|
| 373 |
+
</div>
|
| 374 |
+
</div>
|
| 375 |
+
|
| 376 |
+
<!-- โโ SIGN GUIDE LIGHTBOX POPUP (Top-layer native dialog) โโโโโโโโโโโโโโโโโโโโโโโโ -->
|
| 377 |
+
<dialog class="sign-lightbox-dialog" id="sign-lightbox-overlay">
|
| 378 |
+
<div class="sign-lightbox" id="sign-lightbox">
|
| 379 |
+
<button class="sign-lightbox-close" id="sign-lightbox-close" aria-label="Close preview">×</button>
|
| 380 |
+
<div class="sign-lightbox-img-wrap">
|
| 381 |
+
<img id="sign-lightbox-img" src="" alt="Sign preview" />
|
| 382 |
+
</div>
|
| 383 |
+
<div class="sign-lightbox-letter" id="sign-lightbox-letter"></div>
|
| 384 |
+
<span class="sign-lightbox-type" id="sign-lightbox-type"></span>
|
| 385 |
+
<p class="sign-lightbox-desc" id="sign-lightbox-desc"></p>
|
| 386 |
+
<div class="sign-lightbox-lang" id="sign-lightbox-lang"></div>
|
| 387 |
+
</div>
|
| 388 |
+
</dialog>
|
| 389 |
+
|
| 390 |
+
<div class="toast" id="toast" role="status" aria-live="polite"></div>
|
| 391 |
+
<script src="app.js?v=19"></script>
|
| 392 |
+
</body>
|
| 393 |
+
</html>
|
isl_landmarks_weights.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
isl_signs/1.png
ADDED
|
isl_signs/2.png
ADDED
|
isl_signs/3.png
ADDED
|
isl_signs/4.png
ADDED
|
isl_signs/5.png
ADDED
|
isl_signs/6.png
ADDED
|
isl_signs/7.png
ADDED
|
isl_signs/8.png
ADDED
|
isl_signs/9.png
ADDED
|
isl_signs/A.png
ADDED
|
isl_signs/B.png
ADDED
|
isl_signs/C.png
ADDED
|
isl_signs/D.png
ADDED
|
isl_signs/E.png
ADDED
|
isl_signs/F.png
ADDED
|
isl_signs/G.png
ADDED
|