venmugilrajan commited on
Commit
b1d85c8
ยท
verified ยท
1 Parent(s): 86c2dc4

feat: deploy SignLens web app to Hugging Face Spaces

Browse files
This view is limited to 50 files because it contains too many changes. ย  See raw diff
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ ASL_Alphabet_Chart.png filter=lfs diff=lfs merge=lfs -text
37
+ ISL_Hand_Signs_Official.png filter=lfs diff=lfs merge=lfs -text
ASL_Alphabet_Chart.png ADDED

Git LFS Details

  • SHA256: 2c1dba6ed870a2f0c6e0f20e30548e2ba8acd14f7f7e733b1d5e1d1098472bb8
  • Pointer size: 132 Bytes
  • Size of remote file: 1.16 MB
ISL_Hand_Signs_Official.png ADDED

Git LFS Details

  • SHA256: 792dc1b24e71fe43451a4d5751b9ef0a4a695fe844da0d9c5f7e259c61ae65bf
  • Pointer size: 132 Bytes
  • Size of remote file: 1.73 MB
README.md CHANGED
@@ -1,10 +1,43 @@
1
  ---
2
  title: SignLens
3
- emoji: ๐Ÿ‘€
4
- colorFrom: indigo
5
  colorTo: indigo
6
  sdk: static
7
  pinned: false
 
 
 
 
 
 
 
 
 
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  title: SignLens
3
+ emoji: ๐ŸคŸ
4
+ colorFrom: yellow
5
  colorTo: indigo
6
  sdk: static
7
  pinned: false
8
+ license: apache-2.0
9
+ tags:
10
+ - sign-language-detection
11
+ - computer-vision
12
+ - mediapipe
13
+ - indian-sign-language
14
+ - american-sign-language
15
+ - accessibility
16
+ - neural-network
17
  ---
18
 
19
+ # ๐ŸคŸ SignLens: Real-Time Sign Language Translation & Avatar Practice Studio
20
+
21
+ Welcome to **SignLens** running live on Hugging Face Spaces!
22
+
23
+ SignLens is a high-speed, client-side neural landmark translation engine that translates **Indian Sign Language (ISL)** and **American Sign Language (ASL)** directly in your browser with zero server latency.
24
+
25
+ ## โœจ Features
26
+ 1. **Live Camera Recognition**:
27
+ - High-speed 3D landmark tracking powered by MediaPipe WebVision.
28
+ - Dual-mode support for both **ISL** (Dual-Hand 35 classes) and **ASL** (Single-Hand 28 classes).
29
+ - In-browser neural forward pass with confidence thresholds, hold gauge, and streak confirmation.
30
+ 2. **Auto-Framing Word & Sentence Builder**:
31
+ - Eliminates gesture spam and flickering.
32
+ - Dynamic word boundary detection and Text-to-Speech (TTS) voice readout.
33
+ 3. **VISUALLY โ€” 3D Skeletal Avatar Studio**:
34
+ - Type any phrase or click sample prompts to watch the interactive avatar demonstrate correct hand and arm motions step-by-step.
35
+ - Interactive scrub bar, playback speed control (0.5x, 1.0x, 1.5x), and live letter timeline.
36
+ - Top-right real-human sign reference card synchronized with the skeleton.
37
+ 4. **Official Sign Reference Guide**:
38
+ - Full dictionary of official poses, hand counts (1-Hand vs 2-Hand rules), and high-resolution photo popups.
39
+
40
+ ## ๐Ÿ”— Project Links
41
+ - **Model Card & Evaluation**: [Hugging Face: SignLens-Landmark-Model](https://huggingface.co/venmugilrajan/SignLens-Landmark-Model)
42
+ - **Source Code**: [GitHub: Sign_lang](https://github.com/venmugilrajan/Sign_lang)
43
+ - **Author**: Venmugil Rajan
app.js ADDED
@@ -0,0 +1,2384 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /**
2
+ * SignLens โ€” MediaPipe Landmark Neural Network Edition
3
+ *
4
+ * Runs classification directly in the browser using the weights
5
+ * of our custom MLP Neural Network trained on the 21 hand landmarks.
6
+ */
7
+
8
+ // โ”€โ”€ Config โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
9
+ let STREAK_NEEDED = 5; // frames of same letter to capture (settings sheet, matches native)
10
+ const COOLDOWN_MS = 350; // ms after capture before detecting again (fluid sign transitions)
11
+ let SPACE_FRAMES = 90; // no-hand frames before auto word push (~3.0s at 30fps, plenty of transition time)
12
+ let MIN_CONFIDENCE = 0.50; // minimum classifier confidence (settings sheet, matches native 0.40-0.50)
13
+ const MIN_HAND_SPAN = 35; // minimum pixel distance between wrist and middle MCP to reject ghost hands
14
+
15
+ // โ”€โ”€ MediaPipe Hand Connections โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
16
+ const CONNECTIONS = [
17
+ [0,1],[1,2],[2,3],[3,4],
18
+ [0,5],[5,6],[6,7],[7,8],
19
+ [0,9],[9,10],[10,11],[11,12],
20
+ [0,13],[13,14],[14,15],[15,16],
21
+ [0,17],[17,18],[18,19],[19,20],
22
+ [5,9],[9,13],[13,17],
23
+ ];
24
+
25
+ const NODE_COLORS = {
26
+ 0:'#ffffff',
27
+ 1:'#f2856a',2:'#f2856a',3:'#f2856a',4:'#f2856a',
28
+ 5:'#f0a53f',6:'#f0a53f',7:'#f0a53f',8:'#f0a53f',
29
+ 9:'#6ec9a0',10:'#6ec9a0',11:'#6ec9a0',12:'#6ec9a0',
30
+ 13:'#5aa9e6',14:'#5aa9e6',15:'#5aa9e6',16:'#5aa9e6',
31
+ 17:'#b98ae0',18:'#b98ae0',19:'#b98ae0',20:'#b98ae0',
32
+ };
33
+
34
+ const EDGE_COLORS = [
35
+ '#f2856a','#f2856a','#f2856a','#f2856a', // thumb 0-3
36
+ '#f0a53f','#f0a53f','#f0a53f','#f0a53f', // index 4-7
37
+ '#6ec9a0','#6ec9a0','#6ec9a0','#6ec9a0', // middle 8-11
38
+ '#5aa9e6','#5aa9e6','#5aa9e6','#5aa9e6', // ring 12-15
39
+ '#b98ae0','#b98ae0','#b98ae0','#b98ae0', // pinky 16-19
40
+ '#6a6a76','#6a6a76','#6a6a76', // palm 20-22
41
+ ];
42
+
43
+ // โ”€โ”€ State โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
44
+ let currentMode = 'asl';
45
+ let paused = false;
46
+ let inCooldown = false;
47
+ let cameraActive = false;
48
+
49
+ let currentWord = '';
50
+ let letterConfidences = [];
51
+ let sentenceWords = [];
52
+
53
+ let streakLetter = null;
54
+ let streakCount = 0;
55
+ let noHandCount = 0;
56
+
57
+ // State tracking to prevent runaway letter repeat without releasing hand
58
+ let lastCapturedLetter = '';
59
+ let releasedSinceLastCapture = true;
60
+ let releaseFrames = 0;
61
+ const RELEASE_FRAMES_NEEDED = 3;
62
+
63
+ // State tracking to prevent runaway word repeat without clear user intent
64
+ let lastCommittedWord = '';
65
+
66
+ // Trained Neural Network model weights (kept for OFFLINE fallback only)
67
+ let aslWeights = null;
68
+ let islWeights = null;
69
+ let currentWeights = null;
70
+
71
+ // โ”€โ”€ Server-side inference state โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
72
+ // When Flask is running, all classification is done server-side (predict_proba
73
+ // on the sklearn model) for feature parity with run_native.py.
74
+ // If the server is unreachable, we fall back to the JS MLP.
75
+ let serverAvailable = false; // set true after first successful /health probe
76
+ let serverCheckPending = false;
77
+
78
+ // โ”€โ”€ DOM โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
79
+ const video = document.getElementById('webcam-video');
80
+ const displayCanvas = document.getElementById('display-canvas');
81
+ const dctx = displayCanvas.getContext('2d');
82
+ const noHandOverlay = document.getElementById('no-hand-overlay');
83
+ const detectedLetter = document.getElementById('detected-letter');
84
+ const detectedConf = document.getElementById('detected-conf');
85
+ const streakBar = document.getElementById('streak-bar');
86
+ const streakLabel = document.getElementById('streak-label');
87
+ const handBadge = document.getElementById('hand-badge');
88
+ const statusPill = document.getElementById('status-pill');
89
+ const statusText = document.getElementById('status-text');
90
+ const wordDisplay = document.getElementById('word-display');
91
+ const sentenceDisplay = document.getElementById('sentence-display');
92
+ const wordChips = document.getElementById('word-chips');
93
+ const modeToggle = document.getElementById('mode-toggle');
94
+ const toast = document.getElementById('toast');
95
+
96
+ // โ”€โ”€ Utilities โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€๏ฟฝ๏ฟฝ๏ฟฝโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
97
+ function showToast(msg, ms = 2000) {
98
+ toast.textContent = msg;
99
+ toast.classList.add('show');
100
+ setTimeout(() => toast.classList.remove('show'), ms);
101
+ }
102
+ function setStatus(cls, msg) {
103
+ statusPill.className = `status-pill ${cls}`;
104
+ statusText.textContent = msg;
105
+ }
106
+ function setStreakVisual(pct) {
107
+ streakBar.style.width = `${pct}%`;
108
+ // .dial-ring reads --streak as a registered <number> and interpolates it.
109
+ document.documentElement.style.setProperty('--streak', pct);
110
+ }
111
+ function resetStreak() {
112
+ streakLetter = null; streakCount = 0;
113
+ setStreakVisual(0);
114
+ streakLabel.textContent = `0 / ${STREAK_NEEDED}`;
115
+ }
116
+
117
+ // โ”€โ”€ Word / Sentence โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
118
+ function renderWord() {
119
+ wordDisplay.innerHTML = currentWord
120
+ ? currentWord
121
+ : '<span class="ph">Start signingโ€ฆ</span>';
122
+ }
123
+ function renderSentence() {
124
+ sentenceDisplay.innerHTML = sentenceWords.length
125
+ ? sentenceWords.join(' ')
126
+ : '<em class="ph">Your sentence will appear hereโ€ฆ</em>';
127
+ wordChips.innerHTML = '';
128
+ sentenceWords.forEach(w => {
129
+ const c = document.createElement('span');
130
+ c.className = 'word-chip'; c.textContent = w;
131
+ wordChips.appendChild(c);
132
+ });
133
+ }
134
+ async function pushWord() {
135
+ let w = currentWord.trim();
136
+ if (w) {
137
+ // Attempt spellcheck via backend API if available
138
+ try {
139
+ const res = await fetch('/spellcheck', {
140
+ method: 'POST',
141
+ headers: {'Content-Type': 'application/json'},
142
+ body: JSON.stringify({ word: w, confidences: letterConfidences })
143
+ });
144
+ if (res.ok) {
145
+ const data = await res.json();
146
+ if (data.corrected) {
147
+ w = data.corrected.toUpperCase();
148
+ }
149
+ }
150
+ } catch (e) {
151
+ // Backend not present or offline, keep raw word
152
+ }
153
+
154
+ // Gating: Don't repeat the exact same word back-to-back if triggered by background jitter
155
+ const isDuplicateAutoPush = (sentenceWords.length > 0 && sentenceWords[sentenceWords.length - 1] === w && lastCommittedWord === w);
156
+ if (!isDuplicateAutoPush) {
157
+ sentenceWords.push(w);
158
+ lastCommittedWord = w;
159
+ renderSentence();
160
+ showToast(`โœ“ "${w}"`);
161
+ }
162
+ }
163
+ currentWord = '';
164
+ letterConfidences = [];
165
+ renderWord();
166
+ resetStreak();
167
+ noHandCount = 0;
168
+ lastCapturedLetter = '';
169
+ releasedSinceLastCapture = true;
170
+ releaseFrames = 0;
171
+ }
172
+
173
+ // โ”€โ”€ Letter capture โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
174
+ function captureLetter(letter, confidence = 0.85) {
175
+ if (inCooldown) return;
176
+ const l = letter.toLowerCase();
177
+ if (l === 'space' || l === '_') { pushWord(); return; }
178
+ if (l === 'del' || l === 'delete') {
179
+ currentWord = currentWord.slice(0, -1);
180
+ letterConfidences.pop();
181
+ renderWord();
182
+ resetStreak();
183
+ lastCapturedLetter = '';
184
+ releasedSinceLastCapture = true;
185
+ return;
186
+ }
187
+ if (l === 'nothing') return;
188
+
189
+ currentWord += letter.toUpperCase();
190
+ letterConfidences.push(confidence);
191
+ renderWord();
192
+
193
+ lastCapturedLetter = letter.toUpperCase();
194
+ releasedSinceLastCapture = false;
195
+ releaseFrames = 0;
196
+
197
+ detectedLetter.classList.remove('flash');
198
+ void detectedLetter.offsetWidth;
199
+ detectedLetter.classList.add('flash');
200
+
201
+ resetStreak();
202
+ noHandCount = 0;
203
+ inCooldown = true;
204
+ setTimeout(() => { inCooldown = false; }, COOLDOWN_MS);
205
+ }
206
+
207
+ // โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
208
+ // โ”€โ”€ Landmark Neural Network Classifier โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
209
+ // โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
210
+
211
+ /** 15 finger flexion triplets (vertex is the middle index) */
212
+ const ANGLE_TRIPLETS = [
213
+ [0,1,2], [1,2,3], [2,3,4],
214
+ [0,5,6], [5,6,7], [6,7,8],
215
+ [0,9,10], [9,10,11], [10,11,12],
216
+ [0,13,14], [13,14,15], [14,15,16],
217
+ [0,17,18], [17,18,19], [18,19,20],
218
+ ];
219
+
220
+ /** Angle at vertex b formed by a-b-c, in radians normalized to [0,1] */
221
+ function calculateAngle3D(a, b, c) {
222
+ const bax = a[0]-b[0], bay = a[1]-b[1], baz = a[2]-b[2];
223
+ const bcx = c[0]-b[0], bcy = c[1]-b[1], bcz = c[2]-b[2];
224
+ const dot = bax*bcx + bay*bcy + baz*bcz;
225
+ const nba = Math.sqrt(bax*bax + bay*bay + baz*baz);
226
+ const nbc = Math.sqrt(bcx*bcx + bcy*bcy + bcz*bcz);
227
+ const cos = Math.min(1, Math.max(-1, dot / (nba*nbc + 1e-6)));
228
+ return Math.acos(cos) / Math.PI;
229
+ }
230
+
231
+ /**
232
+ * Builds the 78-D feature vector the MLP was trained on:
233
+ * 63 aspect-corrected, wrist-centred, palm-scaled, rotation-aligned coords
234
+ * + 15 finger flexion angles.
235
+ *
236
+ * Mirrors train_landmark_classifier.extract_features_from_image() exactly,
237
+ * including its rotation convention. Do not "fix" the rotation here alone --
238
+ * the shipped weights were trained with it (see notes).
239
+ */
240
+ function normalizeLandmarks(lm, w, h, handedness = "Right", applyHandednessMirror = false) {
241
+ w = w || video.videoWidth || 640;
242
+ h = h || video.videoHeight || 480;
243
+
244
+ // Aspect-ratio corrected pixel space: (x*w, y*h, z*w)
245
+ const pts = lm.map(p => [p.x * w, p.y * h, p.z * w]);
246
+
247
+ // Mirror Left hand to match Right hand dataset distribution (ASL single hand)
248
+ if (applyHandednessMirror && handedness === "Left") {
249
+ const wristX = pts[0][0];
250
+ pts.forEach(p => {
251
+ p[0] = wristX - (p[0] - wristX);
252
+ });
253
+ }
254
+
255
+ // Centre on the wrist
256
+ const wrist = pts[0];
257
+ const cen = pts.map(p => [p[0]-wrist[0], p[1]-wrist[1], p[2]-wrist[2]]);
258
+
259
+ // Palm scale = 2D wrist -> middle MCP distance
260
+ let scale = Math.sqrt(cen[9][0]*cen[9][0] + cen[9][1]*cen[9][1]);
261
+ if (scale < 1e-4) scale = 1;
262
+
263
+ // 2D rotation keyed on landmark 9 against the negative Y axis
264
+ const angle = Math.atan2(cen[9][0]/scale, -cen[9][1]/scale);
265
+ const ca = Math.cos(angle), sa = Math.sin(angle);
266
+
267
+ const features = [];
268
+ // ISL uses the mathematically correct rotation matrix [[cos, -sin], [sin, cos]]
269
+ // that aligns landmark 9 to vertical 12 o'clock, cancelling any hand tilt.
270
+ const isIslMode = (typeof currentMode !== 'undefined' && currentMode === 'isl');
271
+ cen.forEach(p => {
272
+ const x = p[0]/scale, y = p[1]/scale;
273
+ if (isIslMode) {
274
+ features.push(x*ca + y*sa);
275
+ features.push(-x*sa + y*ca);
276
+ } else {
277
+ // ASL legacy rotation
278
+ features.push(x*ca - y*sa);
279
+ features.push(x*sa + y*ca);
280
+ }
281
+ features.push(p[2]/scale); // z is scaled but not rotated
282
+ });
283
+
284
+ // Angles come from the raw pixel points, not the normalized ones
285
+ ANGLE_TRIPLETS.forEach(([i, j, k]) =>
286
+ features.push(calculateAngle3D(pts[i], pts[j], pts[k]))
287
+ );
288
+
289
+ return features; // 63 + 15 = 78
290
+ }
291
+
292
+ /** Matrix multiplication helper */
293
+ function matMul(input, weights, bias) {
294
+ const output = [];
295
+ const numOutputs = bias.length;
296
+ const numInputs = input.length;
297
+ for (let j = 0; j < numOutputs; j++) {
298
+ let sum = bias[j];
299
+ for (let i = 0; i < numInputs; i++) {
300
+ sum += input[i] * weights[i][j];
301
+ }
302
+ output.push(sum);
303
+ }
304
+ return output;
305
+ }
306
+
307
+ /** ReLU activation function */
308
+ function relu(arr) {
309
+ return arr.map(x => Math.max(0, x));
310
+ }
311
+
312
+ /** Softmax activation function */
313
+ function softmax(arr) {
314
+ const max = Math.max(...arr);
315
+ const exps = arr.map(x => Math.exp(x - max));
316
+ const sum = exps.reduce((a, b) => a + b, 0);
317
+ return exps.map(x => x / sum);
318
+ }
319
+
320
+ /**
321
+ * Probes /health to find out whether the Flask backend is up.
322
+ * Called once at startup and retried whenever a classify() call fails.
323
+ */
324
+ async function probeServer() {
325
+ if (serverCheckPending) return;
326
+ serverCheckPending = true;
327
+ try {
328
+ const r = await fetch('/health', { signal: AbortSignal.timeout(1500) });
329
+ if (r.ok) {
330
+ if (!serverAvailable) {
331
+ serverAvailable = true;
332
+ console.log('โœ… Flask backend detected โ€” using server-side inference');
333
+ setStatus('ready', 'Connected');
334
+ }
335
+ } else {
336
+ serverAvailable = false;
337
+ }
338
+ } catch {
339
+ serverAvailable = false;
340
+ } finally {
341
+ serverCheckPending = false;
342
+ // If server is currently offline, automatically retry in 2.5s
343
+ if (!serverAvailable && cameraActive) {
344
+ setTimeout(probeServer, 2500);
345
+ }
346
+ }
347
+ }
348
+
349
+ /** Load MLP weights from JSON files (used for OFFLINE fallback only) */
350
+ async function loadLandmarkModel() {
351
+ try {
352
+ const resAsl = await fetch('asl_landmarks_weights.json');
353
+ if (resAsl.ok) {
354
+ aslWeights = await resAsl.json();
355
+ console.log('๐Ÿง  ASL Neural Network loaded (offline fallback)');
356
+ }
357
+ const resIsl = await fetch('isl_landmarks_weights.json');
358
+ if (resIsl.ok) {
359
+ islWeights = await resIsl.json();
360
+ console.log('๐Ÿง  ISL Neural Network loaded (offline fallback)');
361
+ }
362
+ currentWeights = currentMode === 'asl' ? aslWeights : islWeights;
363
+ } catch (err) {
364
+ console.error('Failed to load MLP weights:', err);
365
+ }
366
+ }
367
+
368
+ /**
369
+ * Assembles the 78-D (single-hand) or 156-D (dual-hand) feature vector.
370
+ *
371
+ * Sorting is on the RAW wrist x of the unflipped frame, identical to
372
+ * run_native.extract_landmarks(). If these two ever disagree, every
373
+ * two-handed sign silently decodes to the wrong letter.
374
+ */
375
+ function buildFeatureVector(multiHandLandmarks, isDualMode, multiHandedness, w, h) {
376
+ if (!isDualMode) {
377
+ const handedness = (multiHandedness && multiHandedness[0] && multiHandedness[0].label)
378
+ ? multiHandedness[0].label
379
+ : 'Right';
380
+ // In ASL single hand mode, mirror Left hands to match Right hand dataset
381
+ return normalizeLandmarks(multiHandLandmarks[0], w, h, handedness, true);
382
+ }
383
+
384
+ // Dual mode (ISL):
385
+ // Single-hand ISL model is trained on BOTH Left & Right hands natively,
386
+ // and two-handed classes strictly require both hands.
387
+ // No fragile handedness guessing/mirroring is needed.
388
+ const hands = multiHandLandmarks
389
+ .map(lm => ({ x: lm[0].x, feat: normalizeLandmarks(lm, w, h, 'Right', false) }))
390
+ .sort((a, b) => a.x - b.x);
391
+
392
+ const slot1 = hands[0].feat;
393
+ const slot2 = hands.length > 1 ? hands[1].feat : new Array(78).fill(0);
394
+ return slot1.concat(slot2);
395
+ }
396
+
397
+ // Helper to check if hand landmarks represent a valid, non-ghost hand
398
+ function isValidHand(lm) {
399
+ if (!lm || lm.length < 21) return false;
400
+ const w = displayCanvas.width || 640;
401
+ const h = displayCanvas.height || 480;
402
+
403
+ // Wrist (0) to Middle MCP (9) distance in pixels
404
+ const dx = (lm[9].x - lm[0].x) * w;
405
+ const dy = (lm[9].y - lm[0].y) * h;
406
+ const span = Math.sqrt(dx * dx + dy * dy);
407
+
408
+ // Reject tiny noise clusters / background artifacts
409
+ if (span < MIN_HAND_SPAN) return false;
410
+
411
+ // Check landmark coordinates are reasonably within frame [-0.2, 1.2]
412
+ for (let i = 0; i < lm.length; i++) {
413
+ if (isNaN(lm[i].x) || isNaN(lm[i].y)) return false;
414
+ if (lm[i].x < -0.2 || lm[i].x > 1.2 || lm[i].y < -0.2 || lm[i].y > 1.2) return false;
415
+ }
416
+ return true;
417
+ }
418
+
419
+ /**
420
+ * JS MLP fallback โ€” used ONLY when Flask server is unreachable.
421
+ * Runs the MLP forward pass in-browser using the cached weight JSON.
422
+ */
423
+ function classifyJS(featureVector) {
424
+ if (!currentWeights) return { letter: 'nothing', confidence: 0.0 };
425
+
426
+ const x = featureVector;
427
+ if (x.length !== currentWeights.w0.length) {
428
+ return { letter: 'nothing', confidence: 0.0 };
429
+ }
430
+
431
+ let h = relu(matMul(x, currentWeights.w0, currentWeights.b0));
432
+ h = relu(matMul(h, currentWeights.w1, currentWeights.b1));
433
+ if (currentWeights.w2 && currentWeights.b2) {
434
+ if (currentWeights.w3 && currentWeights.b3) {
435
+ h = relu(matMul(h, currentWeights.w2, currentWeights.b2));
436
+ h = matMul(h, currentWeights.w3, currentWeights.b3);
437
+ } else {
438
+ h = matMul(h, currentWeights.w2, currentWeights.b2);
439
+ }
440
+ }
441
+
442
+ let probs = softmax(h);
443
+
444
+ // โ”€โ”€ ISL Hand-Count Constrained Gating โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
445
+ if (currentMode === 'isl' && x.length === 156) {
446
+ const isOneHand = x.slice(78).every(v => Math.abs(v) < 1e-4);
447
+ const islSingleHand = new Set(['1', '2', '3', '4', '5', '6', '7', '8', '9', 'C', 'I', 'J', 'L', 'O', 'Q', 'U', 'V']);
448
+ for (let i = 0; i < probs.length; i++) {
449
+ const cls = currentWeights.classes[i];
450
+ if (isOneHand ? !islSingleHand.has(cls) : islSingleHand.has(cls)) {
451
+ probs[i] = 0;
452
+ }
453
+ }
454
+ const pSum = probs.reduce((a, b) => a + b, 0);
455
+ if (pSum > 0) {
456
+ probs = probs.map(p => p / pSum);
457
+ }
458
+ }
459
+
460
+ let maxIdx = 0, maxProb = 0, secondProb = 0;
461
+ for (let i = 0; i < probs.length; i++) {
462
+ if (probs[i] > maxProb) { secondProb = maxProb; maxProb = probs[i]; maxIdx = i; }
463
+ else if (probs[i] > secondProb) { secondProb = probs[i]; }
464
+ }
465
+ if ((maxProb - secondProb) < 0.08 && maxProb < 0.85) {
466
+ return { letter: 'nothing', confidence: maxProb };
467
+ }
468
+ return { letter: currentWeights.classes[maxIdx], confidence: maxProb };
469
+ }
470
+
471
+ /**
472
+ * PRIMARY classify() โ€” async.
473
+ *
474
+ * Strategy:
475
+ * 1. Build the feature vector in JS (normalization is correct and fast).
476
+ * 2. POST it to Flask /predict_landmarks โ€” uses sklearn's calibrated
477
+ * predict_proba(), identical to run_native.py. This is why signs that
478
+ * work in native Python also work in the browser.
479
+ * 3. If the server is unavailable, fall back to the JS MLP.
480
+ */
481
+ async function classify(multiHandLandmarks, multiHandedness) {
482
+ if (!multiHandLandmarks || multiHandLandmarks.length === 0) {
483
+ return { letter: 'nothing', confidence: 0.0 };
484
+ }
485
+
486
+ // Determine if active model is dual-hand (156-D) or single-hand (78-D)
487
+ const isDualMode = currentWeights ? currentWeights.w0.length === 156 : false;
488
+
489
+ // Build feature vector with true display/camera aspect ratio
490
+ const cw = (displayCanvas && displayCanvas.width) || (video && video.videoWidth) || 640;
491
+ const ch = (displayCanvas && displayCanvas.height) || (video && video.videoHeight) || 480;
492
+ const featureVector = buildFeatureVector(multiHandLandmarks, isDualMode, multiHandedness, cw, ch);
493
+
494
+ // โ”€โ”€ PATH A: Server-side inference (preferred) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
495
+ if (serverAvailable) {
496
+ try {
497
+ const response = await fetch('/predict_landmarks', {
498
+ method: 'POST',
499
+ headers: { 'Content-Type': 'application/json' },
500
+ body: JSON.stringify({ mode: currentMode, landmarks: featureVector }),
501
+ signal: AbortSignal.timeout(1500), // 1500ms resilient timeout per frame
502
+ });
503
+
504
+ if (response.ok) {
505
+ const data = await response.json();
506
+ return {
507
+ letter: data.letter || 'nothing',
508
+ confidence: data.confidence || 0.0,
509
+ top3: data.top3 || [],
510
+ };
511
+ }
512
+
513
+ // Non-200 response โ€” server up but something is wrong
514
+ console.warn('[classify] Server error:', response.status);
515
+ return { letter: 'nothing', confidence: 0.0 };
516
+
517
+ } catch (err) {
518
+ // Timeout or network error โ†’ mark server down and re-probe
519
+ console.warn('[classify] Server unreachable, switching to JS fallback:', err.name);
520
+ serverAvailable = false;
521
+ showToast('โš ๏ธ Server offline โ€” using JS fallback', 3000);
522
+ setStatus('loading', 'JS fallback');
523
+ probeServer(); // async retry in background
524
+ }
525
+ }
526
+
527
+ // โ”€โ”€ PATH B: JS MLP fallback โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
528
+ if (!currentWeights) return { letter: 'nothing', confidence: 0.0 };
529
+ return classifyJS(featureVector);
530
+ }
531
+
532
+ // โ”€โ”€ Process classification result โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
533
+ function processResult(letter, confidence) {
534
+ const conf = Math.round(confidence * 100);
535
+
536
+ detectedLetter.textContent = (letter === 'nothing' || confidence < MIN_CONFIDENCE)
537
+ ? ''
538
+ : letter.toUpperCase();
539
+
540
+ detectedConf.textContent = letter === 'nothing'
541
+ ? 'Unclear gesture'
542
+ : `${conf}% ยท ${letter.toUpperCase()}`;
543
+
544
+ const effective = (confidence >= MIN_CONFIDENCE && letter !== 'nothing') ? letter : null;
545
+
546
+ // Track release for repeating the same letter
547
+ if (!effective || effective !== lastCapturedLetter) {
548
+ releaseFrames++;
549
+ if (releaseFrames >= RELEASE_FRAMES_NEEDED) {
550
+ releasedSinceLastCapture = true;
551
+ }
552
+ } else {
553
+ releaseFrames = 0;
554
+ }
555
+
556
+ if (paused || inCooldown || !effective) {
557
+ if (!effective) resetStreak();
558
+ return;
559
+ }
560
+
561
+ // Prevent spamming the exact same letter repeatedly without releasing hand
562
+ if (effective.toUpperCase() === lastCapturedLetter && !releasedSinceLastCapture) {
563
+ resetStreak();
564
+ return;
565
+ }
566
+
567
+ noHandCount = 0;
568
+
569
+ if (effective === streakLetter) {
570
+ streakCount++;
571
+ } else {
572
+ streakLetter = effective;
573
+ streakCount = 1;
574
+ }
575
+
576
+ const pct = Math.min(100, Math.round((streakCount / STREAK_NEEDED) * 100));
577
+ setStreakVisual(pct);
578
+ streakLabel.textContent = `${Math.min(streakCount, STREAK_NEEDED)} / ${STREAK_NEEDED}`;
579
+
580
+ if (streakCount >= STREAK_NEEDED) {
581
+ captureLetter(streakLetter, confidence);
582
+ }
583
+ }
584
+
585
+ // (handLandmarker and animFrameId declared near initMediaPipe below)
586
+
587
+ // Prevent concurrent classify calls from piling up while a fetch is in flight
588
+ let classifyInFlight = false;
589
+
590
+ async function onResults(results) {
591
+ dctx.save();
592
+ dctx.clearRect(0, 0, displayCanvas.width, displayCanvas.height);
593
+ dctx.drawImage(results.image, 0, 0, displayCanvas.width, displayCanvas.height);
594
+
595
+ // Filter for genuine hands with valid size and geometry
596
+ const validHands = (results.multiHandLandmarks || []).filter(isValidHand);
597
+
598
+ if (validHands.length > 0) {
599
+ noHandOverlay.classList.remove('visible');
600
+ handBadge.className = 'hand-badge live';
601
+ if (currentMode === 'isl') {
602
+ handBadge.textContent = validHands.length > 1 ? '2 HANDS ยท ISL (A-Z)' : '1 HAND ยท ISL (1-9, C, I, L, O, U, V)';
603
+ } else {
604
+ handBadge.textContent = validHands.length > 1 ? '2 HANDS DETECTED' : 'HAND DETECTED';
605
+ }
606
+ validHands.forEach(drawSkeleton);
607
+
608
+ // Skip this frame if the previous classify() hasn't returned yet
609
+ // (avoids a queue of stale requests building up on slow networks)
610
+ if (!classifyInFlight) {
611
+ classifyInFlight = true;
612
+ try {
613
+ const res = await classify(validHands, results.multiHandedness);
614
+ processResult(res.letter, res.confidence);
615
+ } finally {
616
+ classifyInFlight = false;
617
+ }
618
+ }
619
+
620
+ } else {
621
+ noHandOverlay.classList.add('visible');
622
+ handBadge.className = 'hand-badge';
623
+ handBadge.textContent = 'NO HAND';
624
+ detectedLetter.textContent = '';
625
+ detectedConf.textContent = 'Waitingโ€ฆ';
626
+ resetStreak();
627
+
628
+ // Hand was dropped/removed -> immediately grant release permission for next letter
629
+ releaseFrames++;
630
+ if (releaseFrames >= RELEASE_FRAMES_NEEDED) {
631
+ releasedSinceLastCapture = true;
632
+ }
633
+
634
+ if (currentWord) {
635
+ noHandCount++;
636
+ if (noHandCount >= SPACE_FRAMES) {
637
+ pushWord();
638
+ }
639
+ }
640
+ }
641
+ dctx.restore();
642
+ }
643
+
644
+ function drawSkeleton(lm) {
645
+ const w = displayCanvas.width;
646
+ const h = displayCanvas.height;
647
+
648
+ // Draw lines
649
+ CONNECTIONS.forEach(([i, j], idx) => {
650
+ const p1 = lm[i];
651
+ const p2 = lm[j];
652
+ dctx.beginPath();
653
+ dctx.moveTo(p1.x * w, p1.y * h);
654
+ dctx.lineTo(p2.x * w, p2.y * h);
655
+ dctx.strokeStyle = EDGE_COLORS[idx] || '#6a6a76'; // arrays are aligned; neutral if ever not
656
+ dctx.lineWidth = 3;
657
+ dctx.lineCap = 'round';
658
+ dctx.stroke();
659
+ });
660
+
661
+ // Draw points
662
+ lm.forEach((p, i) => {
663
+ const x = p.x * w;
664
+ const y = p.y * h;
665
+ dctx.beginPath();
666
+ dctx.arc(x, y, i === 0 ? 7 : 5, 0, 2 * Math.PI);
667
+ dctx.fillStyle = NODE_COLORS[i] || '#ffffff';
668
+ dctx.fill();
669
+ dctx.strokeStyle = '#ffffff';
670
+ dctx.lineWidth = 1.5;
671
+ dctx.stroke();
672
+ });
673
+ }
674
+
675
+ /**
676
+ * Renders a styled, in-theme camera failure inside the camera panel.
677
+ * Replaces the native alert() MediaPipe would otherwise raise.
678
+ */
679
+ function showCameraError(err) {
680
+ const name = (err && err.name) ? err.name : 'Error';
681
+ const HINTS = {
682
+ NotReadableError: 'Another application is already using the camera. Close it (video call, another browser tab, your editor\u2019s preview pane) and reload.',
683
+ NotAllowedError: 'Camera permission was denied. Allow camera access for this site, then reload.',
684
+ NotFoundError: 'No camera device was found on this machine.',
685
+ NotSupportedError: 'This browser cannot access a camera on this page. A secure origin (https:// or localhost) is required.',
686
+ OverconstrainedError: 'No camera matches the requested 640\u00d7480 video mode.',
687
+ AbortError: 'The camera was released before it could start. Reload to retry.',
688
+ };
689
+ const msg = HINTS[name] || (err && err.message) || 'The camera could not be started.';
690
+
691
+ noHandOverlay.innerHTML =
692
+ '<div class="cam-error-icon" aria-hidden="true">\u26A0</div>' +
693
+ '<div class="cam-error-title">Camera unavailable</div>' +
694
+ '<div class="cam-error-msg"></div>' +
695
+ '<div class="cam-error-code"></div>';
696
+ // textContent, not innerHTML -- the error string is not ours to trust as markup
697
+ noHandOverlay.querySelector('.cam-error-msg').textContent = msg;
698
+ noHandOverlay.querySelector('.cam-error-code').textContent = name;
699
+ noHandOverlay.classList.add('error');
700
+ noHandOverlay.classList.remove('hidden');
701
+ noHandOverlay.setAttribute('role', 'alert');
702
+
703
+ setStatus('err', 'Camera unavailable');
704
+ detectedLetter.textContent = '';
705
+ detectedConf.textContent = 'No camera';
706
+ console.error('[SignLens] camera init failed:', err);
707
+ }
708
+
709
+ // โ”€โ”€ MediaPipe Tasks Vision Camera Loop โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
710
+ let handLandmarker = null;
711
+ let animFrameId = null;
712
+
713
+ /**
714
+ * Converts new Tasks Vision HandLandmarker results into the same shape
715
+ * the rest of app.js expects:
716
+ * multiHandLandmarks: Array of [{x,y,z}, ...21] (same as old API)
717
+ * multiHandedness: Array of {label: 'Right'|'Left'} (same field name)
718
+ */
719
+ function adaptResults(result) {
720
+ const rawLandmarks = result.landmarks || [];
721
+ const rawHandedness = result.handedness || [];
722
+
723
+ const multiHandLandmarks = [];
724
+ const multiHandedness = [];
725
+
726
+ for (let i = 0; i < rawLandmarks.length; i++) {
727
+ const lms = rawLandmarks[i];
728
+ if (!lms || lms.length < 21) continue;
729
+
730
+ // Anatomical validation:
731
+ // Rejects facial false positives while fully supporting profile/curled hand signs like 'O', 'C', 'E', 'S'
732
+ const palmLen = Math.hypot(lms[9].x - lms[0].x, lms[9].y - lms[0].y);
733
+ const totalSpan = Math.max(...lms.map(p => Math.hypot(p.x - lms[0].x, p.y - lms[0].y)));
734
+ if (palmLen < 0.03 || totalSpan < 0.06) {
735
+ continue;
736
+ }
737
+
738
+ multiHandLandmarks.push(lms);
739
+ const h = rawHandedness[i];
740
+ multiHandedness.push({
741
+ label: (h && h[0] && h[0].categoryName) ? h[0].categoryName : 'Right'
742
+ });
743
+ }
744
+
745
+ return { multiHandLandmarks, multiHandedness };
746
+ }
747
+
748
+ async function initMediaPipe() {
749
+ // Probe getUserMedia first so we can show a styled error instead of a
750
+ // native alert() on failure.
751
+ try {
752
+ const probe = await navigator.mediaDevices.getUserMedia({
753
+ video: { width: 640, height: 480 }
754
+ });
755
+ probe.getTracks().forEach(t => t.stop());
756
+ } catch (err) {
757
+ showCameraError(err);
758
+ return false;
759
+ }
760
+
761
+ // Dynamically import the Tasks Vision ES-module bundle.
762
+ // More reliable than a <script> UMD tag: exports land directly in scope,
763
+ // no window.* namespace hunting required.
764
+ const { FilesetResolver, HandLandmarker } = await import(
765
+ 'https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@0.10.14/vision_bundle.mjs'
766
+ );
767
+
768
+ // Resolve WASM files from the same CDN version
769
+ const vision = await FilesetResolver.forVisionTasks(
770
+ 'https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@0.10.14/wasm'
771
+ );
772
+
773
+ handLandmarker = await HandLandmarker.createFromOptions(vision, {
774
+ baseOptions: {
775
+ // Same hand_landmarker model as Python's hand_landmarker.task โ€” identical coordinate space
776
+ modelAssetPath: 'https://storage.googleapis.com/mediapipe-models/hand_landmarker/hand_landmarker/float16/1/hand_landmarker.task',
777
+ delegate: 'GPU',
778
+ },
779
+ runningMode: 'VIDEO',
780
+ numHands: 2,
781
+ minHandDetectionConfidence: 0.25,
782
+ minHandPresenceConfidence: 0.25,
783
+ minTrackingConfidence: 0.25,
784
+ });
785
+
786
+ // Start camera stream
787
+ let stream;
788
+ try {
789
+ stream = await navigator.mediaDevices.getUserMedia({
790
+ video: { width: 640, height: 480, facingMode: 'user' }
791
+ });
792
+ } catch (err) {
793
+ showCameraError(err);
794
+ return false;
795
+ }
796
+
797
+ video.srcObject = stream;
798
+ await new Promise(resolve => { video.onloadedmetadata = resolve; });
799
+ video.play();
800
+
801
+ cameraActive = true;
802
+ setStatus('ready', 'Connected');
803
+
804
+ // Detection loop using requestAnimationFrame for smooth 30fps
805
+ let lastVideoTime = -1;
806
+ function detect() {
807
+ if (!cameraActive) { animFrameId = null; return; }
808
+ animFrameId = requestAnimationFrame(detect);
809
+
810
+ if (video.currentTime === lastVideoTime) return; // no new frame yet
811
+ lastVideoTime = video.currentTime;
812
+
813
+ displayCanvas.width = video.videoWidth || 640;
814
+ displayCanvas.height = video.videoHeight || 480;
815
+
816
+ // detectForVideo uses the video timestamp in ms for temporal smoothing
817
+ const result = handLandmarker.detectForVideo(video, performance.now());
818
+ const adapted = adaptResults(result);
819
+
820
+ // Synthesise a results object matching what onResults() already expects
821
+ onResults({
822
+ image: video,
823
+ multiHandLandmarks: adapted.multiHandLandmarks,
824
+ multiHandedness: adapted.multiHandedness,
825
+ });
826
+ }
827
+ detect();
828
+ return true;
829
+ }
830
+
831
+ // โ”€โ”€ Controls Setup โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
832
+ function setupControls() {
833
+ document.getElementById('btn-backspace').addEventListener('click', () => {
834
+ currentWord = currentWord.slice(0, -1);
835
+ letterConfidences.pop();
836
+ renderWord();
837
+ resetStreak();
838
+ });
839
+
840
+ document.getElementById('btn-add-space').addEventListener('click', () => {
841
+ pushWord();
842
+ });
843
+
844
+ document.getElementById('btn-clear-word').addEventListener('click', () => {
845
+ currentWord = '';
846
+ letterConfidences = [];
847
+ renderWord();
848
+ resetStreak();
849
+ });
850
+
851
+ document.getElementById('btn-clear-sentence').addEventListener('click', () => {
852
+ sentenceWords = [];
853
+ currentWord = '';
854
+ letterConfidences = [];
855
+ renderWord();
856
+ renderSentence();
857
+ resetStreak();
858
+ });
859
+
860
+ document.getElementById('btn-pause-detection').addEventListener('click', function() {
861
+ paused = !paused;
862
+ this.textContent = paused ? 'Resume' : 'Pause';
863
+ this.classList.toggle('active-btn', paused);
864
+ });
865
+
866
+ document.getElementById('btn-camera-toggle').addEventListener('click', function() {
867
+ cameraActive = !cameraActive;
868
+ this.textContent = cameraActive ? 'Stop Camera' : 'Start Camera';
869
+ this.classList.toggle('active-btn', !cameraActive);
870
+ if (!cameraActive && animFrameId) {
871
+ cancelAnimationFrame(animFrameId);
872
+ animFrameId = null;
873
+ } else if (cameraActive && !animFrameId && handLandmarker) {
874
+ // restart loop
875
+ let lastVideoTime = -1;
876
+ function detect() {
877
+ if (!cameraActive) { animFrameId = null; return; }
878
+ animFrameId = requestAnimationFrame(detect);
879
+ if (video.currentTime === lastVideoTime) return;
880
+ lastVideoTime = video.currentTime;
881
+ displayCanvas.width = video.videoWidth || 640;
882
+ displayCanvas.height = video.videoHeight || 480;
883
+ const result = handLandmarker.detectForVideo(video, performance.now());
884
+ const adapted = adaptResults(result);
885
+ onResults({ image: video, multiHandLandmarks: adapted.multiHandLandmarks, multiHandedness: adapted.multiHandedness });
886
+ }
887
+ detect();
888
+ }
889
+ });
890
+
891
+ document.getElementById('btn-copy').addEventListener('click', () => {
892
+ if (sentenceWords.length) {
893
+ navigator.clipboard.writeText(sentenceWords.join(' '));
894
+ showToast('๐Ÿ“‹ Copied to clipboard!');
895
+ }
896
+ });
897
+
898
+ // Global reference to prevent Chrome garbage-collection bug on SpeechSynthesisUtterance
899
+ window._activeUtterance = null;
900
+
901
+ function fallbackServerSpeak(text, btn) {
902
+ fetch('/speak', {
903
+ method: 'POST',
904
+ headers: { 'Content-Type': 'application/json' },
905
+ body: JSON.stringify({ text })
906
+ }).then(res => res.json())
907
+ .then(data => {
908
+ console.log('[SignLens] Server TTS spoke:', data);
909
+ })
910
+ .catch(err => console.error('[SignLens] Server TTS failed:', err))
911
+ .finally(() => {
912
+ if (btn) {
913
+ btn.classList.remove('active');
914
+ btn.textContent = 'Speak';
915
+ }
916
+ });
917
+ }
918
+
919
+ function speakSentence() {
920
+ const text = (sentenceWords.length
921
+ ? sentenceWords.join(' ')
922
+ : (sentenceDisplay.innerText || '').replace(/Your sentence will appear here[โ€ฆ\.]*/i, '')
923
+ ).trim();
924
+
925
+ if (!text) {
926
+ showToast('โš ๏ธ No sentence to speak yet');
927
+ return;
928
+ }
929
+
930
+ const btnSpeak = document.getElementById('btn-speak');
931
+ if (btnSpeak) {
932
+ btnSpeak.classList.add('active');
933
+ btnSpeak.textContent = 'Speakingโ€ฆ';
934
+ }
935
+ showToast(`๐Ÿ”Š Speaking: "${text}"`);
936
+
937
+ if ('speechSynthesis' in window) {
938
+ try {
939
+ window.speechSynthesis.cancel(); // Reset stuck Chromium speech queue
940
+
941
+ const u = new SpeechSynthesisUtterance(text);
942
+ window._activeUtterance = u; // Keep reference to prevent V8 GC drop
943
+
944
+ u.lang = 'en-US';
945
+ u.rate = 0.95;
946
+ u.pitch = 1.0;
947
+ u.volume = 1.0;
948
+
949
+ const voices = window.speechSynthesis.getVoices();
950
+ if (voices && voices.length > 0) {
951
+ const enVoice = voices.find(v => v.lang.startsWith('en') && !v.localService) ||
952
+ voices.find(v => v.lang.startsWith('en')) ||
953
+ voices[0];
954
+ if (enVoice) u.voice = enVoice;
955
+ }
956
+
957
+ u.onend = () => {
958
+ window._activeUtterance = null;
959
+ if (btnSpeak) {
960
+ btnSpeak.classList.remove('active');
961
+ btnSpeak.textContent = 'Speak';
962
+ }
963
+ };
964
+
965
+ u.onerror = (e) => {
966
+ console.warn('[SignLens] Browser speech error, falling back to server TTS:', e);
967
+ window._activeUtterance = null;
968
+ fallbackServerSpeak(text, btnSpeak);
969
+ };
970
+
971
+ window.speechSynthesis.speak(u);
972
+
973
+ if (window.speechSynthesis.paused) {
974
+ window.speechSynthesis.resume();
975
+ }
976
+
977
+ // Safety timeout to reset button state after utterance
978
+ setTimeout(() => {
979
+ if (btnSpeak && btnSpeak.textContent === 'Speakingโ€ฆ') {
980
+ btnSpeak.classList.remove('active');
981
+ btnSpeak.textContent = 'Speak';
982
+ }
983
+ }, Math.max(3000, text.split(' ').length * 1200));
984
+
985
+ } catch (err) {
986
+ console.warn('[SignLens] SpeechSynthesis exception:', err);
987
+ fallbackServerSpeak(text, btnSpeak);
988
+ }
989
+ } else {
990
+ fallbackServerSpeak(text, btnSpeak);
991
+ }
992
+ }
993
+
994
+ document.getElementById('btn-speak').addEventListener('click', speakSentence);
995
+
996
+ // Mode buttons
997
+ const btnAsl = document.getElementById('btn-asl');
998
+ const btnIsl = document.getElementById('btn-isl');
999
+ const btnVisually = document.getElementById('btn-visually');
1000
+ const liveWorkspace = document.getElementById('live-workspace');
1001
+ const visuallyWorkspace = document.getElementById('visually-workspace');
1002
+
1003
+ function isInVisuallyMode() {
1004
+ return visuallyWorkspace && !visuallyWorkspace.classList.contains('hidden');
1005
+ }
1006
+
1007
+ btnAsl.addEventListener('click', () => {
1008
+ currentMode = 'asl';
1009
+ currentWeights = aslWeights;
1010
+ btnAsl.classList.add('active');
1011
+ btnIsl.classList.remove('active');
1012
+ if (btnVisually) btnVisually.classList.remove('active');
1013
+ modeToggle.classList.remove('isl-mode', 'visually-mode');
1014
+ if (liveWorkspace) liveWorkspace.classList.remove('hidden');
1015
+ if (visuallyWorkspace) visuallyWorkspace.classList.add('hidden');
1016
+ resetStreak();
1017
+ showToast('Switched to ASL (Live Camera)');
1018
+ });
1019
+
1020
+ btnIsl.addEventListener('click', () => {
1021
+ currentMode = 'isl';
1022
+ currentWeights = islWeights;
1023
+ btnIsl.classList.add('active');
1024
+ btnAsl.classList.remove('active');
1025
+ if (btnVisually) btnVisually.classList.remove('active');
1026
+ modeToggle.classList.remove('visually-mode');
1027
+ modeToggle.classList.add('isl-mode');
1028
+ if (liveWorkspace) liveWorkspace.classList.remove('hidden');
1029
+ if (visuallyWorkspace) visuallyWorkspace.classList.add('hidden');
1030
+ resetStreak();
1031
+ showToast('Switched to ISL (Live Camera)');
1032
+ });
1033
+
1034
+ if (btnVisually) {
1035
+ btnVisually.addEventListener('click', () => {
1036
+ btnVisually.classList.add('active');
1037
+ btnAsl.classList.remove('active');
1038
+ btnIsl.classList.remove('active');
1039
+ modeToggle.classList.remove('isl-mode');
1040
+ modeToggle.classList.add('visually-mode');
1041
+ if (liveWorkspace) liveWorkspace.classList.add('hidden');
1042
+ if (visuallyWorkspace) visuallyWorkspace.classList.remove('hidden');
1043
+ activatePracticeStudio();
1044
+ showToast('Practice Studio Active');
1045
+ });
1046
+ }
1047
+
1048
+ // Settings sheet. Both thresholds are read live by processResult(),
1049
+ // so no restart is needed after a change.
1050
+ const setConf = document.getElementById('set-confidence');
1051
+ const outConf = document.getElementById('out-confidence');
1052
+ const setStreak = document.getElementById('set-streak');
1053
+ const outStreak = document.getElementById('out-streak');
1054
+ const setTimeoutInput = document.getElementById('set-timeout');
1055
+ const outTimeout = document.getElementById('out-timeout');
1056
+
1057
+ setConf.value = Math.round(MIN_CONFIDENCE * 100);
1058
+ outConf.value = `${setConf.value}%`;
1059
+ setStreak.value = STREAK_NEEDED;
1060
+ outStreak.value = STREAK_NEEDED;
1061
+ if (setTimeoutInput && outTimeout) {
1062
+ setTimeoutInput.value = (SPACE_FRAMES / 30).toFixed(1);
1063
+ outTimeout.value = `${setTimeoutInput.value}s`;
1064
+ }
1065
+
1066
+ setConf.addEventListener('input', () => {
1067
+ MIN_CONFIDENCE = Number(setConf.value) / 100;
1068
+ outConf.value = `${setConf.value}%`;
1069
+ });
1070
+
1071
+ setStreak.addEventListener('input', () => {
1072
+ STREAK_NEEDED = Number(setStreak.value);
1073
+ outStreak.value = STREAK_NEEDED;
1074
+ resetStreak(); // rescale the dial against the new target
1075
+ });
1076
+
1077
+ if (setTimeoutInput && outTimeout) {
1078
+ setTimeoutInput.addEventListener('input', () => {
1079
+ const sec = Number(setTimeoutInput.value);
1080
+ SPACE_FRAMES = Math.round(sec * 30);
1081
+ outTimeout.value = `${sec.toFixed(1)}s`;
1082
+ });
1083
+ }
1084
+
1085
+ // Theme Toggle (Light / Dark Mode)
1086
+ const btnThemeToggle = document.getElementById('btn-theme-toggle');
1087
+ const iconThemeToggle = document.getElementById('theme-toggle-icon');
1088
+
1089
+ function applyTheme(theme) {
1090
+ const isLight = theme === 'light';
1091
+ document.documentElement.setAttribute('data-theme', isLight ? 'light' : 'dark');
1092
+ localStorage.setItem('signlens_theme', isLight ? 'light' : 'dark');
1093
+ if (iconThemeToggle) {
1094
+ iconThemeToggle.textContent = isLight ? '๐ŸŒ™' : 'โ˜€๏ธ';
1095
+ }
1096
+ if (btnThemeToggle) {
1097
+ btnThemeToggle.setAttribute('title', isLight ? 'Switch to Dark Mode' : 'Switch to Light Mode');
1098
+ btnThemeToggle.setAttribute('aria-label', isLight ? 'Switch to Dark Mode' : 'Switch to Light Mode');
1099
+ }
1100
+ // Re-render avatar if in practice mode so canvas background updates immediately
1101
+ if (typeof drawAvatar === 'function') {
1102
+ try { drawAvatar(); } catch (_) {}
1103
+ }
1104
+ }
1105
+
1106
+ // Load saved preference or fallback to dark
1107
+ const savedTheme = localStorage.getItem('signlens_theme') || 'dark';
1108
+ applyTheme(savedTheme);
1109
+
1110
+ if (btnThemeToggle) {
1111
+ btnThemeToggle.addEventListener('click', () => {
1112
+ const currentTheme = document.documentElement.getAttribute('data-theme') || 'dark';
1113
+ const nextTheme = currentTheme === 'light' ? 'dark' : 'light';
1114
+ applyTheme(nextTheme);
1115
+ showToast(nextTheme === 'light' ? 'โ˜€๏ธ Switched to Light Mode' : '๐ŸŒ™ Switched to Dark Mode');
1116
+ });
1117
+ }
1118
+
1119
+ // Keyboard shortcuts
1120
+ document.addEventListener('keydown', (e) => {
1121
+ // Ignore shortcuts while a control has focus, or a sheet is open.
1122
+ if (e.target.closest('input, textarea, [popover]')) return;
1123
+
1124
+ if (e.key === ' ' || e.code === 'Space') {
1125
+ e.preventDefault();
1126
+ pushWord();
1127
+ } else if (e.key === 'Backspace') {
1128
+ currentWord = currentWord.slice(0, -1);
1129
+ letterConfidences.pop();
1130
+ renderWord();
1131
+ } else if (e.key === '1') {
1132
+ btnAsl.click();
1133
+ } else if (e.key === '2') {
1134
+ btnIsl.click();
1135
+ } else if (e.key === '3') {
1136
+ if (btnVisually) btnVisually.click();
1137
+ }
1138
+ });
1139
+ }
1140
+
1141
+ // โ”€โ”€ Sign Reference Guide Modal โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
1142
+ // โ”€โ”€ Sign Reference Guide Modal โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
1143
+ function initSignGuide() {
1144
+ const panelGuide = document.getElementById('panel-guide');
1145
+ const btnOpenGuide = document.getElementById('btn-open-guide');
1146
+ const btnCloseGuide = document.getElementById('btn-close-guide');
1147
+ const tabIsl = document.getElementById('tab-guide-isl');
1148
+ const tabAsl = document.getElementById('tab-guide-asl');
1149
+ const contentIsl = document.getElementById('guide-content-isl');
1150
+ const contentAsl = document.getElementById('guide-content-asl');
1151
+ const singleGrid = document.getElementById('isl-single-cards');
1152
+ const dualGrid = document.getElementById('isl-dual-cards');
1153
+
1154
+ const islDescFallback = {
1155
+ '1': 'Index finger extended upwards',
1156
+ '2': 'Index and middle fingers extended in V-shape',
1157
+ '3': 'Thumb, index, and middle fingers extended',
1158
+ '4': 'Four fingers extended upwards',
1159
+ '5': 'Open hand with all five fingers spread',
1160
+ '6': 'Thumb and little finger extended',
1161
+ '7': 'Thumb, index, and little finger extended',
1162
+ '8': 'Middle, ring, and little fingers extended',
1163
+ '9': 'Thumb touching index finger in circle',
1164
+ 'C': 'Curved hand forming C-shape',
1165
+ 'I': 'Little finger extended upwards',
1166
+ 'L': 'Thumb and index finger forming L-shape',
1167
+ 'O': 'All fingers curved touching thumb in O-shape',
1168
+ 'U': 'Index and middle fingers held together vertically',
1169
+ 'V': 'Index and middle fingers in open V-shape',
1170
+ 'A': 'Dominant index finger touching non-dominant thumb tip',
1171
+ 'B': 'Both hands forming double circles / glasses pose',
1172
+ 'D': 'Dominant index pointing to non-dominant index forming D',
1173
+ 'E': 'Left hand curved C/horizontal, right index touching middle knuckle',
1174
+ 'F': 'Dominant index and middle crossing non-dominant index and middle',
1175
+ 'G': 'Both fists held together vertically with knuckles facing forward',
1176
+ 'H': 'Dominant open palm sweeping flat across non-dominant palm',
1177
+ 'J': 'Index finger drawing J stroke across non-dominant palm',
1178
+ 'K': 'Dominant index finger hooking over non-dominant index finger',
1179
+ 'M': 'Dominant three fingers laid across non-dominant palm',
1180
+ 'N': 'Dominant two fingers laid across non-dominant palm',
1181
+ 'P': 'Dominant index finger forming circle with thumb touching non-dominant tip',
1182
+ 'Q': 'Dominant hand forming circle loop with non-dominant index through',
1183
+ 'R': 'Dominant index finger hooked on non-dominant palm',
1184
+ 'S': 'Dominant pinky hooking non-dominant pinky / fist lock',
1185
+ 'T': 'Dominant index finger placed perpendicularly on non-dominant index side',
1186
+ 'W': 'Interlocking open fingers of both hands upright',
1187
+ 'X': 'Crossing both index fingers in an X',
1188
+ 'Y': 'Dominant index pointing between non-dominant thumb and index',
1189
+ 'Z': 'Dominant open palm held upright against non-dominant palm'
1190
+ };
1191
+
1192
+ function createGuideCard(letter, isTwoHanded, desc) {
1193
+ const el = document.createElement('div');
1194
+ el.className = 'guide-card';
1195
+ el.title = `${letter}: ${desc} (click to enlarge)`;
1196
+ el.dataset.letter = letter;
1197
+ el.dataset.hands = isTwoHanded ? '2' : '1';
1198
+ el.dataset.desc = desc;
1199
+ el.dataset.lang = 'ISL';
1200
+ el.dataset.imgSrc = `isl_signs/${letter}.png`;
1201
+ el.innerHTML = `
1202
+ <div class="guide-card-img-wrap">
1203
+ <img src="isl_signs/${letter}.png" alt="Sign ${letter}" loading="lazy" />
1204
+ </div>
1205
+ <div class="guide-card-body">
1206
+ <span class="guide-card-char">${letter}</span>
1207
+ <span class="guide-card-hands ${isTwoHanded ? 'pill-blue' : 'pill-amber'} pill-badge">${isTwoHanded ? '2 Hands' : '1 Hand'}</span>
1208
+ </div>
1209
+ <p class="guide-card-desc" style="padding: 6px 10px 10px; font-size: 0.72rem; color: var(--text-dim); line-height: 1.35; margin: 0;">${desc}</p>
1210
+ `;
1211
+ el.addEventListener('click', () => openSignLightbox(el));
1212
+ return el;
1213
+ }
1214
+
1215
+ function renderGuideCards() {
1216
+ if (!singleGrid || !dualGrid) return;
1217
+ singleGrid.innerHTML = '';
1218
+ dualGrid.innerHTML = '';
1219
+
1220
+ const isl = (practiceTemplates && practiceTemplates.isl) ? practiceTemplates.isl : {};
1221
+ const singleKeys = ['1', '2', '3', '4', '5', '6', '7', '8', '9', 'C', 'I', 'L', 'O', 'U', 'V'];
1222
+ const dualKeys = ['A', 'B', 'D', 'E', 'F', 'G', 'H', 'J', 'K', 'M', 'N', 'P', 'Q', 'R', 'S', 'T', 'W', 'X', 'Y', 'Z'];
1223
+
1224
+ singleKeys.forEach(k => {
1225
+ const desc = (isl[k] && isl[k].desc) ? isl[k].desc : (islDescFallback[k] || 'Single hand sign');
1226
+ singleGrid.appendChild(createGuideCard(k, false, desc));
1227
+ });
1228
+
1229
+ dualKeys.forEach(k => {
1230
+ const desc = (isl[k] && isl[k].desc) ? isl[k].desc : (islDescFallback[k] || 'Dual hand sign');
1231
+ dualGrid.appendChild(createGuideCard(k, true, desc));
1232
+ });
1233
+ }
1234
+
1235
+ const aslDescFallback = {
1236
+ 'A': 'Fist with thumb on side', 'B': 'All fingers together pointing up',
1237
+ 'C': 'Curved hand in C-shape', 'D': 'Index up, fingers curve to thumb',
1238
+ 'E': 'Fingers bent, thumb tucked', 'F': 'Index-thumb circle, three fingers up',
1239
+ 'G': 'Index and thumb point sideways', 'H': 'Two fingers point sideways',
1240
+ 'I': 'Pinky finger up', 'J': 'Pinky draws J in air',
1241
+ 'K': 'Index up, middle angled, thumb out', 'L': 'L-shape โ€” thumb and index',
1242
+ 'M': 'Three fingers over thumb', 'N': 'Two fingers over thumb',
1243
+ 'O': 'All fingers curve to form O', 'P': 'K shape pointing down',
1244
+ 'Q': 'G shape pointing down', 'R': 'Two fingers crossed',
1245
+ 'S': 'Fist with thumb over fingers', 'T': 'Thumb between index and middle',
1246
+ 'U': 'Two fingers together pointing up', 'V': 'Two fingers spread in V',
1247
+ 'W': 'Three fingers spread out', 'X': 'Index finger hooks',
1248
+ 'Y': 'Thumb and pinky out', 'Z': 'Index draws Z in air'
1249
+ };
1250
+
1251
+ function renderAslCards() {
1252
+ const aslGrid = document.getElementById('asl-cards');
1253
+ if (!aslGrid || aslGrid.children.length > 0) return; // render once
1254
+ aslGrid.innerHTML = '';
1255
+ const aslLetters = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'.split('');
1256
+ aslLetters.forEach(letter => {
1257
+ const desc = aslDescFallback[letter] || 'ASL hand sign';
1258
+ const el = document.createElement('div');
1259
+ el.className = 'guide-card';
1260
+ el.title = `${letter}: ${desc} (click to enlarge)`;
1261
+ el.dataset.letter = letter;
1262
+ el.dataset.hands = '1';
1263
+ el.dataset.desc = desc;
1264
+ el.dataset.lang = 'ASL';
1265
+ el.dataset.imgSrc = `asl_signs/${letter}.png`;
1266
+ el.innerHTML = `
1267
+ <div class="guide-card-img-wrap">
1268
+ <img src="asl_signs/${letter}.png" alt="ASL Sign ${letter}" loading="lazy" />
1269
+ </div>
1270
+ <div class="guide-card-body">
1271
+ <span class="guide-card-char">${letter}</span>
1272
+ <span class="guide-card-hands pill-amber pill-badge">1 Hand</span>
1273
+ </div>
1274
+ <p class="guide-card-desc" style="padding: 6px 10px 10px; font-size: 0.72rem; color: var(--text-dim); line-height: 1.35; margin: 0;">${desc}</p>
1275
+ `;
1276
+ el.addEventListener('click', () => openSignLightbox(el));
1277
+ aslGrid.appendChild(el);
1278
+ });
1279
+ }
1280
+
1281
+ function openGuide() {
1282
+ renderGuideCards();
1283
+ renderAslCards();
1284
+ if (panelGuide) {
1285
+ try {
1286
+ if (panelGuide.showPopover) {
1287
+ panelGuide.showPopover();
1288
+ } else {
1289
+ panelGuide.style.display = 'flex';
1290
+ panelGuide.classList.add('open');
1291
+ }
1292
+ } catch (err) {
1293
+ panelGuide.style.display = 'flex';
1294
+ panelGuide.classList.add('open');
1295
+ }
1296
+ }
1297
+ }
1298
+
1299
+ function closeGuide() {
1300
+ if (panelGuide) {
1301
+ try {
1302
+ if (panelGuide.hidePopover) {
1303
+ panelGuide.hidePopover();
1304
+ } else {
1305
+ panelGuide.style.display = 'none';
1306
+ panelGuide.classList.remove('open');
1307
+ }
1308
+ } catch (err) {
1309
+ panelGuide.style.display = 'none';
1310
+ panelGuide.classList.remove('open');
1311
+ }
1312
+ }
1313
+ }
1314
+
1315
+ if (panelGuide) {
1316
+ // Synchronize inline display style with popover state
1317
+ panelGuide.addEventListener('toggle', (e) => {
1318
+ if (e.newState === 'open') {
1319
+ panelGuide.style.display = 'flex';
1320
+ panelGuide.classList.add('open');
1321
+ } else {
1322
+ panelGuide.style.display = 'none';
1323
+ panelGuide.classList.remove('open');
1324
+ }
1325
+ });
1326
+ // Ensure it starts closed and strictly hidden
1327
+ panelGuide.style.display = 'none';
1328
+ panelGuide.classList.remove('open');
1329
+ }
1330
+
1331
+ const stageGuideBtn = document.getElementById('stage-guide-btn');
1332
+ if (stageGuideBtn) {
1333
+ stageGuideBtn.addEventListener('click', (e) => {
1334
+ e.preventDefault();
1335
+ openGuide();
1336
+ });
1337
+ }
1338
+
1339
+ if (btnOpenGuide) {
1340
+ btnOpenGuide.addEventListener('click', (e) => {
1341
+ e.preventDefault();
1342
+ openGuide();
1343
+ });
1344
+ }
1345
+
1346
+ if (btnCloseGuide) {
1347
+ btnCloseGuide.addEventListener('click', (e) => {
1348
+ e.preventDefault();
1349
+ closeGuide();
1350
+ });
1351
+ }
1352
+
1353
+ if (tabIsl && tabAsl && contentIsl && contentAsl) {
1354
+ tabIsl.addEventListener('click', () => {
1355
+ tabIsl.classList.add('active');
1356
+ tabAsl.classList.remove('active');
1357
+ tabIsl.setAttribute('aria-selected', 'true');
1358
+ tabAsl.setAttribute('aria-selected', 'false');
1359
+ contentIsl.classList.remove('hidden');
1360
+ contentAsl.classList.add('hidden');
1361
+ });
1362
+
1363
+ tabAsl.addEventListener('click', () => {
1364
+ tabAsl.classList.add('active');
1365
+ tabIsl.classList.remove('active');
1366
+ tabAsl.setAttribute('aria-selected', 'true');
1367
+ tabIsl.setAttribute('aria-selected', 'false');
1368
+ contentAsl.classList.remove('hidden');
1369
+ contentIsl.classList.add('hidden');
1370
+ });
1371
+ }
1372
+
1373
+ // โ”€โ”€ Sign Lightbox popup โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
1374
+ const lightboxOverlay = document.getElementById('sign-lightbox-overlay');
1375
+ const lightboxImg = document.getElementById('sign-lightbox-img');
1376
+ const lightboxLetter = document.getElementById('sign-lightbox-letter');
1377
+ const lightboxType = document.getElementById('sign-lightbox-type');
1378
+ const lightboxDesc = document.getElementById('sign-lightbox-desc');
1379
+ const lightboxLang = document.getElementById('sign-lightbox-lang');
1380
+ const lightboxCloseBtn = document.getElementById('sign-lightbox-close');
1381
+
1382
+ function openSignLightbox(dataOrEl) {
1383
+ if (!lightboxOverlay) return;
1384
+
1385
+ let letter = '?';
1386
+ let hands = '1';
1387
+ let desc = '';
1388
+ let lang = 'ISL';
1389
+ let imgSrc = '';
1390
+
1391
+ if (dataOrEl instanceof HTMLElement) {
1392
+ letter = dataOrEl.dataset.letter || '?';
1393
+ hands = dataOrEl.dataset.hands || '1';
1394
+ desc = dataOrEl.dataset.desc || '';
1395
+ lang = dataOrEl.dataset.lang || 'ISL';
1396
+ imgSrc = dataOrEl.dataset.imgSrc || '';
1397
+ } else if (typeof dataOrEl === 'object' && dataOrEl !== null) {
1398
+ letter = dataOrEl.letter || '?';
1399
+ hands = String(dataOrEl.hands || '1');
1400
+ desc = dataOrEl.desc || '';
1401
+ lang = dataOrEl.lang || 'ISL';
1402
+ imgSrc = dataOrEl.imgSrc || '';
1403
+ }
1404
+
1405
+ if (lightboxImg) lightboxImg.src = imgSrc;
1406
+ if (lightboxLetter) lightboxLetter.textContent = `Sign: ${letter}`;
1407
+ if (lightboxType) {
1408
+ const isTwoHand = hands === '2';
1409
+ lightboxType.textContent = isTwoHand ? 'โœŒ๏ธ 2 Hands Required' : '๐Ÿคš 1 Hand Only';
1410
+ lightboxType.className = `sign-lightbox-type ${isTwoHand ? 'two-hand' : 'one-hand'}`;
1411
+ }
1412
+ if (lightboxDesc) lightboxDesc.textContent = desc;
1413
+ if (lightboxLang) lightboxLang.textContent = (lang.toUpperCase() === 'ASL') ? '๐Ÿ‡บ๐Ÿ‡ธ American Sign Language (ASL)' : '๐Ÿ‡ฎ๐Ÿ‡ณ Indian Sign Language (ISL)';
1414
+
1415
+ // Open natively in the browser Top Layer (above all popovers and dialogs)
1416
+ if (typeof lightboxOverlay.showModal === 'function') {
1417
+ if (!lightboxOverlay.open) {
1418
+ lightboxOverlay.showModal();
1419
+ }
1420
+ } else {
1421
+ lightboxOverlay.setAttribute('open', '');
1422
+ }
1423
+ }
1424
+
1425
+ function closeSignLightbox() {
1426
+ if (!lightboxOverlay) return;
1427
+ if (typeof lightboxOverlay.close === 'function') {
1428
+ if (lightboxOverlay.open) {
1429
+ lightboxOverlay.close();
1430
+ }
1431
+ } else {
1432
+ lightboxOverlay.removeAttribute('open');
1433
+ }
1434
+ }
1435
+
1436
+ // Make openSignLightbox available globally for card click handlers
1437
+ window.openSignLightbox = openSignLightbox;
1438
+
1439
+ // Close button
1440
+ if (lightboxCloseBtn) {
1441
+ lightboxCloseBtn.addEventListener('click', (e) => {
1442
+ e.preventDefault();
1443
+ e.stopPropagation();
1444
+ closeSignLightbox();
1445
+ });
1446
+ }
1447
+
1448
+ // Click outside (on backdrop) to close dialog
1449
+ if (lightboxOverlay) {
1450
+ lightboxOverlay.addEventListener('click', (e) => {
1451
+ if (e.target === lightboxOverlay) {
1452
+ closeSignLightbox();
1453
+ }
1454
+ });
1455
+ }
1456
+
1457
+ // ESC key natively closes <dialog>, but add safety handler
1458
+ document.addEventListener('keydown', (e) => {
1459
+ if (e.key === 'Escape' && lightboxOverlay && lightboxOverlay.open) {
1460
+ closeSignLightbox();
1461
+ }
1462
+ });
1463
+
1464
+ window._renderGuideCards = renderGuideCards;
1465
+ window._openSignGuide = openGuide;
1466
+ }
1467
+
1468
+ // โ”€โ”€ Startup โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
1469
+ window.addEventListener('DOMContentLoaded', async () => {
1470
+ displayCanvas.width = 640;
1471
+ displayCanvas.height = 480;
1472
+
1473
+ // Wire controls FIRST: a camera failure must not leave every button dead.
1474
+ setupControls();
1475
+ initPracticeStudio();
1476
+ initSignGuide();
1477
+
1478
+ setStatus('loading', 'Loadingโ€ฆ');
1479
+ try {
1480
+ // 1. Probe for Flask backend (async, result used by classify())
1481
+ probeServer();
1482
+
1483
+ // 2. Always load JS MLP weights for offline fallback
1484
+ await loadLandmarkModel();
1485
+
1486
+ // 3. Start camera
1487
+ await initMediaPipe();
1488
+ } catch (err) {
1489
+ showCameraError(err);
1490
+ }
1491
+ });
1492
+
1493
+ // โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
1494
+ // VISUALLY (PRACTICE STUDIO) IMPLEMENTATION
1495
+ // โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
1496
+
1497
+ let practiceTemplates = { asl: {}, isl: {} };
1498
+ let practiceDialect = 'asl'; // 'asl' | 'isl'
1499
+ let practiceSpeed = 1.0; // 0.5 | 1.0 | 1.5
1500
+
1501
+ // Sequence and animation state
1502
+ let practiceSequence = []; // e.g. ['H', 'I', ' ', 'B', 'A', 'B', 'Y']
1503
+ let practiceCurrentIndex = 0;
1504
+ let practiceIsPlaying = false;
1505
+ let practiceStepStartTime = 0;
1506
+ let practiceAnimFrameId = null;
1507
+
1508
+ // Poses for interpolation: arrays of 21 landmarks [x, y, z]
1509
+ let practiceFromPose = null;
1510
+ let practiceToPose = null;
1511
+ let practiceCurrentPose = null;
1512
+
1513
+ // Default neutral resting pose (21 landmarks)
1514
+ const NEUTRAL_REST_POSE = Array(21).fill(0).map((_, i) => {
1515
+ if (i === 0) return [0, 0, 0];
1516
+ const fingerIdx = Math.floor((i - 1) / 4);
1517
+ const jointIdx = ((i - 1) % 4) + 1;
1518
+ const xOffset = (fingerIdx - 2) * 0.18;
1519
+ const yOffset = -0.35 * jointIdx;
1520
+ return [xOffset, yOffset, 0];
1521
+ });
1522
+
1523
+ function clone21(pts) {
1524
+ if (!pts || !Array.isArray(pts)) return null;
1525
+ return pts.map(pt => [pt[0], pt[1], pt[2] || 0]);
1526
+ }
1527
+
1528
+ function normalizePose(raw) {
1529
+ if (!raw) {
1530
+ return {
1531
+ twoHanded: false,
1532
+ left: null,
1533
+ right: clone21(NEUTRAL_REST_POSE),
1534
+ wristL: [0.0, 0.0],
1535
+ wristR: [0.0, 0.0],
1536
+ desc: ''
1537
+ };
1538
+ }
1539
+ if (Array.isArray(raw)) {
1540
+ return {
1541
+ twoHanded: false,
1542
+ left: null,
1543
+ right: clone21(raw),
1544
+ wristL: [0.0, 0.0],
1545
+ wristR: [0.0, 0.0],
1546
+ desc: ''
1547
+ };
1548
+ }
1549
+ return {
1550
+ twoHanded: !!raw.twoHanded,
1551
+ left: raw.left ? clone21(raw.left) : null,
1552
+ right: raw.right ? clone21(raw.right) : clone21(NEUTRAL_REST_POSE),
1553
+ wristL: raw.wristL ? [raw.wristL[0], raw.wristL[1]] : [0.0, 0.0],
1554
+ wristR: raw.wristR ? [raw.wristR[0], raw.wristR[1]] : [0.0, 0.0],
1555
+ desc: raw.desc || ''
1556
+ };
1557
+ }
1558
+
1559
+ function clonePose(pose) {
1560
+ const norm = normalizePose(pose);
1561
+ return {
1562
+ twoHanded: norm.twoHanded,
1563
+ left: clone21(norm.left),
1564
+ right: clone21(norm.right),
1565
+ wristL: [...norm.wristL],
1566
+ wristR: [...norm.wristR],
1567
+ desc: norm.desc
1568
+ };
1569
+ }
1570
+
1571
+ function initPracticeStudio() {
1572
+ const canvas = document.getElementById('avatar-canvas');
1573
+ if (!canvas) return;
1574
+ const ctx = canvas.getContext('2d');
1575
+
1576
+ // 1. Fetch templates
1577
+ fetch('/sign_templates.json?v=' + Date.now(), { cache: 'no-store' })
1578
+ .then(res => {
1579
+ if (!res.ok) throw new Error('HTTP ' + res.status);
1580
+ return res.json();
1581
+ })
1582
+ .then(data => {
1583
+ practiceTemplates = data;
1584
+ console.log('[PracticeStudio] Loaded sign templates:', Object.keys(data.asl || {}).length, 'ASL,', Object.keys(data.isl || {}).length, 'ISL');
1585
+ // Set initial pose
1586
+ practiceCurrentPose = clonePose(getTemplatePose('REST'));
1587
+ drawAvatar();
1588
+ const panelGuide = document.getElementById('panel-guide');
1589
+ if (panelGuide && (panelGuide.matches(':popover-open') || panelGuide.classList.contains('open'))) {
1590
+ if (window._renderGuideCards) window._renderGuideCards();
1591
+ }
1592
+ })
1593
+ .catch(err => {
1594
+ console.warn('[PracticeStudio] Failed to load sign_templates.json:', err);
1595
+ });
1596
+
1597
+ // 2. DOM elements
1598
+ const txtInput = document.getElementById('practice-input');
1599
+ const charCount = document.getElementById('practice-char-count');
1600
+ const btnMic = document.getElementById('btn-mic');
1601
+ const micLabel = document.getElementById('mic-label');
1602
+ const btnSpeakInput = document.getElementById('btn-speak-input');
1603
+ const btnClearPractice = document.getElementById('btn-clear-practice');
1604
+ const btnStartSign = document.getElementById('btn-start-sign');
1605
+ const sequenceChipsContainer = document.getElementById('sequence-chips');
1606
+ const seqCounter = document.getElementById('seq-counter');
1607
+
1608
+ const btnPracAsl = document.getElementById('btn-practice-asl');
1609
+ const btnPracIsl = document.getElementById('btn-practice-isl');
1610
+
1611
+ const hudChar = document.getElementById('hud-letter-char');
1612
+ const hudSub = document.getElementById('hud-letter-sub');
1613
+ const hudCue = document.getElementById('hud-letter-cue');
1614
+ const hudStatus = document.getElementById('avatar-hud-status');
1615
+ const avatarStatusText = document.getElementById('avatar-status-text');
1616
+
1617
+ // Top-Right Sign Gesture Image Preview Reference
1618
+ const signPreviewWrap = document.getElementById('avatar-sign-preview');
1619
+ const signPreviewImg = document.getElementById('avatar-sign-img');
1620
+ const signPreviewBadge = document.getElementById('avatar-sign-badge');
1621
+ const signPreviewChar = document.getElementById('avatar-sign-char');
1622
+ const signPreviewHands = document.getElementById('avatar-sign-hands');
1623
+ const signPreviewDesc = document.getElementById('avatar-sign-desc');
1624
+
1625
+ if (signPreviewWrap) {
1626
+ signPreviewWrap.addEventListener('click', () => {
1627
+ if (signPreviewImg && signPreviewImg.dataset.letter) {
1628
+ if (typeof openSignLightbox === 'function') {
1629
+ openSignLightbox({
1630
+ letter: signPreviewImg.dataset.letter,
1631
+ hands: signPreviewImg.dataset.hands || '1',
1632
+ desc: signPreviewImg.dataset.desc || '',
1633
+ lang: signPreviewImg.dataset.lang || 'ISL',
1634
+ imgSrc: signPreviewImg.src
1635
+ });
1636
+ }
1637
+ }
1638
+ });
1639
+ }
1640
+
1641
+ const btnPrev = document.getElementById('btn-avatar-prev');
1642
+ const btnPlayPause = document.getElementById('btn-avatar-playpause');
1643
+ const btnNext = document.getElementById('btn-avatar-next');
1644
+ const btnReplay = document.getElementById('btn-avatar-replay');
1645
+ const scrubber = document.getElementById('avatar-scrubber');
1646
+
1647
+ // Resize canvas handler
1648
+ function resizeCanvas() {
1649
+ if (!canvas.parentElement) return;
1650
+ const rect = canvas.parentElement.getBoundingClientRect();
1651
+ const dpr = window.devicePixelRatio || 1;
1652
+ const w = Math.floor(rect.width);
1653
+ const h = Math.floor(rect.height);
1654
+ if (w > 0 && h > 0 && (canvas.width !== w * dpr || canvas.height !== h * dpr)) {
1655
+ canvas.width = w * dpr;
1656
+ canvas.height = h * dpr;
1657
+ drawAvatar();
1658
+ }
1659
+ }
1660
+ window.addEventListener('resize', resizeCanvas);
1661
+ setTimeout(resizeCanvas, 100);
1662
+
1663
+ // Get template for a character
1664
+ function getTemplatePose(char) {
1665
+ const dict = practiceTemplates[practiceDialect] || practiceTemplates.asl || {};
1666
+ const upper = String(char).toUpperCase();
1667
+ if (dict[upper]) return normalizePose(dict[upper]);
1668
+ if (dict['REST']) return normalizePose(dict['REST']);
1669
+ return normalizePose(null);
1670
+ }
1671
+
1672
+ const ASL_SAMPLES = ['HI', 'HELLO', 'THANK YOU', 'HOW ARE YOU', 'BABY'];
1673
+ const ISL_SAMPLES = ['NAMASTE', 'A B C', '1 2 3', 'ONAM', 'INDIA'];
1674
+
1675
+ function updateSampleChips(samples) {
1676
+ const chipsContainer = document.querySelector('.quick-samples');
1677
+ if (!chipsContainer) return;
1678
+ const chips = chipsContainer.querySelectorAll('.sample-chip');
1679
+ chips.forEach((chip, idx) => {
1680
+ if (samples[idx] !== undefined) {
1681
+ chip.dataset.text = samples[idx];
1682
+ chip.textContent = samples[idx];
1683
+ chip.style.display = '';
1684
+ } else {
1685
+ chip.style.display = 'none';
1686
+ }
1687
+ });
1688
+ }
1689
+
1690
+ // Dialect switch inside Practice
1691
+ if (btnPracAsl && btnPracIsl) {
1692
+ btnPracAsl.addEventListener('click', () => {
1693
+ practiceDialect = 'asl';
1694
+ btnPracAsl.classList.add('active');
1695
+ btnPracIsl.classList.remove('active');
1696
+ updateSampleChips(ASL_SAMPLES);
1697
+ rebuildSequence(false);
1698
+ showToast('Practice dialect: ASL');
1699
+ });
1700
+ btnPracIsl.addEventListener('click', () => {
1701
+ practiceDialect = 'isl';
1702
+ btnPracIsl.classList.add('active');
1703
+ btnPracAsl.classList.remove('active');
1704
+ updateSampleChips(ISL_SAMPLES);
1705
+ rebuildSequence(false);
1706
+ showToast('Practice dialect: ISL');
1707
+ });
1708
+ }
1709
+
1710
+ // Speed controls
1711
+ document.querySelectorAll('.speed-btn').forEach(btn => {
1712
+ btn.addEventListener('click', () => {
1713
+ document.querySelectorAll('.speed-btn').forEach(b => b.classList.remove('active'));
1714
+ btn.classList.add('active');
1715
+ practiceSpeed = parseFloat(btn.dataset.speed) || 1.0;
1716
+ });
1717
+ });
1718
+
1719
+ // Quick sample chips
1720
+ document.querySelectorAll('.sample-chip').forEach(chip => {
1721
+ chip.addEventListener('click', () => {
1722
+ const text = chip.dataset.text || '';
1723
+ if (txtInput) {
1724
+ txtInput.value = text;
1725
+ updateCharCount();
1726
+ }
1727
+ rebuildSequence(true);
1728
+ });
1729
+ });
1730
+
1731
+ function updateCharCount() {
1732
+ if (!txtInput || !charCount) return;
1733
+ charCount.textContent = `${txtInput.value.length}/200`;
1734
+ }
1735
+
1736
+ if (txtInput) {
1737
+ txtInput.addEventListener('input', () => {
1738
+ updateCharCount();
1739
+ rebuildSequence(false);
1740
+ });
1741
+ txtInput.addEventListener('keydown', (e) => {
1742
+ if (e.key === 'Enter' && !e.shiftKey) {
1743
+ e.preventDefault();
1744
+ rebuildSequence(true);
1745
+ }
1746
+ });
1747
+ }
1748
+
1749
+ if (btnClearPractice) {
1750
+ btnClearPractice.addEventListener('click', () => {
1751
+ if (txtInput) txtInput.value = '';
1752
+ updateCharCount();
1753
+ practiceSequence = [];
1754
+ practiceCurrentIndex = 0;
1755
+ pausePlayback();
1756
+ renderSequenceChips();
1757
+ updateHUD();
1758
+ practiceCurrentPose = clonePose(getTemplatePose('REST'));
1759
+ drawAvatar();
1760
+ });
1761
+ }
1762
+
1763
+ // Voice Speech-to-Text
1764
+ if (btnMic) {
1765
+ let recognition = null;
1766
+ const SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition;
1767
+
1768
+ if (SpeechRecognition) {
1769
+ recognition = new SpeechRecognition();
1770
+ recognition.continuous = false;
1771
+ recognition.interimResults = false;
1772
+ recognition.lang = 'en-US';
1773
+
1774
+ recognition.onstart = () => {
1775
+ btnMic.classList.add('listening');
1776
+ if (micLabel) micLabel.textContent = 'Listeningโ€ฆ';
1777
+ showToast('Speak now โ€” listeningโ€ฆ');
1778
+ };
1779
+
1780
+ recognition.onresult = (event) => {
1781
+ const transcript = event.results[0][0].transcript;
1782
+ if (txtInput) {
1783
+ txtInput.value = transcript.toUpperCase();
1784
+ updateCharCount();
1785
+ rebuildSequence(true);
1786
+ }
1787
+ };
1788
+
1789
+ recognition.onerror = (event) => {
1790
+ console.warn('[PracticeStudio] Speech recognition error:', event.error);
1791
+ btnMic.classList.remove('listening');
1792
+ if (micLabel) micLabel.textContent = 'Speak';
1793
+ showToast('Voice error: ' + event.error);
1794
+ };
1795
+
1796
+ recognition.onend = () => {
1797
+ btnMic.classList.remove('listening');
1798
+ if (micLabel) micLabel.textContent = 'Speak';
1799
+ };
1800
+
1801
+ btnMic.addEventListener('click', () => {
1802
+ try {
1803
+ if (btnMic.classList.contains('listening')) {
1804
+ recognition.stop();
1805
+ } else {
1806
+ recognition.start();
1807
+ }
1808
+ } catch (err) {
1809
+ console.warn('[PracticeStudio] Speech start exception:', err);
1810
+ }
1811
+ });
1812
+ } else {
1813
+ btnMic.addEventListener('click', () => {
1814
+ showToast('Speech recognition not supported in this browser. Please type.');
1815
+ });
1816
+ }
1817
+ }
1818
+
1819
+ // TTS speak aloud
1820
+ if (btnSpeakInput) {
1821
+ btnSpeakInput.addEventListener('click', () => {
1822
+ const text = (txtInput ? txtInput.value.trim() : '');
1823
+ if (!text) {
1824
+ showToast('Type some words to speak aloud.');
1825
+ return;
1826
+ }
1827
+ if ('speechSynthesis' in window) {
1828
+ window.speechSynthesis.cancel();
1829
+ const utter = new SpeechSynthesisUtterance(text);
1830
+ utter.rate = 0.95;
1831
+ window.speechSynthesis.speak(utter);
1832
+ }
1833
+ });
1834
+ }
1835
+
1836
+ // Start Demonstration Button
1837
+ if (btnStartSign) {
1838
+ btnStartSign.addEventListener('click', () => {
1839
+ rebuildSequence(true);
1840
+ });
1841
+ }
1842
+
1843
+ // Parse text into sequence of signs
1844
+ function rebuildSequence(autoPlay = false) {
1845
+ const raw = (txtInput ? txtInput.value : '').toUpperCase();
1846
+ const cleanChars = [];
1847
+ for (let i = 0; i < raw.length; i++) {
1848
+ const ch = raw[i];
1849
+ if ((ch >= 'A' && ch <= 'Z') || (ch >= '0' && ch <= '9')) {
1850
+ cleanChars.push(ch);
1851
+ } else if (ch === ' ' && cleanChars.length > 0 && cleanChars[cleanChars.length - 1] !== ' ') {
1852
+ cleanChars.push(' ');
1853
+ }
1854
+ }
1855
+ // Remove trailing space
1856
+ if (cleanChars.length > 0 && cleanChars[cleanChars.length - 1] === ' ') {
1857
+ cleanChars.pop();
1858
+ }
1859
+
1860
+ practiceSequence = cleanChars;
1861
+ practiceCurrentIndex = 0;
1862
+ renderSequenceChips();
1863
+ updateHUD();
1864
+
1865
+ if (practiceSequence.length === 0) {
1866
+ practiceCurrentPose = clonePose(getTemplatePose('REST'));
1867
+ drawAvatar();
1868
+ return;
1869
+ }
1870
+
1871
+ if (autoPlay) {
1872
+ startPlayback();
1873
+ } else {
1874
+ jumpToStep(0);
1875
+ }
1876
+ }
1877
+
1878
+ function renderSequenceChips() {
1879
+ if (!sequenceChipsContainer) return;
1880
+ sequenceChipsContainer.innerHTML = '';
1881
+
1882
+ if (practiceSequence.length === 0) {
1883
+ sequenceChipsContainer.innerHTML = '<span class="ph">Enter text above to preview gesture timelineโ€ฆ</span>';
1884
+ if (seqCounter) seqCounter.textContent = '0 / 0';
1885
+ if (scrubber) scrubber.value = 0;
1886
+ return;
1887
+ }
1888
+
1889
+ if (seqCounter) {
1890
+ seqCounter.textContent = `${practiceCurrentIndex + 1} / ${practiceSequence.length}`;
1891
+ }
1892
+
1893
+ practiceSequence.forEach((char, idx) => {
1894
+ const chip = document.createElement('button');
1895
+ chip.className = 'sequence-chip' + (char === ' ' ? ' is-space' : '') + (idx === practiceCurrentIndex ? ' active' : '');
1896
+ chip.textContent = (char === ' ' ? 'โฃ' : char);
1897
+ chip.title = (char === ' ' ? 'Space (Rest pause)' : `Letter ${char}`);
1898
+ chip.addEventListener('click', () => {
1899
+ jumpToStep(idx);
1900
+ });
1901
+ sequenceChipsContainer.appendChild(chip);
1902
+ });
1903
+
1904
+ if (scrubber) {
1905
+ const pct = (practiceSequence.length > 1) ? (practiceCurrentIndex / (practiceSequence.length - 1)) * 100 : 0;
1906
+ scrubber.value = pct;
1907
+ }
1908
+ }
1909
+
1910
+ function updateHUD() {
1911
+ if (practiceSequence.length === 0) {
1912
+ if (hudChar) hudChar.textContent = 'โ€”';
1913
+ if (hudSub) hudSub.textContent = 'Waiting for input';
1914
+ if (hudCue) { hudCue.style.display = 'none'; hudCue.textContent = ''; }
1915
+ if (avatarStatusText) avatarStatusText.textContent = 'Ready';
1916
+ if (hudStatus) hudStatus.classList.remove('playing');
1917
+ if (signPreviewWrap) signPreviewWrap.style.display = 'none';
1918
+ return;
1919
+ }
1920
+
1921
+ const currChar = practiceSequence[practiceCurrentIndex];
1922
+ if (hudChar) {
1923
+ hudChar.textContent = (currChar === ' ' ? 'โฃ' : currChar);
1924
+ }
1925
+
1926
+ const currPose = getTemplatePose(currChar === ' ' ? 'REST' : currChar);
1927
+ const is2H = currPose.twoHanded;
1928
+ const desc = currPose.desc;
1929
+
1930
+ if (hudSub) {
1931
+ if (currChar === ' ') {
1932
+ hudSub.textContent = `Word Boundary Pause (${practiceCurrentIndex + 1}/${practiceSequence.length})`;
1933
+ } else {
1934
+ const handTag = (practiceDialect === 'isl') ? (is2H ? ' โ€ข Dual Hand (Two-Handed)' : ' โ€ข Single Hand') : ' โ€ข Single Hand';
1935
+ hudSub.textContent = `${practiceDialect.toUpperCase()} Sign โ€ข Step ${practiceCurrentIndex + 1} of ${practiceSequence.length}${handTag}`;
1936
+ }
1937
+ }
1938
+
1939
+ if (hudCue) {
1940
+ if (desc && currChar !== ' ') {
1941
+ hudCue.style.display = 'inline-flex';
1942
+ hudCue.textContent = `${is2H ? '๐Ÿ‘ Two-Handed: ' : 'โœ‹ '}${desc}`;
1943
+ } else {
1944
+ hudCue.style.display = 'none';
1945
+ hudCue.textContent = '';
1946
+ }
1947
+ }
1948
+
1949
+ if (avatarStatusText) {
1950
+ avatarStatusText.textContent = practiceIsPlaying ? 'Demonstratingโ€ฆ' : 'Paused';
1951
+ }
1952
+ if (hudStatus) {
1953
+ if (practiceIsPlaying) hudStatus.classList.add('playing');
1954
+ else hudStatus.classList.remove('playing');
1955
+ }
1956
+
1957
+ // Update Top-Right Sign Gesture Image Preview Reference
1958
+ if (signPreviewWrap) {
1959
+ if (currChar && currChar !== ' ') {
1960
+ const lang = (practiceDialect || 'isl').toLowerCase();
1961
+ const imgSrc = (lang === 'asl') ? `asl_signs/${currChar}.png` : `isl_signs/${currChar}.png`;
1962
+ if (signPreviewImg) {
1963
+ signPreviewImg.src = imgSrc;
1964
+ signPreviewImg.dataset.letter = currChar;
1965
+ signPreviewImg.dataset.hands = is2H ? '2' : '1';
1966
+ signPreviewImg.dataset.desc = desc || '';
1967
+ signPreviewImg.dataset.lang = lang.toUpperCase();
1968
+ }
1969
+ if (signPreviewBadge) signPreviewBadge.textContent = lang.toUpperCase();
1970
+ if (signPreviewChar) signPreviewChar.textContent = `Sign: ${currChar}`;
1971
+ if (signPreviewHands) signPreviewHands.textContent = is2H ? '2 Hands' : '1 Hand';
1972
+ if (signPreviewDesc) signPreviewDesc.textContent = desc || (is2H ? 'Dual-hand gesture' : 'Single-hand gesture');
1973
+ signPreviewWrap.style.display = 'flex';
1974
+ } else {
1975
+ signPreviewWrap.style.display = 'none';
1976
+ }
1977
+ }
1978
+ }
1979
+
1980
+ function jumpToStep(idx) {
1981
+ if (practiceSequence.length === 0) return;
1982
+ practiceCurrentIndex = Math.max(0, Math.min(idx, practiceSequence.length - 1));
1983
+ const char = practiceSequence[practiceCurrentIndex];
1984
+ practiceFromPose = practiceCurrentPose ? clonePose(practiceCurrentPose) : clonePose(getTemplatePose('REST'));
1985
+ practiceToPose = clonePose(getTemplatePose(char === ' ' ? 'REST' : char));
1986
+ practiceStepStartTime = performance.now();
1987
+ renderSequenceChips();
1988
+ updateHUD();
1989
+
1990
+ // Scroll active chip into view
1991
+ const activeChip = sequenceChipsContainer ? sequenceChipsContainer.children[practiceCurrentIndex] : null;
1992
+ if (activeChip && activeChip.scrollIntoView) {
1993
+ activeChip.scrollIntoView({ behavior: 'smooth', block: 'nearest', inline: 'center' });
1994
+ }
1995
+ }
1996
+
1997
+ function startPlayback() {
1998
+ if (practiceSequence.length === 0) return;
1999
+ practiceIsPlaying = true;
2000
+ if (btnPlayPause) btnPlayPause.textContent = 'โธ';
2001
+ jumpToStep(practiceCurrentIndex);
2002
+ updateHUD();
2003
+ }
2004
+
2005
+ function pausePlayback() {
2006
+ practiceIsPlaying = false;
2007
+ if (btnPlayPause) btnPlayPause.textContent = 'โ–ถ';
2008
+ updateHUD();
2009
+ }
2010
+
2011
+ // Playback control buttons
2012
+ if (btnPlayPause) {
2013
+ btnPlayPause.addEventListener('click', () => {
2014
+ if (practiceIsPlaying) {
2015
+ pausePlayback();
2016
+ } else {
2017
+ if (practiceCurrentIndex >= practiceSequence.length - 1) {
2018
+ practiceCurrentIndex = 0;
2019
+ }
2020
+ startPlayback();
2021
+ }
2022
+ });
2023
+ }
2024
+
2025
+ if (btnPrev) {
2026
+ btnPrev.addEventListener('click', () => {
2027
+ pausePlayback();
2028
+ jumpToStep(practiceCurrentIndex - 1);
2029
+ });
2030
+ }
2031
+
2032
+ if (btnNext) {
2033
+ btnNext.addEventListener('click', () => {
2034
+ pausePlayback();
2035
+ jumpToStep(practiceCurrentIndex + 1);
2036
+ });
2037
+ }
2038
+
2039
+ if (btnReplay) {
2040
+ btnReplay.addEventListener('click', () => {
2041
+ practiceCurrentIndex = 0;
2042
+ startPlayback();
2043
+ });
2044
+ }
2045
+
2046
+ if (scrubber) {
2047
+ scrubber.addEventListener('input', () => {
2048
+ if (practiceSequence.length === 0) return;
2049
+ pausePlayback();
2050
+ const val = parseFloat(scrubber.value);
2051
+ const targetIdx = Math.round((val / 100) * (practiceSequence.length - 1));
2052
+ jumpToStep(targetIdx);
2053
+ });
2054
+ }
2055
+
2056
+ // Interpolation & Render Loop
2057
+ function lerp(a, b, t) {
2058
+ return a + (b - a) * t;
2059
+ }
2060
+
2061
+ function easeInOutCubic(x) {
2062
+ return x < 0.5 ? 4 * x * x * x : 1 - Math.pow(-2 * x + 2, 3) / 2;
2063
+ }
2064
+
2065
+ function avatarAnimationLoop(now) {
2066
+ practiceAnimFrameId = requestAnimationFrame(avatarAnimationLoop);
2067
+
2068
+ // Calculate step timing based on speed
2069
+ const transitionMs = 450 / practiceSpeed;
2070
+ const holdMs = 700 / practiceSpeed;
2071
+ const totalStepMs = transitionMs + holdMs;
2072
+
2073
+ if (practiceToPose && practiceFromPose) {
2074
+ const elapsed = now - practiceStepStartTime;
2075
+ const progress = Math.min(1.0, elapsed / transitionMs);
2076
+ const t = easeInOutCubic(progress);
2077
+
2078
+ if (!practiceCurrentPose) {
2079
+ practiceCurrentPose = clonePose(practiceFromPose);
2080
+ }
2081
+
2082
+ // Interpolate right hand
2083
+ const fromR = practiceFromPose.right || NEUTRAL_REST_POSE;
2084
+ const toR = practiceToPose.right || NEUTRAL_REST_POSE;
2085
+ if (!practiceCurrentPose.right) practiceCurrentPose.right = clone21(fromR);
2086
+ for (let i = 0; i < 21; i++) {
2087
+ practiceCurrentPose.right[i][0] = lerp(fromR[i][0], toR[i][0], t);
2088
+ practiceCurrentPose.right[i][1] = lerp(fromR[i][1], toR[i][1], t);
2089
+ practiceCurrentPose.right[i][2] = lerp(fromR[i][2], toR[i][2], t);
2090
+ }
2091
+
2092
+ // Interpolate left hand
2093
+ const fromL = practiceFromPose.left;
2094
+ const toL = practiceToPose.left;
2095
+ if (fromL || toL) {
2096
+ const effFromL = fromL || NEUTRAL_REST_POSE;
2097
+ const effToL = toL || NEUTRAL_REST_POSE;
2098
+ if (!practiceCurrentPose.left) practiceCurrentPose.left = clone21(effFromL);
2099
+ for (let i = 0; i < 21; i++) {
2100
+ practiceCurrentPose.left[i][0] = lerp(effFromL[i][0], effToL[i][0], t);
2101
+ practiceCurrentPose.left[i][1] = lerp(effFromL[i][1], effToL[i][1], t);
2102
+ practiceCurrentPose.left[i][2] = lerp(effFromL[i][2], effToL[i][2], t);
2103
+ }
2104
+ practiceCurrentPose.twoHanded = (t > 0.5) ? practiceToPose.twoHanded : practiceFromPose.twoHanded;
2105
+ } else {
2106
+ practiceCurrentPose.left = null;
2107
+ practiceCurrentPose.twoHanded = false;
2108
+ }
2109
+
2110
+ // Interpolate wrist offsets
2111
+ practiceCurrentPose.wristL = [
2112
+ lerp(practiceFromPose.wristL ? practiceFromPose.wristL[0] : 0, practiceToPose.wristL ? practiceToPose.wristL[0] : 0, t),
2113
+ lerp(practiceFromPose.wristL ? practiceFromPose.wristL[1] : 0, practiceToPose.wristL ? practiceToPose.wristL[1] : 0, t)
2114
+ ];
2115
+ practiceCurrentPose.wristR = [
2116
+ lerp(practiceFromPose.wristR ? practiceFromPose.wristR[0] : 0, practiceToPose.wristR ? practiceToPose.wristR[0] : 0, t),
2117
+ lerp(practiceFromPose.wristR ? practiceFromPose.wristR[1] : 0, practiceToPose.wristR ? practiceToPose.wristR[1] : 0, t)
2118
+ ];
2119
+ practiceCurrentPose.desc = practiceToPose.desc;
2120
+
2121
+ if (practiceIsPlaying && elapsed >= totalStepMs) {
2122
+ if (practiceCurrentIndex < practiceSequence.length - 1) {
2123
+ jumpToStep(practiceCurrentIndex + 1);
2124
+ } else {
2125
+ // Completed full sequence
2126
+ pausePlayback();
2127
+ if (avatarStatusText) avatarStatusText.textContent = 'Complete';
2128
+ }
2129
+ }
2130
+ }
2131
+
2132
+ drawAvatar();
2133
+ }
2134
+
2135
+ // Draw 21-landmark hand skeleton with 5 distinct finger colors, palm metacarpals, and pearl beads
2136
+ function drawHandSkeleton(ctx, pose21, wristX, wristY, dpr, scale, isLeft) {
2137
+ if (!pose21 || pose21.length !== 21) return;
2138
+
2139
+ // Project landmarks to screen coordinates
2140
+ const pts = pose21.map(pt => {
2141
+ const lx = pt[0];
2142
+ const ly = pt[1];
2143
+ const lz = pt[2] || 0;
2144
+ const depth = 1.0 / (1.0 - lz * 0.18);
2145
+ return [
2146
+ wristX + lx * scale * depth,
2147
+ wristY + ly * scale * depth,
2148
+ lz
2149
+ ];
2150
+ });
2151
+
2152
+ // Palm Metacarpal Structure (cool titanium slate connecting wrist to the 4 knuckles)
2153
+ const palmCol = 'rgba(145, 160, 175, 0.85)';
2154
+ ctx.strokeStyle = palmCol;
2155
+ ctx.lineWidth = 3.5 * dpr;
2156
+
2157
+ // Metacarpal rays: wrist to MCPs (5, 9, 13, 17)
2158
+ [5, 9, 13, 17].forEach(mcp => {
2159
+ ctx.beginPath();
2160
+ ctx.moveTo(wristX, wristY);
2161
+ ctx.lineTo(pts[mcp][0], pts[mcp][1]);
2162
+ ctx.stroke();
2163
+ });
2164
+
2165
+ // Knuckle transverse arch: connecting MCP 5 -> 9 -> 13 -> 17
2166
+ ctx.lineWidth = 4.5 * dpr;
2167
+ ctx.beginPath();
2168
+ ctx.moveTo(pts[5][0], pts[5][1]);
2169
+ ctx.lineTo(pts[9][0], pts[9][1]);
2170
+ ctx.lineTo(pts[13][0], pts[13][1]);
2171
+ ctx.lineTo(pts[17][0], pts[17][1]);
2172
+ ctx.stroke();
2173
+
2174
+ // Finger Segments with exact reference colors:
2175
+ // Thumb: Coral / Red-Pink (wrist 0 -> 1 -> 2 -> 3 -> 4)
2176
+ // Index: Deep Electric Blue (5 -> 6 -> 7 -> 8)
2177
+ // Middle: Bright Emerald Green (9 -> 10 -> 11 -> 12)
2178
+ // Ring: Vibrant Cyan (13 -> 14 -> 15 -> 16)
2179
+ // Pinky: Warm Orange (17 -> 18 -> 19 -> 20)
2180
+ const fingerGroups = [
2181
+ { indices: [0, 1, 2, 3, 4], color: '#ff4d4d', width: 6.0 * dpr }, // Thumb
2182
+ { indices: [5, 6, 7, 8], color: '#2979ff', width: 6.0 * dpr }, // Index
2183
+ { indices: [9, 10, 11, 12], color: '#00e676', width: 6.0 * dpr }, // Middle
2184
+ { indices: [13, 14, 15, 16], color: '#00e5ff', width: 6.0 * dpr }, // Ring
2185
+ { indices: [17, 18, 19, 20], color: '#ff9800', width: 6.0 * dpr }, // Pinky
2186
+ ];
2187
+
2188
+ fingerGroups.forEach(group => {
2189
+ ctx.strokeStyle = group.color;
2190
+ ctx.lineWidth = group.width;
2191
+ for (let k = 0; k < group.indices.length - 1; k++) {
2192
+ const i1 = group.indices[k];
2193
+ const i2 = group.indices[k + 1];
2194
+ ctx.beginPath();
2195
+ ctx.moveTo(pts[i1][0], pts[i1][1]);
2196
+ ctx.lineTo(pts[i2][0], pts[i2][1]);
2197
+ ctx.stroke();
2198
+ }
2199
+ });
2200
+
2201
+ // Joint caps: Clean pearl white dots
2202
+ ctx.fillStyle = '#ffffff';
2203
+ pts.forEach((pt, idx) => {
2204
+ if (idx === 0) return; // Skip wrist, drawn specially below
2205
+ const radius = (idx % 4 === 0) ? 2.5 * dpr : 2.0 * dpr;
2206
+ ctx.beginPath();
2207
+ ctx.arc(pt[0], pt[1], radius, 0, Math.PI * 2);
2208
+ ctx.fill();
2209
+ });
2210
+
2211
+ // Wrist Joint: Clean pearl white bead
2212
+ ctx.fillStyle = '#ffffff';
2213
+ ctx.beginPath();
2214
+ ctx.arc(wristX, wristY, 5 * dpr, 0, Math.PI * 2);
2215
+ ctx.fill();
2216
+ }
2217
+
2218
+ // Draw Avatar Canvas
2219
+ function drawAvatar() {
2220
+ if (!canvas || !ctx) return;
2221
+ const dpr = window.devicePixelRatio || 1;
2222
+ const w = canvas.width;
2223
+ const h = canvas.height;
2224
+
2225
+ ctx.save();
2226
+ ctx.clearRect(0, 0, w, h);
2227
+
2228
+ // 1. Studio background (theme-aware)
2229
+ const isLight = document.documentElement.getAttribute('data-theme') === 'light';
2230
+ ctx.fillStyle = isLight ? '#eaedf4' : '#161619';
2231
+ ctx.fillRect(0, 0, w, h);
2232
+
2233
+ // 2. Stylized Red Silhouette Avatar (Head, Face, Torso, Arms - matching RyloTranslate)
2234
+ const red = '#d82222';
2235
+ ctx.strokeStyle = red;
2236
+ ctx.fillStyle = red;
2237
+ ctx.lineCap = 'round';
2238
+ ctx.lineJoin = 'round';
2239
+ ctx.shadowBlur = 0;
2240
+
2241
+ // Shoulder & torso anchor points
2242
+ const shoulderY = h * 0.48;
2243
+ const leftShoulderX = w * 0.33;
2244
+ const rightShoulderX = w * 0.71;
2245
+
2246
+ // Head dimensions
2247
+ const headCx = w * 0.51;
2248
+ const headCy = h * 0.25;
2249
+ const headRx = w * 0.082;
2250
+ const headRy = h * 0.125;
2251
+
2252
+ // Head contour
2253
+ ctx.lineWidth = 5 * dpr;
2254
+ ctx.beginPath();
2255
+ ctx.ellipse(headCx, headCy, headRx, headRy, 0, 0, Math.PI * 2);
2256
+ ctx.stroke();
2257
+
2258
+ // Eyebrows (bold expressive arches)
2259
+ ctx.lineWidth = 7 * dpr;
2260
+ ctx.beginPath();
2261
+ ctx.arc(headCx - headRx * 0.44, headCy - headRy * 0.35, headRx * 0.34, Math.PI * 1.1, Math.PI * 1.9, false);
2262
+ ctx.stroke();
2263
+ ctx.beginPath();
2264
+ ctx.arc(headCx + headRx * 0.44, headCy - headRy * 0.35, headRx * 0.34, Math.PI * 1.1, Math.PI * 1.9, false);
2265
+ ctx.stroke();
2266
+
2267
+ // Eyes (almond shaped contours)
2268
+ ctx.lineWidth = 3.5 * dpr;
2269
+ ctx.beginPath();
2270
+ ctx.ellipse(headCx - headRx * 0.42, headCy - headRy * 0.12, headRx * 0.24, headRy * 0.12, 0, 0, Math.PI * 2);
2271
+ ctx.stroke();
2272
+ ctx.beginPath();
2273
+ ctx.ellipse(headCx + headRx * 0.42, headCy - headRy * 0.12, headRx * 0.24, headRy * 0.12, 0, 0, Math.PI * 2);
2274
+ ctx.stroke();
2275
+
2276
+ // Mouth (expressive open oval)
2277
+ ctx.lineWidth = 4.5 * dpr;
2278
+ ctx.beginPath();
2279
+ ctx.ellipse(headCx, headCy + headRy * 0.38, headRx * 0.32, headRy * 0.16, 0, 0, Math.PI * 2);
2280
+ ctx.stroke();
2281
+
2282
+ // Torso Frame:
2283
+ ctx.lineWidth = 5 * dpr;
2284
+
2285
+ // Clavicle / shoulder horizontal bar
2286
+ ctx.beginPath();
2287
+ ctx.moveTo(leftShoulderX, shoulderY);
2288
+ ctx.lineTo(rightShoulderX, shoulderY);
2289
+ ctx.stroke();
2290
+
2291
+ // Torso body lines (tapering downwards)
2292
+ ctx.beginPath();
2293
+ ctx.moveTo(leftShoulderX, shoulderY);
2294
+ ctx.lineTo(w * 0.39, h * 0.98);
2295
+ ctx.moveTo(rightShoulderX, shoulderY);
2296
+ ctx.lineTo(w * 0.65, h * 0.98);
2297
+ ctx.stroke();
2298
+
2299
+ // Determine dual-hand vs single-hand posture
2300
+ const is2H = practiceCurrentPose && practiceCurrentPose.twoHanded && practiceCurrentPose.left;
2301
+ const handScale = Math.min(w * 0.20, h * 0.20);
2302
+
2303
+ if (is2H) {
2304
+ // TWO-HANDED MODE: Both arms raised into signing space
2305
+ const offLx = (practiceCurrentPose.wristL ? practiceCurrentPose.wristL[0] : 0) * handScale;
2306
+ const offLy = (practiceCurrentPose.wristL ? practiceCurrentPose.wristL[1] : 0) * handScale;
2307
+ const offRx = (practiceCurrentPose.wristR ? practiceCurrentPose.wristR[0] : 0) * handScale;
2308
+ const offRy = (practiceCurrentPose.wristR ? practiceCurrentPose.wristR[1] : 0) * handScale;
2309
+
2310
+ const wristL_X = w * 0.37 + offLx;
2311
+ const wristL_Y = h * 0.65 + offLy;
2312
+ const wristR_X = w * 0.63 + offRx;
2313
+ const wristR_Y = h * 0.65 + offRy;
2314
+
2315
+ const elbowL_X = leftShoulderX - w * 0.07;
2316
+ const elbowL_Y = (shoulderY + wristL_Y) / 2 + h * 0.08;
2317
+ const elbowR_X = rightShoulderX + w * 0.07;
2318
+ const elbowR_Y = (shoulderY + wristR_Y) / 2 + h * 0.08;
2319
+
2320
+ // Left Arm (viewer-left): Shoulder -> Elbow -> Wrist
2321
+ ctx.beginPath();
2322
+ ctx.moveTo(leftShoulderX, shoulderY);
2323
+ ctx.lineTo(elbowL_X, elbowL_Y);
2324
+ ctx.lineTo(wristL_X, wristL_Y);
2325
+ ctx.stroke();
2326
+
2327
+ // Right Arm (viewer-right): Shoulder -> Elbow -> Wrist
2328
+ ctx.beginPath();
2329
+ ctx.moveTo(rightShoulderX, shoulderY);
2330
+ ctx.lineTo(elbowR_X, elbowR_Y);
2331
+ ctx.lineTo(wristR_X, wristR_Y);
2332
+ ctx.stroke();
2333
+
2334
+ // Render both hand skeletons
2335
+ drawHandSkeleton(ctx, practiceCurrentPose.left, wristL_X, wristL_Y, dpr, handScale, true);
2336
+ drawHandSkeleton(ctx, practiceCurrentPose.right, wristR_X, wristR_Y, dpr, handScale, false);
2337
+ } else {
2338
+ // SINGLE-HANDED MODE: Dominant signing arm active, other arm resting naturally
2339
+ const wristX = w * 0.34;
2340
+ const wristY = h * 0.65;
2341
+ const elbowL_X = w * 0.24;
2342
+ const elbowL_Y = h * 0.84;
2343
+
2344
+ // Signing Arm (viewer-left): Shoulder -> Elbow -> Wrist
2345
+ ctx.beginPath();
2346
+ ctx.moveTo(leftShoulderX, shoulderY);
2347
+ ctx.lineTo(elbowL_X, elbowL_Y);
2348
+ ctx.lineTo(wristX, wristY);
2349
+ ctx.stroke();
2350
+
2351
+ // Resting Arm (viewer-right): Shoulder -> Elbow -> Forearm down
2352
+ const elbowR_X = w * 0.76;
2353
+ const elbowR_Y = h * 0.82;
2354
+ ctx.beginPath();
2355
+ ctx.moveTo(rightShoulderX, shoulderY);
2356
+ ctx.lineTo(elbowR_X, elbowR_Y);
2357
+ ctx.lineTo(w * 0.68, h * 0.98);
2358
+ ctx.stroke();
2359
+
2360
+ // Render single hand skeleton
2361
+ const singlePose = practiceCurrentPose ? (practiceCurrentPose.right || practiceCurrentPose.left) : null;
2362
+ if (singlePose) {
2363
+ drawHandSkeleton(ctx, singlePose, wristX, wristY, dpr, handScale, false);
2364
+ }
2365
+ }
2366
+
2367
+ ctx.restore();
2368
+ }
2369
+
2370
+ // Start animation loop
2371
+ requestAnimationFrame(avatarAnimationLoop);
2372
+ }
2373
+
2374
+ function activatePracticeStudio() {
2375
+ const canvas = document.getElementById('avatar-canvas');
2376
+ if (canvas && canvas.parentElement) {
2377
+ const rect = canvas.parentElement.getBoundingClientRect();
2378
+ const dpr = window.devicePixelRatio || 1;
2379
+ if (rect.width > 0 && rect.height > 0) {
2380
+ canvas.width = Math.floor(rect.width) * dpr;
2381
+ canvas.height = Math.floor(rect.height) * dpr;
2382
+ }
2383
+ }
2384
+ }
asl_landmarks_weights.json ADDED
The diff for this file is too large to render. See raw diff
 
asl_signs/A.png ADDED
asl_signs/B.png ADDED
asl_signs/C.png ADDED
asl_signs/D.png ADDED
asl_signs/E.png ADDED
asl_signs/F.png ADDED
asl_signs/G.png ADDED
asl_signs/H.png ADDED
asl_signs/I.png ADDED
asl_signs/J.png ADDED
asl_signs/K.png ADDED
asl_signs/L.png ADDED
asl_signs/M.png ADDED
asl_signs/N.png ADDED
asl_signs/O.png ADDED
asl_signs/P.png ADDED
asl_signs/Q.png ADDED
asl_signs/R.png ADDED
asl_signs/S.png ADDED
asl_signs/T.png ADDED
asl_signs/U.png ADDED
asl_signs/V.png ADDED
asl_signs/W.png ADDED
asl_signs/X.png ADDED
asl_signs/Y.png ADDED
asl_signs/Z.png ADDED
index.html CHANGED
@@ -1,19 +1,393 @@
1
- <!doctype html>
2
- <html>
3
- <head>
4
- <meta charset="utf-8" />
5
- <meta name="viewport" content="width=device-width" />
6
- <title>My static Space</title>
7
- <link rel="stylesheet" href="style.css" />
8
- </head>
9
- <body>
10
- <div class="card">
11
- <h1>Welcome to your static Space!</h1>
12
- <p>You can modify this app directly by editing <i>index.html</i> in the Files and versions tab.</p>
13
- <p>
14
- Also don't forget to check the
15
- <a href="https://huggingface.co/docs/hub/spaces" target="_blank">Spaces documentation</a>.
16
- </p>
17
- </div>
18
- </body>
19
- </html>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <!DOCTYPE html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="UTF-8" />
5
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
6
+ <meta name="color-scheme" content="dark" />
7
+ <title>SignLens</title>
8
+
9
+ <link rel="preconnect" href="https://fonts.googleapis.com" />
10
+ <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
11
+ <link href="https://fonts.googleapis.com/css2?family=Geist:wght@300;400;500;600;700&family=Geist+Mono:wght@400;500;600&display=swap" rel="stylesheet" />
12
+ <link rel="stylesheet" href="style.css?v=14" />
13
+
14
+ <!-- MediaPipe Tasks Vision loaded dynamically inside app.js initMediaPipe() -->
15
+ </head>
16
+ <body>
17
+
18
+ <!-- โ•โ• TOP BAR โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ• -->
19
+ <header class="topbar">
20
+
21
+ <div class="brand">
22
+ <!-- Single geometric mark: three ascending strokes, a hand reading as signal. -->
23
+ <svg class="brand-mark" viewBox="0 0 24 24" aria-hidden="true" focusable="false">
24
+ <rect x="3" y="13" width="4" height="8" rx="2" />
25
+ <rect x="10" y="7" width="4" height="14" rx="2" />
26
+ <rect x="17" y="3" width="4" height="18" rx="2" />
27
+ </svg>
28
+ <span class="brand-name">SignLens</span>
29
+ </div>
30
+
31
+ <div class="mode-toggle" id="mode-toggle" role="group" aria-label="Sign language">
32
+ <button class="mode-btn active" data-mode="asl" id="btn-asl" aria-pressed="true">ASL</button>
33
+ <button class="mode-btn" data-mode="isl" id="btn-isl" aria-pressed="false">ISL</button>
34
+ <button class="mode-btn" data-mode="visually" id="btn-visually" aria-pressed="false">VISUALLY</button>
35
+ <span class="mode-slider" aria-hidden="true"></span>
36
+ </div>
37
+
38
+ <div class="topbar-right">
39
+ <button class="ghost-btn guide-btn" popovertarget="panel-guide" id="btn-open-guide" aria-label="Open Sign Guide">
40
+ <svg width="15" height="15" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" style="vertical-align: -2px; margin-right: 5px;"><path d="M2 3h6a4 4 0 0 1 4 4v14a3 3 0 0 0-3-3H2z"></path><path d="M22 3h-6a4 4 0 0 0-4 4v14a3 3 0 0 1 3-3h7z"></path></svg><span>Sign Guide</span>
41
+ </button>
42
+ <button class="ghost-btn" popovertarget="panel-settings">Settings</button>
43
+ <button class="ghost-btn" popovertarget="panel-help">Shortcuts</button>
44
+ <button class="ghost-btn theme-toggle-btn" id="btn-theme-toggle" aria-label="Toggle Light or Dark Mode" title="Toggle Light / Dark Mode">
45
+ <span id="theme-toggle-icon" aria-hidden="true">โ˜€๏ธ</span>
46
+ </button>
47
+ <div class="status-pill" id="status-pill">
48
+ <span class="status-dot" aria-hidden="true"></span>
49
+ <span id="status-text" role="status" aria-live="polite">Connecting</span>
50
+ </div>
51
+ </div>
52
+ </header>
53
+
54
+ <!-- โ•โ• WORKSPACE (LIVE CAMERA) โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ• -->
55
+ <main class="workspace" id="live-workspace">
56
+
57
+ <!-- โ”€โ”€ Camera stage โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ -->
58
+ <section class="stage" aria-label="Camera">
59
+
60
+ <div class="viewport">
61
+ <video id="webcam-video" autoplay playsinline muted></video>
62
+ <canvas id="display-canvas"></canvas>
63
+
64
+ <div class="no-hand-overlay visible" id="no-hand-overlay">
65
+ <div class="idle-title">Show your hand to the camera</div>
66
+ <div class="idle-sub">Tracking starts as soon as a hand is in frame.</div>
67
+ </div>
68
+ </div>
69
+
70
+ <!-- Live readout: the one thing the user actually watches -->
71
+ <div class="readout">
72
+
73
+ <div class="readout-dial">
74
+ <div class="dial-ring" aria-hidden="true"></div>
75
+ <div class="detected-letter" id="detected-letter" role="status" aria-live="polite" aria-atomic="true"></div>
76
+ </div>
77
+
78
+ <div class="readout-body">
79
+ <div class="readout-line">
80
+ <span class="detected-sub" id="detected-conf">Waiting</span>
81
+ <span class="hand-badge" id="hand-badge" role="status" aria-live="polite">NO HAND</span>
82
+ </div>
83
+
84
+ <div class="streak-row">
85
+ <div class="streak-track"><div class="streak-fill" id="streak-bar"></div></div>
86
+ <span class="streak-label" id="streak-label">0 / 5</span>
87
+ </div>
88
+ </div>
89
+
90
+ <div class="stage-actions">
91
+ <button class="ctrl-btn" popovertarget="panel-guide" id="stage-guide-btn" title="View official sign chart">Guide</button>
92
+ <button class="ctrl-btn" id="btn-pause-detection">Pause</button>
93
+ <button class="ctrl-btn active-btn" id="btn-camera-toggle">Stop Camera</button>
94
+ </div>
95
+ </div>
96
+ </section>
97
+
98
+ <!-- โ”€โ”€ Output rail โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ -->
99
+ <aside class="rail">
100
+
101
+ <article class="panel" id="word-panel">
102
+ <header class="panel-head">
103
+ <h2 class="panel-title">Building word</h2>
104
+ </header>
105
+
106
+ <div class="word-display" id="word-display">
107
+ <span class="ph">Start signing</span>
108
+ </div>
109
+
110
+ <div class="panel-actions">
111
+ <button class="btn" id="btn-backspace">Delete</button>
112
+ <button class="btn" id="btn-add-space">Next word</button>
113
+ <button class="btn btn-quiet" id="btn-clear-word">Clear</button>
114
+ </div>
115
+ </article>
116
+
117
+ <article class="panel panel-grow" id="sentence-panel">
118
+ <header class="panel-head">
119
+ <h2 class="panel-title">Sentence</h2>
120
+ <div class="panel-head-actions">
121
+ <button class="btn btn-sm" id="btn-speak">Speak</button>
122
+ <button class="btn btn-sm" id="btn-copy">Copy</button>
123
+ </div>
124
+ </header>
125
+
126
+ <div class="sentence-display" id="sentence-display" role="status" aria-live="polite">
127
+ <em class="ph">Your sentence will appear here</em>
128
+ </div>
129
+
130
+ <div class="word-chips" id="word-chips"></div>
131
+
132
+ <button class="btn btn-danger" id="btn-clear-sentence">Clear all</button>
133
+ </article>
134
+ </aside>
135
+ </main>
136
+
137
+ <!-- โ•โ• VISUALLY (PRACTICE STUDIO) โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ• -->
138
+ <section class="workspace visually-workspace hidden" id="visually-workspace" aria-label="Practice Studio">
139
+ <!-- Left Column: Input, Speech Dictation, and Sequence Breakdown -->
140
+ <aside class="practice-input-rail">
141
+ <article class="panel panel-grow practice-panel">
142
+ <header class="panel-head">
143
+ <div class="practice-head-title">
144
+ <span class="practice-badge">PRACTICE STUDIO</span>
145
+ <h2 class="panel-title">Text to Sign</h2>
146
+ </div>
147
+ <div class="practice-dialect-toggle" id="practice-dialect-toggle">
148
+ <button class="dialect-pill active" id="btn-practice-asl">ASL</button>
149
+ <button class="dialect-pill" id="btn-practice-isl">ISL</button>
150
+ </div>
151
+ </header>
152
+
153
+ <!-- Textarea with integrated voice mic & actions -->
154
+ <div class="practice-textarea-box">
155
+ <textarea id="practice-input" placeholder="Type words or sentence (e.g. 'HI BABY', 'HOW ARE YOU')..." rows="4"></textarea>
156
+ <div class="practice-input-tools">
157
+ <button class="tool-btn mic-btn" id="btn-mic" title="Voice Input (Speech-to-Sign)">
158
+ <svg class="icon" viewBox="0 0 24 24" fill="currentColor">
159
+ <path d="M12 14c1.66 0 3-1.34 3-3V5c0-1.66-1.34-3-3-3S9 3.34 9 5v6c0 1.66 1.34 3 3 3zm5-3c0 2.76-2.24 5-5 5s-5-2.24-5-5H5c0 3.53 2.61 6.43 6 6.92V21h2v-3.08c3.39-.49 6-3.39 6-6.92h-2z"/>
160
+ </svg>
161
+ <span id="mic-label">Speak</span>
162
+ </button>
163
+ <button class="tool-btn" id="btn-speak-input" title="Speak text aloud">
164
+ <svg class="icon" viewBox="0 0 24 24" fill="currentColor">
165
+ <path d="M3 9v6h4l5 5V4L7 9H3zm13.5 3c0-1.77-1.02-3.29-2.5-4.03v8.05c1.48-.73 2.5-2.25 2.5-4.02zM14 3.23v2.06c2.89.86 5 3.54 5 6.71s-2.11 5.85-5 6.71v2.06c4.01-.91 7-4.49 7-8.77s-2.99-7.86-7-8.77z"/>
166
+ </svg>
167
+ </button>
168
+ <button class="tool-btn" id="btn-clear-practice" title="Clear text">Clear</button>
169
+ <span class="char-count" id="practice-char-count">0/200</span>
170
+ </div>
171
+ </div>
172
+
173
+ <!-- Quick sample prompts -->
174
+ <div class="quick-samples">
175
+ <span class="quick-label">Try:</span>
176
+ <button class="sample-chip" data-text="HI">HI</button>
177
+ <button class="sample-chip" data-text="HELLO">HELLO</button>
178
+ <button class="sample-chip" data-text="THANK YOU">THANK YOU</button>
179
+ <button class="sample-chip" data-text="HOW ARE YOU">HOW ARE YOU</button>
180
+ <button class="sample-chip" data-text="BABY">BABY</button>
181
+ </div>
182
+
183
+ <!-- Main Trigger & Speed -->
184
+ <div class="practice-action-row">
185
+ <button class="btn btn-primary btn-demo" id="btn-start-sign">
186
+ <svg class="icon-play" viewBox="0 0 24 24" fill="currentColor"><path d="M8 5v14l11-7z"/></svg>
187
+ <span>Demonstrate Sign</span>
188
+ </button>
189
+
190
+ <div class="speed-control">
191
+ <span class="speed-label">Speed</span>
192
+ <button class="speed-btn" data-speed="0.5">0.5x</button>
193
+ <button class="speed-btn active" data-speed="1.0">1.0x</button>
194
+ <button class="speed-btn" data-speed="1.5">1.5x</button>
195
+ </div>
196
+ </div>
197
+
198
+ <!-- Letter Tape Progression -->
199
+ <div class="sequence-section">
200
+ <div class="sequence-head">
201
+ <span class="seq-title">Sign Sequence Timeline</span>
202
+ <span class="seq-counter" id="seq-counter">0 / 0</span>
203
+ </div>
204
+ <div class="sequence-chips" id="sequence-chips">
205
+ <span class="ph">Enter text above to preview gesture timelineโ€ฆ</span>
206
+ </div>
207
+ </div>
208
+ </article>
209
+ </aside>
210
+
211
+ <!-- Right Column: Interactive 3D Avatar Skeleton Stage -->
212
+ <section class="practice-stage" aria-label="Avatar Stage">
213
+ <div class="avatar-viewport">
214
+ <canvas id="avatar-canvas"></canvas>
215
+
216
+ <!-- Active Letter Glow Badge -->
217
+ <div class="avatar-hud-letter" id="avatar-hud-letter">
218
+ <span class="hud-letter-char" id="hud-letter-char">โ€”</span>
219
+ <span class="hud-letter-sub" id="hud-letter-sub">Waiting for input</span>
220
+ <span class="hud-letter-cue" id="hud-letter-cue" style="display: none;"></span>
221
+ </div>
222
+
223
+ <!-- Avatar Status Badge -->
224
+ <div class="avatar-hud-status" id="avatar-hud-status">
225
+ <span class="status-badge-dot"></span>
226
+ <span id="avatar-status-text">Ready</span>
227
+ </div>
228
+
229
+ <!-- Real Human Sign Reference Card (Top-Right) -->
230
+ <div class="avatar-sign-preview" id="avatar-sign-preview" title="Official sign gesture photo reference" style="display: none;">
231
+ <div class="avatar-sign-img-wrap">
232
+ <img id="avatar-sign-img" src="" alt="Sign Gesture Reference" />
233
+ <span class="avatar-sign-badge" id="avatar-sign-badge">ISL</span>
234
+ </div>
235
+ <div class="avatar-sign-info">
236
+ <div class="avatar-sign-char-row">
237
+ <span class="avatar-sign-char" id="avatar-sign-char">A</span>
238
+ <span class="avatar-sign-hands" id="avatar-sign-hands">1 Hand</span>
239
+ </div>
240
+ <span class="avatar-sign-desc" id="avatar-sign-desc">Sign gesture reference</span>
241
+ </div>
242
+ </div>
243
+
244
+
245
+ <!-- Playback Controls Floating Bar -->
246
+ <div class="avatar-controls-bar">
247
+ <button class="avatar-ctrl-btn" id="btn-avatar-prev" title="Previous letter">โฎ</button>
248
+ <button class="avatar-ctrl-btn play-pause-btn" id="btn-avatar-playpause" title="Play/Pause">โธ</button>
249
+ <button class="avatar-ctrl-btn" id="btn-avatar-next" title="Next letter">โญ</button>
250
+ <button class="avatar-ctrl-btn" id="btn-avatar-replay" title="Replay all">โŸฒ</button>
251
+ <div class="avatar-progress-wrap">
252
+ <input type="range" class="avatar-scrubber" id="avatar-scrubber" min="0" max="100" value="0" step="0.1" />
253
+ </div>
254
+ </div>
255
+ </div>
256
+ </section>
257
+ </section>
258
+
259
+ <!-- โ•โ• SETTINGS (top layer) โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ• -->
260
+ <div popover id="panel-settings" class="sheet" style="display: none;">
261
+ <header class="sheet-head">
262
+ <h2 class="sheet-title">Detection</h2>
263
+ <button class="ghost-btn" popovertarget="panel-settings" popovertargetaction="hide">Close</button>
264
+ </header>
265
+
266
+ <div class="field">
267
+ <label for="set-confidence">Minimum confidence</label>
268
+ <div class="field-row">
269
+ <input type="range" id="set-confidence" min="40" max="95" step="5" value="50" />
270
+ <output class="field-value" id="out-confidence">50%</output>
271
+ </div>
272
+ <p class="field-help">Predictions below this are ignored. Raise it if letters are captured too eagerly.</p>
273
+ </div>
274
+
275
+ <div class="field">
276
+ <label for="set-streak">Frames to confirm</label>
277
+ <div class="field-row">
278
+ <input type="range" id="set-streak" min="3" max="15" step="1" value="5" />
279
+ <output class="field-value" id="out-streak">5</output>
280
+ </div>
281
+ <p class="field-help">How long the same sign must hold before the letter is captured.</p>
282
+ </div>
283
+
284
+ <div class="field">
285
+ <label for="set-timeout">Auto-push word delay</label>
286
+ <div class="field-row">
287
+ <input type="range" id="set-timeout" min="1.0" max="5.0" step="0.5" value="3.0" />
288
+ <output class="field-value" id="out-timeout">3.0s</output>
289
+ </div>
290
+ <p class="field-help">How long you must lower your hand before the word commits to sentence.</p>
291
+ </div>
292
+ </div>
293
+
294
+ <!-- โ•โ• SHORTCUTS (top layer) โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ• -->
295
+ <div popover id="panel-help" class="sheet" style="display: none;">
296
+ <header class="sheet-head">
297
+ <h2 class="sheet-title">How it works</h2>
298
+ <button class="ghost-btn" popovertarget="panel-help" popovertargetaction="hide">Close</button>
299
+ </header>
300
+
301
+ <ol class="steps">
302
+ <li><span class="step-n">1</span> Show a hand sign to the camera.</li>
303
+ <li><span class="step-n">2</span> Hold it steady while the ring fills.</li>
304
+ <li><span class="step-n">3</span> The letter is captured and added to the word.</li>
305
+ <li><span class="step-n">4</span> Drop your hand to commit the word to the sentence.</li>
306
+ </ol>
307
+
308
+ <div class="keys">
309
+ <div class="key-row"><kbd>Space</kbd><span>Commit current word</span></div>
310
+ <div class="key-row"><kbd>Backspace</kbd><span>Delete last letter</span></div>
311
+ <div class="key-row"><kbd>1</kbd><span>Switch to ASL</span></div>
312
+ <div class="key-row"><kbd>2</kbd><span>Switch to ISL</span></div>
313
+ <div class="key-row"><kbd>3</kbd><span>Switch to Practice (VISUALLY)</span></div>
314
+ </div>
315
+ </div>
316
+
317
+ <!-- โ•โ• SIGN REFERENCE GUIDE (top layer modal) โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ• -->
318
+ <div popover id="panel-guide" class="sheet sheet-wide" style="display: none;">
319
+ <header class="sheet-head">
320
+ <div class="sheet-title-group">
321
+ <h2 class="sheet-title">Sign Reference Guide</h2>
322
+ <span class="sheet-sub">Official hand gestures, required hand counts & poses</span>
323
+ </div>
324
+ <div class="guide-tabs" role="tablist">
325
+ <button class="guide-tab-btn active" id="tab-guide-isl" role="tab" aria-selected="true">๐Ÿ‡ฎ๐Ÿ‡ณ ISL (Indian Sign Language)</button>
326
+ <button class="guide-tab-btn" id="tab-guide-asl" role="tab" aria-selected="false">๐Ÿ‡บ๐Ÿ‡ธ ASL (American Sign Language)</button>
327
+ </div>
328
+ <button class="ghost-btn" id="btn-close-guide" popovertarget="panel-guide" popovertargetaction="hide">Close</button>
329
+ </header>
330
+
331
+ <!-- ISL Tab Content -->
332
+ <div class="guide-tab-content" id="guide-content-isl">
333
+ <div class="guide-alert-banner">
334
+ <span class="banner-icon">โš ๏ธ</span>
335
+ <div>
336
+ <strong>Crucial Rule in ISL:</strong> 20 alphabet letters (<kbd>A, B, D, E, F, G, H, J, K, M, N, P, Q, R, S, T, W, X, Y, Z</kbd>) strictly require <strong>TWO HANDS</strong>. Only numbers <kbd>1-9</kbd> and letters <kbd>C, I, L, O, U, V</kbd> use a single hand!
337
+ </div>
338
+ </div>
339
+
340
+ <div class="guide-section">
341
+ <div class="guide-section-head">
342
+ <h3 class="guide-section-title">Single-Hand Gestures (15 Signs)</h3>
343
+ <span class="pill-badge pill-amber">1-Hand Required</span>
344
+ </div>
345
+ <div class="guide-cards-grid" id="isl-single-cards"></div>
346
+ </div>
347
+
348
+ <div class="guide-section" style="margin-top: 28px;">
349
+ <div class="guide-section-head">
350
+ <h3 class="guide-section-title">Dual-Hand Gestures (20 Signs)</h3>
351
+ <span class="pill-badge pill-blue">2-Hands Required</span>
352
+ </div>
353
+ <div class="guide-cards-grid" id="isl-dual-cards"></div>
354
+ </div>
355
+ </div>
356
+
357
+ <!-- ASL Tab Content -->
358
+ <div class="guide-tab-content hidden" id="guide-content-asl">
359
+ <div class="guide-alert-banner">
360
+ <span class="banner-icon">๐Ÿ’ก</span>
361
+ <div>
362
+ <strong>Rule in ASL:</strong> All 26 alphabet letters (<kbd>A to Z</kbd>) are performed using <strong>ONE dominant hand</strong>.
363
+ </div>
364
+ </div>
365
+
366
+ <div class="guide-section">
367
+ <div class="guide-section-head">
368
+ <h3 class="guide-section-title">ASL Alphabet (26 Signs)</h3>
369
+ <span class="pill-badge pill-amber">1-Hand Only</span>
370
+ </div>
371
+ <div class="guide-cards-grid" id="asl-cards"></div>
372
+ </div>
373
+ </div>
374
+ </div>
375
+
376
+ <!-- โ•โ• SIGN GUIDE LIGHTBOX POPUP (Top-layer native dialog) โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ• -->
377
+ <dialog class="sign-lightbox-dialog" id="sign-lightbox-overlay">
378
+ <div class="sign-lightbox" id="sign-lightbox">
379
+ <button class="sign-lightbox-close" id="sign-lightbox-close" aria-label="Close preview">&times;</button>
380
+ <div class="sign-lightbox-img-wrap">
381
+ <img id="sign-lightbox-img" src="" alt="Sign preview" />
382
+ </div>
383
+ <div class="sign-lightbox-letter" id="sign-lightbox-letter"></div>
384
+ <span class="sign-lightbox-type" id="sign-lightbox-type"></span>
385
+ <p class="sign-lightbox-desc" id="sign-lightbox-desc"></p>
386
+ <div class="sign-lightbox-lang" id="sign-lightbox-lang"></div>
387
+ </div>
388
+ </dialog>
389
+
390
+ <div class="toast" id="toast" role="status" aria-live="polite"></div>
391
+ <script src="app.js?v=19"></script>
392
+ </body>
393
+ </html>
isl_landmarks_weights.json ADDED
The diff for this file is too large to render. See raw diff
 
isl_signs/1.png ADDED
isl_signs/2.png ADDED
isl_signs/3.png ADDED
isl_signs/4.png ADDED
isl_signs/5.png ADDED
isl_signs/6.png ADDED
isl_signs/7.png ADDED
isl_signs/8.png ADDED
isl_signs/9.png ADDED
isl_signs/A.png ADDED
isl_signs/B.png ADDED
isl_signs/C.png ADDED
isl_signs/D.png ADDED
isl_signs/E.png ADDED
isl_signs/F.png ADDED
isl_signs/G.png ADDED