ameforge commited on
Commit
942ee04
·
verified ·
1 Parent(s): 12bc7a1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +94 -0
README.md CHANGED
@@ -1,3 +1,97 @@
 
 
 
 
 
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ =============================================================================
3
+ SAM-MM-HAR
4
+ AMEFORGE Lab
5
+ =============================================================================
6
  ---
7
  license: apache-2.0
8
+ language:
9
+ - en
10
+ - fr
11
+ tags:
12
+ - human-activity-recognition
13
+ - multimodal
14
+ - sensor-fusion
15
+ - edge-ai
16
+ - privacy-preserving
17
+ - pytorch
18
+ - sparse-transformer
19
+ library_name: pytorch
20
+ pipeline_tag: video-classification
21
  ---
22
+
23
+ # SAM-MM-HAR
24
+
25
+ **SAM-MM-HAR** is a lightweight multimodal Human Activity Recognition model
26
+ built by **AMEFORGE Lab** (Amega Mike) on a proprietary sparse Transformer
27
+ architecture. It classifies 40 daily activities from privacy-preserving
28
+ non-RGB sensors: Depth, Skeleton, IMU, mmWave Radar, IR and Thermal.
29
+
30
+ Developed for the **CUHK-X Multimodal Human Activity Challenge**
31
+ (co-located with UbiComp 2026).
32
+
33
+ ## Key specs
34
+
35
+ | Property | Value |
36
+ |---|---|
37
+ | Architecture | Sparse Transformer (proprietary — AMEFORGE) |
38
+ | Parameters | {n_params:,} (~{n_params/1e6:.1f}M) |
39
+ | Size on disk | {size:.1f} MB |
40
+ | Classes | 40 daily activities |
41
+ | Modalities | Depth · Skeleton · IMU · mmWave · IR · Thermal |
42
+ | Val accuracy | {val_acc:.1f}% (cross-subject) |
43
+ | Edge ready | ✅ CPU inference < 100 MB |
44
+
45
+ ## Modalities
46
+
47
+ The model handles missing modalities gracefully — any subset works at inference.
48
+
49
+ | Modality | Encoder type |
50
+ |---|---|
51
+ | Depth | Patch Conv2D + sparse attention |
52
+ | IR / Thermal | Patch Conv2D + sparse attention |
53
+ | Skeleton | Joint linear + sparse attention |
54
+ | IMU (6-axis) | Conv1D temporal |
55
+ | mmWave Radar | Patch Conv2D + sparse attention |
56
+
57
+ A **MotionCore** temporal world-model (GRU over per-frame embeddings)
58
+ models human movement dynamics across frames — the key advantage over
59
+ standard frame-by-frame classifiers.
60
+
61
+ ## Classes (40)
62
+
63
+ Wash_face · Brush_teeth · Wash_hands · Comb_hair · Put/Take_off_glasses ·
64
+ Put/Take_off_clothes · Put/Take_off_shoes · Drink_water · Eat · Read_book ·
65
+ Write · Use_phone · Use_laptop · Sit_down · Stand_up · Lie_down · Get_up ·
66
+ Walk · Run · Jump · Clap · Wave · Point · Throw · Kick · Pick_up ·
67
+ Put_down · Open/Close_door · Turn_on/off_light · Sweep_floor · Vacuum ·
68
+ Fall_down · Check_time · Take_body_temperature
69
+
70
+ ## Inference
71
+
72
+ ```python
73
+ import torch
74
+ from huggingface_hub import hf_hub_download
75
+
76
+ ckpt = hf_hub_download("AMFORGE/sam-mm-har", "best.pt")
77
+ # Load with inference.py from the repo
78
+ # python inference.py --checkpoint best.pt --clip /path/to/clip_folder
79
+ ```
80
+
81
+ ## Citation
82
+
83
+ If you use SAM-MM-HAR, please cite:
84
+
85
+ ```bibtex
86
+ @misc{{sam_mm_har,
87
+ title = {{SAM-MM-HAR: Multimodal Human Activity Recognition
88
+ on Privacy-Preserving Sensors}},
89
+ author = {{Amega Mike}},
90
+ year = {{2026}},
91
+ note = {{AMEFORGE Lab. Built on a proprietary sparse Transformer architecture.}},
92
+ }}
93
+ ```
94
+
95
+ ---
96
+ *Architecture internals are proprietary and not disclosed. © AMEFORGE Lab 2026*
97
+ """