tthhanh commited on
Commit
2a5c0c7
·
1 Parent(s): f6d8ef7

docs: reorganize documentation structure and workflow

Browse files
README.md CHANGED
@@ -1,6 +1,6 @@
1
  ---
2
  title: Pozify
3
- emoji: 🏋️
4
  colorFrom: green
5
  colorTo: blue
6
  sdk: gradio
@@ -20,38 +20,53 @@ tags:
20
 
21
  # Pozify
22
 
23
- Pozify is a Gradio Space that turns a short workout clip into a structured form-review report:
24
- exercise detected, reps counted, issues localized by timestamp, an annotated video, and grounded
25
- coach feedback.
26
 
27
- The project was built for the
28
- [Hugging Face Build Small Hackathon](https://huggingface.co/build-small-hackathon). It uses small,
29
- replaceable models instead of a giant end-to-end model: pose extraction, a tiny trained exercise
30
- router, rule-based rep and issue analysis, and a small instruction model for the final explanation.
 
 
31
 
32
- ## Hackathon Fit
 
 
33
 
34
- | Requirement or badge | Pozify status |
35
- | -------------------- | ------------- |
36
- | Gradio app on HF Space | Uses `gr.Server` with a custom static frontend and FastAPI routes in `app.py`. |
37
- | Small models only | Main coach model is Qwen2.5-7B Instruct; trained router has 182,796 params. Total is far below the 32B cap. |
38
- | Well-Tuned | The exercise router was trained on pose-window data and published as `build-small-hackathon/pozify-exercise-router`. |
39
- | Modal-powered | Dataset ingestion, pose feature extraction, training, evaluation, and Hub publishing run on Modal. |
40
- | Llama Champion | Coach summary can run through `llama-server` with `POZIFY_COACH_SUMMARY_PROVIDER=llama_cpp`. |
41
- | Field Notes | See [docs/build-small-hackathon-report.md](docs/build-small-hackathon-report.md). |
42
- | Off-Brand | The app uses a custom frontend under `web/`, not the default Gradio blocks layout. |
 
 
 
 
 
 
 
 
 
 
 
 
43
 
44
- ## What It Does
45
 
46
  ```text
47
- user profile + input video
48
- -> video quality check
49
- -> MediaPipe pose landmarks
50
- -> pose cleaning and normalization
51
- -> trained exercise router
52
  -> exercise-specific rep counter
53
- -> per-rep metrics and variation detection
54
- -> frame-level issue markers
 
55
  -> annotated video renderer
56
  -> grounded coach summary
57
  -> verifier
@@ -65,117 +80,32 @@ Supported router labels:
65
  - `shoulder_press`
66
  - `unknown`
67
 
68
- Pozify is not a medical device. It does not diagnose injuries, claim injury prevention, or replace a
69
- trainer, clinician, or physical therapist.
70
-
71
  ## Model Stack
72
 
73
  | Component | Model or method | Trained here? | Runtime |
74
- | --------- | --------------- | ------------- | ------- |
75
- | Pose extraction | MediaPipe Pose Landmarker Lite | No | CPU, MediaPipe Tasks GPU delegate when available |
76
- | Exercise router | PyTorch BiLSTM over 30-frame pose windows | Yes | Torch CPU/CUDA, HF ZeroGPU wrapped |
77
  | Router baseline | scikit-learn `HistGradientBoostingClassifier` | Yes | CPU fallback/reference |
78
- | Rep counting | Exercise-specific state machines over pose signals | No ML | CPU |
79
  | Issue markers | Transparent rules over per-rep metrics | No ML | CPU |
80
- | Coach summary | `Qwen/Qwen2.5-7B-Instruct` by default | No | HF Inference, local Transformers, or llama.cpp |
81
- | Verifier | Deterministic checks over generated JSON | No ML | CPU |
 
82
 
83
  The trained router is intentionally tiny:
84
 
85
  | Artifact | Count |
86
- | -------- | ----: |
87
  | BiLSTM router trainable params | 182,796 |
88
  | Router input features per frame | 237 |
89
  | Window length | 30 frames |
90
  | Output classes | 4 |
91
 
92
- ## Training Data
93
-
94
- The router starts from
95
- [`RickyRiccio/Real_Time_Exercise_Recognition_Dataset`](https://huggingface.co/datasets/RickyRiccio/Real_Time_Exercise_Recognition_Dataset).
96
- Supported folders are normalized to `squat`, `push_up`, and `shoulder_press`. Unsupported classes,
97
- such as curl variations, are mapped to `unknown` so the router learns when to reject clips rather
98
- than force a supported label.
99
-
100
- Latest Modal training run:
101
-
102
- | Metric | Value |
103
- | ------ | ----: |
104
- | Feature examples | 134 |
105
- | Pose windows | 2,224 |
106
- | Failed feature extractions | 0 |
107
- | Squat windows | 659 |
108
- | Push-up windows | 287 |
109
- | Shoulder press windows | 646 |
110
- | Unknown windows | 632 |
111
-
112
- Feature extraction converts each video to cleaned pose windows:
113
-
114
- - normalized COCO-17 landmarks with visibility
115
- - knee, hip, elbow, and shoulder angles
116
- - relative distances such as hand width over shoulder width
117
- - frame deltas and velocities
118
- - 30-frame tensors for the BiLSTM
119
- - aggregated vectors for the scikit-learn baseline
120
-
121
- Custom negative examples can be uploaded to the Modal data volume at `/data/raw/custom_unknown/`.
122
- The collection protocol is documented in
123
- [docs/custom-data-collection-guide.md](docs/custom-data-collection-guide.md).
124
-
125
- ## Training On Modal
126
-
127
- Modal is used because the expensive work is bursty: download videos, run pose extraction over a
128
- dataset, train/evaluate, publish artifacts, then shut down.
129
-
130
- ```bash
131
- uv run modal setup
132
- uv run modal run scripts/exercise_router_modal.py --stage ingest
133
- uv run modal run scripts/exercise_router_modal.py --stage features
134
- uv run modal run scripts/exercise_router_modal.py --stage train-baseline
135
- uv run modal run scripts/exercise_router_modal.py --stage train-temporal
136
- uv run modal run scripts/exercise_router_modal.py --stage evaluate
137
- ```
138
-
139
- Or run the full flow:
140
-
141
- ```bash
142
- uv run modal run scripts/exercise_router_modal.py \
143
- --stage all \
144
- --repo-id build-small-hackathon/pozify-exercise-router
145
- ```
146
-
147
- Modal resources:
148
-
149
- - `pozify-router-data`: raw dataset, manifests, extracted pose-window feature caches.
150
- - `pozify-router-models`: `baseline.joblib`, `temporal.pt`, metrics, selection JSON, HF upload log.
151
- - `train_temporal`: runs on Modal `A10` GPU.
152
-
153
- Temporal router hyperparameters:
154
-
155
- | Hyperparameter | Value |
156
- | -------------- | ----: |
157
- | Architecture | BiLSTM |
158
- | Epochs | 73 |
159
- | Hidden units | 73 |
160
- | Dropout | 0.2174 |
161
- | Learning rate | 0.0004 |
162
- | Batch size | 54 |
163
- | Final training loss | 0.0003 |
164
-
165
- Current evaluation summary:
166
-
167
- | Model | Artifact | Accuracy | Unknown rejection rate |
168
- | ----- | -------- | -------: | ---------------------: |
169
- | Baseline | `baseline.joblib` | 0.9982 | 0.9968 |
170
- | BiLSTM temporal | `temporal.pt` | 0.9969 | 0.9968 |
171
-
172
- The active runtime artifact is `temporal.pt`; the baseline is retained for comparison and fallback.
173
- Full metrics are in
174
- [docs/exercise-router-training-report.md](docs/exercise-router-training-report.md).
175
-
176
  ## Run The App Locally
177
 
178
- Install dependencies with `uv`, then start the app:
 
179
 
180
  ```bash
181
  uv run python app.py
@@ -187,14 +117,12 @@ The app listens at:
187
  http://127.0.0.1:7860
188
  ```
189
 
190
- You can also run:
191
 
192
- ```bash
193
- uv run gradio app.py
194
- ```
195
 
196
- Real video mode is the default when a video path is provided. No-video demo runs use mock pose data
197
- so the app can still produce deterministic JSON artifacts.
198
 
199
  Force mock mode:
200
 
@@ -208,8 +136,7 @@ Force real mode:
208
  POZIFY_MOCK_MODE=0 uv run python app.py
209
  ```
210
 
211
- MediaPipe downloads `pose_landmarker_lite.task` to `/tmp/pozify-models` on first use. To provide a
212
- pre-downloaded task file:
213
 
214
  ```bash
215
  POZIFY_MEDIAPIPE_POSE_MODEL=/path/to/pose_landmarker_lite.task \
@@ -217,120 +144,128 @@ POZIFY_MOCK_MODE=0 \
217
  uv run python app.py
218
  ```
219
 
220
- ## Router Artifacts From Hugging Face
221
 
222
- Pozify loads the router from this model repo by default:
223
 
224
- ```text
225
- build-small-hackathon/pozify-exercise-router
226
- ```
227
-
228
- Override the repo:
229
 
230
  ```bash
231
- export POZIFY_ROUTER_HF_REPO_ID=owner/other-pozify-router
 
232
  ```
233
 
234
- Disable Hub loading and use local files only:
 
 
 
 
 
235
 
236
  ```bash
237
- export POZIFY_ROUTER_DISABLE_HF=1
 
 
 
238
  ```
239
 
240
- Local active artifacts should live under:
 
241
 
242
- ```text
243
- models/exercise_router/active/
244
- router_selection.json
245
- temporal.pt
246
- baseline.joblib # optional fallback/reference
247
- router.joblib # optional baseline alias
248
- ```
249
 
250
- Publishing details are in
251
- [docs/huggingface-router-release.md](docs/huggingface-router-release.md), and the model card draft is
252
- in [docs/huggingface-router-model-card.md](docs/huggingface-router-model-card.md).
253
 
254
- ## GPU Runtime
255
 
256
- ### Hugging Face ZeroGPU
 
 
 
 
 
 
 
257
 
258
- On a ZeroGPU Space, compute-heavy calls are wrapped with `spaces.GPU`. Hugging Face sets
259
- `SPACES_ZERO_GPU=1`; Pozify then defaults the Torch router to CUDA.
 
 
 
 
 
 
 
260
 
261
- Useful environment variables:
262
 
263
  | Variable | Purpose |
264
- | -------- | ------- |
265
  | `POZIFY_ROUTER_DEVICE` | Override router device, for example `cpu` or `cuda`. |
266
  | `POZIFY_SPACES_GPU_DURATION` | `spaces.GPU` duration in seconds, default `120`. |
267
  | `POZIFY_COACH_SUMMARY_PROVIDER` | `hf_inference`, `local_transformers`, or `llama_cpp`. |
268
  | `POZIFY_COACH_SUMMARY_MODEL` | Coach model id or llama.cpp model alias. |
 
 
 
 
269
 
270
- Fully local Transformers coach mode on ZeroGPU:
271
 
272
  ```bash
273
- POZIFY_COACH_SUMMARY_PROVIDER=local_transformers \
274
- POZIFY_COACH_SUMMARY_MODEL=Qwen/Qwen2.5-7B-Instruct \
275
- POZIFY_SPACES_GPU_DURATION=300 \
276
- uv run python app.py
277
  ```
278
 
279
- ### Local CUDA
280
-
281
- If Torch sees a CUDA GPU:
282
 
283
  ```bash
284
- POZIFY_ROUTER_DEVICE=cuda \
285
- POZIFY_COACH_SUMMARY_PROVIDER=local_transformers \
286
- POZIFY_COACH_SUMMARY_MODEL=Qwen/Qwen2.5-7B-Instruct \
287
- uv run python app.py
 
 
288
  ```
289
 
290
- The router is small enough that CPU inference is usually fine; GPU is more useful for local
291
- Transformers coach generation.
292
 
293
- ### llama.cpp
294
 
295
- Pozify can send the coach-summary prompt to a local `llama-server` that exposes the
296
- OpenAI-compatible `/v1/chat/completions` endpoint. This is the path for running the coach model as a
297
- GGUF with llama.cpp.
298
-
299
- Start a llama.cpp server with GPU offload. Use either a local GGUF file:
300
 
301
  ```bash
302
- llama-server \
303
- --model /path/to/qwen2.5-7b-instruct-q4_k_m.gguf \
304
- --ctx-size 4096 \
305
- --n-gpu-layers 99 \
306
- --host 127.0.0.1 \
307
- --port 8080
308
  ```
309
 
310
- Or let llama.cpp download a GGUF repo from Hugging Face:
311
 
312
  ```bash
313
- llama-server \
314
- --hf-repo owner/qwen2.5-7b-instruct-gguf:Q4_K_M \
315
- --ctx-size 4096 \
316
- --n-gpu-layers 99 \
317
- --host 127.0.0.1 \
318
- --port 8080
319
  ```
320
 
321
- Then run Pozify against that server:
322
 
323
  ```bash
324
- POZIFY_COACH_SUMMARY_PROVIDER=llama_cpp \
325
- POZIFY_COACH_SUMMARY_MODEL=local-qwen2.5-7b-gguf \
326
- POZIFY_LLAMA_CPP_BASE_URL=http://127.0.0.1:8080 \
327
- POZIFY_COACH_SUMMARY_MAX_TOKENS=700 \
328
- uv run python app.py
329
  ```
330
 
331
- Use `--n-gpu-layers all` or a lower layer count if your llama.cpp build supports it and your GPU
332
- memory budget requires it. On CPU-only machines, omit `--n-gpu-layers`; generation will be slower but
333
- the app still works.
 
 
 
334
 
335
  ## Generated Artifacts
336
 
@@ -351,49 +286,43 @@ Each run creates `runs/<run_id>/` with:
351
  - `verification.json`
352
  - `final_report.json`
353
 
354
- JSON artifacts are validated before they are written. The final report records the analysis mode,
355
- pose source, coach provider, coach model, verifier status, and downloadable artifact URLs. The
356
- coach-summary verifier is disabled by default; set `POZIFY_COACH_SUMMARY_BYPASS_VERIFIER=0` or pass
357
- `bypass_verifier=false` to run it.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
358
 
359
  ## Project Structure
360
 
361
  ```text
362
  app.py
363
  web/
364
- index.html
365
- app.js
366
- report.js
367
- styles.css
368
  src/pozify/
369
  pipeline.py
370
  contracts.py
371
  steps/
372
- video_qc.py
373
- pose_landmarker.py
374
- exercise_classifier.py
375
- annotated_renderer.py
376
- coach_summary.py
377
- verifier.py
378
  ml/
379
- exercise_router_features.py
380
- exercise_router_temporal.py
381
- exercise_router_inference.py
382
  slm/
383
- providers.py
384
- prompting.py
385
  exercises/
386
- squat/
387
- push_up/
388
- shoulder_press/
389
  scripts/
390
- exercise_router_modal.py
391
- upload_exercise_router_to_hf.py
392
  docs/
393
- build-small-hackathon-report.md
394
- exercise-router-training-report.md
395
- huggingface-router-release.md
396
- custom-data-collection-guide.md
397
  ```
398
 
399
  ## Development Checks
@@ -410,13 +339,3 @@ Run the real MediaPipe fixture smoke test only when the fixture is available:
410
  POZIFY_RUN_REAL_POSE_TESTS=1 \
411
  uv run python -m unittest tests.test_pose_steps.PoseStepTests.test_real_sample_mov_extracts_pose_landmarks
412
  ```
413
-
414
- ## Notes And Limits
415
-
416
- - Router metrics are from the current Modal feature cache, not a broad production benchmark.
417
- - The app depends on full-body framing and usable pose extraction.
418
- - Unsupported or low-confidence clips should route to `unknown` instead of receiving fabricated
419
- coaching.
420
- - Qwen is prompted and verified; it is not fine-tuned in this project.
421
- - More held-out videos, camera angles, body types, and custom unknown clips are needed before using
422
- Pozify as a robust training product.
 
1
  ---
2
  title: Pozify
3
+ emoji: "🏋️"
4
  colorFrom: green
5
  colorTo: blue
6
  sdk: gradio
 
20
 
21
  # Pozify
22
 
23
+ Pozify turns a short workout video into a structured form-review report:
 
 
24
 
25
+ - exercise detected
26
+ - reps counted
27
+ - variation and issue markers
28
+ - annotated video and clips
29
+ - grounded coach summary
30
+ - verifier-backed confidence and safety notes
31
 
32
+ The app is built as a small-model pipeline, not a giant end-to-end model. It combines transparent
33
+ pose analysis, a trained exercise router, exercise-specific rule logic, deterministic knowledge-card
34
+ retrieval, and a small summary model.
35
 
36
+ Pozify is not a medical device. It does not diagnose injuries, claim injury prevention, or replace a
37
+ qualified trainer, clinician, or physical therapist.
38
+
39
+ ## Current Status
40
+
41
+ The current codebase supports:
42
+
43
+ - web app runtime through `app.py`
44
+ - trained exercise routing for `squat`, `push_up`, `shoulder_press`, and `unknown`
45
+ - grounded coach-summary generation from structured JSON artifacts
46
+ - verifier and conservative fallback summaries
47
+ - Modal training pipelines for both the exercise router and coach-summary model
48
+
49
+ The current safest cloud runtime for coach summary is:
50
+
51
+ - `Qwen/Qwen3-14B`
52
+
53
+ The current fine-tuned coach-summary Hugging Face repo can be trained, merged, and published, but
54
+ it is not yet cleanly usable through the current Hugging Face serverless `chat_completion` path.
55
+ If you want to use that fine-tuned model today, the simplest path is local merged-model inference
56
+ through `POZIFY_COACH_SUMMARY_LOCAL_MODEL_DIR`.
57
 
58
+ ## Product Flow
59
 
60
  ```text
61
+ video + user profile
62
+ -> video QC
63
+ -> MediaPipe pose extraction
64
+ -> pose cleaning
65
+ -> exercise router
66
  -> exercise-specific rep counter
67
+ -> per-rep analysis
68
+ -> variation detection
69
+ -> issue markers
70
  -> annotated video renderer
71
  -> grounded coach summary
72
  -> verifier
 
80
  - `shoulder_press`
81
  - `unknown`
82
 
 
 
 
83
  ## Model Stack
84
 
85
  | Component | Model or method | Trained here? | Runtime |
86
+ | --- | --- | --- | --- |
87
+ | Pose extraction | MediaPipe Pose Landmarker Lite | No | CPU / MediaPipe delegate |
88
+ | Exercise router | PyTorch BiLSTM over 30-frame pose windows | Yes | Torch |
89
  | Router baseline | scikit-learn `HistGradientBoostingClassifier` | Yes | CPU fallback/reference |
90
+ | Rep counting | Exercise-specific state machines | No ML | CPU |
91
  | Issue markers | Transparent rules over per-rep metrics | No ML | CPU |
92
+ | Coach summary | `Qwen/Qwen3-14B` by default | Base model only | HF Inference, local Transformers, or llama.cpp |
93
+ | Coach-summary fine-tune | LoRA / merged checkpoint pipeline on Modal | Yes | Local merged-model path recommended |
94
+ | Verifier | Deterministic safety and grounding checks | No ML | CPU |
95
 
96
  The trained router is intentionally tiny:
97
 
98
  | Artifact | Count |
99
+ | --- | ---: |
100
  | BiLSTM router trainable params | 182,796 |
101
  | Router input features per frame | 237 |
102
  | Window length | 30 frames |
103
  | Output classes | 4 |
104
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
105
  ## Run The App Locally
106
 
107
+ This repo uses a `src/` layout, but `uv` is configured with `package = false`, so the correct local
108
+ entrypoint is:
109
 
110
  ```bash
111
  uv run python app.py
 
117
  http://127.0.0.1:7860
118
  ```
119
 
120
+ ### Mock vs Real Mode
121
 
122
+ By default:
 
 
123
 
124
+ - if no video is provided, Pozify uses mock mode
125
+ - if a real video is uploaded, Pozify runs the full analysis pipeline
126
 
127
  Force mock mode:
128
 
 
136
  POZIFY_MOCK_MODE=0 uv run python app.py
137
  ```
138
 
139
+ If you already have the MediaPipe task file locally:
 
140
 
141
  ```bash
142
  POZIFY_MEDIAPIPE_POSE_MODEL=/path/to/pose_landmarker_lite.task \
 
144
  uv run python app.py
145
  ```
146
 
147
+ ## Coach Summary Runtime Options
148
 
149
+ ### 1. Simplest cloud path
150
 
151
+ Use the default supported cloud model:
 
 
 
 
152
 
153
  ```bash
154
+ export POZIFY_COACH_SUMMARY_MODEL=Qwen/Qwen3-14B
155
+ uv run python app.py
156
  ```
157
 
158
+ This is the easiest path if you want the UI to avoid immediate fallback caused by unsupported custom
159
+ HF serverless routing.
160
+
161
+ ### 2. Use the fine-tuned merged model locally
162
+
163
+ Download the merged repo locally, then point Pozify at it:
164
 
165
  ```bash
166
+ export POZIFY_COACH_SUMMARY_LOCAL_MODEL_DIR=/absolute/path/to/merged_model
167
+ export POZIFY_COACH_SUMMARY_BASE_MODEL=Qwen/Qwen3-14B
168
+ export POZIFY_COACH_SUMMARY_ADAPTER_ID=build-small-hackathon/pozify-coach-summary1
169
+ uv run python app.py
170
  ```
171
 
172
+ This is the simplest way to use `build-small-hackathon/pozify-coach-summary1` today without adding a
173
+ dedicated inference endpoint.
174
 
175
+ ### 3. llama.cpp
 
 
 
 
 
 
176
 
177
+ Pozify can send the coach-summary prompt to a local `llama-server` that exposes the
178
+ OpenAI-compatible `/v1/chat/completions` endpoint.
 
179
 
180
+ Example:
181
 
182
+ ```bash
183
+ llama-server \
184
+ --model /path/to/qwen3-14b-instruct.gguf \
185
+ --ctx-size 4096 \
186
+ --n-gpu-layers 99 \
187
+ --host 127.0.0.1 \
188
+ --port 8080
189
+ ```
190
 
191
+ Then:
192
+
193
+ ```bash
194
+ POZIFY_COACH_SUMMARY_PROVIDER=llama_cpp \
195
+ POZIFY_COACH_SUMMARY_MODEL=local-qwen3-14b-gguf \
196
+ POZIFY_LLAMA_CPP_BASE_URL=http://127.0.0.1:8080 \
197
+ POZIFY_COACH_SUMMARY_MAX_TOKENS=700 \
198
+ uv run python app.py
199
+ ```
200
 
201
+ ### Useful environment variables
202
 
203
  | Variable | Purpose |
204
+ | --- | --- |
205
  | `POZIFY_ROUTER_DEVICE` | Override router device, for example `cpu` or `cuda`. |
206
  | `POZIFY_SPACES_GPU_DURATION` | `spaces.GPU` duration in seconds, default `120`. |
207
  | `POZIFY_COACH_SUMMARY_PROVIDER` | `hf_inference`, `local_transformers`, or `llama_cpp`. |
208
  | `POZIFY_COACH_SUMMARY_MODEL` | Coach model id or llama.cpp model alias. |
209
+ | `POZIFY_COACH_SUMMARY_LOCAL_MODEL_DIR` | Prefer a local merged/model directory for coach summary. |
210
+ | `POZIFY_COACH_SUMMARY_BYPASS_VERIFIER` | Keep model output even when verifier fails. |
211
+
212
+ ## Exercise Router Training
213
 
214
+ Run the full router training and publish flow:
215
 
216
  ```bash
217
+ uv run modal run scripts/exercise_router_modal.py \
218
+ --stage all \
219
+ --repo-id build-small-hackathon/pozify-exercise-router
 
220
  ```
221
 
222
+ Step-by-step:
 
 
223
 
224
  ```bash
225
+ uv run modal run scripts/exercise_router_modal.py --stage ingest
226
+ uv run modal run scripts/exercise_router_modal.py --stage features
227
+ uv run modal run scripts/exercise_router_modal.py --stage train-baseline
228
+ uv run modal run scripts/exercise_router_modal.py --stage train-temporal
229
+ uv run modal run scripts/exercise_router_modal.py --stage evaluate
230
+ uv run modal run scripts/exercise_router_modal.py --stage publish --repo-id build-small-hackathon/pozify-exercise-router
231
  ```
232
 
233
+ The active router artifact is `temporal.pt`; the baseline is retained for comparison and fallback.
 
234
 
235
+ ## Coach Summary Training
236
 
237
+ Build the grounded SFT dataset:
 
 
 
 
238
 
239
  ```bash
240
+ uv run python scripts/build_coach_summary_sft_dataset.py
 
 
 
 
 
241
  ```
242
 
243
+ Run the full coach-summary Modal flow:
244
 
245
  ```bash
246
+ uv run modal run scripts/coach_summary_modal.py \
247
+ --stage all \
248
+ --epochs 2 \
249
+ --style-weight 0.2 \
250
+ --repo-id build-small-hackathon/pozify-coach-summary1
 
251
  ```
252
 
253
+ Step-by-step:
254
 
255
  ```bash
256
+ uv run modal run scripts/coach_summary_modal.py --stage prepare-data
257
+ uv run modal run scripts/coach_summary_modal.py --stage train --epochs 2 --style-weight 0.2
258
+ uv run modal run scripts/coach_summary_modal.py --stage evaluate --limit 5
259
+ uv run modal run scripts/coach_summary_modal.py --stage merge
260
+ uv run modal run scripts/coach_summary_modal.py --stage publish-merged --repo-id build-small-hackathon/pozify-coach-summary1
261
  ```
262
 
263
+ Important runtime note:
264
+
265
+ - publishing the merged repo does not automatically make it usable through HF serverless
266
+ `chat_completion`
267
+ - if you want immediate working inference, use `Qwen/Qwen3-14B`
268
+ - if you want to use the fine-tuned model, use `POZIFY_COACH_SUMMARY_LOCAL_MODEL_DIR`
269
 
270
  ## Generated Artifacts
271
 
 
286
  - `verification.json`
287
  - `final_report.json`
288
 
289
+ JSON artifacts are validated before they are written. The final report records:
290
+
291
+ - analysis mode
292
+ - pose source
293
+ - knowledge-card provenance
294
+ - coach summary provider/model/source
295
+ - verifier status and bypass flags
296
+
297
+ ## Docs Map
298
+
299
+ See [docs/01-docs-index.md](docs/01-docs-index.md) for the ordered documentation map.
300
+
301
+ Most useful operational docs:
302
+
303
+ - [docs/10-overview-build-small-hackathon-report.md](docs/10-overview-build-small-hackathon-report.md)
304
+ - [docs/20-router-training-report.md](docs/20-router-training-report.md)
305
+ - [docs/21-router-huggingface-release.md](docs/21-router-huggingface-release.md)
306
+ - [docs/30-coach-modal-training.md](docs/30-coach-modal-training.md)
307
+ - [docs/31-coach-training-report.md](docs/31-coach-training-report.md)
308
+ - [docs/40-data-custom-collection-guide.md](docs/40-data-custom-collection-guide.md)
309
 
310
  ## Project Structure
311
 
312
  ```text
313
  app.py
314
  web/
 
 
 
 
315
  src/pozify/
316
  pipeline.py
317
  contracts.py
318
  steps/
 
 
 
 
 
 
319
  ml/
 
 
 
320
  slm/
 
 
321
  exercises/
 
 
 
322
  scripts/
 
 
323
  docs/
324
+ demo/
325
+ runs/
 
 
326
  ```
327
 
328
  ## Development Checks
 
339
  POZIFY_RUN_REAL_POSE_TESTS=1 \
340
  uv run python -m unittest tests.test_pose_steps.PoseStepTests.test_real_sample_mov_extracts_pose_landmarks
341
  ```
 
 
 
 
 
 
 
 
 
 
docs/01-docs-index.md ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Pozify Docs Index
2
+
3
+ This folder is ordered by purpose so the current workflow is easier to follow.
4
+
5
+ ## Current Workflow
6
+
7
+ Read in this order if you want to understand and run the current project:
8
+
9
+ 1. [../README.md](../README.md)
10
+ 2. [10-overview-build-small-hackathon-report.md](10-overview-build-small-hackathon-report.md)
11
+ 3. [20-router-training-report.md](20-router-training-report.md)
12
+ 4. [21-router-huggingface-release.md](21-router-huggingface-release.md)
13
+ 5. [30-coach-modal-training.md](30-coach-modal-training.md)
14
+ 6. [31-coach-training-report.md](31-coach-training-report.md)
15
+ 7. [40-data-custom-collection-guide.md](40-data-custom-collection-guide.md)
16
+
17
+ ## Presentation / Demo
18
+
19
+ - [11-overview-demo-video-transcript.md](11-overview-demo-video-transcript.md)
20
+
21
+ ## Planning / Historical Design Notes
22
+
23
+ These files are useful context, but they are not the source of truth for current runtime commands:
24
+
25
+ - [90-archive-implementation-plan.md](90-archive-implementation-plan.md)
26
+ - [91-archive-coach-analysis-implementation-plan.md](91-archive-coach-analysis-implementation-plan.md)
27
+ - [92-archive-coach-model-data-plan.md](92-archive-coach-model-data-plan.md)
28
+ - [99-archive-pozify-project-documentation.md](99-archive-pozify-project-documentation.md)
29
+
30
+ ## Notes
31
+
32
+ - Current coach-summary cloud runtime default: `Qwen/Qwen3-14B`
33
+ - Current fine-tuned coach-summary Hugging Face repo is best used through local merged-model
34
+ inference, not HF serverless `chat_completion`
35
+ - Current app entrypoint: `uv run python app.py`
docs/{build-small-hackathon-report.md → 10-overview-build-small-hackathon-report.md} RENAMED
@@ -1,5 +1,13 @@
1
  # Pozify Build Small Hackathon Report
2
 
 
 
 
 
 
 
 
 
3
  Date: June 14, 2026
4
 
5
  Pozify is a small-model workout form reviewer. A user uploads a short exercise video, adds basic
@@ -49,11 +57,12 @@ pretending every video is one of the supported movements.
49
  | Pose extractor | MediaPipe Pose Landmarker Lite | Fast, practical feature extractor for a Gradio Space. |
50
  | Exercise router | Custom PyTorch BiLSTM | Tiny trainable temporal model over pose windows. |
51
  | Baseline router | scikit-learn HistGradientBoostingClassifier | Strong baseline over engineered vectors and fallback artifact. |
52
- | Coach summary | Qwen2.5-7B-Instruct | Small enough for the hackathon cap, strong at structured JSON explanation. |
53
- | llama.cpp path | Qwen2.5 Instruct GGUF via `llama-server` | Local-first/off-grid coach summary path with GPU offload. |
54
 
55
- The project did not fine-tune Qwen. Instead, Qwen is used as a grounded summarizer over JSON
56
- evidence, and the trained part of the project is the exercise router.
 
57
 
58
  ## What Was Trained
59
 
@@ -203,7 +212,7 @@ logic stay outside the GPU worker. Useful settings:
203
 
204
  ```bash
205
  POZIFY_COACH_SUMMARY_PROVIDER=local_transformers
206
- POZIFY_COACH_SUMMARY_MODEL=Qwen/Qwen2.5-7B-Instruct
207
  POZIFY_SPACES_GPU_DURATION=300
208
  ```
209
 
@@ -218,7 +227,7 @@ Start llama.cpp with a local GGUF:
218
 
219
  ```bash
220
  llama-server \
221
- --model /path/to/qwen2.5-7b-instruct-q4_k_m.gguf \
222
  --ctx-size 4096 \
223
  --n-gpu-layers 99 \
224
  --host 127.0.0.1 \
@@ -229,7 +238,7 @@ Or use a Hugging Face GGUF repo:
229
 
230
  ```bash
231
  llama-server \
232
- --hf-repo owner/qwen2.5-7b-instruct-gguf:Q4_K_M \
233
  --ctx-size 4096 \
234
  --n-gpu-layers 99 \
235
  --host 127.0.0.1 \
@@ -240,7 +249,7 @@ Then point Pozify at it:
240
 
241
  ```bash
242
  POZIFY_COACH_SUMMARY_PROVIDER=llama_cpp \
243
- POZIFY_COACH_SUMMARY_MODEL=local-qwen2.5-7b-gguf \
244
  POZIFY_LLAMA_CPP_BASE_URL=http://127.0.0.1:8080 \
245
  uv run python app.py
246
  ```
 
1
  # Pozify Build Small Hackathon Report
2
 
3
+ Status note:
4
+
5
+ - This report is kept as the hackathon narrative document.
6
+ - Current runtime defaults have moved to `Qwen/Qwen3-14B`.
7
+ - Current coach-summary training now includes a LoRA/merge/publish pipeline on Modal.
8
+ - For the current operational commands, prefer [../README.md](../README.md) and
9
+ [30-coach-modal-training.md](30-coach-modal-training.md).
10
+
11
  Date: June 14, 2026
12
 
13
  Pozify is a small-model workout form reviewer. A user uploads a short exercise video, adds basic
 
57
  | Pose extractor | MediaPipe Pose Landmarker Lite | Fast, practical feature extractor for a Gradio Space. |
58
  | Exercise router | Custom PyTorch BiLSTM | Tiny trainable temporal model over pose windows. |
59
  | Baseline router | scikit-learn HistGradientBoostingClassifier | Strong baseline over engineered vectors and fallback artifact. |
60
+ | Coach summary | Qwen/Qwen3-14B | Current default cloud runtime for structured JSON explanation. |
61
+ | llama.cpp path | Qwen3-14B Instruct GGUF via `llama-server` | Local-first/off-grid coach summary path with GPU offload. |
62
 
63
+ The original hackathon build trained the exercise router first and used Qwen as a grounded
64
+ summarizer over JSON evidence. The current codebase now also contains a coach-summary LoRA / merged
65
+ model training path on Modal.
66
 
67
  ## What Was Trained
68
 
 
212
 
213
  ```bash
214
  POZIFY_COACH_SUMMARY_PROVIDER=local_transformers
215
+ POZIFY_COACH_SUMMARY_MODEL=Qwen/Qwen3-14B
216
  POZIFY_SPACES_GPU_DURATION=300
217
  ```
218
 
 
227
 
228
  ```bash
229
  llama-server \
230
+ --model /path/to/qwen3-14b-instruct-q4_k_m.gguf \
231
  --ctx-size 4096 \
232
  --n-gpu-layers 99 \
233
  --host 127.0.0.1 \
 
238
 
239
  ```bash
240
  llama-server \
241
+ --hf-repo owner/qwen3-14b-instruct-gguf:Q4_K_M \
242
  --ctx-size 4096 \
243
  --n-gpu-layers 99 \
244
  --host 127.0.0.1 \
 
249
 
250
  ```bash
251
  POZIFY_COACH_SUMMARY_PROVIDER=llama_cpp \
252
+ POZIFY_COACH_SUMMARY_MODEL=local-qwen3-14b-gguf \
253
  POZIFY_LLAMA_CPP_BASE_URL=http://127.0.0.1:8080 \
254
  uv run python app.py
255
  ```
docs/{demo-video-transcript.md → 11-overview-demo-video-transcript.md} RENAMED
File without changes
docs/{exercise-router-training-report.md → 20-router-training-report.md} RENAMED
File without changes
docs/{huggingface-router-release.md → 21-router-huggingface-release.md} RENAMED
@@ -61,7 +61,7 @@ Omit `--private` if the model repository should be public.
61
 
62
  The Modal training script can also publish directly from the `pozify-router-models` volume after
63
  evaluation. It reads `HF_TOKEN`, `POZIFY_ROUTER_HF_REPO_ID`, and optional
64
- `POZIFY_ROUTER_HF_PRIVATE=1` from `.env` via `modal.Secret.from_dotenv()`:
65
 
66
  ```bash
67
  uv run modal run scripts/exercise_router_modal.py --stage all
 
61
 
62
  The Modal training script can also publish directly from the `pozify-router-models` volume after
63
  evaluation. It reads `HF_TOKEN`, `POZIFY_ROUTER_HF_REPO_ID`, and optional
64
+ `POZIFY_ROUTER_HF_PRIVATE=1` from local `.env` and injects them into Modal secrets at runtime:
65
 
66
  ```bash
67
  uv run modal run scripts/exercise_router_modal.py --stage all
docs/{huggingface-router-model-card.md → 22-router-huggingface-model-card.md} RENAMED
File without changes
docs/30-coach-modal-training.md ADDED
@@ -0,0 +1,193 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Coach Summary Modal Training
2
+
3
+ This guide explains the current Modal workflow for training, evaluating, merging, and publishing the
4
+ Pozify grounded coach-summary model.
5
+
6
+ ## Scope
7
+
8
+ The pipeline lives in:
9
+
10
+ - `scripts/coach_summary_modal.py`
11
+
12
+ Available stages:
13
+
14
+ - `prepare-data`
15
+ - `train`
16
+ - `evaluate`
17
+ - `merge`
18
+ - `publish`
19
+ - `publish-merged`
20
+ - `all`
21
+
22
+ ## What This Pipeline Produces
23
+
24
+ The pipeline fine-tunes a coach-summary model for Pozify's structured JSON-to-summary task.
25
+
26
+ It can produce:
27
+
28
+ - a LoRA adapter
29
+ - evaluation artifacts
30
+ - a merged full Transformers checkpoint
31
+ - a Hugging Face repo containing the merged checkpoint
32
+
33
+ Important limitation:
34
+
35
+ - Publishing a merged model repo to Hugging Face does **not** currently guarantee that the repo can
36
+ be used through Hugging Face serverless `chat_completion`.
37
+ - In the current codebase, the safest cloud runtime remains `Qwen/Qwen3-14B`.
38
+ - The simplest way to use the fine-tuned merged model today is local inference through
39
+ `POZIFY_COACH_SUMMARY_LOCAL_MODEL_DIR`.
40
+
41
+ ## Requirements
42
+
43
+ 1. Install and authenticate Modal:
44
+
45
+ ```bash
46
+ uv run modal setup
47
+ ```
48
+
49
+ 2. Make sure local `.env` contains at least:
50
+
51
+ - `HF_TOKEN`
52
+ - optional `POZIFY_COACH_SUMMARY_HF_REPO_ID`
53
+ - optional `POZIFY_COACH_SUMMARY_MERGED_HF_REPO_ID`
54
+ - optional `POZIFY_COACH_SUMMARY_MODEL`
55
+ - optional `POZIFY_COACH_SUMMARY_HF_PRIVATE=1`
56
+
57
+ The script reads local `.env` and injects those values into a Modal secret at runtime.
58
+
59
+ ## Input Data
60
+
61
+ The pipeline expects these files:
62
+
63
+ - `data/sft/coach_summary_train.jsonl`
64
+ - `data/sft/coach_summary_eval.jsonl`
65
+ - `data/sft/public_fitness_style.jsonl`
66
+ - `configs/coach_summary_lora.default.json`
67
+
68
+ The default checked-in config currently points to:
69
+
70
+ - base model: `Qwen/Qwen3-14B`
71
+
72
+ ## Recommended Training Flow
73
+
74
+ ### 1. Build the SFT dataset
75
+
76
+ ```bash
77
+ uv run python scripts/build_coach_summary_sft_dataset.py
78
+ ```
79
+
80
+ ### 2. Copy data into the Modal data volume
81
+
82
+ ```bash
83
+ uv run modal run scripts/coach_summary_modal.py --stage prepare-data
84
+ ```
85
+
86
+ ### 3. Train the adapter
87
+
88
+ ```bash
89
+ uv run modal run scripts/coach_summary_modal.py --stage train --epochs 2 --style-weight 0.2
90
+ ```
91
+
92
+ ### 4. Evaluate
93
+
94
+ ```bash
95
+ uv run modal run scripts/coach_summary_modal.py --stage evaluate --limit 5
96
+ ```
97
+
98
+ ### 5. Merge the adapter into a full model
99
+
100
+ ```bash
101
+ uv run modal run scripts/coach_summary_modal.py --stage merge
102
+ ```
103
+
104
+ ### 6. Publish the merged model
105
+
106
+ ```bash
107
+ uv run modal run scripts/coach_summary_modal.py --stage publish-merged --repo-id build-small-hackathon/pozify-coach-summary1
108
+ ```
109
+
110
+ ### Full end-to-end run
111
+
112
+ ```bash
113
+ uv run modal run scripts/coach_summary_modal.py --stage all --epochs 2 --style-weight 0.2 --repo-id build-small-hackathon/pozify-coach-summary1
114
+ ```
115
+
116
+ ## Cheap Smoke-Test Workflow
117
+
118
+ If you want to reduce GPU cost before a full run:
119
+
120
+ ```bash
121
+ uv run modal run scripts/coach_summary_modal.py --stage prepare-data
122
+ uv run modal run scripts/coach_summary_modal.py --stage train --epochs 1 --style-weight 0.2
123
+ uv run modal run scripts/coach_summary_modal.py --stage evaluate --limit 5
124
+ ```
125
+
126
+ If the format and verifier behavior look acceptable, rerun with `--epochs 2` and then `merge` +
127
+ `publish-merged`.
128
+
129
+ ## Model Volume Artifacts
130
+
131
+ The Modal model volume stores:
132
+
133
+ - `adapter/`
134
+ - `merged_model/`
135
+ - `training_config.json`
136
+ - `training_summary.json`
137
+ - `evaluation.json`
138
+ - `hf_upload.json`
139
+ - `merge_summary.json`
140
+ - `hf_merged_upload.json`
141
+ - `README.md`
142
+
143
+ ## Current Runtime Usage
144
+
145
+ ### Safest cloud runtime now
146
+
147
+ ```bash
148
+ export POZIFY_COACH_SUMMARY_MODEL=Qwen/Qwen3-14B
149
+ uv run python app.py
150
+ ```
151
+
152
+ ### Use the merged fine-tuned model locally
153
+
154
+ Download the merged repo or copy the merged directory locally, then:
155
+
156
+ ```bash
157
+ export POZIFY_COACH_SUMMARY_LOCAL_MODEL_DIR=/path/to/merged_model
158
+ export POZIFY_COACH_SUMMARY_BASE_MODEL=Qwen/Qwen3-14B
159
+ export POZIFY_COACH_SUMMARY_ADAPTER_ID=build-small-hackathon/pozify-coach-summary1
160
+ uv run python app.py
161
+ ```
162
+
163
+ ### Not recommended right now
164
+
165
+ Pointing app runtime directly at the merged Hugging Face repo through:
166
+
167
+ ```bash
168
+ export POZIFY_COACH_SUMMARY_MODEL=build-small-hackathon/pozify-coach-summary1
169
+ ```
170
+
171
+ This may still fall back because HF serverless currently rejects that repo as “not a chat model”.
172
+
173
+ ## Evaluation Meaning
174
+
175
+ The coach-summary evaluation is task-specific. It measures:
176
+
177
+ - JSON validity rate
178
+ - verifier pass rate
179
+ - section completeness rate
180
+ - failure count and example failures
181
+
182
+ This is more useful for Pozify than generic text-generation metrics because the product depends on:
183
+
184
+ - schema correctness
185
+ - grounded issue references
186
+ - safe variation handling
187
+ - no diagnosis / no injury-prevention claims
188
+
189
+ ## Related Docs
190
+
191
+ - [../README.md](../README.md)
192
+ - [31-coach-training-report.md](31-coach-training-report.md)
193
+ - [92-archive-coach-model-data-plan.md](92-archive-coach-model-data-plan.md)
docs/31-coach-training-report.md ADDED
@@ -0,0 +1,370 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Coach Summary Training Report
2
+
3
+ Generated from the current Pozify codebase state on June 14, 2026.
4
+
5
+ ## Summary
6
+
7
+ The coach-summary model in Pozify is a grounded JSON-to-JSON generation component, not a generic
8
+ fitness chatbot. Its job is to convert structured workout-analysis artifacts into a safe,
9
+ evidence-bounded `coach_summary.json`, then pass that output through a verifier before it is shown
10
+ in the app.
11
+
12
+ The current training stack is:
13
+
14
+ - Base model: `Qwen/Qwen3-14B`
15
+ - Fine-tuning method: LoRA / QLoRA-style SFT on Modal
16
+ - Training target: Pozify-native structured summary generation
17
+ - Runtime fallback: deterministic conservative summary when model generation fails or verification fails
18
+
19
+ The current codebase already supports:
20
+
21
+ - dataset building from real Pozify run artifacts
22
+ - Modal stages for prepare/train/evaluate/merge/publish
23
+ - verifier-based evaluation
24
+ - merged full-model publishing
25
+ - UI metadata for provider/model/source reporting
26
+
27
+ The main deployment gap at the time of this report is not training infrastructure. The main gap is
28
+ that the published Hugging Face merged repo is still not accepted by the current Hugging Face
29
+ Inference API path used by the app, so the app falls back to rule-based summaries when that repo is
30
+ selected.
31
+
32
+ ## Objective
33
+
34
+ The coach-summary model is designed to generate these grounded sections:
35
+
36
+ - `summary`
37
+ - `what_you_did`
38
+ - `what_looked_good`
39
+ - `what_changed_across_reps`
40
+ - `valid_variation_vs_issue`
41
+ - `top_fixes`
42
+ - `next_session_plan`
43
+ - `confidence_notes`
44
+
45
+ Unlike a free-form assistant, this model is instructed to:
46
+
47
+ - use only structured evidence and retrieved knowledge cards
48
+ - never invent issue labels outside `issue_markers.json`
49
+ - avoid diagnosis language
50
+ - avoid injury-prevention claims
51
+ - preserve valid variation context instead of overcorrecting it
52
+
53
+ This contract is implemented in the prompt builder and enforced again by the verifier.
54
+
55
+ ## Training Data
56
+
57
+ The current SFT data is Pozify-native and built from real run artifacts under `runs/`.
58
+
59
+ ### Source artifacts per example
60
+
61
+ Each SFT row is constructed from:
62
+
63
+ - `user_profile.json`
64
+ - `exercise_classification.json`
65
+ - `reps.json`
66
+ - `rep_analysis.json`
67
+ - `variation.json`
68
+ - `issue_markers.json`
69
+ - `coach_summary.json`
70
+
71
+ The dataset builder reconstructs typed contracts from those files, retrieves the same knowledge
72
+ cards used at runtime, and serializes a chat-style SFT example with:
73
+
74
+ - system prompt
75
+ - user message containing structured evidence JSON
76
+ - assistant message containing the target `coach_summary.json`
77
+
78
+ ### Current local dataset size
79
+
80
+ From the checked-in data in `data/sft/`:
81
+
82
+ | File | Rows |
83
+ | --- | ---: |
84
+ | `coach_summary_train.jsonl` | 22 |
85
+ | `coach_summary_eval.jsonl` | 5 |
86
+ | `public_fitness_style.jsonl` | 854 |
87
+
88
+ The style dataset is much larger than the grounded task dataset, so the training script samples
89
+ only a small portion of style rows according to `style_weight`.
90
+
91
+ ## Prompt and Grounding Design
92
+
93
+ The prompt builder creates a structured instruction block plus a structured evidence block.
94
+
95
+ ### Evidence included in the prompt
96
+
97
+ - `user_profile`
98
+ - `exercise_classification`
99
+ - `variation`
100
+ - `rep_summary`
101
+ - `issue_summary`
102
+ - `priority_cues`
103
+ - `knowledge_cards`
104
+
105
+ ### Core prompt rules
106
+
107
+ - Use only the evidence JSON and retrieved knowledge cards.
108
+ - Do not infer new issues absent from `issue_summary.issues`.
109
+ - Do not diagnose injuries or pathology.
110
+ - Do not claim injury prevention.
111
+ - Do not treat valid variation labels or not-issue labels as errors.
112
+ - Include exact issue or variation labels in backticks when referenced.
113
+ - Return JSON only.
114
+
115
+ This is a good fit for small-model fine-tuning because the task is narrow, repetitive, and highly
116
+ structured.
117
+
118
+ ## Model and Training Setup
119
+
120
+ ### Current default base model
121
+
122
+ The codebase now defaults to:
123
+
124
+ - `Qwen/Qwen3-14B`
125
+
126
+ This is used in:
127
+
128
+ - `configs/coach_summary_lora.default.json`
129
+ - `scripts/train_coach_summary_lora.py`
130
+ - runtime default in `src/pozify/slm/providers.py`
131
+
132
+ ### Default training hyperparameters
133
+
134
+ | Hyperparameter | Value |
135
+ | --- | ---: |
136
+ | Base model | `Qwen/Qwen3-14B` |
137
+ | LoRA rank | 16 |
138
+ | LoRA alpha | 32 |
139
+ | LoRA dropout | 0.05 |
140
+ | Learning rate | 0.0002 |
141
+ | Epochs | 2 |
142
+ | Batch size per device | 1 |
143
+ | Gradient accumulation | 8 |
144
+ | Max sequence length | 4096 |
145
+ | Default style weight | 0.2 |
146
+
147
+ ### Modal implementation
148
+
149
+ The full training pipeline is implemented in `scripts/coach_summary_modal.py` with these stages:
150
+
151
+ - `prepare-data`
152
+ - `train`
153
+ - `evaluate`
154
+ - `merge`
155
+ - `publish`
156
+ - `publish-merged`
157
+ - `all`
158
+
159
+ The training stage:
160
+
161
+ - loads the base model with 4-bit quantization
162
+ - fine-tunes with TRL `SFTTrainer`
163
+ - saves adapter weights and tokenizer to the Modal model volume
164
+
165
+ The merge stage:
166
+
167
+ - loads base model plus adapter
168
+ - calls `merge_and_unload()`
169
+ - writes a merged full checkpoint to `merged_model/`
170
+
171
+ ## Evaluation Design
172
+
173
+ The evaluation stage is not based on generic BLEU or chat preference metrics. It uses task-specific
174
+ checks that match Pozify's production constraints.
175
+
176
+ ### Reported evaluation metrics
177
+
178
+ - JSON validity rate
179
+ - verifier pass rate
180
+ - section completeness rate
181
+ - failure count and example failures
182
+
183
+ ### Verifier checks
184
+
185
+ The verifier currently enforces:
186
+
187
+ - `no_issue_outside_json`
188
+ - `variation_not_overcorrected`
189
+ - `no_diagnosis`
190
+ - `no_injury_prevention_claim`
191
+ - `confidence_notes_present_when_required`
192
+
193
+ This is a strong design choice for Pozify because it measures whether the model behaves safely and
194
+ stays grounded, not just whether it sounds fluent.
195
+
196
+ ## Latest Observed Training Outcome
197
+
198
+ From the latest visible Modal training logs for the full coach-summary flow:
199
+
200
+ ### Training summary
201
+
202
+ | Metric | Value |
203
+ | --- | ---: |
204
+ | Train rows | 22 |
205
+ | Eval rows | 5 |
206
+ | Style rows mixed in | 4 |
207
+ | Merged train rows | 26 |
208
+ | Epochs | 2 |
209
+ | Global steps | 8 |
210
+ | Training loss | 1.0797 |
211
+
212
+ ### Evaluation summary
213
+
214
+ | Metric | Value |
215
+ | --- | ---: |
216
+ | Evaluated count | 5 |
217
+ | JSON valid count | 2 |
218
+ | JSON validity rate | 0.40 |
219
+ | Verifier pass count | 0 |
220
+ | Verifier pass rate | 0.00 |
221
+ | Section completeness rate | 0.00 |
222
+
223
+ ### Observed failure modes
224
+
225
+ - missing required top-level fields such as `summary`
226
+ - malformed JSON with delimiter errors
227
+ - outputs that fail schema extraction before verifier success is even possible
228
+
229
+ These numbers suggest that the current grounded task setup is correct in principle, but the dataset
230
+ is still too small and the model is not yet reliably producing the full required schema.
231
+
232
+ ## Runtime Behavior in the App
233
+
234
+ At runtime the pipeline does this:
235
+
236
+ 1. Generate a model summary from structured evidence.
237
+ 2. Parse the model output into `coach_summary.json`.
238
+ 3. Run the verifier.
239
+ 4. If generation fails, use `fallback_initial`.
240
+ 5. If generation succeeds but verification fails, use `fallback_after_verification` unless bypass is enabled.
241
+
242
+ The final report stores:
243
+
244
+ - `coach_summary_provider`
245
+ - `coach_summary_model`
246
+ - `coach_summary_source`
247
+ - verifier bypass metadata
248
+
249
+ This makes the UI traceable and helps distinguish:
250
+
251
+ - cloud model output
252
+ - local model output
253
+ - conservative fallback output
254
+
255
+ ## Current Deployment Status
256
+
257
+ ### What works
258
+
259
+ - grounded prompt contract
260
+ - deterministic knowledge-card retrieval
261
+ - SFT dataset builder from Pozify artifacts
262
+ - Modal train/evaluate/merge/publish pipeline
263
+ - verifier integration
264
+ - fallback summary integration
265
+ - runtime metadata reporting in UI
266
+
267
+ ### What is currently failing
268
+
269
+ The published merged Hugging Face repo for the fine-tuned coach model is still not accepted by the
270
+ Hugging Face inference route currently used in `HFInferenceCoachSummaryModel`.
271
+
272
+ Observed runtime failure:
273
+
274
+ - `The requested model 'build-small-hackathon/pozify-coach-summary1' is not a chat model.`
275
+
276
+ Because of that, the app falls back even when the runtime resolves the correct repo ID.
277
+
278
+ ### Practical implication
279
+
280
+ For the app to avoid fallback today, the safest temporary runtime choice is a cloud-hosted model
281
+ that is already supported by Hugging Face Inference, such as:
282
+
283
+ - `Qwen/Qwen3-14B`
284
+
285
+ The fine-tuned merged repo is still useful as a training artifact, but it is not yet plug-and-play
286
+ through the current cloud inference path.
287
+
288
+ ## Assessment
289
+
290
+ ### Strengths
291
+
292
+ - The task definition is well-scoped and matches the product.
293
+ - The prompt is strongly grounded on structured artifacts.
294
+ - The verifier is aligned with real product risk, not just language quality.
295
+ - The dataset builder is tightly coupled to actual runtime contracts.
296
+ - The pipeline supports both model generation and deterministic fallback.
297
+
298
+ ### Weaknesses
299
+
300
+ - The grounded SFT dataset is still very small.
301
+ - Schema reliability is not yet strong enough.
302
+ - Verifier pass rate is currently too low for production dependence.
303
+ - Published merged artifacts are not yet working cleanly with the current Hugging Face inference strategy.
304
+ - Modal config reuse can accidentally preserve stale training settings if old volume artifacts are not cleaned.
305
+
306
+ ## Recommendations
307
+
308
+ ### Short term
309
+
310
+ 1. Use `Qwen/Qwen3-14B` as the temporary runtime cloud model.
311
+ 2. Keep the fallback summary enabled in production.
312
+ 3. Expand the grounded SFT dataset from more real runs before increasing training complexity.
313
+ 4. Add stronger output-format controls or post-processing to improve JSON validity.
314
+ 5. Clean or version Modal model volumes when switching base models.
315
+
316
+ ### Medium term
317
+
318
+ 1. Grow the Pozify-native dataset to at least 100 to 300 high-quality examples.
319
+ 2. Evaluate `epochs=1` vs `epochs=2` and `style_weight=0.1` vs `0.2`.
320
+ 3. Add targeted eval slices:
321
+ - no-issue cases
322
+ - valid-variation cases
323
+ - multi-issue cases
324
+ - low-confidence / low-pose-coverage cases
325
+ 4. Improve inference compatibility for the merged repo or support local merged-model inference directly.
326
+
327
+ ## Reproduction Commands
328
+
329
+ Build the grounded SFT dataset from run artifacts:
330
+
331
+ ```bash
332
+ uv run python scripts/build_coach_summary_sft_dataset.py
333
+ ```
334
+
335
+ Run the full Modal training flow:
336
+
337
+ ```bash
338
+ uv run modal run scripts/coach_summary_modal.py --stage all --epochs 2 --style-weight 0.2 --repo-id build-small-hackathon/pozify-coach-summary1
339
+ ```
340
+
341
+ Run the app against a cloud-supported fallback-safe runtime model:
342
+
343
+ ```bash
344
+ unset POZIFY_COACH_SUMMARY_MODEL
345
+ uv run python app.py
346
+ ```
347
+
348
+ Or set the runtime model explicitly:
349
+
350
+ ```bash
351
+ export POZIFY_COACH_SUMMARY_MODEL=Qwen/Qwen3-14B
352
+ uv run python app.py
353
+ ```
354
+
355
+ ## Conclusion
356
+
357
+ Pozify's coach-summary model training architecture is directionally strong. The core design choice
358
+ to train a grounded summary model on structured evidence, then gate it with a verifier, is the
359
+ right one for this product.
360
+
361
+ The current bottleneck is not lack of infrastructure. It is model reliability and inference
362
+ deployment maturity:
363
+
364
+ - too few grounded examples
365
+ - low schema-compliance rate
366
+ - no verifier passes in the latest visible eval
367
+ - published merged repo still not usable through the current HF inference path
368
+
369
+ The codebase is ready for the next iteration. The next high-leverage step is to improve dataset
370
+ size and output reliability while keeping the verifier and fallback architecture intact.
docs/{custom-data-collection-guide.md → 40-data-custom-collection-guide.md} RENAMED
File without changes
docs/{implementation-plan.md → 90-archive-implementation-plan.md} RENAMED
@@ -1,5 +1,10 @@
1
  # Pozify Implementation Plan
2
 
 
 
 
 
 
3
  This plan breaks Pozify into implementation workstreams that can be developed independently while preserving the current JSON data contracts.
4
 
5
  ## Phase 1: Foundation And Contracts
 
1
  # Pozify Implementation Plan
2
 
3
+ Status: historical implementation plan.
4
+
5
+ Use [../README.md](../README.md) and [01-docs-index.md](01-docs-index.md) for the current runtime
6
+ workflow and operational commands.
7
+
8
  This plan breaks Pozify into implementation workstreams that can be developed independently while preserving the current JSON data contracts.
9
 
10
  ## Phase 1: Foundation And Contracts
docs/{coach-analysis-implementation-plan.md → 91-archive-coach-analysis-implementation-plan.md} RENAMED
@@ -1,5 +1,10 @@
1
  # Pozify Coach Analysis Expansion Plan
2
 
 
 
 
 
 
3
  ## Objective
4
 
5
  Extend Pozify's grounded coach-summary pipeline with a dataset-ready knowledge layer so the app can incorporate richer exercise expertise without turning the product into a generic chatbot.
 
1
  # Pozify Coach Analysis Expansion Plan
2
 
3
+ Status: historical implementation plan for the coach-analysis branch work.
4
+
5
+ Use [../README.md](../README.md) and [01-docs-index.md](01-docs-index.md) for the current runtime
6
+ workflow and operational commands.
7
+
8
  ## Objective
9
 
10
  Extend Pozify's grounded coach-summary pipeline with a dataset-ready knowledge layer so the app can incorporate richer exercise expertise without turning the product into a generic chatbot.
docs/{coach-model-data-plan.md → 92-archive-coach-model-data-plan.md} RENAMED
@@ -1,5 +1,11 @@
1
  # Pozify Coach Model Data Plan
2
 
 
 
 
 
 
 
3
  ## Objective
4
 
5
  Define a practical data and modeling plan for improving Pozify's coach summary and coach-style guidance using open datasets and the current codebase.
 
1
  # Pozify Coach Model Data Plan
2
 
3
+ Status: planning and architecture note.
4
+
5
+ This file is useful for model and dataset strategy, but it is not the source of truth for the
6
+ current run/deploy commands. Use [../README.md](../README.md) and
7
+ [30-coach-modal-training.md](30-coach-modal-training.md) for the current workflow.
8
+
9
  ## Objective
10
 
11
  Define a practical data and modeling plan for improving Pozify's coach summary and coach-style guidance using open datasets and the current codebase.
docs/{pozify-project-documentation.md → 99-archive-pozify-project-documentation.md} RENAMED
@@ -1,5 +1,10 @@
1
  # Pozify Project Documentation
2
 
 
 
 
 
 
3
  ## 1. Product Overview
4
 
5
  **Pozify** is a Gradio web app that lets a user upload or record a short workout video, then returns a rep-by-rep movement report built from pose landmarks, movement metrics, frame-level issue markers, and goal-aware coaching feedback.
 
1
  # Pozify Project Documentation
2
 
3
+ Status: large historical product/design document.
4
+
5
+ Some sections may lag behind the current runtime implementation. For the current project entrypoint,
6
+ commands, and model workflow, use [../README.md](../README.md) and [01-docs-index.md](01-docs-index.md).
7
+
8
  ## 1. Product Overview
9
 
10
  **Pozify** is a Gradio web app that lets a user upload or record a short workout video, then returns a rep-by-rep movement report built from pose landmarks, movement metrics, frame-level issue markers, and goal-aware coaching feedback.
docs/coach-summary-modal-training.md DELETED
@@ -1,139 +0,0 @@
1
- # Coach Summary Modal Training
2
-
3
- This guide explains how to train, evaluate, and publish the Pozify coach-summary LoRA adapter on [Modal](https://modal.com).
4
-
5
- ## What The Script Does
6
-
7
- The Modal pipeline lives in:
8
-
9
- - `scripts/coach_summary_modal.py`
10
-
11
- It follows the same stage pattern as the exercise-router Modal pipeline:
12
-
13
- - `prepare-data`
14
- - `train`
15
- - `evaluate`
16
- - `merge`
17
- - `publish`
18
- - `publish-merged`
19
- - `all`
20
-
21
- ## Requirements
22
-
23
- 1. Install the Modal CLI locally.
24
- 2. Authenticate Modal:
25
-
26
- ```bash
27
- modal setup
28
- ```
29
-
30
- 3. Make sure your local `.env` contains:
31
-
32
- - `HF_TOKEN`
33
- - optional `POZIFY_COACH_SUMMARY_HF_REPO_ID`
34
- - optional `POZIFY_COACH_SUMMARY_MERGED_HF_REPO_ID`
35
- - recommended `POZIFY_COACH_SUMMARY_MODEL`
36
- - optional `POZIFY_COACH_SUMMARY_HF_PRIVATE=1`
37
-
38
- The script reads your local `.env` and injects those values into a Modal secret at runtime, so for local `modal run ...` usage you do not need to pre-create a hosted Modal secret just to run `prepare-data`, `train`, or `publish`.
39
-
40
- If you prefer a hosted secret for team/shared environments, you can still create one manually in Modal and adapt the script later.
41
-
42
- ## Data Inputs
43
-
44
- The script expects these local artifacts to already exist in the repo:
45
-
46
- - `data/sft/coach_summary_train.jsonl`
47
- - `data/sft/coach_summary_eval.jsonl`
48
- - `data/sft/public_fitness_style.jsonl`
49
- - `configs/coach_summary_lora.default.json`
50
-
51
- These are copied into the Modal data volume during `prepare-data`.
52
-
53
- ## Stage Commands
54
-
55
- Prepare training data:
56
-
57
- ```bash
58
- modal run scripts/coach_summary_modal.py --stage prepare-data
59
- ```
60
-
61
- Train the LoRA adapter:
62
-
63
- ```bash
64
- modal run scripts/coach_summary_modal.py --stage train --epochs 2 --style-weight 0.2
65
- ```
66
-
67
- Evaluate the trained adapter:
68
-
69
- ```bash
70
- modal run scripts/coach_summary_modal.py --stage evaluate --limit 5
71
- ```
72
-
73
- Publish the adapter and metadata to Hugging Face:
74
-
75
- ```bash
76
- modal run scripts/coach_summary_modal.py --stage publish --repo-id build-small-hackathon/pozify-coach-summary
77
- ```
78
-
79
- Merge the LoRA adapter into a full Transformers checkpoint:
80
-
81
- ```bash
82
- modal run scripts/coach_summary_modal.py --stage merge
83
- ```
84
-
85
- Publish the merged inference-ready model to Hugging Face:
86
-
87
- ```bash
88
- modal run scripts/coach_summary_modal.py --stage publish-merged --repo-id build-small-hackathon/pozify-coach-summary
89
- ```
90
-
91
- If your Hugging Face token does not have permission to create or write under an organization
92
- namespace, publish to a personal repo instead:
93
-
94
- ```bash
95
- modal run scripts/coach_summary_modal.py --stage publish-merged --repo-id <your-username>/pozify-coach-summary
96
- ```
97
-
98
- You can also set this once in `.env`:
99
-
100
- ```bash
101
- POZIFY_COACH_SUMMARY_MODEL=<your-username>/pozify-coach-summary
102
- POZIFY_COACH_SUMMARY_MERGED_HF_REPO_ID=<your-username>/pozify-coach-summary
103
- ```
104
-
105
- Run the full pipeline end to end:
106
-
107
- ```bash
108
- modal run scripts/coach_summary_modal.py --stage all --epochs 2 --style-weight 0.2 --repo-id build-small-hackathon/pozify-coach-summary
109
- ```
110
-
111
- ## Artifacts
112
-
113
- The Modal model volume stores:
114
-
115
- - `adapter/`
116
- - `merged_model/`
117
- - `training_config.json`
118
- - `training_summary.json`
119
- - `evaluation.json`
120
- - `hf_upload.json`
121
- - `merge_summary.json`
122
- - `hf_merged_upload.json`
123
- - `README.md`
124
-
125
- ## Runtime Usage
126
-
127
- For cloud inference in the current Pozify app, point runtime to the merged full model repo:
128
-
129
- ```bash
130
- export POZIFY_COACH_SUMMARY_MODEL=build-small-hackathon/pozify-coach-summary
131
- ```
132
-
133
- `POZIFY_COACH_SUMMARY_ADAPTER_ID` is no longer used as the remote inference target. Keep it only for adapter-centric workflows such as local merge/evaluation.
134
-
135
- If you download the merged model locally instead, use:
136
-
137
- ```bash
138
- export POZIFY_COACH_SUMMARY_LOCAL_MODEL_DIR=/path/to/merged_model
139
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
scripts/exercise_router_modal.py CHANGED
@@ -55,13 +55,13 @@ image = (
55
  )
56
  .add_local_dir("src", "/root/src", copy=True)
57
  .add_local_file(
58
- "docs/huggingface-router-model-card.md",
59
- "/root/docs/huggingface-router-model-card.md",
60
  copy=True,
61
  )
62
  .add_local_file(
63
- "docs/exercise-router-training-report.md",
64
- "/root/docs/exercise-router-training-report.md",
65
  copy=True,
66
  )
67
  )
 
55
  )
56
  .add_local_dir("src", "/root/src", copy=True)
57
  .add_local_file(
58
+ "docs/22-router-huggingface-model-card.md",
59
+ "/root/docs/22-router-huggingface-model-card.md",
60
  copy=True,
61
  )
62
  .add_local_file(
63
+ "docs/20-router-training-report.md",
64
+ "/root/docs/20-router-training-report.md",
65
  copy=True,
66
  )
67
  )
scripts/upload_exercise_router_to_hf.py CHANGED
@@ -7,8 +7,8 @@ from huggingface_hub import HfApi
7
 
8
 
9
  DEFAULT_MODEL_DIR = Path("models/exercise_router/active")
10
- DEFAULT_MODEL_CARD = Path("docs/huggingface-router-model-card.md")
11
- DEFAULT_TRAINING_REPORT = Path("docs/exercise-router-training-report.md")
12
  ARTIFACT_FILENAMES = (
13
  "baseline.joblib",
14
  "router.joblib",
 
7
 
8
 
9
  DEFAULT_MODEL_DIR = Path("models/exercise_router/active")
10
+ DEFAULT_MODEL_CARD = Path("docs/22-router-huggingface-model-card.md")
11
+ DEFAULT_TRAINING_REPORT = Path("docs/20-router-training-report.md")
12
  ARTIFACT_FILENAMES = (
13
  "baseline.joblib",
14
  "router.joblib",