YanTianlong commited on
Commit
057d7fe
·
1 Parent(s): 316b35d

Record ASR smoke test success

Browse files
docs/dependency-inventory.md CHANGED
@@ -123,8 +123,8 @@ No `requirements.txt` found.
123
 
124
  | Model / asset | Source | Target location | Notes |
125
  | --- | --- | --- | --- |
126
- | Qwen ASR | `Qwen/Qwen3-ASR-1.7B` | Hugging Face/model cache or custom local model path | Loaded by `VoiceBridgeASRLoader`. |
127
- | Qwen forced aligner | `Qwen/Qwen3-ForcedAligner-0.6B` | Hugging Face/model cache or custom local model path | Loaded by `VoiceBridgeASRLoader`. |
128
  | VoxCPM2 | `openbmb/VoxCPM2` | persistent target `/data/voicegate_models/voxcpm/VoxCPM2`; ComfyUI link `ComfyUI/models/voxcpm/VoxCPM2` | RunningHub VoxCPM README documents the ComfyUI location. Approx. 4.6 GB. |
129
  | MelBand RoFormer | `Kijai/MelBandRoFormer_comfy` | persistent target `/data/voicegate_models/diffusion_models/MelBandRoFormer_comfy`; ComfyUI link `ComfyUI/models/diffusion_models/MelBandRoFormer_comfy` | Workflow references `MelBandRoFormer_comfy/MelBandRoformer_fp32.safetensors`. |
130
 
@@ -196,7 +196,10 @@ ComfyUI/
196
  | `-- ComfyUI_AudioTools/
197
  `-- models/
198
  |-- voxcpm/VoxCPM2/
199
- `-- diffusion_models/MelBandRoFormer_comfy/
 
 
 
200
  ```
201
 
202
  Model files should be downloaded at runtime or first startup with
 
123
 
124
  | Model / asset | Source | Target location | Notes |
125
  | --- | --- | --- | --- |
126
+ | Qwen ASR | `Qwen/Qwen3-ASR-1.7B` | persistent target `/data/voicegate_models/Qwen3-ASR/Qwen3-ASR-1.7B`; ComfyUI link root `ComfyUI/models/Qwen3-ASR` | Loaded by `VoiceBridgeASRLoader`. |
127
+ | Qwen forced aligner | `Qwen/Qwen3-ForcedAligner-0.6B` | persistent target `/data/voicegate_models/Qwen3-ASR/Qwen3-ForcedAligner-0.6B`; ComfyUI link root `ComfyUI/models/Qwen3-ASR` | Loaded by `VoiceBridgeASRLoader`. |
128
  | VoxCPM2 | `openbmb/VoxCPM2` | persistent target `/data/voicegate_models/voxcpm/VoxCPM2`; ComfyUI link `ComfyUI/models/voxcpm/VoxCPM2` | RunningHub VoxCPM README documents the ComfyUI location. Approx. 4.6 GB. |
129
  | MelBand RoFormer | `Kijai/MelBandRoFormer_comfy` | persistent target `/data/voicegate_models/diffusion_models/MelBandRoFormer_comfy`; ComfyUI link `ComfyUI/models/diffusion_models/MelBandRoFormer_comfy` | Workflow references `MelBandRoFormer_comfy/MelBandRoformer_fp32.safetensors`. |
130
 
 
196
  | `-- ComfyUI_AudioTools/
197
  `-- models/
198
  |-- voxcpm/VoxCPM2/
199
+ |-- diffusion_models/MelBandRoFormer_comfy/
200
+ `-- Qwen3-ASR/
201
+ |-- Qwen3-ASR-1.7B/
202
+ `-- Qwen3-ForcedAligner-0.6B/
203
  ```
204
 
205
  Model files should be downloaded at runtime or first startup with
docs/hf-space-comfyui-step-by-step.zh.md CHANGED
@@ -131,6 +131,9 @@ python scripts/bootstrap_comfy.py --with-models
131
  ```text
132
  ComfyUI/models/voxcpm/VoxCPM2
133
  -> /data/project_models/voxcpm/VoxCPM2
 
 
 
134
  ```
135
 
136
  ## 6. 写最小 Gradio 入口
@@ -345,6 +348,27 @@ Gradio 进程和后续子进程可能看到不同 torch 版本。
345
  - 尽量复用模型进程。
346
  - 先做子流程验证,再优化 UX。
347
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
348
  ## 13. 最终正式化建议
349
 
350
  诊断阶段可以把很多按钮放在 `app.py` 里。正式阶段建议拆分:
 
131
  ```text
132
  ComfyUI/models/voxcpm/VoxCPM2
133
  -> /data/project_models/voxcpm/VoxCPM2
134
+
135
+ ComfyUI/models/Qwen3-ASR
136
+ -> /data/project_models/Qwen3-ASR
137
  ```
138
 
139
  ## 6. 写最小 Gradio 入口
 
348
  - 尽量复用模型进程。
349
  - 先做子流程验证,再优化 UX。
350
 
351
+ ### ZeroGPU duration 和配额预检
352
+
353
+ `@spaces.GPU(duration=...)` 不要随手写得太大。VoiceGate 的 ASR 测试曾经
354
+ 使用 `duration=1800`,被 Space 直接拒绝:
355
+
356
+ ```text
357
+ ZeroGPU illegal duration
358
+ ```
359
+
360
+ 后来改成 `duration=1200`,又因为剩余额度不足被配额预检拒绝:
361
+
362
+ ```text
363
+ You have exceeded your Pro ZeroGPU quota
364
+ ```
365
+
366
+ 最终 ASR 诊断用 `duration=900` 跑通。建议做法:
367
+
368
+ - 先用短输入和较小 duration 验证链路。
369
+ - 模型下载放到非 GPU 的 Prepare 阶段。
370
+ - 只有确认真实执行时间后,再提高 duration。
371
+
372
  ## 13. 最终正式化建议
373
 
374
  诊断阶段可以把很多按钮放在 `app.py` 里。正式阶段建议拆分:
docs/todo.md CHANGED
@@ -109,7 +109,7 @@ from scaffold to a working Gradio app.
109
  - [x] Run a MelBand RoFormer smoke test.
110
  - [x] Run a MelBand RoFormer smoke test inside ZeroGPU.
111
  - [x] Run a tiny TTS-only workflow inside ZeroGPU.
112
- - [ ] Run a short ASR-only workflow.
113
  - [ ] Run SRT split -> VoxCPM -> SRT merge.
114
  - [ ] Run the full short-audio VoiceGate workflow.
115
  - [ ] Confirm output audio and SRT files are downloadable from Gradio.
@@ -127,6 +127,8 @@ from scaffold to a working Gradio app.
127
  - [x] Which exact node repository provides `RH_LLMAPI_NODE`?
128
  - [x] Which exact node repository provides `RunningHub_VoxCPM_*`?
129
  - [ ] Is `flash_attention_2` available and reliable in the ZeroGPU environment?
 
 
130
  - [ ] Does VoxCPM2 fit comfortably in ZeroGPU memory with ASR and
131
  MelBandRoFormer in the same run?
132
  - [x] Where should large model files live? `/data/voicegate_models`, with
 
109
  - [x] Run a MelBand RoFormer smoke test.
110
  - [x] Run a MelBand RoFormer smoke test inside ZeroGPU.
111
  - [x] Run a tiny TTS-only workflow inside ZeroGPU.
112
+ - [x] Run a short ASR-only workflow inside ZeroGPU.
113
  - [ ] Run SRT split -> VoxCPM -> SRT merge.
114
  - [ ] Run the full short-audio VoiceGate workflow.
115
  - [ ] Confirm output audio and SRT files are downloadable from Gradio.
 
127
  - [x] Which exact node repository provides `RH_LLMAPI_NODE`?
128
  - [x] Which exact node repository provides `RunningHub_VoxCPM_*`?
129
  - [ ] Is `flash_attention_2` available and reliable in the ZeroGPU environment?
130
+ - [x] Can ASR run without `flash_attention_2`? Yes. The ASR-only smoke test
131
+ used `attention=sdpa`.
132
  - [ ] Does VoxCPM2 fit comfortably in ZeroGPU memory with ASR and
133
  MelBandRoFormer in the same run?
134
  - [x] Where should large model files live? `/data/voicegate_models`, with
docs/work-log.md CHANGED
@@ -12,7 +12,7 @@ and how they were resolved.
12
  - Space SDK: Gradio
13
  - Local Space wrapper repo: `VoiceGate-hf`
14
  - Local upstream reference checkout: `VoiceGate/`
15
- - Latest confirmed normal runtime commit: `6e4cd3f53d33e87979bdc814e0f1c34e0860f562`
16
  - Current expected Space secret: `DEEPSEEK_API_KEY`
17
  - Default persistent model root: `/data/voicegate_models`
18
 
@@ -35,12 +35,12 @@ Confirmed working:
35
  - DeepSeek-compatible LLM node with the Space secret.
36
  - MelBand RoFormer smoke tests in CPU mode and ZeroGPU mode.
37
  - VoxCPM2 TTS-only smoke test in ZeroGPU mode.
38
- - Persistent model storage for VoxCPM2 and MelBand under `/data`.
 
 
39
 
40
  Not yet confirmed at the start of 2026-06-06:
41
 
42
- - VoiceBridge ASR-only workflow.
43
- - Qwen ASR and Qwen forced aligner model download/cache behavior in the Space.
44
  - SRT split -> VoxCPM -> SRT merge.
45
  - Full short-audio VoiceGate workflow.
46
  - Final user-facing Gradio upload/download UI.
@@ -265,6 +265,37 @@ This is workable for diagnostics, but final production should avoid heavy
265
  runtime `pip install` where possible. Prefer moving stable dependencies into
266
  Space build-time requirements or explicitly controlling pins.
267
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
268
  ## Dependency Pitfalls and Fixes
269
 
270
  `ComfyUI_AudioTools` initially failed to import.
@@ -558,6 +589,71 @@ run was very slow. The final app should minimize cold starts, avoid repeated
558
  ComfyUI/model reloads where possible, and use shorter diagnostic prompts while
559
  tuning.
560
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
561
  ## Secrets and API Keys
562
 
563
  `DEEPSEEK_API_KEY` should be stored only as a Hugging Face Space Secret.
@@ -609,6 +705,9 @@ ComfyUI/models/voxcpm/VoxCPM2
609
 
610
  ComfyUI/models/diffusion_models/MelBandRoFormer_comfy
611
  -> /data/voicegate_models/diffusion_models/MelBandRoFormer_comfy
 
 
 
612
  ```
613
 
614
  Override the root with:
@@ -624,6 +723,8 @@ persistent storage:
624
  /data/voicegate_models/voxcpm/VoxCPM2/model.safetensors
625
  /data/voicegate_models/voxcpm/VoxCPM2/audiovae.pth
626
  /data/voicegate_models/diffusion_models/MelBandRoFormer_comfy/MelBandRoformer_fp32.safetensors
 
 
627
  ```
628
 
629
  Verified symlinks:
@@ -634,6 +735,9 @@ Verified symlinks:
634
 
635
  /home/user/app/ComfyUI/models/diffusion_models/MelBandRoFormer_comfy
636
  -> /data/voicegate_models/diffusion_models/MelBandRoFormer_comfy
 
 
 
637
  ```
638
 
639
  `DEEPSEEK_API_KEY` was also verified as present in the Space environment without
@@ -655,23 +759,15 @@ Model download pitfall:
655
  - `d849d03` Record ComfyUI API smoke test
656
  - `79b8b37` Add GPU smoke test button
657
  - `6e4cd3f` Run Space preparation in background
 
 
658
 
659
  ## Remaining Work
660
 
661
  Next recommended steps:
662
 
663
- 1. Verify the `DEEPSEEK_API_KEY` secret is visible to the Space process without
664
- printing its value.
665
- 2. Add model download or model presence checks for:
666
- - VoxCPM2
667
- - MelBand RoFormer
668
- - Qwen ASR
669
- - Qwen forced aligner
670
- 3. Wire `app.py` to bootstrap and start ComfyUI from the Gradio process.
671
- 4. Wrap the actual VoiceGate execution path in `@spaces.GPU`.
672
- 5. Run progressively larger workflows:
673
- - minimal API workflow
674
- - TTS-only
675
- - ASR-only
676
  - SRT split and merge
677
  - full short-audio VoiceGate workflow
 
 
 
12
  - Space SDK: Gradio
13
  - Local Space wrapper repo: `VoiceGate-hf`
14
  - Local upstream reference checkout: `VoiceGate/`
15
+ - Latest confirmed normal runtime commit: `316b35db739d74d05543d6c8c9dd9c16e0580b17`
16
  - Current expected Space secret: `DEEPSEEK_API_KEY`
17
  - Default persistent model root: `/data/voicegate_models`
18
 
 
35
  - DeepSeek-compatible LLM node with the Space secret.
36
  - MelBand RoFormer smoke tests in CPU mode and ZeroGPU mode.
37
  - VoxCPM2 TTS-only smoke test in ZeroGPU mode.
38
+ - VoiceBridge ASR-only smoke test in ZeroGPU mode.
39
+ - Persistent model storage for VoxCPM2, MelBand, Qwen3-ASR, and Qwen3 forced
40
+ aligner under `/data`.
41
 
42
  Not yet confirmed at the start of 2026-06-06:
43
 
 
 
44
  - SRT split -> VoxCPM -> SRT merge.
45
  - Full short-audio VoiceGate workflow.
46
  - Final user-facing Gradio upload/download UI.
 
265
  runtime `pip install` where possible. Prefer moving stable dependencies into
266
  Space build-time requirements or explicitly controlling pins.
267
 
268
+ ### ZeroGPU Duration and Quota Pitfall
269
+
270
+ The ASR diagnostic was first decorated with:
271
+
272
+ ```python
273
+ @spaces.GPU(duration=1800)
274
+ ```
275
+
276
+ The Space rejected it before execution:
277
+
278
+ ```text
279
+ ZeroGPU illegal duration
280
+ The requested GPU duration is larger than the maximum allowed
281
+ ```
282
+
283
+ After reducing the function to `duration=1200`, the Space still rejected the
284
+ call because the quota precheck reported:
285
+
286
+ ```text
287
+ You have exceeded your Pro ZeroGPU quota (1800s requested vs. 1389s left)
288
+ ```
289
+
290
+ The working diagnostic used:
291
+
292
+ ```python
293
+ @spaces.GPU(duration=900)
294
+ ```
295
+
296
+ For future tests, keep diagnostic durations conservative and increase only when
297
+ the workflow has already proven it needs more time.
298
+
299
  ## Dependency Pitfalls and Fixes
300
 
301
  `ComfyUI_AudioTools` initially failed to import.
 
589
  ComfyUI/model reloads where possible, and use shorter diagnostic prompts while
590
  tuning.
591
 
592
+ ### VoiceBridge ASR-only
593
+
594
+ On 2026-06-06, a minimal VoiceBridge ASR-only workflow was run from the normal
595
+ Gradio runtime inside `@spaces.GPU`.
596
+
597
+ Before running ASR, `scripts/bootstrap_comfy.py` was extended so Qwen ASR models
598
+ also live on persistent storage:
599
+
600
+ ```text
601
+ /home/user/app/ComfyUI/models/Qwen3-ASR
602
+ -> /data/voicegate_models/Qwen3-ASR
603
+ ```
604
+
605
+ The model preparation downloads:
606
+
607
+ ```text
608
+ /data/voicegate_models/Qwen3-ASR/Qwen3-ASR-1.7B
609
+ /data/voicegate_models/Qwen3-ASR/Qwen3-ForcedAligner-0.6B
610
+ ```
611
+
612
+ Minimal workflow:
613
+
614
+ ```text
615
+ LoadAudio
616
+ -> VoiceBridgeASRLoader(attention=sdpa, forced_aligner=Qwen/Qwen3-ForcedAligner-0.6B)
617
+ -> VoiceBridgeASRTranscribe(return_timestamps=True)
618
+ -> GenerateSRT
619
+ -> easy showAnything
620
+ ```
621
+
622
+ Input:
623
+
624
+ ```text
625
+ D:\voicebridge-test-audio\test_audio\2-坤哥.MP3
626
+ ```
627
+
628
+ Result:
629
+
630
+ ```text
631
+ status_str=success
632
+ completed=True
633
+ elapsed_sec=62.4
634
+ ```
635
+
636
+ Returned SRT text:
637
+
638
+ ```text
639
+ 1
640
+ 00:00:02,080 --> 00:00:03,200
641
+ 全民制作人们 大家好
642
+
643
+ 2
644
+ 00:00:03,439 --> 00:00:06,160
645
+ 我是练习时长两年半的个人练习生蔡徐坤
646
+
647
+ 3
648
+ 00:00:06,480 --> 00:00:09,359
649
+ 喜欢唱、跳、rap、篮球、music
650
+ ```
651
+
652
+ This confirms the Qwen3-ASR model, forced aligner, VoiceBridge ASR nodes, and
653
+ SRT generation can run in the Space ZeroGPU path. The smoke test intentionally
654
+ used `attention=sdpa` instead of `flash_attention_2`; `flash_attention_2`
655
+ availability remains unverified.
656
+
657
  ## Secrets and API Keys
658
 
659
  `DEEPSEEK_API_KEY` should be stored only as a Hugging Face Space Secret.
 
705
 
706
  ComfyUI/models/diffusion_models/MelBandRoFormer_comfy
707
  -> /data/voicegate_models/diffusion_models/MelBandRoFormer_comfy
708
+
709
+ ComfyUI/models/Qwen3-ASR
710
+ -> /data/voicegate_models/Qwen3-ASR
711
  ```
712
 
713
  Override the root with:
 
723
  /data/voicegate_models/voxcpm/VoxCPM2/model.safetensors
724
  /data/voicegate_models/voxcpm/VoxCPM2/audiovae.pth
725
  /data/voicegate_models/diffusion_models/MelBandRoFormer_comfy/MelBandRoformer_fp32.safetensors
726
+ /data/voicegate_models/Qwen3-ASR/Qwen3-ASR-1.7B
727
+ /data/voicegate_models/Qwen3-ASR/Qwen3-ForcedAligner-0.6B
728
  ```
729
 
730
  Verified symlinks:
 
735
 
736
  /home/user/app/ComfyUI/models/diffusion_models/MelBandRoFormer_comfy
737
  -> /data/voicegate_models/diffusion_models/MelBandRoFormer_comfy
738
+
739
+ /home/user/app/ComfyUI/models/Qwen3-ASR
740
+ -> /data/voicegate_models/Qwen3-ASR
741
  ```
742
 
743
  `DEEPSEEK_API_KEY` was also verified as present in the Space environment without
 
759
  - `d849d03` Record ComfyUI API smoke test
760
  - `79b8b37` Add GPU smoke test button
761
  - `6e4cd3f` Run Space preparation in background
762
+ - `b39ef30` Add ASR diagnostic workflow and deployment guide
763
+ - `316b35d` Reduce ASR ZeroGPU duration
764
 
765
  ## Remaining Work
766
 
767
  Next recommended steps:
768
 
769
+ 1. Run progressively larger workflows:
 
 
 
 
 
 
 
 
 
 
 
 
770
  - SRT split and merge
771
  - full short-audio VoiceGate workflow
772
+ 2. Replace the diagnostic panel with the first real Gradio upload/download UI.
773
+ 3. Reduce runtime dependency installation and model reload overhead.