PitchIn / README.md
Hayeonbang's picture
Add PitchIn-Q free ZeroGPU demo with verified adapter
df72268 verified
|
Raw History Blame Contribute Delete
2.1 kB
---
title: PitchIn
emoji: 🎙️
colorFrom: green
colorTo: blue
sdk: gradio
sdk_version: 5.49.1
python_version: 3.12
app_file: app.py
pinned: false
short_description: Ask questions about your singing with PitchIn-Q
---
# PitchIn
Upload a short singing recording and ask a question. PitchIn-Q generates English
coaching feedback using Qwen2.5-Omni-7B adapted on Focused QA and Overall Feedback.
This initial research demo accepts 1–60 seconds of audio and questions up to
1,000 characters. Each request is independent. The model may miss or misidentify
issues. This interactive demo is not a benchmark evaluation.
## Deployment
Use **Gradio + ZeroGPU**. Gradio and `spaces` are provided by the Space runtime.
The deployment runtime supplies PyTorch 2.13; torchvision 0.28 matches it and is
needed by the multimodal processor even for audio-only requests. The local
smoke-test environment uses the existing research PyTorch 2.7 / torchvision 0.22.
The Space stores the 617 MiB PitchIn-Q adapter in `adapter/` and the exact
paper processor in `processor/`. It downloads the public base model pinned to
revision `ae9e1690543ffd5c0221dc27f79834d0294cba00`, merges the adapter at startup,
and moves the model to CUDA before requests. No account token is needed by the app.
For local use, `PITCHIN_MODEL_PATH` can point to the existing merged checkpoint;
alternatively set `PITCHIN_BASE_MODEL`, `PITCHIN_ADAPTER_PATH`, and
`PITCHIN_PROCESSOR_PATH` to local directories. Inference is text-only, BF16,
SDPA, with the checkpoint's native user audio/text chat template. No extra
system prompt or vocal separation is applied. Sampling is temperature 0.6,
top-p 0.85, top-k 30, repetition penalty 1.08, and at most 384 output tokens.
The 60-second limit applies only to this demo, not to paper evaluation inputs.
For a local UI-only preview, set `PITCHIN_PREVIEW_ONLY=1`. It is visibly labeled
and refuses inference; it never supplies simulated model responses.
Set `GRADIO_SERVER_NAME=0.0.0.0` in the Space environment. Locally the app binds
only to loopback unless explicitly configured otherwise.