Update VocalRender model card and inference guide

README.md

@@ -2,20 +2,141 @@
 license: apache-2.0
 base_model: openbmb/VoxCPM2
 pipeline_tag: text-to-speech
+language:
+  - zh
 tags:
   - voxcpm
   - voxcpm2
-  - text-to-speech
+  - singing-voice-synthesis
+  - score-conditioned
+  - music
+  - arxiv:2607.27768
 ---
 
 # VocalRender
 
-VocalRender is a fine-tuned model based on
-[VoxCPM2](https://huggingface.co/openbmb/VoxCPM2).
+**VocalRender: Score-Native Singing Voice Synthesis for Real-World Composition**
 
-This repository contains two variants:
+- [Paper](https://arxiv.org/abs/2607.27768)
+- [Code and documentation](https://github.com/pymaster17/VocalRender)
 
-| Variant | Repository path |
-| --- | --- |
-| VocalRender | `VocalRender/` |
-| VocalRender-Pro | `VocalRender-Pro/` |
+VocalRender is a score-native singing voice synthesis (SVS) model built on
+[VoxCPM2](https://huggingface.co/openbmb/VoxCPM2). It synthesizes 48 kHz
+singing directly from lyrics, MIDI pitches, symbolic note values, and tempo,
+without requiring phoneme-level durations, an explicit duration predictor, or
+a time-aligned acoustic reference.
+
+![VocalRender overview](https://raw.githubusercontent.com/pymaster17/VocalRender/main/assets/intro.png)
+
+## How it works
+
+VocalRender combines three components:
+
+1. A score-native interleaved representation serializes BPM followed by each
+   lyric syllable and its associated `(pitch, note-value)` pairs. This retains
+   lyric-to-note alignment and supports melisma.
+2. An Audio VAE represents singing as compact continuous acoustic latents.
+3. An autoregressive diffusion model generates the latent sequence patch by
+   patch and decides when to stop. The VAE decoder renders the result as a
+   waveform.
+
+![VocalRender architecture](https://raw.githubusercontent.com/pymaster17/VocalRender/main/assets/structure.png)
+
+## Checkpoints
+
+This repository contains two variants with the same architecture and different
+training recipes:
+
+| Variant | Repository path | Approximate download |
+| --- | --- | ---: |
+| VocalRender | `VocalRender/` | 9.5 GB |
+| VocalRender-Pro | `VocalRender-Pro/` | 9.5 GB |
+
+Each directory contains the model weights, AudioVAE, model configuration, and
+extended SVS tokenizer required by the inference code.
+
+## Installation
+
+```bash
+git clone --recurse-submodules https://github.com/pymaster17/VocalRender.git
+cd VocalRender
+uv sync
+source .venv/bin/activate
+```
+
+## Download
+
+Download either checkpoint into the repository's `pretrained_models/`
+directory:
+
+```bash
+# VocalRender
+hf download pymaster/VocalRender \
+    --include "VocalRender/*" \
+    --local-dir pretrained_models
+
+# Or VocalRender-Pro
+hf download pymaster/VocalRender \
+    --include "VocalRender-Pro/*" \
+    --local-dir pretrained_models
+```
+
+## Quick inference
+
+The GitHub repository includes a ready-to-use symbolic score example:
+
+```bash
+python scripts/infer_vocalrender_svs_single.py \
+    --ckpt_dir pretrained_models/VocalRender \
+    --json_file examples/inference_input.json \
+    --item_name demo \
+    --output svs_output.wav
+```
+
+Use `pretrained_models/VocalRender-Pro` as `--ckpt_dir` to run the Pro model.
+Full generation requires a CUDA-capable compute node. The example runs without
+reference audio; see the
+[repository README](https://github.com/pymaster17/VocalRender#inference) for
+timbre-prompt and batch-inference options.
+
+## Input format
+
+```json
+{
+  "item_name": "demo",
+  "word": ["我", "的", "孤", "独"],
+  "pitch": [65, 64, 64, 65, 67, 65],
+  "note": ["<NOTE_8>", "<NOTE_32>", "<NOTE_16>", "<NOTE_16>", "<NOTE_16>", "<NOTE_8>"],
+  "pitch2word": [0, 1, 2, 2, 2, 3],
+  "bpm": 64
+}
+```
+
+`pitch2word[i]` identifies the lyric index associated with note `i`, allowing
+one lyric syllable to span multiple notes. `word_dur` and `pitch_dur` are
+optional fields used only for visualization and evaluation; they are not model
+inputs.
+
+## Limitations
+
+- The released checkpoints primarily target Mandarin Chinese singing.
+- Checkpoint loading and generation require substantial disk space, system
+  memory, and GPU memory.
+- Output quality depends on the musical validity and lyric-to-note alignment of
+  the input score.
+
+## Citation
+
+```bibtex
+@article{chen2026vocalrender,
+  title   = {VocalRender: Score-Native Singing Voice Synthesis for Real-World Composition},
+  author  = {Chen, Yukun and Wang, Tianrui and Mu, Zhaoxi and Yang, Xinyu and Chng, EngSiong},
+  journal = {arXiv preprint arXiv:2607.27768},
+  year    = {2026},
+  url     = {https://arxiv.org/abs/2607.27768}
+}
+```
+
+## License
+
+Apache License 2.0. See the repository's `LICENSE` file for details.