title: Hum
emoji: 🎵
colorFrom: indigo
colorTo: blue
sdk: docker
app_port: 7860
fullWidth: true
short_description: Turn a hummed melody into an instrument, or explore YuE2.
models:
- m-a-p/SheetSage2
- m-a-p/MERT-v2-FullSong
- m-a-p/YuE2-3B
- m-a-p/YuE2-Vae
Hum
Upload a short hum or record one in the browser. One request generates seven source-timed comparisons. SheetSage2 extracts notes and beats; FluidSynth uses the full FluidR3 GM bank for the selected instrument and drum kit, plus sampled electric guitar and fingered bass in clean and processed versions. The bass octave is chosen automatically to fit all notes into the pinned sample's measured MIDI range 26..45. If no single octave fits the whole phrase, the route uses the full GM Fingered Bass bank instead of dropping notes. Note starts and lengths are preserved. A continuous-F0 synth follows voice pitch and amplitude. The sampler uses full-size FreePats electric guitar and fingered bass SoundFonts, with FFmpeg compression, filtering, oversampled soft clipping, and a short guitar room echo in processed versions.
If SheetSage2 returns an empty vocal melody, a conservative pYIN fallback extracts source-timed notes from sustained voiced pitch. Spectral attacks can separate repeated syllables at the same pitch; stable abrupt pitch changes can separate legato notes without turning a smooth glide into repeated keypresses. It supplies a simple instrumental ABC for YuE2 and rejects unpitched noise. Fallback note boundaries remain approximate, and the run details identify which transcription was used. SheetSage2's beat grid remains the source for sampled drums when available.
YuE2 is on by default, so the same click also starts three experiments from SheetSage2 ABC: solo, melody-with-drums, and composition. Uncheck its option when only the faster sampler previews are wanted. The drums in these experiments are requested in the style prompt, not imposed as a separate symbolic track. YuE2 can introduce voices or other sounds and does not preserve source timing exactly. Sampler previews appear first; optional YuE2 previews then appear as each generation finishes.
Output audio is downloadable 320 kbit/s CBR MP3. This Space is public: uploads are processed on its Hugging Face server and results are served from temporary job files. The unchecked archive option can be enabled to keep the exact microphone recording or uploaded file in the owner's private Hugging Face Storage Bucket. Archived files use opaque IDs and have a JSON receipt with their SHA-256 hash and duration. They are not committed to the public Space repository or used to train models. Without the option, input files remain temporary. A failed archive write is reported in the run details and does not stop generation.
Current hardware request: A100 80 GB, one queued inference at a time, with 15-minute idle sleep. Models are downloaded at first use and cached on the Space's ephemeral disk. A cold restart can require another download.
Model licenses and source: SheetSage2, MERT-v2-FullSong, YuE2-3B, YuE2-Vae.
Sample library sources, both CC0 1.0: FreePats FSBS Clean Electric Guitar
(full SF2, release 2026-08-07; archive SHA-256
6d338afefe0efb629afd9ba28a5c8c129fc28b97ef1b57a31ebceb7bf0e3bf65),
and FreePats Finger Bass YR
(SF2, release 2019-09-30; archive SHA-256
f5b5cfe04435a9e9037d1f97d515dc596535acf26637db9579d724cd757f179e).
The Docker build verifies both archives before extraction.
This is a research prototype; a single owner-reviewed recording is development evidence, not a general accuracy guarantee.