Spaces:
Running
Running
| title: Kiku AI | |
| emoji: 🎤 | |
| colorFrom: yellow | |
| colorTo: red | |
| sdk: docker | |
| app_port: 7860 | |
| short_description: AI music studio that builds a band around your voice | |
| models: | |
| - m-a-p/SheetSage2 | |
| - m-a-p/YuE2-3B | |
| - ACE-Step/ACE-Step-v1-3.5B | |
| tags: | |
| - audio-to-score | |
| - music-generation | |
| - text-to-music | |
| - music-transcription | |
| # Kiku AI | |
| Record or upload a take, turn it into editable notation, and build a full song | |
| around it. Sign in with Hugging Face to use your own ZeroGPU allowance. | |
| ## What it does | |
| **Sheet music** (`/score`) turns a recording into a lead sheet you can play with a | |
| moving note highlight, edit, undo, restore and download as ABC, MIDI or PDF. | |
| **Studio** (`/`, the page the Space opens on; `/studio` and `/create` redirect there) is a small multitrack DAW: tracks of audio | |
| clips on a timeline, a transport with loop and recording, a mixer with per-track volume, pan, | |
| mute, solo and meters, undo, and WAV export. Clips move, trim, split and duplicate on a beat grid. | |
| Kiku AI works on whatever is selected: | |
| - Select a clip, several clips, or drag across part of a clip (the range tool, `R`, or dragging | |
| on empty space) to **restyle** it, **rewrite** just that stretch, **split** it into stems or | |
| **extend** it. An edit inside one clip is spliced back in place with crossfades; anything | |
| spanning several clips lands on a new track, and the originals are muted rather than deleted. | |
| - **Band** turns a vocal into a band that follows its beat and chords. If the take already has | |
| music behind it, the voice is lifted off first and the old backing's harmony is followed. The | |
| voice in the result is always yours. Drums, bass and keys arrive as separate tracks, and the | |
| project adopts the measured tempo, key and chords (shown in a chord lane). | |
| - With nothing selected, the browser adds one instrument at a time — drums, bass, keys, guitar, | |
| strings, synth or a full band — written to the project's beat and chords and isolated to that | |
| instrument, or generates an instrumental or a song with (synthetic) vocals at the playhead. | |
| Each AI action has a model picker: **ACE-Step 1.5** (the default), **YuE2** or **ACE-Step 1**. | |
| Models that cannot do the selected job are greyed out: YuE2 only writes songs with vocals, and | |
| cannot edit audio. With ACE-Step 1.5, a band is written by its *complete* task directly around | |
| the vocal, and a single instrument by its *lego* task directly into the project. ACE-Step 1 | |
| builds a guide track from the measured beat and chords instead. Stems are always Demucs. | |
| Several jobs can run while you keep editing; each result is placed against the project as it is | |
| when the job finishes. | |
| Score playback defaults to piano. The instrument selector also offers acoustic | |
| and electric guitar, acoustic bass, violin, cello, string ensemble, trumpet, | |
| saxophone, clarinet, and flute. This changes the playback sound of the score; | |
| it does not separate the recording into individual instrument parts. | |
| The interface uses Supreme Regular and Bespoke Stencil Bold from Fontshare, | |
| follows your device's preferred theme until you choose one, and switches themes | |
| with an expanding circular wave. Reduced-motion preferences are respected. | |
| ## Run locally | |
| Requires Node 24 and FFmpeg/FFprobe on PATH. | |
| Run `npm ci`, configure values from `env.example` in `.env.local`, then `npm run dev`. | |
| Open http://localhost:3000. The development-only animation preview is at | |
| http://localhost:3000/preview/transcribe, and the studio without signing in (engine calls | |
| fail there) is at http://localhost:3000/preview/studio. | |
| ## Hosting | |
| The frontend runs on a CPU Docker Space. Inference runs on two public Gradio ZeroGPU Spaces, | |
| each called with the signed-in user's own token: | |
| | Space | Used for | | |
| | --- | --- | | |
| | [Kiku-Engine](https://huggingface.co/spaces/Reubencf/Kiku-Engine) | Transcription (SheetSage2), YuE2 songs, ACE-Step 1 editing and guide-track bands, Demucs stems | | |
| | [Kiku-ACE-Step-15](https://huggingface.co/spaces/Reubencf/Kiku-ACE-Step-15) | ACE-Step 1.5: songs, instrumentals, restyle, rewrite, extend, lego parts and complete bands | | |
| ACE-Step 1.5 needs Python 3.11, Gradio 6 and torch 2.9, which cannot share the engine's YuE2 | |
| stack, so it has its own Space. Both expose the same endpoint names, and the routes pick one | |
| from the model the user chose. | |
| Hosting more than two ZeroGPU Spaces on one account requires PRO. | |
| Before deploying, configure these Space settings: | |
| - Secret `SESSION_SECRET`: 32 random bytes encoded as 64 hexadecimal characters. | |
| - Variable `APP_ORIGIN`: `https://reubencf-kiku-ai.hf.space`. | |
| - Optional `HF_*_SPACE` and `HF_*_ORIGIN` overrides for each engine; see `env.example`. | |
| The OAuth client is published by the API Space, so every origin that signs users | |
| in must appear in its `redirect_uris`. | |
| The Dockerfile includes FFmpeg and the standalone Next.js build. | |
| No owner Hugging Face token is required or used for public requests. The login | |
| flow uses Hugging Face's Client ID Metadata Documents and authorization code | |
| with PKCE, requesting only `openid profile`. User access tokens are encrypted in | |
| HttpOnly cookies, expire within eight hours, and are never returned to frontend JavaScript. | |
| Serve the app over HTTPS and keep the session secret stable across restarts. | |
| Login opens the app at its own origin so it does not depend on third-party cookies. | |
| Every generation route requires a valid session and same-origin submission. | |
| Backend score downloads expire after 24 hours. | |
| ## Limits | |
| Audio for transcription must be 5 seconds to 10 minutes and under 100 MB. Audio sent | |
| from the studio must be 3 seconds to 4 minutes (stem splits up to 6 minutes). The whole recording is | |
| validated and submitted; longer input is rejected, not silently truncated. | |
| A free Hugging Face account carries about 5 minutes of daily ZeroGPU time and a | |
| PRO account about 40, which is the real ceiling on how much anyone can generate | |
| per day. Songs with vocals cost far more of that budget than instrumentals. | |
| A band follows the beat grid and chords measured from your take, not its phrasing, because | |
| ACE-Step's Singing2Accompaniment ControlNet is unreleased. Steady singing follows well; a take | |
| with no clear pulse falls back to matching its key only. Studio edits send at most four minutes | |
| of audio at once. | |
| SheetSage2 produces melody and chord lead sheets, not a complete orchestral score. | |
| Review AI notation before performing. The model weights are CC BY-NC 4.0 | |
| (non-commercial). Fonts are distributed by [Fontshare](https://www.fontshare.com/licenses). | |
| ## Checks | |
| `npm run typecheck` and `npm run build` verify the production frontend. | |
| Before publishing, verify real Hugging Face login, full audio processing, | |
| score playback and editing, both themes, keyboard controls and reduced motion. | |