README.md

metadata

title: Kiku AI
emoji: 🎤
colorFrom: yellow
colorTo: red
sdk: docker
app_port: 7860
short_description: AI music studio that builds a band around your voice
models:
  - m-a-p/SheetSage2
  - m-a-p/YuE2-3B
  - ACE-Step/ACE-Step-v1-3.5B
tags:
  - audio-to-score
  - music-generation
  - text-to-music
  - music-transcription

Kiku AI

Record or upload a take, turn it into editable notation, and build a full song around it. Sign in with Hugging Face to use your own ZeroGPU allowance.

What it does

Sheet music (/score) turns a recording into a lead sheet you can play with a moving note highlight, edit, undo, restore and download as ABC, MIDI or PDF.

Studio (/, the page the Space opens on; /studio and /create redirect there) is a small multitrack DAW: tracks of audio clips on a timeline, a transport with loop and recording, a mixer with per-track volume, pan, mute, solo and meters, undo, and WAV export. Clips move, trim, split and duplicate on a beat grid.

Kiku AI works on whatever is selected:

• Select a clip, several clips, or drag across part of a clip (the range tool, R, or dragging on empty space) to restyle it, rewrite just that stretch, split it into stems or extend it. An edit inside one clip is spliced back in place with crossfades; anything spanning several clips lands on a new track, and the originals are muted rather than deleted.
• Band turns a vocal into a band that follows its beat and chords. If the take already has music behind it, the voice is lifted off first and the old backing's harmony is followed. The voice in the result is always yours. Drums, bass and keys arrive as separate tracks, and the project adopts the measured tempo, key and chords (shown in a chord lane).
• With nothing selected, the browser adds one instrument at a time — drums, bass, keys, guitar, strings, synth or a full band — written to the project's beat and chords and isolated to that instrument, or generates an instrumental or a song with (synthetic) vocals at the playhead.

Each AI action has a model picker: ACE-Step 1.5 (the default), YuE2 or ACE-Step 1. Models that cannot do the selected job are greyed out: YuE2 only writes songs with vocals, and cannot edit audio. With ACE-Step 1.5, a band is written by its complete task directly around the vocal, and a single instrument by its lego task directly into the project. ACE-Step 1 builds a guide track from the measured beat and chords instead. Stems are always Demucs.

Several jobs can run while you keep editing; each result is placed against the project as it is when the job finishes.

Score playback defaults to piano. The instrument selector also offers acoustic and electric guitar, acoustic bass, violin, cello, string ensemble, trumpet, saxophone, clarinet, and flute. This changes the playback sound of the score; it does not separate the recording into individual instrument parts.

The interface uses Supreme Regular and Bespoke Stencil Bold from Fontshare, follows your device's preferred theme until you choose one, and switches themes with an expanding circular wave. Reduced-motion preferences are respected.

Run locally

Requires Node 24 and FFmpeg/FFprobe on PATH. Run npm ci, configure values from env.example in .env.local, then npm run dev. Open http://localhost:3000. The development-only animation preview is at http://localhost:3000/preview/transcribe, and the studio without signing in (engine calls fail there) is at http://localhost:3000/preview/studio.

Hosting

The frontend runs on a CPU Docker Space. Inference runs on two public Gradio ZeroGPU Spaces, each called with the signed-in user's own token:

Space	Used for
Kiku-Engine	Transcription (SheetSage2), YuE2 songs, ACE-Step 1 editing and guide-track bands, Demucs stems
Kiku-ACE-Step-15	ACE-Step 1.5: songs, instrumentals, restyle, rewrite, extend, lego parts and complete bands

ACE-Step 1.5 needs Python 3.11, Gradio 6 and torch 2.9, which cannot share the engine's YuE2 stack, so it has its own Space. Both expose the same endpoint names, and the routes pick one from the model the user chose.

Hosting more than two ZeroGPU Spaces on one account requires PRO.

Before deploying, configure these Space settings:

• Secret SESSION_SECRET: 32 random bytes encoded as 64 hexadecimal characters.
• Variable APP_ORIGIN: https://reubencf-kiku-ai.hf.space.
• Optional HF_*_SPACE and HF_*_ORIGIN overrides for each engine; see env.example.

The OAuth client is published by the API Space, so every origin that signs users in must appear in its redirect_uris.

The Dockerfile includes FFmpeg and the standalone Next.js build. No owner Hugging Face token is required or used for public requests. The login flow uses Hugging Face's Client ID Metadata Documents and authorization code with PKCE, requesting only openid profile. User access tokens are encrypted in HttpOnly cookies, expire within eight hours, and are never returned to frontend JavaScript. Serve the app over HTTPS and keep the session secret stable across restarts. Login opens the app at its own origin so it does not depend on third-party cookies.

Every generation route requires a valid session and same-origin submission. Backend score downloads expire after 24 hours.

Limits

Audio for transcription must be 5 seconds to 10 minutes and under 100 MB. Audio sent from the studio must be 3 seconds to 4 minutes (stem splits up to 6 minutes). The whole recording is validated and submitted; longer input is rejected, not silently truncated.

A free Hugging Face account carries about 5 minutes of daily ZeroGPU time and a PRO account about 40, which is the real ceiling on how much anyone can generate per day. Songs with vocals cost far more of that budget than instrumentals.

A band follows the beat grid and chords measured from your take, not its phrasing, because ACE-Step's Singing2Accompaniment ControlNet is unreleased. Steady singing follows well; a take with no clear pulse falls back to matching its key only. Studio edits send at most four minutes of audio at once.

SheetSage2 produces melody and chord lead sheets, not a complete orchestral score. Review AI notation before performing. The model weights are CC BY-NC 4.0 (non-commercial). Fonts are distributed by Fontshare.

Checks

npm run typecheck and npm run build verify the production frontend. Before publishing, verify real Hugging Face login, full audio processing, score playback and editing, both themes, keyboard controls and reduced motion.