keshavbhandari/modulator

🎹 Change is Key: A Generative Framework for Controllable Musical Modulations

This repository contains the official implementation for "Change is Key: A Generative Framework for Controllable Musical Modulations".

The framework formulates musical modulation as a generative Fill-In-The-Middle (FIM) task using a LLaMA-based architecture. It allows users to select a structural turning point in a MIDI composition, transpose subsequent material to a target key, and autonomously or interactively synthesize a smooth, musically coherent transition bridge.

⚙️ Installation

First, clone this repository and navigate into the modulator folder:

git clone https://github.com/keshavbhandari/modulator.git
cd modulator

Set up your environment using Conda:

module load cuda/12.4.0-gcc-12.2.0 # Optional: For HPC clusters
conda create -n modulate python=3.11 -y
conda activate modulate
conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia -y

Install the project and its dependencies:

pip install -e .

🚀 Quickstart & Usage

The primary script for users is modulate_midi.py. It automatically fetches our pre-trained model weights directly from Hugging Face (keshavbhandari/modulator).

Run the commands below from the root modulator folder.

1. Autonomous Infilling (No Chord Conditioning)

The model relies strictly on left/right musical context and temporal boundary tokens to autonomously bridge into the new key.

python modulate/modulate_midi.py \
--midi_file path/to/input.mid \
--output_path path/to/output.mid \
--semitone_shift 4 \
--chord_mode none

2. Rule-Guided Modulation (Dynamic Chord Baseline)

Condition generation on a dynamically calculated functional chord progression that back-cycles diatonic chords to resolve smoothly into the target key.

python modulate/modulate_midi.py \
--midi_file path/to/input.mid \
--output_path path/to/output.mid \
--semitone_shift 5 \
--chord_mode dynamic

3. LLM-Guided Modulation (Groq API)

Use a Large Language Model (via the Groq API) to plan custom harmonic progression instructions for the generative bridge based on the surrounding harmonic context.

python modulate/modulate_midi.py \
--midi_file path/to/input.mid \
--output_path path/to/output.mid \
--semitone_shift 2 \
--chord_mode llm \
--api_key YOUR_GROQ_API_KEY

4. Optimal Modulation Boundary Detection

Automatically analyze the harmony to identify a structural turning point (e.g., resolving to the tonic) rather than cutting at a generic midpoint.

python modulate/modulate_midi.py \
--midi_file path/to/input.mid \
--output_path path/to/output.mid \
--enable_optimal_modulation_point \
--gap_duration 10000

(Note: --gap_duration is specified in milliseconds. 10000 = 10 seconds).

5. Manual Timestamp Control (Milliseconds)

Explicitly define exact start and end boundaries (in milliseconds) for the left context, the generative gap, and the right context.

python modulate/modulate_midi.py \
--midi_file path/to/input.mid \
--output_path path/to/output.mid \
--left_segment_start_time 70000 \
--left_segment_end_time 110000 \
--right_segment_start_time 117000 \
--right_segment_end_time 140000 \
--semitone_shift 7

6. Simple Infilling (Zero Key Change)

Repair or infill missing segments without transposing the target key by setting the semitone shift to 0.

python modulate/modulate_midi.py \
--midi_file path/to/input.mid \
--output_path path/to/output.mid \
--semitone_shift 0 \
--chord_mode none

🎛️ Script Arguments Reference

Argument	Default	Description
--midi_file	Required	Path to the input .mid file.
--output_path	infilled_output.mid	Target save location for output MIDI.
--semitone_shift	4	Semitone transposition for target key (0 for simple infilling).
--chord_mode	none	Conditioning strategy: none, dynamic, or llm.
--api_key	None	Groq API key (required when --chord_mode llm).
--enable_optimal_modulation_point	False	Analyzes harmony to pick the best structural cut point.
--left_segment_start_time	None	Manual left segment start (ms).
--left_segment_end_time	None	Manual left segment end / gap start (ms).
--right_segment_start_time	None	Manual gap end / right segment start (ms).
--right_segment_end_time	None	Manual right segment end (ms).
--left_segment_duration	30000	Fallback left context duration (ms).
--right_segment_duration	30000	Fallback right context duration (ms).
--gap_duration	5000	Fallback generated bridge duration (ms).
--temperature	1.0	Sampling temperature for the LLaMA model generation.

🧠 Model Training

If you would like to continue training the model or train it on your own custom datasets, you can launch the training script using Accelerate:

accelerate launch --num_processes 1 /data/home/acw769/modulator/modulate/models/infiller_llama/train.py

📜 Citation

If you use this code, model weights, or evaluation framework in your research, please cite our paper:

@article{bhandari2026change,
title={Change is Key: A Generative Framework for Controllable Musical Modulations},
author={Bhandari, Keshav and Bizzarri, Matteo and Wiggins, Geraint A. and Colton, Simon},
year={2026}
}