Spaces:
Sleeping
Sleeping
File size: 9,459 Bytes
f4b8912 b3d9dcd 69ba7ff f4b8912 9c7e38e 69ba7ff | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 | ---
title: Classical Music Generator
emoji: πΌ
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: "4.44.1"
python_version: '3.8'
app_file: app.py
pinned: false
---
# Classical Music Generator
A Transformer-based neural network system for generating multi-instrument classical music using a grid-based representation. This project uses PyTorch and provides an interactive Gradio web interface with interactive music composition controls.
## Overview
Classical Music Generator is an AI-powered music composition tool that generates original classical music arrangements across four instrument families: Strings, Woodwinds, Brass, and Percussion. The system uses a Transformer encoder with per-instrument prediction heads to generate music sequences quantized on a fixed 8-step-per-bar grid at 120 BPM.
### Key Features
- **Multi-instrument composition**: Generates music for Strings, Woodwinds, Brass, and Percussion simultaneously
- **Grid-based representation**: Music is represented as a quantized 8-step-per-bar grid for precise control
- **Pitch control**: Melodic instruments limited to one octave (C4βB4, 12 pitches), percussion mapped to standard drum sounds
- **Transformer architecture**: Uses attention-based Transformer encoder with positional encoding for sequential music generation
- **Interactive web UI**: Gradio-based interface with real-time audio synthesis and visual feedback
- **Music theory constraints**: Built-in support for scale enforcement and note limiting to ensure musical coherence
- **MIDI support**: Full MIDI input/output for compatibility with DAWs and other music software
## Architecture
### Model Details
The core model is a **MusicTransformer** with the following components:
- **Input layer**: Projects 39-dimensional multi-hot grid (12 strings + 12 woodwinds + 12 brass + 3 percussion pitches) to the model embedding
- **Positional encoding**: Sinusoidal positional encoding for temporal sequence information
- **Transformer encoder**:
- Hidden size: 256 dimensions
- Layers: 3 transformer encoder layers
- Attention heads: 4
- Feed-forward dimension: 512
- Dropout: 0.1
- **Output heads**: 4 separate linear heads for per-instrument pitch prediction
- Strings head: 12 pitches
- Woodwinds head: 12 pitches
- Brass head: 12 pitches
- Percussion head: 3 pitches (Kick, Snare, Hi-hat)
### Grid-Based Representation
Music is represented as a sequence of time steps with the following quantization:
- **Tempo**: 120 BPM
- **Time signature**: 4/4
- **Steps per bar**: 8
- **Seconds per step**: 0.125 (125 ms)
- **Input context**: 3 bars (24 steps) of conditioning
- **Output**: Generation continues from seeded context
Each time step is represented as a 39-dimensional binary vector:
```
[strings (12) | woodwinds (12) | brass (12) | percussion (3)]
```
## Installation
### Prerequisites
- Python 3.8 or higher
- PyTorch 2.0+
- CUDA 11.8+ (optional, for GPU acceleration)
### Setup
1. Clone the repository:
```bash
git clone <repository-url>
cd ClassicalMusicGenerator
```
2. Install dependencies:
```bash
pip install -r requirements.txt
```
3. Install system dependencies for audio synthesis:
```bash
# macOS
brew install fluidsynth
# Ubuntu/Debian
sudo apt-get install fluidsynth
# Fedora
sudo dnf install fluidsynth
```
## Running the Project
### 1. Interactive Web Interface
Launch the Gradio-based interactive UI with visual grid controls:
```bash
python app.py
```
Then open your browser to `http://localhost:7860` (or the URL displayed in the terminal).
**Features**:
- Interactive 8-step grid editor for each instrument family
- Real-time audio preview via Web Audio synthesis
- Generate longer compositions with customizable length
- Visual feedback showing active notes and pitch controls
- Color-coded instrument families for easy identification
### 2. Training the Model
To train on your own MIDI dataset:
```bash
python train.py \
--dataset-path data/training_data.npz \
--num-epochs 20 \
--batch-size 32 \
--learning-rate 1e-3
```
**Options**:
- `--dataset-path`: Path to preprocessed dataset (NPZ or PT format)
- `--num-epochs`: Number of training epochs
- `--batch-size`: Batch size for training
- `--learning-rate`: Adam optimizer learning rate
- `--val-split`: Validation split ratio (default: 0.1)
- `--device`: Device to use ('cuda' or 'cpu', auto-detected by default)
- `--save-dir`: Directory to save model checkpoints (default: 'models')
**Resume training**:
The training script automatically resumes from checkpoints if they exist:
```bash
python train.py --dataset-path data/training_data.npz --extra-epochs 10
```
### 3. Data Preprocessing
Prepare MIDI files for training:
```bash
python data_preprocessing.py \
--midi-dir path/to/midi/files \
--output-path data/training_data.npz
```
This script:
- Loads MIDI files from a directory
- Maps instruments to 4 families (Strings, Woodwinds, Brass, Percussion)
- Quantizes notes to the fixed grid (120 BPM, 8 steps/bar)
- Creates input-output pairs for training
- Saves as NumPy NPZ format
### 4. Inference Scripts
For programmatic music generation:
```python
from inference import load_model, generate_midi_and_audio
import numpy as np
# Load pre-trained model
model = load_model("models/transformer_music_best.pt")
# Create seed grids (3 bars of context)
strings_seed = np.zeros((3, 8, 12), dtype=np.float32)
woodwinds_seed = np.zeros((3, 8, 12), dtype=np.float32)
brass_seed = np.zeros((3, 8, 12), dtype=np.float32)
perc_seed = np.zeros((3, 8, 3), dtype=np.float32)
# Generate 1 minute of music
midi_path, wav_path = generate_midi_and_audio(
strings_seed=strings_seed,
woodwinds_seed=woodwinds_seed,
brass_seed=brass_seed,
perc_seed=perc_seed,
target_minutes=1.0,
device="cpu"
)
```
## Project Structure
```
ClassicalMusicGenerator/
βββ app.py # Gradio web interface with grid controls
βββ config.py # Global configuration (grid, model, tempo)
βββ model.py # MusicTransformer architecture
βββ train.py # Training loop with checkpoint support
βββ inference.py # Model loading and generation utilities
βββ data_preprocessing.py # MIDI processing and dataset creation
βββ classicalmusicgenerator.py # Colab notebook (reference)
βββ test_grad.py # Gradient checking utilities
βββ requirements.txt # Python dependencies
βββ models/
β βββ transformer_music_best.pt # Pre-trained model weights
βββ utils/
βββ midi_utils.py # MIDI conversion utilities
βββ sampling_utils.py # Generation constraints and sampling
βββ __pycache__/
```
## Configuration
Edit [config.py](config.py) to customize:
### Grid Parameters
```python
STEPS_PER_BAR = 8 # Time steps per 4/4 bar
INPUT_BARS = 3 # Context bars for conditioning
TEMPO_BPM = 120 # Tempo in beats per minute
```
### Model Hyperparameters
```python
D_MODEL = 256 # Hidden dimension
N_LAYERS = 3 # Transformer layers
N_HEADS = 4 # Attention heads
FFN_DIM = 512 # Feed-forward dimension
DROPOUT = 0.1 # Dropout rate
```
### Generation Constraints
```python
BAR_SIMILARITY_THRESHOLD = 0.9 # Avoid repetitive sections
MAX_RESAMPLE_ATTEMPTS_PER_BAR = 5 # Resampling attempts per bar
MIN_ACTIVE_FAMILIES_PER_BAR = 2 # Minimum instrument families per bar
```
## Dataset Format
The preprocessing creates training data where:
- **X**: Input sequences of shape `[N, INPUT_STEPS, INPUT_FEATURES]` (context)
- **Y**: Target sequences of shape `[N, INPUT_STEPS, INPUT_FEATURES]` (next step prediction)
Where:
- `N` = number of sequences
- `INPUT_STEPS` = 24 (3 bars Γ 8 steps)
- `INPUT_FEATURES` = 39 (12+12+12+3 pitches)
Saved as NPZ or PyTorch PT format with keys `"X"` and `"Y"`.
## Training Tips
- **Dataset size**: Aim for at least 10,000-50,000 sequences for good results
- **Learning rate**: Start with 1e-3, reduce by 0.5Γ if loss plateaus
- **Batch size**: 32 or 64 depending on GPU memory
- **Validation split**: 10% is reasonable; increase to 15-20% for smaller datasets
- **Checkpointing**: Models are auto-saved at each epoch, best model tracked separately
## Troubleshooting
### CUDA Out of Memory
Reduce batch size or use CPU:
```bash
python train.py --dataset-path data/training_data.npz --batch-size 16 --device cpu
```
### Missing Audio Dependencies
Ensure fluidsynth is installed (see Installation section above).
### Gradio Connection Issues
If the web interface won't open, check the terminal output for the correct localhost address, typically `http://localhost:7860`.
## References
- Transformer architecture: "[Attention Is All You Need](https://arxiv.org/abs/1706.03762)" (Vaswani et al., 2017)
- Music representation: Grid-based quantization inspired by [MusicVAE](https://magenta.tensorflow.org/music-vae) and [Jukebox](https://openai.com/blog/jukebox/)
- Implementation uses [PyTorch](https://pytorch.org/), [Gradio](https://gradio.app/), and [pretty_midi](https://github.com/craffel/pretty_midi)
## Contributing
Contributions are welcome! Please:
1. Fork the repository
2. Create a feature branch
3. Commit your changes
4. Submit a pull request |