Music_Generator / README.md
hephaestusonfire's picture
Update README.md
b3d9dcd verified
|
Raw History Blame Contribute Delete
9.46 kB

A newer version of the Gradio SDK is available: 6.32.0

Upgrade
metadata
title: Classical Music Generator
emoji: 🎼
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: 4.44.1
python_version: '3.8'
app_file: app.py
pinned: false

Classical Music Generator

A Transformer-based neural network system for generating multi-instrument classical music using a grid-based representation. This project uses PyTorch and provides an interactive Gradio web interface with interactive music composition controls.

Overview

Classical Music Generator is an AI-powered music composition tool that generates original classical music arrangements across four instrument families: Strings, Woodwinds, Brass, and Percussion. The system uses a Transformer encoder with per-instrument prediction heads to generate music sequences quantized on a fixed 8-step-per-bar grid at 120 BPM.

Key Features

  • Multi-instrument composition: Generates music for Strings, Woodwinds, Brass, and Percussion simultaneously
  • Grid-based representation: Music is represented as a quantized 8-step-per-bar grid for precise control
  • Pitch control: Melodic instruments limited to one octave (C4–B4, 12 pitches), percussion mapped to standard drum sounds
  • Transformer architecture: Uses attention-based Transformer encoder with positional encoding for sequential music generation
  • Interactive web UI: Gradio-based interface with real-time audio synthesis and visual feedback
  • Music theory constraints: Built-in support for scale enforcement and note limiting to ensure musical coherence
  • MIDI support: Full MIDI input/output for compatibility with DAWs and other music software

Architecture

Model Details

The core model is a MusicTransformer with the following components:

  • Input layer: Projects 39-dimensional multi-hot grid (12 strings + 12 woodwinds + 12 brass + 3 percussion pitches) to the model embedding
  • Positional encoding: Sinusoidal positional encoding for temporal sequence information
  • Transformer encoder:
    • Hidden size: 256 dimensions
    • Layers: 3 transformer encoder layers
    • Attention heads: 4
    • Feed-forward dimension: 512
    • Dropout: 0.1
  • Output heads: 4 separate linear heads for per-instrument pitch prediction
    • Strings head: 12 pitches
    • Woodwinds head: 12 pitches
    • Brass head: 12 pitches
    • Percussion head: 3 pitches (Kick, Snare, Hi-hat)

Grid-Based Representation

Music is represented as a sequence of time steps with the following quantization:

  • Tempo: 120 BPM
  • Time signature: 4/4
  • Steps per bar: 8
  • Seconds per step: 0.125 (125 ms)
  • Input context: 3 bars (24 steps) of conditioning
  • Output: Generation continues from seeded context

Each time step is represented as a 39-dimensional binary vector:

[strings (12) | woodwinds (12) | brass (12) | percussion (3)]

Installation

Prerequisites

  • Python 3.8 or higher
  • PyTorch 2.0+
  • CUDA 11.8+ (optional, for GPU acceleration)

Setup

  1. Clone the repository:
git clone <repository-url>
cd ClassicalMusicGenerator
  1. Install dependencies:
pip install -r requirements.txt
  1. Install system dependencies for audio synthesis:
# macOS
brew install fluidsynth

# Ubuntu/Debian
sudo apt-get install fluidsynth

# Fedora
sudo dnf install fluidsynth

Running the Project

1. Interactive Web Interface

Launch the Gradio-based interactive UI with visual grid controls:

python app.py

Then open your browser to http://localhost:7860 (or the URL displayed in the terminal).

Features:

  • Interactive 8-step grid editor for each instrument family
  • Real-time audio preview via Web Audio synthesis
  • Generate longer compositions with customizable length
  • Visual feedback showing active notes and pitch controls
  • Color-coded instrument families for easy identification

2. Training the Model

To train on your own MIDI dataset:

python train.py \
    --dataset-path data/training_data.npz \
    --num-epochs 20 \
    --batch-size 32 \
    --learning-rate 1e-3

Options:

  • --dataset-path: Path to preprocessed dataset (NPZ or PT format)
  • --num-epochs: Number of training epochs
  • --batch-size: Batch size for training
  • --learning-rate: Adam optimizer learning rate
  • --val-split: Validation split ratio (default: 0.1)
  • --device: Device to use ('cuda' or 'cpu', auto-detected by default)
  • --save-dir: Directory to save model checkpoints (default: 'models')

Resume training: The training script automatically resumes from checkpoints if they exist:

python train.py --dataset-path data/training_data.npz --extra-epochs 10

3. Data Preprocessing

Prepare MIDI files for training:

python data_preprocessing.py \
    --midi-dir path/to/midi/files \
    --output-path data/training_data.npz

This script:

  • Loads MIDI files from a directory
  • Maps instruments to 4 families (Strings, Woodwinds, Brass, Percussion)
  • Quantizes notes to the fixed grid (120 BPM, 8 steps/bar)
  • Creates input-output pairs for training
  • Saves as NumPy NPZ format

4. Inference Scripts

For programmatic music generation:

from inference import load_model, generate_midi_and_audio
import numpy as np

# Load pre-trained model
model = load_model("models/transformer_music_best.pt")

# Create seed grids (3 bars of context)
strings_seed = np.zeros((3, 8, 12), dtype=np.float32)
woodwinds_seed = np.zeros((3, 8, 12), dtype=np.float32)
brass_seed = np.zeros((3, 8, 12), dtype=np.float32)
perc_seed = np.zeros((3, 8, 3), dtype=np.float32)

# Generate 1 minute of music
midi_path, wav_path = generate_midi_and_audio(
    strings_seed=strings_seed,
    woodwinds_seed=woodwinds_seed,
    brass_seed=brass_seed,
    perc_seed=perc_seed,
    target_minutes=1.0,
    device="cpu"
)

Project Structure

ClassicalMusicGenerator/
β”œβ”€β”€ app.py                      # Gradio web interface with grid controls
β”œβ”€β”€ config.py                   # Global configuration (grid, model, tempo)
β”œβ”€β”€ model.py                    # MusicTransformer architecture
β”œβ”€β”€ train.py                    # Training loop with checkpoint support
β”œβ”€β”€ inference.py                # Model loading and generation utilities
β”œβ”€β”€ data_preprocessing.py        # MIDI processing and dataset creation
β”œβ”€β”€ classicalmusicgenerator.py   # Colab notebook (reference)
β”œβ”€β”€ test_grad.py                # Gradient checking utilities
β”œβ”€β”€ requirements.txt            # Python dependencies
β”œβ”€β”€ models/
β”‚   └── transformer_music_best.pt    # Pre-trained model weights
└── utils/
    β”œβ”€β”€ midi_utils.py           # MIDI conversion utilities
    β”œβ”€β”€ sampling_utils.py       # Generation constraints and sampling
    └── __pycache__/

Configuration

Edit config.py to customize:

Grid Parameters

STEPS_PER_BAR = 8              # Time steps per 4/4 bar
INPUT_BARS = 3                 # Context bars for conditioning
TEMPO_BPM = 120                # Tempo in beats per minute

Model Hyperparameters

D_MODEL = 256                  # Hidden dimension
N_LAYERS = 3                   # Transformer layers
N_HEADS = 4                    # Attention heads
FFN_DIM = 512                  # Feed-forward dimension
DROPOUT = 0.1                  # Dropout rate

Generation Constraints

BAR_SIMILARITY_THRESHOLD = 0.9          # Avoid repetitive sections
MAX_RESAMPLE_ATTEMPTS_PER_BAR = 5      # Resampling attempts per bar
MIN_ACTIVE_FAMILIES_PER_BAR = 2        # Minimum instrument families per bar

Dataset Format

The preprocessing creates training data where:

  • X: Input sequences of shape [N, INPUT_STEPS, INPUT_FEATURES] (context)
  • Y: Target sequences of shape [N, INPUT_STEPS, INPUT_FEATURES] (next step prediction)

Where:

  • N = number of sequences
  • INPUT_STEPS = 24 (3 bars Γ— 8 steps)
  • INPUT_FEATURES = 39 (12+12+12+3 pitches)

Saved as NPZ or PyTorch PT format with keys "X" and "Y".

Training Tips

  • Dataset size: Aim for at least 10,000-50,000 sequences for good results
  • Learning rate: Start with 1e-3, reduce by 0.5Γ— if loss plateaus
  • Batch size: 32 or 64 depending on GPU memory
  • Validation split: 10% is reasonable; increase to 15-20% for smaller datasets
  • Checkpointing: Models are auto-saved at each epoch, best model tracked separately

Troubleshooting

CUDA Out of Memory

Reduce batch size or use CPU:

python train.py --dataset-path data/training_data.npz --batch-size 16 --device cpu

Missing Audio Dependencies

Ensure fluidsynth is installed (see Installation section above).

Gradio Connection Issues

If the web interface won't open, check the terminal output for the correct localhost address, typically http://localhost:7860.

References

Contributing

Contributions are welcome! Please:

  1. Fork the repository
  2. Create a feature branch
  3. Commit your changes
  4. Submit a pull request