Spaces:
Sleeping
Download README.md from hephaestusonfire/Music_Generator: direct link, hf CLI and curl.
- Browser
- Download file 9.46 kB
-
https://huggingface.co/spaces/hephaestusonfire/Music_Generator/resolve/main/README.md
- Command line
-
hf download hf://spaces/hephaestusonfire/Music_Generator/README.md
-
curl -L -o README.md https://huggingface.co/spaces/hephaestusonfire/Music_Generator/resolve/main/README.md
A newer version of the Gradio SDK is available: 6.32.0
title: Classical Music Generator
emoji: πΌ
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: 4.44.1
python_version: '3.8'
app_file: app.py
pinned: false
Classical Music Generator
A Transformer-based neural network system for generating multi-instrument classical music using a grid-based representation. This project uses PyTorch and provides an interactive Gradio web interface with interactive music composition controls.
Overview
Classical Music Generator is an AI-powered music composition tool that generates original classical music arrangements across four instrument families: Strings, Woodwinds, Brass, and Percussion. The system uses a Transformer encoder with per-instrument prediction heads to generate music sequences quantized on a fixed 8-step-per-bar grid at 120 BPM.
Key Features
- Multi-instrument composition: Generates music for Strings, Woodwinds, Brass, and Percussion simultaneously
- Grid-based representation: Music is represented as a quantized 8-step-per-bar grid for precise control
- Pitch control: Melodic instruments limited to one octave (C4βB4, 12 pitches), percussion mapped to standard drum sounds
- Transformer architecture: Uses attention-based Transformer encoder with positional encoding for sequential music generation
- Interactive web UI: Gradio-based interface with real-time audio synthesis and visual feedback
- Music theory constraints: Built-in support for scale enforcement and note limiting to ensure musical coherence
- MIDI support: Full MIDI input/output for compatibility with DAWs and other music software
Architecture
Model Details
The core model is a MusicTransformer with the following components:
- Input layer: Projects 39-dimensional multi-hot grid (12 strings + 12 woodwinds + 12 brass + 3 percussion pitches) to the model embedding
- Positional encoding: Sinusoidal positional encoding for temporal sequence information
- Transformer encoder:
- Hidden size: 256 dimensions
- Layers: 3 transformer encoder layers
- Attention heads: 4
- Feed-forward dimension: 512
- Dropout: 0.1
- Output heads: 4 separate linear heads for per-instrument pitch prediction
- Strings head: 12 pitches
- Woodwinds head: 12 pitches
- Brass head: 12 pitches
- Percussion head: 3 pitches (Kick, Snare, Hi-hat)
Grid-Based Representation
Music is represented as a sequence of time steps with the following quantization:
- Tempo: 120 BPM
- Time signature: 4/4
- Steps per bar: 8
- Seconds per step: 0.125 (125 ms)
- Input context: 3 bars (24 steps) of conditioning
- Output: Generation continues from seeded context
Each time step is represented as a 39-dimensional binary vector:
[strings (12) | woodwinds (12) | brass (12) | percussion (3)]
Installation
Prerequisites
- Python 3.8 or higher
- PyTorch 2.0+
- CUDA 11.8+ (optional, for GPU acceleration)
Setup
- Clone the repository:
git clone <repository-url>
cd ClassicalMusicGenerator
- Install dependencies:
pip install -r requirements.txt
- Install system dependencies for audio synthesis:
# macOS
brew install fluidsynth
# Ubuntu/Debian
sudo apt-get install fluidsynth
# Fedora
sudo dnf install fluidsynth
Running the Project
1. Interactive Web Interface
Launch the Gradio-based interactive UI with visual grid controls:
python app.py
Then open your browser to http://localhost:7860 (or the URL displayed in the terminal).
Features:
- Interactive 8-step grid editor for each instrument family
- Real-time audio preview via Web Audio synthesis
- Generate longer compositions with customizable length
- Visual feedback showing active notes and pitch controls
- Color-coded instrument families for easy identification
2. Training the Model
To train on your own MIDI dataset:
python train.py \
--dataset-path data/training_data.npz \
--num-epochs 20 \
--batch-size 32 \
--learning-rate 1e-3
Options:
--dataset-path: Path to preprocessed dataset (NPZ or PT format)--num-epochs: Number of training epochs--batch-size: Batch size for training--learning-rate: Adam optimizer learning rate--val-split: Validation split ratio (default: 0.1)--device: Device to use ('cuda' or 'cpu', auto-detected by default)--save-dir: Directory to save model checkpoints (default: 'models')
Resume training: The training script automatically resumes from checkpoints if they exist:
python train.py --dataset-path data/training_data.npz --extra-epochs 10
3. Data Preprocessing
Prepare MIDI files for training:
python data_preprocessing.py \
--midi-dir path/to/midi/files \
--output-path data/training_data.npz
This script:
- Loads MIDI files from a directory
- Maps instruments to 4 families (Strings, Woodwinds, Brass, Percussion)
- Quantizes notes to the fixed grid (120 BPM, 8 steps/bar)
- Creates input-output pairs for training
- Saves as NumPy NPZ format
4. Inference Scripts
For programmatic music generation:
from inference import load_model, generate_midi_and_audio
import numpy as np
# Load pre-trained model
model = load_model("models/transformer_music_best.pt")
# Create seed grids (3 bars of context)
strings_seed = np.zeros((3, 8, 12), dtype=np.float32)
woodwinds_seed = np.zeros((3, 8, 12), dtype=np.float32)
brass_seed = np.zeros((3, 8, 12), dtype=np.float32)
perc_seed = np.zeros((3, 8, 3), dtype=np.float32)
# Generate 1 minute of music
midi_path, wav_path = generate_midi_and_audio(
strings_seed=strings_seed,
woodwinds_seed=woodwinds_seed,
brass_seed=brass_seed,
perc_seed=perc_seed,
target_minutes=1.0,
device="cpu"
)
Project Structure
ClassicalMusicGenerator/
βββ app.py # Gradio web interface with grid controls
βββ config.py # Global configuration (grid, model, tempo)
βββ model.py # MusicTransformer architecture
βββ train.py # Training loop with checkpoint support
βββ inference.py # Model loading and generation utilities
βββ data_preprocessing.py # MIDI processing and dataset creation
βββ classicalmusicgenerator.py # Colab notebook (reference)
βββ test_grad.py # Gradient checking utilities
βββ requirements.txt # Python dependencies
βββ models/
β βββ transformer_music_best.pt # Pre-trained model weights
βββ utils/
βββ midi_utils.py # MIDI conversion utilities
βββ sampling_utils.py # Generation constraints and sampling
βββ __pycache__/
Configuration
Edit config.py to customize:
Grid Parameters
STEPS_PER_BAR = 8 # Time steps per 4/4 bar
INPUT_BARS = 3 # Context bars for conditioning
TEMPO_BPM = 120 # Tempo in beats per minute
Model Hyperparameters
D_MODEL = 256 # Hidden dimension
N_LAYERS = 3 # Transformer layers
N_HEADS = 4 # Attention heads
FFN_DIM = 512 # Feed-forward dimension
DROPOUT = 0.1 # Dropout rate
Generation Constraints
BAR_SIMILARITY_THRESHOLD = 0.9 # Avoid repetitive sections
MAX_RESAMPLE_ATTEMPTS_PER_BAR = 5 # Resampling attempts per bar
MIN_ACTIVE_FAMILIES_PER_BAR = 2 # Minimum instrument families per bar
Dataset Format
The preprocessing creates training data where:
- X: Input sequences of shape
[N, INPUT_STEPS, INPUT_FEATURES](context) - Y: Target sequences of shape
[N, INPUT_STEPS, INPUT_FEATURES](next step prediction)
Where:
N= number of sequencesINPUT_STEPS= 24 (3 bars Γ 8 steps)INPUT_FEATURES= 39 (12+12+12+3 pitches)
Saved as NPZ or PyTorch PT format with keys "X" and "Y".
Training Tips
- Dataset size: Aim for at least 10,000-50,000 sequences for good results
- Learning rate: Start with 1e-3, reduce by 0.5Γ if loss plateaus
- Batch size: 32 or 64 depending on GPU memory
- Validation split: 10% is reasonable; increase to 15-20% for smaller datasets
- Checkpointing: Models are auto-saved at each epoch, best model tracked separately
Troubleshooting
CUDA Out of Memory
Reduce batch size or use CPU:
python train.py --dataset-path data/training_data.npz --batch-size 16 --device cpu
Missing Audio Dependencies
Ensure fluidsynth is installed (see Installation section above).
Gradio Connection Issues
If the web interface won't open, check the terminal output for the correct localhost address, typically http://localhost:7860.
References
- Transformer architecture: "Attention Is All You Need" (Vaswani et al., 2017)
- Music representation: Grid-based quantization inspired by MusicVAE and Jukebox
- Implementation uses PyTorch, Gradio, and pretty_midi
Contributing
Contributions are welcome! Please:
- Fork the repository
- Create a feature branch
- Commit your changes
- Submit a pull request