Spaces:
Sleeping
Sleeping
|
Download README.md from hephaestusonfire/Music_Generator: direct link, hf CLI and curl.
- Browser
- Download file 9.46 kB
-
https://huggingface.co/spaces/hephaestusonfire/Music_Generator/resolve/main/README.md
- Command line
-
hf download hf://spaces/hephaestusonfire/Music_Generator/README.md
-
curl -L -o README.md https://huggingface.co/spaces/hephaestusonfire/Music_Generator/resolve/main/README.md
9.46 kB
| title: Classical Music Generator | |
| emoji: πΌ | |
| colorFrom: purple | |
| colorTo: blue | |
| sdk: gradio | |
| sdk_version: "4.44.1" | |
| python_version: '3.8' | |
| app_file: app.py | |
| pinned: false | |
| # Classical Music Generator | |
| A Transformer-based neural network system for generating multi-instrument classical music using a grid-based representation. This project uses PyTorch and provides an interactive Gradio web interface with interactive music composition controls. | |
| ## Overview | |
| Classical Music Generator is an AI-powered music composition tool that generates original classical music arrangements across four instrument families: Strings, Woodwinds, Brass, and Percussion. The system uses a Transformer encoder with per-instrument prediction heads to generate music sequences quantized on a fixed 8-step-per-bar grid at 120 BPM. | |
| ### Key Features | |
| - **Multi-instrument composition**: Generates music for Strings, Woodwinds, Brass, and Percussion simultaneously | |
| - **Grid-based representation**: Music is represented as a quantized 8-step-per-bar grid for precise control | |
| - **Pitch control**: Melodic instruments limited to one octave (C4βB4, 12 pitches), percussion mapped to standard drum sounds | |
| - **Transformer architecture**: Uses attention-based Transformer encoder with positional encoding for sequential music generation | |
| - **Interactive web UI**: Gradio-based interface with real-time audio synthesis and visual feedback | |
| - **Music theory constraints**: Built-in support for scale enforcement and note limiting to ensure musical coherence | |
| - **MIDI support**: Full MIDI input/output for compatibility with DAWs and other music software | |
| ## Architecture | |
| ### Model Details | |
| The core model is a **MusicTransformer** with the following components: | |
| - **Input layer**: Projects 39-dimensional multi-hot grid (12 strings + 12 woodwinds + 12 brass + 3 percussion pitches) to the model embedding | |
| - **Positional encoding**: Sinusoidal positional encoding for temporal sequence information | |
| - **Transformer encoder**: | |
| - Hidden size: 256 dimensions | |
| - Layers: 3 transformer encoder layers | |
| - Attention heads: 4 | |
| - Feed-forward dimension: 512 | |
| - Dropout: 0.1 | |
| - **Output heads**: 4 separate linear heads for per-instrument pitch prediction | |
| - Strings head: 12 pitches | |
| - Woodwinds head: 12 pitches | |
| - Brass head: 12 pitches | |
| - Percussion head: 3 pitches (Kick, Snare, Hi-hat) | |
| ### Grid-Based Representation | |
| Music is represented as a sequence of time steps with the following quantization: | |
| - **Tempo**: 120 BPM | |
| - **Time signature**: 4/4 | |
| - **Steps per bar**: 8 | |
| - **Seconds per step**: 0.125 (125 ms) | |
| - **Input context**: 3 bars (24 steps) of conditioning | |
| - **Output**: Generation continues from seeded context | |
| Each time step is represented as a 39-dimensional binary vector: | |
| ``` | |
| [strings (12) | woodwinds (12) | brass (12) | percussion (3)] | |
| ``` | |
| ## Installation | |
| ### Prerequisites | |
| - Python 3.8 or higher | |
| - PyTorch 2.0+ | |
| - CUDA 11.8+ (optional, for GPU acceleration) | |
| ### Setup | |
| 1. Clone the repository: | |
| ```bash | |
| git clone <repository-url> | |
| cd ClassicalMusicGenerator | |
| ``` | |
| 2. Install dependencies: | |
| ```bash | |
| pip install -r requirements.txt | |
| ``` | |
| 3. Install system dependencies for audio synthesis: | |
| ```bash | |
| # macOS | |
| brew install fluidsynth | |
| # Ubuntu/Debian | |
| sudo apt-get install fluidsynth | |
| # Fedora | |
| sudo dnf install fluidsynth | |
| ``` | |
| ## Running the Project | |
| ### 1. Interactive Web Interface | |
| Launch the Gradio-based interactive UI with visual grid controls: | |
| ```bash | |
| python app.py | |
| ``` | |
| Then open your browser to `http://localhost:7860` (or the URL displayed in the terminal). | |
| **Features**: | |
| - Interactive 8-step grid editor for each instrument family | |
| - Real-time audio preview via Web Audio synthesis | |
| - Generate longer compositions with customizable length | |
| - Visual feedback showing active notes and pitch controls | |
| - Color-coded instrument families for easy identification | |
| ### 2. Training the Model | |
| To train on your own MIDI dataset: | |
| ```bash | |
| python train.py \ | |
| --dataset-path data/training_data.npz \ | |
| --num-epochs 20 \ | |
| --batch-size 32 \ | |
| --learning-rate 1e-3 | |
| ``` | |
| **Options**: | |
| - `--dataset-path`: Path to preprocessed dataset (NPZ or PT format) | |
| - `--num-epochs`: Number of training epochs | |
| - `--batch-size`: Batch size for training | |
| - `--learning-rate`: Adam optimizer learning rate | |
| - `--val-split`: Validation split ratio (default: 0.1) | |
| - `--device`: Device to use ('cuda' or 'cpu', auto-detected by default) | |
| - `--save-dir`: Directory to save model checkpoints (default: 'models') | |
| **Resume training**: | |
| The training script automatically resumes from checkpoints if they exist: | |
| ```bash | |
| python train.py --dataset-path data/training_data.npz --extra-epochs 10 | |
| ``` | |
| ### 3. Data Preprocessing | |
| Prepare MIDI files for training: | |
| ```bash | |
| python data_preprocessing.py \ | |
| --midi-dir path/to/midi/files \ | |
| --output-path data/training_data.npz | |
| ``` | |
| This script: | |
| - Loads MIDI files from a directory | |
| - Maps instruments to 4 families (Strings, Woodwinds, Brass, Percussion) | |
| - Quantizes notes to the fixed grid (120 BPM, 8 steps/bar) | |
| - Creates input-output pairs for training | |
| - Saves as NumPy NPZ format | |
| ### 4. Inference Scripts | |
| For programmatic music generation: | |
| ```python | |
| from inference import load_model, generate_midi_and_audio | |
| import numpy as np | |
| # Load pre-trained model | |
| model = load_model("models/transformer_music_best.pt") | |
| # Create seed grids (3 bars of context) | |
| strings_seed = np.zeros((3, 8, 12), dtype=np.float32) | |
| woodwinds_seed = np.zeros((3, 8, 12), dtype=np.float32) | |
| brass_seed = np.zeros((3, 8, 12), dtype=np.float32) | |
| perc_seed = np.zeros((3, 8, 3), dtype=np.float32) | |
| # Generate 1 minute of music | |
| midi_path, wav_path = generate_midi_and_audio( | |
| strings_seed=strings_seed, | |
| woodwinds_seed=woodwinds_seed, | |
| brass_seed=brass_seed, | |
| perc_seed=perc_seed, | |
| target_minutes=1.0, | |
| device="cpu" | |
| ) | |
| ``` | |
| ## Project Structure | |
| ``` | |
| ClassicalMusicGenerator/ | |
| βββ app.py # Gradio web interface with grid controls | |
| βββ config.py # Global configuration (grid, model, tempo) | |
| βββ model.py # MusicTransformer architecture | |
| βββ train.py # Training loop with checkpoint support | |
| βββ inference.py # Model loading and generation utilities | |
| βββ data_preprocessing.py # MIDI processing and dataset creation | |
| βββ classicalmusicgenerator.py # Colab notebook (reference) | |
| βββ test_grad.py # Gradient checking utilities | |
| βββ requirements.txt # Python dependencies | |
| βββ models/ | |
| β βββ transformer_music_best.pt # Pre-trained model weights | |
| βββ utils/ | |
| βββ midi_utils.py # MIDI conversion utilities | |
| βββ sampling_utils.py # Generation constraints and sampling | |
| βββ __pycache__/ | |
| ``` | |
| ## Configuration | |
| Edit [config.py](config.py) to customize: | |
| ### Grid Parameters | |
| ```python | |
| STEPS_PER_BAR = 8 # Time steps per 4/4 bar | |
| INPUT_BARS = 3 # Context bars for conditioning | |
| TEMPO_BPM = 120 # Tempo in beats per minute | |
| ``` | |
| ### Model Hyperparameters | |
| ```python | |
| D_MODEL = 256 # Hidden dimension | |
| N_LAYERS = 3 # Transformer layers | |
| N_HEADS = 4 # Attention heads | |
| FFN_DIM = 512 # Feed-forward dimension | |
| DROPOUT = 0.1 # Dropout rate | |
| ``` | |
| ### Generation Constraints | |
| ```python | |
| BAR_SIMILARITY_THRESHOLD = 0.9 # Avoid repetitive sections | |
| MAX_RESAMPLE_ATTEMPTS_PER_BAR = 5 # Resampling attempts per bar | |
| MIN_ACTIVE_FAMILIES_PER_BAR = 2 # Minimum instrument families per bar | |
| ``` | |
| ## Dataset Format | |
| The preprocessing creates training data where: | |
| - **X**: Input sequences of shape `[N, INPUT_STEPS, INPUT_FEATURES]` (context) | |
| - **Y**: Target sequences of shape `[N, INPUT_STEPS, INPUT_FEATURES]` (next step prediction) | |
| Where: | |
| - `N` = number of sequences | |
| - `INPUT_STEPS` = 24 (3 bars Γ 8 steps) | |
| - `INPUT_FEATURES` = 39 (12+12+12+3 pitches) | |
| Saved as NPZ or PyTorch PT format with keys `"X"` and `"Y"`. | |
| ## Training Tips | |
| - **Dataset size**: Aim for at least 10,000-50,000 sequences for good results | |
| - **Learning rate**: Start with 1e-3, reduce by 0.5Γ if loss plateaus | |
| - **Batch size**: 32 or 64 depending on GPU memory | |
| - **Validation split**: 10% is reasonable; increase to 15-20% for smaller datasets | |
| - **Checkpointing**: Models are auto-saved at each epoch, best model tracked separately | |
| ## Troubleshooting | |
| ### CUDA Out of Memory | |
| Reduce batch size or use CPU: | |
| ```bash | |
| python train.py --dataset-path data/training_data.npz --batch-size 16 --device cpu | |
| ``` | |
| ### Missing Audio Dependencies | |
| Ensure fluidsynth is installed (see Installation section above). | |
| ### Gradio Connection Issues | |
| If the web interface won't open, check the terminal output for the correct localhost address, typically `http://localhost:7860`. | |
| ## References | |
| - Transformer architecture: "[Attention Is All You Need](https://arxiv.org/abs/1706.03762)" (Vaswani et al., 2017) | |
| - Music representation: Grid-based quantization inspired by [MusicVAE](https://magenta.tensorflow.org/music-vae) and [Jukebox](https://openai.com/blog/jukebox/) | |
| - Implementation uses [PyTorch](https://pytorch.org/), [Gradio](https://gradio.app/), and [pretty_midi](https://github.com/craffel/pretty_midi) | |
| ## Contributing | |
| Contributions are welcome! Please: | |
| 1. Fork the repository | |
| 2. Create a feature branch | |
| 3. Commit your changes | |
| 4. Submit a pull request |