hephaestusonfire commited on
Commit
9c7e38e
·
verified ·
1 Parent(s): 28ee8d6

Upload 11 files

Browse files
Files changed (11) hide show
  1. README.md +282 -15
  2. app.py +375 -0
  3. config.py +58 -0
  4. data_preprocessing.py +387 -0
  5. inference.py +485 -0
  6. model.py +163 -0
  7. models/transformer_music_best.pt +3 -0
  8. requirements.txt +19 -0
  9. train.py +215 -0
  10. utils/midi_utils.py +242 -0
  11. utils/sampling_utils.py +307 -0
README.md CHANGED
@@ -1,15 +1,282 @@
1
- ---
2
- title: Music Generator
3
- emoji: 🐠
4
- colorFrom: blue
5
- colorTo: gray
6
- sdk: gradio
7
- sdk_version: 6.15.2
8
- python_version: '3.13'
9
- app_file: app.py
10
- pinned: false
11
- license: mit
12
- short_description: 'Music composition tool that generates original music '
13
- ---
14
-
15
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Classical Music Generator
2
+
3
+ A Transformer-based neural network system for generating multi-instrument classical music using a grid-based representation. This project uses PyTorch and provides an interactive Gradio web interface with interactive music composition controls.
4
+
5
+ ## Overview
6
+
7
+ Classical Music Generator is an AI-powered music composition tool that generates original classical music arrangements across four instrument families: Strings, Woodwinds, Brass, and Percussion. The system uses a Transformer encoder with per-instrument prediction heads to generate music sequences quantized on a fixed 8-step-per-bar grid at 120 BPM.
8
+
9
+ ### Key Features
10
+
11
+ - **Multi-instrument composition**: Generates music for Strings, Woodwinds, Brass, and Percussion simultaneously
12
+ - **Grid-based representation**: Music is represented as a quantized 8-step-per-bar grid for precise control
13
+ - **Pitch control**: Melodic instruments limited to one octave (C4–B4, 12 pitches), percussion mapped to standard drum sounds
14
+ - **Transformer architecture**: Uses attention-based Transformer encoder with positional encoding for sequential music generation
15
+ - **Interactive web UI**: Gradio-based interface with real-time audio synthesis and visual feedback
16
+ - **Music theory constraints**: Built-in support for scale enforcement and note limiting to ensure musical coherence
17
+ - **MIDI support**: Full MIDI input/output for compatibility with DAWs and other music software
18
+
19
+ ## Architecture
20
+
21
+ ### Model Details
22
+
23
+ The core model is a **MusicTransformer** with the following components:
24
+
25
+ - **Input layer**: Projects 39-dimensional multi-hot grid (12 strings + 12 woodwinds + 12 brass + 3 percussion pitches) to the model embedding
26
+ - **Positional encoding**: Sinusoidal positional encoding for temporal sequence information
27
+ - **Transformer encoder**:
28
+ - Hidden size: 256 dimensions
29
+ - Layers: 3 transformer encoder layers
30
+ - Attention heads: 4
31
+ - Feed-forward dimension: 512
32
+ - Dropout: 0.1
33
+ - **Output heads**: 4 separate linear heads for per-instrument pitch prediction
34
+ - Strings head: 12 pitches
35
+ - Woodwinds head: 12 pitches
36
+ - Brass head: 12 pitches
37
+ - Percussion head: 3 pitches (Kick, Snare, Hi-hat)
38
+
39
+ ### Grid-Based Representation
40
+
41
+ Music is represented as a sequence of time steps with the following quantization:
42
+
43
+ - **Tempo**: 120 BPM
44
+ - **Time signature**: 4/4
45
+ - **Steps per bar**: 8
46
+ - **Seconds per step**: 0.125 (125 ms)
47
+ - **Input context**: 3 bars (24 steps) of conditioning
48
+ - **Output**: Generation continues from seeded context
49
+
50
+ Each time step is represented as a 39-dimensional binary vector:
51
+ ```
52
+ [strings (12) | woodwinds (12) | brass (12) | percussion (3)]
53
+ ```
54
+
55
+ ## Installation
56
+
57
+ ### Prerequisites
58
+
59
+ - Python 3.8 or higher
60
+ - PyTorch 2.0+
61
+ - CUDA 11.8+ (optional, for GPU acceleration)
62
+
63
+ ### Setup
64
+
65
+ 1. Clone the repository:
66
+ ```bash
67
+ git clone <repository-url>
68
+ cd ClassicalMusicGenerator
69
+ ```
70
+
71
+ 2. Install dependencies:
72
+ ```bash
73
+ pip install -r requirements.txt
74
+ ```
75
+
76
+ 3. Install system dependencies for audio synthesis:
77
+ ```bash
78
+ # macOS
79
+ brew install fluidsynth
80
+
81
+ # Ubuntu/Debian
82
+ sudo apt-get install fluidsynth
83
+
84
+ # Fedora
85
+ sudo dnf install fluidsynth
86
+ ```
87
+
88
+ ## Running the Project
89
+
90
+ ### 1. Interactive Web Interface
91
+
92
+ Launch the Gradio-based interactive UI with visual grid controls:
93
+
94
+ ```bash
95
+ python app.py
96
+ ```
97
+
98
+ Then open your browser to `http://localhost:7860` (or the URL displayed in the terminal).
99
+
100
+ **Features**:
101
+ - Interactive 8-step grid editor for each instrument family
102
+ - Real-time audio preview via Web Audio synthesis
103
+ - Generate longer compositions with customizable length
104
+ - Visual feedback showing active notes and pitch controls
105
+ - Color-coded instrument families for easy identification
106
+
107
+ ### 2. Training the Model
108
+
109
+ To train on your own MIDI dataset:
110
+
111
+ ```bash
112
+ python train.py \
113
+ --dataset-path data/training_data.npz \
114
+ --num-epochs 20 \
115
+ --batch-size 32 \
116
+ --learning-rate 1e-3
117
+ ```
118
+
119
+ **Options**:
120
+ - `--dataset-path`: Path to preprocessed dataset (NPZ or PT format)
121
+ - `--num-epochs`: Number of training epochs
122
+ - `--batch-size`: Batch size for training
123
+ - `--learning-rate`: Adam optimizer learning rate
124
+ - `--val-split`: Validation split ratio (default: 0.1)
125
+ - `--device`: Device to use ('cuda' or 'cpu', auto-detected by default)
126
+ - `--save-dir`: Directory to save model checkpoints (default: 'models')
127
+
128
+ **Resume training**:
129
+ The training script automatically resumes from checkpoints if they exist:
130
+ ```bash
131
+ python train.py --dataset-path data/training_data.npz --extra-epochs 10
132
+ ```
133
+
134
+ ### 3. Data Preprocessing
135
+
136
+ Prepare MIDI files for training:
137
+
138
+ ```bash
139
+ python data_preprocessing.py \
140
+ --midi-dir path/to/midi/files \
141
+ --output-path data/training_data.npz
142
+ ```
143
+
144
+ This script:
145
+ - Loads MIDI files from a directory
146
+ - Maps instruments to 4 families (Strings, Woodwinds, Brass, Percussion)
147
+ - Quantizes notes to the fixed grid (120 BPM, 8 steps/bar)
148
+ - Creates input-output pairs for training
149
+ - Saves as NumPy NPZ format
150
+
151
+ ### 4. Inference Scripts
152
+
153
+ For programmatic music generation:
154
+
155
+ ```python
156
+ from inference import load_model, generate_midi_and_audio
157
+ import numpy as np
158
+
159
+ # Load pre-trained model
160
+ model = load_model("models/transformer_music_best.pt")
161
+
162
+ # Create seed grids (3 bars of context)
163
+ strings_seed = np.zeros((3, 8, 12), dtype=np.float32)
164
+ woodwinds_seed = np.zeros((3, 8, 12), dtype=np.float32)
165
+ brass_seed = np.zeros((3, 8, 12), dtype=np.float32)
166
+ perc_seed = np.zeros((3, 8, 3), dtype=np.float32)
167
+
168
+ # Generate 1 minute of music
169
+ midi_path, wav_path = generate_midi_and_audio(
170
+ strings_seed=strings_seed,
171
+ woodwinds_seed=woodwinds_seed,
172
+ brass_seed=brass_seed,
173
+ perc_seed=perc_seed,
174
+ target_minutes=1.0,
175
+ device="cpu"
176
+ )
177
+ ```
178
+
179
+ ## Project Structure
180
+
181
+ ```
182
+ ClassicalMusicGenerator/
183
+ ├── app.py # Gradio web interface with grid controls
184
+ ├── config.py # Global configuration (grid, model, tempo)
185
+ ├── model.py # MusicTransformer architecture
186
+ ├── train.py # Training loop with checkpoint support
187
+ ├── inference.py # Model loading and generation utilities
188
+ ├── data_preprocessing.py # MIDI processing and dataset creation
189
+ ├── classicalmusicgenerator.py # Colab notebook (reference)
190
+ ├── test_grad.py # Gradient checking utilities
191
+ ├── requirements.txt # Python dependencies
192
+ ├── models/
193
+ │ └── transformer_music_best.pt # Pre-trained model weights
194
+ └── utils/
195
+ ├── midi_utils.py # MIDI conversion utilities
196
+ ├── sampling_utils.py # Generation constraints and sampling
197
+ └── __pycache__/
198
+ ```
199
+
200
+ ## Configuration
201
+
202
+ Edit [config.py](config.py) to customize:
203
+
204
+ ### Grid Parameters
205
+ ```python
206
+ STEPS_PER_BAR = 8 # Time steps per 4/4 bar
207
+ INPUT_BARS = 3 # Context bars for conditioning
208
+ TEMPO_BPM = 120 # Tempo in beats per minute
209
+ ```
210
+
211
+ ### Model Hyperparameters
212
+ ```python
213
+ D_MODEL = 256 # Hidden dimension
214
+ N_LAYERS = 3 # Transformer layers
215
+ N_HEADS = 4 # Attention heads
216
+ FFN_DIM = 512 # Feed-forward dimension
217
+ DROPOUT = 0.1 # Dropout rate
218
+ ```
219
+
220
+ ### Generation Constraints
221
+ ```python
222
+ BAR_SIMILARITY_THRESHOLD = 0.9 # Avoid repetitive sections
223
+ MAX_RESAMPLE_ATTEMPTS_PER_BAR = 5 # Resampling attempts per bar
224
+ MIN_ACTIVE_FAMILIES_PER_BAR = 2 # Minimum instrument families per bar
225
+ ```
226
+
227
+ ## Dataset Format
228
+
229
+ The preprocessing creates training data where:
230
+ - **X**: Input sequences of shape `[N, INPUT_STEPS, INPUT_FEATURES]` (context)
231
+ - **Y**: Target sequences of shape `[N, INPUT_STEPS, INPUT_FEATURES]` (next step prediction)
232
+
233
+ Where:
234
+ - `N` = number of sequences
235
+ - `INPUT_STEPS` = 24 (3 bars × 8 steps)
236
+ - `INPUT_FEATURES` = 39 (12+12+12+3 pitches)
237
+
238
+ Saved as NPZ or PyTorch PT format with keys `"X"` and `"Y"`.
239
+
240
+ ## Training Tips
241
+
242
+ - **Dataset size**: Aim for at least 10,000-50,000 sequences for good results
243
+ - **Learning rate**: Start with 1e-3, reduce by 0.5× if loss plateaus
244
+ - **Batch size**: 32 or 64 depending on GPU memory
245
+ - **Validation split**: 10% is reasonable; increase to 15-20% for smaller datasets
246
+ - **Checkpointing**: Models are auto-saved at each epoch, best model tracked separately
247
+
248
+ ## Troubleshooting
249
+
250
+ ### CUDA Out of Memory
251
+ Reduce batch size or use CPU:
252
+ ```bash
253
+ python train.py --dataset-path data/training_data.npz --batch-size 16 --device cpu
254
+ ```
255
+
256
+ ### Missing Audio Dependencies
257
+ Ensure fluidsynth is installed (see Installation section above).
258
+
259
+ ### Gradio Connection Issues
260
+ If the web interface won't open, check the terminal output for the correct localhost address, typically `http://localhost:7860`.
261
+
262
+ ## License
263
+
264
+ [Specify your license here]
265
+
266
+ ## References
267
+
268
+ - Transformer architecture: "[Attention Is All You Need](https://arxiv.org/abs/1706.03762)" (Vaswani et al., 2017)
269
+ - Music representation: Grid-based quantization inspired by [MusicVAE](https://magenta.tensorflow.org/music-vae) and [Jukebox](https://openai.com/blog/jukebox/)
270
+ - Implementation uses [PyTorch](https://pytorch.org/), [Gradio](https://gradio.app/), and [pretty_midi](https://github.com/craffel/pretty_midi)
271
+
272
+ ## Contributing
273
+
274
+ Contributions are welcome! Please:
275
+ 1. Fork the repository
276
+ 2. Create a feature branch
277
+ 3. Commit your changes
278
+ 4. Submit a pull request
279
+
280
+ ## Contact
281
+
282
+ [Add contact information if desired]
app.py ADDED
@@ -0,0 +1,375 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Gradio app for classical music generation with a grid-based Transformer.
3
+ Completely native, optimized Gradio 6.0 engine with variable frequency pitch
4
+ synthesis and an active coordinate control surface HUD display.
5
+ """
6
+
7
+ import os
8
+ import tempfile
9
+ from typing import Tuple
10
+ import gradio as gr
11
+ import numpy as np
12
+
13
+ import config
14
+ from inference import generate_midi_and_audio
15
+
16
+ # ==============================================================================
17
+ # UI Constants & Color Mappings
18
+ # ==============================================================================
19
+ _MELODIC_NOTE_NAMES = ["C4","C#4","D4","D#4","E4","F4","F#4","G4","G#4","A4","A#4","B4"]
20
+ _PERC_NOTE_NAMES = ["Kick", "Snare", "Hi-hat"]
21
+
22
+ STRINGS_COLOR = "#63b3ed"
23
+ WOODWINDS_COLOR = "#6ed29b"
24
+ BRASS_COLOR = "#fbbf24"
25
+ PERC_COLOR = "#fc814a"
26
+
27
+ # CSS Esthetic styling system overrides to restore the sleek studio grid interface layout
28
+ CSS_ESTHETIC = """
29
+ .instrument-container {
30
+ background-color: #151b26 !important;
31
+ border: 1px solid rgba(255, 255, 255, 0.05) !important;
32
+ padding: 14px !important;
33
+ border-radius: 8px !important;
34
+ }
35
+ .instrument-margin {
36
+ margin-top: 15px !important;
37
+ }
38
+ .instrument-margin-button {
39
+ margin-top: 5px !important;
40
+ }
41
+ .grid-label {
42
+ font-family: monospace !important;
43
+ font-size: 11px !important;
44
+ font-weight: bold !important;
45
+ color: #c9d1e8 !important;
46
+ text-align: right;
47
+ align-self: center;
48
+ }
49
+ .matrix-cell {
50
+ min-width: 20px !important;
51
+ height: 26px !important;
52
+ padding: 0 !important;
53
+ margin: 0 !important;
54
+ border-radius: 2px !important;
55
+ font-size: 0px !important;
56
+ cursor: pointer !important;
57
+ }
58
+ .cell-off {
59
+ background-color: #111827 !important;
60
+ border: 1px solid rgba(255, 255, 255, 0.02) !important;
61
+ }
62
+ .cell-strings-on { background-color: #63b3ed !important; box-shadow: inset 0 0 5px #fff !important; }
63
+ .cell-brass-on { background-color: #fbbf24 !important; box-shadow: inset 0 0 4px #fff !important; }
64
+ .cell-woodwinds-on { background-color: #6ed29b !important; box-shadow: inset 0 0 4px #fff !important; }
65
+ .cell-perc-on { background-color: #fc814a !important; box-shadow: inset 0 0 4px #fff !important; }
66
+
67
+ /* Timeline Bar Separations */
68
+ .bar-line-left {
69
+ border-left: 2px solid rgba(255, 255, 255, 0.25) !important;
70
+ }
71
+ .action-bar-layout {
72
+ margin-top: 10px;
73
+ }
74
+ .hud-container {
75
+ background-color: #0f141c !important;
76
+ border: 1px solid rgba(99, 179, 237, 0.2) !important;
77
+ padding: 12px !important;
78
+ border-radius: 6px !important;
79
+ }
80
+ """
81
+
82
+ # Dynamic Web Audio JS script to synthesize accurate note pitches
83
+ JS_AUDIO_ENGINE = """
84
+ (row, is_perc) => {
85
+ try {
86
+ window.AudioContext = window.AudioContext || window.webkitAudioContext;
87
+ let ctx = window._gradioAudioCtx || new AudioContext();
88
+ window._gradioAudioCtx = ctx;
89
+ if (ctx.state === 'suspended') ctx.resume();
90
+
91
+ let now = ctx.currentTime;
92
+ let gainNode = ctx.createGain();
93
+ gainNode.connect(ctx.destination);
94
+
95
+ if (is_perc === 1) {
96
+ let bufferSize = Math.floor(ctx.sampleRate * 0.12);
97
+ let buffer = ctx.createBuffer(1, bufferSize, ctx.sampleRate);
98
+ let data = buffer.getChannelData(0);
99
+ for (let i = 0; i < bufferSize; i++) data[i] = Math.random() * 2 - 1;
100
+
101
+ let source = ctx.createBufferSource(); source.buffer = buffer;
102
+ let filter = ctx.createBiquadFilter(); filter.type = 'bandpass';
103
+
104
+ if (row === 0) { filter.frequency.setValueAtTime(55, now); filter.Q.setValueAtTime(5, now); gainNode.gain.setValueAtTime(0.8, now); gainNode.gain.exponentialRampToValueAtTime(0.001, now + 0.1); }
105
+ else if (row === 1) { filter.frequency.setValueAtTime(900, now); filter.Q.setValueAtTime(2, now); gainNode.gain.setValueAtTime(0.4, now); gainNode.gain.exponentialRampToValueAtTime(0.001, now + 0.12); }
106
+ else { filter.frequency.setValueAtTime(8000, now); filter.Q.setValueAtTime(8, now); gainNode.gain.setValueAtTime(0.2, now); gainNode.gain.exponentialRampToValueAtTime(0.001, now + 0.04); }
107
+
108
+ source.connect(filter); filter.connect(gainNode);
109
+ source.start(now); source.stop(now + 0.12);
110
+ } else {
111
+ let midiNote = 60 + row;
112
+ let freq = 440 * Math.pow(2, (midiNote - 69) / 12);
113
+
114
+ let osc = ctx.createOscillator();
115
+ osc.type = 'triangle';
116
+ osc.frequency.setValueAtTime(freq, now);
117
+
118
+ gainNode.gain.setValueAtTime(0.25, now);
119
+ gainNode.gain.exponentialRampToValueAtTime(0.001, now + 0.3);
120
+
121
+ osc.connect(gainNode);
122
+ osc.start(now); osc.stop(now + 0.3);
123
+ }
124
+ } catch(e) { console.error(e); }
125
+ return [];
126
+ }
127
+ """
128
+
129
+ # ==============================================================================
130
+ # Pipeline Execution Adapter
131
+ # ==============================================================================
132
+
133
+ def generate(strings, woodwinds, brass, perc, length, temp, tk, lead):
134
+ """Callback parsing python state matrices smoothly straight into your core backend."""
135
+ tmpdir = tempfile.mkdtemp()
136
+ midi_path = os.path.join(tmpdir, "generated.mid")
137
+ audio_path = os.path.join(tmpdir, "generated.wav")
138
+
139
+ def to_backend_layout(flat_grid, r_dims):
140
+ arr = np.array(flat_grid, dtype=np.float32).reshape(r_dims, config.INPUT_STEPS)
141
+ reshaped = arr.reshape(r_dims, config.INPUT_BARS, config.STEPS_PER_BAR)
142
+ return np.transpose(reshaped, (1, 2, 0))
143
+
144
+ s_seed = to_backend_layout(strings, config.PITCHES_MELODIC)
145
+ w_seed = to_backend_layout(woodwinds, config.PITCHES_MELODIC)
146
+ b_seed = to_backend_layout(brass, config.PITCHES_MELODIC)
147
+ p_seed = to_backend_layout(perc, config.PITCHES_PERC)
148
+
149
+ midi_path, audio_path = generate_midi_and_audio(
150
+ strings_seed=s_seed, woodwinds_seed=w_seed, brass_seed=b_seed, perc_seed=p_seed,
151
+ target_minutes=length, temperature=temp, top_k=int(tk),
152
+ midi_out_path=midi_path, audio_out_path=audio_path, device="cpu", primary_family=lead
153
+ )
154
+ return midi_path, (audio_path if os.path.exists(audio_path) else None)
155
+
156
+ # ==============================================================================
157
+ # Interactive Runtime State Logic
158
+ # ==============================================================================
159
+
160
+ def toggle_cell_state(flat_grid, r, c, max_rows, family):
161
+ """Toggles state values natively in pure python, updates button styles, and tracks HUD info."""
162
+ idx = (r * config.INPUT_STEPS) + c
163
+ current_value = flat_grid[idx]
164
+ new_value = 1 if current_value == 0 else 0
165
+ flat_grid[idx] = new_value
166
+
167
+ cls = "cell-off" if new_value == 0 else f"cell-{family}-on"
168
+ if c > 0 and c % config.STEPS_PER_BAR == 0:
169
+ cls += " bar-line-left"
170
+
171
+ bar_num = (c // config.STEPS_PER_BAR) + 1
172
+ step_num = (c % config.STEPS_PER_BAR) + 1
173
+ note_name = _PERC_NOTE_NAMES[r] if family == "perc" else _MELODIC_NOTE_NAMES[r]
174
+ status = "🔴 ACTIVATED" if new_value == 1 else "⚪ DEACTIVATED"
175
+
176
+ hud_info = f"### 🎛️ Selection HUD\n* **Family:** {family.upper()}\n* **Note / Track:** `{note_name}`\n* **Location:** Bar {bar_num}, Step {step_num} (Global Index {c})\n* **Status:** {status}"
177
+
178
+ return flat_grid, gr.update(elem_classes=["matrix-cell", cls]), hud_info
179
+
180
+ def clear_family_grid(max_rows):
181
+ """Clears a whole family block instantly."""
182
+ blank = [0] * (max_rows * config.INPUT_STEPS)
183
+ updates = []
184
+ for r in range(max_rows):
185
+ for c in range(config.INPUT_STEPS):
186
+ cls = "cell-off"
187
+ if c > 0 and c % config.STEPS_PER_BAR == 0:
188
+ cls += " bar-line-left"
189
+ updates.append(gr.update(elem_classes=["matrix-cell", cls]))
190
+ return [blank] + updates
191
+
192
+ # ==============================================================================
193
+ # UI Composition Window
194
+ # ==============================================================================
195
+
196
+ with gr.Blocks(title="Classical Music Grid Generator") as demo:
197
+
198
+ gr.HTML(f"<script>{JS_AUDIO_ENGINE}</script>")
199
+
200
+ gr.HTML("""
201
+ <div style="text-align:center; padding:15px 0;">
202
+ <h1 style="font-size:2.2rem; font-weight:800; background:linear-gradient(135deg, #63b3ed 0%, #9b5de5 50%, #fc814a 100%); -webkit-background-clip:text; -webkit-text-fill-color:transparent; margin:0 0 5px 0;">
203
+ &#127932; Classical Music Grid Generator
204
+ </h1>
205
+ <p style="color:#8892aa; font-size:14px; max-width:650px; margin:0 auto;">
206
+ Click native studio blocks to build arrangement structures. Notes play instant tone reviews as you paint them.
207
+ </p>
208
+ </div>
209
+ """)
210
+
211
+ with gr.Accordion("📖 Setup Blueprint & Manual Guide", open=True):
212
+ gr.HTML("""
213
+ <div style="display:grid; grid-template-columns:repeat(auto-fit,minmax(220px,1fr)); gap:12px; padding:6px 0;">
214
+ <div style="background:rgba(99,179,237,0.06); border:1px solid rgba(99,179,237,0.2); border-radius:8px; padding:12px;">
215
+ <strong style="color:#63b3ed;">1. Draw Framework Across 3 Bars</strong>
216
+ <p style="color:#8892aa; font-size:12px; margin:4px 0 0; line-height:1.4;">
217
+ Click empty slots to activate note triggers across all three bars. Click an active colorful note to erase it from the timeline instantly.
218
+ </p>
219
+ </div>
220
+ <div style="background:rgba(110,210,155,0.06); border:1px solid rgba(110,210,155,0.2); border-radius:8px; padding:12px;">
221
+ <strong style="color:#6ed29b;">2. Coordinate Inspection HUD</strong>
222
+ <p style="color:#8892aa; font-size:12px; margin:4px 0 0; line-height:1.4;">
223
+ Look at the live <strong>Active Selection HUD</strong> panel on the right after any click to inspect precisely which note and step index value was changed.
224
+ </p>
225
+ </div>
226
+ <div style="background:rgba(252,129,74,0.06); border:1px solid rgba(252,129,74,0.2); border-radius:8px; padding:12px;">
227
+ <strong style="color:#fc814a;">3. Compile & Listen</strong>
228
+ <p style="color:#8892aa; font-size:12px; margin:4px 0 0; line-height:1.4;">
229
+ Adjust composition parameter sliders, then click generate. Use the inline media layout player widget to listen to your full classical orchestration piece.
230
+ </p>
231
+ </div>
232
+ </div>
233
+ """)
234
+
235
+ strings_state = gr.State([0] * (config.PITCHES_MELODIC * config.INPUT_STEPS))
236
+ woodwinds_state = gr.State([0] * (config.PITCHES_MELODIC * config.INPUT_STEPS))
237
+ brass_state = gr.State([0] * (config.PITCHES_MELODIC * config.INPUT_STEPS))
238
+ perc_state = gr.State([0] * (config.PITCHES_PERC * config.INPUT_STEPS))
239
+
240
+ with gr.Row():
241
+ # --- LEFT PANEL COLUMN: INSTRUMENT MATRIX SLOTS ---
242
+ with gr.Column(scale=4):
243
+
244
+ # --- STRINGS DECK ---
245
+ with gr.Column(elem_classes="instrument-container"):
246
+ gr.HTML("<div style='color:#63b3ed; font-weight:bold; font-size:13px; margin-bottom:6px;'>🎻 Strings Family (All 3 Bars)</div>")
247
+ s_buttons = []
248
+ for r in reversed(range(config.PITCHES_MELODIC)):
249
+ with gr.Row():
250
+ gr.HTML(f"<div class='grid-label'>{_MELODIC_NOTE_NAMES[r]}</div>", scale=1)
251
+ for c in range(config.INPUT_STEPS):
252
+ cls = "cell-off"
253
+ if c > 0 and c % config.STEPS_PER_BAR == 0:
254
+ cls += " bar-line-left"
255
+ btn = gr.Button("", elem_classes=["matrix-cell", cls], min_width=15, scale=2)
256
+ s_buttons.append((btn, r, c))
257
+ s_clear = gr.Button("🗑️ Clear Strings", size="sm", elem_classes=["instrument-margin-button"])
258
+
259
+ # --- WOODWINDS DECK ---
260
+ with gr.Column(elem_classes="instrument-container instrument-margin"):
261
+ gr.HTML("<div style='color:#6ed29b; font-weight:bold; font-size:13px; margin-bottom:6px;'>🪵 Woodwinds Family (All 3 Bars)</div>")
262
+ w_buttons = []
263
+ for r in reversed(range(config.PITCHES_MELODIC)):
264
+ with gr.Row():
265
+ gr.HTML(f"<div class='grid-label'>{_MELODIC_NOTE_NAMES[r]}</div>", scale=1)
266
+ for c in range(config.INPUT_STEPS):
267
+ cls = "cell-off"
268
+ if c > 0 and c % config.STEPS_PER_BAR == 0:
269
+ cls += " bar-line-left"
270
+ btn = gr.Button("", elem_classes=["matrix-cell", cls], min_width=15, scale=2)
271
+ w_buttons.append((btn, r, c))
272
+ w_clear = gr.Button("🗑️ Clear Woodwinds", size="sm", elem_classes=["instrument-margin-button"])
273
+
274
+ # --- BRASS DECK ---
275
+ with gr.Column(elem_classes="instrument-container instrument-margin"):
276
+ gr.HTML("<div style='color:#fbbf24; font-weight:bold; font-size:13px; margin-bottom:6px;'>🎺 Brass Family (All 3 Bars)</div>")
277
+ b_buttons = []
278
+ for r in reversed(range(config.PITCHES_MELODIC)):
279
+ with gr.Row():
280
+ gr.HTML(f"<div class='grid-label'>{_MELODIC_NOTE_NAMES[r]}</div>", scale=1)
281
+ for c in range(config.INPUT_STEPS):
282
+ cls = "cell-off"
283
+ if c > 0 and c % config.STEPS_PER_BAR == 0:
284
+ cls += " bar-line-left"
285
+ btn = gr.Button("", elem_classes=["matrix-cell", cls], min_width=15, scale=2)
286
+ b_buttons.append((btn, r, c))
287
+ b_clear = gr.Button("🗑️ Clear Brass", size="sm", elem_classes=["instrument-margin-button"])
288
+
289
+ # --- PERCUSSION DECK ---
290
+ with gr.Column(elem_classes="instrument-container instrument-margin"):
291
+ gr.HTML("<div style='color:#fc814a; font-weight:bold; font-size:13px; margin-bottom:6px;'>🥁 Percussion Grid (All 3 Bars)</div>")
292
+ p_buttons = []
293
+ for r in reversed(range(config.PITCHES_PERC)):
294
+ with gr.Row():
295
+ gr.HTML(f"<div class='grid-label'>{_PERC_NOTE_NAMES[r]}</div>", scale=1)
296
+ for c in range(config.INPUT_STEPS):
297
+ cls = "cell-off"
298
+ if c > 0 and c % config.STEPS_PER_BAR == 0:
299
+ cls += " bar-line-left"
300
+ btn = gr.Button("", elem_classes=["matrix-cell", cls], min_width=15, scale=2)
301
+ p_buttons.append((btn, r, c))
302
+ p_clear = gr.Button("🗑️ Clear Percussion", size="sm", elem_classes=["instrument-margin-button"])
303
+
304
+ # --- RIGHT PANEL COLUMN: ACTIVE SELECTION INSPECTION HUD ---
305
+ with gr.Column(scale=1):
306
+ with gr.Column(elem_classes="hud-container"):
307
+ hud_display = gr.Markdown("### 🎛️ Selection HUD\n*Click any block in the orchestration grids to inspect active coordinate parameters.*")
308
+
309
+ # ==============================================================================
310
+ # Core Event Bindings Engine Loops
311
+ # ==============================================================================
312
+
313
+ # Strings Action Pipeline Mapping
314
+ for btn, r, c in s_buttons:
315
+ btn.click(
316
+ fn=lambda grid, row=r, col=c: toggle_cell_state(grid, row, col, config.PITCHES_MELODIC, "strings"),
317
+ inputs=[strings_state], outputs=[strings_state, btn, hud_display]
318
+ ).then(fn=None, inputs=[gr.State(r), gr.State(0)], outputs=None, js="window.jsAudioEngine.playNote")
319
+ s_clear.click(fn=lambda: clear_family_grid(config.PITCHES_MELODIC), inputs=None, outputs=[strings_state] + [b for b,_,_ in s_buttons])
320
+
321
+ # Woodwinds Action Pipeline Mapping
322
+ for btn, r, c in w_buttons:
323
+ btn.click(
324
+ fn=lambda grid, row=r, col=c: toggle_cell_state(grid, row, col, config.PITCHES_MELODIC, "woodwinds"),
325
+ inputs=[woodwinds_state], outputs=[woodwinds_state, btn, hud_display]
326
+ ).then(fn=None, inputs=[gr.State(r), gr.State(0)], outputs=None, js="window.jsAudioEngine.playNote")
327
+ w_clear.click(fn=lambda: clear_family_grid(config.PITCHES_MELODIC), inputs=None, outputs=[woodwinds_state] + [b for b,_,_ in w_buttons])
328
+
329
+ # Brass Action Pipeline Mapping
330
+ for btn, r, c in b_buttons:
331
+ btn.click(
332
+ fn=lambda grid, row=r, col=c: toggle_cell_state(grid, row, col, config.PITCHES_MELODIC, "brass"),
333
+ inputs=[brass_state], outputs=[brass_state, btn, hud_display]
334
+ ).then(fn=None, inputs=[gr.State(r), gr.State(0)], outputs=None, js="window.jsAudioEngine.playNote")
335
+ b_clear.click(fn=lambda: clear_family_grid(config.PITCHES_MELODIC), inputs=None, outputs=[brass_state] + [b for b,_,_ in b_buttons])
336
+
337
+ # Percussion Action Pipeline Mapping
338
+ for btn, r, c in p_buttons:
339
+ btn.click(
340
+ fn=lambda grid, row=r, col=c: toggle_cell_state(grid, row, col, config.PITCHES_PERC, "perc"),
341
+ inputs=[perc_state], outputs=[perc_state, btn, hud_display]
342
+ ).then(fn=None, inputs=[gr.State(r), gr.State(1)], outputs=None, js="window.jsAudioEngine.playNote")
343
+ p_clear.click(fn=lambda: clear_family_grid(config.PITCHES_PERC), inputs=None, outputs=[perc_state] + [b for b,_,_ in p_buttons])
344
+
345
+ # ==============================================================================
346
+ # Global Parameters & Output Management
347
+ # ==============================================================================
348
+ gr.HTML("<div style='margin-top:20px; border-bottom:1px solid rgba(255,255,255,0.08);'></div>")
349
+
350
+ with gr.Row():
351
+ with gr.Column(scale=3):
352
+ with gr.Row():
353
+ target_minutes = gr.Slider(minimum=1.0, maximum=5.0, value=2.0, step=0.5, label="🕐 Target Length (Minutes)")
354
+ temperature = gr.Slider(minimum=0.7, maximum=1.3, value=1.0, step=0.05, label="🌡️ Creativity Temperature")
355
+ with gr.Row():
356
+ top_k = gr.Slider(minimum=2, maximum=12, value=6, step=1, label="🎲 Top-K Distribution Filter")
357
+ primary_family = gr.Dropdown(choices=["strings", "woodwinds", "brass"], value="strings", label="🎼 Primary Orchestration Lead")
358
+
359
+ generate_button = gr.Button("🎵 Execute Transformer Music Generation Pipeline", variant="primary", size="lg")
360
+
361
+ with gr.Column(scale=2):
362
+ audio_output = gr.Audio(label="🔊 Fluidsynth Generated Audio Player (Listen Live)", type="filepath")
363
+ midi_output = gr.File(label="⬇ Download Output Standard MIDI File (.mid)")
364
+
365
+ generate_button.click(
366
+ fn=generate,
367
+ inputs=[
368
+ strings_state, woodwinds_state, brass_state, perc_state,
369
+ target_minutes, temperature, top_k, primary_family
370
+ ],
371
+ outputs=[midi_output, audio_output]
372
+ )
373
+
374
+ if __name__ == "__main__":
375
+ demo.launch(css=CSS_ESTHETIC)
config.py ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Global configuration for the classical music grid Transformer.
3
+ """
4
+
5
+ # =========================
6
+ # Grid and tempo constants
7
+ # =========================
8
+
9
+ # Time grid
10
+ STEPS_PER_BAR: int = 8
11
+ INPUT_BARS: int = 3
12
+ INPUT_STEPS: int = INPUT_BARS * STEPS_PER_BAR # 24
13
+
14
+ # Pitch grid
15
+ PITCHES_MELODIC: int = 12 # one octave: 12 pitch classes
16
+ PITCHES_PERC: int = 3 # e.g., Kick, Snare, Hi-hat
17
+
18
+ # Instrument families (for clarity / future use)
19
+ N_FAMILIES: int = 4 # Strings, Woodwinds, Brass, Percussion
20
+
21
+ # Tempo and meter
22
+ TEMPO_BPM: int = 120
23
+ BEATS_PER_BAR: int = 4 # assuming 4/4 time
24
+
25
+ # Derived timing values
26
+ SECONDS_PER_BEAT: float = 60.0 / TEMPO_BPM
27
+ SECONDS_PER_BAR: float = BEATS_PER_BAR * SECONDS_PER_BEAT
28
+ SECONDS_PER_STEP: float = SECONDS_PER_BAR / STEPS_PER_BAR
29
+
30
+
31
+ # =========================
32
+ # Model hyperparameters
33
+ # =========================
34
+
35
+ # Transformer dimensions
36
+ D_MODEL: int = 256 # hidden size
37
+ N_LAYERS: int = 3 # number of Transformer encoder layers (2–4 is reasonable)
38
+ N_HEADS: int = 4 # attention heads
39
+ FFN_DIM: int = 512 # feed-forward inner dimension
40
+ DROPOUT: float = 0.1 # dropout rate
41
+
42
+ # Input feature size per time step:
43
+ # Strings (12) + Woodwinds (12) + Brass (12) + Percussion (3) = 39
44
+ INPUT_FEATURES: int = (3 * PITCHES_MELODIC) + PITCHES_PERC # 39
45
+
46
+
47
+ # =========================
48
+ # Sampling and constraints
49
+ # =========================
50
+
51
+ # Bar-level similarity threshold (0–1); above this, bar is considered "too similar"
52
+ BAR_SIMILARITY_THRESHOLD: float = 0.9
53
+
54
+ # Maximum number of resampling attempts per bar before accepting anyway
55
+ MAX_RESAMPLE_ATTEMPTS_PER_BAR: int = 5
56
+
57
+ # Minimum number of instrument families that must be active (have ≥1 note) per bar
58
+ MIN_ACTIVE_FAMILIES_PER_BAR: int = 2
data_preprocessing.py ADDED
@@ -0,0 +1,387 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Data preprocessing for grid-based multi-instrument music generation.
3
+
4
+ - Loads MIDI files from a directory (e.g., MusicNet).
5
+ - Maps raw MIDI instruments to 4 families: Strings, Woodwinds, Brass, Percussion.
6
+ - Quantizes notes to a fixed grid at 120 BPM, with 8 steps per bar.
7
+ - Limits melodic pitches to one octave (C4–B4) and maps to indices 0–11.
8
+ - Maps percussion to Kick/Snare/Hi-hat rows.
9
+ - Builds sequences of length SEQ_LEN (in steps), with each time step a 39-dim vector:
10
+ [12 strings, 12 woodwinds, 12 brass, 3 perc]
11
+ - Saves X (inputs) and Y (targets) arrays to disk.
12
+ """
13
+
14
+ from typing import List, Tuple, Dict
15
+
16
+ import glob
17
+ import os
18
+
19
+ import numpy as np
20
+ import pretty_midi
21
+
22
+ import config
23
+
24
+
25
+ # One octave: C4 (60) to B4 (71)
26
+ MELODIC_BASE_PITCH: int = 60
27
+ MELODIC_MAX_PITCH: int = MELODIC_BASE_PITCH + config.PITCHES_MELODIC - 1 # 71
28
+
29
+ # Percussion pitches
30
+ PERC_KICK_PITCH: int = 36 # Bass Drum 1
31
+ PERC_SNARE_PITCH: int = 38 # Acoustic Snare
32
+ PERC_HIHAT_PITCH: int = 42 # Closed Hi-Hat
33
+
34
+
35
+ def load_midi_paths_from_dir(midi_root: str, pattern: str = "**/*.mid*") -> List[str]:
36
+ """
37
+ Recursively collect MIDI file paths from a root directory.
38
+
39
+ Args:
40
+ midi_root: Root directory containing MIDI files.
41
+ pattern: Glob pattern relative to root.
42
+
43
+ Returns:
44
+ List of file paths.
45
+ """
46
+ search_path = os.path.join(midi_root, pattern)
47
+ filepaths = glob.glob(search_path, recursive=True)
48
+ return filepaths
49
+
50
+
51
+ # ============
52
+ # Instrument family mapping
53
+ # ============
54
+
55
+ def get_family_for_program(program: int, is_drum: bool) -> str:
56
+ """
57
+ Map a MIDI program number to one of the four families.
58
+
59
+ Args:
60
+ program: MIDI program number (0–127).
61
+ is_drum: Whether this is a drum channel.
62
+
63
+ Returns:
64
+ One of: "strings", "woodwinds", "brass", "perc".
65
+ """
66
+ if is_drum:
67
+ return "perc"
68
+
69
+ # Very simple heuristic based on General MIDI program ranges.
70
+ # You can refine this mapping later as needed.
71
+
72
+ # Strings: Violins, Violas, Celli, etc. (40–47)
73
+ if 40 <= program <= 47:
74
+ return "strings"
75
+
76
+ # Woodwinds/Flutes/Clarinets: Flute (73), Clarinet (71), etc.
77
+ if 64 <= program <= 79:
78
+ return "woodwinds"
79
+
80
+ # Brass: Trumpet, Trombone, etc. (56–63)
81
+ if 56 <= program <= 63:
82
+ return "brass"
83
+
84
+ # Fallback: treat others as strings
85
+ return "strings"
86
+
87
+
88
+ # ============
89
+ # Quantization and grid building
90
+ # ============
91
+
92
+ def time_to_step_index(t: float) -> int:
93
+ """
94
+ Convert time (seconds) to global step index on the quantized grid.
95
+
96
+ Using config.SECONDS_PER_STEP.
97
+
98
+ Args:
99
+ t: Time in seconds.
100
+
101
+ Returns:
102
+ Global step index (int).
103
+ """
104
+ return int(np.floor(t / config.SECONDS_PER_STEP))
105
+
106
+
107
+ def clip_pitch_to_melodic_range(pitch: int) -> int:
108
+ """
109
+ Map MIDI pitch to [MELODIC_BASE_PITCH, MELODIC_MAX_PITCH] if possible.
110
+
111
+ Returns:
112
+ -1 if outside range (should be ignored), or
113
+ pitch_idx in [0, PITCHES_MELODIC-1] otherwise.
114
+ """
115
+ if pitch < MELODIC_BASE_PITCH or pitch > MELODIC_MAX_PITCH:
116
+ return -1
117
+ return pitch - MELODIC_BASE_PITCH
118
+
119
+
120
+ def instrument_to_family_grids(
121
+ pm: pretty_midi.PrettyMIDI,
122
+ max_bars: int = None,
123
+ ) -> Dict[str, np.ndarray]:
124
+ """
125
+ Convert a PrettyMIDI object into per-family grids.
126
+
127
+ Grids have shape:
128
+ strings: [bars, STEPS_PER_BAR, 12]
129
+ woodwinds: [bars, STEPS_PER_BAR, 12]
130
+ brass: [bars, STEPS_PER_BAR, 12]
131
+ perc: [bars, STEPS_PER_BAR, 3]
132
+
133
+ Args:
134
+ pm: PrettyMIDI object.
135
+ max_bars: Optional cap on number of bars.
136
+
137
+ Returns:
138
+ Dict of family → grid.
139
+ """
140
+ total_seconds = pm.get_end_time()
141
+ bar_duration = config.SECONDS_PER_BAR
142
+ total_bars = int(np.ceil(total_seconds / bar_duration))
143
+
144
+ if max_bars is not None:
145
+ total_bars = min(total_bars, max_bars)
146
+
147
+ T = total_bars * config.STEPS_PER_BAR
148
+
149
+ strings = np.zeros((total_bars, config.STEPS_PER_BAR, config.PITCHES_MELODIC), dtype=np.float32)
150
+ woodwinds = np.zeros_like(strings)
151
+ brass = np.zeros_like(strings)
152
+ perc = np.zeros((total_bars, config.STEPS_PER_BAR, config.PITCHES_PERC), dtype=np.float32)
153
+
154
+ # Helper to convert time to (bar_idx, step_idx)
155
+ def t_to_bar_step(t: float) -> Tuple[int, int]:
156
+ step_idx_global = time_to_step_index(t)
157
+ bar_idx = step_idx_global // config.STEPS_PER_BAR
158
+ step_idx = step_idx_global % config.STEPS_PER_BAR
159
+ if bar_idx >= total_bars:
160
+ bar_idx = total_bars - 1
161
+ return bar_idx, step_idx
162
+
163
+ for inst in pm.instruments:
164
+ is_drum = inst.is_drum
165
+ family = get_family_for_program(inst.program, is_drum=is_drum)
166
+
167
+ if family == "perc":
168
+ for note in inst.notes:
169
+ b, s = t_to_bar_step(note.start)
170
+ if note.pitch == PERC_KICK_PITCH:
171
+ k = 0
172
+ elif note.pitch == PERC_SNARE_PITCH:
173
+ k = 1
174
+ elif note.pitch == PERC_HIHAT_PITCH:
175
+ k = 2
176
+ else:
177
+ continue
178
+ perc[b, s, k] = 1.0
179
+ else:
180
+ # Select grid
181
+ if family == "strings":
182
+ grid = strings
183
+ elif family == "woodwinds":
184
+ grid = woodwinds
185
+ elif family == "brass":
186
+ grid = brass
187
+ else:
188
+ grid = strings # fallback
189
+
190
+ for note in inst.notes:
191
+ b, s = t_to_bar_step(note.start)
192
+ pitch_idx = clip_pitch_to_melodic_range(note.pitch)
193
+ if pitch_idx == -1:
194
+ continue
195
+ grid[b, s, pitch_idx] = 1.0
196
+
197
+ return {
198
+ "strings": strings,
199
+ "woodwinds": woodwinds,
200
+ "brass": brass,
201
+ "perc": perc,
202
+ }
203
+
204
+
205
+ def grids_to_feature_sequences(
206
+ family_grids: Dict[str, np.ndarray],
207
+ ) -> np.ndarray:
208
+ """
209
+ Convert per-family grids into a [T, 39] feature sequence.
210
+
211
+ Families:
212
+ strings: [bars, STEPS_PER_BAR, 12]
213
+ woodwinds: [bars, STEPS_PER_BAR, 12]
214
+ brass: [bars, STEPS_PER_BAR, 12]
215
+ perc: [bars, STEPS_PER_BAR, 3]
216
+
217
+ Returns:
218
+ seq: [T, 39], where T = bars * STEPS_PER_BAR.
219
+ """
220
+ strings = family_grids["strings"]
221
+ woodwinds = family_grids["woodwinds"]
222
+ brass = family_grids["brass"]
223
+ perc = family_grids["perc"]
224
+
225
+ bars, steps_per_bar, n_strings = strings.shape
226
+ T = bars * steps_per_bar
227
+
228
+ # Reshape to [T, pitches]
229
+ strings_flat = strings.reshape(T, n_strings)
230
+ woodwinds_flat = woodwinds.reshape(T, woodwinds.shape[-1])
231
+ brass_flat = brass.reshape(T, brass.shape[-1])
232
+ perc_flat = perc.reshape(T, perc.shape[-1])
233
+
234
+ # Concatenate along feature dimension: [T, 12+12+12+3] = [T, 39]
235
+ seq = np.concatenate(
236
+ [strings_flat, woodwinds_flat, brass_flat, perc_flat],
237
+ axis=-1,
238
+ )
239
+
240
+ return seq
241
+
242
+
243
+ def build_sequences_for_file(
244
+ midi_path: str,
245
+ seq_len_steps: int,
246
+ max_bars: int = None,
247
+ ) -> Tuple[np.ndarray, np.ndarray]:
248
+ """
249
+ Build (X, Y) sequences for a single MIDI file.
250
+
251
+ - Convert MIDI → family grids → [T, 39] sequence.
252
+ - Build sliding window sequences of length seq_len_steps:
253
+ X: first seq_len_steps-1 steps
254
+ Y: next step (for next-step prediction)
255
+ For convenience, we pack them as:
256
+ X: [N, seq_len_steps, 39]
257
+ Y: [N, seq_len_steps, 39] (shifted by 1).
258
+
259
+ Args:
260
+ midi_path: Path to MIDI file.
261
+ seq_len_steps: Sequence length in grid steps (e.g., 64, 128).
262
+ max_bars: Optional max bars to consider from this file.
263
+
264
+ Returns:
265
+ X: [N, seq_len_steps, 39]
266
+ Y: [N, seq_len_steps, 39]
267
+ """
268
+ try:
269
+ pm = pretty_midi.PrettyMIDI(midi_path)
270
+ except Exception:
271
+ # Skip unreadable files
272
+ return np.empty((0, seq_len_steps, config.INPUT_FEATURES), dtype=np.float32), \
273
+ np.empty((0, seq_len_steps, config.INPUT_FEATURES), dtype=np.float32)
274
+
275
+ family_grids = instrument_to_family_grids(pm, max_bars=max_bars)
276
+ seq = grids_to_feature_sequences(family_grids) # [T, 39]
277
+
278
+ T = seq.shape[0]
279
+ if T <= seq_len_steps:
280
+ return np.empty((0, seq_len_steps, config.INPUT_FEATURES), dtype=np.float32), \
281
+ np.empty((0, seq_len_steps, config.INPUT_FEATURES), dtype=np.float32)
282
+
283
+ # Sliding window
284
+ X_list = []
285
+ Y_list = []
286
+ # We'll create sequences of length seq_len_steps,
287
+ # with Y being the same sequence shifted by 1 step.
288
+ for start in range(0, T - seq_len_steps):
289
+ end = start + seq_len_steps
290
+ seq_window = seq[start:end] # [seq_len_steps, 39]
291
+ seq_window_next = seq[start + 1:end + 1] # shifted by 1 step
292
+
293
+ X_list.append(seq_window)
294
+ Y_list.append(seq_window_next)
295
+
296
+ X = np.stack(X_list, axis=0) # [N, seq_len_steps, 39]
297
+ Y = np.stack(Y_list, axis=0) # [N, seq_len_steps, 39]
298
+
299
+ return X.astype(np.float32), Y.astype(np.float32)
300
+
301
+
302
+ def build_dataset_from_dir(
303
+ midi_root: str,
304
+ seq_len_bars: int,
305
+ max_files: int = None,
306
+ max_bars_per_file: int = None,
307
+ pattern: str = "**/*.mid*",
308
+ ) -> Tuple[np.ndarray, np.ndarray]:
309
+ """
310
+ Build a full dataset (X, Y) from all MIDI files in a directory.
311
+
312
+ Args:
313
+ midi_root: Root directory containing MIDI files (e.g., MusicNet midis).
314
+ seq_len_bars: Sequence length in bars.
315
+ max_files: Optional max number of files to use.
316
+ max_bars_per_file: Optional cap on bars per file for preprocessing.
317
+ pattern: Glob pattern for finding midis under midi_root.
318
+
319
+ Returns:
320
+ X_all: [N_total, seq_len_steps, 39]
321
+ Y_all: [N_total, seq_len_steps, 39]
322
+ """
323
+ midi_paths = load_midi_paths_from_dir(midi_root, pattern=pattern)
324
+
325
+ if max_files is not None:
326
+ midi_paths = midi_paths[:max_files]
327
+
328
+ seq_len_steps = seq_len_bars * config.STEPS_PER_BAR
329
+
330
+ X_all_list = []
331
+ Y_all_list = []
332
+
333
+ for path in midi_paths:
334
+ X, Y = build_sequences_for_file(
335
+ midi_path=path,
336
+ seq_len_steps=seq_len_steps,
337
+ max_bars=max_bars_per_file,
338
+ )
339
+ if X.shape[0] == 0:
340
+ continue
341
+ X_all_list.append(X)
342
+ Y_all_list.append(Y)
343
+
344
+ if not X_all_list:
345
+ # No data
346
+ return np.empty((0, seq_len_steps, config.INPUT_FEATURES), dtype=np.float32), \
347
+ np.empty((0, seq_len_steps, config.INPUT_FEATURES), dtype=np.float32)
348
+
349
+ X_all = np.concatenate(X_all_list, axis=0)
350
+ Y_all = np.concatenate(Y_all_list, axis=0)
351
+
352
+ return X_all, Y_all
353
+
354
+
355
+ def save_dataset_npz(
356
+ X: np.ndarray,
357
+ Y: np.ndarray,
358
+ out_path: str,
359
+ ) -> None:
360
+ """
361
+ Save (X, Y) arrays to a .npz file.
362
+
363
+ Args:
364
+ X: [N, seq_len_steps, 39]
365
+ Y: [N, seq_len_steps, 39]
366
+ out_path: path to .npz file.
367
+ """
368
+ np.savez_compressed(out_path, X=X, Y=Y)
369
+
370
+
371
+ def save_dataset_pt(
372
+ X: np.ndarray,
373
+ Y: np.ndarray,
374
+ out_path: str,
375
+ ) -> None:
376
+ """
377
+ Save (X, Y) arrays to a .pt file for PyTorch.
378
+
379
+ Args:
380
+ X: [N, seq_len_steps, 39]
381
+ Y: [N, seq_len_steps, 39]
382
+ out_path: path to .pt file.
383
+ """
384
+ import torch
385
+
386
+ data = {"X": torch.from_numpy(X), "Y": torch.from_numpy(Y)}
387
+ torch.save(data, out_path)
inference.py ADDED
@@ -0,0 +1,485 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Inference utilities: load trained model and generate music from seed grids.
3
+ """
4
+
5
+ from typing import Dict, Tuple
6
+
7
+ import math
8
+ import os
9
+
10
+ import numpy as np
11
+ import torch
12
+
13
+ import config
14
+ from model import build_model
15
+ from utils import sampling_utils, midi_utils
16
+
17
+
18
+ # =========================
19
+ # Model loading
20
+ # =========================
21
+
22
+ _model: torch.nn.Module = None
23
+
24
+
25
+ def load_model(
26
+ weights_path: str = "models/transformer_music_best.pt",
27
+ device: str = "cpu",
28
+ ) -> torch.nn.Module:
29
+ """
30
+ Load the trained MusicTransformer model for inference.
31
+
32
+ Args:
33
+ weights_path: Path to the .pt weights file.
34
+ device: Device to map the model to ("cpu" for HF Spaces).
35
+
36
+ Returns:
37
+ model: Loaded MusicTransformer in eval mode.
38
+ """
39
+ global _model
40
+
41
+ if _model is not None:
42
+ return _model
43
+
44
+ model = build_model(config)
45
+ state_dict = torch.load(weights_path, map_location=device)
46
+ model.load_state_dict(state_dict)
47
+ model.to(device)
48
+ model.eval()
49
+
50
+ _model = model
51
+ return _model
52
+
53
+
54
+ # =========================
55
+ # Seed conversion
56
+ # =========================
57
+
58
+ def seed_grids_to_sequence(
59
+ strings_seed: np.ndarray,
60
+ woodwinds_seed: np.ndarray,
61
+ brass_seed: np.ndarray,
62
+ perc_seed: np.ndarray,
63
+ ) -> np.ndarray:
64
+ """
65
+ Convert 4 seed grids into a [T_seed, 39] feature sequence.
66
+
67
+ Seed shapes:
68
+ strings_seed: [INPUT_BARS, STEPS_PER_BAR, 12]
69
+ woodwinds_seed: [INPUT_BARS, STEPS_PER_BAR, 12]
70
+ brass_seed: [INPUT_BARS, STEPS_PER_BAR, 12]
71
+ perc_seed: [INPUT_BARS, STEPS_PER_BAR, 3]
72
+
73
+ Returns:
74
+ seq: [T_seed, 39] float32
75
+ """
76
+ # Basic checks
77
+ bars, steps, p_s = strings_seed.shape
78
+ assert bars == config.INPUT_BARS
79
+ assert steps == config.STEPS_PER_BAR
80
+ assert p_s == config.PITCHES_MELODIC
81
+
82
+ # Flatten bars × steps → T
83
+ T_seed = bars * steps
84
+
85
+ strings_flat = strings_seed.reshape(T_seed, config.PITCHES_MELODIC)
86
+ woodwinds_flat = woodwinds_seed.reshape(T_seed, config.PITCHES_MELODIC)
87
+ brass_flat = brass_seed.reshape(T_seed, config.PITCHES_MELODIC)
88
+ perc_flat = perc_seed.reshape(T_seed, config.PITCHES_PERC)
89
+
90
+ seq = np.concatenate(
91
+ [strings_flat, woodwinds_flat, brass_flat, perc_flat],
92
+ axis=-1,
93
+ ).astype(np.float32) # [T_seed, 39]
94
+
95
+ return seq
96
+
97
+
98
+ def sequence_to_grids(
99
+ seq: np.ndarray,
100
+ ) -> Dict[str, np.ndarray]:
101
+ """
102
+ Convert a [T, 39] sequence back into per-family grids.
103
+
104
+ Returns:
105
+ {
106
+ "strings": [bars, STEPS_PER_BAR, 12],
107
+ "woodwinds": [bars, STEPS_PER_BAR, 12],
108
+ "brass": [bars, STEPS_PER_BAR, 12],
109
+ "perc": [bars, STEPS_PER_BAR, 3],
110
+ }
111
+ """
112
+ T = seq.shape[0]
113
+ bars = T // config.STEPS_PER_BAR
114
+ seq = seq[: bars * config.STEPS_PER_BAR] # trim to full bars if needed
115
+
116
+ strings = seq[:, 0:12]
117
+ woodwinds = seq[:, 12:24]
118
+ brass = seq[:, 24:36]
119
+ perc = seq[:, 36:39]
120
+
121
+ strings_grid = strings.reshape(bars, config.STEPS_PER_BAR, config.PITCHES_MELODIC)
122
+ woodwinds_grid = woodwinds.reshape(bars, config.STEPS_PER_BAR, config.PITCHES_MELODIC)
123
+ brass_grid = brass.reshape(bars, config.STEPS_PER_BAR, config.PITCHES_MELODIC)
124
+ perc_grid = perc.reshape(bars, config.STEPS_PER_BAR, config.PITCHES_PERC)
125
+
126
+ return {
127
+ "strings": strings_grid,
128
+ "woodwinds": woodwinds_grid,
129
+ "brass": brass_grid,
130
+ "perc": perc_grid,
131
+ }
132
+
133
+
134
+ # =========================
135
+ # Generation
136
+ # =========================
137
+
138
+ def compute_target_bars(target_minutes: float) -> int:
139
+ """
140
+ Compute number of bars needed to approximate target_minutes,
141
+ given fixed tempo and meter.
142
+
143
+ Returns:
144
+ target_bars: int
145
+ """
146
+ total_seconds = target_minutes * 60.0
147
+ seconds_per_bar = config.SECONDS_PER_BAR
148
+ target_bars = math.ceil(total_seconds / seconds_per_bar)
149
+ return target_bars
150
+
151
+
152
+ def generate_from_seed(
153
+ strings_seed: np.ndarray,
154
+ woodwinds_seed: np.ndarray,
155
+ brass_seed: np.ndarray,
156
+ perc_seed: np.ndarray,
157
+ target_minutes: float,
158
+ temperature: float = 1.0,
159
+ top_k: int = 0,
160
+ device: str = "cpu",
161
+ ) -> Dict[str, np.ndarray]:
162
+ """
163
+ Generate full grids from seed grids using the trained model.
164
+
165
+ Args:
166
+ strings_seed, woodwinds_seed, brass_seed, perc_seed:
167
+ Seed grids from UI, shapes:
168
+ [INPUT_BARS, STEPS_PER_BAR, 12] for melodic families,
169
+ [INPUT_BARS, STEPS_PER_BAR, 3] for perc.
170
+ target_minutes: Desired approximate length in minutes.
171
+ temperature: Sampling temperature.
172
+ top_k: top-k filter.
173
+ device: "cpu" or "cuda".
174
+
175
+ Returns:
176
+ full_grids dict:
177
+ {
178
+ "strings": [total_bars, STEPS_PER_BAR, 12],
179
+ "woodwinds": [total_bars, STEPS_PER_BAR, 12],
180
+ "brass": [total_bars, STEPS_PER_BAR, 12],
181
+ "perc": [total_bars, STEPS_PER_BAR, 3],
182
+ }
183
+ This includes seed bars + generated bars.
184
+ """
185
+ model = load_model(device=device) # global cached
186
+
187
+ # Convert seeds to [T_seed, 39]
188
+ seed_seq = seed_grids_to_sequence(
189
+ strings_seed=strings_seed,
190
+ woodwinds_seed=woodwinds_seed,
191
+ brass_seed=brass_seed,
192
+ perc_seed=perc_seed,
193
+ ) # [T_seed, 39]
194
+
195
+ T_seed = seed_seq.shape[0]
196
+ seed_bars = T_seed // config.STEPS_PER_BAR
197
+
198
+ target_bars = compute_target_bars(target_minutes)
199
+ # Ensure target_bars >= seed_bars
200
+ if target_bars <= seed_bars:
201
+ target_bars = seed_bars + 1
202
+
203
+ total_bars_to_generate = target_bars - seed_bars
204
+
205
+ # Start with seed sequence as context
206
+ seq = seed_seq.copy() # [T_cur, 39]
207
+
208
+ # Keep track of bar dicts (including seed bars)
209
+ all_bars: Dict[str, list] = {
210
+ "strings": [],
211
+ "woodwinds": [],
212
+ "brass": [],
213
+ "perc": [],
214
+ }
215
+
216
+ # First, add the seed bars to all_bars
217
+ seed_grids = sequence_to_grids(seed_seq)
218
+ for key in all_bars.keys():
219
+ for b in range(seed_grids[key].shape[0]):
220
+ all_bars[key].append(seed_grids[key][b])
221
+
222
+ # Previous generated bar for similarity check (full dict)
223
+ prev_bar_dict = {
224
+ "strings": all_bars["strings"][-1],
225
+ "woodwinds": all_bars["woodwinds"][-1],
226
+ "brass": all_bars["brass"][-1],
227
+ "perc": all_bars["perc"][-1],
228
+ }
229
+
230
+ for bar_idx in range(total_bars_to_generate):
231
+ # Collect logits for one bar
232
+ bar_step_logits = {
233
+ "strings": [],
234
+ "woodwinds": [],
235
+ "brass": [],
236
+ "perc": [],
237
+ }
238
+
239
+ for step_in_bar in range(config.STEPS_PER_BAR):
240
+ seq_tensor = torch.from_numpy(seq).unsqueeze(0).to(device) # [1, T_cur, 39]
241
+
242
+ with torch.no_grad():
243
+ outputs = model(seq_tensor) # dict of [1, T_cur, pitches]
244
+
245
+ # Take logits for the last time step
246
+ strings_logits = outputs["strings"][:, -1, :] # [1, 12]
247
+ woodwinds_logits = outputs["woodwinds"][:, -1, :]
248
+ brass_logits = outputs["brass"][:, -1, :]
249
+ perc_logits = outputs["perc"][:, -1, :]
250
+
251
+ bar_step_logits["strings"].append(strings_logits)
252
+ bar_step_logits["woodwinds"].append(woodwinds_logits)
253
+ bar_step_logits["brass"].append(brass_logits)
254
+ bar_step_logits["perc"].append(perc_logits)
255
+
256
+ # Append placeholder next-step feature
257
+ next_step_placeholder = np.zeros((1, config.INPUT_FEATURES), dtype=np.float32)
258
+ seq = np.concatenate([seq, next_step_placeholder], axis=0) # [T_cur+1, 39]
259
+
260
+ # Stack logits to [1, T, P] per family
261
+ logits_dict = {
262
+ "strings": torch.stack(bar_step_logits["strings"], dim=1), # [1, T, 12]
263
+ "woodwinds": torch.stack(bar_step_logits["woodwinds"], dim=1),
264
+ "brass": torch.stack(bar_step_logits["brass"], dim=1),
265
+ "perc": torch.stack(bar_step_logits["perc"], dim=1), # [1, T, 3]
266
+ }
267
+
268
+ # Sample a bar with constraints
269
+ bar_dict = sampling_utils.resample_bar_if_needed(
270
+ prev_bar=prev_bar_dict,
271
+ logits_dict=logits_dict,
272
+ temperature=temperature,
273
+ top_k=top_k,
274
+ # using defaults inside resample for min_notes_per_bar, max_notes_per_step
275
+ )
276
+
277
+ # Append this bar
278
+ for key in all_bars.keys():
279
+ all_bars[key].append(bar_dict[key])
280
+
281
+ # Update prev_bar_dict
282
+ prev_bar_dict = bar_dict
283
+
284
+ # Update the last STEPS_PER_BAR placeholders in seq with actual features
285
+ T_cur = seq.shape[0]
286
+ start_idx = T_cur - config.STEPS_PER_BAR
287
+ end_idx = T_cur
288
+
289
+ strings_bar = bar_dict["strings"] # [T, 12]
290
+ woodwinds_bar = bar_dict["woodwinds"]
291
+ brass_bar = bar_dict["brass"]
292
+ perc_bar = bar_dict["perc"]
293
+
294
+ bar_features = np.concatenate(
295
+ [strings_bar, woodwinds_bar, brass_bar, perc_bar],
296
+ axis=-1,
297
+ ).astype(np.float32) # [T, 39]
298
+
299
+ seq[start_idx:end_idx, :] = bar_features
300
+
301
+ # Convert all_bars lists to numpy grids
302
+ strings_grid = np.stack(all_bars["strings"], axis=0) # [bars, STEPS_PER_BAR, 12]
303
+ woodwinds_grid = np.stack(all_bars["woodwinds"], axis=0)
304
+ brass_grid = np.stack(all_bars["brass"], axis=0)
305
+ perc_grid = np.stack(all_bars["perc"], axis=0)
306
+
307
+ return {
308
+ "strings": strings_grid,
309
+ "woodwinds": woodwinds_grid,
310
+ "brass": brass_grid,
311
+ "perc": perc_grid,
312
+ }
313
+
314
+
315
+ # =========================
316
+ # Orchestration from primary family
317
+ # =========================
318
+
319
+ def orchestrate_from_primary(
320
+ full_grids: Dict[str, np.ndarray],
321
+ primary_family: str,
322
+ ) -> Dict[str, np.ndarray]:
323
+ """
324
+ Build other families from a chosen primary melodic family using simple rules.
325
+
326
+ Args:
327
+ full_grids: raw model output:
328
+ "strings": [B, T, 12]
329
+ "woodwinds": [B, T, 12]
330
+ "brass": [B, T, 12]
331
+ "perc": [B, T, 3]
332
+ primary_family: one of "strings", "woodwinds", "brass".
333
+
334
+ Returns:
335
+ new_grids: dict with same keys, but non-primary families derived from primary.
336
+ """
337
+ grids = {k: v.copy() for k, v in full_grids.items()}
338
+
339
+ if primary_family not in ["strings", "woodwinds", "brass"]:
340
+ return grids
341
+
342
+ primary = grids[primary_family] # [B, T, 12]
343
+ B, T, P = primary.shape
344
+
345
+ # Reset melodic families to zero
346
+ for fam in ["strings", "woodwinds", "brass"]:
347
+ grids[fam] = np.zeros_like(primary)
348
+
349
+ strong_steps = [0, 2, 4, 6]
350
+
351
+ def thinned_copy(src: np.ndarray) -> np.ndarray:
352
+ dst = np.zeros_like(src)
353
+ dst[:, strong_steps, :] = src[:, strong_steps, :]
354
+ return dst
355
+
356
+ def root_pattern(src: np.ndarray) -> np.ndarray:
357
+ dst = np.zeros_like(src)
358
+ keep_steps = [0, 4]
359
+ dst[:, keep_steps, :] = src[:, keep_steps, :]
360
+ return dst
361
+
362
+ if primary_family == "strings":
363
+ grids["strings"] = primary
364
+ grids["woodwinds"] = thinned_copy(primary)
365
+ grids["brass"] = root_pattern(primary)
366
+ elif primary_family == "woodwinds":
367
+ grids["woodwinds"] = primary
368
+ grids["strings"] = thinned_copy(primary)
369
+ grids["brass"] = root_pattern(primary)
370
+ elif primary_family == "brass":
371
+ grids["brass"] = primary
372
+ grids["strings"] = thinned_copy(primary)
373
+ grids["woodwinds"] = root_pattern(primary)
374
+
375
+ # Percussion: simple fixed pattern
376
+ perc = np.zeros_like(grids["perc"])
377
+ for b in range(B):
378
+ for t in range(T):
379
+ if t in [0, 4]:
380
+ perc[b, t, 0] = 1.0 # kick-like
381
+ if t in [2, 6]:
382
+ perc[b, t, 1] = 1.0 # snare-like
383
+ grids["perc"] = perc
384
+
385
+ return grids
386
+
387
+
388
+ # =========================
389
+ # MIDI / audio helpers
390
+ # =========================
391
+
392
+ def generate_midi_file(
393
+ strings_seed: np.ndarray,
394
+ woodwinds_seed: np.ndarray,
395
+ brass_seed: np.ndarray,
396
+ perc_seed: np.ndarray,
397
+ target_minutes: float,
398
+ temperature: float,
399
+ top_k: int,
400
+ midi_out_path: str = "output.mid",
401
+ device: str = "cpu",
402
+ primary_family: str = "strings",
403
+ ) -> str:
404
+ """
405
+ High-level helper: generate music from seed grids and write to a MIDI file.
406
+
407
+ Returns:
408
+ Path to the written .mid file.
409
+ """
410
+ full_grids = generate_from_seed(
411
+ strings_seed=strings_seed,
412
+ woodwinds_seed=woodwinds_seed,
413
+ brass_seed=brass_seed,
414
+ perc_seed=perc_seed,
415
+ target_minutes=target_minutes,
416
+ temperature=temperature,
417
+ top_k=top_k,
418
+ device=device,
419
+ )
420
+
421
+ # Rebuild orchestration from chosen primary family
422
+ orchestrated = orchestrate_from_primary(full_grids, primary_family=primary_family)
423
+
424
+ pm = midi_utils.create_pretty_midi_from_grids(
425
+ strings_grid=orchestrated["strings"],
426
+ woodwinds_grid=orchestrated["woodwinds"],
427
+ brass_grid=orchestrated["brass"],
428
+ perc_grid=orchestrated["perc"],
429
+ )
430
+
431
+ os.makedirs(os.path.dirname(midi_out_path) or ".", exist_ok=True)
432
+ return midi_utils.save_midi(pm, midi_out_path)
433
+
434
+
435
+ def generate_midi_and_audio(
436
+ strings_seed: np.ndarray,
437
+ woodwinds_seed: np.ndarray,
438
+ brass_seed: np.ndarray,
439
+ perc_seed: np.ndarray,
440
+ target_minutes: float,
441
+ temperature: float,
442
+ top_k: int,
443
+ midi_out_path: str = "output.mid",
444
+ audio_out_path: str = "output.wav",
445
+ device: str = "cpu",
446
+ primary_family: str = "strings",
447
+ ) -> Tuple[str, str]:
448
+ """
449
+ Generate MIDI and render audio via Fluidsynth.
450
+
451
+ Returns:
452
+ (midi_path, audio_path)
453
+ """
454
+ midi_path = generate_midi_file(
455
+ strings_seed=strings_seed,
456
+ woodwinds_seed=woodwinds_seed,
457
+ brass_seed=brass_seed,
458
+ perc_seed=perc_seed,
459
+ target_minutes=target_minutes,
460
+ temperature=temperature,
461
+ top_k=top_k,
462
+ midi_out_path=midi_out_path,
463
+ device=device,
464
+ primary_family=primary_family,
465
+ )
466
+
467
+ # Render audio (requires fluidsynth and soundfont)
468
+ try:
469
+ import fluidsynth
470
+ import pretty_midi
471
+ from scipy.io import wavfile
472
+
473
+ fs = fluidsynth.Synth()
474
+ sfid = fs.sfload("soundfont.sf2")
475
+ fs.program_select(0, sfid, 0, 0)
476
+
477
+ pm = pretty_midi.PrettyMIDI(midi_path)
478
+ audio = pm.fluidsynth(fs=16000)
479
+ wavfile.write(audio_out_path, 16000, audio.astype(np.float32))
480
+ fs.delete()
481
+ except Exception as e:
482
+ print(f"Audio rendering failed: {e}")
483
+ audio_out_path = ""
484
+
485
+ return midi_path, audio_out_path
model.py ADDED
@@ -0,0 +1,163 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Transformer model for multi-instrument grid-based music generation.
3
+ """
4
+
5
+ from typing import Dict
6
+
7
+ import torch
8
+ import torch.nn as nn
9
+ import torch.nn.functional as F
10
+
11
+ import config
12
+
13
+
14
+ class PositionalEncoding(nn.Module):
15
+ """
16
+ Standard sinusoidal positional encoding for Transformer inputs.
17
+ Expects input of shape [batch, seq_len, d_model].
18
+ """
19
+
20
+ def __init__(self, d_model: int, max_len: int = 2048):
21
+ super().__init__()
22
+ self.d_model = d_model
23
+
24
+ pe = torch.zeros(max_len, d_model) # [max_len, d_model]
25
+ position = torch.arange(0, max_len, dtype=torch.float32).unsqueeze(1)
26
+ div_term = torch.exp(
27
+ torch.arange(0, d_model, 2, dtype=torch.float32)
28
+ * (-torch.log(torch.tensor(10000.0)) / d_model)
29
+ )
30
+
31
+ pe[:, 0::2] = torch.sin(position * div_term) # even indices
32
+ pe[:, 1::2] = torch.cos(position * div_term) # odd indices
33
+
34
+ pe = pe.unsqueeze(0) # [1, max_len, d_model]
35
+ self.register_buffer("pe", pe)
36
+
37
+ def forward(self, x: torch.Tensor) -> torch.Tensor:
38
+ """
39
+ Add positional encoding to input.
40
+
41
+ Args:
42
+ x: [batch, seq_len, d_model]
43
+
44
+ Returns:
45
+ [batch, seq_len, d_model]
46
+ """
47
+ seq_len = x.size(1)
48
+ # pe[:, :seq_len] has shape [1, seq_len, d_model]
49
+ return x + self.pe[:, :seq_len]
50
+
51
+
52
+ class MusicTransformer(nn.Module):
53
+ """
54
+ Transformer encoder with per-instrument output heads.
55
+
56
+ Input:
57
+ x: [batch, seq_len, INPUT_FEATURES] (multi-hot grid)
58
+
59
+ Output:
60
+ dict of logits:
61
+ {
62
+ "strings": [batch, seq_len, 12],
63
+ "woodwinds":[batch, seq_len, 12],
64
+ "brass": [batch, seq_len, 12],
65
+ "perc": [batch, seq_len, 3],
66
+ }
67
+ """
68
+
69
+ def __init__(
70
+ self,
71
+ input_dim: int = config.INPUT_FEATURES,
72
+ d_model: int = config.D_MODEL,
73
+ n_layers: int = config.N_LAYERS,
74
+ n_heads: int = config.N_HEADS,
75
+ ffn_dim: int = config.FFN_DIM,
76
+ dropout: float = config.DROPOUT,
77
+ ):
78
+ super().__init__()
79
+
80
+ self.input_dim = input_dim
81
+ self.d_model = d_model
82
+
83
+ # Project 39-dim multi-hot input to d_model
84
+ self.input_proj = nn.Linear(input_dim, d_model)
85
+
86
+ # Positional encoding
87
+ self.pos_encoding = PositionalEncoding(d_model=d_model)
88
+
89
+ # Transformer encoder
90
+ encoder_layer = nn.TransformerEncoderLayer(
91
+ d_model=d_model,
92
+ nhead=n_heads,
93
+ dim_feedforward=ffn_dim,
94
+ dropout=dropout,
95
+ batch_first=True, # [batch, seq, d_model]
96
+ )
97
+ self.transformer_encoder = nn.TransformerEncoder(
98
+ encoder_layer,
99
+ num_layers=n_layers,
100
+ )
101
+
102
+ # Per-instrument heads
103
+ self.strings_head = nn.Linear(d_model, config.PITCHES_MELODIC) # 12
104
+ self.woodwinds_head = nn.Linear(d_model, config.PITCHES_MELODIC) # 12
105
+ self.brass_head = nn.Linear(d_model, config.PITCHES_MELODIC) # 12
106
+ self.perc_head = nn.Linear(d_model, config.PITCHES_PERC) # 3
107
+
108
+ def forward(self, x: torch.Tensor) -> Dict[str, torch.Tensor]:
109
+ """
110
+ Forward pass.
111
+
112
+ Args:
113
+ x: Tensor of shape [batch, seq_len, INPUT_FEATURES]
114
+
115
+ Returns:
116
+ Dict of logits, each [batch, seq_len, pitches]:
117
+ {
118
+ "strings": logits_S,
119
+ "woodwinds": logits_W,
120
+ "brass": logits_B,
121
+ "perc": logits_P,
122
+ }
123
+ """
124
+ # Project input features to d_model
125
+ h = self.input_proj(x) # [batch, seq_len, d_model]
126
+
127
+ # Add positional encoding
128
+ h = self.pos_encoding(h) # [batch, seq_len, d_model]
129
+
130
+ # Transformer encoder
131
+ # (no src_key_padding_mask here; you can add later if needed)
132
+ h = self.transformer_encoder(h) # [batch, seq_len, d_model]
133
+
134
+ # Per-instrument logits
135
+ logits_strings = self.strings_head(h) # [batch, seq_len, 12]
136
+ logits_woodwinds = self.woodwinds_head(h)
137
+ logits_brass = self.brass_head(h)
138
+ logits_perc = self.perc_head(h) # [batch, seq_len, 3]
139
+
140
+ return {
141
+ "strings": logits_strings,
142
+ "woodwinds": logits_woodwinds,
143
+ "brass": logits_brass,
144
+ "perc": logits_perc,
145
+ }
146
+
147
+
148
+ def build_model(cfg=config) -> MusicTransformer:
149
+ """
150
+ Convenience function to build a MusicTransformer from config values.
151
+
152
+ Returns:
153
+ MusicTransformer instance with parameters from config.py.
154
+ """
155
+ model = MusicTransformer(
156
+ input_dim=cfg.INPUT_FEATURES,
157
+ d_model=cfg.D_MODEL,
158
+ n_layers=cfg.N_LAYERS,
159
+ n_heads=cfg.N_HEADS,
160
+ ffn_dim=cfg.FFN_DIM,
161
+ dropout=cfg.DROPOUT,
162
+ )
163
+ return model
models/transformer_music_best.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4dd4e457a4a7c11a0a0628e8c746cf07d370d8eba0eed7236b65fb2f966e101f
3
+ size 8522289
requirements.txt ADDED
@@ -0,0 +1,19 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Core ML
2
+ torch>=2.0.0
3
+
4
+ # Web UI
5
+ gradio>=4.0.0
6
+
7
+ # MIDI and audio
8
+ pretty_midi>=0.2.9
9
+ pyfluidsynth>=1.3.0
10
+ mido>=1.2.10
11
+
12
+ # Numerics
13
+ numpy>=1.24.0
14
+ scipy>=1.10.0
15
+
16
+ # Utilities (optional but common)
17
+ tqdm>=4.65.0
18
+
19
+ fluidsynth
train.py ADDED
@@ -0,0 +1,215 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ from typing import Tuple
3
+
4
+ import numpy as np
5
+ import torch
6
+ import torch.nn as nn
7
+ from torch.utils.data import DataLoader, TensorDataset, random_split
8
+
9
+ import config
10
+ from model import build_model
11
+
12
+
13
+ def load_dataset_npz(path: str) -> Tuple[torch.Tensor, torch.Tensor]:
14
+ data = np.load(path)
15
+ X = data["X"].astype(np.float32)
16
+ Y = data["Y"].astype(np.float32)
17
+ return torch.from_numpy(X), torch.from_numpy(Y)
18
+
19
+
20
+ def load_dataset_pt(path: str) -> Tuple[torch.Tensor, torch.Tensor]:
21
+ data = torch.load(path)
22
+ return data["X"].float(), data["Y"].float()
23
+
24
+
25
+ def train(
26
+ dataset_path: str,
27
+ use_npz: bool = True,
28
+ batch_size: int = 32,
29
+ num_epochs: int = 20,
30
+ val_split: float = 0.1,
31
+ learning_rate: float = 1e-3,
32
+ device: str = None,
33
+ save_dir: str = "models",
34
+ best_name: str = "transformer_music_best.pt",
35
+ last_name: str = "transformer_music_last.pt",
36
+ checkpoint_name: str = "transformer_music_checkpoint.pt",
37
+ extra_epochs: int = None,
38
+ ) -> None:
39
+ """
40
+ Train MusicTransformer with resume & best-model saving.
41
+
42
+ If extra_epochs is not None, the function will:
43
+ - load checkpoint (if any) and read checkpoint["epoch"] as last_epoch,
44
+ - start from last_epoch + 1,
45
+ - run for exactly extra_epochs epochs (end_epoch = last_epoch + extra_epochs).
46
+
47
+ If extra_epochs is None, num_epochs is treated as a global target:
48
+ - start from start_epoch (1 or checkpoint_epoch+1),
49
+ - run until epoch == num_epochs.
50
+ """
51
+
52
+ if device is None:
53
+ device = "cuda" if torch.cuda.is_available() else "cpu"
54
+
55
+ # Load dataset
56
+ if use_npz:
57
+ X, Y = load_dataset_npz(dataset_path)
58
+ else:
59
+ X, Y = load_dataset_pt(dataset_path)
60
+
61
+ print(f"Loaded dataset: X {X.shape}, Y {Y.shape}")
62
+
63
+ dataset = TensorDataset(X, Y)
64
+
65
+ # Train/val split
66
+ n_total = len(dataset)
67
+ n_val = int(n_total * val_split)
68
+ n_train = n_total - n_val
69
+
70
+ train_ds, val_ds = random_split(dataset, [n_train, n_val])
71
+
72
+ train_loader = DataLoader(train_ds, batch_size=batch_size, shuffle=True)
73
+ val_loader = DataLoader(val_ds, batch_size=batch_size, shuffle=False)
74
+
75
+ # Build model, optimizer, loss
76
+ model = build_model(config).to(device)
77
+ bce = nn.BCEWithLogitsLoss()
78
+ optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)
79
+
80
+ os.makedirs(save_dir, exist_ok=True)
81
+ best_model_path = os.path.join(save_dir, best_name)
82
+ last_model_path = os.path.join(save_dir, last_name)
83
+ checkpoint_path = os.path.join(save_dir, checkpoint_name)
84
+
85
+ # ===== Resume from checkpoint if present =====
86
+ start_epoch = 1
87
+ best_val_loss = float("inf")
88
+
89
+ if os.path.exists(checkpoint_path):
90
+ print(f"Found checkpoint at {checkpoint_path}, loading...")
91
+ checkpoint = torch.load(checkpoint_path, map_location=device)
92
+ model.load_state_dict(checkpoint["model_state_dict"])
93
+ optimizer.load_state_dict(checkpoint["optimizer_state_dict"])
94
+ start_epoch = checkpoint["epoch"] + 1
95
+ best_val_loss = checkpoint["best_val_loss"]
96
+ print(f"Resuming from epoch {start_epoch}, best_val_loss={best_val_loss:.4f}")
97
+
98
+ # Determine end_epoch based on extra_epochs / num_epochs
99
+ if extra_epochs is not None:
100
+ # Run exactly extra_epochs from the current checkpoint point
101
+ end_epoch = start_epoch + extra_epochs - 1
102
+ else:
103
+ # Run until global num_epochs
104
+ end_epoch = num_epochs
105
+
106
+ if start_epoch > end_epoch:
107
+ print(f"No epochs to run (start_epoch={start_epoch}, end_epoch={end_epoch}).")
108
+ print("Training complete.")
109
+ print(f"Best model: {best_model_path}")
110
+ print(f"Last model: {last_model_path}")
111
+ return
112
+
113
+ # ===== Training loop =====
114
+ for epoch in range(start_epoch, end_epoch + 1):
115
+ model.train()
116
+ train_loss_sum = 0.0
117
+ train_batches = 0
118
+
119
+ for X_batch, Y_batch in train_loader:
120
+ X_batch = X_batch.to(device) # [B, T, 39]
121
+ Y_batch = Y_batch.to(device) # [B, T, 39]
122
+
123
+ optimizer.zero_grad()
124
+
125
+ outputs = model(X_batch)
126
+
127
+ # Slice per-instrument targets
128
+ strings_target = Y_batch[:, :, 0:12]
129
+ woodwinds_target = Y_batch[:, :, 12:24]
130
+ brass_target = Y_batch[:, :, 24:36]
131
+ perc_target = Y_batch[:, :, 36:39]
132
+
133
+ loss_strings = bce(outputs["strings"], strings_target)
134
+ loss_wood = bce(outputs["woodwinds"], woodwinds_target)
135
+ loss_brass = bce(outputs["brass"], brass_target)
136
+ loss_perc = bce(outputs["perc"], perc_target)
137
+
138
+ loss = loss_strings + loss_wood + loss_brass + loss_perc
139
+
140
+ loss.backward()
141
+ optimizer.step()
142
+
143
+ train_loss_sum += loss.item()
144
+ train_batches += 1
145
+
146
+ avg_train_loss = train_loss_sum / max(train_batches, 1)
147
+
148
+ # Validation
149
+ model.eval()
150
+ val_loss_sum = 0.0
151
+ val_batches = 0
152
+ with torch.no_grad():
153
+ for X_batch, Y_batch in val_loader:
154
+ X_batch = X_batch.to(device)
155
+ Y_batch = Y_batch.to(device)
156
+
157
+ outputs = model(X_batch)
158
+
159
+ strings_target = Y_batch[:, :, 0:12]
160
+ woodwinds_target = Y_batch[:, :, 12:24]
161
+ brass_target = Y_batch[:, :, 24:36]
162
+ perc_target = Y_batch[:, :, 36:39]
163
+
164
+ loss_strings = bce(outputs["strings"], strings_target)
165
+ loss_wood = bce(outputs["woodwinds"], woodwinds_target)
166
+ loss_brass = bce(outputs["brass"], brass_target)
167
+ loss_perc = bce(outputs["perc"], perc_target)
168
+
169
+ loss = loss_strings + loss_wood + loss_perc + loss_brass
170
+
171
+ val_loss_sum += loss.item()
172
+ val_batches += 1
173
+
174
+ avg_val_loss = val_loss_sum / max(val_batches, 1)
175
+
176
+ print(
177
+ f"Epoch {epoch:03d} | "
178
+ f"Train loss: {avg_train_loss:.4f} | "
179
+ f"Val loss: {avg_val_loss:.4f}"
180
+ )
181
+
182
+ # ===== Save checkpoint (for resume) =====
183
+ checkpoint = {
184
+ "epoch": epoch,
185
+ "model_state_dict": model.state_dict(),
186
+ "optimizer_state_dict": optimizer.state_dict(),
187
+ "best_val_loss": best_val_loss,
188
+ }
189
+ torch.save(checkpoint, checkpoint_path)
190
+ print(f" ↳ Checkpoint saved to {checkpoint_path}")
191
+
192
+ # ===== Best model logic =====
193
+ if avg_val_loss < best_val_loss:
194
+ best_val_loss = avg_val_loss
195
+ torch.save(model.state_dict(), best_model_path)
196
+ print(f" ↳ New best model saved to {best_model_path}")
197
+
198
+ # Always save "last" model as well (optional)
199
+ torch.save(model.state_dict(), last_model_path)
200
+
201
+ print("Training complete.")
202
+ print(f"Best model: {best_model_path}")
203
+ print(f"Last model: {last_model_path}")
204
+
205
+
206
+ if __name__ == "__main__":
207
+ dataset_path = "musicnet_grid_dataset.npz"
208
+ train(
209
+ dataset_path=dataset_path,
210
+ use_npz=True,
211
+ batch_size=32,
212
+ num_epochs=20,
213
+ val_split=0.1,
214
+ learning_rate=1e-3,
215
+ )
utils/midi_utils.py ADDED
@@ -0,0 +1,242 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ MIDI utilities: convert between grid representation and PrettyMIDI.
3
+ """
4
+
5
+ from typing import Dict, Tuple, Optional
6
+
7
+ import numpy as np
8
+ import pretty_midi
9
+
10
+ import config
11
+
12
+
13
+ # Fixed MIDI pitches for one octave, center around middle range.
14
+ # You can adjust these if you want a different register.
15
+ # Example: C4 (60) to B4 (71) → 12 pitches.
16
+ MELODIC_BASE_PITCH: int = 60 # MIDI note for C4
17
+
18
+ # Percussion: General MIDI standard channel 9 (index 9), with typical drum notes.
19
+ PERCUSSION_CHANNEL: int = 9
20
+ PERC_KICK_PITCH: int = 36 # Bass Drum 1
21
+ PERC_SNARE_PITCH: int = 38 # Acoustic Snare
22
+ PERC_HIHAT_PITCH: int = 42 # Closed Hi-Hat
23
+
24
+
25
+ def create_pretty_midi_from_grids(
26
+ strings_grid: np.ndarray,
27
+ woodwinds_grid: np.ndarray,
28
+ brass_grid: np.ndarray,
29
+ perc_grid: np.ndarray,
30
+ velocity: int = 80,
31
+ ) -> pretty_midi.PrettyMIDI:
32
+ """
33
+ Convert per-family grids to a PrettyMIDI object.
34
+
35
+ Args:
36
+ strings_grid: [bars, STEPS_PER_BAR, 12] binary array.
37
+ woodwinds_grid: [bars, STEPS_PER_BAR, 12] binary array.
38
+ brass_grid: [bars, STEPS_PER_BAR, 12] binary array.
39
+ perc_grid: [bars, STEPS_PER_BAR, 3] binary array (Kick, Snare, Hi-hat).
40
+ velocity: MIDI velocity for melodic notes.
41
+
42
+ Returns:
43
+ pm: PrettyMIDI object with 4 instrument tracks.
44
+ """
45
+ # Basic checks
46
+ bars, steps_per_bar, n_pitch_strings = strings_grid.shape
47
+ assert steps_per_bar == config.STEPS_PER_BAR
48
+ assert n_pitch_strings == config.PITCHES_MELODIC
49
+
50
+ # Percussion shape check
51
+ bars_p, steps_per_bar_p, n_perc = perc_grid.shape
52
+ assert bars_p == bars
53
+ assert steps_per_bar_p == config.STEPS_PER_BAR
54
+ assert n_perc == config.PITCHES_PERC
55
+
56
+ pm = pretty_midi.PrettyMIDI()
57
+
58
+ # Create instrument tracks
59
+ strings_inst = pretty_midi.Instrument(
60
+ program=pretty_midi.instrument_name_to_program("Violin")
61
+ )
62
+ woodwinds_inst = pretty_midi.Instrument(
63
+ program=pretty_midi.instrument_name_to_program("Clarinet")
64
+ )
65
+ brass_inst = pretty_midi.Instrument(
66
+ program=pretty_midi.instrument_name_to_program("Trumpet")
67
+ )
68
+ # Percussion uses channel 9; PrettyMIDI uses program but channel is set internally.
69
+ perc_inst = pretty_midi.Instrument(program=0, is_drum=True)
70
+
71
+ # Duration per step
72
+ step_duration = config.SECONDS_PER_STEP
73
+
74
+ # Helper to convert (bar_idx, step_idx, pitch_idx) to start/end times
75
+ def cell_times(bar_idx: int, step_idx: int) -> Tuple[float, float]:
76
+ start = (bar_idx * config.STEPS_PER_BAR + step_idx) * step_duration
77
+ end = start + step_duration # simple: one-step duration
78
+ return start, end
79
+
80
+ # Melodic families
81
+ def add_melodic_notes(grid: np.ndarray, instrument: pretty_midi.Instrument) -> None:
82
+ """
83
+ Add notes to a melodic instrument from a [bars, steps, pitches] grid.
84
+ Each active cell becomes a note of one-step duration.
85
+ """
86
+ for b in range(bars):
87
+ for s in range(config.STEPS_PER_BAR):
88
+ for p in range(config.PITCHES_MELODIC):
89
+ if grid[b, s, p] > 0.5: # treat >0.5 as "on"
90
+ start, end = cell_times(b, s)
91
+ midi_pitch = MELODIC_BASE_PITCH + p
92
+ note = pretty_midi.Note(
93
+ velocity=velocity,
94
+ pitch=midi_pitch,
95
+ start=start,
96
+ end=end,
97
+ )
98
+ instrument.notes.append(note)
99
+
100
+ add_melodic_notes(strings_grid, strings_inst)
101
+ add_melodic_notes(woodwinds_grid, woodwinds_inst)
102
+ add_melodic_notes(brass_grid, brass_inst)
103
+
104
+ # Percussion
105
+ # Map indices 0,1,2 to Kick, Snare, Hi-hat.
106
+ PERC_PITCHES = [PERC_KICK_PITCH, PERC_SNARE_PITCH, PERC_HIHAT_PITCH]
107
+
108
+ for b in range(bars):
109
+ for s in range(config.STEPS_PER_BAR):
110
+ for k in range(config.PITCHES_PERC):
111
+ if perc_grid[b, s, k] > 0.5:
112
+ start, end = cell_times(b, s)
113
+ pitch = PERC_PITCHES[k]
114
+ note = pretty_midi.Note(
115
+ velocity=velocity,
116
+ pitch=pitch,
117
+ start=start,
118
+ end=end,
119
+ )
120
+ perc_inst.notes.append(note)
121
+
122
+ # Append instruments to the PrettyMIDI object
123
+ pm.instruments.append(strings_inst)
124
+ pm.instruments.append(woodwinds_inst)
125
+ pm.instruments.append(brass_inst)
126
+ pm.instruments.append(perc_inst)
127
+
128
+ return pm
129
+
130
+
131
+ def save_midi(
132
+ pm: pretty_midi.PrettyMIDI,
133
+ out_path: str,
134
+ ) -> str:
135
+ """
136
+ Save PrettyMIDI object to a .mid file.
137
+
138
+ Args:
139
+ pm: PrettyMIDI object.
140
+ out_path: Path to write .mid file.
141
+
142
+ Returns:
143
+ out_path: Same path, for convenience.
144
+ """
145
+ pm.write(out_path)
146
+ return out_path
147
+
148
+
149
+ # =========================
150
+ # Optional: MIDI → grid for debugging
151
+ # =========================
152
+
153
+ def midi_to_grids(
154
+ midi_path: str,
155
+ max_bars: Optional[int] = None,
156
+ ) -> Dict[str, np.ndarray]:
157
+ """
158
+ Very simple debugging helper: convert a MIDI file back into rough grids.
159
+ Assumes the file broadly follows the same tempo/grid assumptions.
160
+
161
+ This is intentionally approximate and mainly for inspection, not training.
162
+
163
+ Args:
164
+ midi_path: Path to a MIDI file.
165
+ max_bars: Optional max number of bars to decode.
166
+
167
+ Returns:
168
+ {
169
+ "strings": [bars, STEPS_PER_BAR, 12],
170
+ "woodwinds": [bars, STEPS_PER_BAR, 12],
171
+ "brass": [bars, STEPS_PER_BAR, 12],
172
+ "perc": [bars, STEPS_PER_BAR, 3],
173
+ }
174
+ """
175
+ pm = pretty_midi.PrettyMIDI(midi_path)
176
+
177
+ # Estimate total length in bars
178
+ total_seconds = pm.get_end_time()
179
+ bar_duration = config.SECONDS_PER_BAR
180
+ total_bars = int(np.ceil(total_seconds / bar_duration))
181
+
182
+ if max_bars is not None:
183
+ total_bars = min(total_bars, max_bars)
184
+
185
+ # Initialize empty grids
186
+ strings_grid = np.zeros((total_bars, config.STEPS_PER_BAR, config.PITCHES_MELODIC), dtype=np.float32)
187
+ woodwinds_grid = np.zeros_like(strings_grid)
188
+ brass_grid = np.zeros_like(strings_grid)
189
+ perc_grid = np.zeros((total_bars, config.STEPS_PER_BAR, config.PITCHES_PERC), dtype=np.float32)
190
+
191
+ def time_to_indices(t: float) -> Tuple[int, int]:
192
+ """
193
+ Convert time in seconds to (bar_idx, step_idx), clamped to grid.
194
+ """
195
+ step_idx_global = int(np.floor(t / config.SECONDS_PER_STEP))
196
+ bar_idx = step_idx_global // config.STEPS_PER_BAR
197
+ step_idx = step_idx_global % config.STEPS_PER_BAR
198
+ bar_idx = min(bar_idx, total_bars - 1)
199
+ return bar_idx, step_idx
200
+
201
+ # Very naive family routing: you may refine this mapping later.
202
+ for inst in pm.instruments:
203
+ # Percussion
204
+ if inst.is_drum:
205
+ for note in inst.notes:
206
+ b, s = time_to_indices(note.start)
207
+ if note.pitch == PERC_KICK_PITCH:
208
+ k = 0
209
+ elif note.pitch == PERC_SNARE_PITCH:
210
+ k = 1
211
+ elif note.pitch == PERC_HIHAT_PITCH:
212
+ k = 2
213
+ else:
214
+ continue
215
+ perc_grid[b, s, k] = 1.0
216
+ else:
217
+ # Choose family by program range (very rough heuristic).
218
+ program = inst.program
219
+ # Strings (e.g., Violin, Viola, Cello)
220
+ if program in range(40, 49):
221
+ target_grid = strings_grid
222
+ # Woodwinds
223
+ elif program in range(72, 81) or program in range(64, 73):
224
+ target_grid = woodwinds_grid
225
+ # Brass
226
+ elif program in range(56, 65):
227
+ target_grid = brass_grid
228
+ else:
229
+ target_grid = strings_grid # fallback
230
+
231
+ for note in inst.notes:
232
+ b, s = time_to_indices(note.start)
233
+ pitch_idx = note.pitch - MELODIC_BASE_PITCH
234
+ if 0 <= pitch_idx < config.PITCHES_MELODIC:
235
+ target_grid[b, s, pitch_idx] = 1.0
236
+
237
+ return {
238
+ "strings": strings_grid,
239
+ "woodwinds": woodwinds_grid,
240
+ "brass": brass_grid,
241
+ "perc": perc_grid,
242
+ }
utils/sampling_utils.py ADDED
@@ -0,0 +1,307 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Sampling and bar-level constraints for music generation.
3
+ """
4
+
5
+ from typing import Dict
6
+
7
+ import numpy as np
8
+ import torch
9
+ import torch.nn.functional as F # noqa: F401
10
+
11
+ import config
12
+
13
+ # =========================
14
+ # Simple music-theory rules
15
+ # =========================
16
+
17
+ # C major scale degrees within one octave (0–11):
18
+ # C, D, E, F, G, A, B
19
+ C_MAJOR_INDICES = [0, 2, 4, 5, 7, 9, 11]
20
+
21
+
22
+ def enforce_scale(
23
+ bar: Dict[str, np.ndarray],
24
+ allowed_indices=C_MAJOR_INDICES,
25
+ ) -> Dict[str, np.ndarray]:
26
+ """
27
+ Zero out melodic pitches not in the allowed scale for each melodic family.
28
+
29
+ Args:
30
+ bar: dict with keys "strings", "woodwinds", "brass", "perc".
31
+ allowed_indices: list of pitch indices (0–11) allowed in the scale.
32
+
33
+ Returns:
34
+ bar with non-scale pitches zeroed for melodic families.
35
+ """
36
+ mask = np.zeros((config.PITCHES_MELODIC,), dtype=np.float32)
37
+ mask[allowed_indices] = 1.0
38
+
39
+ for key in ["strings", "woodwinds", "brass"]:
40
+ grid = bar[key] # [T, 12]
41
+ bar[key] = grid * mask # broadcast over time
42
+
43
+ return bar
44
+
45
+
46
+ def limit_notes_per_step(
47
+ bar: Dict[str, np.ndarray],
48
+ max_notes_per_step: int = 4,
49
+ ) -> Dict[str, np.ndarray]:
50
+ """
51
+ For each melodic family and time step, restrict to at most max_notes_per_step notes.
52
+
53
+ This reduces dense vertical clusters and produces simpler chords.
54
+ """
55
+ for key in ["strings", "woodwinds", "brass"]:
56
+ grid = bar[key] # [T, pitches]
57
+ T, P = grid.shape
58
+ for t in range(T):
59
+ on_indices = np.where(grid[t] > 0.5)[0]
60
+ if len(on_indices) > max_notes_per_step:
61
+ # Keep the first few; zero out the rest
62
+ to_keep = on_indices[:max_notes_per_step]
63
+ mask = np.zeros_like(grid[t])
64
+ mask[to_keep] = 1.0
65
+ grid[t] = mask
66
+ bar[key] = grid
67
+ return bar
68
+
69
+
70
+ def count_total_notes(
71
+ bar: Dict[str, np.ndarray],
72
+ ) -> int:
73
+ """
74
+ Count total number of "on" notes across all instrument families in a bar.
75
+ """
76
+ total = 0
77
+ for key in ["strings", "woodwinds", "brass", "perc"]:
78
+ total += np.count_nonzero(bar[key] > 0.5)
79
+ return int(total)
80
+
81
+
82
+ def emphasize_strong_beats(
83
+ bar: Dict[str, np.ndarray],
84
+ ) -> Dict[str, np.ndarray]:
85
+ """
86
+ Encourage notes on strong beats (0, 2, 4, 6) for melodic families.
87
+ If a strong beat has no note in a family, randomly turn one on.
88
+ """
89
+ strong_steps = [0, 2, 4, 6]
90
+ for key in ["strings", "woodwinds", "brass"]:
91
+ grid = bar[key] # [T, 12]
92
+ T, P = grid.shape
93
+ for t in strong_steps:
94
+ if t >= T:
95
+ continue
96
+ if np.sum(grid[t] > 0.5) == 0:
97
+ p = np.random.randint(0, P)
98
+ grid[t, p] = 1.0
99
+ bar[key] = grid
100
+ return bar
101
+
102
+
103
+ # =========================
104
+ # Logit → binary grid sampling
105
+ # =========================
106
+
107
+ def sample_binary_from_logits(
108
+ logits: torch.Tensor,
109
+ temperature: float = 1.0,
110
+ top_k: int = 0,
111
+ ) -> torch.Tensor:
112
+ """
113
+ Sample binary activations from logits for one instrument.
114
+
115
+ Args:
116
+ logits: [T, P] tensor (time steps, pitches).
117
+ temperature: Temperature for scaling; >1.0 → more random.
118
+ top_k: If >0, keep only top_k logits per time step before sampling.
119
+
120
+ Returns:
121
+ activations: [T, P] tensor of 0/1 (float).
122
+ """
123
+ if temperature <= 0:
124
+ raise ValueError("temperature must be > 0")
125
+
126
+ # Scale by temperature
127
+ scaled_logits = logits / temperature
128
+
129
+ # Optional top-k per time step
130
+ if top_k > 0:
131
+ # Clamp k so it never exceeds number of pitches for this instrument
132
+ num_pitches = scaled_logits.shape[-1]
133
+ k = min(top_k, num_pitches)
134
+
135
+ values, indices = torch.topk(scaled_logits, k=k, dim=-1)
136
+ mask = torch.full_like(scaled_logits, float("-inf"))
137
+ mask.scatter_(-1, indices, values)
138
+ scaled_logits = mask
139
+
140
+ # Convert to probabilities
141
+ probs = torch.sigmoid(scaled_logits) # [T, P]
142
+
143
+ # Bernoulli sampling
144
+ bernoulli_samples = torch.bernoulli(probs)
145
+
146
+ return bernoulli_samples
147
+
148
+
149
+ def sample_bar_from_logits_dict(
150
+ logits_dict: Dict[str, torch.Tensor],
151
+ temperature: float = 1.0,
152
+ top_k: int = 0,
153
+ ) -> Dict[str, np.ndarray]:
154
+ """
155
+ Sample a full bar (all instruments) from per-instrument logits.
156
+
157
+ Args:
158
+ logits_dict:
159
+ "strings": [T, 12],
160
+ "woodwinds": [T, 12],
161
+ "brass": [T, 12],
162
+ "perc": [T, 3],
163
+ where T should be config.STEPS_PER_BAR.
164
+ temperature: Temperature for sampling.
165
+ top_k: top-k filter per instrument per time step.
166
+
167
+ Returns:
168
+ Dict of numpy arrays with shape [STEPS_PER_BAR, pitches] and 0/1 values.
169
+ """
170
+ bar_grids: Dict[str, np.ndarray] = {}
171
+
172
+ for name, logits in logits_dict.items():
173
+ # Expect [T, P]
174
+ if logits.dim() == 3:
175
+ logits = logits.squeeze(0) # [T, P]
176
+ assert logits.shape[0] == config.STEPS_PER_BAR, f"{name}: wrong T"
177
+
178
+ samples = sample_binary_from_logits(
179
+ logits=logits,
180
+ temperature=temperature,
181
+ top_k=top_k,
182
+ ) # [T, P]
183
+
184
+ bar_grids[name] = samples.cpu().numpy().astype(np.float32)
185
+
186
+ return bar_grids
187
+
188
+
189
+ # =========================
190
+ # Bar similarity and resampling
191
+ # =========================
192
+
193
+ def bar_similarity(
194
+ bar1: Dict[str, np.ndarray],
195
+ bar2: Dict[str, np.ndarray],
196
+ ) -> float:
197
+ """
198
+ Compute normalized similarity between two bars.
199
+
200
+ Both bars are dicts:
201
+ "strings": [STEPS_PER_BAR, 12],
202
+ "woodwinds": [STEPS_PER_BAR, 12],
203
+ "brass": [STEPS_PER_BAR, 12],
204
+ "perc": [STEPS_PER_BAR, 3],
205
+
206
+ Similarity is defined as:
207
+ 1 - (Hamming distance / total_cells)
208
+ """
209
+ flat1_list = []
210
+ flat2_list = []
211
+ for key in ["strings", "woodwinds", "brass", "perc"]:
212
+ g1 = bar1[key].astype(np.float32)
213
+ g2 = bar2[key].astype(np.float32)
214
+ flat1_list.append(g1.reshape(-1))
215
+ flat2_list.append(g2.reshape(-1))
216
+
217
+ flat1 = np.concatenate(flat1_list)
218
+ flat2 = np.concatenate(flat2_list)
219
+
220
+ total_cells = flat1.size
221
+ if total_cells == 0:
222
+ return 0.0
223
+
224
+ diff = np.abs(flat1 - flat2)
225
+ hamming = np.count_nonzero(diff)
226
+
227
+ similarity = 1.0 - (hamming / float(total_cells))
228
+ return float(similarity)
229
+
230
+
231
+ def count_active_families(
232
+ bar: Dict[str, np.ndarray],
233
+ ) -> int:
234
+ """
235
+ Count how many instrument families have at least one active note in the bar.
236
+ """
237
+ active_count = 0
238
+ for key in ["strings", "woodwinds", "brass", "perc"]:
239
+ grid = bar[key]
240
+ if np.any(grid > 0.5):
241
+ active_count += 1
242
+ return active_count
243
+
244
+
245
+ def resample_bar_if_needed(
246
+ prev_bar: Dict[str, np.ndarray],
247
+ logits_dict: Dict[str, torch.Tensor],
248
+ temperature: float,
249
+ top_k: int,
250
+ similarity_threshold: float = config.BAR_SIMILARITY_THRESHOLD,
251
+ max_attempts: int = config.MAX_RESAMPLE_ATTEMPTS_PER_BAR,
252
+ min_active_families: int = config.MIN_ACTIVE_FAMILIES_PER_BAR,
253
+ min_notes_per_bar: int = 2,
254
+ max_notes_per_step: int = 4,
255
+ ) -> Dict[str, np.ndarray]:
256
+ """
257
+ Sample a bar from logits, enforcing:
258
+ - Not too similar to prev_bar (if prev_bar is provided).
259
+ - At least min_active_families have notes.
260
+ - At least min_notes_per_bar total notes.
261
+ - At most max_notes_per_step notes per step per melodic family.
262
+ """
263
+ attempt = 0
264
+ best_bar = None
265
+ best_similarity = -1.0
266
+
267
+ while attempt < max_attempts:
268
+ attempt += 1
269
+
270
+ # Sample raw bar
271
+ candidate_bar = sample_bar_from_logits_dict(
272
+ logits_dict=logits_dict,
273
+ temperature=temperature,
274
+ top_k=top_k,
275
+ )
276
+
277
+ # Apply theory rules
278
+ candidate_bar = enforce_scale(candidate_bar)
279
+ candidate_bar = emphasize_strong_beats(candidate_bar)
280
+ candidate_bar = limit_notes_per_step(candidate_bar, max_notes_per_step=max_notes_per_step)
281
+
282
+ # Activity and density
283
+ active = count_active_families(candidate_bar)
284
+ total_notes = count_total_notes(candidate_bar)
285
+
286
+ # If no prev_bar, only activity + density constraints
287
+ if prev_bar is None:
288
+ if active >= min_active_families and total_notes >= min_notes_per_bar:
289
+ return candidate_bar
290
+ if best_bar is None:
291
+ best_bar = candidate_bar
292
+ continue
293
+
294
+ # Compare to previous bar
295
+ sim = bar_similarity(candidate_bar, prev_bar)
296
+
297
+ # Track best candidate for fallback
298
+ if sim > best_similarity:
299
+ best_similarity = sim
300
+ best_bar = candidate_bar
301
+
302
+ # Check constraints: not too similar, enough active families, enough notes
303
+ if (sim <= similarity_threshold) and (active >= min_active_families) and (total_notes >= min_notes_per_bar):
304
+ return candidate_bar
305
+
306
+ # Fallback
307
+ return best_bar