Spaces:
Running
Create a web-based audio visualizer that generates immersive, procedurally created visuals based on uploaded audio files. The system should leverage medically backed data on synesthesia, auditory perception, and cross-modal sensory integration to simulate how people with synesthesia may “see” music.
Browse filesCore Requirements:
Audio Input & Processing
Users can upload .wav or .mp3 files.
The engine extracts frequency bands, amplitude, tempo, and timbral qualities.
Use FFT or similar techniques to generate real-time analysis.
Medical/Scientific Backing
Base color, motion, and particle mapping rules on published neurological and psychological studies of synesthesia (e.g., sound-color, pitch-shape, rhythm-motion correlations).
Allow configurable “synesthesia modes” such as Chromesthesia (sound → color), Auditory-Tactile (sound → vibration patterns), etc.
Visualization Engine
Render visuals in WebGL (Three.js or similar) for smooth, GPU-accelerated performance.
Procedurally generate glowing particle fields, fluid motions, and geometric transformations tied to music intensity and frequency.
Ensure visuals evolve dynamically over time, not just repeating loops.
UI/UX
Simple upload interface for audio files.
Play/Pause controls with timeline scrubbing.
Option to toggle different visualization “modes” (scientifically inspired presets).
Responsive design so it works on desktop and mobile browsers.
Extendability
Architecture should allow adding new medically informed mappings later (e.g., linking timbre → texture, harmony → spatial depth).
Export visuals as high-resolution video or shareable web links.
Deliverables:
A web application prototype (React + Three.js or similar) hosted in the browser.
A modular procedural visualization engine that maps sound features to medically grounded sensory equivalents.
Clean, well-documented code so mappings can be extended with new research data. - Initial Deployment
- README.md +7 -5
- index.html +358 -18
- prompts.txt +49 -0
|
@@ -1,10 +1,12 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
|
|
|
|
|
|
| 8 |
---
|
| 9 |
|
| 10 |
-
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|
|
|
|
| 1 |
---
|
| 2 |
+
title: synesthesia
|
| 3 |
+
emoji: 🐳
|
| 4 |
+
colorFrom: gray
|
| 5 |
+
colorTo: green
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
| 8 |
+
tags:
|
| 9 |
+
- deepsite
|
| 10 |
---
|
| 11 |
|
| 12 |
+
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|
|
@@ -1,19 +1,359 @@
|
|
| 1 |
-
<!
|
| 2 |
-
<html>
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
</html>
|
|
|
|
| 1 |
+
<!DOCTYPE html>
|
| 2 |
+
<html lang="en">
|
| 3 |
+
<head>
|
| 4 |
+
<meta charset="UTF-8">
|
| 5 |
+
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
| 6 |
+
<title>Synesthetic Audio Visualizer</title>
|
| 7 |
+
<script src="https://cdn.tailwindcss.com"></script>
|
| 8 |
+
<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r128/three.min.js"></script>
|
| 9 |
+
<script src="https://cdn.jsdelivr.net/npm/vanta@latest/dist/vanta.globe.min.js"></script>
|
| 10 |
+
<script src="https://unpkg.com/feather-icons"></script>
|
| 11 |
+
<script src="https://cdn.jsdelivr.net/npm/p5@1.4.0/lib/p5.js"></script>
|
| 12 |
+
<style>
|
| 13 |
+
#audioCanvas {
|
| 14 |
+
position: absolute;
|
| 15 |
+
top: 0;
|
| 16 |
+
left: 0;
|
| 17 |
+
width: 100%;
|
| 18 |
+
height: 100%;
|
| 19 |
+
z-index: -1;
|
| 20 |
+
}
|
| 21 |
+
.file-upload:hover {
|
| 22 |
+
transform: translateY(-2px);
|
| 23 |
+
box-shadow: 0 10px 20px rgba(0, 0, 0, 0.2);
|
| 24 |
+
}
|
| 25 |
+
.synesthesia-mode:hover {
|
| 26 |
+
transform: scale(1.05);
|
| 27 |
+
transition: all 0.3s ease;
|
| 28 |
+
}
|
| 29 |
+
</style>
|
| 30 |
+
</head>
|
| 31 |
+
<body class="bg-gray-900 text-white min-h-screen flex flex-col">
|
| 32 |
+
<div id="vanta-bg" class="fixed inset-0 opacity-20"></div>
|
| 33 |
+
|
| 34 |
+
<header class="py-6 px-4 sm:px-6 lg:px-8 text-center">
|
| 35 |
+
<h1 class="text-4xl md:text-5xl font-bold mb-2 bg-clip-text text-transparent bg-gradient-to-r from-purple-400 via-pink-500 to-red-500">
|
| 36 |
+
Synesthetic Audio Visualizer
|
| 37 |
+
</h1>
|
| 38 |
+
<p class="text-lg text-gray-300 max-w-2xl mx-auto">
|
| 39 |
+
Experience music through the lens of synesthesia. Scientifically grounded audio-visual mappings.
|
| 40 |
+
</p>
|
| 41 |
+
</header>
|
| 42 |
+
|
| 43 |
+
<main class="flex-grow container mx-auto px-4 py-8 flex flex-col items-center">
|
| 44 |
+
<div class="w-full max-w-2xl bg-gray-800 bg-opacity-70 backdrop-blur-lg rounded-xl shadow-2xl p-6 mb-8">
|
| 45 |
+
<div class="flex flex-col sm:flex-row gap-4 mb-6">
|
| 46 |
+
<label class="file-upload flex-1 cursor-pointer">
|
| 47 |
+
<input type="file" id="audioUpload" accept=".mp3,.wav" class="hidden">
|
| 48 |
+
<div class="flex items-center justify-center bg-gray-700 hover:bg-gray-600 rounded-lg p-4 transition-all duration-300">
|
| 49 |
+
<i data-feather="upload" class="mr-2"></i>
|
| 50 |
+
<span>Choose Audio File</span>
|
| 51 |
+
</div>
|
| 52 |
+
</label>
|
| 53 |
+
<button id="playBtn" class="bg-purple-600 hover:bg-purple-700 px-6 py-3 rounded-lg font-medium flex items-center justify-center transition-colors">
|
| 54 |
+
<i data-feather="play" class="mr-2"></i>
|
| 55 |
+
Play
|
| 56 |
+
</button>
|
| 57 |
+
</div>
|
| 58 |
+
|
| 59 |
+
<div class="mb-6">
|
| 60 |
+
<div class="flex justify-between text-sm text-gray-400 mb-1">
|
| 61 |
+
<span id="currentTime">0:00</span>
|
| 62 |
+
<span id="totalTime">0:00</span>
|
| 63 |
+
</div>
|
| 64 |
+
<input type="range" id="progressBar" class="w-full h-2 bg-gray-700 rounded-lg appearance-none cursor-pointer" value="0">
|
| 65 |
+
</div>
|
| 66 |
+
|
| 67 |
+
<div class="grid grid-cols-2 sm:grid-cols-3 gap-3">
|
| 68 |
+
<button class="synesthesia-mode bg-gradient-to-br from-blue-500 to-indigo-600 px-4 py-3 rounded-lg font-medium transition-all" data-mode="chromesthesia">
|
| 69 |
+
Chromesthesia
|
| 70 |
+
</button>
|
| 71 |
+
<button class="synesthesia-mode bg-gradient-to-br from-green-500 to-teal-600 px-4 py-3 rounded-lg font-medium transition-all" data-mode="tactile">
|
| 72 |
+
Auditory-Tactile
|
| 73 |
+
</button>
|
| 74 |
+
<button class="synesthesia-mode bg-gradient-to-br from-red-500 to-orange-600 px-4 py-3 rounded-lg font-medium transition-all" data-mode="spatial">
|
| 75 |
+
Spatial Harmony
|
| 76 |
+
</button>
|
| 77 |
+
</div>
|
| 78 |
+
</div>
|
| 79 |
+
|
| 80 |
+
<div id="visualizerContainer" class="relative w-full max-w-4xl aspect-square rounded-xl overflow-hidden shadow-2xl">
|
| 81 |
+
<div id="audioCanvas"></div>
|
| 82 |
+
<div class="absolute inset-0 flex items-center justify-center bg-black bg-opacity-30" id="uploadPrompt">
|
| 83 |
+
<div class="text-center p-6">
|
| 84 |
+
<i data-feather="music" class="w-12 h-12 mx-auto mb-4 text-gray-300"></i>
|
| 85 |
+
<h3 class="text-xl font-medium mb-2">Upload an audio file</h3>
|
| 86 |
+
<p class="text-gray-300">Supported formats: MP3, WAV</p>
|
| 87 |
+
</div>
|
| 88 |
+
</div>
|
| 89 |
+
</div>
|
| 90 |
+
</main>
|
| 91 |
+
|
| 92 |
+
<footer class="py-6 px-4 text-center text-gray-400 text-sm">
|
| 93 |
+
<p>Based on neurological research into synesthesia and cross-modal perception</p>
|
| 94 |
+
</footer>
|
| 95 |
+
|
| 96 |
+
<script>
|
| 97 |
+
feather.replace();
|
| 98 |
+
|
| 99 |
+
// Initialize Vanta.js background
|
| 100 |
+
VANTA.GLOBE({
|
| 101 |
+
el: "#vanta-bg",
|
| 102 |
+
mouseControls: true,
|
| 103 |
+
touchControls: true,
|
| 104 |
+
gyroControls: false,
|
| 105 |
+
minHeight: 200.00,
|
| 106 |
+
minWidth: 200.00,
|
| 107 |
+
scale: 1.00,
|
| 108 |
+
scaleMobile: 1.00,
|
| 109 |
+
color: 0x8a4fff,
|
| 110 |
+
backgroundColor: 0x09090b,
|
| 111 |
+
size: 0.8
|
| 112 |
+
});
|
| 113 |
+
|
| 114 |
+
// Audio visualization setup
|
| 115 |
+
let audioContext, analyser, dataArray, source;
|
| 116 |
+
let isPlaying = false;
|
| 117 |
+
let currentMode = 'chromesthesia';
|
| 118 |
+
let audioElement, gainNode;
|
| 119 |
+
|
| 120 |
+
document.getElementById('audioUpload').addEventListener('change', function(e) {
|
| 121 |
+
const file = e.target.files[0];
|
| 122 |
+
if (file) {
|
| 123 |
+
document.getElementById('uploadPrompt').style.display = 'none';
|
| 124 |
+
|
| 125 |
+
if (audioElement) {
|
| 126 |
+
audioElement.pause();
|
| 127 |
+
audioElement = null;
|
| 128 |
+
}
|
| 129 |
+
|
| 130 |
+
audioElement = new Audio(URL.createObjectURL(file));
|
| 131 |
+
document.getElementById('playBtn').innerHTML = '<i data-feather="play" class="mr-2"></i>Play';
|
| 132 |
+
feather.replace();
|
| 133 |
+
|
| 134 |
+
// Set up audio context on first interaction
|
| 135 |
+
if (!audioContext) {
|
| 136 |
+
audioContext = new (window.AudioContext || window.webkitAudioContext)();
|
| 137 |
+
analyser = audioContext.createAnalyser();
|
| 138 |
+
analyser.fftSize = 256;
|
| 139 |
+
gainNode = audioContext.createGain();
|
| 140 |
+
|
| 141 |
+
source = audioContext.createMediaElementSource(audioElement);
|
| 142 |
+
source.connect(gainNode);
|
| 143 |
+
gainNode.connect(analyser);
|
| 144 |
+
analyser.connect(audioContext.destination);
|
| 145 |
+
|
| 146 |
+
dataArray = new Uint8Array(analyser.frequencyBinCount);
|
| 147 |
+
}
|
| 148 |
+
|
| 149 |
+
// Set up duration display
|
| 150 |
+
audioElement.addEventListener('loadedmetadata', function() {
|
| 151 |
+
const duration = audioElement.duration;
|
| 152 |
+
const minutes = Math.floor(duration / 60);
|
| 153 |
+
const seconds = Math.floor(duration % 60).toString().padStart(2, '0');
|
| 154 |
+
document.getElementById('totalTime').textContent = `${minutes}:${seconds}`;
|
| 155 |
+
});
|
| 156 |
+
|
| 157 |
+
// Update progress bar
|
| 158 |
+
audioElement.addEventListener('timeupdate', function() {
|
| 159 |
+
const currentTime = audioElement.currentTime;
|
| 160 |
+
const duration = audioElement.duration || 1;
|
| 161 |
+
const minutes = Math.floor(currentTime / 60);
|
| 162 |
+
const seconds = Math.floor(currentTime % 60).toString().padStart(2, '0');
|
| 163 |
+
document.getElementById('currentTime').textContent = `${minutes}:${seconds}`;
|
| 164 |
+
document.getElementById('progressBar').value = (currentTime / duration) * 100;
|
| 165 |
+
});
|
| 166 |
+
|
| 167 |
+
// Set up progress bar scrubbing
|
| 168 |
+
document.getElementById('progressBar').addEventListener('input', function() {
|
| 169 |
+
if (audioElement.duration) {
|
| 170 |
+
audioElement.currentTime = (this.value / 100) * audioElement.duration;
|
| 171 |
+
}
|
| 172 |
+
});
|
| 173 |
+
|
| 174 |
+
// Initialize p5.js visualizer
|
| 175 |
+
new p5(function(p) {
|
| 176 |
+
let particles = [];
|
| 177 |
+
|
| 178 |
+
p.setup = function() {
|
| 179 |
+
const container = document.getElementById('visualizerContainer');
|
| 180 |
+
const canvas = p.createCanvas(container.clientWidth, container.clientHeight);
|
| 181 |
+
canvas.parent('audioCanvas');
|
| 182 |
+
|
| 183 |
+
// Initialize particles based on mode
|
| 184 |
+
initParticles();
|
| 185 |
+
};
|
| 186 |
+
|
| 187 |
+
p.windowResized = function() {
|
| 188 |
+
const container = document.getElementById('visualizerContainer');
|
| 189 |
+
p.resizeCanvas(container.clientWidth, container.clientHeight);
|
| 190 |
+
};
|
| 191 |
+
|
| 192 |
+
function initParticles() {
|
| 193 |
+
particles = [];
|
| 194 |
+
const count = currentMode === 'spatial' ? 500 : 200;
|
| 195 |
+
|
| 196 |
+
for (let i = 0; i < count; i++) {
|
| 197 |
+
particles.push({
|
| 198 |
+
x: p.random(p.width),
|
| 199 |
+
y: p.random(p.height),
|
| 200 |
+
size: p.random(2, 6),
|
| 201 |
+
speed: p.random(0.5, 2),
|
| 202 |
+
angle: p.random(p.TWO_PI),
|
| 203 |
+
color: getRandomColor(),
|
| 204 |
+
freqIndex: Math.floor(p.random(analyser.frequencyBinCount))
|
| 205 |
+
});
|
| 206 |
+
}
|
| 207 |
+
}
|
| 208 |
+
|
| 209 |
+
function getRandomColor() {
|
| 210 |
+
if (currentMode === 'chromesthesia') {
|
| 211 |
+
return p.color(
|
| 212 |
+
p.random(100, 255),
|
| 213 |
+
p.random(100, 255),
|
| 214 |
+
p.random(100, 255),
|
| 215 |
+
p.random(150, 220)
|
| 216 |
+
);
|
| 217 |
+
} else if (currentMode === 'tactile') {
|
| 218 |
+
return p.color(
|
| 219 |
+
100 + p.random(100),
|
| 220 |
+
100 + p.random(100),
|
| 221 |
+
255,
|
| 222 |
+
p.random(180, 220)
|
| 223 |
+
);
|
| 224 |
+
} else {
|
| 225 |
+
return p.color(
|
| 226 |
+
255,
|
| 227 |
+
100 + p.random(100),
|
| 228 |
+
100 + p.random(100),
|
| 229 |
+
p.random(180, 220)
|
| 230 |
+
);
|
| 231 |
+
}
|
| 232 |
+
}
|
| 233 |
+
|
| 234 |
+
p.draw = function() {
|
| 235 |
+
p.background(0, 0, 0, 25);
|
| 236 |
+
|
| 237 |
+
if (analyser && audioElement && !audioElement.paused) {
|
| 238 |
+
analyser.getByteFrequencyData(dataArray);
|
| 239 |
+
|
| 240 |
+
// Draw particles based on current mode
|
| 241 |
+
if (currentMode === 'chromesthesia') {
|
| 242 |
+
drawChromesthesia();
|
| 243 |
+
} else if (currentMode === 'tactile') {
|
| 244 |
+
drawTactile();
|
| 245 |
+
} else {
|
| 246 |
+
drawSpatial();
|
| 247 |
+
}
|
| 248 |
+
}
|
| 249 |
+
};
|
| 250 |
+
|
| 251 |
+
function drawChromesthesia() {
|
| 252 |
+
p.noStroke();
|
| 253 |
+
const centerX = p.width / 2;
|
| 254 |
+
const centerY = p.height / 2;
|
| 255 |
+
|
| 256 |
+
particles.forEach(part => {
|
| 257 |
+
const freqValue = dataArray[part.freqIndex] / 255;
|
| 258 |
+
|
| 259 |
+
// Move particle in circular pattern
|
| 260 |
+
part.angle += part.speed * 0.01;
|
| 261 |
+
part.x = centerX + Math.cos(part.angle) * (100 + freqValue * 200);
|
| 262 |
+
part.y = centerY + Math.sin(part.angle) * (100 + freqValue * 200);
|
| 263 |
+
|
| 264 |
+
// Size and color change with frequency
|
| 265 |
+
const pulseSize = part.size * (1 + freqValue * 2);
|
| 266 |
+
part.color.setAlpha(150 + freqValue * 105);
|
| 267 |
+
|
| 268 |
+
p.fill(part.color);
|
| 269 |
+
p.circle(part.x, part.y, pulseSize);
|
| 270 |
+
});
|
| 271 |
+
}
|
| 272 |
+
|
| 273 |
+
function drawTactile() {
|
| 274 |
+
p.noStroke();
|
| 275 |
+
const centerY = p.height / 2;
|
| 276 |
+
|
| 277 |
+
particles.forEach(part => {
|
| 278 |
+
const freqValue = dataArray[part.freqIndex] / 255;
|
| 279 |
+
|
| 280 |
+
// Vertical wave motion
|
| 281 |
+
part.y = centerY + Math.sin(p.frameCount * 0.02 * part.speed + part.x * 0.01) * 100 * freqValue;
|
| 282 |
+
|
| 283 |
+
// Pulsing effect
|
| 284 |
+
const pulseSize = part.size * (1 + freqValue * 3);
|
| 285 |
+
part.color.setAlpha(180 + freqValue * 75);
|
| 286 |
+
|
| 287 |
+
p.fill(part.color);
|
| 288 |
+
p.ellipse(part.x, part.y, pulseSize, pulseSize * 0.6);
|
| 289 |
+
|
| 290 |
+
// Move horizontally slightly
|
| 291 |
+
part.x += part.speed * 0.1;
|
| 292 |
+
if (part.x > p.width) part.x = 0;
|
| 293 |
+
});
|
| 294 |
+
}
|
| 295 |
+
|
| 296 |
+
function drawSpatial() {
|
| 297 |
+
p.noStroke();
|
| 298 |
+
const centerX = p.width / 2;
|
| 299 |
+
const centerY = p.height / 2;
|
| 300 |
+
|
| 301 |
+
particles.forEach(part => {
|
| 302 |
+
const freqValue = dataArray[part.freqIndex] / 255;
|
| 303 |
+
|
| 304 |
+
// 3D effect with perspective
|
| 305 |
+
const z = freqValue * 5;
|
| 306 |
+
const scale = p.map(z, 0, 5, 0.2, 1.5);
|
| 307 |
+
const x = centerX + (part.x - centerX) * scale;
|
| 308 |
+
const y = centerY + (part.y - centerY) * scale;
|
| 309 |
+
|
| 310 |
+
part.color.setAlpha(100 + freqValue * 155);
|
| 311 |
+
p.fill(part.color);
|
| 312 |
+
p.circle(x, y, part.size * scale);
|
| 313 |
+
|
| 314 |
+
// Connect particles that are close
|
| 315 |
+
p.stroke(part.color);
|
| 316 |
+
p.strokeWeight(0.5);
|
| 317 |
+
particles.forEach(other => {
|
| 318 |
+
if (other !== part && p.dist(x, y, other.x, other.y) < 50 * scale) {
|
| 319 |
+
p.line(x, y, other.x, other.y);
|
| 320 |
+
}
|
| 321 |
+
});
|
| 322 |
+
});
|
| 323 |
+
}
|
| 324 |
+
}, 'audioCanvas');
|
| 325 |
+
}
|
| 326 |
+
});
|
| 327 |
+
|
| 328 |
+
// Play/pause button
|
| 329 |
+
document.getElementById('playBtn').addEventListener('click', function() {
|
| 330 |
+
if (audioElement) {
|
| 331 |
+
if (audioElement.paused) {
|
| 332 |
+
audioElement.play();
|
| 333 |
+
this.innerHTML = '<i data-feather="pause" class="mr-2"></i>Pause';
|
| 334 |
+
if (audioContext.state === 'suspended') {
|
| 335 |
+
audioContext.resume();
|
| 336 |
+
}
|
| 337 |
+
} else {
|
| 338 |
+
audioElement.pause();
|
| 339 |
+
this.innerHTML = '<i data-feather="play" class="mr-2"></i>Play';
|
| 340 |
+
}
|
| 341 |
+
feather.replace();
|
| 342 |
+
}
|
| 343 |
+
});
|
| 344 |
+
|
| 345 |
+
// Synesthesia mode buttons
|
| 346 |
+
document.querySelectorAll('.synesthesia-mode').forEach(button => {
|
| 347 |
+
button.addEventListener('click', function() {
|
| 348 |
+
currentMode = this.dataset.mode;
|
| 349 |
+
document.querySelectorAll('.synesthesia-mode').forEach(btn => {
|
| 350 |
+
btn.classList.remove('ring-2', 'ring-white');
|
| 351 |
+
});
|
| 352 |
+
this.classList.add('ring-2', 'ring-white');
|
| 353 |
+
|
| 354 |
+
// Reinitialize visualization (handled in p5 instance)
|
| 355 |
+
});
|
| 356 |
+
});
|
| 357 |
+
</script>
|
| 358 |
+
</body>
|
| 359 |
</html>
|
|
@@ -0,0 +1,49 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Create a web-based audio visualizer that generates immersive, procedurally created visuals based on uploaded audio files. The system should leverage medically backed data on synesthesia, auditory perception, and cross-modal sensory integration to simulate how people with synesthesia may “see” music.
|
| 2 |
+
|
| 3 |
+
Core Requirements:
|
| 4 |
+
|
| 5 |
+
Audio Input & Processing
|
| 6 |
+
|
| 7 |
+
Users can upload .wav or .mp3 files.
|
| 8 |
+
|
| 9 |
+
The engine extracts frequency bands, amplitude, tempo, and timbral qualities.
|
| 10 |
+
|
| 11 |
+
Use FFT or similar techniques to generate real-time analysis.
|
| 12 |
+
|
| 13 |
+
Medical/Scientific Backing
|
| 14 |
+
|
| 15 |
+
Base color, motion, and particle mapping rules on published neurological and psychological studies of synesthesia (e.g., sound-color, pitch-shape, rhythm-motion correlations).
|
| 16 |
+
|
| 17 |
+
Allow configurable “synesthesia modes” such as Chromesthesia (sound → color), Auditory-Tactile (sound → vibration patterns), etc.
|
| 18 |
+
|
| 19 |
+
Visualization Engine
|
| 20 |
+
|
| 21 |
+
Render visuals in WebGL (Three.js or similar) for smooth, GPU-accelerated performance.
|
| 22 |
+
|
| 23 |
+
Procedurally generate glowing particle fields, fluid motions, and geometric transformations tied to music intensity and frequency.
|
| 24 |
+
|
| 25 |
+
Ensure visuals evolve dynamically over time, not just repeating loops.
|
| 26 |
+
|
| 27 |
+
UI/UX
|
| 28 |
+
|
| 29 |
+
Simple upload interface for audio files.
|
| 30 |
+
|
| 31 |
+
Play/Pause controls with timeline scrubbing.
|
| 32 |
+
|
| 33 |
+
Option to toggle different visualization “modes” (scientifically inspired presets).
|
| 34 |
+
|
| 35 |
+
Responsive design so it works on desktop and mobile browsers.
|
| 36 |
+
|
| 37 |
+
Extendability
|
| 38 |
+
|
| 39 |
+
Architecture should allow adding new medically informed mappings later (e.g., linking timbre → texture, harmony → spatial depth).
|
| 40 |
+
|
| 41 |
+
Export visuals as high-resolution video or shareable web links.
|
| 42 |
+
|
| 43 |
+
Deliverables:
|
| 44 |
+
|
| 45 |
+
A web application prototype (React + Three.js or similar) hosted in the browser.
|
| 46 |
+
|
| 47 |
+
A modular procedural visualization engine that maps sound features to medically grounded sensory equivalents.
|
| 48 |
+
|
| 49 |
+
Clean, well-documented code so mappings can be extended with new research data.
|