Synesthetica is an experimental interactive canvas where painting becomes music.
Every brush stroke generates sound in real time, transforming visual gestures into melodic compositions.
Instead of creating music with instruments, Synesthetica lets you compose through movement and color.
Try the experience here:
👉 https://synesthetica.vercel.app/
Synesthetica explores the idea of synesthesia — the blending of senses.
In this project:
- Visual gestures become sound
- Colors influence musical tone
- Brush movement creates rhythm
- Each painting becomes a replayable performance
The goal is to rethink creative tools and explore a multisensory interaction between vision and sound.
- Free-form painting canvas
- Real-time sound generation
- Gesture-based melody creation
- Replay mode to watch the painting process as a performance
- Composition recording
- Minimal interface focused on creativity
- Works across desktop, tablet, and mobile
The system maps visual input to sound:
- Brush strokes → musical notes
- Stroke speed → rhythm
- Canvas position → pitch
- Color → instrument or tone
This creates a dynamic musical composition while painting.
Synesthetica explores how creative tools can evolve when gesture, sound, and visual expression merge into one experience.
Synesthetica is inspired by:
- Synesthetic art experiences
- Generative music systems
- Interactive design experiments
- Performance-based creative tools
Possible future improvements:
- Collaborative painting sessions
- Export compositions as audio or video
- AI-assisted harmony generation
- Public gallery of user creations
Rohit
Exploring creative interactions between design, motion, and sound.
MIT License