Interactive Museum Installations - Simulation Of Ancient Musical Instrument ​

This thesis presents the design and implementation of an interactive audio-embedded system that enables museum visitors to engage with ancient musical instruments through a digitally controlled physical replica that responds to user gestures. The core objective is to emulate the physical interaction and auditory feedback of traditional instruments by translating gestural inputs into dynamic sound outputs. The system is built around an ESP32 microcontroller that processes real-time data from an inertial measurement unit (IMU), composed of accelerometer and gyroscope sensors. These sensors capture motion dynamics such as angular velocity and tilt orientation, which are analyzed to determine the user’s gestural intent. Based on this analysis, the system plays back pre-recorded sound files stored in its internal memory, with playback volume dynamically adjusted in response to the intensity of movement. Audio output is delivered through a digital I2S interface connected to a speaker module, enabling high-fidelity reproduction of sound samples. In addition to audio feedback, a vibration motor is used to provide haptic feedback, with vibration strength also modulated proportionally to the user's motion. This dual-modality response—combining sound and vibration—enhances the realism and immersiveness of the interaction. The system is capable of recognizing varying motion patterns and mapping them to different levels of sound output, simulating the tactile nuances of playing real historical instruments. Moreover, gesture detection and response logic are implemented using a finite state machine, allowing consistent and reliable transitions between interaction phases. By combining embedded hardware, motion sensing, and audio-haptic feedback, this work offers a novel approach to preserving and reinterpreting musical heritage through interactive technology. The resulting prototype provides a meaningful and educational experience for museum visitors, bridging the gap between historical artifacts and modern digital engagement.