Real-Time MIR Algorithms for Music-Reactive Game World Generation
(2022)DOI: 10.18420/MUC2022-MCI-WS03-225
Abstract
In music education, an important learning objective is to acquire a sense of rhythm and melody, e.g., by tapping or singing along to a musical performance. In this context, music games are becoming increasingly popular due to their accessibility and interactivity. However, existing games are often tailored to a specific musical repertoire. In this paper, we will explore the potential of music information retrieval (MIR) algorithms for generating dynamic game worlds that are reactive to musical input in real-time, e.g., to music from a stereo system or a live performance. Such games may be used to explore musical performances in an interactive and playful way. As a case study, we therefore developed a jump-andrun game prototype that challenges players to tap along the beat of the music in their environment. Its music-reactive game world relies on past and predicted beat positions estimated by a real-time beat tracking algorithm.
