Soundscape generation with stable diffusion using abstract emotion descriptors

This thesis explores the generation of soundscapes and audio sequences using the Riffusion model, which leverages stable diffusion techniques to produce spectrograms from abstract emotion signifiers such as valence and arousal scores. Utilizing the DEAM dataset, the project involves creating spectrograms from song files and training a new model capable of generating audio based on provided emotion vectors in the discrete form of labels clustered in the valence-arousal latent space. The integration of emotion-driven soundscapes has significant implications in the field of Internet and Communication Technologies for Internet and Multimedia. By enabling the generation of audio that reflects specific emotional states, this work can enhance multimedia applications, offering more immersive and personalized user experiences. Potential applications can significantly impact engagement and effectiveness. The research contributes to the growing body of work on emotion-aware multimedia, presenting a novel approach to audio generation that bridges the gap between computational models and human emotional experience. The findings underscore the transformative potential of combining advanced machine learning techniques with multimedia content creation, paving the way for innovative applications in various domains of internet and communication technologies.