Most of the current tools for working with sound work on single soundfiles, use 2D graphics and offer limited interaction to the user. In this paper we describe a set of tools for working with collections of sounds that are based on interactive 3D graphics. These tools form two families: sound analysis visualization displays and model-based controllers for sound synthesis algorithms. We describe the general techniques we have used to develop these tools and give specific case studies from each family. Several collections of sounds were used for development and evaluation. These are: a set of musical instrument tones, a set of sound effects, a set of FM radio audio clips belonging to several music genres, and a set of mp3 rock song snippets.

Download

Musical signals exhibit periodic temporal structure that create the sensation of rhythm. In order to model, analyze, and retrieve musical signals it is important to automatically extract rhythmic information. To somewhat simplify the problem, automatic algorithms typically only extract information about the main beat of the signal which can be loosely defined as the regular periodic sequence of pulses corresponding to where a human would tap his foot while listening to the music. In these algorithms, the beat is characterized by its frequency (tempo), phase (accent locations) and a confidence measure about its detection. The main focus of this paper is the concept of Beat Strength, which will be loosely defined as one rhythmic characteristic that could allow to discriminate between two pieces of music having the same tempo. Using this definition, we might say that a piece of Hard Rock has a higher beat strength than a piece of Classical Music at the same tempo. Characteristics related to Beat Strength have been implicitely used in automatic beat detection algorithms and shown to be as important as tempo information for music classification and retrieval. In the work presented in this paper, a user study exploring the perception of Beat Strength was conducted and the results were used to calibrate and explore automatic Beat Strength measures based on the calculation of Beat Histograms.

Download

The process of creating an audio mosaic consists of the concatenation of segments of sound. Segments are chosen to correspond best with a description of a target sound specified by the desired features of the final mosaic. Current audio mosaicing techniques take advantage of the description of future target units in order to make more intelligent decisions when choosing individual segments. In this paper, we investigate ways to expand mosaicing techniques in order to use the mosaicing process as an interactive means of musical expression in real time. In our system, the user can interactively choose the specification of the target as well as the source signals from which the mosaic is composed. These means of control are incorporated into MoSievius, a framework intended for the rapid implementation of different interactive mosaicing techniques. Its integral means of control, the Sound Sieve, provides real-time control over the source selection process when creating an audio mosaic. We discuss a number of new real-time effects that can be achieved through use of the Sound Sieve.

Download

We present two simple perceptually motivated audio effects designed to increase the perceived sensory dissonance/roughness (a process we call “dissonancization”) of audio input. The first involves heterodyning multiple bands of the audio signal at different frequencies to break each sinusoid in each band into two sinusoids separated in frequency by the amount that Kameoka and Kuriyagawa [1] predict will produce a maximally dissonant effect. The second attempts to increase the depth of modulation caused by existing beating partials by exponentiating the amplitude envelope within small bands, enhancing the perceived roughness already present in the signal. The first algorithm can produce very dramatic effects even for very consonant inputs, whereas the second tends to produce a more subtle effect. Both algorithms are quite simple to understand and implement and computationally inexpensive enough to be used in real time, but produce perceptually interesting results. The effects can be selectively applied so as to affect only desired frequency ranges, and can be continuously controlled (e.g. in a performance context) to have more or less impact.

Download