Paper Archive

In this paper we present a new method to compute frame based audio similarities, based on nearest neighbour density estimation. We do not recommend it is as a practical method for large collections because of the high runtime. Rather, we use this new method for a detailed analysis to get a deeper insight on how a bag of frames approach (BOF) determines similarities among songs, and in particular, to identify those audio frames that make two songs similar from a machine’s point of view. Our analysis reveals that audio frames of very low energy, which are of course not the most salient with respect to human perception, have a surprisingly big influence on current similarity measures. Based on this observation we propose to remove these low-energy frames before computing song models and show, via classification experiments, that the proposed frame selection strategy improves the audio similarity measure.

Download

This paper discusses architectural aspects of a software library for unified metadata management in audio processing applications. The data incorporates editorial, production, acoustical and musicological features for a variety of use cases, ranging from adaptive audio effects to alternative metadata based visualisation. Our system is designed to capture information, prescribed by modular ontology schema. This advocates the development of intelligent user interfaces and advanced media workflows in music production environments. In an effort to reach these goals, we argue for the need of modularity and interoperable semantics in representing information. We discuss the advantages of extensible Semantic Web ontologies as opposed to using specialised but disharmonious metadata formats. Concepts and techniques permitting seamless integration with existing audio production software are described in detail.

Download

Soon after the Echo Nest Remix API was made publicly available and open source, the primary author began aggressively enhancing the Python framework for re-editing music based on perceptually-based musical analyses. The basic principles of this API – integrating content-based metadata with the underlying signal – are described in the paper, then the authors’ enhancements are described. The libraries moved from supporting an imperative coding style to incorporating influences from functional programming and domain specific languages to allow for a much more fluent, terse coding style, allowing users to concentrate on the functions needed to find the portions of the song that were interesting, and modifying them. The paper then goes on to describe enhancements involving mixing multiple sources with one another and enabling user-created and user-modifiable effects that are controlled by direct manipulation of the objects that represent the sound. Revelations that the Remix API does not need to be as integrated as it currently is point to future directions for the API at the end of the paper.

Download

We present a real-time implementation of SMS synthesis in Pure Data. This instrument focuses on interaction with the ability to continuously synthesize any frame position within an SMS sound representation, in any order, thereby freeing time from other parameters such as frequency or spectral shape. The instrument can be controlled expressively with a Wacom Tablet that offers both coupled and absolute controls with good precision. A prototype graphical interface in python is presented that helps to interact with the SMS data through visualization. In this system, any sound sample with interesting spectral features turns into a playable instrument. The processing functionality originates in the SMS C code written almost 20 years ago, now re-factored into the open source library, libsms, also wrapped into a python module. A set of externals for Pure Data, called smspd, was made using this library to facilitate on-the-fly analysis, flexible modifications, and interactive synthesis. We discuss new transformations are introduced based on the possibilities of this system and ideas for higher-level, feature based transformations that benefit from the interactivity of this system.

Download