Download Musical Instrument Identification in Continuous Recordings
Recognition of musical instruments in multi-instrumental, polyphonic music is a difficult challenge which is yet far from being solved. Successful instrument recognition techniques in solos (monophonic or polyphonic recordings of single instruments) can help to deal with this task. We introduce an instrument recognition process in solo recordings of a set of instruments (bassoon, clarinet, flute, guitar, piano, cello and violin), which yields a high recognition rate. A large and very diverse solo database (108 different solos, all by different performers) is used in order to encompass the different sound possibilities of each instrument and evaluate the generalization abilities of the classification process. First we bring classification results using a very extensive collection of features (62 different feature types), and then use our GDE feature selection algorithm to select a smaller feature set with a relatively short computation time, which allows us to perform instrument recognition in solos in real-time, with only a slight decrease in recognition rate. We demonstrate that our real-time solo classifier can also be useful for instrument recognition in duet performances, and improved using simple “source reduction”.
Download Audio-Based Gesture Extraction on the ESITAR Controller
Using sensors to extract gestural information for control parameters of digital audio effects is common practice. There has also been research using machine learning techniques to classify specific gestures based on audio feature analysis. In this paper, we will describe our experiments in training a computer to map the appropriate audio-based features to look like sensor data, in order to potentially eliminate the need for sensors. Specifically, we will show our experiments using the ESitar, a digitally enhanced sensor based controller modeled after the traditional North Indian sitar. We utilize multivariate linear regression to map continuous audio features to continuous gestural data.
Download On Finding Melodic Lines in Audio Recordings
The paper presents our approach to the problem of finding melodic line(s) in polyphonic audio recordings. The approach is composed of two different stages, partially rooted in psychoacoustic theories of music perception: the first stage is dedicated to finding regions with strong and stable pitch (melodic fragments), while in the second stage, these fragments are grouped according to their properties (pitch, loudness...) into clusters which represent melodic lines of the piece. Expectation Maximization algorithm is used in both stages to find the dominant pitch in a region, and to train Gaussian Mixture Models that group fragments into melodies. The paper presents the entire process in more detail and provides some initial results.
Download Audio Processing Using Haskell
The software for most today’s applications including signal processing applications is written in imperative languages. Imperative programs are fast because they are designed close to the architecture of the widespread computers, but they don’t match the structure of signal processing very well. In contrast to that, functional programming and especially lazy evaluation perfectly models many common operations on signals. Haskell is a statically typed, lazy functional programming language which allow for a very elegant and concise programming style. We want to sketch how to process signals, how to improve safety by the use of physical units, and how to compose music using this language.
Download Event-Synchronous Music Synthesis
This work presents a novel framework for music synthesis, based on the perceptual structure analysis of pre-existing musical signals, for example taken from a personal MP3 database. We raise the important issue of grounding music analysis on perception, and propose a bottom-up approach to music analysis, as well as modeling, and synthesis. A model of segmentation for polyphonic signals is described, and is qualitatively validated through several artifact-free music resynthesis experiments, e.g., reversing the ordering of sound events (notes), without reversing their waveforms. Then, a compact “timbre” structure analysis, and a method for song description in the form of an “audio DNA” sequence is presented. Finally, we propose novel applications, such as music cross-synthesis, or time-domain audio compression, enabled through simple sound similarity measures, and clustering.
Download Effect of Latency on Playing Accuracy of Two Gesture Controlled Continuous Sound Instruments Without Tactile Feedback
The paper reports results from an experimental study quantifying how latency affects the playing accuracy of two continuous sound instruments. 11 subjects played a conventional Theremin and a virtual reality Theremin. Both instruments provided the user only audio feedback. The subjects performed two tasks under different instrument latencies. They attempted to match the pitch of the instrument to a sample pitch and they played along a short sample melody and a metronome. Both the sample sound and the instrument’s sound were recorded on different channels of a sound file. Later the pitch of the sounds was extracted and user performance analyzed. The results show that the time required to match a given pitch degrades about five times the introduced latency suggesting that the feedback latency cumulates over the whole task. Errors while playing along a sample melody increased 80% by average on the highest latency of 240ms. Latencies until 120ms increased the errors only slightly.
Download Adaptive Effects Based on STFT, Using a Source-Filter Model
This paper takes the opportunity of presenting a set of new adaptive effects to propose a generic scheme for adaptive effects built upon a spectral source-filter decomposition and a Short-Time Fourier analysis-resynthesis. This allows for a better formalization of the involved signal processing algorithms and leads to a simple classification of adaptive effects already presented in the literature, that falls into this category. We discuss the motivation and the advantages of combining source-filter modeling and phase vocoder representation for the design of adaptive digital audio effects. Then we detail the general structure that includes STFT analysis and re-synthesis scheme, the source filter decomposition, and an adaptive control unit composed of a feature extraction system and a sound mapping unit that might be driven by a gestural control section.
Download Exponential Weighting Method for Sample-by-Sample Update of Warped AR-Model
Auto-regressive (AR) modeling is a powerful tool having many ap­ plications in audio signal processing. The modeling procedure can be focused to low or high frequency range using frequency warp­ ing. Conventionally the AR-modeling procedure is accomplished with frame-by-frame processing which introduces latency. As with any frame-by-frame algorithm full frame has to be available for the algorithm before any output can be produced. This latency makes AR-modeling more or less unusable in real-time sound effects es­ pecially when long frame lengths are required. In this paper we introduce an exponential weighting (EW) method for sample-bysample update of the warped AR-model. This method reduces the latency down to the order of the AR-model.
Download Sound Source Separation: Azimuth Discrimination and Resynthesis
In this paper we present a novel sound source separation algorithm which requires no prior knowledge, no learning, assisted or otherwise, and performs the task of separation based purely on azimuth discrimination within the stereo field. The algorithm exploits the use of the pan pot as a means to achieve image localisation within stereophonic recordings. As such, only an interaural intensity difference exists between left and right channels for a single source. We use gain scaling and phase cancellation techniques to expose frequency dependent nulls across the azimuth domain, from which source separation and resynthesis is carried out. We present results obtained from real recordings, and show that for musical recordings, the algorithm improves upon the output quality of current source separation schemes.
Download Acoustical Simulations of the Human Vocal Tract Using the 1D and 2D Digital Waveguide Software Model
This paper details software under development that uses the digital waveguide physical model to represent the sound creation mechanism and environment associated with the production of speech, specifically the human vocal tract. Focus is directed towards a comparison between the existing 1D waveguide method, on which several studies have already been conducted, and the developing 2D waveguide mesh method. The construction of the two models and the application of the tract geometry is examined, in addition, the inclusion of dynamic articulatory variations to increase the ability of such systems to create natural sounding speech is discussed. Results obtained from each suggest that the 2D model is capable of producing similarly accurate vowel spectra to that already accomplished with the 1D version, although speech-like sounds created with the 2D mesh appear to exhibit greater realism.