Download Multimodal Interfaces for Expressive Sound Control This paper introduces research issues on multimodal interaction and interfaces for expressive sound control. We introduce Multisensory Integrated Expressive Environments (MIEEs) as a framework for Mixed Reality applications in the performing arts. Paradigmatic contexts for applications of MIEEs are multimedia concerts, interactive dance / music / video installations, interactive museum exhibitions, distributed cooperative environments for theatre and artistic expression. MIEEs are user-centred systems able to interpret the high-level information conveyed by performers through their expressive gestures and to establish an effective multisensory experience taking into account expressive, emotional, affective content. The lecture discusses some main issues for MIEEs and presents the EyesWeb (www.eyesweb.org) open software platform which has been recently redesigned (version 4) in order to better address MIEE requirements. Short live demonstrations are also presented.
Download Neural Grey-Box Guitar Amplifier Modelling with Limited Data This paper combines recurrent neural networks (RNNs) with the discretised Kirchhoff nodal analysis (DK-method) to create a grey-box guitar amplifier model. Both the objective and subjective results suggest that the proposed model is able to outperform a baseline black-box RNN model in the task of modelling a guitar amplifier, including realistically recreating the behaviour of the amplifier equaliser circuit, whilst requiring significantly less training data. Furthermore, we adapt the linear part of the DK-method in a deep learning scenario to derive multiple state-space filters simultaneously. We frequency sample the filter transfer functions in parallel and perform frequency domain filtering to considerably reduce the required training times compared to recursive state-space filtering. This study shows that it is a powerful idea to separately model the linear and nonlinear parts of a guitar amplifier using supervised learning.
Download Simulating guitar distortion circuits using wave digital and nonlinear state-space formulations This work extends previous research on numerical solution of nonlinear systems in musical acoustics to the realm of nonlinear musical circuits. Wave digital principles and nonlinear state-space simulators provide two alternative approaches explored in this work. These methods are used to simulate voltage amplification stages typically used in guitar distortion or amplifier circuits. Block level analysis of the entire circuit suggests a strategy based upon the nonlinear filter composition technique for connecting amplifier stages while accounting for the way these stages interact. Formulations are given for the bright switch, the diode clipper, a transistor amplifier, and a triode amplifier.
Download Eigensystem Realization of Violin Bridge Admittances Modeling violin bridge admittance is a long-standing problem in musical acoustics, with applications in sound analysis, synthesis, and virtual instrument design. In this work, we investigate the use of the Eigensystem Realization Algorithm (ERA) for deriving reduced-order state-space models directly from measured impulse responses. The proposed approach allows us to extract dominant system dynamics and obtain compact realizations without requiring explicit modal parameterization. We evaluate ERA on a dataset of modern and historical violins and compare it against established modal and state-space identification methods. Experimental results demonstrate that ERA outperforms existing approaches by achieving lower reconstruction errors in both the time and frequency domains while preserving perceptually relevant characteristics of the bridge response. Furthermore, we show that the state-space realizations obtained using ERA reproduce the target frequency-dependent energy decay more accurately than models obtained using the baseline methods. These findings support the use of ERA as an efficient and flexible alternative for modeling violin bridge admittances, with applications that span from audio synthesis and processing to instrument virtualization.
Download Digitally Moving An Electric Guitar Pickup This paper describes a technique to transform the sound of an arbitrarily selected magnetic pickup into another pickup selection on the same electric guitar. This is a first step towards replicating an arbitrary electric guitar timbre in an audio recording using the signal from another guitar as input. We record 1458 individual notes from the pickups of a single guitar, varying the string, fret, plucking position, and dynamics of the tones in order to create a controlled dataset for training and testing our approach. Given an input signal and a target signal, a least squares estimator is used to obtain the coefficients of a finite impulse response (FIR) filter to match the desired magnetic pickup position. We use spectral difference to measure the error of the emulation, and test the effects of independent variables fret, dynamics, plucking position and repetition on the accuracy. A small reduction in accuracy was observed for different repetitions; moderate errors arose when the playing style (plucking position and dynamics) were varied; and there were large differences between output and target when the training and test data comprised different notes (fret positions). We explain results in terms of the acoustics of the vibrating strings.
Download Fractionally-addressed Delay Lines While traditional implementations of digital delay lines are based on a circular buffer accessed by two pointers, we propose an implementation where a single fractional pointer is used both for reading and writing operations. On modern general-purpose architectures, the proposed method is nearly as efficient as the popular interpolated circular buffer, but it offers better performance in terms of frequency-dependent attenuation and response to delay-length modulations.
Download Energy-stable modelling of contacting modal objects with piece-wise linear interaction force In discrete-time digital models of contact of vibrating objects stability and therefore control over system energy is an important issue. While numerical approximation is problematic in this context digital algorithms may meat this challenge when based on exact mathematical solution of the underlying equation. The latter may generally be possible under certain conditions of linearity. While a system of contacting solid objects is non-linear by definition, piece-wise linear models may be used. Here however the aspect of “switching” between different linear phases is crucial. An approach is presented for exact preservation of system energy when passing between different phases of contact. One basic principle used may be pictured as inserting appropriate ideal, massless and perfectly stiff, “connection rods” at discrete moments of phase switching. Theoretic foundations are introduced and the general technique is explained and tested at two simple examples.
Download Model-Based Obstacle Sonification for the Navigation of Visually Impaired Persons This paper proposes a sonification model for encoding visual 3D information into sounds, inspired by the impact properties of the objects encountered during blind navigation. The proposed model is compared against two sonification models developed for orientation and mobility, chosen based on their common technical requirements. An extensive validation of the proposed model is reported; five legally blind and five normally sighted participants evaluated the proposed model as compared to the two competitive models on a simplified experimental navigation scenario. The evaluation addressed not only the accuracy of the responses in terms of psychophysical measurements but also the cognitive load and emotional stress of the participants by means of biophysiological signals and evaluation questionnaires. Results show that the proposed impact sound model adequately conveys the relevant information to the participants with low cognitive load, following a short training session.
Download Amp-Space: A Large-Scale Dataset for Fine-Grained Timbre Transformation We release Amp-Space, a large-scale dataset of paired audio
samples: a source audio signal, and an output signal, the result of
a timbre transformation. The types of transformations we study
are from blackbox musical tools (amplifiers, stompboxes, studio
effects) traditionally used to shape the sound of guitar, bass, or
synthesizer sounds. For each sample of transformed audio, the
set of parameters used to create it are given. Samples are from
both real and simulated devices, the latter allowing for orders of
magnitude greater data than found in comparable datasets. We
demonstrate potential use cases of this data by (a) pre-training a
conditional WaveNet model on synthetic data and show that it reduces the number of samples necessary to digitally reproduce a
real musical device, and (b) training a variational autoencoder to
shape a continuous space of timbre transformations for creating
new sounds through interpolation.
Download Simulation of Textured Audio Harmonics Using Random Fractal Phaselets We present a method of simulating audio signals using the principles of random fractal geometry which, in the context of this paper, is concerned with the analysis of statistically self-affine ‘phaselets’. The approach is used to generate audio signals that are characterised by texture and timbre through the Fractal Dimension such as those associated with bowed stringed instruments. The paper provides a short overview on potential simulation methods using Artificial Neural Networks and Evolutionary Computing and on the problems associated with using a deterministic approach based on solutions to the acoustic wave equation. This serves to quantify the origins of the ‘noise’ associated with multiple scattering events that characterise texture and timbre in an audio signal. We then explore a method to compute the phaselet of a phase signal which is the primary phase function from which a phase signal is, to a good approximation, a periodic replica and show that, by modelling the phaselet as a random fractal signal, it can be characterised by the Fractal Dimension. The Fractal Dimension is then used to synthesise a phaselet from which the phase function is computed through multiple concatenations of the phaselet. The paper provides details of the principal steps associated with the method considered and examines some example results, providing a URL to m-coded functions for interested readers to repeat the results obtained and develop the algorithms further.