Download Effect of Latency on Playing Accuracy of Two Gesture Controlled Continuous Sound Instruments Without Tactile Feedback
The paper reports results from an experimental study quantifying how latency affects the playing accuracy of two continuous sound instruments. 11 subjects played a conventional Theremin and a virtual reality Theremin. Both instruments provided the user only audio feedback. The subjects performed two tasks under different instrument latencies. They attempted to match the pitch of the instrument to a sample pitch and they played along a short sample melody and a metronome. Both the sample sound and the instrument’s sound were recorded on different channels of a sound file. Later the pitch of the sounds was extracted and user performance analyzed. The results show that the time required to match a given pitch degrades about five times the introduced latency suggesting that the feedback latency cumulates over the whole task. Errors while playing along a sample melody increased 80% by average on the highest latency of 240ms. Latencies until 120ms increased the errors only slightly.
Download Adaptive Effects Based on STFT, Using a Source-Filter Model
This paper takes the opportunity of presenting a set of new adaptive effects to propose a generic scheme for adaptive effects built upon a spectral source-filter decomposition and a Short-Time Fourier analysis-resynthesis. This allows for a better formalization of the involved signal processing algorithms and leads to a simple classification of adaptive effects already presented in the literature, that falls into this category. We discuss the motivation and the advantages of combining source-filter modeling and phase vocoder representation for the design of adaptive digital audio effects. Then we detail the general structure that includes STFT analysis and re-synthesis scheme, the source filter decomposition, and an adaptive control unit composed of a feature extraction system and a sound mapping unit that might be driven by a gestural control section.
Download Differentiable All-Pass Filters for Phase Response Estimation and Automatic Signal Alignment
Virtual analog (VA) audio effects are increasingly based on neural networks and deep learning frameworks. Due to the underlying black-box methodology, a successful model will learn to approximate the data it is presented, including potential errors such as latency and audio dropouts as well as non-linear characteristics and frequency-dependent phase shifts produced by the hardware. The latter is of particular interest as the learned phase-response might cause unwanted audible artifacts when the effect is used for creative processing techniques such as dry-wet mixing or parallel compression. To overcome these artifacts we propose differentiable signal processing tools and deep optimization structures for automatically tuning all-pass filters to predict the phase response of different VA simulations, and align processed signals that are out of phase. The approaches are assessed using objective metrics while listening tests evaluate their ability to enhance the quality of parallel path processing techniques. Ultimately, an overparameterized, BiasNet-based, all-pass model is proposed for the optimization problem under consideration, resulting in models that can estimate all-pass filter coefficients to align a dry signal with its affected, wet, equivalent.
Download Automatic Violin Synthesis Using Expressive Musical Term Features
The control of interpretational properties such as duration, vibrato, and dynamics is important in music performance. Musicians continuously manipulate such properties to achieve different expressive intentions. This paper presents a synthesis system that automatically converts a mechanical, deadpan interpretation to distinct expressions by controlling these expressive factors. Extending from a prior work on expressive musical term analysis, we derive a subset of essential features as the control parameters, such as the relative time position of the energy peak in a note and the mean temporal length of the notes. An algorithm is proposed to manipulate the energy contour (i.e. for dynamics) of a note. The intended expressions of the synthesized sounds are evaluated in terms of the ability of the machine model developed in the prior work. Ten musical expressions such as Risoluto and Maestoso are considered, and the evaluation is done using held-out music pieces. Our evaluations show that it is easier for the machine to recognize the expressions of the synthetic version, comparing to those of the real recordings of an amateur student. While a listening test is under construction as a next step for further performance validation, this work represents to our best knowledge a first attempt to build and quantitatively evaluate a system for EMT analysis/synthesis.
Download Faust2android: a Faust Architecture For Android
faust2android is a tool that turns a FAUST program into an Android application. Signal processing tasks as well as accessing the audio record and playback resources are done natively in C++ using the Android Native Development Toolkit (NDK). User interface and other components of the application are programmed in JAVA. The implementation as well as issues related to real-time signal processing on Android platforms are discussed. faust2android is part of a larger project whose goal is to build a full FAUST environment for Android: FAUST D ROID.
Download Artificial Intelligence based Modeling of Musical Instruments
In this paper, a novel research tool, which allows real-time implementation and evaluation of sound synthesis of musical instrument, is described. The tool is a PC-based application and allows the user to evaluate the effects of parameter changes on the sound quality in an intuitive manner. Tuning makes use of a Genetic Algorithm (GA) technique. Flute and plucked string modeling examples are used to illustrate the capabilities of the tool.
Download Sound synthesis using an allpass filter chain with audio‐rate coefficient modulation
This paper describes a sound synthesis technique that modulates the coefficients of allpass filter chains using audio-rate frequencies. It was found that modulating a single allpass filter section produces a feedback AM–like spectrum, and that its bandwidth is extended and further processed by non-sinusoidal FM when the sections are cascaded. The cascade length parameter provides dynamic bandwidth control to prevent upper range aliasing artifacts, and the amount of spectral content within that band can be controlled using a modulation index parameter. The technique is capable of synthesizing rich and evolving timbres, including those resembling classic virtual analog waveforms. It can also be used as an audio effect with pitch-tracked input sources. Software and sound examples are available at http://www.acoustics.hut.fi/publications/papers/dafx09-cm/
Download Physical Modeling for Spatial Sound Synthesis
This contribution combines techniques for sound synthesis and spatial reproduction for a joint synthesis of the sound production and sound propagation properties of virtual string instruments. The generated sound field is reproduced on a massive multichannel loudspeaker system using wave field synthesis techniques. From physical descriptions of string vibrations and sound waves by partial differential equations follows an algorithmic procedure for synthesis-driven wave field reproduction. Its processing steps are derived by mathematical analysis and signal processing principles. Three different building blocks are addressed: The simulation of string vibrations, a model for the radiation pattern of the generated acoustical waves, and the determination of the driving signals for the multichannel loudspeaker array. The proposed method allows the spatial reproduction of synthetic spatial sound without the need for pre-recorded or pre-synthesized source tracks.
Download Real-time Finite Difference Physical Models of Musical Instruments on a Field Programmable Gate Array (FPGA)
Real-time sound synthesis of musical instruments based on solving differential equations is of great interest in Musical Acoustics especially in terms of linking geometry features of musical instruments to sound features. A major restriction of accurate physical models is the computational effort. One could state that the calculation cost is directly linked to the geometrical and material accuracy of a physical model and so to the validity of the results. This work presents a methodology for implementing realtime models of whole instrument geometries modelled with the Finite Differences Method (FDM) on a Field Programmable Gate Array (FPGA), a device capable of massively parallel computations. Examples of three real-time musical instrument implementations are given, a Banjo, a Violin and a Chinese Ruan.
Download A Minimal Passive Model of the Operational Amplifier: Application to Sallen-Key Analog Filters
This papers stems from the fact that, whereas there are passive models of transistors and tubes, a minimal passive model of the operational amplifier does not seem to exist. A new behavioural model is presented that is memoryless, fully described by its interaction ports, with a minimal number of equations, for which a passive power balance can be defined. The proposed model handles saturation, asymmetric power supply, and can be used with nonideal voltage references. To illustrate the model in audio applications, the non-inverting voltage amplifier and a saturating Sallen-Key lowpass filter are considered.