Download Some Physical Audio Effects This paper presents a survey of various audio effects that can be physically applied to a rigidly-terminated vibrating string. The string’s resonant behavior is described, and then the ability of active feedback control to “reprogram" the physics of the string is explained. Active damping, which is a direct result of applying classical control techniques, provides for an effect based on amplitude modulation (AM). Traditional electric guitar sustain techniques are elaborated upon, which suggest another approach for ensuring marginal stability of the system even in the presence of an arbitrary nonlinear and/or time-varying effect unit in the feedback loop. This approach involves placing a dynamic range limiter in the feedback loop and does not introduce significant harmonic distortion other than that due to the effect unit. The maximum RMS level of the system’s output can be easily bounded if reasonable conditions are met by the dynamic range limiter. Finally, nonlinear and time-varying feedback control loops are applied experimentally to artificially induce frequency modulation (FM) at a low rate and AM at a high rate. These effects can be interpreted musically as vibrato and as a sort of resonant ring modulation, respectively.
Download On the Challenges of Embedded Real-Time Music Information Retrieval Real-time applications of Music Information Retrieval (MIR) have been gaining interest as of recently. However, as deep learning becomes more and more ubiquitous for music analysis tasks, several challenges and limitations need to be overcome to deliver accurate and quick real-time MIR systems. In addition, modern embedded computers offer great potential for compact systems that use MIR algorithms, such as digital musical instruments. However, embedded computing hardware is generally resource constrained, posing additional limitations. In this paper, we identify and discuss the challenges and limitations of embedded real-time MIR. Furthermore, we discuss potential solutions to these challenges, and demonstrate their validity by presenting an embedded real-time classifier of expressive acoustic guitar techniques. The classifier achieved 99.2% accuracy in distinguishing pitched and percussive techniques and a 99.1% average accuracy in distinguishing four distinct percussive techniques with a fifth class for pitched sounds. The full classification task is a considerably more complex learning problem, with our preliminary results reaching only 56.5% accuracy. The results were produced with an average latency of 30.7 ms.
Download Discretization of Parametric Analog Circuits for Real-Time Simulations The real-time simulation of analog circuits by digital systems becomes problematic when parametric components like potentiometers are involved. In this case the coefficients defining the digital system will change and have to be adapted. One common solution is to recalculate the coefficients in real-time, a possibly computationally expensive operation. With a view to the simulation using state-space representations, two parametric subcircuits found in typical guitar amplifiers are analyzed, namely the tone stack, a linear passive network used as simple equalizer and a distorting preamplifier, limiting the signal amplitude with LEDs. Solutions using trapezoidal rule discretization are presented and discussed. It is shown, that the computational costs in case of recalculation of the coefficients are reduced compared to the related DK-method, due to minimized matrix formulations. The simulation results are compared to reference data and show good match.
Download Neural Modelling of Time-Varying Effects This paper proposes a grey-box neural network based approach
to modelling LFO modulated time-varying effects.
The neural
network model receives both the unprocessed audio, as well as
the LFO signal, as input. This allows complete control over the
model’s LFO frequency and shape. The neural networks are trained
using guitar audio, which has to be processed by the target effect
and also annotated with the predicted LFO signal before training.
A measurement signal based on regularly spaced chirps was used
to accurately predict the LFO signal. The model architecture has
been previously shown to be capable of running in real-time on a
modern desktop computer, whilst using relatively little processing
power. We validate our approach creating models of both a phaser
and a flanger effects pedal, and theoretically it can be applied to
any LFO modulated time-varying effect. In the best case, an errorto-signal ratio of 1.3% is achieved when modelling a flanger pedal,
and previous work has shown that this corresponds to the model
being nearly indistinguishable from the target device.
Download New techniques and Effects in Model-based Sound Synthesis Physical modeling and model-based sound synthesis have recently been among the most active topics of computer music and audio research. In the modeling approach one typically tries to simulate and duplicate the most prominent sound generation properties of the acoustic musical instrument under study. If desired, the models developed may then be modified in order to create sounds that are not common or even possible from physically realizable instruments. In addition to physically related principles it is possible to combine physical models with other synthesis and signal processing methods to realize hybrid modeling techniques.
This article is written as an overview of some recent results in model-based sound synthesis and related signal processing techniques. The focus is on modeling and synthesizing plucked string sounds, although the techniques may find much more widespread application. First, as a background, an advanced linear model of the acoustic guitar is discussed along with model control principles. Then the methodology to include inherent nonlinearities and time-varying features is introduced. Examples of string instrument nonlinearities are studied in the context of two specific instruments, the kantele and the tanbur, which exhibit interesting nonlinear effects.
Download Keytar: Melodic control of multisensory feedback from virtual strings A multisensory virtual environment has been designed, aiming at recreating a realistic interaction with a set of vibrating strings. Haptic, auditory and visual cues progressively istantiate the environment: force and tactile feedback are provided by a robotic arm reporting for string reaction, string surface properties, and furthermore defining the physical touchpoint in form of a virtual plectrum embodied by the arm stylus. Auditory feedback is instantaneously synthesized as a result of the contacts of this plectrum against the strings, reproducing guitar sounds. A simple visual scenario contextualizes the plectrum in action along with the vibrating strings. Notes and chords are selected using a keyboard controller, in ways that one hand is engaged in the creation of a melody while the other hand plucks virtual strings. Such components have been integrated within the Unity3D simulation environment for game development, and run altogether on a PC. As also declared by a group of users testing a monophonic Keytar prototype with no keyboard control, the most significant contribution to the realism of the strings is given by the haptic feedback, in particular by the textural nuances that the robotic arm synthesizes while reproducing physical attributes of a metal surface. Their opinion, hence, argues in favor of the importance of factors others than auditory feedback for the design of new musical interfaces.
Download Instrument reusability scheme in PWGLSynth This paper presents our recent developments that aim to make the instrument definition process of our visual synthesis environment more accessible to a broader audience. There are several novel aspects that aim to overcome some of the classical limitations found in sound synthesis systems. After an introductory section we discuss two advanced examples where scores containing several model-based instrument parts can be realized without having to edit the original instrument definitions. In the first one, we can duplicate instrumental parts of a single instrument definition. In the latter example, we can mix in a single score several instrument models with the help of a mixer patch that is automatically created by system.
Download Low-Latency Bass Separation Using Harmonic-Percussion Decomposition Many recent approaches to musical source separation rely on modelbased inference methods that take into account the signal’s harmonic structure. To address the particular case of low-latency bass separation, we propose a method that combines harmonic decomposition using a Tikhonov regularization-based algorithm, with the peak contrast analysis of the pitch likelihood function. Our experiment compares the separation performance of this method to a naive low-pass filter, a state-of-the-art NMF-based method and a near-optimal binary mask. The proposed low-latency method achieves results similar to the NMF-based high-latency approach at a lower computational cost. Therefore the method is valid for real-time implementations.
Download A Hierarchical Deep Learning Approach for Minority Instrument Detection Identifying instrument activities within audio excerpts is vital in music information retrieval, with significant implications for music cataloging and discovery. Prior deep learning endeavors in musical instrument recognition have predominantly emphasized instrument classes with ample data availability. Recent studies have demonstrated the applicability of hierarchical classification in detecting instrument activities in orchestral music, even with limited fine-grained annotations at the instrument level. Based on the Hornbostel-Sachs classification, such a hierarchical classification system is evaluated using the MedleyDB dataset, renowned for its diversity and richness concerning various instruments and music genres. This work presents various strategies to integrate hierarchical structures into models and tests a new class of models for hierarchical music prediction. This study showcases more reliable coarse-level instrument detection by bridging the gap between detailed instrument identification and group-level recognition, paving the way for further advancements in this domain.
Download Unsupervised Estimation of Nonlinear Audio Effects: Comparing Diffusion-Based and Adversarial Approaches Accurately estimating nonlinear audio effects without access to
paired input-output signals remains a challenging problem. This
work studies unsupervised probabilistic approaches for solving this
task. We introduce a method, novel for this application, based
on diffusion generative models for blind system identification, enabling the estimation of unknown nonlinear effects using blackand gray-box models. This study compares this method with a
previously proposed adversarial approach, analyzing the performance of both methods under different parameterizations of the
effect operator and varying lengths of available effected recordings. Through experiments on guitar distortion effects, we show
that the diffusion-based approach provides more stable results and
is less sensitive to data availability, while the adversarial approach
is superior at estimating more pronounced distortion effects. Our
findings contribute to the robust unsupervised blind estimation of
audio effects, demonstrating the potential of diffusion models for
system identification in music technology.