Download Acoustical Simulations of the Human Vocal Tract Using the 1D and 2D Digital Waveguide Software Model This paper details software under development that uses the digital waveguide physical model to represent the sound creation mechanism and environment associated with the production of speech, specifically the human vocal tract. Focus is directed towards a comparison between the existing 1D waveguide method, on which several studies have already been conducted, and the developing 2D waveguide mesh method. The construction of the two models and the application of the tract geometry is examined, in addition, the inclusion of dynamic articulatory variations to increase the ability of such systems to create natural sounding speech is discussed. Results obtained from each suggest that the 2D model is capable of producing similarly accurate vowel spectra to that already accomplished with the 1D version, although speech-like sounds created with the 2D mesh appear to exhibit greater realism.
Download Diagonal Complex-Valued State Space Models for System Identification and Modeling of Metal Plate Reverbs Accurate and interpretable modeling of plate reverbs remains an important challenge in virtual analog modeling of audio effects. While existing neural network-based black-box approaches already achieve high-quality synthesis and strong perceptual quality, they often lack the possibility to identify the underlying physically meaningful complex, long-memory modal behavior. In this work, we address this limitation by proposing a restricted complex-valued diagonal State Space Model (SSM), showing its equivalence to a parallel second-order all-pole filter, also utilizing efficient training via parallel state computation using the parallel scan algorithm. Additionally, we propose a Matrix Pencil (MP) guided eigenvalue initialization, improving synthesis quality and system identification performance.
Download On the Nonlinear Commuted Synthesis of the Piano In this paper a novel method is presented for the physics-based sound synthesis of the piano, based on digital waveguides. The approach combines the advantages of the commuted synthesis technique and the methods using a nonlinear hammer model. The interaction force of the hammer-string contact is computed by an auxiliary digital waveguide connected to a nonlinear hammer model. This force signal is used as a target impulse response for designing a low-order digital filter real-time. The piano sound is calculated by filtering the soundboard response with the hammer filter and feeding this signal to a synthesizer digital waveguide. A new method is presented for separating the contribution of the interaction force and the soundboard in measured piano tones. For modeling beating, a new technique is proposed based on a simplified pitch-shift effect. Considerations on modeling the effect of sustain pedal are also given. It is shown that the technique of designing the hammer filter real-time is not only useful for digital waveguide modeling, but it can be combined with sampling synthesis too.
Download Differentiable Piano Model for Midi-to-Audio Performance Synthesis Recent neural-based synthesis models have achieved impressive results for musical instrument sound generation. In particular, the Differentiable Digital Signal Processing (DDSP) framework enables the usage of spectral modeling analysis and synthesis techniques in fully differentiable architectures. Yet currently, it has only been used for modeling monophonic instruments. Leveraging the interpretability and modularity of this framework, the present work introduces a polyphonic differentiable model for piano sound synthesis, conditioned on Musical Instrument Digital Interface (MIDI) inputs. The model architecture is motivated by high-level acoustic modeling knowledge of the instrument which, in tandem with the sound structure priors inherent to the DDSP components, makes for a lightweight, interpretable and realistic sounding piano model. The proposed model has been evaluated in a listening test, demonstrating improved sound quality compared to a benchmark neural-based piano model, with significantly less parameters and even with reduced training data. The same listening test indicates that physical-modeling-based models still achieve better quality, but the differentiability of our lightened approach encourages its usage in other musical tasks dealing with polyphonic audio and symbolic data.
Download Simplified Guitar Bridge Model for the Displacement Wave Representation in Digital Waveguides In this paper, we present a simplified model for the string-bridge interaction in guitars or other string instruments simulated by digital waveguides. The bridge model is devised for the displacement wave representation in order to be integrated with other models for string interactions with the player and with other parts of the instrument, whose simulation and implementation is easier in this representation. The model is based on a multiplierless scattering matrix representing the string-bridge interaction. Although not completely physically inspired, we show that this junction is sufficiently general to accommodate a variety of transfer functions under the sole requirement of passivity and avoids integration constants mismatch when the bridge is in turn modeled by a digital waveguide. The model is completed with simple methods to introduce horizontal and vertical polarizations of the string displacement and sympathetic vibrations of other strings. The aim of this paper is not to provide the most general methods for sound synthesis of guitar but, rather, to point at low computational cost and scalable solutions suitable for real-time implementations where the synthesizer is running together with several other audio applications.
Download Towards Efficient Modelling of String Dynamics: A Comparison of State Space and Koopman Based Deep Learning Methods This paper presents an examination of State Space Models (SSM) and Koopman-based deep learning methods for modelling the dynamics of both linear and non-linear stiff strings. Through experiments with datasets generated under different initial conditions and sample rates, we assess the capacity of these models to accurately model the complex behaviours observed in string dynamics. Our findings indicate that our proposed Koopman-based model performs as well as or better than other existing approaches in nonlinear cases for long-sequence modelling. We inform the design of these architectures with the structure of the problems at hand. Although challenges remain in extending model predictions beyond the training horizon (i.e., extrapolation), the focus of our investigation lies in the models’ ability to generalise across different initial conditions within the training time interval. This research contributes insights into the physical modelling of dynamical systems (in particular those addressing musical acoustics) by offering a comparative overview of these and previous methods and introducing innovative strategies for model improvement. Our results highlight the efficacy of these models in simulating non-linear dynamics and emphasise their wide-ranging applicability in accurately modelling dynamical systems over extended sequences.
Download Control Parameters for Reed Wind Instruments or Organ Pipes with Reed Induced Flow Sound synthesis of a pipe coupled with a reed requires to finely tune the physical parameters of the underlying model. Although the pipe geometry is often well known, the 1 degree of freedom reed model’s parameters are effective coefficients (mass, section, etc) and are difficult to assess. Studies of this coupled system have essentially focused on models without the reed induced flow, and have exhibited two dimensionless parameters γ and ζ, which respectively describe the ratio between feeding pressure and closing reed pressure, and a dimensionless opening of the reed at rest. Including the reed flow in the model, then performing a scaling of the equations, leads to a new third dimensionless quantity that we will call κ. Varying the reed frequency with constant (γ, ζ, κ) on different pipe dimensions shows a certain stability of the model once put under this form. Using a real-time sound synthesis tool, the parameter space (γ, ζ, κ) is explored while the damping of the reed is also varied.
Download Physics-Based and Spike-Guided Tools for Sound Design In this paper we present graphical tools and parameters search algorithms for the timbre space exploration and design of complex sounds generated by physical modeling synthesis. The tools are built around a sparse representation of sounds based on Gammatone functions and provide the designer with both a graphical and an auditory insight. The auditory representation of a number of reference sounds, located as landmarks in a 2D sound design space, provides the designer with an effective aid to direct his search for new sounds. The sonic landmarks can either be synthetic sounds chosen by the user or be automatically derived by using clever parameter search and clustering algorithms. The proposed probabilistic method in this paper makes use of the sparse representations to model the distance between sparsely represented sounds. A subsequent optimization model minimizes those distances to estimate the optimal parameters, which generate the landmark sounds on the given auditory landscape.
Download Modal Audio Effects: A Carillon Case Study Modal representations—decomposing the resonances of objects into their vibrational modes has historically been a powerful tool for studying and synthesizing the sounds of physical objects, but it also provides a flexible framework for abstract sound synthesis. In this paper, we demonstrate a variety of musically relevant ways to modify the model upon resynthesis employing a carillon model as a case study. Using a set of audio recordings of the sixty bells of the Robert and Ann Lurie Carillon recorded at the University of Michigan, we present a modal analysis of these recordings, in which we decompose the sound of each bell into a sum of decaying sinusoids. Each sinusoid is characterized by a modal frequency, exponential decay rate, and initial complex amplitude. This analysis yields insight into the timbre of each individual bell as well as the entire carillon as an ensemble. It also yields a powerful parametric synthesis model for reproducing bell sounds and bell-based audio effects.
Download Pitch glide analysis and synthesis from Recorded Tones Pitch glide is an important effect that occurs in nearly all plucked string instruments. In essence, large amplitude waves traveling on a string during the note onset increases the string tension above its nominal value, and therefore cause the pitch to temporarily increase. Measurements are presented showing an exponential relaxation of all the partial frequencies to their nominal values with a time-constant related to the decay rate of transverse waves propagating on the string. This exponential pitch trajectory is supported by a simple physical model in which the increased tension is somewhat counterbalanced by the increased length of the string. Finally, a method for synthesizing the plucked string via a novel hybrid digital waveguide-modal synthesis model is presented with implementation details for time-varying resonators.