Download Gestural Auditory and Visual Interactive Platform This paper introduces GAVIP, an interactive and immersive platform allowing for audio-visual virtual objects to be controlled in real-time by physical gestures and with a high degree of intermodal coherency. The focus is particularly put on two scenarios exploring the interaction between a user and the audio, visual, and spatial synthesis of a virtual world. This platform can be seen as an extended virtual musical instrument that allows an interaction with three modalities: the audio, visual and spatial modality. Intermodal coherency is thus of particular importance in this context. Possibilities and limitations offered by the two developed scenarios are discussed and future work presented.
Download Physical Model of the Chinese Yehu for Sound Synthesis The yehu is a Chinese bowed string instrument featuring a resonator carved from a coconut shell, a seashell-based bridge, and two silk strings. This paper proposes a physical model of the yehu and reports on simulations using a finite-difference scheme with measurement-based physical characterization. The proposed model consists of two stiff strings coupled at the bridge, a bow with elastic bow hairs, a stopping finger, and a modal model of the bridge. A non-iterative solver based on energy quadratization is used to model the finger–string contact force, while an iterative solver is used for elasto-plastic bow-string friction force. The bridge-body model is based on a modal characterization obtained from the measured bridge admittance. The measured radiation transfer function is represented as a bank of parallel second-order filters and is applied to the simulated bridge force to incorporate body radiation characteristics. Finally, computational performance tests are conducted, showing that the proposed model is capable of real-time computation.
Download Recognition Of Ellipsoids From Acoustic Cues Ideal three-dimensional resonators are “labeled” (identified) by infinite sequences of resonance modes, whose distribution depends on the resonator shape. We are investigating the ability of human beings to recognize these shapes by auditory spectral cues. Rather than focusing on a precise simulation of the resonator, we want to understand if the recognition takes place using simplified “cartoon” models, just providing the first resonances that identify a shape. In fact, such models can be easily translated into efficient algorithms for real-time sound synthesis in contexts of human-machine interaction, where the resonator shape and other rendering parameters can be interactively manipulated. This paper describes the method we have followed to come up with an application that, executed in real-time, can be used in listening tests of shape recognition and together with human-computer interfaces.
Download Physics-Informed Deep Learning for Nonlinear Friction Model of Bow-String Interaction This study investigates the use of an unsupervised, physicsinformed deep learning framework to model a one-degree-offreedom mass-spring system subjected to a nonlinear friction bow
force and governed by a set of ordinary differential equations.
Specifically, it examines the application of Physics-Informed Neural Networks (PINNs) and Physics-Informed Deep Operator Networks (PI-DeepONets). Our findings demonstrate that PINNs successfully address the problem across different bow force scenarios,
while PI-DeepONets perform well under low bow forces but encounter difficulties at higher forces. Additionally, we analyze the
Hessian eigenvalue density and visualize the loss landscape. Overall, the presence of large Hessian eigenvalues and sharp minima
indicates highly ill-conditioned optimization.
These results underscore the promise of physics-informed
deep learning for nonlinear modelling in musical acoustics, while
also revealing the limitations of relying solely on physics-based
approaches to capture complex nonlinearities. We demonstrate
that PI-DeepONets, with their ability to generalize across varying parameters, are well-suited for sound synthesis. Furthermore,
we demonstrate that the limitations of PI-DeepONets under higher
forces can be mitigated by integrating observation data within a
hybrid supervised-unsupervised framework. This suggests that a
hybrid supervised-unsupervised DeepONets framework could be
a promising direction for future practical applications.
Download Creating Music by Means of ´Physical Thinking`: The Musician Oriented Genesis Environment Mass-interaction physical modeling scheme is often cited as the traditional physical modeling technique, but surprisingly some of the possibilities for musical creation it allows have not yet been pointed out. GENESIS is a graphical environment based on the CORDIS-ANIMA mass-interaction paradigm and designed for musicians. It was conceived so as to help the user "think physical"; that is, to discover and experiment with new ways of creating music, which is necessary when using physical modeling. The paper introduces version 1.5 of the GENESIS environment. Its major features and ergonomic aspects are exposed – especially model representation, low and high level modeling tools, multisensorial simulation facilities. Examples of composer's works are presented.
Download The PluckSynth touch string In this paper the problem of the synthesis of plucked strings by means of physically inspired models is reconsidered in the context of the player’s interaction with the virtual instrument. While solutions for the synthesis of guitar tones have been proposed, which are excellent from the acoustic point of view, the problem of the control of the physical parameters directly by the player has not received sufficient attention. In this paper we revive a simple model previously presented by Cuzzucoli and Lombardo for the player’s touch. We show that the model is affected by an inconsistency that can be removed by introducing the finger/pick perturbation in a balanced form on the digital waveguide. The results, together with a more comprehensive model of the guitar have been implemented in a VST plugin, which is the starting point for further research.
Download Modal Audio Effects: A Carillon Case Study Modal representations—decomposing the resonances of objects into their vibrational modes has historically been a powerful tool for studying and synthesizing the sounds of physical objects, but it also provides a flexible framework for abstract sound synthesis. In this paper, we demonstrate a variety of musically relevant ways to modify the model upon resynthesis employing a carillon model as a case study. Using a set of audio recordings of the sixty bells of the Robert and Ann Lurie Carillon recorded at the University of Michigan, we present a modal analysis of these recordings, in which we decompose the sound of each bell into a sum of decaying sinusoids. Each sinusoid is characterized by a modal frequency, exponential decay rate, and initial complex amplitude. This analysis yields insight into the timbre of each individual bell as well as the entire carillon as an ensemble. It also yields a powerful parametric synthesis model for reproducing bell sounds and bell-based audio effects.
Download Recent Advances in Physical Modeling with K- and W-Techniques Physical (or physics-based) modeling of musical instruments is one of the main research fields in computer music. A basic question, with increasing research interest recently, is to understand how different discrete-time modeling paradigms are interrelated and can be combined, whereby wave modeling with wave quantities (W-methods) and Kirchhoff quantities (K-methods) can be understood in the same theoretical framework. This paper presents recent results from the HUT Sound Source Modeling group, both in the form of theoretical discussions and by examples of Kvs. W-modeling in sound synthesis of musical instruments.
Download On the limits of real-time physical modelling synthesis with a modular environment One goal of physical modelling synthesis is the creation of new virtual instruments. Modular approaches, whereby a set of basic primitive elements can be connected to form a more complex instrument have a long history in audio synthesis. This paper examines such modular methods using finite difference schemes, within the constraints of real-time audio systems. Focusing on consumer hardware and the application of parallel programming techniques for CPU processors, useable combinations of 1D and 2D objects are demonstrated. These can form the basis for a modular synthesis environment that is implemented in a standard plug-in architecture such as an Audio Unit, and controllable via a MIDI keyboard. Optimisation techniques such as vectorization and multi-threading are examined in order to maximise the performance of these computationally demanding systems.
Download A Corpus-Driven Parametric Modal Reverberator A parametric modal reverberator is presented in which synthesis parameters are derived from a large, curated corpus of room impulse responses (IRs). The collected responses are subjected to modal decomposition, yielding per-mode frequencies, damping coefficients, and residue amplitudes, together with a short early-reflection finite impulse response (FIR) filter. From the decomposed data, a feature table is constructed per IR comprising standard acoustic indices, per-band damping and density statistics, amplitude distributions, and FIR descriptors—50 variables in total. Six acoustically meaningful user controls are selected; since these exhibit substantial pairwise correlations across the corpus, they are orthogonalised via principal component analysis (PCA) prior to regression.