Download Deep Regularized RNNs for Virtual Analog
Virtual analog (VA) modeling methods seek to emulate analog audio hardware using digital signal processing (DSP). Modeling approaches fall into three broad categories: white-box methods, which use detailed device knowledge for accurate simulation; gray-box methods that use generic DSP blocks to model the system; and black-box methods, which rely solely on opaque models learned from input–output data. A category of architectures used widely in black-box modeling are recurrent neural networks (RNNs). To model device controls, the control values can be provided as conditioning input to the network. However, when the conditioning is time-varied, the models are susceptible to producing noise artifacts. Regularization of the RNN dynamics significantly reduces these artifacts, though at a loss in modeling accuracy. This paper closes the dynamics regularization quality gap by introducing deep control-conditioned LSTMs and a gammatone filterbank (GFB) loss. Experiments indicate that the proposed method achieves comparable modeling performance as unregularized baselines while avoiding the noise artifacts caused by time-varying control inputs.
Download Quantifying Nonlinear Behavior in Digital Moog Ladder Filters: Cross-Implementation Comparison and Common-Core Ablation
Digital Moog ladder filters are often compared by their linear frequency response, even though musically important differences emerge under nonlinear operation. This paper presents an open, reproducible SPICE-referenced evaluation framework that combines two components: a cross-implementation comparison of five digital ladder-filter implementations against the same SPICE reference, and a controlled common-core ablation. The cross-implementation comparison shows that close linear agreement can mask substantial nonlinear divergence, while the ablation shows that ladder nonlinear behavior depends not only on saturator choice and nonlinearity placement, but also on the non-uniform contribution of different ladder stages: in partial ablations of a SPICE-referenced TPT/ZDF core, retaining earlier-stage nonlinearities preserves harmonic behavior better than retaining later-stage nonlinearities alone. Beyond the specific comparisons reported here, the framework provides a reproducible basis for evaluating fidelity–cost trade-offs when nonlinear structure must be simplified.
Download Eigensystem Realization of Violin Bridge Admittances
Modeling violin bridge admittance is a long-standing problem in musical acoustics, with applications in sound analysis, synthesis, and virtual instrument design. In this work, we investigate the use of the Eigensystem Realization Algorithm (ERA) for deriving reduced-order state-space models directly from measured impulse responses. The proposed approach allows us to extract dominant system dynamics and obtain compact realizations without requiring explicit modal parameterization. We evaluate ERA on a dataset of modern and historical violins and compare it against established modal and state-space identification methods. Experimental results demonstrate that ERA outperforms existing approaches by achieving lower reconstruction errors in both the time and frequency domains while preserving perceptually relevant characteristics of the bridge response. Furthermore, we show that the state-space realizations obtained using ERA reproduce the target frequency-dependent energy decay more accurately than models obtained using the baseline methods. These findings support the use of ERA as an efficient and flexible alternative for modeling violin bridge admittances, with applications that span from audio synthesis and processing to instrument virtualization.
Download Benchmarking Integrated GPU Acceleration of Real-Time Neural Audio Inference on Snapdragon
This paper investigates whether integrated GPUs on Qualcomm Snapdragon SoCs can accelerate streaming inference of neural audio models. Five models spanning three orders of magnitude in parameter count are benchmarked across three inference approaches (best available CPU, QNN CPU, and QNN GPU). Results reveal when GPU acceleration offers meaningful gains, when per-call overhead negates benefits, and how model size and architecture determine GPU suitability.
Download A Comparative Study of Kolmogorov-Arnold Networks and Multi-Layer Perceptrons for Virtual Analog Modeling in Wave Digital Filters
The design of Virtual Analog (VA) algorithms has traditionally been divided between white-box (physics-based) and black-box (data-driven) approaches. Recent work has shown that hybrid methods, combining physical modeling with neural networks, can effectively leverage the strengths of both paradigms. In particular, Wave Digital Filters (WDFs) can be coupled with Multi-Layer Perceptrons (MLPs) to model circuits with multiple nonlinearities in a fully explicit manner. In this paper, we present a comparative study investigating the use of Kolmogorov-Arnold Networks (KANs) for VA modeling within the WDF framework. Unlike MLPs, KANs shift the learning paradigm by parameterizing activation functions instead of relying exclusively on learned weight matrices, potentially enabling more compact representations. Results show that, for our case study, KANs achieve accuracy comparable to MLPs while requiring approximately 70% fewer parameters at the cost of increased computational complexity. These findings suggest that KANs may represent a promising alternative in scenarios where memory footprint is a primary constraint, such as embedded audio applications, or when target models feature numerous nonlinear elements.
Download Physical Model of the Chinese Yehu for Sound Synthesis
The yehu is a Chinese bowed string instrument featuring a resonator carved from a coconut shell, a seashell-based bridge, and two silk strings. This paper proposes a physical model of the yehu and reports on simulations using a finite-difference scheme with measurement-based physical characterization. The proposed model consists of two stiff strings coupled at the bridge, a bow with elastic bow hairs, a stopping finger, and a modal model of the bridge. A non-iterative solver based on energy quadratization is used to model the finger–string contact force, while an iterative solver is used for elasto-plastic bow-string friction force. The bridge-body model is based on a modal characterization obtained from the measured bridge admittance. The measured radiation transfer function is represented as a bank of parallel second-order filters and is applied to the simulated bridge force to incorporate body radiation characteristics. Finally, computational performance tests are conducted, showing that the proposed model is capable of real-time computation.
Download FoleySet: A Multi-Level Human-Annotated Foley Sound Dataset
We introduce FoleySet, a human-annotated Foley sound dataset designed to support research on sound-event understanding and Foley sound generation. The dataset provides annotations at multiple levels of granularity, capturing both broad event categories and more detailed semantic or production-related attributes. This multi-level structure supports tasks such as classification, retrieval, captioning, and controllable generation. FoleySet is intended to address the limited availability of systematically annotated Foley material and to provide a common resource for evaluating models across different levels of semantic detail.
Download PAEDB: A Synthetic Primary-Ambient Dataset Generation Pipeline for Automatic Upmixing Using Deep Neural Networks
Automatic blind upmixing aims to convert audio from a smaller channel format (e.g. mono or stereo) into a multichannel format using estimates of direct and diffuse spatial statistics within the signal. Current approaches rely on primary-ambient extraction (PAE) algorithms, which lack real-world context through limited processing windows. Deep learning music source separation (MSS) models have been applied in voice-primary-ambient extraction (VPA) upmixing systems for handling direct components, but still rely on DSP methods of surround channel generation. This work further investigates utilizing source separation within VPA upmixing, focusing specifically on the task of stereo decorrelation and ambience extraction for 5.1 surround. We also release PAEDB (Primary–Ambient Extraction Dataset), a high-quality music dataset derived from MUSDB18-HQ and MoisesDB, comprising 1,809 primary–ambient stem pairs totaling over 550 hours of audio. The performance of selected DNNs trained on PAEDB is then evaluated using signal metrics and a listening study. Our findings indicate that DNNs can effectively model the behavior of PAE algorithms, establishing PAEDB as a strong foundation for ML upmixing systems and underscoring the need for higher-quality multichannel data to advance beyond conventional methods.
Download Modal Structure of Plate Boundaries and Klein Bottle Reverberation
Physical modeling sound synthesis has achieved remarkable success in terms of its fidelity to reality. In many cases, since modeling of the physical system is performed on the sounding objects that already exist in the real world, observation precedes the model itself. Departing from this convention, this paper aims to physically model the acoustic characteristics of objects that do not necessarily exist in reality. Specifically, we study wave propagation on compact two-dimensional (2D) manifolds that are non-orientable surfaces, such as the Klein bottle that cannot be embedded in three-dimensional Euclidean space without self-intersection. We derive closed-form expressions for the eigenfrequencies and mode shapes of non-orientable 2D topologies and study their acoustic characteristics. The modal structures are verified through comparison with finite-difference time-domain simulations. The results demonstrate how the topological character formed by the boundaries influences the acoustic resonances, and how the quotient-space framework provides a practical route to reverb synthesis on geometries with no physical counterpart.
Download Using the Distribution Derivative Method to Model Acoustic Musical Instrument Sounds with Polynomial AM-FM Sinusoids
The oscillatory modes of musical instrument sounds are commonly modeled with time-varying sinusoids. Several estimation methods model quasi-stationary oscillations accurately, yielding a high-quality representation. However, nonstationary oscillations such as attack transients are still very challenging to model accurately. In this work, we propose to model musical instrument sounds with polynomial modulation sinusoids (PMS) estimated with the distribution derivative method (DDM). DDM gives accurate parameter estimations for PMS with arbitrary order, allowing great flexibility in modeling temporal changes inside analysis frames as amplitude and frequency modulations. We used 39 musical instrument sounds to compare DDM objectively against the standard (SM+) and an adaptive sinusoidal model (eaQHM) using time and frequency error measures. We showed that DDM captures more oscillatory energy than SM+ or eaQHM by modeling PMS more accurately. A MUSHRA listening test with 18 selected sounds confirmed that DDM has higher perceptual quality than both SM+ and eaQHM and that DDM is almost perceptually indistinguishable from the original sounds.