Paper Archive

We present Clean2FX, a study and demo of label-conditioned clean-to-effect transformation for electric guitar audio. Given a clean guitar input and a target effect label, the task is to synthesize the corresponding effected signal while preserving the musical content. Training and evaluation pairs are constructed from EGFxSet real, single-tone recordings by assembling matched clean/effected chords, melodies, and mixed timelines. This allows for controlled comparison across effects. We evaluate four neural approaches under a common spectrogram-based transformation setting: two variational autoencoders and two U-Net models that differ in whether they operate on linear or log-magnitude representations. Performance is measured using linear-magnitude spectrogram MSE and Fréchet Audio Distance. The U-Net models outperform the variational autoencoder variants. Per-effect results show that distortion effects are most readily improved, whereas delay and reverb effects exhibit weaker FAD gains despite substantial spectral-error reductions. A conditioning-sensitivity diagnostic provides evidence that the best model responds to target labels rather than collapsing to a single transformation. Our demo website compares two models applied on real-world guitar performances outside training and validation data, providing audio and spectrogram examples of the practical clean-to-effect behavior.

Download

Expressive piano performance generation needs symbolic pitch, timing, and dynamics. We evaluate six tokenization strategies for a Transformer that generates classical piano performances. Our tokenizations add velocity, beat annotations, and sustain pedal, from note-only to full representations. We pretrain on MAESTRO, then finetune on ASAP with beat-level annotations. The model uses anticipatory-style note encoding with cross-attention on composer and genre. FAD on the ASAP test set shows that note + velocity + pedal and full modes achieve the lowest mean FAD (1.76 and 1.97). Both beat the note-only baseline (3.10). Beat tokens show mixed, category-dependent effects and do not improve the best modes on average.

Download

This project introduces an automatic method for enhancing audio quality in singing and speech. Using recordings from the LibriSpeech and Smule DAMP dataset, I applied a set of degradations and tested a set of audio effect "remedies" designed to reverse them: a high shelf filter, de-esser, noise reduction, and high-pass filter. I used the CLAP (Contrastive Language-Audio Pretraining) model to estimate recording quality and recommend remedies by comparing audio clips to descriptive text prompts in the shared embedding space. To evaluate my method, I conducted a large-scale listener study with 234 participants and 4,600 ratings. While CLAP encoded some relevant information of vocal recording quality, it often favored remedies like noise reduction while listeners preferred the original clips, suggesting that perceptual artifacts introduced by enhancement may not be captured by CLAP. My findings underscore the value of human judgment: embedding models can guide enhancement, but perceptual validation remains valuable. Audio examples are available online on the DAFx demo website.

Download

Processing and recording multitrack audio from an electronic drum set is demanding of computational power and hardware resources. In this paper, we present a complete musical instrument system capable of up to 16-channel percussion sampling, processing, and recording, all in real time. The system leverages a field programmable gate array (FPGA) for parallel audio processing and includes audio effects such as pitch shift, delay, reverb, distortion, a virtual analog low-pass filter, and bit crush. The FPGA also provides interfaces for other system hardware, including an Ethernet audio interface and various audio effect control interfaces. The final design has a cost of under $500 and utilizes about half of the hardware resources on an entry-level FPGA, providing a future platform for more advanced percussion synthesis using real-time physical modeling.

Download

Live performance leaves little room for a sensor pipeline that misfires; when a gesture fails to map correctly to an intended effect, the error is immediately audible. This paper presents L-Bow, a wrist-worn six-degree-of-freedom (6-DoF) inertial measurement unit (IMU) controller for augmented violin performance. By removing intermediate software layers, L-Bow integrates gesture sensing, six performance modes, and a shared digital effects chain within a single, self-contained Csound file. This is achieved using Csound's native arduinoRead opcode for direct serial communication rather than an external Python–OSC bridge. The paper discusses the architecture of this system and its implications for designing dependable, low-maintenance interactive digital audio effects.

Download

We present InstructFX2FX, a system for sequential audio effect refinement through multi-turn natural-language instructions. Existing text-to-effect systems are largely single-shot, mapping one textual descriptor to one preset. Real audio engineering is instead sequential: engineers refine an existing effect chain through successive instructions. This poses a stateful problem that single-shot systems do not address: given the current effect parameters state and a new instruction, update the sound while preserving what earlier instructions already achieved. InstructFX2FX addresses this with a hybrid architecture that divides labor between a language model and CLAP-guided optimization. The LLM serves as a high-level planner that selects effects and proposes the initial parameter state, motivated by recent evidence that LLMs can outperform CLAP-based optimization for single-turn text-to-effect mapping; CLAP-guided optimization then refines the existing parameter state, providing a more stable and robust refinement mechanism than LLM reprompting. In the demo, attendees drive a dry recording through successive natural-language instructions: after each turn, they choose how strongly the effect is applied, then issue the next instruction based on what still differs from the sound they intend. In a preliminary evaluation on SocialFX-derived descriptor pairs, CLAP-guided refinement achieves lower DSP-feature MMD than an LLM+LLM initialize-then-reprompt baseline on 9 of 10 pairs. Trajectory analysis further shows that, for differentiable effects, optimization tends to gradually move the audio toward the new target while retaining the effects of the previous instruction, highlighting the potential for gradual refinement. Audio demo and source code are available online.

Download

Overlap-add (OLA) is the simplest approach to audio time stretching. Methods like the phase vocoder (PV) and waveform-similarity OLA (WSOLA) offer higher quality results but require operations like the FFT or cross-correlation. On low power embedded hardware, this cost adds up quickly. We present a content-adaptive OLA method, an order of magnitude cheaper than PV or WSOLA, whose dominant artifacts are added saturation and some spectral contrast loss. Our method reduces uniformly sampled signals to sets of timestamped local extrema, a sparse representation where the distance between points encodes the signal's information density directly into the buffer. In OLA, the crossfade duration is fixed, but no one value suits both transients and sustained sounds. We use the extrema density to inform the crossfade duration, adapting it to the signal's local content on a sample-by-sample basis. We compare our method against OLA, WSOLA, and PV using objective metrics and a listening test. Our method coherently stretches audio, preserving transients across a wide range of stretch ratios and capturing dense, layered material cleanly.

Download

This work introduces a novel framework for Concatenation-Driven Convolution (CDC), unifying concatenative synthesis and real-time convolution into a single integrated audio processing paradigm. While concatenative synthesis has traditionally been used for corpus-based sound generation and convolution has served as a largely static filtering technique, the proposed approach reconceptualizes impulse responses (IRs) as dynamic, navigable sonic material. In the CDC framework, a corpus of audio segments is analyzed using perceptual features and organized via a self-organizing map (SOM), enabling intuitive, gesture-based traversal of a structured timbral space; the resulting concatenative output is treated as a continuously evolving impulse response and injected directly into a partitioned convolution engine. Its central technical contribution is single-engine frequency-domain kernel interpolation: rather than crossfading the outputs of two convolution engines, the FFT-domain kernels of the current and target IRs are interpolated within a single engine, preserving the internal convolution state across IR transitions and avoiding the warm-up energy loss inherent to dual-engine crossfading.

Download

Shimmer reverberation is an effect used in music production to deliver ethereal, pitch-shifted textures and evolving ambient soundscapes. This paper explores the synthesis of shimmer effects using the feedback delay network architecture, a popular real-time reverberator. We propose five distinct approaches for integrating nonlinear and time-varying operations into the feedback loop, focusing on expanding the harmonic content while adhering to energy-preservation and stability criteria. Our approach can generate a wide range of sonic characteristics, from harmonically rich distortions to musically coherent pitch-shifted reverberation, while maintaining stability and controllable decay behavior.

Download

This paper presents a framework for designing creative reverbs using parametric orthogonal feedback matrices on Feedback Delay Networks (FDNs) through recursive Kronecker products of 2D rotation and reflection matrices. By parameterizing each 2×2 kernel with a single angle, we construct a family of 2M×2M orthogonal matrices that maintain losslessness while enabling continuous control over network topology. We then exploit their recursive definition to compute the feedback operation with an O(N log₂ N) divide-and-conquer algorithm that matches the Fast Walsh-Hadamard Transform time complexity while offering parametric flexibility. Strategic manipulation of individual kernel angles enables creative sound design applications, such as stereo cross-coupling, selective freeze, and time-varying modulation for resonance breaking.

Download