Download Adapting Diffusion-Based Music Synthesis to Speech and Singing Voice Conversion Recent diffusion-based generative models have achieved strong results in domain-specific audio generation tasks such as speech, singing, and instrumental music synthesis. However, these models are typically specialized and do not generalize well to mixed or intermediate audio types. In this work, we adapt a diffusion-based model originally designed for multi-instrument music synthesis to voice conversion, covering both speech and singing within a unified framework. Specifically, we extend musical note-based conditioning to include phonetic posteriorgrams (PPGs) and pitch contours, and reinterpret timbre conditioning as speaker or singer identity via feature-wise linear modulation. Experiments show that the adapted model matches or surpasses a dedicated voice conversion system in terms of naturalness and performer similarity, while maintaining accurate pitch control across speech and singing. At the same time, we observe limitations in phonetic fidelity and a degradation in vocal quality when incorporating instrumental training data. Furthermore, we demonstrate that off-the-shelf feature extractors provide effective conditioning signals, enabling large-scale self-supervised training without manual annotations. These results highlight the potential of cross-domain model transfer towards unified audio generation systems capable of handling speech, singing, and music.
Download Pulsetable Synthesis of Wind Instrument Tones We revisit pulsetable synthesis, an efficient technique for generating plausible and expressive wind instrument tones. Based on the principles of pulse forming theory, this method models sound production as the periodic repetition of shaped pulses characterizing the target instruments' spectral envelope. In this approach, single-cycle waveforms, referred to as pulses, are stored in pulsetables indexed by their corresponding fundamental frequency. During synthesis, the pulses are read from these tables to form a periodic waveform, which is further shaped by time-varying low-pass filtering, amplification, and reverberation. These processes are guided by control signal contours that describe how fundamental frequency, brightness, and loudness evolve over time. Through case studies with real-world wind instrument recordings, we show how the interplay between these control signals gives rise to articulations such as attack transients, vibrato, and growl. Finally, we discuss the potential of this framework for integration into Differentiable Digital Signal Processing (DDSP) models, where neural networks could learn synthesis parameters directly from training data.