This paper will describe the approach of modeling and sonifying the interaction with a pen on surfaces. The main acoustic parts and the dynamic behavior of the interaction are identified and a synthesis model is proposed to imitate the sound emanation during typical interactions on surfaces. Although a surface is twodimensional, modeling acoustical qualities of surfaces has to employ volumes to form resonances. Specific qualities of surfaces like the roughness and the texture are imitated by a noise generator which is controlled by the pen movement in real-time to achieve a maximum of acceptance of the sound effect. The effect will be used one hand to produce natural and coherent interaction on nearly silent electronic white boards or pen-tablets, i.e., reinventing of lost sound qualities. On the other hand modeling and sonifying pen strokes on surfaces allow to convey information about the properties of different areas or the current state of a windows of a computer display by using this sound feedback. Keywords : sound model, human-computer interaction, real-time, disappearing computer, audio feedback, sonification, mixed reality, multi-modal

Download