Serious games can be used for training orientation and mobility skills of visually impaired children and youngsters. Here we present a serious game for training sound localization skills and concepts usually covered at orientation and mobility classes, such as front/back and left/right. In addition, the game helps the players to train simple body rotation mobility skills. The game was designed for touch screen mobile devices and has an audio virtual environment created with 3D spatialized audio obtained with head-related transfer functions. The results from a usability test with blind students show that the game can have a positive impact on the players’ skills, namely on their motor coordination and localization skills, as well as on their self-confidence.

Download

In order to contribute to the access of blind and visually impaired (BVI) people to the study of chemistry, we are developing Navmol, an application that helps BVI chemistry students to interpret molecular structures. This application uses sound to transmit the information about the structure of the molecules. Navmol uses voice synthesis and describes the molecules using the clock polar type coordinates. In order to help the users to mentally conceptualize the molecular structure representations more easily, we propose to use 2D spatial audio. This way, the audio signal generated by the application gives the user the perception of sound originating from the directions of the bonds between the atoms in the molecules. The sound spatialization is obtained with head related transfer functions. The results of a usability study show that the combination of spatial audio with the description of the molecules using the clock reference system helps BVI users to understand the molecules’ structure.

Download

The abundance and ever growing expansion of user-generated content defines a paradigm in multimedia consumption. While user immersion through audio has gained relevance in the later years due to the growing interest in virtual and augmented reality immersion technologies, the existent user-generated content visualization techniques are still not making use of immersion technologies. Here we propose a new technique to visualize multimedia content that provides immersion through the audio. While our technique focus on audio immersion, we also propose to combine it with a video interface that aims at providing an enveloping visual experience to end-users. The technique combines professional audio recordings with user-generated audio recordings of the same event. Immersion is granted through the spatialization of the user generated audio content with head related transfer functions.

Download