Exploring the Use of Audio-Visual Feedback within 3D Virtual Environments to Provide Complex Sensory Cues for Scenario-Based Learning

Abstract

The continuous quest for ever increasing fidelity in 3D virtual worlds is running parallel to the emergence and adoption of low-cost technologies to implement such environments. In education and training, complex simulations can now be implemented on standard desktop technologies. However, such tools lack the means to represent multisensory data beyond audio-visual feedback. This paper reports on a study that involved the design, development and implementation of a 3D learning environment for underground mine evacuation. The requirements of the environment are discussed in terms of the sensory information that needs to be conveyed and techniques are described to achieve this using multiple modes of representation, appropriate levels of abstraction and synesthesia to make up for the lack of tactile and olfactory sensory cues. The study found that audio-visual cues that used such techniques were effective in communicating complex sensory information for novice miners.