Ask Me

Three Use Cases of Multimodal AI in Education

Abstract

In this poster, we share ongoing research into the use of realistic AI-generated avatars for education in a large Australian business school. These highly realistic avatars have been used as educational presenters to provoke critical discussions on ethical AI by immersing students in text-to-video technology (Vallis et al., 2024). From this initial study, an emergent typology for avatars in educational media has been developed, to help educators design for teaching and learning based on learning purposes and modes of delivery Britton & Vallis, 2023; Vallis & Britton, 2024). Such practical, theoretical guidance in designing, developing, and deploying AI in education is much needed (Dawson et al., 2023). Drawing upon learnings to date, we further explore synthetic media designs for effective and ethical use in education. The poster presents three use cases of synthetic media with QR codes to demonstrate their application and to facilitate discussion at the conference. An additional fourth QR code will be added to gather participant feedback on which use case they found most compelling and educational, as well as their reasons for their preferences.

The first use case explores short video presentations, where synthetic media offers students a choice of accents, voices, and faces to present educational content (Dao et al., 2021). This approach aims to free teacher time for more learner guidance and communication, potentially enhancing learning outcomes. Video-based learning can enhance student engagement and understanding by providing flexible, interactive, and visually stimulating experiences (Sabli? et al., 2021).

The second use case is ‘Ask me’, a trial of a realistic avatar that answers frequently asked questions. Multimodal AI, like GPT-4o, with integrated voice, video, and audio capabilities, can simulate real-world scenarios for educational scenarios with engaging and realistic experiences (Mollick & Mollick, 2023). Future use cases may include specific learning activities such as role-play and rehearsal.

Multimodal feedback on student progress is presented in the third use case. A lifelike digital representation of a teacher has been designed to closely mimic a real human teacher’s appearance and expressions. Simple video feedback, generated based on student interactions with university systems and combined with a teacher’s text, is sent via email to enhance engagement. This trial is designed to augment feedback, rather than replace interaction between students and teachers (Bozkurt & Sharma, 2023).

By presenting these use cases, we aim to provide practical insights and stimulate discussions on the ethical and effective use of synthetic media in education. Our preliminary findings indicate benefits to using AI-generated avatars, if the ethical implications and the quality of human-AI interactions are carefully considered in the learning design (Vallis et al., 2024).