Exploring adult learners’ experience with VR-generated feedback for improving online presentation

Poor presentations could be due to presenters’ lack of presentation skills, and lack of informed knowledge for improvement and feedback to self-monitor the speaking process. In a typical training setting, feedback involves using either video or human-intensive feedback that may not explicitly and effectively target at oral speaking components such as facial expression, eye contact, and gestures. Authentic feedback should be real-timed and aimed to improve presenters’ speaking skills and help them make decisions for targeted improvement. This pilot study reports 50 participants’ experiences. It investigates two groups of participants (VR and Non-VR) and their oral speaking experience, informed by feedback (VR-generated system or human supported), and analyses milestones in their presentations. Participants in the VR group receive VR-generated feedback while those in the Non-VR group receive computer and human feedback. Both groups work towards improving their speaking skills in subsequent presentation activities. Preliminary findings and implications will be discussed.


Introduction
Oral presentation is a core competence in a large variety of professions in the world. The quality of a presentation influences professional performance, career development, and effective communication in society (Van Ginkel et al., 2015). However, effective presentation skills are not mastered by everyone. Industries and institutions usually have extensive presentation training for their members, but its success relies on lots of human resources and different scenarios. An individual's competence in presentation determines whether the learners can fully comprehend and interpret what the individual has conveyed. An effective oral presentation is more than just proper pronunciation, intonation or articulation, and it would involve eye contact, gestures, verbal fluency and handling of questions that help learners to internalize and deepen their thinking (Ho, 2014). Poor presentation could be due to the lack of presentation skills, self-knowledge, attitude, or limited feedback and guidance during training (Van Ginkel et al., 2017). According to Knowle's characteristics of andragogy, adult learners are more self-directed, and they know how to better understand and appreciate the learning process and experience compared to child learners (Knowle et al., 2015). Transformational learning is the 'social process of construing and appropriating a new or revised interpretation of the meaning of one's experience as a guide to action' (Taylor, 2001, p. 220). Transformative learning is an essential part of adult learning as it determines adult learners' course of actions (Authors, 2022). Through transformational learning, adult learners are able to critically reflect on their learning and take action to improve. Hence, it is crucial for adult learners to add value to their training by taking account of their learning experience and be self-directed in working to improve their oral presentation skills.
Feedback is the key to improving oral presentation skills. Effective feedback could encourage learners' oral presentation performance (Van Ginkel et al., 2017). Compared to human feedback, VR is an automatic descriptor of presentation behaviour with multimodal data for oral presentation skills. From the previous studies, it is observed that the feedback presented inside Virtual Reality (VR) simulations is composed of multiple dimensions.
(1) the timing of the feedback (real-time or delayed) Studies have suggested that immediate feedback has a significant impact on learning (Van Ginkel et al., 2015). The real-time feedback within the VR presentation system could remind the presenters of the time given to their eye gazing, their voice volume, speech pace and fix these for them at one selected point during the presentation process (e.g., Belboukhaddaoui, & Van Ginkel, 2019). The delayed feedback includes pause timings, measures of voice qualities, the use of gestures (e.g., gestures to emphasise, gestures too much), gaze direction and time (e.g., El-Yamri et al., 2019).
(2) the modality of the feedback (visual, auditory, haptic) The feedback could be presented in the form of texts, pictures, video, and audio to help users reflect on their performance (e.g., Schneider et al., 2019).
(3) the complexity of the feedback The VR can provide detailed data for the presenter, such as the pause times, the frequency of using one word and, even, confidence level (e.g., Chollet et al., 2015;Hinojo-Lucena et al., 2020). These multi-sensor data could be analysed and transcribed as comprehensible feedback for the presenters to reflect on their presentation performance. Past methods to provide feedback on oral presentations include video recording and/or human feedback but these are not explicitly targeted at oral speaking components such as facial expression, eye-contact and expression of words spoken. Therefore, there is a need for consistent and concrete evidence on these non-verbal dimensions to be included in feedback to presenters to improve their oral presentations.
VR with 3D interactive environments provides support for situated and constructivist-based learning activities by allowing participants to immerse and interact directly with the people and surrounding (e.g., Chiou, 2020;Schneider et al.,2019). The VR environment can provide good control of the presented stimulus and open up various feedback channels (Theelen et al., 2020). The VR presentation training could support presenters in various aspects during and after the VR training process to improve their oral presentation skills.
First, presenters could receive real-time feedback and adjust their behaviours promptly during the VR presentation training process. The VR-feedback system is designed to support the development of oral speaking skills by providing more complex feedback such as eye contact and body gestures. Second, it allows presenters to review their practice and receive delayed feedback after the VR training. Presenters could review their presentation videos and look at their performance development based on the objective measurements. Third, some research also designs the reflection session within the VR system, and presenters have opportunities to practise behaviours repeatedly following the instructions from the system. Fourth, some of the studies employed design-based research as a methodology. This methodology incorporates the process of designing, developing, and evaluating the VR presentation training prototype. The process of practising, improving and reflecting helps presenters master presentation skills effectively. A wide range of institutions or individuals could employ this method for job interviews, public speaking training, work or academic presentation and many other real-life scenarios to benefit individuals who have the need to improve their oral presentation skills. The studies require a formative evaluation process, which consists of several iterations for investigating the effective implementation of VR modules for improving oral presentation skills. Therefore, this study examines the development of presenters' oral presentation skills with the support of VR-feedback system. As part of the major study, this pilot study investigates listenability and speaking pace of the speakers. Specifically, this pilot study focuses on two research questions: 1. What are the participants' perception of virtual presentations experienced in the three oral presentations? 2. Are there significant differences between the VR and Non-VR groups' performance in terms of their (a) listenability? (b) speaking pace?

Method
A pilot study was conducted from November 2021 to February 2022 on a group of 50 participants in Singapore. It adopted a randomised controlled, single-blinded 2-group pre-test and post-test design. The research assistant responsible for data collection was blinded to the group allocation (VR or Non-VR group) of the participants. Participants were anonymised and randomised to the intervention (with VR) and control groups. The team explored VR training strategies to improve trainers' speaking in a training situation. Developed in 2016, VirtualSpeech was adopted for this study because it is easy to use. The team adopted the Essential Public Speaking package for this study. Through email, participants were invited to take part in this study. The participants were briefed about the experiment and their consent obtained. The participants took part in three oral presentation activities (Stages 3, 4 and 5) as shown in Figure 1, each stage lasting around 15 minutes. At Stage 3 and 4, both groups were given the same treatment. However, at Stage 5, the VR group would wear the Oculus while the Non-VR group would not wear the Oculus to carry out their presentations. Stage 3 All participants did a warm-up practice to prepare them for the subsequent stages via an impromptu speech exercise. Stage 4 All participants orally presented a 5-minute presentation to a small group of virtual audience in a simulated zoom meeting setting. Prior to the presentation, they were given 10 minutes to prepare a script on the topic of "Living in the COVID-19 pandemic". Stage 5 All participants were given another 10 minutes to use the feedback given by VirtualSpeech to enhance their presentation. They were then tasked to orally present the same 5-minute presentation to a larger group of audience in a simulated TEDx Style Theatre setting. The randomly categorised participants in the VR group were required to put on a VR Oculus headset for the assessment of their presentation. This allowed participants to be immersed in the environment for their presentation. The participants in the Non-VR group continued to use the computer as the mode of assessment in the presence of the human After each stage of their experience, the participants were required to complete a survey and reflection log. All videos recorded were anonymized for data analysis. To answer research question 1, the participants' responses were captured via the self-evaluation survey (Table 1). To answer research question 2, the VR system generated the score of each participant based on their listenability (Table 2) and speaking pace (Table 3) were measured. The mean for listenability and speaking pace were calculated for both groups. T-tests were carried out to identify if there are any significant differences between the means. Figure 1 shows participants taking part in the different stages.  Table 1 shows all participants' scores based on their experiences in Stages 3, 4 and 5. With 1 being very bad and 5 being very good, the average score was 3.5 out of 5-point Likert scale. The participants were asked if they thought the practice in Stage 3 prepares them for using technology in presentation. With 1 being strongly disagree and 5 being strongly agree, the average score is 3.6. For Stage 4 and Stage 5, 4-point Likert scale was used. The average scores were 2.5 and 2.88 for Stage 4 and Stage 5 respectively. Table 2 shows the listenability scores of the participants in Stages 3, 4 and 5. The findings show that the VR group obtained higher mean scores for all the three stages. The difference between the mean score for listenability of the VR group and Non-VR group in Stage 5 is especially large; the values are 8.56 and 2.56 respectively. There was a significant difference between the mean for listenability of the VR and Non-VR groups (p < 0.05). This could suggest that participants' listenability improved with the VR intervention in Stage 5. The values of the standard deviation for the VR group and Non-VR group are 0.712 and 1.89 in Stage 5, which might suggest that the differences of listenability in the VR group are smaller than the Non-VR group in Stage 5.  For speaking pace (Table 3), there is no significant difference between the mean values of the VR and Non-VR groups at Stage 5. The mean values for speaking pace in Stage 3, 4 and 5 for the VR group were 3.92, 5.52 and 5.96 respectively and those for the Non-VR group were 2.76, 5.00, and 4.76 respectively. The results suggest that the speaking pace of the VR group gradually improved after practice at every stage except the non-VR group.

Conclusion
This preliminary study focuses on developing the presenters' oral presentation skills with the support of VRfeedback system and two features, listenability and speaking pace, were the primary focus in the VR-feedback environment. As there is no significant difference for the speaking pace between the VR and Non-VR groups, additional tests may be needed to examine the effect. From the data, it could be concluded that the VR-feedback environment has provided a positive effect and more guidance is given to presenters who practise using online presentations. The implications of this study are that VR is an appropriate medium to train adult learners in listenability in oral presentations. Although speaking pace was not shown to be significant, the VR group still attained a higher mean score as compared to the Non-VR group. This shows that VR is able to improve adult learners' speaking pace for presentations. So, the use of VR in oral presentation training could still be the mainspring for confidence in adult learners. Listenability and speaking pace are important in presentations be it online or onsite because they help to engage the audience during the presentation. In this study, the sample size was small (N = 25 per group), and, therefore, the outcomes of this study cannot be generalised. It is recommended that a larger sample size be used for future studies. Other features like gestures and eye-contact are equally important to assess adult learners during presentations. Hence, these other features could be explored in future studies as well.