Thanks for the feedback: Reconnecting students with educators using a metacommunication feedback tool

Given the vast number of learning and teaching tools available to assist in the online delivery of courses, academics and educators have found it difficult to identify which tools students find the most useful for learning. This paper discusses the development of a new tool conceptualised and developed by the authors of this concise paper, which is designed to elicit immediate student feedback about online learning content and activities. Results from a small pilot study examining the potential of the tool to measure and improve student engagement in a second-year criminology elective delivered online are also presented. Providing opportunities for students to give immediate feedback is vital to improving the quality of learning and teaching. Educators can (re)connect with students studying online, improve their understanding of the needs of online learners, and tailor learning content and activities throughout the semester leading to improved student satisfaction, student-teacher relationships, and learner outcomes.


Background
The student cohort is changing with more students studying online than ever before. The typical university student is no longer the domestic school leaver studying on-campus. Today, university cohorts are diverse, consisting of a wide range of students who differ in age, cultural background, socio-economic status, geographical location, and physical and cognitive abilities (Department of Education, Skills, and Employment, 2021). As such, there has been an increased demand for flexible online learning environments that meet the needs of this diverse cohort who may be limited in their ability to study on-campus (Dyment et al., 2020). This demand has led most Australian universities to provide opportunities for students to study online or multimodally (a combination of on-campus and online). However, many higher educational institutes are illequipped for online teaching.
Attempts to transition from traditional face-to-face only mode to multimodal modes rely on lecture capture tools and the digitisation of texts. While this was a well-intentioned attempt to meet the needs of all students, research has found that this passive, teacher-centred approach to content delivery disengages students (Dyment et al., 2020;Smith & Kaya, 2021;Wammes et al., 2019). As a result, educators are exploring how new technologies could be used to deliver course content in more engaging ways. However, given the extensive availability of new educational tools, it can be difficult to identify which tool(s) students find most useful and engaging. As such, the authors of this paper designed and implemented a new user reaction toolbar to elicit immediate student feedback on the educational technologies used in an online course. Through identifying which educational technologies students engage with most, educators can better understand the needs of students, promptly address any issues throughout the semester, and improve student satisfaction and student-teacher relationships.

Project Description
Reaction icons are a ubiquitous feature of social media networking sites, such as Facebook, Instagram, Twitter, WhatsApp, and TikTok (Eftekhar et al., 2014;Hayes et al., 2016;Meier et al., 2014;Sumner et al., 2018). Identified as a form of metacommunication, reaction icons allow users to communicate an array of contentbased messages using minimalistic nonverbal communication (Bryant et al., 2011;Hayes et al., 2016). With research showing that reactions are used in social media to convey enjoyment, approval of content, or acknowledge interest (Sumner et al., 2018), the authors of this paper hypothesised that reactions could be used in online learning environments to obtain immediate student feedback on learning content and activities as one way to assess whether students find the learning materials and activities useful and engaging. The reaction toolbar was developed using learning tools interoperability (LTI) specifications and was designed to mimic the visual appearance of user reaction tools used on the popular social media networking site, Facebook. Figure 1 shows the final design of the reaction toolbar. A prompt question, 'Did you find this useful?', precedes three reaction iconsthumbs up, thumbs down, and provide feedback. Figure 2 shows the seamless placement of the reaction toolbar under a podcast learning activity. Feedback provided by students is anonymous which helps to facilitate a safe learning space where students can express themselves honestly without fear of being identified or judged (Shaheen et al., 2021). Once selected, the thumbs up and down icons are highlighted in yellow (Figures 3 and 4, respectively). The feedback icon turns yellow once feedback is provided in the pop-up comment box ( Figures 5 and 6, respectively). Each interaction made by a student is saved and can be changed or updated at any time by the student. For example, if a student provides feedback and the educator responds to it (e.g., makes a change), then the student can return to the comment and edit it. Students can only see their own feedback and comments.  (Table 1) was also developed to allow the educator to address any identified issues for the current cohort. Through examining the feedback and adapting the learning content and activities to meet the needs and expectations of students, the feedback loop can be closed (Harvey, 2022). Research has shown that closing the feedback loop is vital to increasing student engagement and satisfaction, as it signals to students that they are valued by the institution (Young & Jerome, 2020). As such, the educator in the pilot regularly analysed the feedback provided by students and responded by thanking students for their feedback and providing solutions to issues raised. Closing the loop is also important to educational institutions because research has shown that if students do not see improvements made as a result of their feedback, they are less likely to provide feedback in the future (Leckey & Neill, 2001;Watson, 2003).

Methodology
To test the functionality and potential of the reaction toolbar, a small, mixed methods pilot study was conducted in a second-year criminology elective in Semester 1, 2022. The study was approved by the institution's Human Research Ethics Committee (H22REA014). After gaining ethics approval, a total of 104 reaction toolbars were attached to all online learning content and activities including videos, discussions, H5P content, podcasts, Mentimeter, Padlet, images, and a Latest Announcements feed. An animated video explaining the reaction toolbar was created and posted in the Announcement Forum in the learning management system (Moodle), along with the study information and participant information sheet. In addition, a questionnaire was provided to students via the institutional survey tool to examine their experiences in the course. The questionnaire consisted of five questions and took students 10-15 minutes to complete. The questions focused on overall course satisfaction, engagement with course materials, and the best/worst aspects of the course, with a final open-ended question for additional feedback. Time was allocated during the last class of the semester to allow students to complete the questionnaire. A total of 14 students were enrolled in the course.

Results
Preliminary results from the pilot show that students value the current learning content and activities used in the course. Table 2 below summarises the results of the pilot study. Eleven students from a possible 14 (78% response rate) provided immediate feedback using the reaction toolbar to 24% of the learning content and activities. In total, there were 24 positive reactions, one negative reaction, and one positive comment thanking the teacher for the short course content videos which the student found 'very easy to listen to and engaging'.
In addition to these results, feedback provided via the questionnaire further indicated that students found the online learning content and activities 'extremely useful' and contributed to an increase in their engagement with the course. This was particularly prevalent for the following learning content and activities: module notes (delivered via Moodle books), interactive learning activities (e.g., in-line discussions, H5P activities), and online lectures. Comparing current data to data from the same course in Semester 1, 2021 further demonstrates that replacing traditional forms of content delivery (e.g., PDF documents) with new technologies (i.e., Moodle books, H5P activities) improves student engagement (Figure 7). Figure 7: Changes in student engagement between S1 2021 and S1 2022

Discussion
The reaction toolbar developed for this project aims to allow educators to create a learning experience for students based on frequent feedback, which is critical to learning (Schell et al., 2013). As Harvey (2011) notes, student feedback is "one of the most effective tools in the ongoing improvement of the quality of higher education" (p. 21). It can assist educators understand the learning experiences of students, evaluate student reactions to current learning content and activities, and improve course design and delivery (Marsh, 2007). While there are some concerns that students are over-surveyed, studies have shown that providing regular feedback gives students an increased sense of being supported and valued (Lane & Meth, 2021).
The shift from on-campus to online learning environments has significantly impacted student-teacher relationships (Vagos & Carvalhais, 2022). Contact hours have decreased, which has impacted the ability of teachers to assess student reactions to learning content and activities (Almahasees et al., 2021). While face-toface modalities provide opportunities for teachers to observe nonverbal communication (e.g., head nod) used to evaluate student interest and engagement, online teaching does not afford the same opportunity (Sumner et al., 2018). Although online students can provide feedback through online student evaluations, these are frequently distributed towards the middle or end of the teaching period. As such, feedback provided is most likely to be 0 20 40 60 80 100 1 2 3 4 5 6 7 8 9 10 11 12 S1 2021 S1 2022 used to adjust learning content and activities for the next course delivery and cohort. Any current issues are unlikely to be addressed, which may lead to student dissatisfaction (Marsh, 2007).
The reaction toolbar created and developed by the authors of this paper provides a new approach to collecting student reactions and feedback. While the findings of this pilot study are limited due to the small number of participants, the results are encouraging. It suggests that through listening and responding to student feedback, educators can (re)connect with students studying online and multimodally, improve their understanding of online learner needs, and make timely improvements to learning content and activities. With the ever-increasing number of educational technologies on offer, this approach has the potential to future-proof online learning environments and ensure that learning content and activities always meet the needs of the students.