Generating personalised profiles of student engagement to predict student performance and support student learning using LMS data

Abstract

With the rising popularity of delivering educational content in the online space, this study set out to better understand how students are increasingly engaging with Learning Management Systems (LMSs) for their learning (Gupta, Muralidharan, & Raghavan, 2021; Henrie, Halverson, & Graham, 2018; Conijn, Snijders, Kleingeld, & Matzat, 2017).

A data mining approach was used to investigate whether student activity data from LMSs could be used, first in generating profiles of student engagement and, second, in predicting student academic performance (Romero & Ventura, 2013; Tempelaar, Rienties, & Giesbers, 2015; Conijn et al., 2017).

LMS activity data was collected throughout a semester for a cohort of undergraduate students (N=534) enrolled in a biomedical science subject. This activity data comprised of total views and downloads of lecture recordings and average time spent viewing lecture recordings, total number of page clicks, total number of LMS course and content page clicks (e.g., lecture lessons), total number of discussion forum posts and views, and total number of formative quiz attempts and reviews (Wang, Chen, & Anderson, 2019; Conijn et al., 2017). K-means cluster analysis was performed to generate profiles of student engagement using this digital LMS data. Measures of student activity derived from this digital LMS data were used to predict academic performance (students' final unit totals), using linear regression and ANOVA.

We demonstrate the utility of LMS log data in constructing 'profiles' of student engagement, and in establishing predictors of academic performance. In line with existing literature, average time spent viewing lecture recordings was determined to be a significant predictor of academic performance. Clustering analyses revealed three distinct clusters, or ‘profiles’ of student engagement, each representing key differences in student engagement activity, with greater homogeneity in student engagement behaviour within clusters, and greater heterogeneity across clusters.

The current findings provide valuable insights on how our students are increasingly engaging with LMSs for their learning. As delivery of education towards the online space continues to gain momentum, evidence-based approaches such as those utilized in this study will help inform and enhance teaching practices (Jovanovi?, Gaševi?, Pardo, & Dawson, 2020; Ullmann, Wild, & Scott, 2018; Conijn et al., 2017). A better understanding of how students are differentially engaging with LMSs will also help support the development of teaching resources that more closely mirror and complement online student learning behaviors (Dyckhoff, Zielke, Bültmann, Chatti, & Schroeder, 2013; Conijn et al., 2017).