At the heart of a diverse technology: Applying a realist evaluation methodology to a university live streaming programme

Abstract

Researching web-based lecture technologies in higher education is a complex undertaking (Morris, Swinnerton, & Coop, 2019), making experimental trials and summative evaluations impractical. The implications of existing literature in this area are therefore uncertain (Kay, 2012; O’Callaghan, Neumann, Jones, & Creed, 2017). In this paper we report on the use of an alternative research design, realistic evaluation (King, Dawson, Rothberg, & Batmaz, 2017; Pawson and Tilley, 1997), to gain insight into the impact of a university lecture live-streaming initiative. The initiative provided synchronous and asynchronous access to video of 129 weekly lectures using two different platforms. Surveys of 306 students and 49 staff, a focus group of five students, and an exploratory study of trace data of 359 students indicated that student access to the videos was lower and more idiosyncratic than expected. The ability to identify useful content, appropriate instructional methods, and participation by peers were reported to encourage students to become involved. Our results confirm the importance of specific contexts and forms of behaviour in encouraging beneficial lecture video use. We also show time of initial view to be a promising way to study students’ strategies for using lecture video content.