Manaaki Tauira Course Enhancements Program: Improving outcomes of minority students (a pilot study)

This paper reports on a pilot study of a course enhancements program at Lincoln University, New Zealand, designed to enhance learner success for all students, but particularly those from minority groups. The pilot study includes two phases, each of six-months duration – course redevelopment and course delivery. During the first phase, 10 academics were selected to engage in self-reflection, complete a four-week online asynchronous course, develop an action plan for course redevelopment, attend a two-day in-person workshop to implement the action plan, and meet several times with an academic development facilitator. This blended approach resulted in academics experiencing a variety of in-person and online learning activities and approaches that could be used in their own courses, and the opportunity to collaborate and share their course redevelopment journey with other academics. This paper highlights the approach taken in the pilot study and the strategies used to bring academics on the journey.


Introduction
Regularly reviewing courses to ensure they are fit for purpose and designed with student learning outcomes in mind is becoming an increasingly important academic activity in today's ever-changing higher educational learning and teaching environment.Improvements in course design can lead to improvements in the quality of teaching, student learning experiences and student learning outcomes (Robinson et al., 2021;Ariovich and Walker, 2014;Swank and Whitton, 2019).Well-designed courses "a) consider the distinctive characteristics of the students in the course; b) have effective course design components (e.g., meaningful learning outcomes, engaging learning experiences, and authentic assessments); and c) demonstrate clear alignment between the various components of the course" (Robinson et al., 2021, p. 100).
The Maanaki Tauira Course Enhancement Program (hereafter MT CEP) was introduced at Lincoln University in Aotearoa New Zealand as a strategy to help improve learner success of all students, and in particular those from minority student groups (i.e., those identifying as Māori, Pasifika, and disabled students).It features a multicomponent professional development program delivered in a blended format that models student-focussed teaching practices, and includes opportunities for academic collaboration, community of practice, and selfreflection.This paper reports on this initial pilot study of the MT CEP.
The four key identified focus areas for course redevelopment were: course design, assessment, student engagement, and course delivery.The pilot study consisted of two phasescourse redevelopment (during Semester Two, 2023) and course delivery (during Semester One, 2024).Through the first phase, participating academics reflected on their own teaching, completed a four-week online asynchronous course, identified their personal strengths and competencies as well as highlighted the strengths and areas for improvement in their course design, and shared these with their lead program facilitator and other participating academics.In this first phase, the 10 participating academics were learners themselves as they discovered new ideas that could be used to enhance their role as a teacher and facilitator of learning.The intended outcomes of the first phase are pedagogical change, increased empathy of students and student learning, and development of leaders and change agents.The intended outcomes of the second phase are increased student success measures including pass rate, course satisfaction, and student feedback.
The remainder of this paper is structured as follows.The next section provides a brief overview of relevant literature on academic professional development and course design.This is followed by an outline of the pilot study with particular focus on the first of two key phasescourse redevelopment.The paper concludes with a discussion of the strategies used to bring academics on the journey.

Literature review
Effective professional development is widely recognised as a crucial factor in enhancing the quality and pertinence of education and learning.Nevertheless, numerous obstacles and constraints impede academics from engaging in fruitful professional learning experiences (Inamorato dos Santos et al., 2019).This is despite there being many varied approaches to academic professional development in higher education as noted by Prebble et al. (2004) and explored by Sancar et al. (2021) who look specifically at the transition from traditional to alternative approaches.The need for informal (significant, "backstage") conversations to be included to achieve "meaningful change in academic development" was noted by Pleschová et al. (2021, p. 203).Furthermore, the demand for classroom implementation and reflection in professional development has also been highlighted (Teräs, 2016;Ling and MacKenzie, 2001;Wargo, 2022).Questions arise as to the effectiveness and value of short training courses in professional development (Teräs, 2016;Makopoulou et al., 2021), yet the literature reveals that professional development that is delivered over a longer duration (Teräs, 2016, Lawless andPellegrino, 2007), and includes opportunities for collegial sharing, collaboration, development of community, promotion of teachers' agency, situated design, and practical application to be more effective (Brodie, 2021;Ling and MacKenzie, 2001;Owston et al, 2008;Pischetola et al., 2023).
In addition to the use of a multi-component program, prior studies have shown that the use of blended delivery (Brooks, 2010;Evans et al., 2020), modelling active learning and the online learning environment (Borup and Evmenova, 2019;King, 2015;Wargo, 2022), and providing an authentic student experience (Evans et al., 2020;Lefoe et al., 2009) are effective elements of professional development programs.Borup and Evmenova (2019, p. 1) found that "the ways the course was delivered and the online teaching methods modelled by the course instructor appeared to have a larger impact on perceptions and attitudes towards online learning".Regardless of the approach used, there is consensus that a well-designed and implemented academic professional development program will lead to improved student learning outcomes (Sancar et al., 2021).
The importance of linking professional development to student learning outcomes and curriculum has been noted (e.g., Cohen andHill, 2001, Garet et al., 2001).Further, Webster-Wright (2009), states that to move towards a model of continuing professional learning, it is important to link professional development to "ongoing and situated learning" (p.703) through course redevelopment.Robinson et al. (2021) suggest focusing on the missing components, ineffective components, and misaligned components of courses will lead to welldesigned courses, that will ultimately improve the quality of teaching.Reflection on course design and discussions with peers and experienced colleagues helped academics to critically evaluate their assumptions and beliefs and see alternative perspectives, leading to transformational learning (Wargo, 2022).This discussion suggests that a professional development program that includes multi-components, is delivered in a blended format that models teaching practice, includes opportunities for collaboration, community, and reflection, while focusing on course redevelopment would be effective in creating both short-term and long-term changes in teaching practice and student learning outcomes.The following section outlines a pilot study of such a program.

The pilot study
The MT CEP was designed to support academics at Lincoln University who are the examiners (i.e., lead faculty member for the course) of the priority courses through a range of activities in two key phasescourse redevelopment and course delivery as outlined below.The first ten priority courses to be included in the pilot were selected using several criteria including the pass rate for all students, pass rate for Māori and Pasifika students, participation rates for Māori and Pasifika students, pass rate for disabled students, course satisfaction scores from course evaluations, academic progress reports, class size, course level, the program/s the course contributes to, and student representative feedback and student feedback received by the students' association.
Staff in the Centre for Learning and Teaching (CeLT) at Lincoln University developed the MT CEP which is characterised by a multi-component program using a blended delivery format that draws on in-person and online asynchronous activities.Participating academics are provided with opportunities for collaboration and building communities of practice through conversations with other participating academics and the program facilitators.

Phases and activities
As noted above, the MT CEP includes two phases: course redevelopment and course delivery.The course redevelopment phase commenced in July 2023 and ran for an approximately six-month period in which participating academics redevelop their course based on an action plan they created through personal reflection, dialogue with peers, and conversations with the program facilitators.The course delivery phase is also approximately six-months in duration and includes the same academics delivering their redeveloped courses for the first time.The following sections provide specific details on each of these two phases.

Phase one -Course redevelopment
In our pilot program, phase one occurred between July-December 2023.The design of this phase was tempered with the knowledge that participating academics did not receive much, if any, workload release, and that their involvement in the program was determined by a priority course selection process rather than their own selfselection.

Course selection
Various strategies were used to obtain buy-in from the participating academics.The participating academics were informed that a course they are examiner for had been chosen for inclusion in the MT CEP by the Provost.Faculty leadership then consulted the participating academics about workload implications and concerns.Attendance at an in-person launch forum provided participating academics with an overview of the program, a chance to meet other participating academics, an opportunity to hear the experiences of an academic who had been through a self-selected course development process with the program facilitators, and an opportunity to ask questions and seek clarification about their involvement.A key factor in achieving buy-in was the development of a trusting relationship between the participating academic and the academic development facilitator including an understanding that they were involved in the program based on course factors and that the program was designed to address flaws in course design, not their teaching practice.Providing recognition of their involvement and achievements for inclusion in performance reviews and promotion was also important.

Course review
A full course review was undertaken for each priority course by the program facilitators in conjunction with each participating academic.A five-step process was used, forming the basis for identifying the specific elements for course redevelopment: Step 1: Set goals for the course redevelopment Step 2: Review existing course components Step 3: Source and integrate existing feedback Step 4: Record reflections, findings, and observations Step 5: Identify key areas for course enhancement

Completion of an online asynchronous course
An online asynchronous course, delivered using the university's Moodle-based learning management system, was designed for the participating academics to devote roughly five hours per week over a four-week period.The course models the online learning environment in which academics teach and exposes them to a variety of new and engaging student-centered learning activities that could be adapted within their own courses.The course works through targeted information on each of the four key focus areas (course design, assessment, student engagement, and course delivery) and thereby guides the participating academics to self-identify potential areas for course redevelopment.

Development of an action plan outlining specific aspects for course enhancement
A personalised action plan was developed by each participating academic with support from the program facilitators.The course review identified specific areas for course redevelopment, whilst the action plan solicited additional specific and personalised aspects within the focus areas for course redevelopment.Development of the action plan commenced at the initial consultation with the lead academic development facilitator and continued through the online course and the two-day in-person workshop.The action plan was further refined, as necessary, leading up to and during the first course delivery.

Implementation of the agreed action plan
The agreed action plan outlining the course redevelopment was implemented by the participating academic with support from the program facilitators.Many of the required course changes were made during the two-day inperson workshop and in the period following the workshop prior to course delivery through collaboration with other participating academics and the program facilitators.

Phase two -Course delivery
Phase two of our pilot program will commence in February 2024 and run through until July 2024, a period that covers Semester One and the delivery of each of the 10 redeveloped courses.A key part of this pilot is evaluating our success or failure, which will be done, in part, via a post-delivery review as noted below.

Delivery of redeveloped course
During the first course delivery of the redeveloped course, the academic development facilitator will continue to work with the participating academic to implement the agreed action plan.At least one teaching observation will be conducted in the first half of semester and regular check-ins will be scheduled to support ongoing course redevelopment.

Post-delivery review
A post-delivery review will be undertaken, by the academic development facilitator in conjunction with each participating academic, to assess the effectiveness of the course redevelopment in increasing student success (as measured by the pass rates, course satisfaction scores and student feedback).The post-delivery review will be held as soon as possible once the final course grades and student course evaluations are available.

Conclusion
This paper has outlined a pilot study for a professional development program delivered in a blended format that models teaching practice and includes opportunities for collaboration, community, and reflection.The professional development program focussed on course redevelopment through an examination of four key focus areascourse design, assessment, student engagement, and course delivery.
One of the key challenges with professional development is evaluating the effectiveness and tracing the effects on classroom practice and student learning outcomes (Rutz et al., 2012).During the first phase of the pilot program, participating academics were asked to complete an initial reflection on their teaching and their priority course.During the second phase, a post-delivery review will be conducted along with further questionnaires to identify the effects of the Course Enhancements Program on both teaching practice and student learning outcomes.The feedback from participating academics will be used to improve the program and promote its benefits to future participants.
Future offerings of the Course Enhancements Program will need to consider whether a pre-determined rather than self-selected approach to course selection is appropriate for achieving the desired outcomes, and whether any changes to the course selection criteria are needed.Feedback from participating academics and the program facilitators will also inform changes to future deliveries.