Responding to Generative AI

Results from an action-research study to improve university assessments

Abstract

There are few technologies which have such great a potential impact on the design and delivery of tertiary education, as found with Generative Artificial Intelligence (Gen-AI). Student’s deep learning and the integrity of student assessment in particular is impacted by this emerging technology. This presentation will report on a three-cycle action research project to redesign assessments for effective learning in an Australian university Health Science (HS) faculty.

Cycle 1 reviewed assessments using an innovative appraisal tool for assessments which was designed by the presenting HS academics that aligned recent Gen-AI and academic integrity guidelines from Australian institutions (Lodge et al., 2023a; Monash University, 2023; Flinders University, 2023; TEQSA, 2022; Torrens University n.d.). The tool was reviewed and improved for use in Cycle 2, which involved reviewing the remaining 16 HS subjects and designing a team-based strategy for implementing assessment reform. Cycle 3 involves implementing identified strategies for redesigning assessments across all HS subjects. We will report on the action-research methods used, including the development and use of the assessment appraisal tool, as well as the Cycle 3 early findings from the HS subjects assessment review. Results from the subsequent strategic assessment design will be presented using example assessment improvements based on the study findings.

The findings highlight the need for a unified approach to assessment reform, and the potential benefits of systematic, team-based approaches to address emerging technologies. Action-research methodologies such as this one, employed with subject coordinators' involvement and training in evaluation, encourages pro-active subject quality improvement as well as scholarly thinking among university academics. This approach may be relevant to all academics interested in actively creating assessment reform amid the transformative education environment precipitated by the use of Gen-AI.