A risk-based approach to mitigating the Gen(AI) challenge to assessment integrity

The Programmatic Assessment Security Project (PASP)

Abstract

In the evolving landscape of higher education, emerging technologies such as Generative AI (GenAI) present both opportunities and challenges. The rapid increase in access to, and sophistication of, GenAI poses significant challenges for our institution and the wider sector (Bearman & Luckin 2020, Bobula 2024, Chan & Hu 2023). The most immediate of these challenges is the increased risk to the existing approaches used to assure that students achieve the stated outcomes of their learning experiences. Director of Higher Education Integrity Unit, TEQSA, Dr Helen Gniel argues ‘We are entering an age where institutions can no longer ensure the integrity of every single assessment, and it makes sense to think at that bigger level  […] what are the mechanisms that you have in place to be confident that an individual has demonstrated the learning outcomes to receive that degree’(TEQSA information request: AI risk mitigation, webinar, 21 March 2024). While higher education institutions are exploring opportunities to realise the potential benefits of GenAI, our immediate focus is on mitigating these risks. This poster describes institutional Programmatic Assessment Security Project (PASP) at the University of Newcastle (UON), a pioneering four-stage initiative designed to address academic integrity risks associated with inappropriate uses of GenAI in summative assessment (Lodge 2024). Academic integrity risks in this context refers to the potential for students to engage in academic misconduct by using GenAI tools in ways that violate institutional policies or ethical standards for assessment. These risks pose a threat to the validity and reliability of summative assessments, potentially compromising the integrity of academic qualifications and the learning process itself.

Led by the central teaching and learning support unit, in collaboration with the Artificial Intelligence Working Group (AIWG) a group of expert practitioners and academic stakeholders at UON, the PASP aims to secure assessments across a program of study, assuring student attainment of program learning outcomes. This innovative approach aligns with the sector’s focus on programmatic assessment and supports compliance with regulatory requirements, including TEQSA’s mid 2024 request for institutions to submit a detailed action plan to mitigate the risks that GenAI poses to the integrity of awards (TEQSA 2024).

This poster showcases exemplary innovation in practice through the PASP, advancing research and practice in technology-enhanced learning (TEL). The PASP employs a risk-based approach, identifying seven risk factors for inappropriate student use of GenAI in assessment tasks. The first step in this project involved Course Coordinators completing an Assessment Reflection Survey, guiding them to reflect on individual assessment tasks in relation to these risk factors. The survey data were then used to compile an understanding of the varying levels of assessment risk across the institution’s programs. The PASP will deliver a defined series of secure assessments, delivered at key moments across a program of study to assure student attainment of program learning outcomes. This poster provides insight into UON’s contemporary and agile approach to TEL and assessment redesign, with potential for staff at other institutions to adopt similar responses.