Modifying assessment tasks to ensure learning occurs in the era of artificial intelligence.

The launch of ChatGPT 3.5 in November 2022 has had a profound influence on teaching and learning from both students’ and staff members’ engagement. It is unclear how students are engaging with AI in creating submissions for assessments tasks, potentially impacting their learning. It is also unclear how educators are modifying their assessment design and how this may be impacting the learning outcomes of their cohorts. University systems run at a slow pace meaning that policies, course and program changes are not able to engage at the rate staff need in order to respond to this new technology. Staff have been left to deal with the use of AI in assessment tasks, often with little guidance or sufficient digital skills to make relevant changes or modifications. This workshop will focus on the role of Artificial Intelligence (AI) and assessment. Participants will be given structured activities to understand and develop activities that provide learning for students without being compromised by artificial intelligence. They will apply their learning in the workshop to an assessment they are using or plan to use.


Background
AI has had a significant impact on many industries over the last few years including architectural design (Bölek et al., 2023;Pena, et al., 2021), visual arts (Caramiaux & Fdili, 2022;Santos, et al. 2021), customer service (Song, et al., 2022) and marketing (Mehta et al., 2022;Vlačić et al., 2021).Since the launch of ChatGPT 3.5 late in 2022 there have been reports in the literature that this AI tool can pass legal (Choi et al., 2023;Mazurek, 2023) and medical (Humar, 2023) exams, provide answers to most questions posed in education and has been either banned or explicitly acknowledged as an issue that is of critical importance in schools and higher education institutions.
Whilst staff and students may not be universally aware of the capacities of AI, leadership of institutions have been working to consider the ramifications of AI tools in the learning space (Malinka et al., 2023;Ouyang et al., 2022).Staff have been left to deal with the use of AI in assessment tasks, often with little guidance or sufficient digital skills to make relevant changes or modifications.University systems also run at a slow pace meaning that policies, course and program changes are not able to engage at the rate staff need in order to respond to this new technology.
AI generated text is created through a process of data harvesting and many clever algorithms which examine a large database for likely answers to questions posed of it.ChatGPT (OpenAI) behaves like a chatbot on the surface and has a text window in which users can type questions.The response to this question is vastly different than what you might encounter in a typical search engine, providing clearly articulated text written in good English, similar to the dialogue you might genuinely expect from a human.Whilst ChatGPT does not 'know' what it is writing in the same way we do, it has been trained to mimic a human (Ariyaratne et al., 2023).It is relatively easy to subvert AI detection tools such as those in common tools in higher education such as Turnitin.We need to focus on learning (Al-Husseiny, 2023;Sullivan et al., 2023) rather than 'catching' violations of academic integrity and to do that our focus must be on assessment.

Objectives
In this workshop we will engage with common assessment types and assess how 'broken' they may be in our courses when students have access to AI. Participants will work in groups, first to analyse the potential weaknesses of a task, identify what learning outcomes the task aims to measure and then create a new task that will not have those weaknesses.We will use the SAMR (Substitution, Augmentation, Modification, and Redefinition) approach to frame the changes made to the assessment tasks.The specific learning outcomes are that participants will be able to: 1. Analyse existing assessment tasks with an understanding of how AI might subvert the intention of the tasks, 2. Demonstrate the use of SAMR in examining and changing assessment tasks, and 3. Design different assessment tasks to embrace or resist AI that ensure learning.
The intended audience for this workshop are any members of a higher education institution, either professional or academic staff who are involved in course design or run assessments for their students.No prior knowledge of artificial intelligence is required, and groups will be formed that have varying levels of expertise with this technology.
Workshop Structure -Presentation Mode: Face-to-face (5 -50 Participants) Where participants don't have access to technology eg for Mentimeter, an equivalent no-technology task will be allocated.

What do you need to bring?
Attendees need to bring one assignment they'd like to work on with regard to AI, but a mobile device that can run ChatGPT or other AI tools relevant to your discipline would be helpful.Group work will use butchers' paper, markers and sticky notes, but the use of AI tools devices will add to the depth of discussion.