Student use of Generative AI

Findings of a multi-institutional large-scale survey

Abstract

Despite the potential and challenges of generative AI (GenAI) in higher education, there remains a significant gap in empirical research to guide strategic decision-making. Current studies are often conceptual or small-scale explorations of staff and student experiences, ranging from analyses of the potential advantages of GenAI, through to expressions of serious concerns about risks for higher education (e.g. Chan & Hu, 2023).  Large-scale and comprehensive, data-driven investigations remain rare, however. In particular, there is a need to explore the experiences of students, to better understand what they know, how they are using GenAI, why they are using it, and their attitudes about the use of AI in their studies, by their educators and in their future lives (e.g. Lodge et al., 2023; O’Dea, 2024).

This Pecha Kucha reports on the second phase of a multi-institutional collaborative project. The project’s overall research question is ‘How do university students make meaning about GenAI in relation to themselves as learners?’ The first phase involved focus groups at each university, followed by a large-scale online survey. The analysis of the focus group data informed the design of a cross-sectional survey to explore students’ perspectives on GenAI on topics such as knowledge and access, use and usefulness, and attitudes.

The survey development was an iterative and significant undertaking. Current literature was unable to provide a comprehensive picture of student use and attitudes. Given the rapidly changing GenAI landscape, we were also aware that student experience and attitudes are contextually dependent and may also evolve quickly. Whilst valuable, existing studies have not connected the varied dimensions of the student experience we seek to investigate.

While keeping these studies in mind, this project used a grounded thematic approach to build its own survey instrument, beginning with 4 focus groups of 11 scholars to identify the key issues which needed to be explored within the Australian university context. This stage was followed by iterative development of the survey, in which over 200 suggested edits were provided by 17 scholars as well as a reference group of Deputy/Pro-Vice Chancellors. When completed, the survey was piloted by undergraduate students.

The anonymous online survey was conducted in the second half of 2024 at four Australian Universities located in Victoria, New South Wales, and Queensland. Human Research Ethics Committee approval was granted at each University prior to conducting this study.

The data is currently being collected. To date, the survey has been completed by 8,340 students. The Pecha Kucha will present key descriptive data on selected components of the survey including student knowledge and access, GenAI literacies and skills, perceptions of use and usefulness, and motivations and barriers for when they choose to (not) use GenAI. We will present findings of the whole sample, as well specific equity groups who may have experienced disadvantage or under-representation. For example, preliminary analysis already indicates significant differences in the ways in which international students use and trust generative AI. The findings will provide an evidence base to inform policies and practices that leverage GenAI's benefits while mitigating its risks.