Online Assessment in Australian University Business Schools: A Snapshot of Usage and Challenges

Whilst most institutions have been using online assessment submission for many decades, the move to fully online delivery during the COVID-19 pandemic meant more traditional forms of assessment also had to move online. In this national study we captured a snapshot of online assessment usage across business disciplines from 97 survey participants from universities in Australia and identified the challenges reported by participants. We found the three most predominant forms of online assessments are written reports and essays, online exams/quizzes and live or recorded presentations. We categorised the reported challenges into 13 groups, and they include academic integrity; additional time and effort for teachers; technology access and service consistency; student preferences and expectations; and changes to feedback. We conclude with a discussion on these challenges. This study contributes not only a better understanding of the usage of online assessment in Australian business education, but also provides a benchmark with which to track and evaluate future shifts in assessment practice.


Introduction
At the beginning of the COVID-19 pandemic, most institutions within the Australian higher education sector switched to emergency remote teaching (Seraj et al., 2022) in order to continue operating. Prior to 2020 fully online delivery comprised a minority of the higher education deliveries in Australia for reasons including visa conditions for onshore international students that restricted online delivery to one-third of teaching in a course (National Code of Practice for Providers of Education and Training to Overseas Students, 2018).
The COVID-19 situation has been transformative for Australian university business programs catering to international students, given these students were unable to travel and enter Australia during 2020-2021. The impact of the pandemic on international student numbers and the flow-on effects to institutional funding has been far reaching particularly in business disciplines since they have some of the highest international student enrolments across the sector (Zhang et al., 2016). As a result, business disciplines had to considerably change their delivery models while progressing from remote emergency teaching to online delivery.
There has been considerable attention directed at pedagogical perspectives of online assessment such as the ability to provide immediate and meaningful feedback (Mate & Weidenhofer, 2021) and the implications of online assessment for academic integrity (e.g., Reedy et al., 2021). In addition to its advantages, the use of online assessment brings practical challenges, for example automated marking can provide immediate feedback but may also be less specific and personalised to the individual student and thus less useful for formative assessment, and remote invigilation creates challenges for student authentication and academic integrity (Mate & Weidenhofer, 2021). In response, a range of strategies may be applied to mitigate these challenges, for example, the provision of authentic assessment to inhibit plagiarism or contracting out, use of a wider variety of performance-based assessments where the identity of the student can be directly verified, adoption of technology-driven remote invigilation of online examinations, as well as feedback and feedforward. At the same time, it is acknowledged that further work is needed and a report on the future of assessment by Pauli and Ferrell (2020) identifies five key five-year targets for online assessment to support student learning and staff confidence: improved authenticity, accessibility, automation, continuous improvement and security.
Business educators in Australia have recently been responding to the unfolding COVID-19 related learning circumstances, and many have been required to design, implement and refine new online assessment practices.
However, there is little information available concerning what types of online assessment have been adopted in Australian university business programs, the level of innovation in these assessments or the practical challenges faced when designing and using online assessment. This research study seeks to provide evidence concerning these issues in relation to the forms of online assessment in use within Australian business school programs.
Research questions: 1. What forms of online assessment are currently being used in Australian university business programs? 2. What challenges do academics and other educators face when designing and implementing online assessment in Australian university business programs?
This snapshot of online assessment practices contributes two-fold. First, it helps us understand the educational practices in use during and after the COVID-19 pivot to online assessment delivery and the challenges educators face in delivering and designing online assessments. Second, the snapshot of current practice can assist with the evaluation of the future sector-wide efforts to improve and innovate online assessment practices.

Method
The study was approved by the University of Sydney Human Research Ethics Committee. An online survey developed from a comprehensive literature review (Brodzeli, 2022) was used to collect data from academics involved in educational delivery and/or design within Australian university business programs. The survey asked participants to share their online assessment practices and to provide additional detail about one specific assessment including a description of the challenges they had experienced with that assessment.
The survey was distributed via the Australian Business Deans Council (ABDC). An email including an invitation to complete the survey was sent from the ABDC to member institutions with a request to cascade the survey to their staff. A follow-up reminder was sent approximately six weeks later. The survey invitation and link were also shared through social media channels Twitter and LinkedIn, with appropriate professional associations tagged to these posts.
To document the forms of online assessment currently being used by respondents, descriptive statistical analysis was conducted on the survey responses concerning assessment type, individual/group modality and invigilation method. The Chi-square test of independence was used to examine the relationships between assessment type, professional accreditation status, weighting, program level and cohort size. The free-text responses were initially coded by one research team member to identify challenges for online assessment and the coding system was independently applied by a second researcher with any differences resolved through consensus.

Results and discussion
A total of 97 people completed the survey. Respondents identified multiple roles and affiliations within Australian business schools. Discipline affiliations were business (n=31), accounting (n=29), management (n=17), finance (n=15), human resources (n=12), marketing (n=11), economics (n=10), business law (n=8), innovation and entrepreneurship (n=8), information systems (n=7), business analytics (n=6), actuarial studies (n=2), property (n=2), and one respondent each for financial planning, IT/engineering, law, leadership, risk management, tax, and tourism and event management. A majority of the respondents held a role as a unit coordinator (n=85) and/or lecturer (n= 69), with other roles including program coordinators (n=36), tutors (n=32) and education developers/learning designers (n=15). There was one associate dean and one head of school, both of whom also coordinated units. For more detail about the survey sample see Huber et al. (2022).
Respondents were asked whether their courses were accredited by a professional membership body. Around two-thirds (n=64) reported that the courses they worked on were professionally accredited, 18 respondents reported that they worked on courses that were not professionally accredited, and a further 15 respondents were unsure of the accreditation status.

Forms of online assessment in use
Participants were asked to report all forms of online assessment that they had used. Table 1 shows the extent each type of online assessment was reported in the sample. Almost all respondents reported using written assessments and/or online exams/quizzes. Around two-thirds of respondents were using some form of presentation, either performed live or as pre-recorded/digitally-created multi-media. Within this group, 53 respondents were using live presentations and 50 respondents were using recorded/multi-media presentations. Close to a third were using participation as an assessment, with most of these respondents using in-class participation and a smaller proportion assessing students' participation outside class times. Online discussions, reflective journals and self/peer-assessment had similar levels of use at just under a third of respondents. Portfolios and interactive simulations/cases/games were each used by around one-sixth of respondents. Design or creative works, laboratory/practical assessments and online self-guided internships were used less frequently (see Table 1). Notably, the categories presented in the survey appear to be comprehensive, with only one participant selecting 'other' to record an online self-guided internship. Participation (in-class and out-of-class) 32 (33) In-class participation 31 (32) Out-of-class participation 9 (9) Online discussion 29 (30) Reflective journal 28 (29) Self/peer assessment 28 (29) Portfolio 16 (16) Simulation, interactive case or serious game 16 (16) Design product or creative work 8 (8) Laboratory/practical assessment 4 (4) Other (Online Self-Guided Internship) 1 (1) Note: Respondents were asked to report all forms of online assessment they had used

Use of individual and group assessments
Respondents indicated whether they used each form of online assessment as a group or an individual assessment, or in both modes (see Figure 1). Some forms of online assessment were substantially more likely to be used as individual rather than group assessments. These included online exams/quizzes, reflective journals, portfolios, and both in-class and out-of-class participation. Online discussions and written assessments were also more commonly used as individual assessments rather than group, although this preference was less pronounced and group modes were often used. The other forms of assessment were more balanced between individual and group approaches. These included recorded/multi-media assessment, live oral presentations, self/peer assessment and simulation/interactive cases/games.
These results raise the question of why some types of online assessment are much more likely to be set as individual assessments compared with other types of assessment. Considering the many permutations of group assessment available (e.g., Davies, 2009) and the recognized importance of improving student preparation and motivation for working in groups (e.g., Peklaj & Levpušček, 2006), this may indicate opportunities for expanding innovation with these assessments into group modes. On the face of it there are obvious reasons why reflective journals and portfolios are used to assess individual learning activity, however there may also be further learning opportunities for more collaborative reflection-based assessments and portfolios that could enhance skills in giving and receiving peer feedback while managing staff workload (e.g., Uijl & Filius, 2022). In relation to exams, Villarroel et al. (2020) argue that active learning is promoted when small groups are asked to respond to exam-based problems designed to be authentic and to assess higher-order learning, as students must engage with each other to present and evaluate arguments. Notably, such assessments develop a range of skills in collaboration, communication and negotiation valued in most workplaces. Further analysis into the selection of group or individual assessment modes for online assessment could also highlight the distinction between assessments that are completed and assessed in groups, and assessments that are completed in groups but individually graded (Davies, 2009;Rico-Juan, 2021).

Figure 1: Use of Assessment types as Individual or Group Assessments
Note. Forms of assessment with fewer than 10 respondents are not displayed in Figure 1.

Invigilation methods for online exams and quizzes
The survey asked respondents who had used online exams/quizzes (n=85) for further details about the question types and invigilation methods. Respondents were able to select multiple answers. Table 2 presents these results, alongside an analysis of invigilation types according to whether the program was professionally accredited. Regarding the relationship of invigilation method and question type, the data indicates that human invigilation was used with similar prevalence across all types of online exam/quiz. Also, there were similar patterns in invigilation method for numerical calculations and multiple-choice questions. On the other hand, extended written answers/essays had relatively higher prevalence of take-home (non-invigilated) approaches and less frequently used computer invigilated approaches. This may point to a lack of access to fit-for-purpose computer invigilation tools for extended written assessments. Emerging tools such as Cadmus (https://www.cadmus.io/) may fill this requirement.  The analysis of invigilation method and professional accreditation status indicates a similar pattern of people and computer based approaches across professionally-accredited and non-accredited programs. However there is a contrast in the prevalence of non-invigilated approaches, with accredited programs more often using scheduled timed sessions and non-accredited programs relatively favouring take-home exams.

Relationship of assessment type to accreditation, assessment weighting, course level and size of cohort
Respondents were asked to nominate one specific online assessment and provide additional detail about the design and implementation of that assessment. This allowed for more granular data as participants were considering the design of a particular assessment rather than their assessment practice in general.
Statistical tests were used to identify whether there was a relationship between the assessment type and (a) whether it was required for accreditation, (b) the assessment weighting, (c) the course level (undergraduate vs postgraduate) and (d) the size of the cohort. As some forms of assessment were rarely nominated by respondents, three categories were used: online exams/quizzes, written assessments, and 'other' (which included all other forms of assessment such as participation, live/recorded performances and portfolios). The only significant relationships found were between the type of assessment and assessment weighting and cohort size (see Table 3). Exams/quizzes and written assignments were more likely to be used for high-stakes (over 30% weighted) assessment compared with other forms of assessment such as presentations and participation. Regarding cohort size, most of the 'other' assessments were being used with cohorts of fewer than 100 students, while exams/quizzes and written assessments were used more consistently regardless of cohort size. Significant relationships were not found between assessment type and either accreditation requirement or course level.
These findings may both be explained in relation to resource requirements. The 'other' assessments category defined here includes a range of assessments that do not lend themselves to ease of standardisation or to automated marking and feedback, such as live or multimedia presentation, participation and reflective journals that must be marked by educators and are thus relatively resource-intensive, particularly as markers must be trained to achieve reliable marking. Thus, these resource-intensive assessments may be selected for smaller cohorts, and for lower-weighted assessments where concerns about marking consistency may be lower.

Challenges to implementing innovative online assessments
One of the key findings from the survey was that the most commonly used online assessments are traditional quizzes, exams and written assessments (see Table 1) where much has been written about translating these assessments to the online environment (Apps et al., 2020;Butler-Henderson & Crawford, 2020;Cramp et al., 2019). To explore the challenges of online assessments for educators we asked participants to reflect on a specific assessment that they had translated or designed for online delivery and 50 respondents described one or more challenges they had faced with their selected online assessment. Analysis of these responses identified 13 challenges, listed in Table 4, with some responses coded to multiple categories. The categories are described below with illustrative quotes. As many of the challenges were contextual to the type of online assessment being implemented, information about the type of assessment is also included.

Academic integrity
The challenges of academic integrity within online assessment are well documented (e.g. Holden, Norris & Kuhlmeier, 2021). Respondents described identity verification 'It is difficult to guarantee (100%) that the student is the one who completes the assessment.'written assessment, and collusion 'Student collusion is very difficult to manage.'online exam/quiz, as specific challenges, as well as highlighting the difficulty of conducting remote invigilation 'Given the online environment validating business [organisations] used for the group assignment outside of Australia has been a challenge.' -written assessment. The challenge of academic integrity was well-represented in this survey with 16 respondents commenting on it, however it was only raised in relation to online exams/quizzes and written assessments.

Additional teacher time and effort
The respondents described a multitude of ways in which online assessment could result in increased workload or resourcing for teachers. This included additional time for question development, marking, technical set-up of the assessment 'Technologies not working appropriately for large classes… Increased workload not accounted for in workload considerations.'written assessment, and responding to students' reported technical issues and appeals 'Student claims of technical difficulties that do not require any proof leading to further assessments that extend the semester and increase academic workload.'online exam/quiz. Some respondents commented that their workload had increased in some areas, but decreased in others 'Requires a lot of work by assessment writer (coordinator) but substantially reduces end of semester stress and burden across the teaching team.'online exam/quiz. Twelve respondents described this as a challenge which goes against the belief that moving assessment online is more efficient (Arney et al., 2012).

Technology access and service consistency
Challenges relating to the availability and stability of technology were frequently mentioned (11 respondents). Unreliable internet for students and limited access to appropriate hardware were the most common explanations identified for these issues. Some respondents noted ways of responding to this challenge, including rescheduling live presentations and functionality that allows auto-submit of answers. This challenge was mostly raised in relation to synchronous assessments 'Highly dependent on technology' -live oral; 'Not all students have access to appropriate hardware or reliable internet coverage.'online exam/quiz, however was also reported for asynchronous assessments 'Upload times to the LMS for submission sometimes cause issues as well.'recorded/multi-media assessment.
Technology functionality and usability Seven respondents described challenges relating to the use of the technology (Mate & Weidenhofer, 2022), including whether the functionality provided was appropriate 'Technologies not working appropriately for large classes.'written assessment, and challenges around teachers and students learning how to use the systems 'Some students struggled with the technical ability required to record a video.'recorded/multi-media assessment. In some cases this resulted in downstream challenges for academics including issues around academic integrity 'IT issues where the student has completed some of the assessment and then is locked out of the assessment -not a good student experience but good for academic integrity'online exam/quiz, and the need to allocate additional teacher time/effort to the assessment Host software platform (Mahara) does not integrate properly with marking platform (Moodle) so when students 'submit' a portfolio in Mahara, it's locked off from being able to be graded in Moodle. Constant source of pain in the submission window of having to unlock/reject submissions in Mahara, email students new instructions, students panicking and repeating the same path that didn't submit.reflective journal.

Student preferences and expectations
Five respondents noted that the shift to more innovative assessments can require students to perform in ways they find uncomfortable compared with more traditional forms of assessment (Sokhanvar et al., 2021). For example, some students reportedly preferred assessment methods with pen and paper 'Some students would rather do exams with pen and paper than on a computer. Pen and paper is not authentic as they wouldn't do this in the workplace. With a computer-based exam they can use the tools they would use in the profession such as Excel and searching Accounting Standards online.'online exam/quiz, or that align with rote learning 'Students generally like the Discussion Board activity but some would prefer assessment where they regurgitate the text book.'online discussion. Online assessment modes may increase stress for students who are less comfortable with an assessment, particularly when it relates to culturally or otherwise socially sensitive topics 'Requiring students to debate issues that might include culture or social-economic sensitivity will inevitably place stress on students. However, this is appropriate for a capstone subject.'recorded/multi-media assessment.

Logistics and timing
Five respondents who were describing live oral assessments noted challenges related to timing 'We can't have a long simulated meeting due to time pressures.', -scheduling/rescheduling. 'The larger the cohort the more stimulus pieces are needed and the logistics of arranging the presentations timing.' and other logistical issues including challenges for students to manage their own logistics 'Hard for students who share accommodation.' This challenge was only raised in relation to live oral assessments.
Changes to how feedback is provided Four respondents described challenges relating to the provision of feedback in online assessment (Arney et al., 2012). These were driven by changes to teaching processes that occurred as a result of adopting online feedback rather than by difficulties in providing effective feedback per se 'Having this assessment task online has meant that the feedback can be provided to students more rapidly than offline assessment. However, that does mean that marking and feedback time for academic staff is spread across multiple submissions.'reflective journal.
Differentiating students against the criteria when grading Three respondents reported challenges with distinguishing students at different levels of performance when grading online assessments 'Finding ways to distinguish high-achieving students due to the open book nature of online assessment.'online exam/quiz. This challenge can also impact students, who likewise need to be able understand the characteristics of good performance with reference to the criteria (Boud et al., 2018) 'Oral presentations are not always black & white in terms of right & wrong, so its a bit harder for students to digest why they are wrong (despite the volume of feedback provided)' -Recorded/multi-media assessment.

Students do not demonstrate all assessment criteria
Another challenge was that educators considered that students were often not demonstrating all of the assessment criteria in their assessment submissions. The reasons for this were varied, including time constraints making demonstration of complex skills difficult 'The challenge is that we can't have a long simulated meeting due to time pressures. This means students struggle to display all their interdisciplinary skills in the time frame.' live oral assessment; and engaging the teaching team with the deeper pedagogical aspects of the assessment (noted by three respondents).
Preparing students for the format Innovative assessments can require a shift in mindset or further capability development for students (Cramp et al., 2019), and three respondents described the preparation of students for the online assessment as a challenge 'Online presentations are relatively new to students'live oral. One respondent noted that this can also have unintended benefits for students 'Students with limited English sometimes struggle a bit to start with, but the ongoing practice actually works in their favour by providing regular practice in written English over the semester.'online discussion.
Getting students to take low-stakes assessments seriously A few respondents commented that it was challenging to engage students in low-stakes assessments, such as low-grade discussion contributions and optional peer-feedback.
Some students don't take it seriously-until they receive their first round of feedback in Week 4 of the 12 Week semester, and then they sit up and pay attention. In fairness, the first 4 weeks are accordingly down-weighted compared with the following 8 weeks.online discussion.

Designing assessments that align with all criteria
A few respondents reported that moving exams/quizzes online meant that they needed to write questions differently and that it was a challenge to align the questions with the learning outcomes they wanted students to demonstrate 'It took some time to develop high quality questions and ensure fairness to the students in terms of whether they had truly encountered the material being tested'online exam/quiz. Some teaching staff don't recognise the integration of functions that is integral to the relevance of this assessment task. They see it as a weekly summative requirement to "spit out what you have learned this week" of dubious relevance, rather than as formative assessment that operated on multiple levels.online discussion.
Institutional/departmental support A few respondents described a lack of support from the department or central university services, which compounded challenges they were having with the online assessments 'Little support from department with designing and implementing anything.'recorded/multi-media assessment; 'Lack of support from Uni central support sections.'written assessment. This aligns with the findings of Cramp et al. (2019).

Institutional policy
One respondent reported that an institutional policy around the use of online exam proctoring restricted their options for designing mid-term assessments and generated challenges in aligning the assessment to what they wanted to assess.
My institution uses proctoring for final exams but not for the mid-term assessment described herein. Thus the assessment is effectively open book […] the inability to effectively restrict access to other materials limits the range of suitable questions. It means I need to use more application and am less able to allocate some marks to demonstration of knowledge.online exam/quiz.

Limitations
The survey was distributed to Australian university business faculty staff involved in the delivery and/or design of online assessment and attracted 97 respondents who were primarily working in universities. While this included people from across the range of business disciplines, there was a higher proportional representation of some disciplines. The sample therefore had some underrepresentation of non-university business education and of some business disciplines, which leaves open the possibility that some areas within the business education community have a substantially different profile of online assessment usage and challenges.
The analysis and identification of challenges was based on each respondent selecting only one online assessment for more in-depth description. This approach will have identified many of the challenges that respondents had at top of mind, but we do not claim that this list of challenges has reached saturation and includes all challenges that business academics may experience. More targeted data collection on the challenges of implementing innovative online assessments may reveal other challenges. Also, the identified challenges relate to online assessments that have already been implemented, and there may be other perceived challenges that inhibit business academics from implementing innovative online assessment in the first place. Finally, the survey included staff only, and although the explanations for challenges often referred to aspects of the student experience, the student perspective would be better addressed directly with a student sample.

Conclusions
This study has identified the range of online assessment practices in use in Australian business education through the pandemic years and the challenges faced by academics in implementing them. These challenges are unlikely to be limited to business disciplines but rather to be common across other disciplines.
Our survey indicates that traditional forms of assessment continue to dominate the assessment regime, with exams, quizzes, written assessments and to a lesser extent live and recorded presentations comprising the majority of assessments reported in this study. Despite this, it is also clear that innovation has been occurring as evidenced by the widespread use of forms of assessment such as online reflective journals, online discussions, self/peer assessment, portfolios, and simulations or games. The results also indicate that this innovation is predominantly taking place in smaller cohorts and with a focus on lower-weighted assessments. Further work may be needed across the sector to address the additional complexities of innovating online assessments within large scale cohorts.
Based on our findings on invigilation of exams and other assessments, we note the importance of shifting from pure invigilation to identity verification. 'At home' invigilation methods through use of cameras can create privacy issues and reduce the student experience (Milone et al., 2017;Huber et al., 2022). Invigilation concerns can be addressed if further support is provided to staff to rethink assessment and the need for invigilation by considering alternative forms of assessment such as performance-type approaches, whereby the student's identity can be verified. If we must stay with online exams then work is needed on reducing the student and staff cognitive load by offering practice opportunities before the exam is run (Cramp et al., 2019).
This study contributes not only a better understanding of the usage of online assessment after the temporary adoption of emergency remote teaching in Australian business education, but also provides a benchmark with which to track and evaluate future shifts in assessment practice. The links between student employability and authentic assessment are becoming more evident (e.g. Sokhanvar et al., 2021) and a further shift away from traditional exams, quizzes and written assessments towards more performative and reflective forms of assessment can be expected in the future.
The findings also suggest there may be further scope for innovation in the use of group and collaborative modalities for some forms of assessment, particularly exams, quizzes, reflective journals and portfolios. Few respondents using these forms of assessment reported using group modes, and while there are obvious reasons why individual assessment might be prioritised with these assessments, the findings also prompt the question of whether additional student learning and assessment opportunities could be realised with deeper exploration of collaboration. We concur with Villerroel et al. (2020) that the use of group assessments for non-invigilated, lowstakes assessments such as quizzes may improve learning outcomes while reducing the risk of normalising academic misconduct.
The analysis of challenges to implementing online assessment provides insight into some of the areas that need ongoing focused attention in order to better facilitate assessment innovation. These challenges highlight the integrated nature of assessment design for online delivery, with overlapping relationships between challenges of alignment of assessment and learning, change management, online technologies, institutional policy and support, and the distributed nature of contemporary teaching practice. Some challenges were notably more common for some assessment types, with timing and logistics particularly impacting live oral presentations and academic integrity concerns being emphasized for exams, quizzes and written assessment and less for other forms of assessment.
Participants in our study submitted their innovative assessment practices via the survey and these have been transposed onto the project website https://bizonlineassessment.com/. Many of these examples can be used as solutions to overcome the challenges identified in this study.