Measuring and Developing Digitally Adept Students with Assurance of Learning (AoL) Rubric

Abstract

This presentation provides an overview of our innovative and comprehensive process for embedding student digital and professional capabilities and assuring the learning of these capabilities for our Business School’s accreditation and quality assurance purposes.  To avoid surface level compliance and ensure all academics were engaged in the change process, we remapped our Program Learning Outcomes to key Competency Goals that met the strategic direction of our Business School, and the University more broadly.

Five Competency Goals were developed: Global Citizenship; Ethical Reasoning; Analysis and Problem Solving; Job Ready; Digitally Adept. These were divided into 12 Objectives and underpinned by a suite of carefully designed rubrics to be workshopped and formalised as Assurance of Learning (AoL) Rubrics (Calma, 2021). We included our Business School’s commitment to the Principles of Responsible Management Education (PRME) and ensured the United Nations, Sustainable Development Goals (SDGs) were met in our core competency areas and learning objectives.  Because the process followed the strategic direction of our business school, this process gave us a vehicle to showcase our priority areas, which included the innovative approaches taken by academics to enhance the Digitally Adept competency goal in student learning and assessment. An important aspect of the formalised process was the development of a digital Data Extraction Tool (rDET), developed by Ishpal Sandhu and Gillian Vesty, which allowed us to extract components of a rubric that matched each of the 5 competency goals and 12 learning objectives, not only for our AoL sample, but for all students in a course, regardless of course numbers.  This automated data collection tool has made it possible to easily capture a wide range of data to reveal distinct areas for improvement in terms of curriculum alignment, assessment design and/or changes that can be made to enhance student experience while getting them ready for life and work.

In a multi-dimensional constructive alignment process (Sridharan et al., 2015), the team validated the Competency Goals and Objectives, ensuring they were captured in assessments across all programs, and in all delivery locations. The suite of formal rubrics developed to measure the competency goals became the centrepiece of conversation among the program management teams. To ensure AoL sampling was a true representation of the practices across the Business School, all academics were asked to include the rubrics in their assessments. 

Importantly, the alignment process undertaken was embedded in formal governance mechanisms and reporting through the hierarchy of committees. The rubrics and template developed was the mechanism to expose current practice gaps and/or achievements of each of the digitally adept learning objectives and competency goals. Recommendations for change in process and/or curriculum were made, and appropriate resourcing provided on agreement. It was important that successful achievement of the digitally adept learning objectives showcased the move away from exams to exemplary authentic learning assessment designs.