Lightweight Mapping of Identify Verification Methods and Secondary Course Aspects: “Swiss Cheese” Modelling

Curriculum mapping in Australia has grown from early notions of constructive alignment to a governed ongoing commitment by institutions to the curriculum structures they have developed. These consider primary aspects of a degree: learning outcomes and how they are achieved. In this paper I describe the need for a descriptive set of mappings for secondary (or quality) aspects of a degree. Notably, mapping the identity verified assessment mechanisms used across a degree can support conversations with accreditors and reviewers around defences against cheating. In 2020 and 2021, many universities were forced to cancel paper-based examinations and academics altered their assessment mechanisms. Mapping the mechanisms that were adopted can reconnect those individual choices with their purpose within the degree. The paper is an experience report from the development of an accreditation submission for a suite of computer science cognate degrees. Other descriptive mappings developed within the submission are also described.


Introduction
In Australia, course and curriculum mapping developed from Biggs's notion of constructive alignmenta learning design approach of identifying specific learning outcomes and then constructively designing the learning activities to meet those objectives (Biggs 1996). This developed from Cohen's (1987) earlier work on "instructional alignment"the match between curriculum and assessment methodsthough commentary on producing visual maps of curricula as part of the planning process can be found from the 1970s (Hausmann, 1974). This approach to curriculum design has collaborative benefits, as the mapping process fosters conversation between colleagues about degree content (Uchiyama & Rudin, 2009). Weingards-de Meij and Merx (2018) found that online mapping improves the visibility of this exercise, improving collaboration.
Curriculum maps can be used in a variety of ways. Veltri et al. (2015) propose using the curriculum map as an instrument for improving Information Systems courses by measuring aspects such as the relative weighting of different outcomes within the assessment in a subject, the depth to which an outcome is covered, and the saturation of coverage of an outcome by multiple units within a degree. Rawle et al. (2017) found that there are differences between disciplines within an institution in how curriculum maps are developed and how they are used by academics. Recently, Kay et al. (2022) suggest the incorporation of mapping into an open-learner model -a data representation of a student's progress against a hierarchy of objectives, from analytics of their activities.
Curriculum maps have become part of the governance processes of universities. For example, by 2008, most undergraduate medical schools in Canada and the UK had begun producing curriculum maps of their courses (Willet, 2008). In 2022, many Australian universities use curriculum management systems, e.g. CourseLoop. which can hold curriculum maps against specified standards. These maps become a governed commitment to the curriculum structure that has been developed. Changes may require several committee stages of approval.
The governed maps typically cover the primary aspects of a course: the learning outcomes and how they are achieved. In this paper, I suggest there is also a need to produce descriptive maps that can describe secondary aspects of a course at a moment in time. Particularly, consider the question "Which forms of identity verification in assessment (IVA) are used in subjects across a degree?" In 2020 and 2021, like many institutions, the University of New England (UNE) lost the capacity to offer supervised paper-based examinations. Computer science academics reacted rapidly by adopting alternative assessment mechanisms. In 2022, how should the degrees' IVA strategies be described to an accreditor? Mapping the IVA strategies adopted in subjects across the degree can reconnect those individual decisions with their purpose within the degree structure. Descriptive mappings also enable quality questions to be discussed across the sweep of a degree: e.g., is there enough identity verification in the degree and are the methods sufficiently robust and diverse?
The paper is presented as an experience report and case study from a particular context: the need to generate mappings and structural diagrams for our 2022 accreditation submission to the Australian Computer Society (ACS). Secondary mappings were also necessary for some other factors. For example, identifying which topics within non-mandatory alternative bodies of knowledge are currently covered within our degrees.

Mapping Identity Verification as a "Swiss Cheese" Model of Mitigation
In 2020, like many institutions, UNE lost the ability to offer supervised paper-based examinations but was able to offer online proctored examinations through a third-party proctoring provider. Because data sharing with the proctoring company was not part of students' agreement with the university at the time of enrolment, university policy had to permit students to opt out of online proctoring if they did not accept this data sharing. Many computer science students opted out, frequently objecting to the requirement to install proctoring software that they viewed as spyware or to clauses in the proctoring vendor's Terms of Service or Privacy Policy. In some computer science subjects in 2020 that retained proctored examinations, opt-out rates were above 50%.
Some academics saw an inequity in the fact that any student who wished to avoid being proctored could decide to object to data sharing. Most computer science subjects dropped proctored exams and focused solely on the alternative assessment that students opting out of a proctored exam would take. For example, some academics asked students to produce video talk-throughs of programs they had produced or video reflections on test answers, so that part of a student's submission would include them discussing their work in an identifiable manner. Others took approaches of personalising assessment. Over 2020 and 2021, the approaches to mitigating and preventing academic misconduct between students rapidly diversified. As this occurred by force majeure, it occurred on an subject-by-subject basis as academics made their own choices.
Within the university, policy and processes around academic integrity and how students apply for an exemption from online proctoring have evolved. However, these institutional approaches focus on the broad institutional context, rather than discipline-specific matters, such as computer science's high opt-out rate from proctoring.
In late 2021 and early 2022, I faced the issue of compiling our 2022 accreditation submission to the Australian Computer Society, including addressing the clause "There will be mechanisms to address identity management in a virtual environment" (McDonald, 2021, p.17). Given that proctored examinations now only existed in a subset of subjects and given that students can, in policy, opt out of proctoring by rejecting the privacy agreement, how should I accurately describe and present our identity management strategy, to enable the conversations on identity management we need to have with our reviewers? It is no longer about the proctoring policy, but about the variety of mechanisms that academics have adopted within the subjects they coordinate.
The strategy I adopted was to create mappings of the mechanisms of IVA used in each subject across each degree. This is inspired by Reason's (1990) observation, when developing the Swiss Cheese model of accident causation, that each layer of defence is an imperfect barrier, which will prevent some forms of threat but is susceptible to others. By using a variety of mitigation approaches across the subjects within a degree, the range of techniques that a cheating student would need to employ across a degree becomes larger. I argue this makes it more difficult (or at least, more effort) to traverse a degree illicitly successfully. Past literature suggests that using varied or multi-layered approaches within a subject's assessment may help deter students from engaging in cheating (Baird & Clare, 2017;Rundle et al., 2020). Mapping those strategies across the degree enables a shift in perspective to seeing the degree, rather than an individual subject within it, as the item being defended against cheating. As it is the degree, not each subject, that is accredited, this seems appropriate.
In our submission, this is presented as a matrix of the identity verification or mitigation methods in each unit. An excerpt of one diagram is shown in Figure 1. This provides an at-a-glance summary of the current IVA strategy, to engage the accreditation panel in a conversation around the level of granularity at which this should be modelled and what breadth and depth of measures are deemed sufficient. The mechanisms were identified by examining the handbook and learning management system content for each subject, and then asking academics to review and amend the matrix that was generated. It forms a categorisation that is local to our discipline within our institution at a point in time: • Proctored exams • Plagiarism detection software, e.g. TurnItIn • Oral or live assessments, in which an academic interacts directly with the student • Video submissions, in which a student's face or voice appear • Personalised assessment. This may be through adapting the tasks or questions given to each student, or it may be inherent personalisation that occurs through the nature of the assessment work • Group work, which would require the complicity of the group rather than only an individual student • Data trails, such as version control logs of students' work in progress.
Some assessments may use multiple forms of mitigation: for example, a group software project that involves video demonstrations and talk-throughs of their work, with the software development captured in version control, and individual reflections on students' work in the project that can be subject to TurnItIn. However, to enable an at-a-glance map, we capture the data only at subject level.

Descriptive (Less Governed) Mappings
The primary aspect of this paper is the introduction of IVA mappings. Part of the rationale for choosing this approach, however, was that other accreditation aspects can also be answered using descriptive mappings.
For example, ACS's accreditation criteria (McDonald, 2021, p.19) identifies discipline specific knowledge areas as "those identified in international curricula, standards and other sources relevant to the program's domain," and states that "There is no requirement to adopt such a curriculum, but the program should be influenced by it". This is a looser relationship than a governed curriculum map. Rather than specifying coverage criteria, it asks how the course is influenced by an international curricula and there may be more than one to choose from. There can also be a different strength of relationship between different mappings. The specialist accreditation requirement for Data Science degrees requires that a program will "contain at least [one equivalent full-time year] of content drawn from an appropriate Data Science body of knowledge … compatible with CBoK." (McDonald, 2021, p.20). The ACS Core Body of Knowledge (CBoK) is the primary content specification courses must adhere to, but specialist courses are asked to draw enough content from "an appropriate" data science body of knowledge compatible with it.
Though not part of our accreditation submission, we have in the past been asked by stakeholders to account for the range of programming languages we use within a degree. This is another descriptive mapping rather than a governed specification: the programming language used does not always appear in the learning outcomes.
In each of these cases, the criteria suggest a quality judgment across the course: whether there is enough influence from the identified bodies of knowledge. This is similar to the Swiss Cheese modelling of IVA methods, in that we present a matrix across the degree, to enable reviewers to consider whether coverage is broad enough and strong enough.
The use of descriptive mappings also supports their use as a design tool, in keeping with the early uses of curriculum maps (Cohen, 1987) and their use in collaborative design (Uchiyama & Rudin, 2009). If maps are convenient and fast to update and visualise, they can be used as an interactive artifact in team discussions. Changes can be sketched out and the academic team can consider where it wishes to introduce new forms of IVA, programming languages, or secondary content.

Lightweight Production of Descriptive Mappings
A potential limitation of using a mappings approach is that it introduces the burden of creating and maintaining the additional mappings. This can be significant: for our accreditation, forty-two diagrams and mappings were created. As well as the descriptive mappings, this also included visualisations for curriculum structure and for CBoK coverage, that present an overview of each degree in a format that is familiar to the panel. To be able to use the mappings as a design tool after the accreditation (for considering changes), I also wanted them to be fast to update so that potential changes could be sketched out quickly and shown to team meetings.
With so many mappings or diagrams to produce, I needed to create an efficient means for their production and to ensure consistency in how a subject appears in multiple degrees. (The same subject cannot contain video assessment in one degree, but lack it in another.) As a lightweight mechanism for this, I used a static site generator designed for education (Billingsley, 2021) to produce the diagrams, as illustrated in Figure 2. This permitted the documentation lead to define category definitions and pages that would be produced. The site loaded brief degree ("course") and subject ("unit") definitions dynamically from two JavaScript files, and automatically rendered mappings and structural diagrams. This allowed other academics to edit their subject definition, hit reload in their browser, and all forty-two tables and diagrams would be updated consistently.

Figure 2: Producing secondary mappings and structural diagrams using a static site generator
Being code, the site is kept under version control and managed in a Git repository. However, as the subject definitions are loaded dynamically, it was possible to enable instant collaboration by cloning it into a shared folder (via OneDrive). Academics could make their changes without worrying about cloning or committing to the repository themselves. Periodically, I would commit the accrued changes from the team. Although the mappings tool itself is not the main contribution, it is described to demonstrate that it is feasible to produce and maintain many different mappings if they can be generated from easily-edited common data sources.

Conclusion
In our case study, the descriptive mappings filled their purpose. While the final version of the accreditation report is still with the accreditors, the documentation was used (and put on-screen by the panel in some panel sessions) and the overall quality of documentation was commended in the debriefing session. As a design team, it enabled us as a team to visualise and discuss the layout of our degrees. Since the submission, the ability to produce lightweight mappings has also helped me to sketch out and present to our team potential designs for future minors or majors.
The Swiss Cheese approach to mapping and describing the IVA strategy within the degree enabled conversations that otherwise it would have been difficult to hold: what is the IVA strategy for the degree, now that we cannot solely rely on proctored examinations? Policies and approaches to academic integrity are often set at an institutional level; the IVA mappings enabled us to hold discussions at the degree level. It also enabled a broader conversation to be held. Whereas the handbook only has one field for IVA (whether there is a supervised examination), the IVA mappings supported discussion of more diverse strategies.