Graduate attributes in action: How a university meta-model has been transferred to a high school context with implications for business settings

This paper addresses an opportunity to harmonise the development of graduate attributes both before and after university engagement. Foregrounding these qualities and capabilities in the assessment of student work brings an opportunity to consider an over-arching meta-model that may serve to harmonise many disparate general capability frameworks. The meta-model, Capability Results Inventory, referred to as CAPRI, emerged through the categorisation of assessment criteria and was then applied with subcategories to forty six degree courses. Implementations at university level and one high school using the same assessment software are described. The meta-model is then mapped for selected 21st Century skill frameworks often used in business contexts. The paper concludes that the examples of graduate attribute integration in both university and high school assessment could also be transferrable to employee performance and development.


Introduction
This paper builds on the authors previous work on graduate attributes implemented and assessed using software (Boud et.al 2015, Thompson, 2016, Thompson, Lawson, 2018), with the intention of encouraging the application of a graduate attribute meta-model to other contexts.The term graduate attributes is common in Australian universities but 'learner profiles' is a popular current term.Earlier terms include 'key skills' (Drew, Thorpe, & Bannister, 2002), 'generic attributes' (Wright, 1995), 'key competences' (Mayer, 1992), 'transferable skills' (Assiter, 1995), and the terms 'employability skills' and 'soft skills', commonly used in business (BIHEC, 2007, Vasilieva, 2022).However, these are not necessarily made explicit in the assessment of student work or employee performance and there is scant evidence of them being mapped or harmonised in meaningful ways.
Assessment and grading do not take place in a vacuum.Professional judgements about the quality of student work together with interpretations of such judgements are always made against some background framework or information.(Sadler, 2005 p.177)The frameworks that group assessment criteria for professional judgements in both education and business are many, complex and varied.The assessment criteria for tasks ought to be the conduit through which students gain an appreciation of their development rather than bypassed to look at an overall mark or grade.The wording of such criteria can often obscure in complicated rubrics the genuine developmental intent of a project or unit of study.Rowntree highlights the importance of assessment criteria in the first of 17 recommendations in his pivotal book 'Assessing Students: How Shall We Know Them?' (Rowntree, 1987): 1. Articulate as clearly as possible the criteria by which we assessthe aims and objectives we espouse, what qualities we look for in students, in general and in individual cases; let us strive to become more aware of our implicit assessment constructs, and constantly question why we value the qualities we do.(Rowntree, 1987 p.240) Foregrounding attributes through a meta-model coded at the assessment criteria level means that a simple overview can be illustrated whilst retaining the complexity and specificity in each diverse context.However, unless such a meta-model is understood and valued by students, employees, tutors and supervisors, learner profile interpretations may not be useful.There is growing evidence that a meta-model could be a useful harmonising agency in education, business and government: A worldwide scan of the trends in and pressures on qualifications found that governments and educators everywhere are scrambling to ensure that qualifications remain a trusted and useful currency, underpinned by standards, comparable, and useful for learners and employers alike.(Milligan et al., 2022 p.19) To illustrate this potential an Australian graduate attribute meta-model and current implementations are described, followed by a mapping of the meta-model for a selection of 21st Century skill frameworks.

Development of the meta-model: CAPRI (pronounced 'capree')
CAPRI is both a mnemonic for five capability categories and an acronym for Capability Assessment Progress Results Inventory.The development of the meta-model originated with the author's university teaching context and Director of Teaching and Learning role.The intention was to shift students' focus from marks for subjects to their development of capabilities or attributes, over time and across subject boundaries.The foregrounding of these capabilities in day-to-day marking rather than general statements in documentation led to the idea of grouping assessment criteria for many subjects across different degree programs.Five categories of criteria emerged as a meta-model that could then be appropriately sub-categorised for various degree contexts.Early implementation of this idea in a live teaching situation revealed the need for consistent and visual integration in assessment and feedback for subjects in a degree course.This led to the idea of colour-coding, alpha-numeric coding and symbol allocation to assessment criteria that could then be grouped into visual feedback for staff and students (Thompson, 2016).The author had designed assessment software called ReView to facilitate this visual approach that was later commercialised by the University (https://youtu.be/vR4OfCEVTK8).In 2016 this approach to assessment received a 'highly commended' from the ACODE and Pearson 'Innovation in Technology Enhanced Learning Awards' (https://youtu.be/ufJ55Wgh2YI).

CAPRI meta-model category descriptions
The five categories at the top level of the meta-model are defined broadly with titles that can vary whilst maintaining the mnemonic for the first letter of each category title C A P R I as follows:

 Innovation and Creativity
Associative colour and symbol ideas derived from the author's design masters and education PhD underpinned the correlation of visual feedback and capability development using the UTS ReView marking software.General descriptions and variation in the titles of the five categories have been used to guide the development of Course Intended Learning Outcomes and their associated assessment criteria at the University of Technology Sydney (Thompson, DG 2016 p.200).The titles and descriptions below were versions used as a starting point for degree program or school subject staff to subcategorise under the five colour-coded categories of the metamodel: (Colour -Yellow; Symbol -Pentagon).This category concerns the quality and clarity of such things as oral presentations, written essays, explanations, and visual presentations.

C -Communication and Collaboration
In addition, the development of communication, in-group interactions and various team roles.A -Attitudes and Values (Colour -Blue; Symbol -Circle).This category is to do with respect for one's own work and the work of others, including ethics.Developing care, understanding, and patience, with consideration for others' points of view.It covers aspects of integrity, honesty, accountability, diversity, and the acknowledgement of indigenous and multicultural perspectives.P -Practical and Professional (Colour -Red; Symbol -Square).This category of development involves technical skills, digital skills, financial skills, organisational skills and operational techniques together with the methods and experience / knowledge required to function as a professional in a broad range of environments.R -Research and Critical Thinking (Colour -White/Grey; Symbol -Triangle).This category of development involves fact-finding, literature surveys, research methods and the ability to think analytically.Also developing Critical Thinking and the ability to make informed criticism of one's own work and the work of others.I -Innovation and Creativity (Colour -Green; Symbol -Star).This category has to do with inventiveness, versatility, thoughtful risk-taking, imagination, creative concepts, innovative problem-solving, natural curiosity, creative experimentation, and the innovative application of technologies and processes.
In practice the subcategories were also labelled with a number code and a hyphen, e.g.C-1, C-2, A-1, P-4 etc to enable further complex data analysis.

Implementations of the CAPRI meta-model
The meta-model CAPRI and UTS ReView software aimed to foreground capability development in the day-today marking of student work, rather than relying on course guidelines or additional assessment processes.However, the focus on total marks and banded grades impacts students' approach to assessment very early in their educational engagement.It constitutes a major influence for both parent and students at high school and primary levels and is reinforced by media reporting and Government policy.Thirteen years exposure to this focus can often mean that marks-driven attitudes are difficult to dislodge in both staff and student approaches in both university and business contexts.The following brief descriptions serve to highlight the opportunities for change if a flexible meta-model is used that can then harmonise profiles across educational sectors.This would require Government policy levers and a shift from simplistic ranking processes.

High School Example: Research report
The Australian Government Australian Curriculum Assessment and Reporting Authority (ACARA) produced a general capability framework for high schools.Seven categories in addition to the eight subject-based Key Learning Areas added an extra layer of assessment to current practices and standardised tests were introduced to assess two of the seven (numeracy and literacy).A Government approved research project used the CAPRI meta-model and UTS ReView software with two New South Wales high schools in a pilot study to integrate rather than add-on capability assessment to current assessment practices.(The integration of capability assessment and reporting for Stage 4 student work in NSW public secondary schools.PDF available at https://app.education.nsw.gov.au/serap/ResearchRecord/Summary?id=209).This 2020 report included the following conclusions: This pilot study found that integrating general capability assessment and reporting with the marking of student work in Stage 4 Project Based Learning programs is possible with a suitable capability framework.However, the ACARA categorisation of capabilities was found to be confusing, inconsistent and impractical when integrated with assessment.In both schools there was improved teacher and student awareness of the role of student capability development as part of subject and project assessment.However, from the results it was evident that the integration rather than addition of capability assessment and reporting is vital.Adopting a process where teachers are expected to assess ACARA General Capabilities in parallel with traditional reporting is not viable.
Liverpool Boys High School in Sydney was awarded Australian Secondary School of the Year in 2019 largely because of its implementation of capability assessment using the CAPRI meta-model across the years from 7-10.This now includes an interpretation of CAPRI for the requirements of the High School Certificate (HSC) in years 11 and 12. Together with other certification the school now provides certificates for school-leavers with their individual CAPRI colour profile and feedback on performance.

University Example: University of Technology Sydney
CAPRI and the UTS ReView software were used to integrate graduate attributes into assessment for all degree programs in the Design, Architecture and Building Faculty.This amounted to rewriting Course Intended Learning Outcomes under the five CAPRI categories for 46 postgraduate and undergraduate degrees.The subjects in these courses were gradually linked to degree-specific subcategories of the CAPRI meta-model developed through workshops with Course Directors and teaching staff over a three-year transition period.
During this time the five meta-model categories were found to encompass all the Course Intended Learning Outcomes with no requirement for an extra category.The UTS ReView assessment software imported all degree, subject and task data daily to maintain accuracy with API's to Canvas and other University systems.Assessment criteria were coded to the CAPRI meta-model through linking to the new degree-specific Course Intended Learning Outcomes.The system was then used in the day-to-day marking of student tasks within subjects and enabled students to also self-assess using visual marking sliders (Thompson, 2016 p.210).Automatic visual feedback of criteria weighting in pie charts and bar charts allowed staff to reflect on whether those weightings matched the true intent of the task or project and led to many curriculum changes and refinements.One Unit Coordinator commented: 'You know I thought this subject was all about practical tax accounting skills but looking at the small red it looks like I'm focusing this subject on research… and there's no blue in there which is a bit of a worry !'. Students also had access to pie charts and bar charts and line grids showing their progressive development of learning outcomes and the five CAPRI meta-model categories.One student commented: 'My yellow is really big… actually over the whole degree I've got an HD average in yellow… will you sign this if I do a screenshot?'

CAPRI meta-model mapping to 21st Century skills frameworks
The following mapping process serves to indicate how some common frameworks used in both business and education contexts could be harmonised through the CAPRI meta-model.This is clearly a surface mapping approach and would need much more granularity of meaning through sub-categorisation.Again  the Attitudes and Values category is missing.This becomes obvious when colour coding is used.This is the category of the meta-model that relates to ethical approaches and sensitivity to sustainability issues.

Enterprise Skills CAPRI categories
Harvard professor in psychology Howard Gardner was an originator of the notion of multiple intelligences.The categories from his book Five Minds for the Future (Gardner, 2007) were not intended to be a model for education but they do suggest a 'meta' approach to the intelligences we need to encourage young people to develop.They also overtly include ethical approaches and further reading of the descriptions in his book relate very closely to the CAPRI meta-model categories that emerged through categorising assessment criteria:

Discussion
The main issue identified in the author's educational interventions that focus on graduate attributes and aim to reduce the focus on total marks and grades has been 'engagement'.This is not just engagement of students with their own development journey; or teachers' engagement with a shift from content delivery and memory testing; or governments' valuing something other than conflated rankings.These are major changes that relies upon parents seeing through the exam-based marks-driven rhetoric together with educational institutions' relegating their certification role in favour of a more developmental one.Society at large also seems to delight in memory tests and TV quizzes with large financial rewards when memory is one of the lowest functions of the brain.The default educational position to measure this as a valuable indicator of success is indefensible.The advent of AI and burgeoning increase in contract cheating, absenteeism and suicidal behaviours is testament to the need for refocusing on the attributes that education ought to espouse.Experiences from previous interventions have highlighted the need for a coordinated engagement strategy: Firstly, the design and implementation of communications that target students, academic teaching staff and other stakeholders explaining the benefits of assessment focused on student capabilities and outcomes rather than final marks and grades.Secondly, the revision of curriculum learning outcomes aligned through learning activities to assessment tasks that foreground graduated capability development.This would have to span subject boundaries to be relevant and consistent, using a meta-model such as CAPRI to code assessment criteria.The main learning mode has been subject based but tends to be most successful in project or problem-based learning contexts.Thirdly, the integration of software-enabled marking and feedback that makes visible the links to both general capabilities and learning outcomes whilst facilitating student self-assessment against related criteria.

Conclusions
The adoption of a attribute meta-model such as CAPRI could facilitate a shift from the focus on total marks, grades and rankings as the main indicator of value to one of a meaningful profile.It would be important however for this to be visual and continually updating according to the inputs of criteria-based assessments from both educational and business contexts.
Interventions to foreground the development of a meta-model of capabilities in the early stages of educational assessment could be a pivotal part of a strategy for real improvement.The three-part strategy outlined in the discussion section of this paper would need engagement by those in education and government with a vision to change and improve the current default assessment regimes.The provision of meaningful updatable attribute profiles instead of Grade Point Averages may inform students about their career or further study inclinations and give the business world less reason to complain about the mismatch between GPA and work-readiness.


Communication Attitudes and Values in the CAPRI meta-model is missing from the Enterprise Skills list.