Developing a micro-credential for Learning Designers: A Delphi Study

Teaching and learning online have increased as a consequence of the pandemic. There is an increasing need for learning design proficiency both in terms of new roles and also new expertise. In recognition of this, the Tertiary Education Commission in Aotearoa | New Zealand is supporting the design of a micro-credential to develop introductory knowledge in this field. Given the complexity and breadth of competencies needed in this role determining the foundational components is challenging. Prior research focused on professional standards and employer requirements only capture one perspective of the role. In this Delphi study, we sought the views of thirty-four learning designers to determine the essential competencies required for learning design in the current (post)pandemic context. Findings demonstrate the importance of understanding the nuances of different modes of learning, pedagogy before technology and the value of relationships.


Introduction
Over the last few years, there has been a dramatic uptake of digital learning. While the COVID-19 pandemic has exacerbated this rapid move to online learning, the adoption of learning facilitated by technology has steadily increased over the years. As a result, there has been growing interest in the role of the learning designer (LD) to facilitate and support educators to design effective online learning experiences (Slade, 2018). As a consequence of the pandemic, there has been a greater focus on online-only or high-end blended models of learning in all sectors of education (Heggart & Dickson-Deane, 2022). Support is therefore needed within all areas to support the effective development of learning, making learning design a critical component of many educational institutions.
The implications of the pandemic and the move to online or technology-facilitated learning, have been twofold. 1; An increased requirement for those in the learning design field, especially those with expertise in digital technologies and online spaces, to redesign and develop effective learning that can be undertaken online. 2; New roles and expertise are needed to support educators in designing and developing learning, especially when learning is undertaken online.
A search on Seek.co.nz for the term "online learning" in June 2022 resulted in 228 job vacancies. While this search uncovered a wide range of jobs, that include aspects of online learning, it does acknowledge the growing influence of online where aspects of learning happen online, either fully online or as a part of the learning process (e.g hybrid learning, hi-flex, technology-enhanced face-to-face learning).
The rapid adoption of technology in education, accelerated by the pandemic, has meant that educators have had to increasingly grapple with how technology can support learning. As the pedagogical approach of designing learning is fundamentally shaped by the mode (online, face-to-face, blended etc) in which learning happens, educators are requiring new competencies to adapt to different approaches. This has meant that while educators have needed to develop their own capabilities for teaching online there has also been an increased demand for new roles to provide additional support to these educators (often referred to as Subject Matter Experts (SME).
The outcome of this demand has meant a plethora of new roles and job titles that have arisen, over time, created to support the design and development of online learning. These titles have included, content developer, curriculum coordinator, curriculum developer, e-learning specialist, e-learning producer, instructional designer, and learning technologist (Sun, et al, 2018) but fall under the broad area of learning design (a term we will use ASCILITE 2022 The University of Sydney e22198-2 in this study to represent the broad area). Anecdotal experience, from talking to various sectors in our region, has highlighted the struggle employers are facing in seeking to fill these growing vacancies in the range of roles that cover these areas of learning design. Many providers have to hire and provide on-the-job training to upskill people outside the LD field. A review of where LDs come from, in the Australian sector, has highlighted the diversity of backgrounds and lack of formal training in this field (Sage & Sankey, 2021).
As a result, there is a current need for training in this area, to up-skilling educators to engage in this area and to train LD to work with SMEs to support and create learning content. The upskilling, in both areas, has led to the proposed development of a set of micro-credentials that could serve as a mechanism for those wanting to upskill in this area. Micro-credentials are increasingly being used as a way to support people to retrain or upskill to address skills shortages in the workforce. They provide a mechanism for learners to rapidly gain recognition for smaller components of learning that focus on skills or knowledge of particular importance to the workplace (Miller & Jorre de St Jorre, 2022). Though the adoption of micro-credentials within the high education has received some criticism (see Wheelahan & Moodie, 2021), it has continued to be promoted as a way to address societal issues, particularly employability and life-long learning-related agendas, and equity and social inclusion (Desmarchelier & Cary, 2022).
As one way to meet this need, the University of Canterbury, supported by the Tertiary Education Commission (TEC), has started the development of a micro-credential to provide an overview of the core skills, knowledge and abilities that are needed for designing digital learning experiences. While there has been a wide range of different competencies and frameworks developed (e.g. Aragon & Johnson, 2002;Piskurich & Sander, 1998;Arinto, 2013) the field of learning design is rapidly evolving.
A micro-credential would clearly not be able to cover this expanse. There are already entire books (for example Clark, 2021 andDalziel, 2015), and postgraduate qualifications devoted to becoming a LD. However, there clearly is a need for something in between being thrown in the deep end as a person new to designing digital learning and becoming an expert in the field.
To discover what current needs and gaps the micro-credential should focus on we adopted a Delphi technique to gather input from experts and colleagues in the field as our starting point. The purpose of adopting a Delphi technique was that it provided the opportunity to collaboratively involve a range of experts in the field (Okoli & Pawlowski, 2004). As the field of learning is broad, we wanted to gather a wide range of perceptions in a way that could also be consolidated. The adoption of this technique enabled us to gain expert opinions and build consensus to inform the development of curricula. This approach has been used extensively as a way to inform the development of curricula (Sitlington & Coetzer, 2015). The outcome of this process was to provide the first steps to the development of these courses. So, while the process would not provide the final design of these courses it would rather highlight the perceptions of the field as to what needs to be covered. This would be then used to support the design of the course, where this feedback would be used to design a cohesive set of courses that acknowledge the process of learning design.
In this article, we explore the outcomes of this Delphi study and propose the critical content, understandings and abilities that should be developed in a micro-credential to support those designing digital learning. As defined by MacLean and Scott (2011), these roles "require the application of theories of learning and instruction to the creation of learning material and the design of online learning experiences" (p557). We use the term LD to represent this broad field of learning design.

Methodology
The study adopted a web-based Delphi technique to address the focal question: What foundational skills and knowledge do LDs need in our current context? The study was undertaken in two iterations, the first was designed to gather opinions and the second version was focused on consolidating these opinions.
The surveys were designed to be undertaken anonymously utilizing the Qualtrics platform. The study was submitted and approved by the University of Canterbury Human Research Ethics Committee before the commencement of this study (2022/35/LR-PS).
Before the commencement of the study, invitations were circulated via email to experts that had already been involved in the initial consultation for these micro-credentials. These experts came from suitably qualified senior practitioners in the sector. We utilized our networks within the professional community from our own work experience. We also asked these experts to send the invitations to anyone they felt may be able to contribute to this process. While some researchers have advocated for rigorous guidelines for selecting experts (Okoli & Pawlowski, 2004), we decided to determine our expert panel more broadly whereby experts were determined by the individual's involvement in the work of learning design. This approach has been adopted in other studies such as Brill et al. (2006). In this study, we had a wide variety of participants from a variety of different roles in the learning design space. The majority (n=12) had titles including the term LD, including four having a role including the term head, manager or senior. The next most common were those with the title of Instructional Designer (n=5), with two including the title of either senior or head, and those having Curriculum Development or Education Specialist or Advisor their titles (n=5). Three participants had the title of Learning Technologist with seven having a variety of titles including Education Innovation Designer, Head of Digital skills, Online and Digital Learning Coordinator and Lecturer. Two participants did not include a title.
Of these participants, most were employed by a Government Education Provider (n= 26), while five were employed as Private Training Providers with only two employed in business. Most respondents came from New Zealand (n=14), but responses also came from the UK (n=7), Canada (n=5), Australia (n=3), USA (n=2) and one person from Nigeria and Burundi respectively. One person declined to answer these two questions.
There is little agreement in the literature about the size of an expert panel (Keeney et al., 2001). While numbers can vary between 10 and 40 participants (Sitlington & Coetzer, 2015), there is always some attrition when running multiple phases required for a Delphi Study. 34 people participated in round one and whilst 23 agreed to participate in round 2 only 17 people completed round 2. The survey ran over two weeks with each survey running over one week.
The first round of the Delphi study was designed to elicit feedback and brainstorm ideas. While the structure can differ our first survey included structured questions seeking the participants' perception of importance along with open-ended questions. This approach was important as it provided a structure to the study but also allowed for flexibility and spontaneous contributions from participants (Critcher and Gladstone, 1998). The survey was sectioned into three parts. In the first two; 1) the knowledge areas required by LDs, and 2) the skills that they need to develop the participants were given items drawn from the literature and asked to indicate the importance of these items using a Likert-scale. They were then asked to add any additional items they felt were important but had not been included in our initial lists. In the last part, we asked the participants to rate the pedagogical approaches or learning theories that LDs need to be aware of and draw into their design. We also asked if anything was missing from our original list.
In Round 2 we fed back the results of the Round 1 survey. In this phase we grouped the information and skills into topic areas of 1) Learning Design Concepts, 2) Assessment Approaches, 3) Learning Approaches 4) Research Concepts 5) People and project management. We then asked the panellists to indicate if these "Needed to be covered for understanding only", "Needed to be applied (demonstrated in the assessment for example)" or "were not relevant or would be covered later" (realizing that the three courses are only considered to be foundational). When categorizing these items into two groups (understanding/application) we asked them to rate these in order of importance. We did not ask participants to comment on the learning theories/pedagogies as we had already asked the participants to rate and we did not receive many additional areas.
The first and second round was undertaken anonymously, with the participant demographics captured in the invitation/recruitment survey not linked to their answers in the two rounds of the study. Therefore it was not possible to explore how their responses and perceptions may have varied across the different demographics, sectors and regions.

Findings of the Delphi study Round One Findings
The first survey focused on gathering the panel's perception of the most critical competencies that would be required for a foundational course (Table 1). The panel were asked to rate the importance of a range of skills and knowledge areas, that would cover a range of competencies, on a Likert scale of 1-5, with five being the most important. These items were drawn from a wide variety of sources including exploring job vacancies as well as key literature (e.g. Kang, & Ritzhaupt, 2015). The aim was not to provide the panel with an exclusive list but rather to provide a starting point for them to add their own ideas (table 2).
As identified by the panellists, the most important knowledge includes an understanding of the pedagogical application based on the context of learning, and how this differs between different modes e.g. online, face-to-face, mobile, and blended. This also included an understanding of different approaches to learning and learning theory to specific knowledge areas related to LD, such as the roles, constructive alignment and learning experience (LX). As for skills, the panel felt that assessment and constructive alignment was a critical skill along with general skills required by LD, including communication, presentation, evaluation and project management. Analysis of the open-ended responses from each section demonstrated that panellists found the distinction between knowledge and skills hard to make. This raised the question of whether knowledge and skills were too simple in their dichotomy as pointed out by one panel member "I think knowledge is only one part of it, need a way to develop practical skills"... 'beware over-emphasis on technology/ academic theory, instead ground in practical application". It was clear that our definition of skills were not helpful as we didn't intend to emphasize one over the other.
Additional areas were grouped thematically into areas related to learning design concepts and approaches and included items like, Māori pedagogies, copywriting skills with audience focus, accessibility flipped learning and new models like Hyflex learning. It also included items related to assessment approaches and include items such as authentic assessment, feedback and approaches like collaborative assessment, and research concepts and include items such as data analytics, evaluation and research skills (for evidence-based design/approaches). The panel also included more general skills that a LD would need to carry out their role and related to engaging with stakeholders and managing projects, items included relationship-building skills, change management and dealing with stakeholders.
The last section included a list of learning theories and learning design models which the panel were asked to rank from 1 -13 with 1 being the most important. The most popular learning design models were ADDIE (x̄=3.69, SD=3.14), Bloom's Taxonomy (x̄=4.72. SD=3.26), and TPACK and SAMR (x̄=5.97, SD=3.38) were rated the highest alongside learning theories, like constructivism and social constructivism (x̄=6.34, SD=3.73) and Vygotsky's Zone of Proximal Development and Scaffolding (x̄=6.44, SD=3.52). However additional, more contemporary theories and models, were mentioned by the panel and included Critical thinking/First principles thinking, humanizing online Learning, Community of inquiry, Networked learning, Relationship-based learning and Threshold concepts.
So, while many people were able to rank these it was clear that while some theories/models appeared lower in the ranked order that this did not mean they should be covered and also that these would not all be relevant in all contexts. As highlighted by one panel member, "Although a sound knowledge of theory is important it is more important to be able to read a situation and apply appropriate approaches. Models date so if I was designing ASCILITE 2022 The University of Sydney e22198-5

Round Two Findings
Based on the findings from round one we adopted the five categories identified from round one. For the collated list from Tables 1 and 2, the panel were asked to categorise if this needed to be covered for more understanding or application. The panel did not need to categorise all items and they needed to list them in order of importance (from most to least importance). Table 2 shows the outcome of this ranking and categorising. In order to identify the top items, we looked at the rating of each item as top 3 in each category and weighted them, a weighting of three for the first ranking, two for the second ranking and 1 for the third weighting these were then added together to get the top-rated items in each category.  (12) Online class dynamics and management (10) Asynchronous learning approaches (9) Active learning approaches (23) Learner engagement (13) Online class dynamics and management (12) Inclusive and accessible design (9) Asynchronous learning approaches (9) Research Concepts Audience identification (19) Data analytics (13) User evaluation methods (13) Research skills (for evidence-based design/approaches) (12) Evaluation, analysis and design critique (21) Understanding learner context (17) Research skills (for evidence-based design/approaches) (13) Designing and gathering user data (i.e focus groups, interviews etc) (10) People and project management.
Positive relationship and stakeholder management (22) The roles of LD and SME (and other stakeholders) (16) Change management (12) Collaboration and relationship building (10) Conflict management and negotiation (9) Effective communication strategies (25) Writing and presentation skills (14) Collaboration and relationship building (14) Project management (11) Note: Number in brackets indicate the weightings based on their ranked order of first, second or third place.

Pedagogy above technology
The most important aspect foregrounded by the panellists was an understanding of the nuances between different modes of learning. How does learning need to be designed differently for online/face-to-face, blended, and mobile? Whilst this has always been important (Kumar Basak, et al. (2018) this is even more critical in the (post) pandemic context as educators have to move rapidly and responsively between different modes. "Focus on course design rather than technology. A course design should be able to be applied to any modality. Technology should be transparent and used only to achieve the learning goals and objectives." Pedagogy was noted as more important than technology eg how to use various tools. As noted by a panellist "The tools and platforms are the least important in the skill set of a quality curriculum and learning designer. The skills to use tools are transferable and can be picked up easily." Skills, needed to focus on learning and not technology "practical skills relating to creating the better and aligned learning environment is important, more so than technical skills." and "Focus on course design rather than technology. A course design should be able to be applied to any modality. Technology should be transparent and used only to achieve the learning goals and objectives.".
Almost all panellists ranked designing learning outcomes as necessary, and in terms of assessment approachesconstructive alignment. These are core pedagogical competencies of any educator so it is interesting to think about the similarities and differences of these in a digital context.
Overall, the mainstream global LD approach ADDIE was the one most people felt was essential to cover. This concurs with the employer views where competencies related to analysis and design, particularly the ability to use ADDIE was one of the most frequently cited skills (Klein & Kelly, 2018). This is interesting as this model was not initially created as a learning design tool. In our bi-cultural contexts in Aotearoa New Zealand, we have a responsibility to uphold the Treaty | Te Tiriti. One does wonder how a framework created in the seventies in the US for military training purposes to do that?. (Bouchrika, 2020). One panellist did note the importance of more culturally relevant approaches but also acknowledged the complexity of doing this within the scope of a micro-credential around learning design. "while I think culturally responsive design and Māori pedagogies are extremely important, I'm not sure a micro-credential could cover enough to have people confident enough to be assessed on it (they could be micro-credentials on their own).".

Designing for engagement
Active learning design and approaches were rated highly in both rounds along with learner engagement in round 2. Clearly this is an emerging area as the additional contemporary theories highlighted by panellists included a focus on participation and collaboration eg Humanizing online learning, Community of inquiry, Networked learning, and Relationship-based learning. Whilst designing instructional activities is sometimes listed by professional organizations who publish design standards  it is not explicitly mentioned as a key competency in other research exploring this topic. This might be because the interrogation of key competencies for learning designers has mostly been through job postings (Kang & Ritzhaupt, 2015, Sun et.al. 2018, Wang et.al 2021. However, it could also be that in the current context where online learning has been the predominant form of education during the pandemic this has become an issue of increasing importance for learning designers, and educators. This also foregrounds the importance of peer-to-peer relationships and consciously creating opportunities for this to occur across multiple times and spaces. It also demonstrated the importance of newer learning theories that explicitly foreground social learning theory.

People relationships are above project management
Effective communication and positive relationships were rated more highly than skills in project management, Panellists confirmed that what was important was not just the what but the how of LD. "The values-based approach and cultural competence is more essential to focus on, particularly in Aotearoa". This is an interesting observation because this is not something that is necessarily taught but is a critical part of the role.  (Wang, et al., 2021) which found collaboration and communication among the top three skills and consulting with subject matter experts amongst the top three concerning abilities. In the current context, however, this is perhaps more challenging as work and learning oscillates between virtual and face-to-face depending on the circumstances of individuals. Hybrid working and learning are likely to be a reality of the future.

Research and evaluation
Evaluation, analysis and design critique was the most highly ranked of the research concepts.
However, in openended comments it appeared that if something had to give this would be it. While I think evaluation is really important I am conscious that there's only so much that can be done in a foundational course. At most maybe some kind of short proposal on how they intend to gather user data, and what qual vs quant methods could be used?. It was also noted that "Not entirely sure that research is a high priority for most learning designers or something that they are commonly allowed to do".

The dichotomy of knowledge and skills
Previously frameworks for learning design competencies such as the Knowledge, Skills, Abilities (KSA) framework Ritzhaupt, 2021, Kang andRitzhaput, 2015) have sought to organize competencies within these spheres. However, the current context is making this more complex/ nuanced. One of the challenges highlighted in the first round was that almost every topic had an aspect of knowledge and skills. Panellists found it hard to separate these and the dichotomy between the two was seen as, to some extent, arbitrary.
In addition, the changing nature of tools across contexts suggested that the principles of LD were more important to learn than the tools "LDs will be expected to be able to learn different platforms as they move between jobs, so I would provide more of a conceptual overview of what the purpose and key features of different tools (e.g. LMS) are and practical ways that students can develop skills with specific versions"

Conclusion
The increasing need for learning designers in the workplace and educators to upskill in the design of online learning makes a case for a foundational introductory micro qualification in learning design. The views from the Delphi study confirm the breadth and complexity of the learning design role. They also offer concern that this process might be too simplistic. "They are really only things picked from a pre-selected list. Learning design is a complex process and set of skills. see my Book Learning Experience design. Honestly I find this approach a little simplistic". Clearly, it is necessary to be strategic about what a micro-credential can focus on given it clearly can't cover everything. So, what foundational skills and knowledge are absolutely essential for a learning designer?
Because technology changes both across contexts and time it is essential to have a solid foundation in pedagogy including relevant learning theories, designing learning outcomes, assessment strategies and constructive alignment. A clear understanding of the differences in pedagogy and practice across modes is critical. LDs need knowledge about the learning process and models to provide a structured approach to the design process.
However, relationships proved to be a key aspect of the LD role. Understanding strategies on how to engage learners, and scaffolding peer-to-peer relationships through collaboration, inclusion, and interaction are core aspects of the role. LDs need to know how to create positive relationships and communicate effectively with stakeholders. Building relationships is seen as more important than managing projects. Developing and maintaining relationships in the (post)digital context, where the lines between online and offline, virtual and face-to-face are blurring, is more challenging than ever. These skills are not something one learns "on paper"