Design-Based Research: Enhancing pedagogical design

This concise paper explores how Design-Based Research (DBR) can practically guide pedagogical innovation through collaborative curriculum design. We illustrate this with four short case studies (vignettes) of DBR implemented in different discipline contexts in higher education. An examination of the common characteristics of these four project designs show how DBR can be practically implemented as a transferable educational design framework. Future work includes deeper analysis and reporting on the longitudinal impact on student learning of these projects in a subsequent full journal paper.


Introduction
The literature exploring pedagogical innovation in the field of Technology Enhanced Learning (TEL) is dominated by examples of case studies using quasi-experimental methods that often result in no significant difference in pedagogical outcomes (Reeves, 1995(Reeves, , 2005;;Reeves et al., 2010;Reeves & Lin, 2020).Reeves argues that DBR is one solution to generating knowledge on real world impact for stakeholders and uses the synonym Educational Design Research (EDR) when applying DBR to educational research.However, there is a gap in the literature: there are few practical examples of how to implement DBR to solve pedagogical problems or design innovation in teaching and learning using educational technologies.McKenney and Reeves (2019) have produced a definitive book on conducting educational design research, but it is very detailed and treated as a canon of DBR rather than a pragmatic guide.In producing such a detailed EDR framework McKenney and Reeves have set the bar high for EDR project implementation.What is needed are more practical examples of the implementation of DBR in a variety of disciplinary contexts.This concise paper describes four examples of DBR in different higher education disciplines and briefly examines the common characteristics of a pragmatic DBR framework.Amiel and Reeves (2008) argue that DBR is an ethical approach to pedagogical innovation to solve real world learning problems.DBR avoids the common constraints of quasi-experimental approaches to educational research.DBR establishes and starts with what is already known about a problem and then asks what an appropriate approach or technology is to solve the problem.Instead of taking a randomised control group comparative approach to learning interventions DBR takes a design approach.DBR does not require a 'control group' or randomised selection of participants as DBR does not focus upon making comparisons between different participant experience.Rather, it seeks to solve a defined pedagogical problem through iterative design and evaluation of best practice in a situated context, often through innovative TEL approaches.Therefore, groups of learners are not treated differently as that may lead to disadvantaging their learning experience.DBR takes a collaborative approach to learning design, starting with the formation of a collaboration between the key stakeholders (practitioners and students) and educational researchers and educational designers.DBR is beneficial for academics who are experts in their discipline but not experts in learning design and tend to default to utilising disciplinary-based research approaches for educational research and design.DBR is a pragmatic framework for addressing pedagogical problems or innovation through four phases: problem analysis, develop solutions to the identified problem, explore and evaluate the impact of the design in iterative interventions in real learning situations, leading to the development of transferable design principles and theory building (McKenney & Reeves, 2019).Drawing on a short guide to DBR (Cochrane, 2022), the four phases of DBR are briefly expanded in the following sections: Analysis and exploration, Design and construction, Evaluation and reflection, Design Principles and theory building.

Analysis and exploration
The first phase of DBR involves identification of the curriculum design problem or innovation, for examplehow to design an authentic student-centred project that is scaffolded across a curriculum and the critical issues surrounding the specific learning environments.This is followed by the exploration of supporting literature to identify initial design principles to address these issues and identify gaps in the literature.

Design and construction
The second phase of DBR draws upon the design principles identified in the first phase of DBR and leads to prototyping of the collaborative curriculum design informed by the identified design principles.A design thinking approach can be utilised to guide the design and construction phase (Cochrane & Munn, 2020).

Evaluation and reflection
The third phase of DBR involves evaluation of the prototype curriculum and subsequent collaborative curriculum redesign through user feedback (students and project team peers), and refinement of the design principles.This then leads to a Phase 2-3 Loop: Iterative redesign and re-evaluation of the collaborative curriculum design.

Design Principles and theory building
The final phase of DBR leads to development of transferable design principles and dissemination of findings for application to other learning contexts.

DBR vignettes
In this section we explore the design of four short case studies (vignettes) of implementing DBR in different disciplinary contexts in higher education: Health Education (Galvin, 2023), Electrical and Mechanical Engineering (Buskes et al., 2023), Biomedical Engineering (Lam et al., 2021), Music Performance (Osborne et al., 2022).As this concise paper summarises prior publications of each project, and all authors are also authors of this concise paper, ethics consent is implicit for an analysis of the design principles of each project.

Clinical reasoning development
Technology is ranked as one of the most important factors influencing education in Australia, with a growing demand for digital innovation to elevate online opportunities for clinical education.Learning the skill of clinical reasoning is central to clinical education and includes ignoring part of available information to focus on fewer relevant predictors for a clinical outcome.While there are documented reasons why problem-based learning (PBL) and team-based learning (TBL) are both used in clinical education to improve decision making, research has mostly focused on assessment scores and learner perceptions when comparing Face-to-Face (F2F) traditional lecture style with a group learning experience.To explore how to enhance clinical reasoning using digital innovation this longitudinal doctorate research project used a DBR methodological framework and qualitative reflexive thematic analysis (RTA) method for data interpretation.An initial scoping literature review and pilot cycle of this study resulted in establishing four key factors that enhance clinical reasoning development: (1) cognitive learning is impacted by group work, (2) growing professional identities have influence on clinical reasoning development, (3) ongoing peer interaction influences development of clinical reasoning skills, and (4) the role of the teacher is important when learning clinical reasoning.A set of draft design principles led to the development of a hybrid approach to PBL by incorporating design elements from traditional PBL with the more current TBL and an online decision wheel tool for learners to practice clinical decision making independently and in groups.The project involved five DBR action cycles (12-week Trimesters).At the conclusion of this project, six final design principles were generated with implications for learning design, health science education, and higher education more broadly.Learnings generated reveal that clinical reasoning skills are enhanced when having a regular central teacher guide, adequate coaching support, simple learning designs, time for reflexive practice, and by encouraging the ability for a diversity of key stakeholders to collaborate and be partners in curriculum.
Alignment with the four stages of DBR 1. Analysisproblem identification (How to enhance clinical reasoning development by exploring individual and group rational decision making across undergraduate health science courses at Torrens University), literature review, establishment of a collaborative design team of students and academics.2. Design prototype intervention (design of a new problem-based learning model with the integration of an online decision wheel tool used across the 17 subjects).3. Evaluation (implementation of the new problem-based learning design and online decision wheel in learning activities and assessment with stakeholdersstudents/academic team/learning designers) -Re-Design / Evaluation iterative loop.
4. Development of six transferable design principles for enhancing clinical reasoning development and individual/group rational decision-making revolving around the collaborative stakeholder-led design of an authentic shared project that scaffolds the integration of learning outcomes across multiple subjects.

Transforming energy and pedagogy
Engineers ultimately work in multi-disciplinary workplaces, yet degree structures and siloing of subjects typically prevent students from interacting with those outside of their own discipline.As products and technology become increasingly complex, engineers can no longer do design in isolation.This project seeks to deliver a cross-disciplinary and cross-subject design capstone project that serves to expose students to the boundaries of multiple engineering disciplines, how to specify and verify design requirements, and how to communicate across technical boundaries via appropriate abstractions, including with industry partners.The initial phase of the project involved a literature review of existing collaborative design projects and relevant frameworks to employ.Studies on professional skills that are most highly valued by engineering companies and the gap that graduates are perceived to have in these were used as a basis for forming the teaching and learning activities associated with the project.The design team includes collaboration between engineering discipline expertise, educational technology expertise, and Industry partners.The design phase involves the development of a real-world project that involves an engineering student design team comprised of both electrical and mechanical engineering students through the design of a speaker system, including the electrical, mechanical, and acoustic components.The third phase will evaluate the prototype design and the design team experience across the different stakeholdersstudents, staff, and industry.An industry partner will be invited to evaluate student team final designs/builds and provide critical industry feedback on the project impact.This will then inform the development of transferable design principles for designing authentic (real world) transdisciplinary learning environments in collaboration with industry.
Alignment with the four stages of DBR: 1. Analysisproblem identification (Threshold Concepts: transdisciplinary collaboration, authentic learning), literature review, establishment of a collaborative design team of students and academics.2. Design prototype intervention (design of authentic learning environment).3. Evaluation (implementation of prototype with stakeholdersstudents/industry partner) -Re-Design / Evaluation iterative loop.4. Development of transferable design principles for designing authentic (real world) transdisciplinary learning environments in collaboration with industry (A loudspeaker driver manufacturer based in Melbourne).

The bionic limb
Biomedical engineering is a discipline that integrates concepts from mechanical, electrical, and chemical engineering, as well as computer science to develop technologies that improve human health.Most existing biomedical engineering curricula, however, do not reflect this transdisciplinary integration, as these concepts are introduced to students in separate subjects with minimal or no cross-curricular references or alignment of learning and assessment activities.To tackle this problem, we launched a collaborative curriculum redesign project centred around the student-led collaborative design of a bionic limb.This redesign has allowed us to link four core subjects across the major, covering key concepts in programming and modelling, biomechanics, electronics, and the engineering design process.This led to the design of a prototype bionic limb and associated teaching and learning materials that integrated learning and assessment activities across four core subjects of the undergraduate biomedical engineering systems major.The initial problem analysis identified three factors that impacted the efficacy of the subject curriculum in the transfer of skills and knowledge of necessary for biomedical engineering: (i) varying levels of prior exposure to concepts in electric circuits, mechanics and computer programming; (ii) a preference for rote-learning of steps or formulae; and (iii) compartmentalisation rather than integration of concepts taught in electrical circuits, mechanics and computer programming.This led to the development of a bionic limb that facilitated conceptual alignment across the four subjects and were identified and mapped onto specific activities and assessments to be considered in the design and function of the limb.A series of authentic scaffolded tasks constructed around the material design of the bionic limb (force and moment analysis, stress and strain analysis, materials testing, CAD design) was developed.Two project-based group assignments were established that required students to synthesise concepts of engineering design, mechanics and computational analysis principles.Three workshops were established to learn the effect of capacitors, resistors and inductors on the operation of the bionic limb.To increase the degree of interconnectedness between the four subjects, evaluation and feedback identified the importance of a common learner-centric ecology of resources (Luckin, 2008) be introduced to support student collaboration across the subjects.Current plans for this revolve around the student-driven curation of ePortfolios to document their progress and learning as they complete the sequence of four core subjects.This has led to the development of design principles for transdisciplinary design from the subject-level to the degree-level.
Alignment with the four stages of DBR: 1. Analysisproblem identification (How to integrate complex concepts across disciplines to tackle real world problems: transdisciplinary collaboration, authentic learning), literature review, establishment of a collaborative design team of students and academics.2. Design prototype intervention (design of the bionic limb as a project shared across the four subjects) 3. Evaluation (implementation of the bionic limb and learning activities with stakeholdersstudents/academic team) -Re-Design / Evaluation iterative loop.4. Development of transferable design principles for designing authentic (real world) transdisciplinary curriculum revolving around the collaborative student-led design of an authentic shared project that scaffolds the integration of learning outcomes across multiple subjects.

Virtual performance lab
This project explores the use of a simulated performance environment in virtual reality (VR) to increase student engagement and wellbeing, enhance learning outcomes, and provide equitable learning opportunities, by presenting an alternative to the traditional dichotomy of face to face versus online teaching and learning.A performance intervention will be delivered via VR for students across three music performance subjects to down-regulate performance anxiety and improve focus and resilience.This project will generate a protocol for use of VR in individual and group performance teaching, representing a fundamental innovation in curriculum delivery.By limiting the scope of this project to music performance students we are able to deeply understand the experiences of the students and the impact of the intervention on wellbeing and performance outcomes.The potential future pedagogical applications of this project, however, extend beyond the disciplinary boundaries of music.Simulation training is used to develop performance skills in various disciplines, particularly where in-situ training is either impossible or unsafe to implement.Such training enables learners to acclimatise to real-life stressors and anxiety-inducing scenarios in physically and/or psychologically safe environments, to protect against performance decrements which reveal themselves in high pressure contexts rather than low-stress practice sessions.The project therefore has the potential to inform future teaching and learning interventions in other disciplines that encounter high-stress and/or dangerous scenarios, such as surgery and other medical interventions, sports training, engineering scenarios, public speaking, and other performing arts practices.
Identifying and triangulating design principles from immersive reality projects in these various contexts will support ongoing work to refine the development of a design framework and model to enhance performance in higher education.
Alignment with the four stages of DBR: 1. Analysisproblem identification (Reducing performance anxiety for music students, through an authentic immersive learning experience), literature review, establishment of a collaborative design team of students and academics.2. Design prototype intervention (design of authentic immersive learning environment and evaluation methods).3. Evaluation (implementation of prototype with stakeholdersstudents/academics) -Re-Design / Evaluation iterative loop.4. Development of transferable design principles for designing authentic immersive simulation of real world music performance environments that could also be applied to other high-stress learning environments.

Discussion
The example of DBR projects range from works in progress to a recently completed study and this paper focuses upon the design of these projects.Evaluation of the impact on student learning outcomes are part of each project design and will be reported on in subsequent publications.There are common characteristics to each of the DBR project examples.Each of the DBR project examples begins with a defined problem or innovation statement that focuses upon designing authentic learning environments.Collaboration of key stakeholders is integrated into these projects.While transdisciplinarity is a common theme across the projects it is not a requirement of DBR, but often an outcome of the design of authentic learning.Each project involves iterative design, implementation, evaluation and redesign leading to the development of transferable design principles.The choice of appropriate technologies and evaluation strategies is unique to each project context but informed by transferable design principles.

Conclusions
DBR utilises design principles and design thinking to improve pedagogical design and innovation.We have described four examples illustrating how McKenney and Reeves four phases of EDR/DBR can be achieved through supporting learning design via collaborative learning design teams using a pragmatic iterative design and evaluation approach within a variety of discipline contexts.Next stages include a deeper analysis in a full journal paper.In this way we hope to dispel some of the barriers to wider adoption of DBR in learning design.