Designing adaptive online support for problem-based learning

This paper describes the work-in-progress of a novel solution to the problem of designing an LMS course structure that supports students in a problem-based learning course. The students come with a wide variety of prior experience in the field, they need to learn a set of complex skills that build on each other, they need to work partially asynchronously, and the design solution needs to be scalable. A lean weekly structure is proposed that is embedded with the synchronous teaching, intricately connected to a parallel hub of support material whose design allows students to construct their own bespoke learning journey. The course design draws on cognitive load theory to direct student attention only to what will be useful to them at any point in the course, and the support materials themselves follow universal design for learning principles to cater for a wide range of student learning needs.


Introduction
Problem-based learning is a well-established pedagogy in business education (see, for example, Stinson & Milter, 1996).Learning is driven by students solving ill-defined problems, usually in collaboration with their peers (Barrows, 1996).That this learning requires extensive scaffolding to ensure student success is wellestablished (Hmelo-Silver et al., 2007).The question of how to scaffold student learning, by providing process diagrams, case study videos, explicit descriptions and definitions, worked examples, sequencing of tasks, to the point where they can tackle such problems in with little more than light educator guidance, has been partially addressed in a variety of ways in the literature (Hmelo-Silver et al., 2007;Rienties et al., 2012;Tawfik & Kolodner, 2016;Larson et al., 2021).However, studies such as these tend to focus on the benefits of the support artefacts in isolation.This paper addresses the conference theme of pedagogy with the research question: how can an online manifestation of problem-based pedagogy support a diverse student cohort?We propose course structuring on a learning management system (LMS) as a design solution to this issue, based on cognitive load and inclusive design principles that offers students at elbow support with whatever learning artefact will benefit them most at any time.Some promising forays into using cognitive load theory to improve problem-based pedagogy learning design already exist.Chen (2016) looked at reducing the extraneous load caused by technological shortcomings in online problem-based learning.Jalani & Sern (2015), looked at breaking down complex problems into chunks.Buchner et al. (2023) looked at where direct instruction would prove most effective in the timeline of a problem-based learning session.This paper seeks to add to this corpus with an integrated design premise, rather than just addressing one aspect of the learning experience, formulated as an action research project with the potential to become the initial cycle of a design-based research project.

Context
The development described in this paper was conducted as part of a major strategic educational project (ethics approval 2019/892) in the School of Business at the University of Sydney called Connected Learning at Scale (Vallis et al., 2020).The project is underpinned by three pedagogical principles, the first of which focuses on information engagement.Application of this principle involves drawing on theories, experiences, and practices to foster student engagement with and deep understanding of learning material (Bryant, 2022).This is the principal embodied in the work presented in this paper.
The course coordinator of the course discussed in this paper sought support from the co-authors, an educational developer and a learning designer, as part of the above project to solve the problem of a lack of engagement with course materials by his students, and their struggles with tackling complex problems.In the course, an introduction to transport logistics, students needed to learn to use Excel to a high level, define problems, construct mathematical models, master a range of data analysis techniques and interpretation skills, and to be able to communicate solutions in plain English.The course coordinator intended to use problem-based pedagogy, so had loaded a wide range of support material onto the LMS so that students could master concepts and practice their skills outside of class and tackle complex problems in class.

Problem
The course coordinator wanted to use problem-based learning in this subject, however he reported that students were not accessing the material held on the LMS except when directed to in class, and he had instead reverted to using a direct instruction pedagogy in class.Students were having trouble understanding and applying concepts and were leaning heavily on the course coordinator for personalised support during class, which prevented him from working through authentic examples with the class.He was also having to organise extra drop-in sessions for extra support.The student cohort (Semester 1 2023) was surveyed on their learning experiences in the course (under ethics approval 2019/892, as above).They reported having their most valuable learning experiences in the synchronous sessions, which aligns with the course coordinator's observation.What students felt was lacking was encapsulated by this student quote from the end of semester survey, 'I would prefer more detail on how concepts & materials cover each week applying on real world business [sic]'.Such material was available, so this illustrates the disconnect the course coordinator reported about the students from the learning materials.Learning analytics from the LMS showed that support materials directly used in class were usually accessed by all students, but support materials in pre-work and support modules on the LMS were accessed by less than half of the students, and some material was not accessed at all.The observation of the design team (the co-authors and the course coordinator) was that these materials were difficult to find and were not clearly aligned with core course material.We decided to frame the initial iteration of addressing this educational problem as an action research project, which then had the potential to unfold into further cycles in design-based research, as it appeared that any solutions may be more generally applicable to problem-based pedagogy courses in other subjects.
The co-authors examined the support material that was available on the LMS.There were text definitions, explanatory videos, industry videos, partially worked examples, graduated practice sets with model answers and downloadable sample answer Excel spreadsheets and were all well-suited to support the course material.In short, exactly the kind of support material that the literature suggests is needed to scaffold problem-based learning, and what the students were asking for.We concluded that the support materials were not therefore the main issue, rather the way they were organised and presentedthey were too hard to find when needed.For instance, a set of practice questions was recommended to students after a class.In the original Week 3 LMS page, the instructions for using the practice questions are: there are many practice questions available for this topic.<link to 62-page PDF containing all practice questions for the course>.Answers to the multiple-choice questions are provided in the question book and answers to the practice computer questions will be posted after week 5.The workbook <link to 134-page PDF workbook> has a worked solution to practice question 4.01.
So the steps that a student would have to take to engage with a "computer question" (the more complex of the practice questions) were: 1. Download and open the practice questions book 2. Check the topic for the week (in this case, linear programming) 3. Find the chapter on linear programming in the practice questions book 4. Locate the "computer questions" sections in that chapter 5. Choose a question that would suit their skill level (the questions were of increasing difficulty, but this was not made explicit in the book) 6. Realise that a data set needs to be downloaded to complete the questions 7. Go back to the LMS to search for and download the relevant data set 8. Work through the questions 9. Either wait for two weeks to check answers, or search to see if the answers were given in the separate practice workbook.
This was the minimum workflow for students wishing to practice their skills.If they needed to review a concept or a process, they would need to go to yet another part of the LMS to find the appropriate video or text explanation, or go back through the three-hour workshop recording.There were similar issues with locating other support material, for instance a large set of concept explanation videos were held in an LMS module separate to the weekly modules simply called 'Online preparation modules'.

Reducing cognitive load
There was clearly a large cognitive load on students trying to engage with the learning materials outside of class.
Cognitive load theory (Sweller et al., 2011) posits that learning occurs when changes in what is stored in longterm memory happen.This is mediated by making connections between elements in working memory, some of which are drawn from long-term memory and some of which are provided by the learning environment.
Learning tasks can impose different types of cognitive load on working memory: intrinsic, extraneous, and germane.Intrinsic load is the inherent difficulty of the content or skill being learned, and it depends on the learner's prior knowledge and the complexity of the material.The complexity of the material is high in problembased learning, particularly at the postgraduate level in this context.Extraneous load is the unnecessary or irrelevant information or activities that distract or interfere with the learning process, and it depends on the design and delivery of the instruction.This is something learning design can minimise.Germane load is the beneficial cognitive effort that helps the learner construct and store schemas in the long-term memory, and it depends on the learner's motivation and metacognition.Learning design can also help with learner motivation by indicating the relevance of tasks and artefacts to the learning journey within the course and to authentic workplace practices.

Universal Design for Learning (UDL)
We not only used cognitive load theory in the redesign of the course presence on the LMS, we also used Universal Design for Learning principles (CAST, n.d.) in that design, and also in redesigning some of the support materials.
The students in this post-graduate course come from a wide variety of professional backgrounds and have varying proficiency in English.As the problems in transport logistics are always expressed as a paragraph of text, one of the skills needed is decoding that text to identify the elements of the problem, and then convert them into a mathematical model.Rather than trying to take a needs-based approach to making the support materials more accessible (such as providing text translations in students' home languages) the co-authors decided to take a Universal Design for Learning (CAST, n.d.) approach, as being inclusive, but also sustainablethere would then be no need to identify what specific needs each cohort contained every semester.We therefore provided a range of graphics-based illustrations of concepts to supplement the existing text and video based explanations, and added check your understanding activities with each concept in the support module and in the weekly materials.The principles are most commonly manifested in the following way (more specific examples are given in Table 1):

Multiple means of engagement
Signposted links out to support material optimised individual choice, it heightened the salience of objectives and provided varying resources to optimise challenge.

Multiple means of representation
Using contextualised and signposted links to the support materials allowed students a level of self-regulation, the levels of challenge were made clear for each practice question.Concepts were illustrated in the support module with a variety of text-based and graphics-based artefacts and activities.

Multiple means of action and expression
Students were able to build fluencies with graduated levels of support.The signposting also allowed students to manage which resources they needed to interact with.

Proposed course design
The cognitive load on the students in this course was unnecessarily high due to the large extraneous load involved in locating asynchronous learning materials.The co-authors sought to reduce this load to the bare minimum by organising much of the support material into a support module that was separate, but linked, to the weekly learning modules on the LMS.We organised the support materials by concept, and linked sections that built on each other so that students could easily move back a section to check a fundamental definition or application.We pared back the weekly learning modules to a pre-work, in class, and post-work page for consistency of experience.The structure was as follows: • Pre-class o Description of the concepts and skills that would be important for the in class activities In this way the student encounters a lean, consistent weekly course presence on the LMS from which they can seamlessly tailor their own support, reducing extraneous cognitive load.Signposting (usually a descriptive sentence) support artefacts allows students to identify their usefulness to their own learning journey, increasing metacognition and increasing beneficial germane load.We again use Week 3 as an example.Table 1 articulates the schema for the pre-class LMS page, with mapping to cognitive load theory instructional effects (Sweller, 2020) and UDL principles: The content for each page fits on a single screen on a laptop.This design reduces extrinsic load to a minimum, the clear signposting allows students to make the choices that are right for them easily.

Discussion
The proposed course structure looks at first glance like a simple case of good course design.However, the concept of separating the support material from the weekly course structure on the LMS and designing access as contextualised links appears to be new in the literature, particularly in business education.So too is articulating Universal Design for Learning principles in support materials for problem-based learning in business education.
Using cognitive load theory in designing problem-based learning is not new, but the research has tended to focus on the redesign of one aspect of a course in isolation, rather than the full structure as we have done here.The combination of designing for the most productive use of cognitive load on the LMS with UDL principles unlocks a scalable, inclusive support solution for a problem-based learning course with a student cohort that has a wide variety of prior experience and learning needs, for instance in the postgraduate space.

Evaluation
The course design described in this paper was put in place for Semester 2 of 2023.We plan to evaluate this redesign to see how easy students found it to locate resources, how often they used the practice questions and which ones, and how popular the graphic representations of concepts were versus text-based ones, and how popular each of the support resources were, with a view to foregrounding those students found most useful in the next iteration of the course design and identifying any support gaps.We plan to use learning analytics, focus groups with some of the students, and a semi-structured interview with the course coordinator in our analysis.
Examination of workshop recordings early in the semester show that solving problems does seem to be taking up more of the workshop time than direct instruction than last semester, a promising start.

o
Link to, and description of, the reading o Links to appropriate support module artefacts (practice questions, concept activities) if students felt that they needed the extra preparation o Check your understanding activity o Key terms • In-class o Details and data sets of the problems that would be worked through in the workshop • Post-class page o Lecture recording o A base set of practice questions with worked answers available o Check your understanding activity o Links to graduated practice questions (also with worked answers) in the support module for further practice.