Future prospects of artificial intelligence in education: Developing strategic scenarios to engage educators

The application of artificial intelligence in education (AIEd) is a growing field of research. While the potential benefits of AIEd drive interest and investment, developing ethical AIEd applications that mitigate the risk of entrenching inequalities requires input from key stakeholders like teachers and students. The present project aims to give voice to the perspectives of educators in higher education on the possible futures of AIEd. Towards this end, we developed a set of strategic scenarios, each describing a hypothetical AIEd application, and invited educators to evaluate the scenarios in a series of international focus-group discussions. The present paper describes the methodology for developing the scenarios and how they can be used to engage stakeholders in future-oriented discussions. This paper aims to assist interested readers in using or modifying the scenarios for their own research.


Introduction
Artificial intelligence (AI) has drawn increasing attention for its potential to transform education (Tuomi, 2018;Zawacki-Richter et al., 2019).However, when implemented without adequate attention to their ethical implications, AI applications risk exacerbating social inequalities (Hu et al., 2019) and can systematically "(dis)advantage some groups of students and teachers over others" (Facer & Selwyn, 2021, p. 13).Multiple policy documents have advocated for multi-stakeholder dialogues to mitigate the risks of AI (e.g., OECD, 2022; UNESCO, 2021; see Fatima et al., 2020, for a review of national AI strategies); however, not all stakeholders may have an equal voice at the table (IEEE, 2019).In the context of AI in education (AIEd), Zawacki-Richter et al.'s (2019) systematic review of AIEd research found that very few articles were led by authors from education and fewer discussed the ethical implications of applying AI in education.Therefore, there is a need to engage educators and give voice to their perspectives on the possible futures of education, and specifically how AI is developed for and implemented in education.
Many authors have contributed discussions on the potential impacts of AI on education (e.g., Bozkurt et al., 2023;Popenici & Kerr, 2017;Porayska-Pomsta & Rajendran, 2019;Schiff, 2021;Selwyn et al., 2020).These discussions serve to examine different facets of education, including trends in educational technology (e.g., Pinkwart, 2016), teaching and learning practices (e.g., Porayska-Pomsta, 2016), and how key factors may shape the future of AIEd (e.g., Baker et al., 2019).More broadly, the process of imagining possible futures can serve a variety of functions, including planning for different contingencies (e.g., Fink & Siebe, 2011) and collaborating to decide which futures we should work towards and which we should avoid (Bai et al., 2022;Nørgård, 2022).Therefore, while the future is inaccessible to direct observation, the exercise of imagining and discussing possible futures offers stakeholders an opportunity to contribute their perspectives towards shaping the future.We invited educators working in higher education to share their perspectives on the possible futures of AIEd by evaluating four strategic scenarios, each describing the implementation of a hypothetical AIEd application.Adapting methods from future management (Fink & Siebe, 2011), we developed these four scenarios to reflect possible outcomes of some key macro-and meso-factors (or strategy elements) that combine to determine the possibilities of different AIEd applications.We then invited educators across six countries and multiple disciplines to a series of focus-group discussions to evaluate the scenarios and suggest changes to the applications.Using these scenarios as a basis for focus-group discussions allowed us to make the abstract strategy elements concrete, and allowed participants to discuss and define the issues that they found most important.At present, this first, qualitative phase of data collection is almost complete and a second, quantitative phase is planned to gather further perspectives through an international, online survey.
We have previously discussed the rationale for choosing the strategy elements that shaped the strategic scenarios (Bai et al., 2022) and presented a thematic analysis of preliminary focus-group transcripts (Bai et al., 2023).In the present paper, we expand on the methodology for developing the scenarios and discuss how they can be used to structure focus groups and stimulate rich discussions.We hope that, by describing the process through which we developed the scenarios, this paper can facilitate adoption by interested readers who wish to use or modify the scenarios in their own research.

Developing strategic scenarios
There are diverse ways for imagining futures (e.g., Nørgård, 2022) and diverse scenario methods for assessing technology (e.g., Kosow et al., 2008).We choose to adapt the methodology used by Gutschow et al. (2016; see also Gutschow & Jörgens, 2019) because their scenario method offers a structured way to define possible contexts for future AIEd applications, and to make explicit the underlying assumptions behind these different possibilities (Bai et al., 2022).Gutschow et al.'s (2016) method traces back to ideas from Fink and Siebe (2011).While Fink and Siebe's book is closely concerned with the development of strategy alternatives in companies, Gutschow et al.'s procedure is also transferable to any future scenarios outside of a business context.Their study demonstrated this flexibility by developing scenarios for the validation of informal and non-formal learning.In particular, Gutschow et al. based their method on three steps in Fink and Siebe (2011): 1. Identify strategy elements 2. Describe and evaluate future options 3. Create and describe strategy alternatives The "strategy alternatives" in the final step correspond to Gutshow et al.'s strategic scenarios.The following sections describe the steps taken to develop the strategic scenarios used in the present study and presented in Table 2.
Step 1: Identify strategy elements Fink and Siebe (2011, p. 87) refer to the strategy elements as the "key elements" that determine a scenario.A literature review is usually conducted to identify strategy elements; however, the researchers' background knowledge also plays an important role.In the present study, we first collated a draft set of strategy elements from a diverse range of AI and AIEd readings; these included basic research (e.g., Mayfield & Black 2020), critical discussions (e.g., Bender et al., 2021), speculative futures (e.g., Selwyn et al., 2020), legal commentary (e.g., Wachter et al., 2017) and policy development efforts (e.g., Wendehorst & Woopen, 2018).Despite the range of perspectives sought out through our readings, the identification of strategy elements involves some degree of subjectivity and requires input from a diverse team (Bai et al., 2022).
In Fink and Siebe's (2011) work on developing scenarios in a business context, managers from a company form the scenario team and contribute their respective expertise to the selection of strategy elements.They emphasise that "it is essential for the acceptance of the strategy options that all team members identify with the selection of the key elements"1 (p.87).As a result, the process can involve lengthy discussions within the team and may become iterative as strategy elements are revised after draft scenarios have been created.In Gutschow et al. (2016), the scenario team was formed by the researchers involved in the study.Similarly, in the present study, our scenario team was composed of international researchers from the _______ who evaluated the draft set of strategy elements and provided feedback throughout the strategy-development process.After discussions within the team, we decided on a final set of five strategy elements; these are: 1. Access to data 2. Funding 3. Role of developers 4. Role of teachers 5. Scope of applications These strategy elements reflect key factors that operate on the macro-and meso-level, that could have a large impact on the future of AIEd, and that contain a high level of uncertainty.The rationale for choosing each of these strategy elements is discussed in more depth in Bai et al. (2022).
Step 2: Describe and evaluate future options In the second step, our team developed alternative future options for each strategy element (Table 1).These future options represent characteristics or attributes of the strategy elements and can be loosely interpreted as possible outcomes of each element.Fink and Siebe (2011) warn that a large number of future options often emerge from the team's discussion and that the challenge is to select candidate options that can contribute to the final scenarios (i.e., those that have a high probability of occurring within one of the scenarios).In the present study, we made multiple rounds of revisions as future options were added, removed, or merged based on feedback and discussions within the research team.Despite our iterative process for evaluating candidate options, not all of the final future options were represented in the final scenarios.For example, none of the final scenarios reflected completely laissez-faire nor heavily regulated access to data; nevertheless, we included these two extremes as boundary conditions in our discussion of possible outcomes (Bai et al., 2022).Therefore, some future options functioned to structure our discussion of the strategy elements, rather than as components in the subsequent scenarios.
The process for finalising the set of strategy elements and future options in Table 1 was informed by our own readings of a sample of the available AI and AIEd literature.Therefore, we encourage readers to bring their own expertise to critique and modify these elements in their own research.
Step 3: Create and describe strategy alternatives In the last step of Gutschow et al.'s (2016) method, the strategy alternatives (i.e., strategic scenarios) are developed through combining different sets of future options.To facilitate this process, the strategy elements and their associated future options can be presented in a morphological box (Table 1; see Zwicky, 1966;Gutschow et al., 2016).Various future possibilities can be generated by drawing zig-zagging lines down the morphological box to create sets of conditions that allow for different possibilities of AIEd applications.This approach borrows from morphological analysis, which traces back to Fritz Zwicky and has been used for complex political problems as well as for developing future scenarios (Ritchey, 2013).In general, it is a method for analysing issues where a variety of non-quantifiable (i.e., qualitative) factors influence a given problem.As Ritchey describes it: "Essentially, general morphological analysis is a method for identifying and investigating the total set of possible relationships or 'configurations' contained in a given problem complex" (p.3).Using such an approach, the strategy elements in Table 1 serve as parameters and their associated future options correspond to possible values of each parameter.A possible solution to a problem or research question results from marking one value of each parameter so that different solutions are defined by different combinations of values.
If all values in Table 1 could be freely combined, there would be a total of 640 (5x4x4x4x2) possible combinations to serve as the basis for possible scenarios.Such a large number of scenarios cannot be meaningfully examined and the solution space can be reduced via a cross-consistency assessment (Ritchey, 2013).Specifically, if there is a contradiction or incompatibility between two values of different rows, all solutions that involve the combination of these two values are deemed invalid.Additionally, links between values of different rows can also reduce the solution space if a particular value of one parameter is always accompanied by a particular value of another parameter.As an example, if the development of an AIEd application is funded by the private sector (i.e., primarily market-based or public-private funding), it is unlikely that such funding models would lead to open (i.e., free) access of the application.Conversely, private funding models are likely to be associated with limited and proprietary access to data as companies are incentivised to protect the economic value of the user data (Hoofnagle et al., 2019).
After accounting for contradictions and reductions, we configured four sets of future options that we deemed to be plausible (see middle column of Table 2).While this process necessarily involves a degree of subjectivity in determining what is 'plausible', it also allowed us to define the possible contexts for our hypothetical AIEd applications explicitly.We hope that this transparency will assist readers in examining the scenariodevelopment process and creating different combinations of future options to suit their own research purposes.
As a final, additional step, we instantiated the scenarios by mapping the sets of future options onto four active areas of AIEd research, as identified in Zawacki-Richter et al.'s (2019) systematic review.Specifically, Table 2 shows how each set of future options serves as the context in which the following categories of application were implemented: 1) profiling and prediction, 2) assessment and evaluation, 3) adaptive systems and personalisation, and 4) intelligent tutoring systems.Table 2 presents the full texts of the scenario descriptions; each description aims to describe in broad terms, but in sufficient detail, how the hypothetical AIEd application works (i.e., what data it uses as inputs and what information it outputs), and reflect the future options that make the application possible (e.g., how the system was funded, the role of the developers, and how students and teachers interact with the system).

Scenario
Set of future options Scenario description (in English) Scenario 1.

Profiling and prediction
Limited and proprietary access to data; funding through public-private partnership; subscription-based (i.e., software-as-aservice); retraining of teachers; comprehensive scope of application A prediction system is implemented in your institution to predict students' performance and their risk of dropout.The system collects a range of data from each student (for example, assignment grades, attendance, and interactions with the institution's online systems) to calculate the probability of the student achieving a particular grade in each of their courses.Teachers can view the tracked data and the system's predictions via a dashboard on the institution's virtual platforms.In addition, when a student is classified as "at risk", the system sends student support a notification and a personalized list of suggested interventions.The support staff make the final decision about which intervention to implement.The system was developed by a joint collaboration between academic researchers and a private company.It is provided to your institution for a reduced subscription cost in exchange for pseudonymized student data that the company uses to improve the system's performance.

Scenario 2. Assessment and evaluation
Limited and proprietary access to data; primarily marketbased funding; subscription-based (i.e., software-as-aservice); division of labour; limited scope of application An automated essay scoring system is implemented in an introductory course in your field.The system is recommended for courses with large classes and is trained with a subset of essays that are hand marked by teachers.These hand-marked essays can be randomly sampled from the submitted essays or reused from a previous year.The system then analyses the remaining unmarked essays and automatically assigns a grade to each essay, along with a confidence rating.Essays marked low confidence are flagged for teachers to review.The system was developed by a private company.It is provided to your institution for a reduced subscription cost in exchange for pseudonymized student essays that the company uses to improve the system's performance.Scenario 3.

Adaptive systems and personalisation
Limited but open access to data; funding through public-private partnership; developers as coresearchers; retraining of teachers; comprehensive scope of application As part of a pilot study, a multi-function learning management platform is implemented in your institution.The system collects a range of data from each student (for example, assignment grades, interactions with the institution's online and learning systems, and use of campus facilities) to monitor their performance and development and make recommendations for personalized learning paths, student support services, and future courses.Teachers can monitor students' progress via a dashboard and override the system's recommendations if they disagree.At the end of each semester, teachers and developers conduct a review of the system's performance and adjust the system's settings in a quasi-experimental design.In addition, students may opt-out and withdraw their data at any point.The system is being developed by a joint collaboration between academic researchers and a private company.During its development, the system is provided to your institution free of charge in exchange for pseudonymized student data that the company uses to improve the system's performance.Scenario 4.

Intelligent tutoring system
Limited and sporadic access to data; primarily government funding; open access; division of labour; limited scope of application An intelligent tutoring system is implemented in an introductory (first-year) course in your field.The system includes a range of inbuilt learning tasks that cover basic and advanced concepts.The system analyses student performance on each task (for example, time taken to complete task, types of errors made, and performance in similar tasks) to give personalized feedback and decide on the next appropriate task.The system sends teachers a notification if the student is stuck or struggling with a particular task or concept.The system was developed by a university research group, and trained with data collected from the researchers' institution.It is provided to your institution free of charge as open-source software (i.e., an Open Educational Resource) and no student data is sent outside of your institution to improve the system's performance.

Using strategic scenarios in focus groups
The goal of the present project is to seek out the perspectives of educators on AIEd and the possible futures of education.This goal informed our choice to ground the strategic scenarios in active strands of AIEd research as a way of lowering the barrier to entry into the discussions.Making the scenarios concrete, plausible, and evidence-based meant that participants were not required to be experts in the nuances of the strategy elements and could instead draw on their expertise as educators to highlight what they found most salient in each scenario.
We found the strategic scenarios to be effective in stimulating rich and nuanced discussions (Bai et al., 2023).In each focus group, we opened the discussion with two general, warm-up questions and then introduced the scenarios.For each scenario, we asked participants to evaluate what they saw as the potential benefits and challenges of implementing such an application in their institution, and what they would add or change about the application.This last prompt gave participants the opportunity to co-create preferred visions of the future and refine what they saw as ethical uses of AI.Together, these prompts generated a variety of interesting responses, with participants often reflecting on their current teaching practices, advocating for students, and reexamining their fundamental assumptions about teaching and learning.As one participant noted during a pilot interview: When we look at scenario two, it makes me wonder about… what I think of myself as being a teacher and what my normative assumptions are… about good teaching, right?And good examination.So I do not regularly think about it but I just do it because I think what I do is good […] but once you present these scenarios, they confront me with a situation in which some part of my doings will be replaced by artificial intelligence, so I start thinking about… why is it that I think my teaching and my examination is appropriate, what are the underlying assumptions about what good teaching is, when I teach and that… I think these questions are triggered through a scenario in which these tasks are no longer done by a teacher.This extract demonstrates how the scenarios can be used to stimulate self-reflection and a deeper consideration of the role of teachers and what constitutes 'good' teaching practices.An analysis of three focus-group discussions from Japan, Spain, and Germany revealed further insights, as the scenarios prompted educators to express concerns about the accuracy of the AIEd applications and how the use of these applications may affect students' and their own behaviour (Bai et al., 2023).Such contributions are highly relevant and valuable to the AIEd discourse, as advances in AI technologies allow more tasks within current teaching practices to be automated (Molenaar, 2021).Thus, the concerns raised by educators deserve careful consideration as AI becomes integrated into education systems.

Limitations and alternative approaches
While the rich data from focus-group discussions provides support for the utility of the strategic scenarios presented in Table 2, there are limitations to our process for developing these scenarios.Firstly, the identification of strategy elements and future options was filtered through our subjective readings of a sample of the AI and AIEd literature.We attempted to mitigate some of these biases through discussions within our diverse team of international researchers at the _______.However, as we acknowledged previously, "the visions of possible futures presented here are neither complete nor objective" (Bai et al., 2022).Secondly, the strategic scenarios were developed at a particular point within a dynamic social and technological context.This was made clear to us as developments in the regulation of data and the release of OpenAI's ChatGPT prompted us to revisit our initial discussion (Bai et al., in press).
While it is likely that the specific scenarios developed in this project may have a limited shelf-life, the methodology for developing strategic scenarios will likely remain an important tool for future studies and for engaging with other stakeholders (e.g., students, institutional decision-makers, developers, etc.).For example, an alternative approach may be to narrow the focus and develop alternative scenarios for only one application (e.g., examine the possibilities for intelligent tutoring systems under different configurations of future options).Such an approach could serve as one component in the design process for developing a particular AIEd system and would necessarily involve some modifications to match the research goal (e.g., providing more detailed descriptions of how the systems work).Thus, the flexibility and utility of Gutschow et al.'s (2016) general scenario method lends itself to a variety of possible applications and research goals.
Lastly, we note that there are a variety of alternative approaches to imagining the future.In contrast to our methodology, Nørgård (2022) provides a compelling case for speculative design that embraces the ethics of hopepunk and challenges readers to imagine futures beyond what is merely projected or probable.The scenarios presented in the present paper may be limited to "probable futures… that are forecasted based on current trends, quantitative data, extrapolation from similar cases in the past or present" (p.165).Nevertheless, when used to engage educators and other stakeholders in discussions, the strategic scenarios in Table 2 can serve as objects "to think with together" (Nørgård, 2022, p. 172).Therefore, our approach aligns with the participatory ethos advocated by Nørgård; we hope that this project, by giving voice to educators thinking together, will help to mitigate the potential risks of future AIEd applications and contribute to the multi-stakeholder dialogue to shape a more-inclusive future for education.
Authors' note: This project is funded by _______.Thanks to _______ for their active collaboration and contributions to shaping the strategy elements, future options, and scenario descriptions.Thanks also to _______ for useful feedback on an earlier draft of this paper.