Seeing the wood for the trees: Insights into the complexity of developing pre- service teachers’ digital competencies for future teaching

Developing digital competencies is a critical component in pre-service teacher training for future practice. However, this is a complex process which includes a range of strategies, and little is known about how they should be implemented. Methods that can address this complexity are needed to improve understanding of strategies to develop digital competency in teacher training. In this paper, data mining approaches are used to explore this issue, through analysis of preservice teachers’ experiences with six key digital competency strategies. Specifically, association rules analysis was conducted on a questionnaire dataset of 931 preservice teachers’ experiences in their training, with 24 different practices representing the six strategies. Results showed four distinct clusters of associated strategies, which illustrates how the approach was able to reveal some complexity among digital competency strategies. The most important strategies were Authentic Experiences, Instructional Design and Role Models, showing multiple relationships and effects across all four clusters. Implications for practice suggest certain combinations of strategies are necessary to support pre- service teachers’ developing digital competencies. Future directions for research are discussed.


Introduction
To build pre-service teachers' digital competency and encourage effective integration of digital technologies in future teaching, training needs to incorporate a range of key strategies. However, research has shown that this is a complex process which requires combinations of strategies (Mouza, Karchmer-Klein, Nandakumar, Ozden, & Hu, 2014). Also, relationships among these strategies are complex. Research has suggested that a systems approach is needed to understand this complexity and how strategies should come together in practice (Tondeur, Aesaert, Prestridge, & Consuegra, 2018). Without a better understanding of how strategies for digital competence relate, how to best design pre-service teacher training to develop digital competence will remain obscured.
To explore the complexity of strategies and how they organize, methods able to address and visualize these phenomena are needed. In this paper, we present and initial exploration into complex relationships among digital competency strategies in teacher training. To do this, data mining techniques are employed to re-analyze and visualize questionnaire data from Tondeur et al.'s (2018) study exploring six effective strategies to develop preservice teachers competencies to use ICT in their practice (see Tondeur et al., 2012). Data was analysed using a combined approach including association rules analysis and directed graphs to visualize the resulting relationships (see Howard, Ma, & Yang, 2016). Findings provide insight into complex relationships among strategies and how they were related in students' experiences in teacher training. Implications for practice andn future research will be explored.

Digital competency
To prepare pre-service teachers to effectively use digital technologies in their future teaching, a range of strategies are needed (Kay, 2007;Mouza et al., 2014). Tondeur et al.'s (2012) synthesis of qualitative evidence identified that these strategies should include: i) scaffolded authentic experiences, ii) collaboration, iii) learning to use digital technologies by design, iv) continuous feedback, v) reflecting on the role of digital technologies, and vi) teacher educator role models. These strategies have been combined to create a Synthesis of Qualitative Evidence (SQD) model (see Tondeur et al., 2018). The SQD-model includes three levels of consideration when preparing pre-service teachers for technology use (Tondeur et al., 2018). The outer level includes systematic and systemic change efforts, along with aligning theory and practice. The second level considers aspects of the institution, such as technology planning and leadership, training staff, access to resources, or cooperation within and between the institutions. The inner circle includes the six micro level strategies such as using teacher educators as role models, or scaffolding authentic technology experiences (see Table 1). These six strategies will be the focus of the current analysis.
The review of the six SQD strategies found that effective preparation of pre-service teachers' digital competencies required attention to the separate strategies and combinations of strategies (Tondeur et al., 2012). Tondeur et al. (2018) analysis of combined strategies revealed complex relationships among those needed for pre-service teachers to successfully develop digital competency, and this was difficult to address.

Complexity of developing competencies for technology integration
The complexity of digital competencies is well known and often discussed as a 'wicked' problem (e.g. Borko, Whitcomb, & Liston, 2009). Effective digital technology use is "complex, contextual, and multi-faceted" (Kimmons, Miller, Amador, Desjardins, & Hall, 2015). However, factors of technology integration are often studied in isolation or in ways that are not able to account for complex relationships (Howard & Thompson, 2016). Consequently, strategies to prepare future teachers are also a complicated challenge.
As stated above, complex relationships exist among strategies to train pre-service teachers for technology use in teaching and learning. Using a system approach, Howard and Thompson (2016) have previously illustrated some of the complexity among factors known to have an effect on technology use in the classroom, such as teacher and students' beliefs about digital technologies, and school leadership. Analysis revealed complex and dynamic interactions among these factors and feedback in the system. A similar level of complexity needs to be taken into consideration when designing learning to develop pre-service teachers' digital competencies. To be effective, it is necessary to employ multiple strategies to develop digital competence. It is also important that pre-service teachers' are provided opportunities to develop and critically engage with technology-enhanced learning designs, and that they reflect on and evaluate these designs (Kimmons et al., 2015).
The multiple effects of relationships among these factors require methods able to address complexity to be understood. Data mining approaches can address some of this complexity, particularly large numbers of interrelated factors. Broadly, data mining can be defined as a process of automating discovery of patterns in large datasets. This is an inductive process, which provides insights based on patterns emerging from data, which is understood as 'knowledge discovery' (Romero & Ventura, 2013). Data mining approaches are agnostic of content or discipline, so they can be applied to any dataset. Importantly, it does not assume a linear model, which is a significant difference between traditional statistical procedures. This allows the complexity of relationships among data to be revealed, which can then be further analyzed for contextual meaning and relevance. This brings us to the main aim of the current study.

Purpose of the study
The aim of the current analysis is to explore the use of data mining to better understand the complexity of digital competency strategies in teacher training. Data mining approaches are well suited to exploring complexity and extracting patterns from datasets. In regard to digital competency, this approach can provide insight into complex relationships among the six key strategies and how they were related in relation to students' experiences in teacher training. This has the potential to inform learning design in initial teacher training and improve pre-service teachers' preparedness to enter the classroom.

The dataset
To explore the complexity of digital competency strategies, the current research will extend Tondeur et al. (2018) multilevel analysis of pre-service teachers' experiences with digital competency strategies in training. Pre-service teachers were asked to indicate their agreement with statements about their perceived support in teacher-training institutions with respect to the six SQD-strategies, e.g. "During my pre-service training I saw good examples of ICT practice that inspired me to use ICT" and "I received sufficient help in designing lessons that integrated ICT". The dataset included 931 final-year pre-service teachers from 20 teacher training institutions in Belgium. Seventytwo percent of respondents were females, which is representative of pre-service teachers in Flanders. The average age was 24.7 years (SD=7.02 years). Over half of the pre-service teachers had obtained a Bachelor's degree (57.8%) and 42.2% had obtained a specific teacher training degree.

Analysis
Initial analysis was a multivariate hierarchical regression, using a two-level design with pre-service teachers clustered in training institutions (Tondeur et al., 2018). The main result was that "more pre-service teachers perceive occurrences of the SQD-strategies during their teacher education, the higher their competence to use ICT for learning processes and to strengthen their instructional practice" (Tondeur et al., 2018, p. 38). However, multiple strategies were required to be successful.
The current analysis employed association rules analysis to explore which relationships among strategies were most frequently occurring in the dataset and how they related. Howard et al. (2016a) have previously used this approach to study perceptions of digital technology use in learning. Association rules analysis is commonly used for identifying relationships in a dataset. Relationships are identified based on the frequency of factors appearing together as a 'rule' in the data. Relationships occurring frequently are thought to be more 'important'.
A limitation of the approach is that smaller, less frequently occurring, but important relationships, can be missed. However, it provides a good starting point to explore datasets and identify initial phenomena. Relationships are expressed as rules, in the form of A→ C. 'A' is understood as the 'antecedent' while 'C' is understood as the 'consequent'. It can be read as 'IF A, THEN C'. Each part of the rule may contain single or multiple factors. The importance of relationships is assessed using three key measurements: support, confidence and lift. Support identifies the degree to which the antecedent and consequent occur simultaneously in a given dataset. Confidence indicates how frequently the consequent follows the antecedent. Lift indicates the correlation between antecedent and consequent, which can predict performance (see Howard et al., 2016a).
In this analysis, 80% Support and Confidence was used as the rule selection criteria. Twenty-four SQD items were included in analysis, addressing the six key factors, and each with five possible responses (i.e. five-point Likerttype scale). Possible answers to items are treated categorically, e.g. for each item Agree and Agree Strongly are treated as separate categories. Given that 24 variables were analysed, each with five possible responses, the number of resulting rules was very large. Therefore, for the initial exploration the top 20 rules were selected for analysis based on their levels of Support (>.10) and Confidence (>.80), which resulted in four clusters of associations (relationships). Solutions for the full set of rules were tested, but they did not show significantly different clustering. The 20 rules were then visualized as a directed graph. A directed graph is defined by showing relationships among the variables as 'ordered pairs'. Arrows indicate the relationship and its direction, with the beginning of the arrow being the Antecedent and the arrowhead being the Consequent. Clusters of variables in the dataset have been circled, but are only heuristics for the purpose of discussion. Clusters have not been calculated for this initial analysis.

Results
The association rules analysis revealed a total of 7,812 rules, with an average length of 3.76 items (combined antecedent and consequent). The average support was .11 and confidence was .84, meaning the antecedents and consequents occurred together 11% of the time in the dataset and, of that, 84% of the time the consequent followed the antecedent. The average lift was 3.71, meaning the rule was 3.71 times more predictive than random choice.
Figures 1 presents a directed graph visualizing the 20 most important (frequently occurring) rules in the dataset. Four clusters can be observed in Figure 1, with two clusters connected by FEE3 and two clusters occurring independently. Rule consequents are at the centre of each cluster. The antecedents are fanned around the outside. For example, in Cluster 1 the rule 'REF3, AUT3-> DES3' can be understood as: if a student Agreed 'There was room for discussion about our experiences with ICT in the classroom (REF3)' and 'I was stimulated to gain experience with ICT in class practice (AUT3)', they were also likely to Agree 'We were supported to develop educational material using ICT.' REF3 and AUT3 are the antecedents and agreement on both must be present to be likely to result in agreement with DES3. Of the 24 SQD items, 11 items occurred in the 20 rules and they were all 'Agree' categories. Each of the six strategies were represented by at least one antecedent or consequent in the rules. The most frequently occurring strategies were Authentic Experiences (AUT), Instructional Design (DES) and Role Models (ROL), items appeared in 18, 13 and 10 rules respectively. The most frequently occurring individual item was AUT3, occurring in 18 rules (seven as antecedent). The second most frequently occurring item was DES3, appearing in 11 rules (seven as antecedent). Key relationships andimplications are discussed in the next section.

Discussion and conclusions
Association rule analysis and graphing of the six SQD strategies (24 items) is able to demonstrate some of the complexity among digital competencies. Importantly, the current analysis only presents an initial descriptive discussion of relationships appearing among the SQD-strategies, which will guide future research and application. The main finding is that the four observed clusters are able to demonstrate some complex relationships among strategies. In the following section three key results will be addressed: the presence of Role Models, the power of Feedback, and importance of Instructional Design and Authentic Experiences.
First, Tondeur et al. (2018) found strong positive beliefs about experiences with Role Models items and less positive beliefs about Feedback. In the current analysis, all four clusters include Role Models items as antecedents, with ROL4 and ROL3 as consequents in Clusters 3 and 4. The high frequency of Role Model items in the rules suggests the importance of Role Models as a strategy. This also reflects results from other studies (e.g. Kauffman, 2015). In contrast, only one Feedback item (FEE3) is observed in the clusters, but it is a unique result. While not frequently occurring, FEE3, suggest a critical link between Clusters 1 and 2. This relationship demonstrates that if students feel positively about feedback on their developing digital competencies, they are likely to feel positively about support received to develop materials (DES3) and gaining experience using ICTs in classroom practice (AUT3). While less positive in this dataset, the importance of feedback across learning is well identified (e.g. Boulton, 2014). FEE3 is not frequently occurring across the four clusters, but it has a singular direct effect on Instructional Design and Authentic Experiences. Other rules all include multiple antecedents. This result suggests FEE3 has a more direct effect on pre-service teachers' developing competencies. In regard training, ensuring sufficient feedback about digital competencies may be a direct way to increase pre-service teachers' positive feeling about Instructional Design (DES3).
Finally, the most complex group of practices relates to students feeling positively about AUT3. Authentic experiences (AUT) can be interpreted as the most important strategy in the dataset, as they are the most frequently occurring in rules. Authentic experiences allow pre-service teachers to apply their knowledge of digital technologies, leading to a better understanding of links between theory and practice. AUT3, as an antecedent, suggests an effect in three clusters. In practice, AUT3 could be addressed individually and have a positive effect on experiences with several strategies. However, it was also the consequent in Cluster 2, which is very complex. The variety of strategies (as antecedents) presents a range of ways instructors may consider creating an experience that could result in a positive effect on Authentic Experiences. Given the high frequency of DES3 in Cluster 2 and its potential impact on AUT3, this could be an important area of focus.
The findings provide a way to view the complexity of digital competence strategies as a system, using association rules and directed graphs. A limitation of the study is that less frequently occurring relationships may be missed, but may be important. However, these are captured in the analysis, and can be explored separately. Future research should explore confirming these relationships and pre-service teachers' experiences, to validate clusters and relationships. Being able to isolate relationships provides a better understanding of preservice teachers' digital competencies and how they can be developed for future teaching practice.