Towards meaning-making with interactive visualisations

Learning analytics systems often use interactive visualisations to display information. Interactivity is typically used by visualisation designers to reduce cognitive load for users. However, our research suggests that interactivity holds significant value beyond the reduction of cognitive load. Informed by theories about meaning, perception and experience, we propose that interactive visualisations promote emergent meaning-making processes which should be accounted for in the design of interactive learning analytics visualisations. We present findings from a qualitative study of four teachers engaging with interactive visualisations in an Australian university. The study used a think-a-loud protocol and a semi-structured interview which were coded according to theory-informed constructs of dimensions of meaning and interaction opportunities. Our findings suggest that interactive visualisations that had been designed with regard to meaning-making stimulated users to engage more deeply with the data and explore it at different resolutions, from overview to detail. The interactive visualisations afforded more opportunities for users to gain understanding and insights they found meaningful. While this is a small study, we argue that it opens up promising avenues for further investigation; provides a practical approach to gather further useful data about interactivity and meaning-making; and suggests new principles that may be helpful for designers of interactive learning analytics visualisations.


Introduction
Visualisations transform or map data to visual properties such as position, size, or colour so we can take advantage of the human visual system to understand and make sense of complex data.Interactive visualisations (e.g., zoom, filter, search, etc.) have become a popular design strategy to visualise high-dimensional data (Yi et al., 2007), addressing a key limitation of static visualisations by allowing users to select which data are displayed and how they are presented.
Learning Analytics (LA) systems often use visualisations to deliver insights to different stakeholders (Verbert et al., 2020).However, there can often be a gap between the intended and actual use of a 'one-size-fits-all' visualisation design (Bodily & Verbert, 2017;Jivet et al., 2018;Sedrakyan et al., 2019).This can result in problems such as users not engaging with visualisations, misaligned perceptions of metrics, or user information needs not being addressed effectively.This paper highlights the significance of using interactions to modify visualisation properties in order to facilitate meaning-making for the user.We argue that improving this meaning-making process increases the likelihood of the user finding meaningful information and helps facilitate interpretation when engaging with interactive visualisations.
To understand how users perceive visualisation's properties researchers have typically drawn on cognitive science for insights about improving working memory through guiding a user's attention and managing cognitive load (Ware, 2004).Researchers map cognitive processes that are likely to occur while interpreting a visualisation to create a predictive model of how users might perceive visualisation properties and patterns.This has resulted in a range of different models and guidelines about visual properties that are easy to perceive and use working memory efficiently (e.g., Padilla et al., 2018;Ware, 2004).
However, perception alone does not lead to meaningful information, as users need to make meaning from what they perceive for those perceptions to be useful.Perceiving a visualisation's property might afford the user an opportunity to make meaning, but their ability to seize this opportunity also relies on previous knowledge (Norman, 1993) perceptions are contextual.Meaning-making always occurs within a context of knowledge, feelings, emotions, environment, and socio-cultural factors.Thus, meaning-making is an interactive and ongoing negotiation between artifacts of meaning and a person's social experience (Wenger, 1998).
Although current research acknowledges perception is not the same as meaning (Hullman & Diakopoulos, 2011;Sengers & Gaver, 2006), there is a tendency to assume that specific meaning naturally flows from specific perceptions.For example, a western designer without knowledge of other cultures might assume that perception of the colour red means 'wrong' or 'error' or 'stop' (despite the perception of red holding different meanings in some other cultures).In visualisation design, there is commonly a focus on guiding users to perceive visualisation properties while expecting those properties to transmit a specific intended meaning held in mind by the designer.This assumption is fraught, and open to the impact of cognitive biases and does not necessarily account for the data storytelling involved in guiding users' navigation of visualisation (Echeverria et al., 2018;Hullman et al., 2013).This paper argues that meaning cannot be found in visualisation properties.Meaning emerges from the meaning-making process afforded by the interaction between the perceptions coming from visualisation properties with the user's personal experiences and interests (Wenger, 1998).This underpinning idea centres on designing opportunities for users to interact with visualisation's properties enabling them to craft perceptions that support their meaning-making process.
This proposition has been investigated in a program of research addressing the research question: What are the implications of considering meaning emerging from interactions instead of being embedded in visualisations' properties for the design of interactive visualisations?
In this paper, we present a component of this work in the form of a case study.Four teachers at an Australian university were invited to participate in a meaning-making process while engaging with a system of interconnected visualisations.The system was composed of two or more interactive visualisations linked together providing multiple views while querying student reflective writing data.The students' reflective writing was the result of engaging in a task of reflecting on their own learning within units (subjects) that the teachers had taught.The interactivity in each visualisation was designed to provide teachers with the opportunity to navigate different levels of granularity to compare or associate data appropriate to their own perspective and experience (Sedrakyan et al., 2019).

Case study -Designing interaction opportunities for meaning-making
Figure 1 shows the dashboard that teachers engaged within the case study.The dashboard was developed and designed by the first author using the D3.js1 visualisation library in Typescript.Code for these visualisations is available on GitHub2 and an open-source library for developing visualisations for reflective writing.
We use the term interaction opportunities to refer to interactive elements in a visualisation that affords a user the opportunity to explore data and make meaning based on their individual needs.They are not merely eventresponse computational interactions, instead, they serve to enable users to craft perceptions that are likely to support a meaning-making process (Sengers & Gaver, 2006).Interaction opportunities are based on the concept of coupling that argues that computational interactions (e.g., click, zoom, drag, etc.) cannot be studied individually as users are likely to group computational interactions to achieve an aim (Dourish, 2004).Users are likely to make meaning not only from the final state of visualisation properties but also from the process of modifying visualisation properties.In this study, we explore interaction opportunities for comparison and association as tasks usually required by teachers (Sedrakyan et al., 2019) and transition as recommended for interactive visualisations (Hullman et al., 2013).Dervin & Naumer (2018) describe transmission theories and communicative theories as the two prominent foundational bases for designing interfaces.Transmission theories usually attached to perception research focus on asking whether users are perceiving what they are expected to.Data is encoded in visualisation properties in ways that are expected to hold and transmit meaning.On the other hand, communicative theories usually attached to meaning research focus on asking how humans understand, interpret, and find insights.Under this view, meaning emerges from meaning-making as the interaction between perceptions and the user's personal experience.Designing interaction opportunities aligns with communicative theories in the sense that aim to offer users opportunities to interact and craft perceptions that better balance and align with their experience instead of transmitting information.The designer animates important elements

Users' actions activate animations on most elements
To explore the effect of designing interactive visualisations following interaction opportunities on users' meaning-making, this case study provided a dashboard for teachers in a reflective writing learning context at an Australian university.Reflective writing is a common approach to help learners make sense of previous experiences and how these can be used in the future (Shum et al., 2022).Reflection in learning is profoundly personal and based on subjectivity beyond cognitive processes Learners in reflective writing can recall emotions, feelings, activities, and the environment of a specific moment by reading the reflexive text.
Most research on reflective writing revolves around supporting authors to make sense of their reflections in either an educational (e.g., Gibson et al., 2017;Knight et al., 2020) or professional context (Shum et al., 2022).However, teachers can find meaningful insights when exploring the reflections of their students.One major barrier teachers face when exploring students' reflections is the amount of reflexive text they need to read to be able to quickly gather insights about the group or individuals to adjust or respond to learner ideas.Thus, this case study focuses on providing a system of interconnected visualisations to support teachers to make meaning of their students' reflective writing.
The visualisations use data from the GoingOK platform for reflective writing3 which offers a large quantity of reflective writing data, including pseudonyms, timestamps, group codes, reflexive text and reflection state points (0 -distressed to 100 -soaring).GoingOK arranges students in group codes (units/courses) where they can submit many reflections.
The data collection involved two steps to investigate how teachers made meaning supported by interaction opportunities in visualisations.First, a think-aloud component where teachers narrate why they are doing specific actions as they interact with the interface.This standard protocol for usability testing allows researchers to access the participant's cognitive processes.Second, a semi-structured interview after the interaction with the system of interconnected interactive visualisations.The semi-structured interview has at its core the qualitative aspects of the concept of meaning.Ethical approval was obtained from the University Human Research Committee (UHREC) at the Queensland University of Technology (QUT) with approval number LR 2021-4788-5838.Written informed consent was obtained from participants before starting data collection.
We adopt Martela & Steger's (2016) view of meaning as having three components: 1) coherence as the cognitive component to understanding experience, 2) purpose as the goal-oriented component and 3) significance as the value worth component.This evaluation was selected as human-computer interaction research leans to meaning in life psychology research to investigate the concept of meaning (e.g., Mekler & Hornbaek, 2016).Interview questions such as "Were there any interactions that helped you understand the data?If so, which and how?" were used to investigate the impact of interaction opportunities on meaning-making, while questions such as "Do you think the dashboard's experience is likely to change what are you going to do with the insights?If so, how?" provided opportunities to explore meaning-making to the middle to long-term.
Participants interacted with the interactive visualisations for as long as they desired while thinking out loud.
Once finished, the semi-structured interview gathered insights into the underlying intentions, motivations and meaning of the interaction opportunities that occurred in the interactive visualisations.Data familiarity effects were not minimized as meaning-making acknowledges the importance of the relationship between the user and the data (Feng et al., 2018).
Participants were current users of the GoingOK platform and current lecturers or sessional academics at an Australian university with the role of group administrators to access students' data.There were four participants: 1 male and 3 female, 2 from the School of Education and 2 from the School of Information Systems.To preserve the anonymity of the participants, they appear as T1, T2, T3 and T4.The small number of qualitative interviews was reasonable given the in-depth analysis and exploration of complex data and early exploration.
The visualisations displayed students' reflective writing grouped by unit (group code) and interaction opportunities (comparison, association, and transition).Figure 1(a) shows screenshots of comparison interaction opportunities where teachers can add/remove groups changing the data of all visualisations.Figure 1(b) shows an example of association interaction opportunities where teachers can click a reflection point to create a line connecting all the reflection points of the student, filter the reflexive text below and highlight the specific reflexive text clicked.Transitions interaction opportunities are triggered anytime an interaction occurs to create awareness of change.

Findings
The think-aloud data along with the semi-structured interview were divided into sentences based on natural silences.Each sentence was coded based on the meaning components and the interaction opportunities used (Table 2).The analysis of the data consisted of finding how the meaning components were related to interaction opportunities and how this supported teachers in finding meaning in the students' reflective writing.Meaning was indicated when the participants indicated that they could see coherence, identified purpose, or made a statement indicating something was significant or valued.This relationship was established by finding code cooccurrences where interaction opportunities that were designed for comparison, association and transition cooccurred with indications of meaning-making coded for coherence, purpose or significance.The researchers identified co-occurrences (e.g., coherence and comparison appearing in the same sentence) and then verified by reading the content to establish if the relationship supported teachers in finding meaning (Table 2).From the analysis, the following four themes emerged, 1) comparisons were a starting point for meaningmaking, 2) interaction opportunities afforded fast and easy access to meaningful information, 3) building a coherent narrative was the first step in meaning-making and 4) interaction opportunities afforded significant self-evaluation.
Comparisons give positions but the meaning is in the details.
Teachers agreed that comparison and association interaction opportunities are not mutually exclusive, rather both are required to have a complete picture in navigating and exploring students' reflections.T1 said, "The numbers were important, but only because they gave me a position to the individual and I found it much more interesting or much more reassuring perhaps, or I needed to hear the story that we have with the number".T4 added, "In that scatter plot density plot [timeline] as well, I could actually see when more reflections being done and seeing that aggregate actually gave me an idea of what to look for when I was doing the looking at individual reflections".T2 mentioned, "Both [overalls and specifics] are equally important.Because it's really useful to see the trends but on there, they are meaningless".
These comments highlight the importance of successfully combining comparison and association interaction opportunities.Teachers use comparisons to be able to find outliers or information that does not match what they were seeing during their classes.T2 highlighted, "When I just look at it as one big block like I don't know if I've seen everything that I should, or understand everything that I should, and being able to zoom in and get different views I think is really useful".T3 mentioned, "So the big picture was interesting because it tells you overall how people were feeling, but the small picture becomes most important when there are sort of outliers".
Teachers initially found it challenging to find coherence as they found some positions (reflection state point in the timeline) disturbing and started to make assumptions (searching for significance) about the underlying reasons for those positions.However, the ability to find further information (increase in purpose) provided enough materiality for teachers to have a more coherent understanding of the position supporting the meaningmaking process to find meaningful insights about the student's reflections.
Figure 2 shows how teachers were able to visually compare reflection points and locate outliers.The interactive visualisations allowed selecting an outlier to connect all the reflections of a student giving an overview of their narrative over time.

Figure 2. Comparison and association interaction opportunities to find and connect outliers
Association interaction opportunities enable faster and easier access to meaningful information.
Association interaction opportunities provided computational support highly regarded by teachers.Teachers agreed that the information presented by association interaction opportunities could be found manually.However, computational support allowed teachers to inquire about the data from different perspectives faster.T1 mentioned, "That's why it was fantastic to be able to say click on a scatter point [student reflection in the timeline]... based on whether that you know, one of the two extremes soaring or distressed.And then read what was the context.If I had simply gone to the responses there would have been a lot of reading".T3 added, "Because of the interactivity and the other two graphs and the scatter one [timeline]...I was able to find more meaningful information more quickly and therefore it had more meaning to me purely because I had access to it".T4 supported, "Actually clicking in the timeline and seeing the relevant reflections, that really helped".
Association interaction opportunities become crucial in understanding the students' context; what they said, when they said it and the reflection point associated with the reflection text.All these links are present in the data.However, the visualisations allow the teachers to interactively find these associations, whereas, without interactions teachers struggle to connect reflection points with their reflection text.
Association interaction opportunities such as clicking a reflection state point to highlight the specific reflection text, provided the computational support to enable access to fine-grain detail data, which is complex and timeconsuming to find manually.As a result, the teachers were able to navigate from a top-level perspective to specific groups/students to drill down into the student's reflection text to read all the details.It is important to highlight that teachers often want to make meaning of students' reflections.However, the meaning-making process without computational support becomes difficult due to time scarcity.Most teachers agreed that student's reflections can be a valuable source of feedback for them, but the meaning is not easily accessible without computational support.Thus, interaction opportunities not only allow teachers to find coherence, purpose and significance but also faster and easier.
Figure 3 shows the student's reflexive text filtered by the selection made in Figure 2.These visualisations provide an overview of the student's reflections and connect a reflective state point with its reflexive text.

Figure 3. Student's reflective data overview.
Coherence is the most salient meaning component while interacting with interaction opportunities.The analysis revealed that interaction opportunities connect to each of the meaning components (coherence, purpose and significance).However, interaction opportunities strongly connect to the coherence component.
Coherence is the cognitive component that relates to understanding patterns of experiences.Coherence translates into how interaction opportunities helped teachers to understand the data.
The coherence component appeared first in the discussion as teachers initially tried to make sense of the data.T1 said "Somebody on the 17th on a Saturday, so a day they're not at uni presumably but sitting down doing the work not coping but very shortly afterwards presumably.Yeah, ok, so you know things can change quickly".T4 stated "Let me have a look, 49 users, 275 reflections.Pretty sure we had more students than that.So, I'm assuming we had few people actually writing reflections.That's interesting".Teachers started to build a coherent narrative that connected what they saw on the visualisations with what they experienced in the classrooms.
The semi-structured interview revealed the underlying reasoning behind the coherence made.T1 mentioned, "I will be interested in that person because that person is representative of the whole group with their distressed", T2 added, "It would be great to see how they're going overall rather than just I guess maybe jump to conclusions when you see one instance of distress" and T3 said, "So when you've got someone who's really unhappy with the way you've taught or if there's someone who's really happy with the way you're taught or something, you kind of, those are the ones that you want to look at".Teachers tried to make sense (coherence) of outliers so they can evaluate (purpose) their teaching practices, which is important (significance) for them.
Interaction opportunities support teachers to self-evaluate their practices.
The future-oriented interview questions revealed how the interaction opportunities can support teachers to evaluate their teaching practices.When teachers were asked, "Do you think the dashboard's experience is likely to change what are you going to do with the insights?If so, how?" T3 said, "That would change how I teach based on getting a better picture of when students were struggling so that I could reflect on why that might be given other kinds of milestones in the unit [course]".T2 mentioned, "You could say sure there's this many people who are distressed but unless you actually know if they're distressed because you as a teacher have not provided them with what they needed or said something offensive.That's very different to seeing someone's actually got major health issues going on".
Teachers' practices are significant to them but finding specific feedback from a general overview is very difficult.T4 said, "Like if I look at student feedback that comes to me via, you know surveys or something like that, that tells me how students are feeling about stuff, I can interact with them in class every day and I can, you know, hear their voice or see their face and see how tired they are, how confused they are...The real strength of GoingOK [reflective writing platform] is actually being able to see their thoughts in a way that they're willing to put down on paper".Interaction opportunities helped teachers to find personalised meaningful feedback, instead of trying to know in advance which feedback will be meaningful in general for teachers.T2 acknowledged the potential advantages of the visualisations as follows "Honestly in higher education when we have these units [courses] with so many students in them, we can't provide individual feedback to every student.And we also can't get to know every student and how they're going.Yeah so a tool like this would be extremely useful I think in higher education".This personalisation gives a sense of meaning when analysing the data leading to meaningful learning and self-actualisation.

Discussion
Bodily & Verbert (2017) demonstrated that LA traditionally evaluates visualisations based on usability metrics such as task time or completion that have little relationship to the actual purpose of visualisations.Jivet et al. (2018) provide a comprehensive review of visualisations evaluation in LA research and recommend evaluation be extended to include understanding and interpretation of the data, fulfilment of goals and impact on motivation, in addition to usability metrics.This study answers that call by including an evaluation that focuses on meaning components such as coherence, purpose and significance and introduces interaction opportunities as a framework for interactive visualisations that promote users' meaning-making.
The main difference between this study and other information visualisation research is that we ask the question of what visualisations can mean when users have the opportunity to bring their experience and engage in interactions, instead of what the visualisation says.We care about what a pattern means to the user rather than how a pattern was identified.This subtle difference gives us access to meaning components that are not usually explored such as coherence, purpose and significance.Most of the literature on visualisations focuses on how users perceive certain visualisation properties that lead to pattern identification (Padilla et al., 2018;Ware, 2004).However, being able to perceive patterns does not mean that users can interpret the patterns within their context.In this sense, this study is based on the tension existing in the meaning-making process between what has been perceived and the user's personal experience (Wenger, 1998).Considering meaning-making, it is fruitful to design interaction opportunities as it gives room for the user to bring their experience to the table.
The findings of this study open the discussion of what role can computational interactions play in designing interactive visualisations considering meaning-making.Traditionally computational interactions were seen as a means to reduce cognitive load and working memory (Norman, 1993).Consequently, most of the research targeting computational interactions is either focused on creating taxonomies for design or studying the effect of a single computational interaction such as search, click, drag, etc. (e.g., Feng et al., 2018;Yi et al., 2007).However, the findings suggest that computational interactions can have a greater role to play such as promoting the meaning-making process.
The role of visualisations can be thought of as a curated environment of modifiable properties that afford users opportunities to engage in a meaning-making process.The modification of properties allows users to construct perceptions that facilitate the construction of a coherent narrative, discovering possible purposes and finding significant information.Users when interpreting visualisations are going to use all the resources available to them within visualisation and outside of it (Zittoun & Brinkmann, 2012).This means that users do not depend on a single computational interaction, rather they use a combination that suits their needs causing a coupling effect (Dourish, 2004).Thus, the need to design interaction opportunities instead of focusing on taxonomies or individual computational interactions.
Designing interaction opportunities at its core aims to support users to explore the data interactively supporting their meaning-making process and crafting an environment where meaning is more likely to emerge.The explorative tools come from users' needs, as such the users in this study are teachers and LA literature argues that teachers want to compare and associate the data representing their classrooms (Sedrakyan et al., 2019).Users in other contexts might have different exploration needs.Thus, designing for interaction opportunities means discovering how users want to explore the data instead of assuming what users need to know.
Interactions in visualisation can give users a sense of control over the data (Buja et al., 1996).Designing for interaction opportunities gave the same sense of control to teachers to explore the data but more importantly, enabled fast and easy access to meaningful information.Teachers initially tried to make a coherent narrative of the data they were seeing considering their experience in the classrooms.If something was not aligned with their personal experience, teachers had the power to interactively explore the underlying reason by navigating from an overview to a fine-grained detail instead of relying on assumptions.This meaning-making process supported by technology created a sense of meaning as teachers decided when an overview or a fine-grain detail was needed.The role of technology was to support them instead of telling them what information was important.As a result, teachers had a high sense of coherence, purpose and significance leading to a high sense of meaning.This outcome was evidenced when teachers used interaction opportunities to self-evaluate their practices in a way that other feedback methods such as surveys do not support.A high sense of meaning has proved to be a reliable factor in adoption as a shift occurs from efficiency in early usage to meaningful usefulness in the middle to long-term usage (Karapanos et al., 2009).
Designing visualisations based on interaction opportunities enables a personalisation in meaning-making that is likely to address common LA problems.For instance, teachers decided what to see and when to see it clarifying any unclear metric or seeing the metric just when needed providing support for different needs.Although teachers found interaction opportunities useful, they used them in different ways as there were minimal constraints on when to use one interaction or another.The guidance that teachers were given in the visualisations was where interactive elements were and their functions.However, there was no indication or suggestion on when to use them.Finally, teachers agreed that the visualisations would be extremely helpful in current higher education settings suggesting that most teachers would find it easy to adopt the visualisations.

Conclusions
This case study presents some limitations given the qualitative approach.Qualitative research is often not able to create generalisations.The reflective writing context provides an initial interest from teachers that may have an impact on the data exploration.Teachers had an interest in exploring the student's reflective data.Further research will develop a case study focussing on students to explore if interaction opportunities can improve the learning experience of students involved in reflective writing.
This paper presents the first case study of a work-in-progress that provides results on the effect that interaction opportunities can have on teachers' meaning-making.The results highlight the importance of enabling teachers to explore the data and find insights that are relevant to them based on their experience.The ability to explore and navigate the data from overview to fine-grained detail provided teachers the opportunity to craft a coherent narrative, discovering different purposes and finding significant information.Teachers in their exploratory effort tried to compare and associate the data with or without computational support.However, the need for computational support for these tasks becomes evident when the quantity of data overcomes the teacher's time.Current higher education structures with units and classes with many people enrolled make this computational support a priority.So far, designing interaction opportunities appears to promote meaning-making processes and results to date suggest that designing comparison, association and transition opportunities can guide the design of computational interactivity to stimulate meaningful learning in a reflective text context.

Figure 1 .
Figure 1.System of interconnected visualisations with interaction opportunities to support teachers to make meaning from their students' reflections.

Table 1 . Differences between designing for transmission and designing for interactive opportunities.
Table1explores how comparison and association techniques are commonly used by designers when exploring data.Designers decide on the users' behalf what is meaningful and encode that meaning into visualisation's properties.However, interaction opportunities hand data exploration decisions to the user; the designer's role is to facilitate opportunities for meaning-making.