Constructing a design framework and de- sign methodology from educational de- sign research on real-world educational technology development

Acknowledgements ative study, a design framework for developing real-world educational technologies and guidelines for conducting EDR are proposed. The design framework highlights four essential aspects – content, pedagogy, practice, and technology – that should be considered when developing educational technologies to ensure their educational benefits, feasibility, and successful real-world utilisation and adoption. The proposed guidelines for conducting EDR, such as exploring design alternatives and employing appropriate design construction and evaluation methods, can assist other researchers, including a single doctoral student, in embracing opportunities and overcoming the challenges that may

Educational design research (EDR) seeks to contribute to both practice and theory by developing solutions that improve educational practice and generating usable and generalisable knowledge. Most EDR researchers tend to focus on reporting their research contributions to educational practice. Therefore, there is a need for disseminating research that pays more attention to the theoretical contributions of EDR so that those outside a particular EDR project can benefit. This paper focuses on the theoretical contributions, particularly the design framework and design methodological knowledge, of a 6-year EDR enquiry that aimed to develop educational technologies that promote primary school mathematics learning and classroom practice. Informed by the literature and direct experiences of working in collaboration with teachers and various disciplines during this iter-to develop a solution to a similar educational challenge in another context, while design methodologies can help other researchers overcome challenges in conducting their EDR.
A growing body of EDR has responded to the increasing utilisation of technologies in educational environments. Recently, various technological solutions -for example, a digital video game (Lambert & Jacobsen, 2019), a mixed reality simulation (Cowling & Birt, 2018), and a virtual learning environment (Bergdahl et al., 2018) -have been developed and implemented in different educational contexts. Hence, it is beneficial to share usable and generalisable knowledge gained from previous EDR on technology-enhanced learning with others so that they can accomplish their future work in the area by building on what has already been learnt. As such, this paper reports the theoretical contributions of a 6-year EDR enquiry that aimed to develop educational technologies that promote primary school mathematics learning and classroom practice. Informed by the literature and my direct experiences of working in collaboration with teachers and various disciplines during this EDR on technology-enhanced learning, this paper seeks to further knowledge of: 1. Design frameworks: key aspects to be taken into account when developing educational technologies to ensure their educational benefits, feasibility, and successful real-world utilisation and adoption.

Educational problem
Linear equation solving, an important area in algebra, is often challenging for students at different educational levels to master (e.g., McNeil et al., 2019;Poon & Leung, 2010), particularly when they lack sufficient understanding of key concepts for solving equations, such as equations and equivalence (e.g., Knuth et al., 2006;McNeil et al., 2019). Conceptual understanding (i.e., understandings of mathematical concepts, operations, and relations) is one of the most important mathematical proficiencies (Kilpatrick et al., 2001). While strong conceptual understanding has several benefits for mathematics learning, insufficient conceptual understanding can hinder students' learning and performance in mathematics (e.g., Andamon & Tan, 2018;Kilpatrick et al., 2001).
Manipulative materials, such as beads and base-10 blocks, have long been used, particularly in preschools and primary schools, as hands-on learning tools that allow students to concretely explore abstract mathematical concepts through different senses. There is evidence that when manipulatives are used meaningfully, they can promote students' understanding of mathematical concepts. For example, when they are used for developing conceptual understanding (instead of attaining procedural fluency) and making links between various representations constructed through the manipulatives and mathematical symbols of the concept to be learnt (e.g., Kilpatrick et al., 2001;McNeil & Jarvin, 2007;Uttal et al., 2013). Nevertheless, disagreement exists between manipulatives' pedagogical benefits and classroom practice. Despite primary and lower secondary school teachers regarding manipulatives as useful learning tools, for instructional activities in their classrooms, they usually favour traditional teacher-centred and paperand-pencil instruction over manipulatives (e.g., Marshall & Swan, 2008;Toptaș et al., 2012). This finding implies that pedagogically sound manipulatives may not be adopted in the classroom, possibly because of classroom-practice-related reasons. Thus, there is a call for a study on the application of the theoretical knowledge of manipulatives for the promotion of manipulative use in the classroom to enhance students' mathematical concept understanding.

Study overview
This study employed an EDR approach to bridge the gap between research on manipulatives and its direct practical contributions to realworld educational challenges (i.e., the disagreement between manipulatives' pedagogical benefits and classroom practice). It aimed to investigate the use of manipulatives in real educational contexts and then to develop a research-informed manipulative that not only enhances primary school students' understanding of equation-solving concepts, but also promotes its utilisation and adoption in the classroom. This study was conducted from 2015-2020 and was my doctoral research. It was self-initiated and not part of any research project; thus, I conducted this 6-year EDR enquiry independently.
Drawing on the widely used EDR process proposed by McKenney and Reeves (2019, pp. 83-84), Figure 1 shows the overall process of this study. The study involved multiple iterations of investigation, design and construction, and evaluation and reflection. The research comprised three main phases (i.e., initial research, concept development, and design development), which were divided into six iterative subcycles. Although the process flow depicted in Figure 1 moves from left to right, the actual process was not linear, but rather iterative (i.e., results from one element repeatedly fed into others) and flexible (i.e., some sub-cycles were revisited).

2.2
Volume 5 | Issue 2 | 2021 | Article 38 4 Figure 1: Overall EDR process of the study The empirical study took place in primary and lower secondary schools in Finland. The uniform quality of the Finnish education system and its teachers, coupled with the students' homogeneous mathematics performance regardless of their socioeconomic background (Organisation for Economic Cooperation andDevelopment, 2017, 2020), enabled the study to be conducted in any school. Mixed methods research, which combines qualitative and quantitative research (see e.g., Creswell & Plano Clark, 2017), was employed for data collection, analysis, and interpretation to better understand the real-world complexity (e.g., Anderson & Shuttuck, 2012;McKenney & Reeves, 2019). Research ethics and integrity were assured; the study was conducted according to the guidelines of the Finnish National Board on Research Integrity (2009,2012) and All European Academies (2017). Altogether, 18 teachers (teaching experience 3-27 years), 98 primary school students (aged 9-12), and 65 lower secondary school students (aged 13-16) participated in different phases of the study. Table 1 summarises the mixed methods research design of the empirical sub-cycles (i.e., initial fieldwork, concept evaluation, and design evaluation). The research design and results of the initial fieldwork have been reported in detail in Lehtonen and Joutsenlahti (2017). The concept evaluation has not been published elsewhere; its research design is elaborated in Section 2.3.2 and results in Section 3.3. The design and development of the design solution (i.e., design principles refinement, technological development and implementation, features and interactions, prototyping, and future development) and its evaluation (i.e., research design and results) have been thoroughly reported in Lehtonen et al. (2020).

Research phases
Phase 1: Initial research Initial research was undertaken to gain knowledge of the following two key areas to construct a context theory: (1) the to-be-solved educational problem and the target educational context (i.e., challenges, opportunities, and needs regarding teaching and learning equation solving in primary school classrooms) and (2)

2.3
educational games for equation solving in terms of their key benefits and limitations to mathematics classrooms. Based on the literature review and existing manipulative analysis findings, fieldwork was conducted in real classrooms to define the problem in practice, understand the real context, and investigate how existing manipulatives support or hinder classroom activities (see Lehtonen & Joutsenlahti, 2017).
Class interventions were implemented with four primary school teachers and their students (n = 74) with no/low prior knowledge of equation solving. Students with different attainment levels were equally divided into either learning with paper-and-pencil (n = 25) or learning with manipulatives (physical manipulative: n = 25, virtual manipulative: n = 24). The teachers were interviewed before the class interventions about their prior experiences and needs in using manipulatives, and after the interventions about their experiences and opinions of the interventions. All the interventions were observed and video recorded. Afterwards, all the students completed the same paper-based test with no access to the manipulatives and evaluated their learning experiences and achievements.

Phase 2: Concept development
This phase aimed to generate alternative design concepts informed by Phase 1 results, evaluate the generated concepts, and select promising one(s) for further development (see Ulrich & Eppinger, 2016). I used the Phase 1 findings to identify design opportunities and tentative design principles that provided initial ideas about how a manipulative can promote students' equation-solving concept understanding and its use in the classroom. Based on that, I explored different potential solutions by first generating ideas and then reviewing each from not only rational and analytical viewpoints, but also instinctive and intuitive viewpoints, as recommended by McKenney and Reeves (2019) and Ulrich and Eppinger (2016). Four generated ideas were selected for development as potential design concepts. Then, I built a nonfunctional mock-up to describe each concept in terms of its key functional features and initial visual appearance.
I conducted a concept evaluation with 12 primary school teachers (four participated in the initial fieldwork) via questionnaire and interview to determine the potential viability of each envisioned concept in the target setting and to collect teachers' feedback for further design decisions. First, I introduced all the concepts to the teachers and asked them to assess how well each concept was likely to benefit students' learning and conform to classroom and school practice. A scale of 1 (not at all) to 4 (very well) was used. They were also asked to provide an explanation of their rating responses. Teachers were selected to evaluate the concepts because they were sufficiently knowledgeable about educational contexts to envision concept implementation and adoption possibilities. Furthermore, according to McKenney and Reeves (2019), it would not be socially responsible to excessively interrupt normal classrooms for evaluation of underdeveloped solutions.

Phase 3: Design development
This phase aimed to develop the selected concept(s) according to the design principles, evaluate the design, and reflect on the evaluation results (see McKenney & Reeves, 2019;Ulrich & Eppinger, 2016). To inform the development of the design solution, I conducted another literature review (e.g., technology-enhanced learning and tangible technologies) and investigated relevant educational products (e.g., textbooks and educational technologies). Subsequently, I incorporated key knowledge underlined by the literature, the existing solution investigation, and Phase 2 concept evaluation results to refine the design principles. Based on the concept evaluation, the most promising concept was selected for further development, guided by the refined design principles. When developing the selected concept, I also took into account the teachers' positive feedback from Phase 2 about other concepts to incorporate their strengths into the developed solution.
Consequently, physical artifacts and class activities were developed as the solution to the educational problem.
The next step was prototyping. As I have a strong background in design (i.e., educational, graphic, user interface, and product design), I built most parts of the working prototypes myself and designed the graphics and interface of a tablet application that was part of the developed manipulative. I sought and received expert information technology and communication assistance for prototyping the application's technological aspects. Finally, the manipulative (i.e., the tablet application and related physical objects) was developed and prototyped in collaboration with a team of six university students and their supervisor at the Faculty of Information Technology and Communication Sciences as part of their coursework.
The development team and I took part in the development and formative evaluation of the manipulative prototype according to our own expertise and discipline. The development team was responsible for technology implementation and the iterative process of programming and use-case scenario testing, while I was responsible for educational, graphics, interface, and product design. Continual discussion was required for issues that required the expertise of both parties. The collaboration involved daily communication (e.g., email, and Slack: a realtime communication platform for teamwork) and monthly face-toface official meetings. Unfortunately, at the end of the 4-month collaboration, only some components of the working prototype fully functioned as envisioned. Moreover, the technical stability of the prototype was heavily dependent on external factors, particularly classroom lighting conditions. Consequently, Microsoft PowerPoint was used to build a Wizard of Oz rapid prototype (see Beaudouin-Lafon & Mackay, 2012) of the application for class intervention evaluation.

2.3.3
I conducted a field test to evaluate the developed solution in real classrooms in terms of its pedagogical benefits, usability, and compatibility with classroom and school practice (see Lehtonen et al., 2020). The class interventions were implemented with two primary school teachers and their students (n = 24) with no/low prior knowledge of equations solving. Students with different attainment levels were divided equally into two groups: learning with the developed manipulative (n = 12) or learning without it (n = 12). Apart from that, both groups used the same components of the developed solution (i.e., a teacher guide, a worksheet, and class activities) during the class interventions. The processes for the intervention and paper-based posttest were similar to those of the Phase 1 initial fieldwork. Additionally, the students who used the developed manipulative individually participated in a thinking aloud session, in which they were asked to solve equations with the manipulative and simultaneously explain about their actions. They also completed a questionnaire and were interviewed regarding their perceptions of usefulness and usability of the manipulative. Lower secondary school students (n = 65), who had learnt equation solving as a part of their normal school curricula (representing traditional classrooms), took the same paper-based test, but without participating in the class intervention. To investigate the extent to which the developed solution enhanced students' learning achievement compared to traditional instruction, the paper-based test performance of the students participating in the class interventions was compared to that of the students who did not take part. After the interventions, both teachers evaluated the developed manipulative using a similar process to those of the Phase 2 concept evaluation. The manipulative was also evaluated by one special education teacher and three lower secondary school teachers to examine the possibility of its implementation and adoption in educational settings outside the studied context.
Based on the results of the formative evaluation during the working prototype development, some physical components that interacted with the application were revised to improve the technical stability of the manipulative. The field test results and reflection informed minor refinement of the manipulative to enhance its pedagogical value, usability, and practicality.

Developing a design solution
This section describes how the design solution was developed to meet the research objectives (i.e., to enhance students' equation-solving concept understanding and conform to classroom and school practice), what informed its development, and whether it accomplished the desired objectives.

Design principle construction
The initial research results from Phase 1 revealed important pedagogical factors that tend to influence the successful manipulative utilisation and adoption in the classroom (Lehtonen & Joutsenlahti, 2017). Based on this, tentative design principles were identified and used to guide potential design concept exploration. It should be noted that different practical factors that could pose challenges to manipulative utilisation and adoption were also discovered. They were not the main consideration when generating design concepts, but were used later to evaluate the generated concepts and to further develop the design solution.
In terms of pedagogy, discovery learning was identified in the literature (e.g., Bruner, 1961;Neber, 2012) as important to students' meaningful learning with manipulatives. Thus, the manipulative was designed to help students learn equation solving through firsthand experience with guidance and scaffolding. To ensure the benefits of the manipulative to students' conceptual understanding, it was also designed based on the literature (e.g., Joutsenlahti & Kulju, 2017;Lesh et al., 1987) and Phase 1 initial fieldwork to help students link various representations of equation-solving concepts (i.e., physical actions, pictures, mathematical symbols, and natural language) and to express their mathematical thinking through multiple modes. Additionally, informed by social constructivism, the manipulative was designed to encourage students to work in pairs/small groups, thereby co-constructing their knowledge through peer interaction (e.g., Slavin, 2010;Vygotsky, 1978).
Regarding to-be-learnt content, the manipulative was designed to help students learn equation solving by concretising fundamental concepts for solving equations: mathematical equivalence (i.e., both sides of an equation are equal), different terms in an equation (i.e., constants and unknowns), and equation solving (i.e., finding the values of the unknowns so that the equation is true or showing that there is no real number-value solution to the equation). These were emphasised in the literature (e.g., McNeil et al., 2019;Otten et al., 2019;Poon & Leung, 2010). To achieve the subject-matter objective, the manipulative employed the balance model (see e.g., Otten et al., 2019) to emphasise the relational (rather than operational) meaning of the equal sign and used different physical objects to represent constants (e.g., 3) and unknowns (e.g., 3x).
Empirical findings from the initial research indicated that the physical manipulative provided more benefits (e.g., concrete representation, tactile-kinesthetic interaction, flexibility, and ease of use) than the virtual manipulative. Nevertheless, the virtual manipulative had several advantages over the physical one, for example, visual and symbolic links, real-time guidance and scaffolding, and interactivity. The discovered strengths and limitations of both manipulative types were taken into account when exploring different possibilities. Consequently, four potential design concepts were generated ( Figure 2).

Concept design
All four concepts share similar core ideas informed by the Phase 1 findings, as described in the previous section. The concepts differ in how they utilise existing technologies to meet the design objectives. Hence, the generated design concepts were not driven by technologies, but rather by what technologies could offer to solve the educational problem. This design approach prevented the technology before pedagogy effect, which occurs when a technology is chosen prior to the identification of an educational problem (Watson, 2001;cf. technology-driven products;Ulrich & Eppinger, 2016).
Concept A (Figure 2a) consists of a physical balance scale and physical objects: black boxes as unknowns and base-10 blocks as constants (e.g., yellow units representing ones and green rods representing tens). Each side of the scale represents each side of the equation, while the balance or tilting of the scale represents the relationship (i.e., equality or inequality, respectively) between the mathematical expressions on each side of the equation. Concept B (Figure 2b) consists of a physical balance scale with a digital display, which shows math sentences of the current equation-solving process, and physical objects. Concept C (Figure 2c) consists of a tablet application, a mirror placed in front of a tablet camera for physical object detection, physical objects, and a mat divided into two parts representing both sides of the scale. Math sentences of the current equation-solving process

3.2
and graphical images of the physical objects currently on and off the scale are displayed on a tablet screen. Concept D (Figure 2d) consists of a tablet application and physical objects. The tablet touchscreen directly detects physical objects on the screen and provides outputs (i.e., graphical images and mathematical symbols) for seamless interaction.

Concept evaluation
Each generated concept was evaluated by teachers in terms of its potential pedagogical benefits (i.e., enhancing students' understanding of equation-solving concepts, discovery learning, social interaction, and multimodal expression of mathematical thinking) and compatibility with classroom and school practice (e.g., acquisition budget, storage space, organisation and preparation, and class management). The evaluation criteria were informed by previous studies (e.g., Bedir & Özbek, 2016;Marshall & Swan, 2008) and Phase 1 fieldwork. To take policymakers' needs into account, the pedagogical criteria were also guided by the current Finnish National Core Curriculum (NCC) for Basic Education (Finnish National Agency for Education [EDUFI], 2016).
The teachers (N = 12) rated each concept according to the evaluation criteria, and the average scores for each concept are shown in Figure  3. All the concepts were rated relatively high (3-3.5 on the 4-point scale of 1 [not at all] to 4 [very well]) in terms of their potential benefits for students' learning. However, only Concept D was highly rated (M = 3.5, SD = 0.78) for its compatibility with classroom and school practice compared to the others, which were rated below 3 (well). At the end of the concept evaluation, the teachers were asked to make an acquisition decision by taking both pedagogical and practical factors into account. Most teachers (n = 9) would acquire Concept D for their own class, followed by Concept B (n = 6), Concept A (n = 5), and Concept C (n = 2). No teachers mentioned that they would not acquire Concept D for their class, but six would not acquire Concept C, and four would not acquire Concept A or Concept B. According to the teachers' explanations, there were no major differences between the concepts regarding their pedagogical benefits, so compatibility with classroom and school practice was the decisive factor in the teachers' acquisition decisions. Concept D was the most desirable because of its highly perceived pedagogical benefits, as well as its simplicity, usability, compactness, portability, durability, compatibility with existing school tablets, attractiveness to diverse learners (in terms of age and attaining levels), and enduring utility. Hence, Concept D was selected for further development.

3.3
Figure 3: Average scores of each concept rated by 12 teachers regarding how well they potentially meet each criterion. Scores ranged from 1 (not at all) to 4 (very well).

Solution design and development
Based on the refined design principles (e.g., concretising key equationsolving concepts; being in agreement with curriculum; supporting multimodality, discovery learning, and social interaction; and being feasible for classroom and school practice; Lehtonen et al., 2020), physical artifacts (i.e., a manipulative, student worksheets, and teacher guides) and class activities (how the manipulative and worksheet should be used in the classroom) were proposed as the solution to the identified educational problem. The manipulative was developed from Concept D as a student learning tool to be used for recommended class activities to ensure meaningful learning. Relying heavily on the literature mentioned in Section 3.1, the class activities were designed to promote students' understanding of equation-solving concepts through discovery learning, social interaction, and multimodal mathematical thinking expression. It is recommended that, under the teacher's supervision, students use the manipulative in pairs/small groups to model and solve an equation shown on the worksheet and then individually write the equation-solving processes and a solution(s) on their own worksheet.
The instructional materials include two student worksheets (the first for lower graders to learn to solve equations by substituting values for the unknown and the second for upper graders to learn to solve equations algebraically [ Figure 4]) and two corresponding teacher guides. The instructional materials were developed based on the NCC (EDUFI, 2016), mathematics textbooks currently used in Finland, and the materials used during Phase 1 class interventions. The materials were evaluated and critiqued by the doctoral research supervisors acting as experts in subject matter and mathematics education and then refined according to the experts' advice. Figure 4: Extracts from the student worksheets of a fourth grader (above) and a fifth grader (below).
Each worksheet was designed to be used for one 45-minute lesson to facilitate students' conceptual understanding by working with multiple representations of equations. The worksheets show a summary of the contents and then eight equations presented through three different representations: mathematical symbols, pictures, and word problems. On the worksheets, students are required to first translate each equation into two other representations, then symbolically, pictorially, or verbally demonstrate how they have solved the equation, and finally provide the answer. The teacher guides were designed to assist teachers in planning and implementing a lesson for learning to solve equations using the manipulative and the worksheet. Each teacher guide consists of five parts: a story that introduces the equation solving, lesson objectives, suggestions for the lesson procedure, advice on how to explain the content to students, and additional information about the theoretical framework behind the class activities.
During the collaboration with the development team, the manipulative was developed based on the refined design principles and technological feasibility. Figure 5a shows the developed manipulative, including the tablet application and two types of physical objects: (1) specially designed X-Boxes representing unknowns and (2) existing and widely used base-10 blocks representing constants (see Lehtonen et al., 2020). The use of base-10 blocks was positively supported by the teachers during the concept evaluation, because teachers and students are usually familiar with them and they are commonly available in schools. The application has two levels containing the same exercises as in the worksheets for students at different grade levels to learn to solve equations. This expands the manipulative utility across primary grades to increase the likelihood of manipulative acquisition. Students manipulate physical objects on a tablet screen to model and solve equations, while the application provides corresponding guidance and scaffolding in textual, pictorial, mathematical symbolic, or audio form. The graphics and user interface design of the application was kept simple, allowing students to easily navigate the application and concentrate on the content. While the design of the manipulative was much the same as the original concept, some trade-off decisions were made in the prototype to balance pedagogical, practical, and feasible factors, all of which were not achievable concurrently. For instance, at the beginning of the prototyping, it became clear that the original object-tracking idea (i.e., conductive technology) used in Concept D for the interaction between the physical objects and the application was not feasible due to the limitations of current tablet touchscreen technology. This led to a change from the original object tracking to image recognition via an external USB web camera (Figure 5b). A more complicated installation of the manipulative was required, which resulted in reduced practicality. However, at that time, it was technologically feasible and affordable. Importantly, it allowed for a single point of interaction (i.e., for students to manipulate physical objects on a tablet screen and look at the outputs on the screen without losing concentration); this was a critical design principle, which was informed by Concept D that should not be traded off. Another decision was to simplify a graphical representation of the balance scale on the tablet screen to overcome the limited screen space (see the original scale in Figure 2d vs. the simplified one in Figure 5a). Despite the fact that the simplified representation was not completely in line with physics, based on teachers' responses during the Phase 3 design evaluation, the simplification was acceptable for primary and lower secondary school mathematics.
Overall, the proposed solution gained highly favourable results from the design evaluation (Lehtonen et al., 2020), thereby suggesting its successful classroom utilisation and adoption. The paper-based test results, class intervention observations, and students' and teachers' responses revealed that the design solution improved students' equation-solving learning and achievement. Moreover, based on the class intervention observations and the students' responses, the developed manipulative was easy and enjoyable to use. Most students found it helpful for their learning and would like to use it in the future. Teachers rated the manipulative highly (M = 3.4 on the 4-point scale, SD = 0.48) for its compatibility with classroom and school practice. All teachers would acquire the manipulative for their classrooms, and the number of acquired manipulatives would depend on their utilisation purpose (e.g., for the whole class or remedial teaching) and school budget. Based on the design evaluation and reflection, minor revisions (e.g., redesign of the X-Box to overcome image recognition challenges) were made; new features (e.g., free experiments with the balance scale for enhancing pedagogical value) were added to the forthcoming design.

Conducting EDR
This section highlights the lessons I learnt from conducting the EDR. Based on my research journals, field notes, and communication records with the development team and teachers, here I reflect on the benefits of EDR and the challenges that emerged during my study.

Iterations
As iteration is a key characteristic of EDR (e.g., Anderson & Shattuck, 2012), the study comprised six sub-cycles. The multiple iterations of investigation, development, assessment, and refinement helped me gradually develop my theoretical understanding of the real educational context (i.e., challenges and opportunities regarding utilisation of manipulatives for teaching and learning equation-solving concepts in primary schools) and how my proposed solution could enhance students' learning and classroom practice. Moreover, through this iterative process, I was able to design, test, and refine the solution to ensure its feasibility and successful implementation and adoption in the classroom. Despite the benefits, the iterations were intensive and required considerable resources. As a single researcher, it took me 6 years to complete the study.

Data triangulation
I collected empirical data from various sources (i.e., teachers and students of different grade levels) using a range of methods (i.e., multiple instruments and various data formats), as recommended in the EDR literature (

Various participants
In line with the EDR literature (e.g., Ørngreen, 2015), the involvement of teachers and students, who were potential implementors and learners of the proposed solution, largely informed the solution development towards the desired outcomes. The concept and design evaluation results indicate that the developed solution is likely to promote students' conceptual understanding of equation solving and classroom practice. Therefore, it has the potential to be implemented and adopted into real classrooms. The design evaluation also revealed students' adaptations of the manipulative use to meet their needs in ways that would not have been discovered without their involvement.
Collaborating with teachers and students not only contributed to the advancement of theoretical knowledge and the improvement of educational practice, but also benefitted teachers and students. Most teachers stated that participation helped them realise how manipulatives can enhance students' conceptual understanding. One even changed her strong prior perception that manipulatives only benefit young students. Consequently, all teachers mentioned their intention to regularly incorporate manipulatives into their classrooms. After the class interventions, one class actually adopted a discussion about their own mathematical thinking with peers, which was implemented during the class interventions as part of their mathematics class. Some teachers expressed their appreciation for the opportunity to participate, stating that participation enabled them to experience how different instructional methods influenced their students' mathematics learning and to understand how to best support their students.
Despite these benefits, I encountered challenges in involving teachers and students. Participant recruitment required careful planning and determined actions for practical and ethical reasons. It was difficult to access teachers who were willing to participate in class interventions that required intensive organisation and resources. Approval for the students' participation had to be obtained well in advance from the city's children and youth service director, school principals, and students' guardians. Additionally, unanticipated rearrangement or cancellation of participation occurred several times. tion. We collaborated successfully due to good communication, mutual respect, and shared understanding. Regular face-to-face and online communication enabled us to interact and share ideas and work progress. In short, our complementary expertise enabled us to overcome the research and design challenges.

Multidisciplinary collaboration
Nevertheless, I encountered some difficulties in the multidisciplinary collaboration. It was challenging for other collaborators to gain insight into the complex educational problems, and it took me almost a year to establish the collaboration with the development team. Another challenge resulted from the experience level of the development team members: undergraduate and graduate students. Most had no experience with the technologies used for the developed manipulative, thereby requiring training and additional study. Moreover, because the students participated in the study as part of their coursework without other incentives, it was difficult to secure their full-time commitment to our collaborative project.

Technological innovations
Technological possibilities play an important role in product design and development (e.g., Ulrich & Eppinger, 2016). While the developed manipulative that employs innovative technologies provided satisfactory results, the development of such educational technology resulted in several challenges. For example, only at the beginning of the manipulative prototyping did it become clear that the envisioned objecttracking idea was not feasible with off-the-shelf technology. Consequently, a considerable amount of the development team's time was devoted to finding other object-tracking alternatives, leaving less time for actual prototyping. Moreover, there were other technical difficulties in getting the prototype functioning properly. Due to limited time and the demanding task, the development team could not develop and construct a fully working prototype that was stable and contained all the needed features for the class interventions, despite the extended project deadline. Thus, I had to build the Wizard of Oz prototype for class intervention evaluation instead.

Alternative designs
Given my design experience, I acknowledge that working with alternative designs reduces the possibility of discovering later in the development process that the developed design might not be the best solution (e.g., Ulrich & Eppinger, 2016). Thus, I thoroughly explored alternative solutions at an early stage of concept development (Phase 2). The concept evaluation with teachers not only helped me better understand the educational context but also identify the issues to be addressed prior to further development and class interventions. This resulted in the efficient utilisation of resources. The teachers' feedback also allowed me to confidently select a concept for further development.

4.6
Additionally, the evaluation of different concepts was a good method for collecting in-depth information from teachers. During the initial research, when teachers were asked to recall their experiences or provide their perceptions of manipulatives, their responses were rather superficial. However, during the concept evaluation, the concepts worked as concrete examples of manipulatives and enabled teachers to provide more detailed and relevant responses to similar questions.

Solitary researcher
Without an accompanying research team, I was involved in every process of the EDR and had multiple roles. On a positive note, I had a deep understanding of the whole process. However, inevitably, this was also challenging due to the complex nature of EDR and the scope of the research, which required considerable time, patience, and multidisciplinary expertise from a single researcher. With a multidisciplinary background and without a restricted timetable, I conducted the study mainly independently over a 6-year period. However, the study still suffered from limited human resources. It required a multidisciplinary theoretical foundation, which I did my best to master. As the only researcher, I could operate only one Wizard of Oz rapid prototype at a time during the class interventions, thereby resulting in a small sample size of students working with the developed manipulative. Moreover, I was only able to carry out the concept evaluation and one implementation of the developed solution in classrooms. It is unlikely that a single implementation of the proposed solution in a real educational setting is sufficient to collect evidence indicating the success of the solution. Thus, I have planned postdoctoral research that could contribute to the research validity by constructing a number of working prototypes of the refined design solution that can be implemented in other educational contexts with larger sample sizes.
Working alone also had a negative impact on the research reliability. It was not possible to achieve researcher triangulation during the data collection and analysis. As cautioned in the EDR literature (e.g., Plomp, 2013), taking on multiple roles (i.e., researcher, designer, and evaluator) also challenged my maintenance of objectivity. It should be noted that I did not take on an implementor's role in any class interventions; teachers were the designated implementors. Thus, there was no researcher or designer influence on any class interventions. An early developed design is rarely flawless, and there is always room for improvement. Therefore, I saw the design evaluation as a means of gathering feedback to improve the proposed solution rather than demonstrating its perfection. Furthermore, I wanted to ensure that the research was EDR, not just a design and development project. So, I consciously acted according to Edelson's (2002) recommendation as an EDR researcher (instead of only a designer) to develop a novel solution to improve educational practice and use design implementation as a strategy to construct theoretical knowledge that makes EDR different from design practice (Easterday et al., 2017).

Looking back and moving forward
Informed by the literature and my experiences of developing the design solution (Section 3) and conducting this EDR (Section 4), I have constructed usable and generalisable knowledge of design frameworks and design methodologies. I am sharing this knowledge so that others outside this EDR project may benefit from it.
Design framework for developing real-world educational technologies Brown (1992) Figure 6 shows that the framework takes into account four essential aspects -content, pedagogy, practice, and technologythat contribute to educational benefits, feasibility, and real-world utilisation and adoption of educational technologies.

Content
Undeniably, an understanding of to-be-learnt subject matter content is compulsory for the development of any educational solutions. For 5.0

5.1
example, to develop the manipulative and the student worksheets, which covered what was to be learnt, I needed knowledge of equation solving, important concepts required for understanding equation solving, different models for teaching equations, and equation-solving content stated in the NCC (EDUFI, 2016) and used in textbooks.

Pedagogy
An understanding of pedagogy (i.e., how to teach and learn the particular subject matter content in the target educational context) is required to ensure the meaningful use of the proposed design solution.
For example, I needed to know how to use manipulatives to enhance students' understanding of equation-solving concepts. Social constructivism (e.g., discovery learning and social interaction), multimodal expression of mathematical thinking, and teaching and learning approaches recommended by the NCC (EDUFI, 2016) were used to guide my design solution.

Practice
Knowledge of practice in the target educational context largely contributes to the successful real-world implementation and adoption of educational solutions. In my case, the conformity of the manipulative to classroom and school practice (e.g., acquisition budget and class management) proved to be an important factor in teachers' acquisition decisions.

Technology
To design feasible educational technologies, it is necessary to know about technological possibilities, including what technologies are available, what they can offer, and how they work. For example, to develop the manipulative in collaboration with the development team, to some extent, I needed to understand digital technologies, tangible technologies (see Ishii & Ullmer, 2012), and different object tracking alternatives that could be used to solve the target educational problem.
It is worth noting that the importance of each aspect in the framework is usually not equal and largely depends on the nature of the educational problem, target setting, and possible technologies. In line with McKenney and Reeves (2019), during my study, design decision-making typically involved simultaneous consideration of multiple aspects. Often, I had to make trade-off decisions to fine-tune the manipulative that would best achieve the research objectives (Ulrich & Eppinger, 2016). For example: Concept D had the most pedagogical value but was not technologically feasible; Concept C was technologically feasi-ble but not very practical or pedagogical; and Concept B was pedagogical and technologically feasible but not affordable. My theoretical, practical, and technological understandings gained from the literature, fieldwork, and multidisciplinary collaboration assisted me in making trade-off decisions.
Unintentionally, there are some similarities between the real-world educational technology design framework and the Technological Pedagogical Content Knowledge (TPACK) framework (see Koehler & Mishra, 2009). The TPACK framework builds on Shulman's (1986) construct of pedagogical content knowledge and includes technological knowledge to address essential teacher knowledge for integrating technology into teaching and learning, whereas my proposed design framework is rooted in design and development practice to inform important aspects to be considered when designing educational technologies for the real world.

Guidelines for conducting EDR
The nature of EDR (e.g., being situated in real educational settings and involving multiple iterations) results in a multifaceted and intensive enquiry requiring substantial resources (Kelly, 2013) and long-term endeavour (Collins et al., 2004). Consequently, doctoral students are often hesitant about employing EDR (Goff & Getenet, 2017;Herrington et al., 2007). Based on my experience of conducting my doctoral EDR, I agree with scholars (e.g., Herrington et al., 2007;Kennedy-Clark, 2013;McKenney & Reeves, 2019) that EDR can be undertaken even by a single doctoral student. I certainly encourage others to engage in EDR for its numerous benefits, as evidenced in my study.
Various EDR models, such as those by Easterday et al. (2017) and McKenney and Reeves (2019), have been employed; however, due to the uniqueness of each piece of EDR, researchers are typically required to adapt these models appropriately. Therefore, instead of proposing another model, I provide general guidelines (Figure 7) for conducting EDR to assist other researchers in embracing opportunities and overcoming the challenges that may emerge from their EDR.

Balance research scope and available resources
Because EDR is resource-intensive, it is important to estimate the resources available and then plan the EDR accordingly. This will ensure that the research is achievable and preserves the real-world iterative enquiry. I agree with Kennedy-Clark (2013) that a single researcher or a few researchers should aim for a less intensive small-scale enquiry, while a larger research team could strive for a more intensive largescale one. In addition to the research team size, the research team type (e.g., monodisciplinary vs. multidisciplinary) and other resources (e.g., time and budget) should be taken into account when planning EDR. For example, financial support can play a crucial role in prototyping technological solutions, as in my study. Complex and long-term research can also be broken down into feasible components, for example, doctoral and postdoctoral research as in my and Goff's (2016, as cited in Goff & Getenet, 2017 cases or several small-scale studies for doctoral students to individually conduct, as recommended by Anderson and Shattuck (2012). Alternatively, results from a single iteration can be used to inform further research, as in Di Biase's (2020) case.

Sensibly collect and use data
The triangulation of data, as recommended by EDR scholars (e.g., McKenney & Reeves, 2019) and evidenced in my study, can contribute to a better understanding of multifaceted real-world phenomena and the trustworthiness of EDR. However, an endeavour to triangulate data often requires intensive resources to collect a significant amount of data (Collins et al., 2004). Due to limited resources, researchers, particularly doctoral students like Goff (2016, as cited in Goff & Getenet, 2017) and me, often select only the data that is clearly relevant to their research questions for analysis. It would be wise if the data collection and analysis were thoroughly planned and implemented by taking into account resource use and data triangulation. Moreover, aligning with the recommendation of Collins et al. (2004), sharing collected data (i.e., open data) can undoubtedly advance the research community.

Engage different research participants
The success of the design solution in the real world depends largely on various people who are directly and indirectly involved in its implementation and adoption. The engagement of different stakeholders in EDR has proven indispensable for my study and those of others (e.g., Bergdahl et al., 2018;Cowling & Birt, 2018). For instance, it can be vital for understanding the complex problems in real-world settings, developing a solution to meet the needs of those involved, and achieving respondent triangulation. Thus, I agree with McKenney and Reeves (2019) that different types of participants, including direct users (i.e., teachers and students) and other relevant stakeholders (e.g., schools, parents, and policymakers), should be appropriately engaged in different phases of EDR. In line with Herrington et al. (2007), stakeholder participation should aim to benefit both the EDR (i.e., scientific and practical outputs) and the participants (i.e., societal outputs). Moreover, participation often requires intensive and long-term collaboration, thereby yielding possible difficulties in recruitment and execution for practical and ethical reasons. Thus, the collaboration should be carefully planned and implemented by taking into account relevant issues, such as research permission, social responsibility (e.g., interruptions to the participants' normal activities), and suitable timing for all involved.

Collaborate with other researchers and disciplines
It is possible for a researcher to undertake EDR alone, as in Di Biase's (2020) case. However, solving multifaceted problems or developing complex solutions, such as technological innovations, often requires a larger number of people from different disciplines. I support the recommendation of other scholars (e.g., Kennedy-Clark, 2013; McKenney & Reeves, 2019) that a multidisciplinary research team can improve the feasibility, rigour, robustness, and trustworthiness of EDR. For example, my research certainly benefited from the skills and expertise of my supervisors (in mathematics and pedagogy) and the development team (in computer science). Good teamwork (e.g., a shared vision and understanding, strong group cohesion, and respect for others) and communication can promote successful collaboration (McKenney & Reeves, 2019).

Explore alternatives, build, and test sensibly
It is essential in the design field to explore alternative ideas before selecting promising ones for further development (e.g., Ulrich & Ep-pinger, 2016). However, only a few EDR researchers have explicitly reported working with alternative solutions (Lehtonen et al., 2019;Ørngreen, 2015). Exploring alternative solutions in the early stages, as evidenced in my study, can help to ensure that the solution developed with considerable resources is the best answer to the educational problem (McKenney & Reeves, 2019;Ørngreen, 2015). Moreover, I agree with Easterday et al. (2017) that researchers should employ construction and evaluation methods appropriate for their theoretical knowledge level and design stage to use resources efficiently. For example, early in my study, the mock-up of each concept was simply built and evaluated only via teacher interviews and questionnaires to identify the issues to be solved and to quickly exclude unsuccessful concepts. Later on, the prototypes of the selected concept were carefully built and evaluated by teachers and students through various methods to validate the efficacy of the developed theory and solution.
Be ready for changes EDR is flexible and adaptive in nature. It is conducted in complex realworld contexts full of variables, unlike laboratory settings. Moreover, unpredictable changes consistently occur during the iterative process of investigation, design, testing, and refining. Thus, the research design of later cycles usually needs to be adjusted based on the results from previous cycles (e.g., Herrington et al., 2007;Plomp, 2013). Often, there are other situations in which adjustments or changes are required. In my case, for example, a teacher withdrew from the field research on short notice, and the working prototype developed in collaboration with the development team did not function reliably for class interventions. Thus, I agree with Kennedy-Clark (2013) that EDR researchers should be prepared to react promptly to adjustments and changes.
While this paper appears to contribute to the design framework and methodological knowledge of EDR in technology-enhanced learning, the presented framework and guidelines were built on only one researcher's experience from a single EDR. Therefore, more similar studies by other researchers could help validate the results of this paper and advance the design framework and methodological knowledge of EDR.
Daranee Lehtonen is a doctoral researcher and instructor at the Faculty of Education and Culture, Tampere University, Finland. She has a background in design, particularly educational design, and primary school education. Her research interests include educational technology, technology-enhanced learning, digitalisation, and multimodality in different primary school subjects.

Author Profile
Author Details +ls