Design as critical engagement in and for education

This paper aims to outline an approach of critical engagement in educational research that moves beyond traditional formats of academic critique. The aim of our contribution is to explore the critical dimension of reflexive design for education and contrast it with recently advocated notions of design-based research. Rather than providing a full-fledged methodology, our intent is to motivate more critical design efforts in education and provide examples on how this could be done. Towards this end we outline a framework of critical research questions for design-based research and provide examples of how critical design could be practiced in education.


Introduction
Education and educational systems are artificial phenomena in the sense that they emanate from human intervention and effort.Irrespective of whether we look at policies, curricula, instructional measures, tools, networks, or environments, educational processes are essentially shaped by man-made inventions and artifacts.As a consequence educational processes are not uniform and lasting but contingent on the socio-material, and historical conditions in which they take place, or as Robert Ebel a former president of the American Educational Research Association put it: The process of education is not a natural phenomenon of the kind that has sometimes rewarded scientific investigation.It is not one of the givens in our universe.It is man-made, designed to serve our purposes and meet our needs.It is not governed by any natural laws.It is not in need of research to find out how it works.It is in need of creative invention to make it work better.(in Farley, 1982, p. 18) Given an understanding of education as a man-made phenomenon and the felt need for improvement, it is not surprising that there has been a growing interest in design-oriented approaches to education such as design-based research (e.g.Brown, 1992;Collins, 1992) and instructional design (e.g.Reigeluth & Schank, 1999).Even though forerunners can even be traced further back into the history of education and educational research, these efforts are particularly interesting as they make direct reference to the notion of design as a starting point for educational research.While we are sympathetic with the overall project of design-based research, especially in its attempt to increase the relevance of educational research (e.g.van den Akker, Gravemeijer, McKenney, & Nieveen, 2006), we are concerned that many proponents of design-based research and educational design have adopted a narrow notion of design and, willingly or not, building on a rather uncritical and positivist approach to education.In particular, we are skeptical about claims that instructional technology and design research are inherently socially responsible as they are directed towards problems related to learning and human development (cf.Reeves, Herrington, & Oliver, 2005).
While we do not want to discredit the intentions of those engaged in design-based research and educational design, we believe it is important to challenge the frequently adopted notion of design as an engineering process and broaden the perspective towards more recent models of reflexive design and design research.In doing so it becomes possible to move beyond merely instrumental accounts of (educational) technology and raise awareness for the political dimension inherent to any design effort in the field of education.In the next section we briefly summarize the guiding ideas of design-based research and challenge the frequently adopted notion of design as engineering.Against this background we then introduce an alternative model that foregrounds the reflexive dimension of design, discuss the routes it opens for critical engagement, and outline a framework of critical research questions for design-based research.To give an idea on how critical design in education could play out, we provide three case examples, highlighting different types of research questions.We conclude with an attempt to position reflexive design in the broader field of educational research approaches and pinpoint some of the more recent trends we believe to deserve more critical attention and could be approached in a reflexive design manner.

Design-based research and the engineering model of design
To motivate our interest in alternative forms of design research in education, it seems important to have a closer look at a prevalent conception of design in educational research, the engineering model, a model recurrently adopted by advocates of design-based research in education.
The origin of design-based research is often associated with the works of Brown (1992) and Collins (1992) and has been taken up by a variety of educational researchers and special issues of several well-known journals (cf.Anderson & Shattuck, 2012).Throughout the last two decades design-based research has been applied to the design, development and evaluation of various types of educational interventions, including programs, teaching-learning strategies, materials, products and systems (Plomp, 2009).As argued by Bell (2004), design-based research is not a unified methodology but rather a group of approaches, which are premised on the idea that the design and implementation of an educational innovation is a way to gain insight into the nature and conditions of learning.Barab and Squire (2004) offered a broad definition in that they characterized design-based research as "a series of approaches, with the intent of producing new theories, artifacts, and practices that account for and potentially impact learning and teaching in naturalistic settings" (p.2).Based on a review of design-based research efforts (Anderson & Shattuck, 2012) suggest that respective studies are characterized by (a) their situatedness in a real educational context, (b) their focus on the design and testing of a significant intervention, (c) the use of mixed methods, (d) the iterative advancement of the design in multiple iterations, (e) a collaborative partnership between researchers and practitioners, and (f) a focus on theoretical understanding and design principles.Further introductions to design-based research have been provided for example by van den Akker et al. (2006), Kelly, Lesh, and Baek (2008), as well as Plomp and Nieveen (2009).
Despite the positive reception of design-based research by many educational researchers, there has also been substantial debate on the methodological underpinnings of respective approaches (e.g.Shavelson, Phillips, Towne, & Feuer, 2003;Kelly, 2004).However, the focus of these debates has been on the scientific

2.0
rigor and validity of designerly efforts towards educational research, while the underlying conception of design has hardly been questioned.Also among the proponents of design-based research there seems to dominate a view that design, even though severely demanding from a practical perspective, is rather unproblematic from a methodological perspective.For example Plomp (2009) suggests that the design process in design-based research can be subsumed under more generic models of educational and instructional design: It is like all systematic educational and instructional design processes -therefore cyclical in character: analysis, design, evaluation and revision activities are iterated until a satisfying balance between ideals (‚the intended') and realization has been achieved.(p.13)Similar models have been suggested by Bannan-Ritland (2003), Reeves (2006), Middleton, Gorard, Taylor, and Bannan-Ritland (2008), or Hjalmarson and Lesh (2008).What these models have in common is the idea that design is best to be understood as a sequence of analysis, synthesis, and subsequent evaluation.According to this view, which is at the heart of an engineering model of design (cf.Löwgren, 1995), design is aimed to find a solution for a given problem from which the requirements and solutions can be derived.The need for an iterative procedure here is basically seen in the complexity of the problem, the lack of access to relevant information and social and human factors that effect the processing of the relevant information (cf.Middleton et al., 2008).Even though some proponents of design-based research such as Hoadley (2002) or Barab, Thomas, Dodge, Squire and Newell (2004) draw a more complex picture of the design process, the engineering model of design appears to be the dominant perspective.While engineering is explicitly used as a role model by authors such as Middleton et al. (2008) and Hjalmarson and Lesh (2008) others such as Cobb, Confrey, diSessa, Lehrer and Schauble (2003) or Burkhardt and Schoenfeld (2003) apparently equate design with engineering.
While to some extent such a perspective might appear obvious as it resembles the dominant instrumental paradigm in educational design (Visscher-Voerman & Gustafson, 2004), we deem it problematic, not only because it might give a misleading account of design as an epistemic process as suggested for example by (Gedenryd, 1998), but also as it undermines the critical dimension of design.In a nutshell our concerns toward the latter is threefold: 1.The engineering model of design assumes that problems are objectively given and provide static (even though not necessarily fully known) properties of the design space.However, such a position ignores that the articulation of a problem already implies that a conceptual framework and value system has been adopted against which a perceived situation appears to be problematic.Even participatory approaches fall short towards this end as they presuppose that future users and stakeholders can be identified at the beginning of the design process (cf.Ehn, 2008).
2. The engineering model of design runs danger of reducing education to a technological process with a clear separation of means and ends, in which the means are supposed to be neutral with respect to the ends pursued (e.g.Biesta, 2007).
Respective claims are most evident in the field of e-learning, where for example it has been claimed that standards for the description of Learning Object (IEEE LOM, 2002) or Learning Analytics as an abstract concept (Greller & Drachsler, 2012) would be "pedagogically neutral".While design-based researchers put focus on what works, the value-ladenness of the means they employ and their broader impact is not an inherent concern in this perspective.
3. The engineering model of design stipulates the idea that design ideally boils down to a series of well-described, discrete, rational, and structured methodological steps (cf.Löwgren, 1995).Such a perspective, however, ignores the fact respective methods are not innocuous, but have an impact on the interests articulated and the ideas emerging (cf.Cohn, Sim, & Dourish, 2010).
The above account of design-based research, which is in fact a multifaceted endeavor, is of course limited and might do an injustice to its proponents.However, we believe that in order to increase the relevance and impact of design-based research efforts, its crucial not just to overcome positivist accounts of research but also narrow understandings of design.

Reflexive design
Despite its popularity in the field of education research, the engineering model of design is by far not the only model of design.
In fact, an ongoing discussion of various design paradigms is not only evident in the field of design theory (e.g.Dorst & Dijkhuis, 1995;Bayazit, 2004) but can also be found in the fields of interaction design (e.g.Löwgren, 1995;Fallman, 2003) as well as educational design (e.g.Edmonds, Branch, & Mukherjee, 1994;Visscher-Voerman & Gustafson, 2004).While an encompassing comparison of the different paradigms and their systematization is beyond the scope of this paper, our focus in the following will be on a model of design that has alternatively been labeled as the "pragmatic account" (Fallman, 2003), the "situativity approach" (Visser, 2006), and an "evolutionary-systemic perspective" (Cronen, 2001).In the following we will call this approach, that has been recurrently been traced back to the works of Schön (1983Schön ( , 1987) ) and Dewey (1938), the reflexive model of design.
The reflexive model of design differs from the engineering model in that it starts from the premise that design is an inherently social activity embedded and mediated by the situation it arises from and aims to change (cf.Löwgren & Stolterman, 2004).Design from this perspective is not just an intellectual process, but a process embedded in and shaped by the material world in which it takes place (cf.Schön & Bennett, 1996).Furthermore, the reflexive model of design posits that the sequence of analysis, synthesis and evaluation, constitutive to the engineering model, is essentially misleading as it ignores the inherently epi- stemic nature of the design effort (e.g.Gedenryd, 1998).In a nutshell, proponents of the reflexive model of design, hold in common the following seven assumptions on design: 1. Design is creative and generative in that it produces new, often unexpected forms and substantially alters the situations it responds to (e.g.Winograd, 1997;Jonas, 2004); 2. Design is conscious and reflexive in that it is an intentional activity, even though concrete effects can hardly be predicted and are often up to processes far beyond the designer's control (e.g.Winograd, 1997;Löwgren & Stolterman, 2004); 3. Design is anticipatory in that it aims to envisage possible futures and to create new and viable options of action.Design not only responds to what is but also raises the question of what might or could be (e.g.Jonas, 2004;Zamenopoulos & Alexiou, 2007); 4. Design is focused on the "ultimate particular" (Stolterman, 2008), in that it aims to respond to a unique situation and aims to develop a solution with specific functions and characteristics, which might not work or even be relevant somewhere else or at another point in time; 5. Design is integrative in that it is confronted with the entire complexity of the situation it responds to.While the designer's focus might be limited to certain aspects of the situation or the product, no aspect of the situation can be bracketed once the product has been put into use (e.g.Jonas, 2004); 6. Design is a conversation, both with the stakeholders involved as well as the materials used.The ideas relevant to design not just exist in the designer's mind but are developed and tested in conversation with others, be it peers, clients, customers, sketches or prototypes (e.g.Schön & Bennett, 1996;Cross, 1999); 7. Design arises from a position of not-knowing and uncertainty in the sense, that both the situation the designers is confronted with as well as the change s/he wants to bring about are essentially uncertain and only take shape in the process of design itself (e.g.Zamenopoulos & Alex-iou, 2007).
In the field of educational design similar understandings of design have been suggested for example by Tripp and Bichelmeyer (1990), Rathbun, Saito, andGoodrum (1997), andWillis (2009).
To our understanding the design-based research work of Hoadley ( 2002) and Barab et al. (2004) also points into this direction.
In the following we argue that this kind of knowledge is not just instrumental or practical in nature, but can also help us to come to terms with and reflect on the values we pursue, to open up new spaces of action, and to foreground the matters of concern.

Towards a critical notion of reflexive design
As argued by Fallman (2003), in taking an active stance and trying to bring about intentional change, design-oriented research goes beyond simple critique of technology and technolo-

4.0
Volume gical development.The question however is in which sense reflexive design and respective research approaches could qualify as critical then and whether they are actually apt to overcome the blind spots inherent to the engineering model of design.Towards this end we will first discuss the notion of critical design and outline some of the ways it can be practiced as suggested in the literature on design research.
While authors such as Dunne and Raby (2001) aimed to position "critical design" as a distinct approach to design research, arising in opposition to what they call "affirmative design", Sengers, Boehner, David, and Kaye (2005), as well as Bardzell and Bardzell (2013) more recently developed broader and less dogmatic notions of critical design and research.We take these broader notions of critical design as a starting point as they avoid simplifying claims about the critical nature of a particular design and open up room for a variety of critical design practices rather than a unified approach.
Aiming to leverage different notions of critical thought as practical resources for design, Bardzell and Bardzell (2013) suggest the following definition: a design research project may be judged "critical" to the extents that it proposes a perspective-changing holistic account of a given phenomenon, and that this account is grounded in speculative theory, reflects a dialogical methodology, improves the public's cultural competence, and is reflexively aware of itself as an actor -with both power and constraints -within the social work it is seeking to change.(p.3304) Even though we do not claim that all forms of reflexive design and inquiry are necessarily critical in nature, they are, in contrast to the engineering model of design, at least in principle compatible with the characteristics suggest above.
First, according to the above definition, critical design entails a shift in perspective, providing an alternative explanation of the phenomena the design is concerned with.However such as shift implies that in principle there can be multiple equally reasonable perspectives at one and the same phenomena.Such a position is clearly at odds with the idea of problems as objectively given, but echoes the idea that the design space is actually (re-) framed in the design process (e.g.Schön, 1983).
Second, the focus in critical design is not on verifiable truthclaims but on thought-provoking interpretations that challenge taken for granted accounts.The emphasis hence is not on general technological rules and design principles as aimed for in the engineering model and respective research traditions, but on the search for promising alternatives and potential futures.
Proponents of reflexive design also stress that predictive statements are quite problematic, as design essentially aims to change the situation it responds to (cf.Löwgren, 1995).
Third, the definition of critical design suggests a dialogical approach to design in the sense that meaning making and learning arise from the "struggle, heterogeneity, and polyphony of human expressions and experiences" (Bardzell & Bardzell, 2013, p. 3303) contesting the idea of a privileged perspective the designer or someone else could take.Again, this perspective is closer to the reflexive model of design stressing that designers are not just information processors but engaged in an ongoing conversation with the design space, the stakeholders, as well as the material world (e.g.Fallman, 2003).
Fourth, critical design is supposed to foster the public's cultural competence in providing means and vocabularies for the skeptical analysis of prevalent ideas.In adopting an instrumental notion of technology the engineering model of design makes standard criteria for the assessment of technology such as its effectiveness, efficiency, accessibility, or attractiveness the essential hallmarks of excellence (e.g.Koper, 2007).Reflexive design in instead is aimed to develop sensitivities as well as nuanced vocabularies for the qualities the design products (e.g.Löwgren & Stolterman, 2004) and the unfolding practices.
Finally, critical design calls for a reflective awareness for the socio-cultural and epistemological conditions in which design takes place.Again such a position runs counter the idea of design as a rational process of problem solving, but calls for a situated assessment of the methods enacted (e.g.Löwgren & Stolterman, 2004).
A similarly programmatic account has been suggested by Sengers, Boehner, David, and Kaye ( 2005) who essentially argued that critical reflection on the values embedded in technology, and the practices it supports should be a central concern of design.Again, the principles they develop appear to be quite compatible with the notion of reflexive design, we sketched above.
Even though ideas pertinent to critical design can be traced back to a variety of predecessors (e.g.Sengers et al., 2005), a variety of "critical design practices" (Bowen, 2007) has been described in the literature.Instead of trying to give an exhaustive overview of this seemingly flourishing field, we restrict ourselves to brief comments on some practices that address the blind spots of engineering design and which we deem particularly promising in the field of educational design research.In doing so we aim to point out that critical design might be more an issue of style rather than dogma (cf.Duschlbauer, Lanz, & Hattmannsdorfer, 2012).
One of the practices that has recurrently been associated with critical design is the development of concepts or products to probe into social practices and underlying values (Sengers et al., 2005).The overall idea is to envision products that are technically feasible but appear unacceptable, useless or questionable under the given social and cultural conditions.While Dunne and Gaver (1997) described this kind approach as "value fiction" and provide examples on design artifacts that are supposed to trigger general debate, authors such as Bowen (2007) used critical artifacts as means to spur reflection and discussion in a participatory design context.
While the use of technology as a probe can be understood as way to foster reflection on the means we are using and challenge the premise of their neutrality, reflexive design can also be geared to demonstrate that improvements are not just technically but in fact possible and even desirable.Such a practice, which is akin to the idea of a design for micro-utopias (Wood, 2007), essentially aims to show that a difference can be made and that other objectives are thinkable and in reach.Respective practices not only provide a proof of concept but also give rise to new aspirations and contribute to a broader discussion of the objectives we strive for.
Finally, there are also practices of critical design that challenge traditional role models of the designer and strengthen the role of the user as competent practitioners.One of these practices is bricolage, which can be understood as a form of "immediate design", in which participants make creative re-use of available technologies -hardware, software, infrastructure, and combinations thereof -to devise new tools for themselves or others (cf.Büscher, Gill, Mogensen, & Shapiro, 2001).As an extreme form of participatory design it actively undermines the prevalent understanding of the user as a consumer but invites him/her to reflect on and make proactive use of the ready-at-hand.

Critical research questions for design-based research
Moving from an engineering model towards a reflexive account of design implies a change in the kind of knowledge claims deemed relevant.In fact, shifting from an instrumental to a critical stance gives rise to a set of value-rational questions, questions we rarely see in the foreground of design-based research efforts.In the following we outline a critical research agenda for design-based research, which extends the kind of research questions targeted by the engineering model of design.
Following authors such as van den Akker (1999), Goldkuhl (2004), Plomp (2009) and others, we assume that design-based research efforts are essentially aimed at the advancement of prescriptive statements in the form of design principles.According to van den Akker (1999) design principles are heuristic statements of the following form: If you want to design intervention X for the purpose/function Y in context Z, then you are best advised to give that intervention the characteristics A, B, and C [substantive emphasis], and to do that via procedures K, L, and M [procedural emphasis], because of arguments P, Q, and R. (p. 9) The arguments P, Q, and R thereby are supposed to be explanatory in nature, or as Plomp (2009, p. 18) put it, they provide an "understanding of the 'how and why' of the functioning of the intervention".In a nutshell a design principle hence entails a specification of the design intervention (WHAT), the expected results (OUTCOME), the presumed working principles (HOW), as well as the conditions deemed relevant (CONTEXT).Adopting the visual syntax suggested by Pawson and Tilley (1997), the generic structure of a design principle can be depicted as shown in Figure 1.From a research perspective respective design principles are however not the starting point, but at best the outcomes of a design-based research effort.The challenge of the research process hence is to eliminate or reduce the unknowns in the above schema.Depending on which elements are known or unknown we are facing different research questions.If we look at a design principle from an instrumental perspective, as suggested by the engineering model of design, we can identify four basic research questions: 1. We might have a design intervention and know about the context and the way the intervention is working, but do not know or are uncertain about the OUTCOMES.This question is at the heart of most efforts in evaluation.For example, we might test whether a given intervention has the intended effects and/or if it produces some unexpected side effects.
2. Conversely, even though we might know the expected outcome, and a working principle suitable to bring about the intended outcome in the given context, we might lack the knowledge of WHAT is to be done, the concrete characteristics of the intervention.The question here is how to instantiate an idea about what is supposed to work in a given context.For example, we might conceive an intelligent tutoring system, but lack the knowledge on how to implement such a system.
3.Even if we know the characteristics of the intervention, the context as well as the outcomes, we might have no proper idea on the underlying working principles, the HOW.Here the focus is on our understanding of what is actually going on.For example, we might ponder on the question whether improved learning outcomes are due to the novel didactic approach implemented or students' interest in using some advanced technology.
4. Finally, we might also lack knowledge of the CONTEXT we are operating in.The question here is on those 'background conditions' required for an intervention to take effect.For example, we might have limited knowledge on the students' meta-cognitive or technical skills to make effective use of intervention offered.
While the above questions are relevant to design-based research, irrespective of the model chosen, they however only focus on issues of instrumental rationality.As such answers to these questions do not tell us what ought to be done or what is the best to be done under given circumstances.To address these kinds of questions, we have to approach the design principle from a critical perspective, i.e. foreground issues of value rationality (cf.Goldkuhl, 2004).Again we can identify four basic types of questions.
1. We might ask whether the (expected) OUTCOMES of an intervention are in fact desirable.The question is how we can legitimate the ends we are pursuing in relation to values.For example, we might raise the question of how we can legitimate the constant monitoring of students in a learning environment in order to provide an intelligent tutoring system with the required information.
2. We might ask about the rules, conventions, and normative commitments in the CONTEXT the intervention is aimed at.
The question is geared towards the implicit and explicit values and convictions as well as the potential conflicts among those acting in the context we are operating in.For example, we might ask whether students, teachers and administration have similar ideas on the purpose of a certain educational program.
3. Additionally, we might ask HOW the design intervention effects the power relations among those involved.The question is concerned with the mechanisms of power triggered by a given intervention.For example, we might be interested to understand, whether a design intervention reinforces the students' role as someone in need of help or if it allows her/him to take an active stand.
4. Finally, we might also ask WHAT should be done.Here the question is which course of action seems preferable in light of the potential outcomes and effects on those involved.This question takes into account not only what is known about a particular intervention, but also about other options available to those in charge of the design process.
While the second set of questions reaches beyond the scope of a social science aimed to emulate the idea of a value free science, they are clearly within the scope of what Flyvbjerg (2001) has called a "phronetic social science" a form of social science with a focus on value-rational deliberation.Rather than conceiving design-based research merely as a kind of applied science, we believe it is at least equally important to consider what is good and bad regarding the interventions we are devising and the design principles we are suggesting.In this sense we are line with Goldkuhl (2004) who argued that design theories are in need of multiple forms of grounding and cannot be based on empirical findings and explanatory theory only.

Examples of critical design in education
To give a more vivid idea on how reflexive design in education can play out, we provide three case vignettes, highlighting different facets of critical engagement in educational design and research.The focus is on projects we have been actively involved in ourselves more recently.We do not claim that these are ideal examples of critical design, but deem them useful to give some idea on how different types of research questions might be addressed.Towards this end, the case vignettes are meant to be a trigger for discussion rather than an endpoint.

Case vignette 1: Performance independent ECTS-points
As part of the project "Diskurs auf dem Campus" (discourse on campus, http://www.diskurs.uni-kiel.de/),which was aimed to trigger reflection and discussion among students and faculty about their understanding of the university as an institution and the future role of learning and teaching, the project team inter alia organized an interactive event aimed to probe into personal motivations in an educational system, in which learning progress is rewarded with credit points.In the interactive event that lasted for a day, the project team had setup a counter in the entrance hall of the university's main building (see Figure 2).At this desk students but also faculty were invited to apply for so-called Performance Independent ECTS-Points as part of a fictional Empowering Creative Transformation System.To receive the credit points applicants had to fill in a form basically asking them to specify what they would invest these credit points in and how many credit points they thought this engagement would be worth.In a two step process, the applicants then had to explain there plans to the first counter clerk, while the second counter clerk read the proposal, increased without comment the requested amount of credit points and handed these over to the applicant in the form of little plastic balls 1 .

6.0
1 A more detailed documentation of the intervention is available at http://www.diskurs.uni-kiel.deEven though this project is not about educational technology in a narrow sense, it is essentially concerned with the role of credit points as a structural means widely used in higher education.In this sense, the issuing of an alternative credit system can be seen as form of critical artifact questioning the underlying assumptions of the currently used credit system and its impact on learning.In foregrounding the applicants' interests and competencies, the intervention hints at an alternative perspective on factors critical to learning.In being open-ended and deliberately ambiguous the intervention does not entail strong claims on how learning is to be organized, but simply insinuates the possibility of a different narrative.Asking applicants to share their plans in a fictional situation, the project team was interested to learn about the students' perspective rather than testing any specific hypothesis.Regarding the critical research agenda suggested above, the intervention was aimed to shed light on the rules, conventions, and normative commitments relevant in this particular context.

Case vignette 2: User driven analysis and reflection of knowledge creation processes
IIn the field of educational technology there has been a growing interest in Learning Analytics, i.e. the use of automatically collected usage data "to inform and support learners, teachers and institutions in better understanding and predicting personal learning needs and performance" (Greller & Drachsler, 2012, p. 42).As mentioned earlier in this paper, we are skeptical about the proclaimed neutrality of Learning Analytics as an educational means.Hence, when being asked to conceptualize an analytic tool for a computer-supported collaborative learning environment, we set out to identify the underlying premises of this emerging technical paradigm and compared these to a theoretical model of learning as knowledge creation (Richter et al., 2012).This conceptual juxtaposition revealed several contradicting claims: (1) While Learning Analytics assumes that patterns of behavior are more or less stable, a practice-oriented perspective holds that they are dynamically evolving.

6.2
(2) While Learning Analytics builds on the premise that significant processes can be inferred from users interaction with the system, the model of learning as knowledge creation assumes that any account of human behavior is necessarily incomplete and subjective.
(3) Rather than striving for general and statistically robust patterns, the knowledge creation perspective draws attention to situated occurrences and critical events.(4) Rather than superimposing normative assumptions about good or bad, productive or unproductive practices, the theoretical model stresses the importance of different perspectives and the relevance of personal and collective agency.Drawing on this analysis, we started to develop an analytic tool enabling users to make sense of the data themselves and in collaboration with others rather than confronting users with predefined queries and indicators.Figure 3 shows a screenshot of one of the implemented analytic tools, allowing users to explore into their collaborative efforts.
In contrast to the first case, which was aimed to explore into the rules, conventions and normative commitments within a given context, the emphasis in this case has been on what is actually desirable and what should be done.The theoretical analysis of recent efforts in Learning Analytics revealed that the underlying premises are by no means neutral and that the intended outcomes are not necessarily desirable, as they imply an imbalance of power between those in charge of the algorithms and the users.Furthermore, an attempt was made to develop a viable alternative to the dominant model of learning Analytics, an alternative aimed to enable students and teachers to reflect on their particular ways of working.Towards this end the implementation and testing of the analytic tools, can be understood as a proof of concept that analytic tools in education can move beyond normative accounts and empower people to reflect on and advance their own practices.

Case vignette 3: Personal learning environment
While the preceding cases where geared towards critical reflection on the context, the outcomes and the means available, this case vignette finally is focused on how the design process affects the relations of power among the stakeholders involved.Even though the notion of Personal Learning Environment (PLE) arose from a critical stance towards currently used learning management systems (Wilson et al., 2009) aiming to help learners take control of and manage their own learning, the development of such personal environments however often still draws on the traditional distinction between designers and users.Trying to challenge this dichotomy, students in a compulsory first-year course were asked to conceptualize, implement, evaluate, and revise a Personal Learning Environment (PLE) themselves.Adopting a bricolage approach, the students were asked not to devise new technologies but built their PLEs from off-the-shelf software and social media applications.Rather than focusing on technical development as such, the intent was to encourage students to creatively explore the potentialities of the technologies already available, and to trigger reflection on their own learning needs and practices.Figure 4 reproduces part of a storyboard a student group developed to communicate their idea.
The bricolage approach in this case not just challenges the distinction between the designer and user but also the notion that technologies are or should be used as envisaged by someone else.In doing so, it opens up room for multiple interpretations and ways of working rather than advocating a one-best-way model.In providing immediate experiences in the use of the emerging assemblages and technical mash-ups, the approach also stresses the dialogical character of the design process.Rather than asking how a certain technology can bring about some intended outcome, the intervention was aimed to shed light on the mechanisms of power entailed in the provision of an educational technologies and how these mechanisms can be altered.

Conclusions
The intent of this paper has been to show how reflexive design raises an additional set of questions and opens up new options for educational research.The concepts of reflexive and critical design, we sketched here, provide an alternative to the still dominant engineering model of design, in which analysis and synthesis are seen as distinct steps in a problem-solving process.Reflexive design instead assumes that neither the problem nor the possible solutions are given but are actually created in the process of design.Design in this perspective does not start from clear objectives, categories, and normative commitments but aims to figure out what is desirable and how we can make this come about.In this sense, design inevitably entails a moment of not knowing and uncertainty, which can only be overcome by an active transformation of the situation, or as Löwgren and Stolterman (2004, p. 9) put it "if the outcome can be predicted, it is by definition not a design process".However, design should also not be mixed up with blind action or mere trial-and-error as it is an intentional process that requires our commitment and that might fail, if we do not manage to figure out what is desirable, if we are unaware of the rules, conventions, and normative commitments relevant to the context we are acting in, if we ignore the mechanisms of power entailed in our design effort, or if we do not see what ought to be done.
As we have shown, such a perspective is also in line with recent accounts of critical design, which in turn offer a variety of novel practices to foster reflection on the objectives we aim at, the means we use and the processes we enact.While of course design-based research is not the only form of educational research, it can provide an inspiring addition in the multifaceted enterprise of educational research.As education is a man-made enterprise, we believe that it is essential that students, teachers, as well as researchers in all fields of education do not understand themselves as mere consumers and passive bystanders of the tools and systems they are supposed to use, but take a proactive stance, and develop a designerly attitude towards the contexts they are engaged in.Design-based research, from this perspective, is not primarily a form of applied science, aimed to deepen our understanding of learning and to devise respective means, but an effort to deliberate about the kind of education (Bildung) we deem valuable and worthwhile.
Figure 1: Generic elements of a design principle

Figure 2 :
Figure 2: Applicants for Performance Independent ECTS-Points at the counter in the entrance hall of the university's main building.(Photo: H. Allert, CC-BY-ND-4.0)

Table 1 : Summary of the Critical Research Agenda.
Table 1 provides a summary of the questions entailed in the critical research agenda.