Design-based research in economic education

Dieter Euler University of St. Gallen Switzerland Michèle Collenberg University of St. Gallen Switzerland Tobias Jenert University of Paderborn Germany This article describes the phases of a design-based research (DBR) process using the example of an economic education project. The project was carried out in the „urban economics“ subject area for economic education in upper secondary schools (15-18 years). The research questions, research design, and central research results are described in detail. Finally, the experiences from the project are incorporated to reflect the DBR‘s specific performance characteristics. economic education secondary school ethics


Presentation of the problem
Two basic methodological orientations can be distinguished in empirical classroom-based research and they can be designated as impact-based and design-based research (Euler, 2011).While impact-based research in the form of empirical-quantitative classroom-based research already has a long tradition, design-based research has only become refined as research theory and practice during the past three decades.
Design-based research was mainly developed in response to criticism in terms of the lack of practical application of the findings from empirical-quantitative impact-based research.Numerous contributions document that many scientific findings from this tradition remain irrelevant, inaccessible, or incomprehensible for educational practice (see, for example, Euler, 1996;2007;2009).In view of this criticism of empirical impact-based research, the question that arises relates to design-based research's special efficiency for generating scientific findings and shaping teaching practice.Using a concrete DBR research project on the development of ethical-reflective competences in economic education as an example, this question is dealt with and examined in the following steps: • The following section 2 first outlines DBR as a research methodology that starts with a different initial question and promises specific research approaches and results.
• Section 3 presents a concrete DBR project in the field of economic education and outlines -in detail -the research questions, the research design, as well as the key research results.
• Section 4 of this paper presents the experiences from the project and reflects on the requirements for a DBR.

Starting points: Major characteristics of design-based research
DBR's starting point is not whether an existing teaching arrangement is effective; instead, it examines how a desirable goal can best be achieved in a given context through an intervention that still has to be developed.The goal is to find innovative practical solutions to unresolved problems; in other words, the goal is not only to investigate existing realities, but also to explore future opportunities.DBR is therefore not primarily interested in an approach of "proving that…", but rather in "exploring and testing what…".This research is characterized by the aspiration to link the development of innovative solutions for practical educational problems with the acquisition of scientific knowledge.DBR aims at contributing to the development of "innovative educational environments" (Brown, 1992, 141) and simultaneously developing practice-relevant theories."Such research, based strongly on prior research and theory and carried out in educational settings, seeks to trace the evolution of learning in complex, messy classrooms and schools, test and build theories of teaching and learning, and produce instructional tools that survive the challenges of everyday practice" (Shavelson et al., 2003, 25).Accordingly, DBR is defined as "the systematic study of designing, developing and evaluating educational interventions (such as programs, teaching-learning strategies and materials, products and systems) as solutions for complex problems in educational practice, which also aims at advancing our knowledge about the characteristics of these interventions and the processes of designing and developing them" (Plomp, 2007, 13).DBR's research and development process predominantly consists of defining characteristic phases.Although the numerous process models by various authors (see McKenney & Reeves, 2012, 73;Reinking & Bradley, 2008, 67 ff.) differ in their number of phases and notional descriptions, their basic structures are quite similar.The following model outlines the basic course of a DBR process and identifies the desired results for the individual process phases (see Euler, 2014, 20): documented."One of the distinctive characteristics of the design experiment methodology is that the research team deepens its understanding of the phenomenon under investigation while the experiment is in progress" (Cobb et al., 2003, 12).Defining the relationship between science and practice in terms of mutual interaction is a key element of the approach.In the course of forming, testing, and applying the scientific theory development, stakeholders from academia and practice can pursue their different goals and interests in cooperation with each other (see Euler, 1994, 239).Their interests differ in that: • science is primarily interested in collecting and studying viable theories, whereas creating a field of practice is secondary; • practice is primarily interested in developing solutions for problems, which are considered relevant and urgent, while it regards developing and formulating suitable everyday theories as secondary.

Research questions
The research project outlined below addresses a currently unresolved question of economic education.Especially after the financial crisis in 2008, the economic and social sciences, as well as business practice, focused increasingly on the question of the ethical foundations of economic activity (see Datar, Garvin & Cullen, 2010).This also includes an emphasis on the fact that economic issues are inevitably embedded in social, political, and cultural contexts.An understanding of economic activity therefore requires the inclusion of the contexts in which it takes place.One of these contexts consists of the values and norms that underpin the economic actions of politicians, companies, and private individuals.
This assumption leads to new challenges for economic education: Which goal orientation should an economic education adopt so that it integrates the ethical dimensions of action?Which teaching arrangements are suitable for promoting such goals?
The challenges were to be taken up -within the context of a DBR project -for teaching at Swiss secondary schools and addressed through concrete research and develop- ment activities.The project was based on the following, still general objective: Future economic citizens should gain competence to reflect on the ethical foundations of economic facts and to draw conclusions for their own actions (ethical-reflective competence).This objective resulted in the following key research questions: 1. Which concrete learning objectives characterize econo mic education that integrates ethical-reflective competences in upper secondary schools?
2. How can these learning objectives be promoted within the context of concrete didactic interventions?

Theory-based specification of the problem
The available scientific findings as well as the activatable common beliefs of experienced practitioners are evaluated in an iterative process.Based on these theoretical foundations, the main research questions, which were roughly formulated at the beginning, are clarified.In the project, this initially meant specifying the target construct of "ethical-reflective competences" and establishing a technical reference with the topic "urban economy".
In pedagogical practice, dealing with values seems to be quite ambivalent.On the one hand, and at least rhetorically, moral development plays an important role, not only in schools, but also in vocational and management education.For example, after the financial crisis in 2008 the "knowing -doing -being" trinity is widely preached in management education (Datar, Garvin & Cullen, 2010), with the "being"-dimension aimed at emphasizing the development of values, attitudes, and profes-sional identities ("acting with integrity, honesty, fairness; being aware of own personal strengths and weaknesses; reflecting and committing to the purpose and goals of organizations", see Euler & Feixas, 2013).On the other hand, the objectives and processes of value development are only broadly reflected.Dealing with values often raises a sneaking suspicion that students will be brainwashed.In contrast, one can also argue that ignoring the reflection of values in (economic) action would increase the risk of subtly manipulating the student."Values and attitudes are always part of human action.As a teacher, you can only decide to consciously reflect on them or unconsciously let them smolder.Silence means consent and reinforces the existing reality -could this be the imposition of values which one actually wants to avoid?" (Euler, 2015, 24).
Values are regarded as the people's statements on what they find valuable and desirable when it comes to (econo- mic) issues or (economic) actions (Euler & Hahn, 2014, 167 ff.).Values can therefore be assigned to things and human actions (Horle-bein, 1997, 6).Values may be phrased in general and abstract terms (e.g., "banks want to maximize their profits") or they can be phrased in more concrete terms (e.g., "the practice of the bank XY during the financial crisis towards their customers was ruthless and irresponsible").Literature provides numerous lists of values, which indicate those most preferred by their author.For example, Adorno (1967) introduced a list with virtues such as critical self-reflection, self-determination, resistance, critical awareness towards ideologies, appreciation, and leniency.
The construct of "ethical-reflective competences" aims, amongst others, at enabling learners to clarify the goals and values behind (economic) activities, to make value-conscious decisions, and to reflect on value-based activities with regard to their possible consequences.It is therefore not a question of transmitting the "right" norms or values, but rather of developing the competence to reflect on ethically relevant questions (in the economy) and to conduct respective ethical discourses.Terminologically, an understanding of ethics is embraced as "reflecting on morality," or more specifically: as reflecting and critically questioning existing values and norms (Reemtsma-Theis, 1998, 5).
For the implementation in economics education, the rough construct of "ethical-reflective competences" is initially captured in the following facets (see Kühner & Euler, 2017;Euler, Collenberg & Kühner, 2018): • Clarifying values: Guiding question: Which values are pursued in the (economic) action of specific economic stakeholders?Students are requested to explicitly clarify, describe, analyze, and substantiate the values underlying the behavior of people.
• Assessing the consequences of value-based decisions (of others): Guiding question: How does economic action impact on those who are affected by it?Students are requested to evaluate the impact of a specific action in order to indicate the extent to which the students find the action acceptable.
• Identifying and reflecting on alternative decisions and value bases: Guiding question: Which alternative action is conceivable and which values underlie those alternative actions?Students are requested to create alternatives to given actions and reflect on their value base.
• Making value-conscious decisions: Guiding question: Which decisions should be taken when values conflict?Students are requested to deal with dilemma situations and make a decision, which is supported with plausible arguments.
• Discussing the consequences of decisions taken (by themselves): Guiding question: How does a decision, once made, impact on those who are affected by it?Students are again requested to evaluate their decision's consequences for others and to take responsibility for their position.
At the same time, numerous methods of value development at different levels of abstraction are proposed, particularly in moral pedagogy, as well as in ethics and business didactics.The spectrum ranges from basic methodical concepts, such as the dilemma method, to the Socratic dialog, or the pro-contra discussion, as well as progress models and building block collections (Ulrich, 1996;Reetzmann & Grammes, 2014).
The didactic intervention was developed in the economic subject area of "urban economy".These developments took place within the context of the "iconomix" edu-cational program provided by the Swiss National Bank (www.iconomix.ch).The "iconomix" educational program provides teaching material for interested economics teachers in upper secondary schools.The materials comprise different media (e.g.knowledge sheets, commentary for the teacher, simulations) and address learning objectives at different aspiration levels.The concrete way in which the teachers use the teaching materials remains their prerogative.They can use the materials in the way suggested in the commentary for the teacher, but they can also include individual elements from the materials (e.g.tasks, slides) and use them selectively in their existing lessons.
The "urban economy" teaching arrangement is to be used primarily for teaching economics at grammar schools.In addition to basic economic categories, this module deals with urban economic contexts.Discussions will include economic issues such as: Where, within a city, is living more expensive and where is it cheaper?How do population growth and changes in transport infrastructure or spatial planning measures affect the choice of location, the prices of houses and apartments, the spread of the settlement area, and the coexistence of the various population groups?Numerous ethical dimensions -that offer anchor points for the promotion of ethical-reflective competences -can be included, and reflected upon, within the economic question.
The teaching arrangement was developed in line with the principles of problem-based learning (see Euler & Hahn 2014, 118 ff.;Euler & Kühner, 2017).It consists of a total of three complex problem situations, each of which is structured into subtasks.
• Problem situation 1 -"Where should I live?" Students are put into the shoes of a young university graduate who has just landed a new job in London.The question she faces is where to live, given a certain income.
• Problem situation 2 -"Should urban sprawl be limited?" Students are drawn into the conflict concerning London's "Green Belt".While some people argue for erecting many buildings in the green area, others fight for the preservation of nature.This constellation offers a substantial starting point for economic, as well as ethical reflections, which are guided by corresponding tasks.The learners will, thus, be introduced to a demonstration for the conservation of the green belt.They should analyze the positions of different interest groups and identify the underlying values.Finally, they should formulate and justify their own value-based position.
• Problem situation 3 -"How does an economic decision in the urban context impact on social segregation?" Students face rental price increases in the London district of Brixton.In the design of the subtasks, ethical questions -in addition to economic questions -are introduced and prepared for a reflective approach.The rental price increases are first analyzed according to the urban economic model.
Then, the students reflect on the consequences for social segregation in the community; they also identify conflicts, and clarify underlying values.Based on real events, such as the Tottenham riots in 2011, different ways to deal with upcoming conflicts are discussed.
The teachers can teach the three problem situations as a complete unit for a total of approximately five hours or in individually selected parts.

Development of the prototype
The teaching arrangement's first prototype is created on the basis of the evaluated literature and the teachers' experiences.The developments essentially comprise the following focal points: • Development of the economic subject structure "urban economy".
• Development of the didactic subject structure "ethical dimensions of economic action" as integration of the respective ethical problem content -and thus reflection content -into the economic subject structure.
• Development of the individual problem situations with a transformation of the subject structure into a problem structure.This involves formulating subjectively challenging tasks for triggering cognitively demanding learning activities, with the help of which, among other things, the development of defined ethical-reflective competences is to be triggered.

Generating design principles
The development of the teaching arrangement aims, on the one hand, at providing a sustainable concept for concrete teaching contexts in different classrooms, on the basis of which the teachers design a promotion of desired competences.On the other hand, the pillars of the teaching arrangement are to be worked out during the testing process by means of an accompanying evaluation, which support and substantially promote the achievement of objectives.These "supporting pillars" are formulated as design principles.
In this sense, design principles aim at statements about means-ends relationships that are not connected with the claim of a general theory, but are more general than singular statements about an individual case.Design principles "recommend how to address a specific class of issues in a range of settings" (McKenney & Reeves, 2012, 19).They capture "regularity in messy, complex settings" (DiSessa & Cobb, 2004, 84), but are inevitably limited in their scope and generalizability to the research contexts of their origin (e.g.classroom, school level, subject, age group).Ulrich & Probst (1991, 66 ff.) speak in this context of "order patterns", which lead to the fact that the states of a system cannot be predicted exactly, but can be determined within limits and with imponderables.Using the example of a tree, they illustrate that although the condition of the tree with its leaves, flowers, buds, and fruits cannot be determined for a specific hour in advance, it is possible to say, within a time frame, when the tree will blossom, when its fruits will ripen, or when the leaves will fall.
The desired scope and the degree of concretization in the formulation of a design principle are in a tense relationship.While research ideally strives for the most general findings possible, the practitioner seeks concrete guidance for a practical situation that needs to be designed.The more abstract the principle's formulation, the greater the range, but the less definite the practical instructions become.Conversely: The more concrete the principle's formulation, the narrower the scope, but the more specific the practical instructions become.
The first design principles are formulated in the DBR process as a result of the theoretical foundation.They initially rely on a literature analysis and the exploration of available practical experience.In this phase, they do not quite have the characteristics of proven principles; instead, they resemble unproven hypotheses and are called design assumptions (Raatz, 2015, 25).Accordingly, they are often not yet very differentiated.They have two functions for the subsequent research and development processes: (1) They identify the key components to be implemented in the development of the first intervention.
(2) They define the objects to be evaluated within the context of the upcoming testing.The path from design assumptions to design principles runs through targeted steps of testing and evaluation.This part of the DBR research process could be described as "patient, yet economic experimentation... in which it is important that the researcher understands how to be surprised" (Oelkers, 2014, 91).For each testing cycle, it is necessary to determine which aspects of the teaching arrangement ought to be examined with which evaluation methods.
Design principles capture the concretized relationships between the supporting pillars -or central teaching activities -of the teaching arrangement and the facets of the target construct.The principles provide information on essential components, which are incorporated into the development of the teaching arrangement and transformed into concrete teaching activities -in line with a design characteristic.At the same time, a design principle addresses the facets of the learning objective, which the respective teaching activities ought to target and promote.The following example illustrates the relationship:

Development of the testing and evaluation concept
The developed prototype with its underlying design assumptions forms the basis for testing and evaluation.
Idealtypically, different phases with different focal points are distinguished (McKenney & Reeves, 2012, 136 ff.).In the alpha phase, the primary focus is on evaluating the arrangement's inner coherence and consistency, as well as its practicability.Within the framework of a developer screening, it is possible to examine whether the design principles are adequately implemented, the didactic decisions are sufficiently justified, and the teaching arrangement is feasible within the scope of the assumed contextual conditions.In the beta phase, the evaluation focuses on identifying possible optimizations in the teaching arrangement.For example, it is of interest which expected and unexpected activities learners and teachers show with regard to certain design principles.Or: Framework conditions, which influence the learning processes and should be considered more, become clear.In the subsequent gamma phase, the focus is on examining the arrangement's effectiveness and effect with regard to the desired objectives.Due to the usually small sample-size and the lack of a control group, the validity of the results is limited.Nevertheless, it is possible to gain further indications that will improve and stabilize the arrangement.
The "urban economy" teaching arrangement was tested, evaluated, and revised at individual points in three cycles with learners from a total of nineteen classes.
• The first testing with four teachers in eight classes fo-

3.5
Volume cused on problem situations 1 and 2 (see section 3.2).Before the testing, a focus group of teachers considered the design's coherence and practicability.The evaluation was based on different methods.Each of the eleven design principles was assigned to one or more of the following data collection methods: classroom observation; interviews after the lessons with 3-4 learners and also with the teacher; analyzing the learners' documentation of the lessons; analyzing tests with reference to the module.The design principles for data generation are operationalized by using appropriate instruments (observation and interview guidelines, categories for document analysis).The testing resulted in numerous hints for improving different facets of the design: clarity of information and problem descriptions; aspiration level and motivational power of specific learning phases and tasks; appraisal of the learning gained by the students.As a consequence, the unit was extensively redesigned.
• The second testing with four teachers in six classes dealt with all three problem situations.The evaluation methods were similar to those used in the first testing, although the instruments were fine-tuned to the redesigned intervention.Especially the components of the intervention, which had turned out critical in the first testing, were emphasized.The results were therefore more specific.As a major result, the testing identified learning phases, which were not dealt with by the students with the expected cognitive depth.The intervention was redesigned again along the critical testing results.
• A third and final testing was conducted with five teachers in five classes.Apart from grammar schools (majoring in economics and law), vocational Matura classes, or a specialized secondary school, the class became part of the testing.Certain selected results are reported in the next section.

Research results
The evaluation of the data generated in the individual testing and evaluation cycles was performed primarily with reference to the underlying design assumptions.The results were initially task-specific references to successful, challenging, or particularly critical learning phases.Such references include suggestions for the linguistic optimization of texts and tasks; or streamlining, extending, or changing the sequencing of individual learning steps; or highlighting critical moments in the learning process that require special attention from the teacher.Some of these references are condensed -in the comparative analysis

3.6
Volume of the cases during the course of the cycles -to findings, which are identified and highlighted as cross-situational.
At the end of the testing and evaluation process, the "urban economy" teaching arrangement consisted of a total of eleven design principles.The following five are aimed at achieving individual facets of the target construct "ethical-reflective competences." Target facet "Clarifying values" • Design principle 1: The problem statement should sketch exemplary values to which arguments for concrete situations can be traced back (teaching activity), so that learners identify values underlying expressed arguments.
(Learning activity in the sense of concretization of the target facet).
• Design principle 2: The problem statement should introduce characters that enable an interpretation of their emotional state so that learners can gain access to possible values through the perception of emotions.
Target facet "Identifying and reflecting on alternative decisions and value bases" • Design principle 3: The problem statement should introduce conflicting role profiles so that learners discuss different positions and associated values, and agree on possible value conflicts.
Target facet "Making value-conscious decisions" • Design principle 4: The problem statement should allow multiple and/or contradictory perspectives on facts, as well as the actions of actors, so that learners can use them as reference points for developing their own values and formulating their own position.
• Design principle 5: The problem statement should introduce a dilemma situation so that learners develop proposals for balancing conflicting perspectives.
Without tracing the evaluation results of the individual design principles in detail here (see Euler, Collenberg & Kühner, 2018), some of the overarching findings with regard to the promotion of ethical-reflective competences are summarized: • The structuring of the learning process along the pro blem-based arrangements, in which economic tasks are combined with ethically relevant situation tasks, proves to be sustainable in principle.• Role-playing with the didactic intention that the learners adopt the perspective of different actors with different life situations and sometimes conflicting interests, are to be worked out in a differentiated way with regard to the role profiles -for indepth reflection on the emotional and affective dimensions of action.
• The traceability of emotions and arguments to underlying values in people's actions is very challenging for upper secondary learners and requires more or less pronounced support from teachers.One of the difficulties is that learners often lack a differentiated terminology for precisely capturing and describing emotions, as well as values.
• Without the teacher's differentiated support, the target group's learners often find it difficult to make a well-founded statement or to place their own values in dilemma situations.
• The development of mediating positions or corresponding action measures in value conflicts also represents a great challenge for learners.Here too, the learners require more or less extensive support from the teacher.
• Interpreting emotions and reflecting on values is also unusual and therefore challenging for certain teachers.Accordingly, it is imperative for them to be methodically prepared for supporting learners or moderating learning pro-cesses.

Cooperation science -practice
The cooperation between science and practice is a specific characteristic of DBR (see section 2).The roles and responsibilities within this cooperation may differ according to the willingness and competences of the actors who are involved.Within the project under discussion, the tasks were divided as follows: Volume The practical design entails a teaching arrangement created within a DBR project, which was tested and evaluated over several cycles, and which could develop a high degree of probation and robustness in this process.The result is didactic concepts and materials that the teacher can use and adapt according to the specific conditions of their lessons.In this respect, the transfer of the research results into a practical application is considerably shorter than is the case with empirical impact-based research.
Acquiring scientific knowledge entails creating -with design principles -a specific type of statement.Design principles -in the sense of medium-range theories -were developed in the project through repeated cycles of development, testing and (formative) evaluation of innovative practical concepts.They go beyond recording a singular case.However, in their generalization range and the statements' degree of abstraction they ultimately remain limited to the context of the respective test field.
The outline of the project process illustrates that design principles are ultimately based on two foundations: On the one hand, they are the substrate for dealing with scientific literature and the experiences of cooperating practitioners.In this phase they have a hypothetical character and are called design assumptions.On the other hand, design principles develop, refine, and expand in the process of testing during teaching practice.While -in empirical impact-based research -an arrangement (usually only theoretically based) is examined comparatively early and with larger samples, DBR initially strives to make the arrangement robust via a series of smaller testing and evaluation cycles for a defined field of practice."Therefore, we usually 'bet low' by conducting small studies, and then pursue the most promising results" (Schwartz et al., 2005, 20).In this context, a distinction is made between the singular forms of the arrangement and the overarching design principles.
How can DBR's potentials and limits be assessed in summary?What are the specifics compared to empirical impact-based research?Based on the above outlined rese- arch project, the specifics can be summarized as follows: • DBR leads to developing interventions for the innovative solution of new or still blurred problems.If the ways to achieve a desired objective are not yet clear, DBR strives to first sharpen the intervention before an arbitrary treat-ment -with a high investigation effort -is examined.
• By developing solutions to problems in authentic contexts, these are usually better adapted to the respective practical conditions and, thus, have a high practical relevance.
• Involving experienced practitioners in the clarification of the problem, the development and testing of solution drafts, as well as the evaluation and interpretation of test experiences, increases the probability of transferring the results beyond the narrower practical context of testing.
• DBR combines development with testing and (initially formative) evaluation of problem solutions, which are also developed as medium-range theories."Some of the most powerful findings will be serendipitous, and the data to support them will be marshaled post hoc." (Schoenfeld, 2006, 202).This corresponds to Dewey's view and his concept of "collateral learning" (quoted in: Reinking & Bradley, 2008, 51).
• The findings of a DBR do not become "general theories" -they offer design principles; "they provide guidance and direction, but do not give ‚certainties'" (Plomp, 2007, 22).
By comparing and differentiating the two research methodologies, it becomes possible to recognize what is specific about DBR.Against this background, the two basic orientations of empirical classroom-based research are to be finally compared:

Figure :
Figure: Research and development cycles within DBR

Figure :
Figure: Structural relationship in the generation and development of design principles

Volume 2 |
Issue 1 | 2018 | Article 15 2 | Issue 1 | 2018 | Article 15 Specification: Performance of design-based research DBR was introduced in section 2 as a possible alternative to empirical classroom-based research in the tradition of impact research.It was also linked to the question about the specific performance of this variant of classroom-based research for acquiring scientific knowledge and designing teaching practice.The outlined research project -based on economic education -illustrates both sides of the DBR and now enables an assessment of its potentials and limits.