How Florist Apprentices Explore Bouquet Designs Supporting Design Space Exploration for Vocational Students

Context: Exploring the design space is an important process in a design task. In this study, we considered design space exploration for the learners in vocational education and training (VET). The goal of the study was to investigate how they explore the design space while focusing on the effect of a graph-like interface on the learner’s understanding of the design space. With florists as the target profession, we investigated how the apprentices explore design variations, what they would gain from such activity, and how we can support this process. Approach: We developed a web application called BloomGraph that allows learners to explore design variations. It provides a graph-based interface that enables the systematic variation of design. Using the BloomGraph application, we conducted an experimental study with 44 florist apprentices in Switzerland to investigate the effect of the graph-based interface which provides a structured way of exploring the design space. The experimental group was given the graph-based interface to explore design variations while the control group had a linear-based interface. We compared them in terms of the number of bouquets explored, time of exploration, diversity of bouquets explored, and the learning gain in terms of the understanding of the design space measured using pre and post-tests. We also analyzed the strategies adopted by the participants for the graph navigation and the visual exploration behavior using the eye gaze data. Findings: Our analysis shows that the graph-based interface fosters a better understanding of the size of the design space and more efficient navigation towards a goal design in terms of the number of intermediate designs but with longer exploration of each intermediate design compared to the linear-based interface. Regarding the behavioral patterns in graph exploration, the participants who showed more strategic behavior in the design choices acquired a better understanding of the design space. Additionally, we trained a model that predicts the next choice of a learner using eye tracking data. It provides a reasonable accuracy that opens new possibilities for future studies. Conclusion: The findings of this study support the feasibility of design space exploration as a digital activity for VET learners and show how the learners can benefit from it. The contribution of the paper includes the validation of the idea with florist apprentices and the demonstration of how the process can be supported using a structured interface and the learner behavior analysis. This paper shows how a design exploration activity can provide an added value in the learning of an apprentice in a design-related VET system. we observed a significant difference between the two conditions ( χ2 = 5.71, df with experimental diversity. We also observed a significant difference between the two conditions in the number of revisits to the same bouquets ( χ2 = 15.6, df = p .001). The experimental group showed a higher number of revisits ( M = SD = 6.64) compared to the control group ( M = 5.52, SD = 3.54).


Introduction
The majority of the vocational education and training (VET) systems in Switzerland have a dual-track structure where students learn in schools for one or two days per week while they do an apprenticeship at workplaces for the remaining days. Their professional competence is developed from their experience in the workplace and reinforced by the theoretical knowledge learned in school. The idea behind the dual-track system is based on the concept of learning through experience, which has been explored with various theoretical models, such as experiential learning and situated learning (Kolb & Kolb, 2009;Lave & Wenger, 1991). The dual-track approach is particularly well-suited in VET systems when the profession involves handcraft and practical design tasks since the importance of hands-on experience is greater in such cases. For example, the jobs of wood craftsmen, fashion designers, jewelry makers, textile designers, ceramic designers, gardeners and florists all involve the creation of new designs where the practical experience plays a major role in their learning. Although the dual VET is considered an effective system for developing professional competence, one of the main challenges is to provide a rich experience. The practical experience gained from a workplace is often limited to the specific situations to which the apprentices are exposed and it does not usually cover the whole spectrum of the skills related to the profession. This limitation of experience is introduced by the type and the style of the workplace. Some companies are specialized in very specific tasks within the profession. For example, a florist apprentice working at an airport flower shop can be repeatedly making bouquets of the same design for a welcoming purpose and it is unlikely that he or she will have much experience on wedding or funeral bouquets during the apprenticeship. Furthermore, companies often establish their own style of design and the experience as an apprentice is biased towards the style of the company. And lastly, the tasks given to the apprentices are usually distant from the ones they will have to perform further down their career path. As an apprentice, they may make the daily flower, but not get the chance to do the original designs.
Our interest is on how to enhance the learning experience of VET apprentices by supporting their exploration through digital variations of designs. Exposure to design variations is an important part of learning in most design-related vocations. It can help the learners in acquiring a better understanding of the design space which plays an important role in finding solutions to a design task (Dobricki et al., 2020;Kolodner & Wills, 1993;Kulkarni et al., 2014;Tohidi et al., 2006). However, in the current situation, an apprentice in the workplace may be tasked with making a single design without having the opportunity to explore many designs and understand how the design is one of many possible solutions to the problem. For example, imagine a florist apprentice who made a wedding bouquet at the workplace. Her experience in the real world might end there, but with an additional digital activity, she can expand her experience from making one bouquet to exploring many variations of the design.
In this paper, our focus was on florists as the target profession and we specifically investigate how we could support the exploration of the design space. In order to allow florist apprentices to explore variations of a design, we implemented a web application called "Bloom-Graph". BloomGraph provides a graph-based interface to navigate through flower bouquet designs. The key concept behind the design of BloomGraph is the axes of transformation -each axis of the graph leads to transforming a particular attribute of the design and, therefore, learners can systematically navigate the design space. We hypothesize the positive effect of the disentanglement of design attributes on understanding of the design space. While it is common in the field of cognitive science to consider design as an exploration of the combinatorial space of independent design features, it might not be the case for apprentices. They might not be able to mentally construct multidimensional abstract space or feel comfortable to navigate through it as they are often very much concerned by the concrete shapes of their craft (Ngoon et al., 2019). This concern has triggered the question for our experimental study: Given a digital tool that provides variations of bouquet designs, how would florist apprentices explore the space? We investigate how they explore a set of variations focusing on the effect on their understanding of the structure of the given dataset. Specifically, we address 68 Design Space Exploration for Vocational Students two research questions in order to achieve the goal: (RQ1) What is the effect of the graph interface on the design exploration in terms of their understanding of the design space and (RQ2) What are the strategic behaviors of the learners in the graph exploration?
Through this paper, we contribute to apprentice experiences in VET systems. We demonstrate how apprentices in a VET system can enrich their experience by exploring design variations in the digital space and how they can benefit from this additional experience. The experimental study presented in this paper shows how the learning outcome is affected by a graph-based interface for exploration and the exploration strategy adopted.

Related Studies
The benefits of examples and variations in a design task have been demonstrated in many studies. Exposure to design alternatives provides awareness of the design space of potential options (Kolodner & Wills, 1993). Additional value is provided through the design process as people recognize and compare the alternatives (Tohidi et al., 2006). When comparing designs, learners are focused on the common relational structure, aiding abstraction of the underlying schema (Gick & Holyoak, 1983). The benefit of having examples in terms of understanding the design space is the core idea supporting our work presented in this paper and tested in the reported study.
While having examples can provide some benefits in a design task, there are mixed results to their effect in terms of design outcome. One negative effect is the conformity toward examples. The studies by Marsh et al. (1996) and Smith et al. (1993) show that design outcomes are likely to contain features of the examples and the conformity effect is increased with the increased number of examples. However, the negative effect of examples is debatable as there are also many positive results in the literature. Exposure to multiple designs can reduce fixation in a design task (Jansson & Smith, 1991) and early and repeated exposure to examples improves creativity in design (Kulkarni et al., 2014).
These mixed results may be due to the support that is provided around the use of examples. During the learning process, it is often not enough to just provide a resource to learners but scaffolds may need to be provided to guide the integration of the resource into the learning process (Durkin & Rittle-Johnson, 2012;Kollar et al., 2006;Roll et al., 2011). In the case of examples, it is important to investigate how people explore the given examples and what is the effect of the exploration. Without exploration, people often interpret the frame of the design problem too narrowly (Kershaw & Ohlsson, 2004) and designers may choose a design concept too early and fail to identify a valuable direction (Cross, 2004). In terms of the exploration strategies, Ball et al. (2004) compare novices and experts on how they make use of examples in design analysis. In their study, experts show more schema-driven use of examples than case-driven, with respect to novices. Najar et al. (2015) show that the students with higher levels of knowledge pay more attention to the schema of the data than the weaker students. Our interest lies in the same domain as these studies. In this paper, we investigate how people explore given designs while focusing on the effect on their understanding of the structure of the given design space.
As the way one explores design examples has an effect in a design task, some studies have investigated how to support the design exploration. Lee et al. (2010) propose an interactive example gallery for a web design task. Their study shows that structured corpus navigation can help users find inspirational examples and facilitate design. Ritchie et al. (2011) show that a systematic design exploration tool can help users finding relevant and inspiring design examples. Their tool allows users to explore the examples ordered according to the styles. Some studies propose methods to generate design variations (Alhashim et al., 2014;Talton et al., 2009); however, they focus on the algorithm rather than how they can be used in a learning context. Our study also proposes a design exploration tool for a design task, but what differentiates it from the previous work is that it allows learners to explore the design space by actively selecting what features to vary. We use the design variations that are systematically generated based on the current design as a means of exploring the design space.
As we are proposing design space exploration as a digital activity that would take a part in the learning experience of VET learners, it is important to consider how it can fit in the context of VET. One of the issues in the dual VET systems is that apprentices often perceive the gap between different learning locations (Taylor & Freeman, 2011) and the knowledge learned often remains encapsulated in its original context (Renkl et al., 1996). The experience from workplaces is usually concrete as it is situated in the target field, however, the experience per se is often not enough (Boud et al., 2013;Kolb, 1984). In order to enrich the workplace experience and better connect to the knowledge from other contexts, Schwendimann et al. (2015) propose a pedagogical model called "Erfarraum" for designing educational technologies for dual VET systems. The Erfarraum model proposes the design of shared digital spaces for reflecting on experiences acquired in different contexts in which VET takes place. It does not refer to a specific technology for digital learning, but rather a framework for designing a digital activity that allows bridging the gap that exists between different learning contexts. And the activity in the digital space should allow learners to reflect on their realworld experiences by augmenting and enriching them. The design of our study is based on the Erfarraum model where we consider design space exploration as a means of enriching the real-world experience.
There have been some recent efforts in this direction to design a digital activity that fosters reflecting on experiences in dual VET systems. One of the ways that have been explored is the use of online learning journals. Cattaneo et al. (2015) have studied the use of online learning journals for apprentice chefs to capture workplace experiences and share them in school classrooms. They reported positive results in terms of effectiveness and satisfaction. 70

Design Space Exploration for Vocational Students
In a similar study, Mauroux et al. (2016) showed that promoting reflective writing in the learning journals had a positive effect on the metacognitive learning strategies as well as the performance on the final exams. Other digital activities that share the same approach include augmenting video recordings from workplaces for chefs and car mechanics (Motta et al., 2014), tabletop activity that allows simulating warehouse layout for logisticians (Cuendet et al., 2013), and augmented reality application for carpenters that lets you apply loads and visualize forces on physical structures (Lucignano, 2018). These studies demonstrated positive effects on VET learners in terms of their motivation and academic performance. Based on the positive findings from these examples, this paper also studies how a digital activity can enrich the experience of VET. Considering design space exploration as a means to achieve the goal, our focus is on understanding the behavior of learners and how it affects the learning outcome.

BloomGraph
For this study, we have developed a web application called BloomGraph that can support florist design exploration (Kim et al., 2019). The interface of the BloomGraph application is shown in Fig. 1. As seen on the left side, the navigation graph consists of the center node that shows the current bouquet design and four proposed variations around it. The proposed design in each axis is a variation of the current bouquet in terms of a specific attribute: Color, form, texture and spacing. These attributes were chosen through discussions with florist teachers to align with the concepts of bouquet designs that were being taught. When a user selects one of the variations, it comes to the center and a set of new variations of that design are proposed. Above the graph, there is a history bar that shows all the designs the user went through. Using the history bar, a user can backtrack to previous designs. On the right side, there is an interactive 3D viewer. The current bouquet design is shown in 3D and the user can rotate or zoom in/out. In the viewer, the user can also see the names of flowers by hovering the mouse pointer over them.
BloomGraph allows users to explore the design space as they follow the nodes in a graph. In the graph, each axis leads to a different variation of the current design. By clicking the nodes in different axes, users can vary the design systematically in terms of the important attributes, thus providing a structured way of exploring the design variations. The attributes that make the design are disentangled by the interface design so that learners can consider design as the exploration of the combinatorial space of independent design features. As users go through a series of designs as they travel along the nodes, the history bar is the way to show them where they have been and where they are now. It allows users to not only backtrack to previous stages, but also visually see the exploration path while going through the designs. 71

Kim, Oertel, Dillenbourg
The BloomGraph application is implemented using Meteor framework written in JavaScript. For the front-end rendering, we used React and D3.js libraries. The interactive 3D viewer is implemented using the API provided by BloomyPro (BloomyPro, 2019).

Methods
In this section, we describe the details of the experimental study we conducted using Bloom-Graph and our analysis methods. As mentioned in the introduction, the goal of the experiment is to investigate how florist apprentices would explore the space of design variations given the graph-based interface of BloomGraph.

72
Design Space Exploration for Vocational Students

Experimental Setup
Participants In order to recruit the participants for the experiment, we first contacted florist teachers in vocational schools in Switzerland. Four teachers from three schools agreed to run the experiment in their classes. We did not have any criteria for eliminating students and invited all the students from the classes to participate. As the result, 44 florist apprentices (43 females and 1 male) participated in this study and they were aged between 15 and 61 years (M = 28.4, SD = 15.2). The unbalanced gender ratio comes from the nature of the profession. The wide range of the age is due to one of the classes being a retraining. Thirty-four subjects were in their first year of the three-year program of the basic vocation education and the remainder in their second year. All participants were asked to sign the consent form if they agreed to participate after reading the information sheet. Participants under 18 years old had to provide a signed parental consent form, which had been distributed to them two weeks in advance of the experiment.

Experimental Design
For the study, we used a between-subjects design with 23 participants in the experimental condition using the graph-based interface of the BloomGraph application and 21 participants in the control condition using the linear-based interface. We controlled for the type of the class and the school year when assigning the participants to the two conditions.

Procedure
The task for the participants was to select a bouquet design that is most appropriate for a virtual customer. Together with a florist teacher, we developed two scenarios that resemble realworld situations -first scenario for the birthday party of an old lady and the second one for a wedding. Additional details given to the participants included basic customer information, when and where the event was taking place, and their preference on colors. The experiment started with the general introduction of the study. Participants signed the consent form if they agreed to participate. Then they were asked to do a pre-test. Once they finished the pre-test, we went through an example tutorial together so that they could familiarize with the interface. The tutorial was given in the interface that they were assigned to base on the grouping. The example tutorial did not involve flower bouquets, but some simple geometric primitives so that they would focus on the interface, not the designs themselves. Once they finished the tutorial, they were given the actual tasks of selecting the bouquet designs for the virtual customers. They were asked to do two exercises, one for each scenario. The order of the scenarios was counterbalanced. Once the participants completed the two tasks, they were asked to do a post-test followed by a questionnaire to gather basic demographic information and the feedback on their experience with the application.

Materials
For the task, the participants either worked with a graph-based interface or linear-based interface depending on their assigned condition. The graph-based interface was designed as described in the previous section. It disentangles the four dimensions of the design space and presents the bouquets in a structured manner. For the control condition, linear-based interface is used and it is shown in Fig. 2. In the linear-based interface, four random variations of the current design are proposed in a linear formation. It is the same dataset as the experimental condition, presented in an unstructured way. It resembles the way people go through a catalog or a search result. In order to provide equal amount of information as the experimental condition, each variation in the linear condition comes with a tag that shows which attributes have been changed from the current design. As with the graph condition, to control for the availability of other features, the participants had access to their history and the 3D viewer. The difference between the two interfaces is the way we present the same data and it is to test our hypothesis on the effect of a graph interface on the design exploration.

Figure 2: Linear Interface for the Control Condition in the Experiment
For each scenario, we created a set of bouquet designs. They were created systematically so that users can vary one attribute of the design at a time. Each attribute could take one of three values we designed (e.g., the color attribute, the main theme color of the bouquet, can be either red, pink or write). Combining the three values for each of the four attributes, the dataset consists of 81 bouquet designs per scenario.

Design Space Exploration for Vocational Students
To measure the participants' learning of the design space, we created pre-test and post-test measures. In particular, we wanted to see if they could understand the bouquet design process as a combinatorial problem where a design is a combination of attributes that can take different values. We were interested in how their understanding can be improved through the BloomGraph activity. We designed two types of questions -one on identifying the design that shares an attribute with a given set of designs and the other on identifying the most appropriate design that could be placed in a missing spot in a connected structure of designs. The questions were multiple choice questions and the participants were asked to choose the one that was most appropriate. Each test was made up of five questions of each type and each question was worth 10 points, therefore the maximum score of 100 points. We created two versions of tests and counterbalanced their use as a pre-test and post-test.
A questionnaire was given at the end of the experiment and it included questions on demographic information and their experience with the BloomGraph application. In terms of their experience with their application, we asked five 7-point Likert scale questions on the usability and satisfaction. Additionally, we asked them to provide an estimate for the total number of bouquets in the given dataset for each scenario during the experiment. Estimation on the size of the dataset is one of the factors that reflect the participant's understanding of the given space.

Measures
During the experiment, we collected data in two ways. All the interactions of the users with the interface of the application have been logged. The log included the designs they went through and the time stamp for each action. Using the log, we can analyze the behavior of users and the strategies adopted by them. In addition to the application log, we recorded the eye gaze of the participants. We used a screen-based eye tracking device from SMI and recorded the binocular gaze at 250 Hz. The purpose of the eye gaze recording is to investigate the visual behavior of users.

Analyses
In order to answer the two research questions, we analyzed the data in two parts. The first part of the analysis is to answer the first research question on the effect of the graph interface on design exploration. The second part of the analysis focuses on the behaviors of the participants using the graph-based interface in order to answer the second research question on the strategies adopted in the graph exploration. Details of the measures we analyzed are described in this section.
In the first part of the analysis, to see the effect of the graph interface on the exploration patterns of the learners, we compared the experimental group with the control group on different metrics: Number of bouquets explored before making a choice, number of revisits to the same bouquets, total exploration time, and time spent per bouquet. We also compared the learning gains between the two conditions. We used the difference between the post-test and the pre-test scores as the learning gain. Another factor we looked at is the estimation on how many bouquets they think there were in each scenario. We also investigated the diversity of the bouquets they went through. The diversity measure was calculated for each participant using the concept of entropy as follows: Where N is the number of attributes of the bouquet design and M is the number of values that each attribute can take. And p ij is the percentage of value j for attribute i in all the bouquets that one participant visited. In the experiment, N = 4 and M = 3.
In the second part of the analysis, we took a closer look at the experimental group in order to investigate the strategies adopted in the graph exploration. As our interest is on the strategy used for navigating different dimensions of attributes, we looked at the sequences of the attributes that the participants selected, specifically the average number of consecutive clicks on one axis. Higher number of consecutive clicks in one axis means exploring more variations in terms of that attribute consecutively, or going deeper in that dimension. This strategy can be described as more consistent in choices since it has a priority on the consistency in attributes. On the other hand, lower number of consecutive clicks in one axis shows lower consistency in attribute choices, but higher priority on the diversity in exploration. For example, Fig. 3 shows two click sequences where the colors represent different attributes chosen to be changed. In our definition, Sequence 1 shows more consistent strategy whereas Sequence 2 is less consistent. The purpose of using this measure to characterize the strategy is to investigate the effect of the disentanglement of the dimensions on the exploration paths with respect to the dimensions.

Figure 3: Examples of Exploration Pattern
Regarding the second research question, we also investigated the visual exploration behavior using eye tracking data. In order to measure how visually explorative they are, we defined visual-explorative-ness with the number of fixations on the four proposed variations before 76 Design Space Exploration for Vocational Students clicking one. This measure shows how much they visually explored the proposed options for making a choice. We looked at the correlation between the visual-explorative-ness and the consistency in the attribute choices in order to investigate the relationship between the visual behavior and the exploration strategy. Lastly, we investigated how we can use the eye gaze data to predict the next click of the learner. We used a support vector machine (SVM) to train the prediction model with the features extracted from the gaze data. We extracted features from the gaze events before each click and generated feature vectors. The feature vector included the number of fixations on each node, the time spent on each node, and the node with the maximum time spent. We randomly selected 80% of the eye tracking data to train the model and the remaining 20% was used to test the performance.

Results
As described in the previous section, we analyzed the data in two parts and we present the results in this section. For the statistical tests, we used Kruskal-Wallis test due to the nonnormality of the distributions of data.

Number of Bouquets and Exploration Time
Comparing the exploration behavior in the two conditions, we first observed that the number of bouquets participants explored for a task was significantly different (χ2 = 8.60, df = 1, p < .01). In the graph condition, they explored fewer bouquets (M = 13.2, SD = 4.46) before making their choices compared to the linear condition (M = 18.5, SD = 9.42). We observed a large variation in the linear condition. In terms of the exploration time, we observed that it was significantly longer in the graph condition (χ2 = 5.71, df = 1, p < .05). Therefore, in the graph condition, participants spent more time on fewer bouquets. The results are shown in Fig. 4.

Bouquets Visited
In terms of which bouquets the participants visited during the tasks, we analyzed the following factors: (i) Diversity of bouquets with the diversity measure defined above and (ii) number of revisits to the same bouquets. For the diversity of bouquets explored, we observed a significant difference between the two conditions (χ2 = 5.71, df = 1, p < .05) with the experimental condition having higher diversity. We also observed a significant difference between the two conditions in the number of revisits to the same bouquets (χ2 = 15.6, df = 1 p < .001). The experimental group showed a higher number of revisits (M = 10.5, SD = 6.64) compared to the control group (M = 5.52, SD = 3.54). 77 Kim, Oertel, Dillenbourg 3), the difference between the two conditions was not significant (χ2 = 2.45, df = 1, p = .12). There was, however, a significant difference in the accuracy of the estimation on the design space size (χ2 = 12.3, df = 1, p < .001). From the questionnaire, the participants in the graph condition showed significantly better estimation on how many bouquet designs were present in a scenario (M = 52.8, SD = 12.1) compared to the linear condition (M = 29.0, SD = 14.4). The actual number of bouquet designs in each scenario was 81.

Categorization Based on Exploration Strategy
In order to investigate the strategies of the participants on the graph exploration, we first investigated how consistent they are in terms of their choice of the bouquets in the graph axes. Fig. 5 shows the sequence of clicks in the graph condition. Using the measure of consistency defined above, we divided the participants in the graph condition (N = 23) into two groups. The group with higher consistency (N = 12) had 3.03 consecutive clicks in the same axis in average (SD = 0.81) and the group with lower consistency (N = 11) 1.45 clicks (SD = 0.17). The difference was significant (χ2 = 16.5, df = 1, p < .001).
With the categorization based on the consistency measure, we observed that the group of higher consistency had significantly higher learning gain compared to the group of lower consistency (χ2 = 5.77, df = 1, p < .05). Fig. 6 shows the learning gains of the two groups as well as that of the linear group. The difference between the high consistency group and the linear group is also significant (χ2 = 7.26, df = 1, p < .01). Furthermore, another measure we looked at with respect to the exploration strategy is the estimation of the space size. We noticed that the consistency of their exploration was positively correlated to the accuracy of the estimation (r = 0.48, p < .05).

Figure 5: Sequences of Clicks in the Graph Condition
Note. Each row represents one trial and the colors represent different attributes selected to change. Note that the lengths of the sequences show the numbers of clicks, not the time. 79 Kim, Oertel, Dillenbourg

Figure 6: Comparison of the Learning Gains of the two Groups From the Graph Condition Along With the Linear Condition
Note. The two groups from the graph condition are based on consistency measure of their exploration strategies.
The learning gain is calculated as the difference between the pre-test and post-test scores.

Visual Exploration and Prediction on Next Choice
Using the eye tracking data, we investigated visual exploration behavior of the participants. The group of higher consistency showed the visual-explorative-ness of 1.82 (SD = 0.45) and the group of lower consistency 2.41 (SD = 0.35). As individuals, we observed that the visualexplorative-ness is negatively correlated to the consistency in choices (r = -0.57, p < .05). In other words, the participants with higher consistency in attribute choices were also visually focused on each axis at a time. This result is logical and as expected since in order to be more divergent in choices, one would consider more options visually. Using this correlation observed between the gaze behavior and the choices they make, we investigated whether we can predict the next choice in the navigation based on the eye gaze data. We trained a prediction model using SVM, as described in the previous section. This was done with different grouping -first on all participants in the graph condition then separately for the two groups with different strategies. In order to see how early we can predict a learner's next choice, we made the prediction every 2 seconds up to 12 seconds. The average interval between two clicks was 11.2 seconds. Fig. 7 shows the test result of the prediction models. The prediction accuracy for the high consistency group is the highest among the 3 groups and it reaches approximately 80% after 10 seconds. Overall, all 3 groups show a reasonable performance and the prediction accuracy reaches 50% as early as the first 3 seconds and over 70% in 10 seconds.

Discussion and Conclusions
The goal of this paper is to investigate how VET apprentices explore the design space and how we can support this process. In order to achieve this goal, we designed a bouquet design exploration application for florist apprentices and conducted an experimental study. The experiment was designed to answer two research questions on the effect of a graph-based interface and the strategies adopted by participants in the graph exploration.
Regarding our first research question on the effect of the graph interface on the design exploration, the analysis allows for a number of observations. First, the participants in the graph condition went through fewer number of bouquets before selecting final designs, compared to the ones in the linear condition. This is logical since the graph allows more direct navigation than the linear presentation. It is debatable whether it is better to navigate efficiently to the goal design or to be exposed to more designs (Jansson & Smith, 1991;Marsh et al., 1996). But what it shows us is that the apprentices were able to navigate using the graph and find their ways more efficiently. In terms of time, they spent more time on the intermediate bouquet designs before making their final choices. This also gives a hint that they are not just randomly selecting next ones on the graph, but trying to understand the structure. In terms of which bouquets they went through, the participants were exposed to more diverse designs when using the graph-based interface. The graph group also showed a higher number of revisits to the same designs while exploring. Overall, we interpret these observations as the evidence of some strategy-driven behavior in the exploration that has been further investigated in the second part of the analysis. In terms of the understanding of the design space, the participants who used the graph-based interface provided significantly better estimations on the size of the design space after the activity. In summary, the graph-based interface led to more efficient navigation towards the goal, exposure to more diverse designs, and better understanding of the design space. These results are in agreement with the previous findings on the advantages of structured and systematic design exploration in design-related learning (Lee et al., 2010;Ritchie et al., 2011). Our findings show that the graph-based interface can support the design exploration and the effectiveness of such design exploration tool can be valid for VET learners.
The second research question was on the strategy in the graph exploration. With the group-ing of the participants of the graph condition based on how consistent they are in terms of their choices of attributes, we observed some interesting results. The participants with more consistent strategy showed higher learning gains and they also had better estimation on the size of the design space. Consistency in the graph exploration appears to be an effective strategy that leads to a better learning outcome in our measures. One possible explanation is that the disentanglement of dimensions allowed disentangled exploration which lead to a better understanding of the design space. On the other hand, the participants who had the strategy with lower consistency were less successful. Their behavior can be also seen as jumping among different attributes and trying to be opportunistic in finding what they like. It can be also interpreted as less strategic in the exploration. These results are in agreement with the findings of Ball et al. (2004) and Najar et al. (2015) on the advantage of strategic approach in design exploration. A schema-driven approach in example search can be more desirable in a design-related task and our study demonstrates it in terms of the understanding of the design space. Moreover, as we observed the correlation between the visual exploration behavior and the choices they make, we trained a prediction model that predicts the next bouquet choices with a reasonable accuracy using the visual features from eye tracking data. In summary, the strategy adopted by the learners in the graph exploration had a significant effect on the learning outcome in our experiment and the visual behavior provided additional insight into their exploration behavior which opens new possibilities for future studies.
Based on our analysis, the participants in both conditions showed positive learning gain from the BloomGraph activity in terms of understanding of the design space. This finding supports the idea of design space exploration as a means of enhancing the learning experience of VET apprentices while demonstrating another practice of the Erfarraum model. When we focused on the participants with graph-based interface, we observed that some of them be-nefited more from having the structured way of navigation. We found that the exploration strategy adopted by the learners had a significant effect on the learning outcome. The question to be addressed now is how we can guide the learners so that they can maximize the benefit they take from the activity. One approach that could be suitable for digital applications like BloomGraph is the use of automated feedback. The result of the prediction model we presented in this study is interesting in this aspect. Based on the prediction of the behavior of a user, we can provide online feedback while the user is using the application. In this way, we can guide the learner so that they can adopt a desirable strategy in the exploration. It would be interesting as future work to investigate the effectiveness of different types of feedback such as direct/indirect or immediate/delayed feedback in this context (Corbett & Anderson, 2001;Roll et al., 2011).
The question we wanted to investigate in the experimental study was whether the apprentices in vocational education can understand conceptual design as a combinatorial problem of different attributes and navigate through them using a graph interface. This might sound as a trivial problem, but it might not be the case for the apprentices in vocational education as the process requires a certain level of abstraction. One might argue that designing a bouquet may not be a problem to be approached scientifically, but rather with a free mind of creativity. However, creativity is not unrelated to the understanding of the design space. Understanding what is available as a designer with the awareness of the constraints that exist in the problem space is an important aspect for creativity (Joyce, 2009;Rosso, 2014). We believe that fostering a better understanding of the design space through supporting design exploration has a positive impact on the creativity in a design task.
This paper demonstrated how a design exploration activity can provide an added value in the learning of an apprentice in a design-related VET system. The contribution of this paper is the validation of the idea with florist apprentices and we demonstrated how we can support the process by investigating the effect of a structured interface and the exploration strategy. Our results show that the apprentices can benefit from such activity by acquiring a better understanding of the design space and the learning outcome can be further improved by the graph-based interface as well as the strategy adopted in the exploration. The results support the potential of design exploration as a means of enhancing the learning experience of VET learners.
Although this work contributes to our understanding of designing digital activities for VET learners, there are limitations to the study that should be considered and addressed in future work. In this paper, we investigated the feasibility of design exploration for VET learners while focusing on their behavior. As we validated the feasibility of the idea, the next question is on how to design and integrate such activity to the learning journey of VET students. It is another research question of how it can be used to enhance the learning experience in a broader scale and it needs further investigation. It requires exploring the fit of such digital activity to the dual-track VET systems, especially in connection to the real-world experience. Another factor to be explored is the generalizability of the results to other professions. While the current study focused on florist education, there are many design-related professions that could benefit from design space exploration and the cross-profession generalizability will be an interesting factor to investigate.