Effectiveness of Collaboration in VET: Measuring Skills for Solving Complex Vocational Problems With a Multidimensional Authentic Technology-Based Assessment

Context: Dealing with professional complexity has been of scientific interest in the research field of vocational education and training for decades. So far, there is a lack of empirical evidence regarding how professional complexity finds its way into learning processes in VET. A common option is to model complexity through authentic simulations and/or problem-solving tasks. This study does both: Complex problem-solving tasks are integrated into an authentic office simulation and are expanded to include collaborative elements, using computer-based agents. Collaboration is used to improve learning, but it is also an authentic representation of current work processes, so we ask whether apprentices perform better in the individual or in the collaborative test setting. Methods: To compare skills used for complex problem-solving tasks within an authentic business simulation, the test settings are systematically varied between individual and collaborative test settings. The test environment is a technology-based assessment (TBA). Test tasks reflect a complex professional problem; they are presented in an office simulation, in which apprentices can collaborate with a computer-based agent. Multi-group confirmatory factor analysis is conducted to test for measurement invariance across test settings, and the two-dimensional Rasch-model incorporating between-item multidimensionality with


1
Designing Learning Processes in VET: The Increasing Importance of Collaboration Increasingly rapid changes in technology and the use of new information and communication technologies are factors that explain the emergence of teamwork and collaboration in work processes.Consequently, academic inquiries into the transversal competencies which are necessary to cope with complex requirements at the workplace suggest a re-modeling of the content of vocational training in line with the transformations in the practical field.In this section, we outline our theoretical conceptualization of skills needed to solve complex problems individually and collaboratively.

Modeling the Complexity of Practice for Learning Processes in VET
Work processes are complex, and their complexity is increasing.This development can be explained by increasing rationalization through technical progress and the use of new information and communication technologies (Achtenhagen et al., 1992).Consequently, work processes are changing.Current research programs at national and international level are looking at future technologies and show that "digitization of work", or "Industry 4.0", is leading to a change in work and production processes across all industries (e.g., Hasenbeck, 2019;Hirsch-Kreinsen, 2017).The current, especially digitally driven, transformations have far-reaching effects on existing organizational structures and processes, on customer relationships, and on business models, which are being realigned according to the digitalization of products, services, and processes (Sczogiel et al., 2019).Transformation in business and work processes has an impact on commercial workplaces: Jobs are changing and with that, the competency requirements as basis for professionally adequate action are changing, too (Brötz et al., 2014).In a complex environment, teamwork and collaboration become more important (Schlicht, 2019).A closer look at the developments of Industry 4.0 provides opportunities for making vocational education and training (VET) more attractive (Esser, 2015).In this context, learning processes in vocational training should be made to focus more on competency development (Anselmann et al., 2022;Spöttl & Windelband, 2021) and require apprentices to make enterprise-specific decisions in more complex and authentic scenarios.Regarding the resulting consequences for content-related training priorities, teamwork skills, negotiation skills, and the willingness to learn and collaborate (Ahrens & Spöttl, 2015) are particularly relevant -these transversal competencies are at the core of the framework curricula for vocational training.
The few empirical studies on the re-modeling of the content of vocational training in line with the transformations in the practical field that are currently available focus especially on participation and communication as conditions for success in the construction of learning processes.The study by Schlicht (2019), for example, points to the following learning situations that are gaining in importance: -Learning situations that emphasize communication and cooperation in business processes.Communication and cooperation are highly relevant for shaping social and business relationships, for entrepreneurial success, and for the personal development of experts and leaders.
-Learning situations that address typical (industry-specific) problem situations linked to the current and future (projected) developments of a sustainable economy.
Learning processes should consider both the digital and employee-related changes that are currently taking place in real work processes.

Skills Needed to Solve Complex Problem-Solving Tasks
Complex work and training situations are, e.g., work processes in which, in addition to routine activities, economic decisions must be taken, and work results presented and reflected on.Work is increasingly done in project teams and across departmental boundaries.Such work requirements show characteristics of complex problems.Thus, coping with complex requirements is not only about accuracy and speed, but above all about acting strategically and coping with a "strategic moment".This includes -the ability to control cognitive operations, -the availability of heurisms, and -the "wisdom" of the problem solver (Dörner, 1986).
Furthermore, a complex problem demands the operational intelligence of the problem solver.This includes, e.g., a balanced elaboration and negotiation of goals and self-management.Solving a complex problem thus implies the efficient interaction between a solver and the situational demands and requires cognitive and social skills and knowledge, as well as emotional and personal regulation (Frensch & Funke, 1995).Against this background, we model and assess two sets of skills for solving a complex problem: Cognitive and social skills (Funke, 2003;Hesse et al., 2015, Table 1).The cognitive skills focus on the complex problem-solving process itself, and in particular on task regulation and knowledge building in the solution process; the social skills address the interpersonal or interactional skills for successful work processes and are primarily reflected in individual opportunities for participation, perspective taking, and social regulation (Andrews-Todd & Kerr, 2019;Davier et al., 2018;Hesse et al., 2015).(Hesse et al., 2015) Cognitive To solve complex problems, both skills must be applied, regardless of whether the problem is dealt with individually or collaboratively.Both dimensions are correlated.The relevance of social skills for cognitive processes is quite obvious, which has also been referenced in team research (Salas et al., 2017).With reference to these findings, it can be assumed that collaboration has a positive effect on the solution quality of a complex problem.On the one hand, the competencies of several people are needed for complex work processes, and on the other hand, the focus is increasingly on professional solutions developed collaboratively through

Effectiveness of Collaboration in Solving Complex Vocational Problems
changed forms of learning and work.Accordingly, collaboration in complex work processes can be defined as the potential to act in a cognitively and socially appropriate manner in specific, technically complex problem situations.So far, complex problem scenarios in VET have usually been modeled as situations that had to be mastered alone.To add more authenticity to the learning tasks here, problem scenarios are expanded to include collaborative components.This can be done in different ways; in this study, computer-based agents are used as simulated colleagues, who interactively offer (standardized) support.Regarding the changes in commercial workplaces, it is helpful that technical progress in the field of computer technology, network technology, and telecommunication (can) create new possibilities for cooperation and more efficiency in the collaborative processing of a task (Barkley et al., 2014;Borghoff & Schlichter, 2000;Haake & Pfister, 2010).For example, computer-supported cooperative learning seeks to make working on a collaborative task more efficient and easier (Barron, 2003;Dillenbourg & Traum, 2006).A special kind of collaboration is that between learners and computer-based agents.The first technical implementation of such collaboration was in PISA 2015 (He et al., 2017).The computer-based agents communicated with the learner in text-based chats and simultaneously evaluated the learner's problem-solving skills based on the given, pre-formulated answers.Thus, the agents were able to observe performance, knowledge, skills, and psychometric abilities and enabled a standardized large-scale assessment (LSA; Graesser et al., 2017).

Goal and Research Question of the Study
The studies on problem-solving in complex problems (Funke, 2003) and collaborative problem-solving (Hesse et al., 2015) clearly indicate that problem-solving is a multidimensional construct.In the present study, we assume a two-dimensional construct that differentiates between cognitive and social skills.The cognitive skill bundle is operationalized via task regulation and knowledge construction; the social skill bundle includes participation, perspective taking, and social regulation as indicators.
The studies on the didactics of complex problems in VET (Rausch et al., 2017;Seifried et al., 2016;Winther & Klotz, 2016;Winther, 2011) show that learning in complex learning arrangements can improve vocational competencies in economic domains.So far, however, there are no findings on how well learners can handle complex, vocational tasks.The studies (summarized for ASCOT: Beck et al., 2016) show that apprentices have difficulties in grasping complexity and translating it into task solutions.Transferred to the assessments, this means that test items are often not used to their full potential because the apprentices cannot fully resolve the complexity.One may assume that learners are better at dealing with complexity when they collaborate.Thus, in this study, collaboration takes place with simulated colleagues in the form of computer-based agents.

Paeßens, Ma, Winther
Against the background of the theoretical considerations, two research questions are addressed: 1. Can the two-dimensional construct of cognitive and social skills for complex problemsolving be empirically represented?
2. Can the complexity of vocational situations be dealt with more successfully with collaborative support from a computer-based agent?
3 Measurement Environment: Using a Business Simulation as TBA To answer the research questions, a complex learning arrangement is used.Commercial tasks for learning and measurement are deployed through the office simulation LUCA.LUCA computer-based agents could offer learners different kinds of hints/scaffolds (learning environment) and at the same time record and measures learners' performance (measurement environment).In this section, we describe how test items are integrated in LUCA and how LUCA is used both for individual and collaborative settings.

Understanding the Task "Supplier Selection"
LUCA is an adaptive learning environment for enabling complex learning processes (Rausch et al., 2021).LUCA simulates real everyday working life and thus supports commercial learning (learning environment).The office simulation can also be used as a measurement environment.For this purpose, complex problem-solving tasks are set as performance tests.The present study refers to the complex problem-solving task "supplier selection" as an example.
In the authentic problem scenario "supplier selection", the apprentices are required to select a suitable supplier from several offers.The task contains a large amount of information and additional attachments that must be viewed and processed.The task "supplier selection" is a domain-typical problem scenario in the business context (design principles of complex scenarios, see Ma et al., 2022;Paeßens & Winther, 2021;Paeßens et al., 2022;Paeßens & Winther, 2023b).The scenario can be placed in the value creation processes of companies and here in particular in the area of purchasing.The vocational requirement is to decide for a supplier, which is worked out by the apprentices in the office simulation in various substeps.The apprentices first view various offers and then prepare a utility value analysis to evaluate the offers.Supplier selection has a high curricular and practical relevance in the field of commercial-vocational education, involves various economic procedures, and focuses on reasoned decisions; thus, it is particularly suitable as an illustrative example for a complex vocational situation.

Effectiveness of Collaboration in Solving Complex Vocational Problems
In this authentic vocational task, the complexity of vocational activity was constructed in such a way that it can be experienced and processed by the apprentices.Working with complexity is systematically but equally varied for both test settings.As theoretically described, both test settings have in common that cognitive and social skills are necessary for solving complex problems (cf.Hesse et al., 2015).The cognitive and social skills involved in both, individual and collaborative problem-solving are measured with validated test items (for individual problem-solving: Paeßens et al., 2023a; for collaborative problem-solving: Paeßens & Winther, 2023b).Table 2 outlines how cognitive and social skills are modeled as theoretically described sets of sub-performances and measured through the different indicators.In the present study, only task regulation is tested as an indicator for cognitive skills.The different indicators testing the corresponding sub-performances are collected in standardized task items and then embedded in various complex problem scenarios.Therefore, the cognitive and social skill sets are evaluated based on the responses of learners to task items.For example, in the problem scenario "supplier selection" for social regulation (a sub-performance of social skills), learners' skill-level is evaluated according to the criteria of quantity and quality of the reasons for choosing a supplier.To obtain a better insight into the tasks in the applied technology-based assessment (TBA), an indicator within the complex task for perspective taking (a sub-performance of social skills) is presented in Figure 1.Learners work through complex problem scenarios in an authentic office environment where various applications (email client, spreadsheet, ERP, etc.) are available.The department manager "Timur Demir" explains in his email with which weighting the factors price, delivery time, and quality should be considered in the benefit analysis.The learners are expected to transfer the information into their own mental model and ultimately into the benefit analysis for other colleagues.Learners in both, the collaborative and the individual test settings need to be able to understand information from other colleagues and their intentions and then integrate this into their work process.This item shows that, in addition to cognitive skills, social skills are also necessary for complex problem-solving in both test settings.

Expanding the Task by Collaborative Interaction
In this study, the existing items from the technology-based office simulation are used in two different test settings.The sub-performances are equivalent in content in both test settings.
The collaborative test setting is expanded with various interaction formats that provide general as well as specific solution hints and concretely address both cognitive and social skills.Learners are given guidance via computer-based agents.For example, overlays in the office simulation (see Figure 2) simulate instructor interactions or contacts with colleagues.When learners respond to the computer-based agent, they receive procedural information by email that addresses, e.g., specific perspective taking, prompts for individual activities in the overall collaborative process, or suggests general problem-solving heuristics.To sum up, the measurement environment is based on a business simulation, in which day by day work tasks have to be mastered.The performance of the apprentices in solving the task "supplier selection" is recorded.This problem-solving task requires both cognitive and social skills.While the apprentices have to master the complex problem-solving task on their own in the individual test setting, they are supported by a computer-based agent in the collaborative test setting.Thus, the computer-based agent provides on the one hand interactions with learners to support problem-solving processes and on the other hand a diagnostic function (for the use of this particular strength of LUCA, see Paeßens & Winther, 2023c).The measurement environment fulfills the requirements of individual and collaborative learning processes in the way that -Vocational learning takes place largely in social situations -namely, in a business process in which third parties are involved.

Effectiveness of Collaboration in Solving Complex Vocational Problems
-Sub-performance can be assigned to both cognitive and social skills, which are important for problem-solving of complex problems.
-Computer-based agents are used in collaborative test settings to help deal with complexity.

Research Design
The theoretical conceptualization emphasizes that skills needed to solve complex problems is a multidimensional construct composed of two key dimensions, one of which is cognitive skills and the other one is social skills.In this section, we adapt the theory-driven framework of cognitive and social skills involved in complex problem-solving into statistical model to process the data statistically.

Model and Hypothesis
The hypothesized two-dimensionality of the social and cognitive abilities involved in problem-solving in individual and collaborative settings is illustrated in the model below (Figure 3).In this model, the cognitive dimension and the social dimension are two distinct latent dimensions.That is, there are learners who might have high ability in one dimension but not in the other.However, according to the theoretical framework, the two dimensions are related (for the relation between the two dimensions, see Hesse et al., 2015; chapter 1).To test this latent two-dimensional ability construct, the digital authentic assessment described above was used.The assessment is designed to measure the abilities in the cognitive and social dimensions involved in solving the different items.Multidimensional IRT models (MIRT) can be employed to investigate the construct validity of tests with multiple dimensions (Baghaei, 2013;Embretson, 1980;Field, 2013;Janssen & De Boeck, 1999;Santelices & Caspary, 2009;Wilson & Moore, 2011) and assess learners' abilities separately for the dimensions involved.
Therefore, we state the following two hypotheses: 1.The theoretical two-dimensional construct of the test form is valid.To confirm this hypothesis, the two-dimensional construct should be compared with the unidimensional construct regarding the fit indices.
2. Learners show significantly higher ability in both, the cognitive and the social dimension in the collaborative test setting than in the individual test setting.To test this hypothesis, the comparability between the collaborative and the individual test settings should first be confirmed.After the comparison of learners' abilities in both test settings, it should also be analyzed which dimension will be influenced more strongly by the test setting.

Sample
505 commercial apprentices from Germany participated in the study.240 were male, 240 female, 3 divers, and 22 values for sex are missing due to technical problems.The age of the learners ranged from 17 to 42 years (M = 20.92;SD = 2.41).These data were collected at 21 vocational schools and in 28 classes in North Rhine-Westphalia.All participants were in their first year of training.250 of them worked in the individual test setting and 255 were assigned to the collaborative test setting.The assignment was random at class level.

Measures
The 11 items within the task are developed according to theory-based design principles and are empirically validated (Paeßens et al., 2023a;Paeßens & Winther, 2023b).The construction of the items is presented in Table 2.The item "participation" in the social dimension, for example, refers to the sub-performance how a problem-solver considers a test result from a supplier that will be sent to them by email later on.The 11 items are embedded in the webbased office simulation and a questionnaire.Seven items belong to the social dimension and four items to the cognitive dimension.Equal test items were used in two test settings; in the individual test setting, the learners processed the supplier selection alone, while in the collaborative test setting, agents were available as a group partner to interact with the learners.

Statistical Analyses and Results
The analysis presented in this section begins with testing the first hypothesis and involves two primary steps: (1) Comparing model fit between the Unidimensional Rasch model and the two-dimensional Rasch model -and then (2) testing for measurement invariance between individual and collaborative test settings.The primary purpose of these two steps is to examine how the theoretical two-dimensional construct of the test form performs.The analysis then proceeds to the examination of the second hypothesis that the complexity of vocational situations can be dealt with more successfully with collaborative support from a computerbased agent.

Comparing Model Fit: The Unidimensional Rasch Model and the Two-Dimensional Rasch Model
This step aims to test the hypothesis that this is a two-dimensional construct.For this purpose, a unidimensional and a multidimensional Rasch model are applied to the test results and we compare the fits of these models.The results of the ANOVA test are is summarized 58

Effectiveness of Collaboration in Solving Complex Vocational Problems
in Table 3.The two-dimensional model with a significantly smaller model deviance, larger log likelihood, smaller AIC, and smaller BIC fits better to the data compared to the unidimensional model.The theoretical construct of the cognitive and social dimensions is thus, statistically confirmed.

Testing for Measurement Invariance
To test for measurement invariance, we ran five rounds of Multi-Group Confirmatory Factor Analysis (MG-CFA) using the R statistics package.The main outputs of model fits are summarized in Table 4. Firstly, we ran an MG-CFA without cross group equality constraints; this configural model shows a good fit (CFI = 0.975, RMSEA = 0.053, SRMR = 0.061) according to Kline (2010).Following configural invariance, we tested for metric invariance with equality of the factor loadings across groups.Although model fit indices (CFI = 0.963, RMSEA = 0.061, SRMR=0.077)show that the metric model is acceptable, a change of 0.012 in CFI (ΔCFI > 0.01) when comparing the metric model and the configural model implies noninvariance according to Chen (2007).Chen (2007) recommends the following alternative cutoff criteria for model comparisons due to the sensitivity of the most commonly used χ2 test for goodness of fit (Cochran, 1952)

Paeßens, Ma, Winther
This study shows that partial metric invariance is achieved across the test settings when releasing only one item of social dimension among the total 11 items.Then we tested for scalar invariance by constraining the factor loadings and the intercepts across groups.Partial scalar invariance is established when releasing two additional intercept parameters for two items.A total of three items were excluded from further analyses.The reasons why they have significant influence on measurement invariance is discussed further down.The established measurement invariance implies that the instrument assesses the psychometric equivalence of a construct across test settings.

Comparing Abilities: The Individual Group and the Collaborative Group
Before person abilities are estimated using the multidimensional Rasch model, we examine correlations between the two dimensions and reliabilities of the two dimensions.As Table 5 shows, a correlation coefficient of .342 is considered moderate correlation between the cognitive and social dimensions in the individual test setting and is smaller than a correlation coefficient of .542 in the collaborative test setting, which corresponds to a large effect size.This finding supports the theoretical consideration that the correlation between cognitive and social dimensions in the collaborative test setting is stronger than in the individual test setting.The EAP reliability of dimensions ranges from 0.591 to 0.690 (see Table 5).Note that the EAP reliability is sensitive to the length of the test, and two items from the social dimension as well as one item from the cognitive dimension were excluded after testing the measurement invariance.We used the Spearman-Brown formula to predict the reliability of the original test.After Spearman-Brown correction, the lowest EAP reliability is .68,slightly less than 0.7.Since the test is not used as a psychometric scale or an individual diagnosis, this value is considered appropriate for the empirical structuring and description of the ability model.

Effectiveness of Collaboration in Solving Complex Vocational Problems
We can see in Table 5 that the leaners in the collaborative group show higher abilities in both, the cognitive and social dimensions.Two rounds of one-way analysis of variance were conducted to test whether the means of the individual group and the collaborative group differ statistically significantly.The summary of ANOVA shows that the choice of test setting has a significant impact both on the ability of the cognitive dimension (F (1,503) = 38.41,p < .001,partial η2 = .071,n = 505) and on the ability of the social dimension (F (1,503) = 7.64, p < .001,partial η2 = .015,n = 505).The partial eta square is converted here into the effect size f according to Cohen (1992).For the cognitive dimension, the effect size is f = .276and corresponds to a medium effect according to Cohen (1988).Regarding the social dimension, the effect size is f = .123and corresponds to a medium effect according to Cohen (1988).

Conclusion and Discussion
Work processes are complex and are increasing in complexity through transformations.This complexity must be taught in the VET processes.For this purpose, we offer complex learning arrangements and complex vocational tasks.The web-based office simulation LUCA is a complex learning arrangement and, as it were, a TBA; commercial competency development can be fostered and the performance in typical commercial situations can be assessed with it.Two test settings were implemented in LUCA for this study: -Test setting 1: Solving an authentic problem with high complexity alone; -Test setting 2: Solving an authentic problem with high complexity supported by a computer-based agent.
In test setting 1, apprentices must deal with the complexity of the problem on their own; in test setting 2, they receive specific support.It is important that this support is not classic solution support, but rather simulates authentic collaboration situations.Test setting 2 thus extends authenticity.To be able to meaningfully compare both test settings, an indicator model is chosen; in this case, components of complex problem-solving (cf.Hesse et al., 2015).
Complex problem-solving -whether handled with individually or collaboratively -requires the knowledge necessary for problem-solving (regarding both, task regulation and knowledge construction) and social skills (perspective taking, social regulation, and identified participation).While in test setting 2, the cognitive and social aspects are fostered by the agent, this support in dealing with complexity does not exist in test setting 1.The hypotheses are confirmed: (1) The test settings both measure complex problem-solving.The construct is two-dimensional in both test settings.Cognitive and social skills are necessary to solve a complex vocational task.(2) Higher performance is shown in the collaborative test setting.

Paeßens, Ma, Winther
The results confirm that the demands of a complex problem are both cognitive and social and learners are better able to deal with complexity through modeled collaboration than when they solve the task without a tutor.Furthermore, the cognitive dimension is more sensitive to the change of the test setting.This result can be interpreted such that by listening to other viewpoints, considering other positions, and reconsidering own ideas, more cognitive processes are activated to gain a more complete understanding as a group.For the practice of VET, the findings imply that (1), collaboration is effective if the task involved is designed based on didactic principles, especially in terms of its complexity and (2), a collaborative approach actively engages learners to process and synthesize information both in the cognitive and the social dimensions.Regarding the findings in the collaborative test setting, we need to discuss to which extent the learners collaborate.Overall, it can be stated that the interaction with an agent represents a basic form of collaboration.The agent responds adaptively to learner input and, thus, simulates collaboration.To evaluate social embeddedness, Braunstein et al. (2022) develop a theory-driven framework and adapt it for the office simulation LUCA (Rausch et al., 2021).Within the office simulation, the agent can be classified as a social interaction.The use of computer-based agents makes an empirical and future-relevant contribution to VET.Besides this empirical goal, web-based office simulations also have an impact on the practice of VET.VET should respond to digitization, which is changing the world of work and will have employees undergo further training to be able to act competently within the new business processes in companies and on the labor market (Hirsch-Kreinsen, 2017;Dengler & Matthes, 2015).Whether at school or at the workplace: Collaboration should be facilitated and integrated into learning and working processes.
The complex LUCA environment has also important implications for the way researchers and practitioners organize collaboration.Firstly, as a learning environment, LUCA can provide instructional interventions, learners receive different kinds of hints and scaffolds from the computer-based agent to construct their collaboration process in the form of humanto-agent interaction.Secondly, a strength of LUCA is that it combines a specific pedagogy with integrated measures, so it is also an innovative assessment tool.Collaboration at the highest level -between colleagues or learners as a form of human-to-human interaction (see framework Braunstein et al., 2022) -requires different tools for measuring interaction.The tools must be able to evaluate the collaboration process in a standardized and, ideally, automated way.Motivational and emotional factors influence the collaboration process.Various approaches impressively show that attributing individual performance in a group process is challenging (vignettes in King et al., 2004, forced decisions in Salgado & Táuriz, 2014, thirdparty evaluations in Connelly & Ones, 2010;Oh et al., 2011, Situational Judgment Test in Motowidlo et al., 1990;Weekley et al., 2004;Whetzel &McDaniel, 2009, cooperative games andsimulations in Griffin, 2017;Hao et al., 2017, multiplayer games in Zhu & Bergner, 2017).

Effectiveness of Collaboration in Solving Complex Vocational Problems
In PISA 2015, collaboration with automated agents was standardized for learners (Graesser et al., 2018;He et al., 2017).The innovation here was that individual performance, particularly social and cognitive skills, could be observed at individual level in a group setting (He et al., 2017;Organisation for Economic Co-operation and Development, 2017).Standardization seems necessary in an LSA, although standardization could also be achieved using an external test instrument to measure collaboration.The use of automated assessments, such as those generated by RIFF, has innovative potential.Automated evaluation promises to observe learners' collaboration performance in real time and make it accessible to empirical assessment.While it has, so far, only been possible to realize the attribution of individual performances with the help of agents in collaborative assessments (cf.Graesser et al., 2017;He et al., 2017), in RIFF, learners can collaborate with each other "for real".Instructional designs have been developed in RIFF that can be used to build and develop technical as well as collaboration skills for commercial apprentices (see Paeßens & Winther, 2021;Paeßens et al., 2022; findings of this study).Collaboration is not the ideal modus operandi in every learning and working situation.The problem has to be complex and different learners have to bring in their specific knowledge to work out a solution.In the future, it should be worked out which complex (sub) tasks are suitable for collaborative settings to foster learners' abilities in the collaboration process as well as in the collaborative performance (first approaches in Ma et al., 2022).Another limitation of the study is that we limited the number and range of items for this first-time validation.We recommend that future research extends the items by taking domain-specific and domain-related competencies into account.

Biographical Notes
Jessica Paeßens is a PhD student and her research focuses on collaborative problem-solving among commercial apprentices.The research, which is both empirical and relevant for (school) practice, focuses on the extent to which collaboration can be empirically measured at individual and group level.Measuring collaboration at the individual level is challenging; she uses innovative digital tools from practice for it.With the help of these tools, she explores how collaboration processes and the resulting performance are related and which tasks are suitable for collaborative problem-solving.

Figure 1 :
Figure 1: Representation of the Primarily Social Sub-Performance "Perspective Taking" in the Office Simulation LUCA

Figure 2 :
Figure 2: Simulated Interaction with a Colleague "Rene Eßer" and Response of the Computer-Based Agent in the Form of an Email From the Colleague

Figure 3 :
Figure 3: Schematic Illustration of the Two-Dimensional Rasch-Model Incorporating Between-Item Multidimensionality (adapted fromHartig & Höhler, 2009) to sample size: For testing loading invariance with a sample size larger than 300, a change of ≥ -.010 in CFI, supplemented by a change of ≥.010 in RMSEA or a change of ≥.030 in SRMR would indicate non-invariance; for testing intercept or residual invariance, a change of ≥ -.010 in CFI, supplemented by a change of ≥ .010 in RMSEA or a change of ≥ 0.10 in SRMR would indicate non-invariance.

Beifang
Ma is a PhD student and her current work focusing on psychometrics and educational measurement includes the structural equation modeling and validating of instruments in the research area of competence assessments in vocational education and training.She works on a project investigating the growth of professional competence and decisive context factors influencing it in vocational education and training.She aims at characterizing the trajectories and causal relationships of professional competence development to optimize the quality of training and education.Esther Winther, Dr rer.pol.habil., is Full Professor of Vocational Education and Training, her main research interests are in the areas of empirical teaching-learning research with focus on psychometric competence modeling and measurement, the development of training and continuing education programs, and the conception of innovative and digital teaching and learning scenarios for professional and operational fields of action.As a project participant in large national research networks (DFG priority program, SAW research funding, BMBF research initiatives ASCOT and ASCOT+), she develops innovative and workplace-oriented assessments that record the status quo and the foreseeable development of vocational competences.

Table 1 :
Cognitive and Social Skills Needed to Solve Complex Problem-Solving Tasks

Table 2 :
Indicators Measuring Sub-Performance for Cognitive and Social Skills Task regulation: Attaching the modified benefit analysis to the email Perspective taking: Transfer of the weighting into the benefit analysis Participation: Considering the test result sent by email later on

Table 3 :
Model Fit Statistics for the Unidimensional and Multidimensional Models

Table 4 :
Model Fit Incidents for 3 Levels of Measurement Invariance

Table 5 :
Correlations and Reliabilities of Dimensions and Average Person Abilities on Dimensions