The Interaction Effect of Group Formation and Individual Preparation on Computer Support Collaborative Engineering Design: An Exploratory Study

Collaboration, as one of the most essential skill sets for engineering students, remains challenging for both engineering educators and student engineers. This study explores the role of pedagogical support individual preparation before collaboration on engineering students’ collaborative learning outcome as well as the process. A total of 36 dyads engineering students from a university engineering class participated in the study, divided into 2 group formations: homogeneous group (two similarly experienced student engineers) and heterogeneous group (one experienced student engineer and one less experienced student engineer). All dyads completed two engineering design tasks in two conditions: immediate collaboration (control condition) and individual preparation before collaboration (experimental condition) in a face-to-face (F2F) computer-supported collaborative learning (CSCL) context. The results indicated the interaction effect of individual preparation and group formation on students’ collaborative knowledge co-construction. The homogeneous group formation produced higher quality knowledge co-construction and design solutions when there was an individual preparation before collaboration than immediate collaboration. Meanwile, the heterogeneous group formation produced lower quality knowledge co-construction and design solutions when there was an individual preparation before collaboration than immediate collaboration. These findings expand the current understanding of individual preparation before collaboration and provide insights for the design of computer-supported collaborative learning in classrooms.


Introduction
Over the last three decades, global educational stakeholders put increasing emphasis on developing comprehensive competencies in pre-service engineers, including creativity, lifelong learning, communication skills, teamwork, and collaboration (Jamison et al., 2022). However, high-quality collaboration is not easy to be practiced in current engineering classrooms. Researchers have identified challenges in the knowledge coconstruction process in university-level engineering classrooms: lack of idea diversity and innovation, make minimal contributions, superficial discussion and analysis, and hasty decision makings (Marra et al., 2016). To address these challenges, appropriate technological and pedagogical support for collaboration is needed to enhance students' collaborative learning in engineering classrooms (Jamison et al., 2022).
Since the 1980s, the field of Computer-Supported Collaborative Learning (CSCL) builds the relationship between the social interactions in learning and computational objects, exploring how to help groups of learners achieve higher level collaboration with digital technologies (Hmelo-Silver & Jeong, 2020;Koschmann, 1996). The technology environment itself does not guarantee successful collaborative learning, appropriate pedagogical support is needed to facilitate students' productive collaborative learning in engineering classrooms. One pedagogical approach is to individually prepare learners before their collaboration (e.g., Lam & Muldner, 2017;Tsovaltzi et al., 2015). Individual preparation before collaboration refers to "devoting a part of the learning time for learning individually prior to learning collaboratively" (Mende et al., 2021, p.30). Though being applied in various collaboration script practices (Loll & Pinkwart, 2013), the effect of individual preparation before collaboration remains underexplored in the engineering design context. In particular, it is still an open question about its effect on student engineers' design outcomes and processes, considering the fact that different group formations of student engineers have different levels of engineering experiences. This exploratory study examines the role of individual preparation before collaboration, when applied in different group formation situations, on students' design solution quality as final products as well as their engineering design strategies applied in the knowledge construction process.

Literature review Individual preparation before collaboration
One valuable instructional approach widely applied in the CSCL context is collaboration scripts Rummel & Spada, 2007). Collaboration scripts are designed to "promote productive interactions by designing the environment such that suggestions of different degrees of coercion are made to the collaborating students, engaging them in specific activities that otherwise might not occur" (Weinberger, 2011, p. 190). The scripted collaboration usually guides students on what to do, what roles to play, and what sequences of activities to perform during collaboration (Carmien et al., 2007). One component or activity that has been involved in various collaboration scripts is the individual preparation phase before students started talking (Loll & Pinkwart, 2013), also known as individual preparation before collaboration (Mende et al., 2021).
The mechanism of individual preparation before collaboration can be explained by the preparation paradigm brought up by Lam and Kapur (2018). The main role of individual preparation before collaboration, compared with direct collaboration, lies in several aspects: activating prior knowledge, exposing knowledge gaps, facilitating engagement, and increasing sensitivity to noticing (Lam & Kapur, 2018). The individual preparation before collaboration, recognised in previous empirical studies (e.g., Beers et al., 2006;Farrokhnia et al., 2019), showed higher learning gains (Salomon, 1997;Stahl, 2006), higher motivation for group work (Van Boxtel et al., 2000), better collaboration products (Engelmann et al., 2014;Engelmann et al., 2009) as well as more indepth knowledge co-construction discourse (Tan et al., 2021). Meanwhile, a series of studies by Tsovaltzi et al. (2015) identified knowledge solidification as a result of individual preparation before collaboration, and that students became less likely to accept alternative opinions during the following discussion, which directly interrupted the knowledge co-construction at the group level. In summary, the empirical studies of the effect of individual preparation before collaboration had mixed findings: both cognitive activation advantages and possible coordination challenges (Mende et al., 2021).
In addition, only a few collaborative learning contexts and subjects have been investigated in the existing literature: ecology concept mapping activities, physics problem solving, collaborative argumentation, and collaborative writing. Most individual preparation studies were conducted in K-12 school settings, making it difficult to transfer empirical implications to university-level learning contexts. It remains under-explored what is the effect of individual preparation before collaboration in the university-level engineering problem-solving context, which requires ongoing idea generation, problem analysis, negotiation, and decision-making practices between student engineers in a group.

Group formation
To optimise collaborative learning efficiency, the application of pedagogical supports requires systematic considerations such as the collaborating students' gender, ethnicity, motivational level, learning ability, and familiarity with each other, as identified by Lei et al. (2010) and Chen and Kuo (2019). One of the commonly discussed factors is group formation (heterogeneous or homogeneous) in terms of the learners' level of knowledge related to the learning task (Chen & Kuo, 2019). A series of studies by Webb (1984); Webb et al. (1995) found that students with a higher level of knowledge contributed more to the collaboration work and gave more explanations while the students with a lower level of knowledge tend to be off-task. A heterogeneous group had a better teacher-student relationship and more meaningful peer interactions than homogeneous groups.
The prior experiences and skillsets in engineering design play an important role in pre-service engineers' participation and contribution to the collaboration process (Song & Becker, 2014), as demonstrated in their engineering design strategies and their decision-making approaches (Atman et al., 2007;Dym et al., 2005). Novice engineers and expert engineers tend to adopt different collaboration strategies when solving complex problems (Song & Becker, 2014). For example, experienced engineers spent more time exploring and framing the design problem while less-experienced engineers would rush to easy solutions without deep research. Experienced engineers tend to make critical evaluations and analyses surrounding each solution and alternative solutions while less-experienced engineers overlook the potential risks and limitations. From the knowledge coconstruction perspective (Weinberger & Fischer, 2006), the different decision-making ways represent the different consensus-building approaches: quick consensus building, integration-oriented consensus building, and conflict-oriented consensus building. The hasty decision-making of less experienced engineers was related to lower quality of knowledge co-construction when the members barely build on each other's contributions (Deken et al., 2012). The experienced engineers may go through higher level of knowledge co-construction interactions when they were engaged in critical and analytical discourses and integrate multiple perspectives before making common decisions (Atman et al., 2007;Weinberger & Fischer, 2006).
As indicated in the preparatory mechanism (Lam & Kapur, 2018), one essential role of an individual preparation activity is to activate learners' prior knowledge and get them ready for a more critical and analytical group discussion activity. From this perspective, the less-experienced students are given more chance to understand the design problem before joining the discussion, so that they may contribute to the collaboration with more indepth discussions, resulting in a higher quality of knowledge co-construction. Experienced students, however, are likely to get either "more-prepared" or "over-prepared" when they are given more time to analyze the given problems, generate their own solutions, and probably get a fixed standpoint, as reported in Tsovaltzi et al. (2015)'s study. The preparation activity, therefore, may either benefit or hinder the subsequent knowledge coconstruction process for experienced students. Therefore, individual preparation may affect the collaboration process and outcomes of different group formations with a different or similar levels of engineering experiences in different ways.

Research Questions
With an interest in supporting and optimizing engineering students' collaborative learning process, this study provided pedagogical support for collaborative learning in engineering classrooms by taking students' prior engineering experiences into consideration, as the individual preparation may have a differentiated effect on the groups with different level of engineering experiences when they understand, analyze, and make decisions on the given problems in the engineering design. The groups were composed in two ways: more experienced student engineers and less-experienced student engineers (heterogeneous dyad) and similarly experienced student engineers (homogeneous dyad). The study is to examine whether and how group formation factor could mediate the effect of the individual preparation before collaboration on engineering students' collaborative learning processes and outcomes in engineering problem-solving settings, in particular, the knowledge coconstruction patterns demonstrated in different group formations. There are two research questions to be addressed: 1. Does individual preparation before collaboration influence the quality of engineering design solutions in different group formations in terms of students' prior experience? 2. Does individual preparation before collaboration influence the engineering design strategies in different group formations in the process of knowledge construction?

Research Design
This study employed an explanatory mixed method design approach, with both quantitative and qualitative data collected and analysed to examine students' collaborative learning under different individual preparation conditions. The quasi-experimental design was employed to compare the design solution quality in the two conditions across two group formations to address the first research question. To further explain the difference in collaboration product quality among the two group formations identified in RQ1, two dyads, one homogeneous dyad, and one heterogeneous dyad were randomly selected to examine the collaboration processes and outcomes in detail.

Participants and learning context
A total of 72 students in a 4th-year engineering mainstream class "Mechatronics Engineering Design" at a Singapore university participated in this study. These students were engaged with a series of conceptual engineering design projects on an autonomous robot. The overall design problem is to design an autonomous robot that is able to perform certain functions in the given arena. To solve these design problems, student engineers were expected to analyze the design problem, identify the required functions, and generate alternative design solutions.
The learning environment was face-to-face CSCL. During the task, each dyad sat face-to-face with every student having one laptop/tablet. Students' collaboration was supported by the online collaboration platform Miro (miro.com), shown in Figure 1. This platform was chosen in this class as it allows collaborating students to co-edit, sketch, and use multi-media tools to represent their idea in real-time, which are essential practices required in early-stage engineering design. Each dyad finished two design problems following two conditions in this study: immediate collaboration (control condition) and individual preparation before collaboration (experimental condition). In the control condition, students collaborated on the shared digital board from the very beginning for one hour. In the experimental condition, students did preparation on their individual digital board for 15 minutes, which contained the same content as their collaboration board, and they move to the shared digital board and discuss for the rest of 45 minutes. This 15-minutes individual preparation invited students to examine the design problem and generate at least one design solution.

Figure 1: An exemplar shared digital board
The participants were randomly assigned to 36 dyads. There were two group formations: homogeneous (similarly experienced student engineers, N=21) and heterogeneous (one experienced and one less-experienced student engineer, N=15). The "experience" here refers to the students' prior engineering design experiences that were self-reported in the pre-survey. The more experienced student engineers were those who graduated from robotics engineering in polytechnic college before joining the university, who participated in relatively more mechanical engineering design projects than less-experienced student engineers who began robotics engineering training at university. The experienced student engineers had studied robotics engineering for more than 6 years before registering for this class while the less-experienced student engineers had studied robotics engineering for less than 4 years.

Data collection and research instruments
The data collected included each group's engineering design solutions in their shared digital boards and their verbal discussions. Content analysis was conducted to evaluate each group's engineering design artifacts in the digital board based on the coding scheme. The unit of analysis is one group's design solutions presented in the online platform. The coding scheme for engineering design solutions was adapted from the evaluation of the design concepts model (Roozenburg & Eekels, 1995) according to the design problems in this study context. It has four dimensions: diversity, elaboration, novelty, and functionality. Each dimension was rated on a scale of 0 to 5. Two expert engineers (the class instructor and another expert engineer outside this class) coded the quality of all the dyads' design solutions in two conditions. The Cronbach's alpha for the four dimensions are: 0.97 for diversity, 0.97 for elaboration, 0.97 for novelty, and 0.96 for functionality. The inter-coder reliability was high for all dimensions. The quality of each group's design solution was calculated by averaging the scores of four dimensions.
This study randomly chooses dyad A and dyad B from the homogeneous and heterogeneous group formations respectively. Their verbal discussions were transcribed into transcripts and qualitative content analysis was conducted to identify the different collaboration behaviours based on the coding scheme. The unit of analysis for coding was one utterance of a distinctive idea conveyed by students in the same group. The coding scheme of students' verbal discussion was adapted from the CommonKADS conceptual modelling language framework which has been widely applied to analyze collaboration and communication in engineering design contexts (Santirojanakul, 2018). To investigate students' knowledge of co-construction in engineering design, this study adopted the argumentative knowledge co-construction framework (Weinberger & Fischer, 2006) from consensus-building perspectives, including quick consensus building, integration-oriented consensus building, and conflict-oriented consensus building. Two trained coders coded the data. The inter-coder reliability was good (Cohen's Kappa=0.702). The explanation and examples of each code are given in Table 1.

Effect of individual preparation before collaboration and group formation on engineering design solutions quality
To answer RQ1, 36 dyads' design solutions were coded and compared under the two conditions. A two-way ANOVA was conducted to understand if there was an interaction between the two factors: group formation and conditions. There was no significant main effect of condition, F(1, 34)=0.315, p=0.578. This indicates that the 36 dyads had no significant difference between the two conditions. There was no significant main effect of group formation, F(1, 34)=0.118, p=0.734. This indicates that the design quality of the homogeneous group had no significant difference from that of the heterogeneous group. However, there was a significant interaction between the group formation and the condition, F(2, 36)=6.25, p=0.017. This indicates the difference in design solution quality between the control condition and the experimental condition is not equivalent in the two group formations. The homogeneous group showed a significantly higher score in the experimental condition than the control condition, F(1, 36)=5.623, p=0.024, while the heterogeneous group showed a higher score in the control condition than the experimental condition. As shown in Figure 2, for homogeneous groups, their artefact quality mean score was higher in the experimental condition and lower in the control condition. While for the heterogeneous group, their artefact quality mean score was higher in the control condition and lower in the experimental condition.

Effect of individual preparation before collaboration on students' engineering design strategies in different group formations
To explain the different effects of individual preparation on different group formations, the verbal discussions of two dyads A and B chosen from each group formation were further analysed using Epistemic Network Analysis (ENA) to identify the connections among the different collaboration behaviours in different conditions for each group. ENA is a learning analytic method that can quantify the co-occurrence of different codes and identify the connections among elements of interest (Shaffer et al., 2016). The two groups' collaboration behaviours codes with their timestamps were imported into ENA web tool version 3.0 (http://www.epistemicnetwork.org/) to model their engineering design networks under different individual preparation conditions. The size of stanza was set to four, meaning that the ENA calculated the co-occurrence of the codes in every four collaboration behaviours. For each group, there were two network graphs created for the two conditions, indicating the connections among different engineering strategies and consensus-building approaches being applied by each group.
ENA produced a weighted comparison network result for groups A and B respectively, shown in Figures 3 and  4. In this network graph, each node represents one collaboration behaviour code. The connections among the different nodes indicate the co-occurrence of the two codes. The thicker lines represent stronger and more frequent connections between elements. The comparison plot was retrieved by subtracting the same group's networks under two conditions (one colour representing one condition), in which the connections in the experimental condition are subtracted from connections in the control condition. The colour represents a certain condition that has a stronger connection in their network. Figure 3 is the comparison plot of homogeneous group A (similarly experienced student engineers) in two conditions. The purple lines indicate connections stronger in the control condition and the pink lines indicate connections that are stronger in the experimental condition. Several patterns can be identified based on the ENA result. First, the two experienced students tended to focus more on design problem definition, by talking about the design goals, given resources, and design considerations. They tended to work on the detailed design specifications by checking the resources available, such as the quantity and size of sensors and motors. When there was an individual preparation activity before the discussion, the two experienced students spent more time on solution generation and solution analysis, when they tended to generate alternative solutions and conduct a comprehensive evaluation of the solutions by discussing and weighing the pros and cons of the alternative solutions. More integration-oriented consensus building and conflict-oriented consensus building were applied instead of quick-consensus building.  Figure 4 is the comparison plot of heterogeneous group B (one experienced student engineer and one less experienced student engineer) in two conditions. The orange lines indicate connections stronger in the control condition and the blue lines indicate connections that are stronger in the experimental condition. The effect of individual preparation for group B can be recognised in the ENA result, whereas the two conditions weighed towards the left and right part of the space respectively. In the immediate collaboration condition (orange lines), the two students proposed multiple alternative solutions by simulating the task contexts together, such as how the ball goes and how to deal with the opponent's attack. These solutions were mostly produced in the integration-oriented consensus-building approach, as two student engineers tried to integrate each other's opinions. When there was individual preparation before collaboration, the two students altered the way of solution generating and made a great effort to determine the design specifications of one solution. Interestingly, the quick consensus building stood out in this process, indicating mostly rushed decision-making and few common contributions from both collaborators.
To summarise, the ENA results report different patterns of collaboration behaviour connections for the two group formations under different individual preparation before collaboration conditions. For both homogeneous and heterogeneous groups, student engineers applied different ways of defining the design problem, solution generation and analysis as well as decision-making under the two conditions. However, the individual preparation activity played different roles with different group formations. In homogeneous group A, the individual preparation activity encouraged students to move beyond one single solution, come up with alternative solutions, and engage in critical evaluation and analysis of each solution. These strategies represented more informed engineering design thinking activities and higher quality of knowledge coconstruction. The individual preparation activity for the heterogeneous group B, however, played a different role in the way they worked on the design problem. With an individual preparation, group B was less likely to embrace multiple perspectives and integrate both sides' opinions, instead, there were overarching discussions surrounding the design specifications surrounding one solution via continuously quick consensus buildings. In comparison, the immediate collaboration condition for group B seemed to involve higher quality of knowledge co-construction and multiple perspectives.

Conclusions and Discussions
This study examined the effect of individual preparation before collaboration on engineering students' collaborative learning in different group formations. The groups with similar experience tend to produce higher quality design solutions in the individual preparation condition than the immediate collaboration condition. The group with different experience tend to produce lower quality design solutions in the individual preparation condition than the immediate collaboration condition. The content analysis of the two groups' verbal discussion revealed that the individual preparation played different roles with different group formations in terms of how student engineers define the design problem, generate solutions, analyze solutions, as well as make decisions. Heterogeneous groups tended to reach consensus and make decisions by integrating both sides' opinions when there was an individual preparation, compared to more quick-consensus building practices in the immediate collaboration condition. These behaviours all indicated the higher-order thinking process when students were more "prepared" for the teamwork (Mende et al., 2021), which also corresponds to the higher quality of their design solutions as the collaboration product (Farrokhnia et al., 2019;Tan et al., 2021).
The individual preparation before collaboration, however, played a different role in the heterogeneous group consisting of one experienced and one less experienced student engineer. The heterogeneous group were less likely to integrate each other's contributions after individual preparation but instead, rushed to decision-making surrounding the design specifications. In contrast, the immediate collaboration condition seemed to allow more transactive dialogues when multiple solutions were discussed and evaluated before the two collaborators made common decisions. This surprising finding can be explained by the possible disadvantage of individual preparation before collaboration, pointed out by previous scholars (Mende et al., 2021), that learners may experience knowledge solidification if they were given some time to develop their own ideas before joining the discussion. In this study, the more experienced student engineer ended up developing his own design solution in the 15 minutes and he was keen on detailing this solution in the following discussion, during which the less experienced student engineer may find it difficult to contribute his ideas to the conversation.
These findings contribute to the existing individual preparation before collaboration studies in the field of collaboration scripts with its application and examination in the real-world university-level engineering classroom context. The in-depth analysis of different group formations shed light on a strategised and adaptive use of this pedagogical support in CSCL practices. Besides, the process-oriented analysis approach illustrates the importance of evaluating both design outcomes and design processes to realise a comprehensive understanding of collaborative learning in engineering classrooms. There are limitations in this study. Firstly, the university engineering classroom context and the specific design tasks may pose challenges to transferring the findings to other learning contexts. Second, there was a relatively small sample size due to the real-world classroom limitation, making it difficult to explore the interaction impact between different group formations and different individual preparation designs on student design solution quality. Bigger scale experimental studies are needed to identify the possible interaction effect of group formation and individual preparation on students' collaborative learning.