Ill-structured problem-solving is increasingly seen as an important skillset in everyday life (Jonassen, 1997). Indeed, the recent emphasis on so-called 21st century skills require an array of problem-solving competencies, including critical-thinking, information literacy, and collaboration (Graesser et al., 2020). As theorists reflect on the skillsets needed in practice, educators have explored various ways to facilitate problem-solving in the classroom to better support domain-specific knowledge structure development, which is an important element of effective problem solving (Renkl, 2011). One of the most widely used approaches includes problem-based learning (PBL), which asks learners to solve ill-structured problems that are similar to the types of problems that practitioners employ (Barrows, 1996). Rather than a lecture about the concepts in a decontextualized manner, learners try to resolve the ill-structured case with their peers as they define the problem space (Dolmans et al., 2016). Teachers in this instructional strategy adopt a more facilitative role as they guide inquiry and provided requisite scaffolding through the problem-solving process. Theorists (Schank, 1999;Tawfik & Kolodner, 2016) argue that this experiential approach to learning affords opportunities for complex reasoning and comprehensive knowledge structures (Tawfik et al., 2020;Tawfik & Kolodner, 2016). Large scale reviews of PBL and similar strategies show gains in deep learning (Dolmans et al., 2016), conceptual knowledge (Car et al., 2019;Sayyah et al., 2017), performance (Lazonder & Harmsen, 2016), and long-term retention (Yew & Goh, 2016).
An important element of PBL is how learners interact with their peers, which requires individuals to engage in collaborative meaning-making and co-construction of knowledge (Suthers, 2006). In the first stages of collaborative problem-solving, learners focus on problem representation to describe the relevant variables and causal reasoning for why the issue occurred within the case (Delahunty et al., 2020). As an individual moves toward solution generation, learners work with the peers to resolve the case through collective decision-making and describe the justification for their solution (Sharan et al., 2013). The importance of collaborative problem-solving is especially important in online contexts where the learner-learner interaction is most pronounced (Hara, 2000;Janssen & Kirschner, 2020;Jeong & Hmelo-Silver, 2016). Indeed, a recent study by Zhu et al. (2020) found that learners' continuous intention was mediated through multiple interactions, including those of their peers. Related research shows that peer scaffolding in online learning is a key determinant in predicting learning outcomes as they collectively solve problems (Shin et al., 2020).
One way to understand problem-solving is through the lens of case-based reasoning (CBR) theory, which describes the ways in which an individual retains a case and reuses prior experiences. Although research shows that learners share ideas with relative ease (Matuk & Linn, 2018), they rarely engage in discourse that challenges, build on the ideas of their peers (Lucas et al., 2014;Saqr et al., 2020), regulate tasks (Hadwin et al., 2018), and sustain interaction (Uttamchandani et al., 2020). This is problematic for problem solving because peers serve as an important scaffold as learners interact with differing perspectives and divergent evidence presented by others during problem-solving. Moreover, problem-solving and knowledge construction are highly integrative (Wang et al., 2013), so it is important to explore how understanding of a case is refined over time and across multiple learning cycles. However, there is a lack of studies to explore the knowledge building process involved in problem solving, especially with an eye toward identifying successful and less successful problem-solvers. Thus, our main aim in this study is precisely to specify the collaborative problem-solving processes leading to successful or less successful problem-solving performances by visualizing and tracking individuals' and groups' knowledge structures before, during, and after collaborative problem-solving.
Deep learning of a domain entails more than just being able to recall information, and also includes the ability to understand the structural characteristics of a phenomena and transfer solutions toward a novel situation (Belland et al., 2009). Such applications are maximized when learners arrive at a level of comprehension that reflects the underlying domain-specific knowledge structure (Linn, 2000). Based on seminal theory articles and empirical literature, knowledge structure here is defined as the organization of domain key concepts stored in long-term memory (Clariana, 2010).
Knowledge structure is important for problem solving in multiple respects. For example, Trumpower and Sarwar (2010) note that "knowledge structures … play a more direct causal role in enabling good performance" (p.427). Shin et al. (2003) reported that complex, ill-structured problem-solving scores were most predicted by knowledge structure scores among other cognitive and non-cognitive factors, suggesting that developed knowledge structures likely lead to better solutions for ill-structured problems. These and other studies between experts and novices have demonstrated that success in ill-structured problem solving depends on the content and the structure of knowledge about the domain (Tawfik et al., 2020).
In terms of applying theory to practice, theories argue that PBL best supports knowledge structures because of the rich contextual information embedded within the case (Schank, 1999;Tawfik & Kolodner, 2016). According to case-based reasoning (Schank, 1999;Tawfik & Kolodner, 2016), as an individual engages in problem-solving, they reference prior cases to solve the new case. If they deem the solution as relevant, they will map that knowledge structure onto the new problem. If not, the learner will engage in further inquiry to resolve the case. That said, novices can often struggle to develop an appropriate knowledge structure about the phenomena without detailed and structured guidance from a more knowledgeable peer. When learning through cases and problem-solving, educators invite the learner to articulate their existing knowledge structure with their peers, provide normative models/cases, distinguish between the normative models and their preexisting knowledge structure, and reflect on what was learned and so move toward the knowledge structure they are targeting. Thus, learners' knowledge structures are constantly refined as they recognize, define, and organize new situation during collaborative problem-solving. In this view, knowledge structure is not a static, fixed property; rather, it is a dynamic and dependent upon the individual's prior knowledge and collective knowledge among other learners.
Learners continually learn and update their knowledge structures through problem solving. CBR theory argues that problem solving and knowledge construction are highly integrative and reciprocate each other (Wang et al., 2013). It is thus crucial to understand how knowledge structure can be better consolidated and so advanced in problem solving process. Gogus et al. (2009) proposed that the progress of learning can be represented as the change of knowledge structure in the direction toward expert-like knowledge structure. Therefore, knowledge structures serve as a way to understand and measure one's deep learning (e.g., problem solving). However, itis difficult to capture the complex structure and process of knowledge structures during problem solving since both problem solving and knowledge construction are complex cognitive processes. Although many efforts have been made to examine the post-hoc outcomes of knowledge structure as a problem-solving performance by the characteristics of learners, tasks, and contexts, there has been very little studies to explore both the outcomes of knowledge structure and the processes that lead to the outcomes related to problem solving. To address this gap, this study analyzed problem-solving processes and outcomes between successful vs. less successful problem-solvers using a network approach as to obtain a comprehensive picture of how individuals engaged in collaborative problem-solving. Specifically, we are interested in gaining significant insights into key research questions such as:
We employed ill-structured problems in the contexts of STEM, namely physics, astronomy, and biology (see Appendix for an example). Specifically, students were instructed as such: define problems and goals, search and select appropriate information, organize selected information, choose a potential solution, and develop justification of their solutions and selections. Accordingly, the ill-structured problem-solving scores in the analysis are based on the qualities of selecting appropriate information, organizing the selected information, choosing a potential solution, and developing justifications of the solution (Shin et al., 2003). The research team, consisting of one researcher, nine content expert teachers (three teachers per subject), and two experts in test development, developed this set of ill-structured problem items and rubrics in Korean for these students.
Prior to collaboration, the triad members were individually required to watch course-related 30-minute video lectures developed by three teachers for this investigation. The video lectures were designed to include both problem-related and solution-related contents. After watching the video, participants were asked to map the entire video lecture based on their understanding of the contents using the web-based mapping system called, Graphical Interface of Knowledge Structure-Map (GIKS-Map). Participants accessed the GIKS-Map embedded in the online OSS system with their assigned individual ID and then later asked to create their Premaps. All were provided with the same list of 20 key terms from the lecture that they could use for their individual Premaps with the statement, "Use any appropriate words in your concept map, but here are a few important words that you could use", i.e., open-ended concept mapping. The provided 20 key terms were selected by the three teachers from each subject, including both problem-related and solution-related key terms. The participants worked alone at their own pace to create their Premaps, but on average they spent about 15-20 minutes to complete the Premaps. All of the individual Premaps were then converted into network graphs for analysis by the GIKS-Map (the procedure is detailed below).
During the collaboration phase (one day after the pre-collaboration), triad members worked together online in a synchronous collaboration mode in the OSS setting that allows for video communications (40 minutes). The group task was to determine how to resolve the given problem. The students' verbal communications were recorded and converted into network graphs for analysis by GIKS-Voice embedded in the online OSS system (the procedure is detailed below).
After the collaboration (one day after the collaboration), all participants were individually required to write a problem-solving essay using the GIKS-Text. Participants accessed the GIKS-Text embedded in the OSS system with their assigned individual ID then they were asked to write and submit their problem-solving essays (30 minutes). Their essays were then converted into network graphs by GIKS-Text for analysis (the procedure is detailed below).
All of participants' mapping (before collaboration), speaking (during collaboration), and writing (after collaboration) were converted into Pathfinder Networks (PFnets), a graph-theoretic psychometric network scaling measure (Tossell et al., 2010), in order to compare each PFnet to one another (e.g., participants' map to map, map to writing, speaking to writing, etc.) and also to expert-derived problem and solution referent PFnets. The Pathfinder algorithm is a psychometric data reduction network scaling approach which is hypothesized to capture the underlying organization or structure of the data (see for details Tossell et al., 2010). Pathfinder scaling has been successfully applied to reveal the strongest associations in sets of associations by removing less important or weak association data across highly diverse domains, for example, from satellite images studies (Barb et al., 2013) to brain image studies (Li & Clariana, 2019). In this investigation, the PFnets are used to represent the strongest connection between keywords from participants' mapping, speaking, and writing as a proxy of knowledge structure.
Following Clariana et al. (2013), three teachers from each subject (physics, astronomy, biology) worked together to establish the (a) problem referent maps that contained the information of the fully explicated problem space and (b) solution referent maps that contained only the solution-relevant information. For creating the referent maps, the three teachers were provided with a list of all the terms used by the participants in their Pre maps, Group speaking, and Post essays (represented as Pre-PFnets, Group-PFnets, Post-PFnets), arranged in order of frequency of occurrence. While considering this list and the lesson content, the teachers collaborated face-to-face to reach a consensus on the essential terms for the problem space and for the subset solution space for all three subjects; these terms were then used to establish the problem PFnets and solution PFnets (see Figure 1 for example). This problem and solution PFnets were used as the referent PFnets for comparing to the students' Pre-, During, and Post-PFnets.
The software tool GIKS-Map (Kim, 2017) was employed to convert students' individual Pre maps into PFnets. The GIKS-Map works by capturing the raw proximity data as the pair-wise distance between terms in the map (Tang & Clariana, 2017) and then transformed the proximity data into PFnets using Pathfinder network algorithm embedded in the GIKS-Map. In this study, we claim that the PFnet from the GIKS-Map represents the most salient connections between key concepts in the original maps, or knowledge structure of the map.
The software tool GIKS-Text (Kim et al., 2019) was used to convert essays to PFnets. The GIKS-Text system works by capturing the raw proximity data as the sequence of selected key terms in a text, adding only "1" or "0" to indicate the sequential occurrence of the key terms in the text. Then the pair-wise term sequence data from a text can be visually represented as PFnets using Pathfinder algorithm embedded in the GIKS-Text as in the process for maps to PFnets (see for details, Clariana et al., 2014;Kim et al., 2019;Kim & Clariana, 2017). Here we claim that the resulting PFnet from the GIKS-Text represents the most salient linkages between key concepts in the text, or knowledge structure of the text.
The software tool GIKS-Voice was used to convert speaking to PFnets. GIKS-Voice is an extension of GIKS-Text by adding IBM Watson's speech-to-text engine, broadly recognized as the most reliable of its kind (Moslehi et al., 2016). Thus, the GIKS-Voice can convert people speaking into a text transcript that is then transformed into PFnets as in the GIKS-Text. Here we claim that the resulting PFnet from the GIKS-Voice represents the most salient linkages between key concepts in individual speaking, or knowledge structure of the speaker.
All the PFnets derived from participants' mapping, speaking, and writing were analyzed and compared by two graph-theoretical measures, including network similarity of the PFnet (e.g., Kim & Clariana, 2019) and (3) decree centrality of the PFnets (e.g., Clariana et al., 2013) because the different methods capture different aspects of structural similarity inherent in the mapping, speaking, and writing data. First, we compared PFnets by network similarity, calculated by common links divided by the average number of links in the two PFnets, with the value of 0 (no similarity) to 1 (perfect similarity). PFnet similarity scores have been extensively and empirically used in various studies (Clariana et al., 2014;Coronges et al., 2007;Draper, 2013). To address the research gap, we calculated the similarity (Research Question 1) to one another in order to assess how collaboration impacts the success vs. less success problem-solvers' knowledge structure by comparing their PFnets before, during, and after collaboration and (Research Question 2 and 3) to the referent PFnets in order to assess how the successful vs. less successful problem-solvers develop problem representation and solution generation by comparing their PFnets to the Problem and Solution referent PFnets.
Second, additional PFnet analysis, degree centrality, was also conducted as an alternative measure of PFnets. This degree centrality can provide both a local-level measure of a PFnet (node degree centrality; a measure of node importance) and a global-level measure of a PFnet (graph degree centrality; a measure of network form or structure). The node degree identifies the relative importance of each node in a network in terms of the number of links that the node has with all other nodes (see for details, Ifenthaler, 2010). To obtain the node degree vectors, a node degree vector table was established to count the number of links of all key terms in participants' PFnets and referent's PFnets (see Figure 2 for an example). Then, the vectors in the table can be statistically further analyzed and compared in several ways including descriptive, correlation, and inferential statistics (see for detail, Clariana et al., 2013;Engelmann et al., 2014;Kim, 2017;Kim & Clariana, 2015, 2017, 2019).
In addition to this node degree centrality, we applied graph degree centrality to measure the structure or form of a PFnet. Clariana et al. (2013) quantified four concept map layout forms using graph centrality as a numerical measure of conceptual typology (see Figure 3), with 0 -0.2 (linear), 0.2 -0.4 (hierarchical), 0.4 -0.6 (network), and 0.6 -1 (star). Here graph centrality was used as a holistic visual measurement of mental representations to distinguish the qualitatively different mental representations between successful and less successful problem-solvers. The graph centrality values for all PFnets were calculated based on the node degree vectors (refer to Figure 2) using the equations described in Clariana et al. (2013) study. A growing number of studies have been using the graph degree centrality in highly diverse domains (e.g., Engelmann et al., 2014;Tawfik et al., 2019).
The data for analysis includes the human-rater measures of problem-solving essays and the PFnets from individual Pre maps, Group discussion, and individual Post essays. First, the human rater measures of post essays are presented, and then PFnet data are described and compared in two ways including (A) Similarity of PFnets, and (B) Centrality of PFnets.
Three subject-expert teacher raters scored each problem-solving Post essay for accuracy using a consensus rubric (0 ~ 10 point scale) for each problem over the investigation period. Polychoric correlations were computed between the three expert teachers' scores as a measure of inter-rater reliability (see Table 1). Then we ranked the scores in order and selected top 10 students (as successful problem-solvers) and bottom 10 students (as less successful problem-solvers) by each problem over the period (60 successful and 60 less successful solvers in total). An independent-sample t-test was run to determine if there was difference between successful vs. less successful problem-solvers on problem solving performance (as problem-solving post essays scores). It shows that the success students had significantly higher average mean scores than less successful students, (M = 7.2 vs. M = 3.5, d = 1.18, p < .001).
To consider how problem-solving collaboration impacted the successful vs. less successful problem-solvers' knowledge structures (Research Question 1), we tracked the changes in their knowledge structures (KS), represented as PFnets, before, during, and after collaboration. For that, we calculated the similarity between their Pre-to-Post, Pre-to-Group, and Group-to-Post PFnets. It is expected that we can identify how differently successful vs. less successful solvers have developed their KS due to collaboration by comparing their Pre-, Group-, and Post-KSs within the same participants. The results are presented in Table 2.
This data set shows that the less successful students' Post KS had a strong relationship with their Pre KS (sim = 0.76) compared to the successful (d = 0.68, p < .001), while the successful students' Post KS were more like the Group KS (sim = 0.84) compared to the less successful group (d = 0.79, p < .001). This suggests that the less successful problem-solvers paid less attention to their Group discussion, their Post essays were more dependent on their Pre maps and their initial unique knowledge, that is, the less successful problem-solvers have the "prior knowledge-oriented" characteristic for solving a problem. However, the successful solvers paid more attention to their Group discussion, so their Post essays were more dependent on their Group discussion and the group's knowledge, that is, the successful problem-solvers have the "group knowledge-oriented" characteristic for addressing the case.
To consider how students developed their KS leading to successful or less successful problem solving (Research Question 2 and 3), participants' KSs captured at different times (as Pre-PFnets, Group-PFnets, and Post-PFnets) are separately compared to the problem referent and to the solution referent PFnets. Similarity to the problem referent. Analysis of the participants' similarity to the Problem referents (Research Question 2) was analyzed by a one-between, one-within mixed ANOVA with the between-subjects factors type of problem solver (successful and less successful) and the withinsubjects factor time (Pre, Group, Post). Means are shown in Table 3. There was no outlier assessed by boxplot. The similarity values were normally distributed, as assessed by Shapiro-Wilk's test of normality (p > .05). There was homogeneity of variances (p > .05) and covariances (p > .001), as assessed by Levene's test of homogeneity of variances and Box's M test, respectively.
There was a statistically significant interaction between the type of problem-solver and time on the similarity to the Problem, F(3, 84) = 107.77, p < .001, partial η 2 = .837 (see the left panel of Figure 4). Therefore, simple main effects were run. The similarity to the Problem was not significantly different between the less successful group (M = 0.47, SD = 0.22) and the successful group (M = 0.49, SD = 0.19) at the Pre map, F(3, 84) = 0.40, p = .539, partial η 2 = .04. However, the similarity was significantly greater in the less successful group at the Group discussion (M = 0.51, SD = 0.29) compared to the successful group (M = 0.33, SD = 0.20), F(3, 84) = 7.406, p = .001, partial η 2 = .135, a mean difference of 0.18, 95% CI [0.04, 0.21]. The similarity was also significantly greater in the less successful group at Post essays (M = 0.58, SD = 0.26), compared to the successful group (M = 0.29, SD = 0.10), F(3, 84) = 12.94, p = .001, partial η 2 = .199, a mean difference of 0.29, 95% CI [0.17, 0.28].
For Pre maps, both the successful and less successful groups were more like the Problem referents. For Group speaking, the less successful students' speaking in Group discussion were more related to the Problem referents compared to the successful students' speaking in Group. For Post essays, the less successful students' Post essays were even more related to the Problem referents than the successful students' Post essays. This suggests that the less successful problem-solvers paid more attention to problem space during and after the group discussion than the successful problem-solvers; that is, the less successful problem-solvers tend to develop the "problem-focused" KS for solving a problem.
Similarity to the solution referent. Analysis of the participants' similarity to the Solution referents (Research Question 3) was analyzed by a one-between, one-within mixed ANOVA with the between-subjects factors type of problem solver (successful and less successful) and the withinsubjects factor time Group, Post). There was no outlier assessed by boxplot. The similarity values were normally distributed, as assessed by Shapiro-Wilk's test of normality (p > .05). There was homogeneity of variances (p > .05) and covariances (p > .001), as assessed by Levene's test of homogeneity of variances and Box's M test, respectively.
There was a statistically significant interaction between the type of problem-solver and time on the similarity to the Solution, F(3, 84) = 6.406, p = .003, partial η 2 = .192 (see the right panel of Figure 4). Therefore, simple main effects were run. The similarity to the Solution was For Pre maps, both the successful and less successful groups were not like the solution referents (as reported above, both groups Pre maps look like the Problem referents). For Group speaking, the successful students' speaking in Group were more related to the Solution referents compared to the less successful students. For Post essays, the successful students' Post essays were more related to the Solution referents than the less successful Post essays. This suggests that the successful problem-solvers paid more attention to solution space during and after the group discussion than the less successful problem-solvers, that is, the successful problem-solvers tend to develop the "solution-focused" KS for solving a problem.
Graph centrality is a numerical holistic measure of graphs form, or structure, that ranges from 0-0.2 (linear), 0.2-0.4 (hierarchical), 0.4-0.6 (network), and 0.6-1 (star). The results are presented in Table 4. For Pre maps, both successful and less successful students' Pre maps had a Problem referent-like-network structure (C graph = 0.47 -0.58; note: Problem referent C graph = 0.51). For Group speaking, the less successful students' speaking in Group had a Problem referent-like-network structure (C graph = 0.45 -0.55) while the successful students' speaking in Group had a Solution referent-like-hierarchical structure (C graph = 0.27 -0.37; note: Solution referent C graph = 0.35). For Post essays, the less successful Post essays had their map (problem)-like-network structure, while the successful Post essays had their Group speaking (solution)-like-hierarchical structure. These average graph centrality results corroborate those from the similarity of PFnets to one another (see Table 2) and the similarity of PFnet to the referents (see Table 3), confirming that the successful and less successful problem-solvers were differently impacted by collaboration (prior knowledge-oriented vs. group knowledge-oriented) and so established different knowledge structures leading to different performance (problem-focused vs. solution-focused).
Node centrality is a numerical measure of the relative importance of each node in a graph, as described above. Following Clariana et al. (2013) study, we used Proxscal multidimensional scaling (SPSS 20.0) to visually represent the average group PFnet as a point in a 2-dimensional space.
In the multidimensional scaling (MDS) representation (Figure 5), the problem referent map fell toward the left of the figure while the solution referent map fell toward the right. This MDS representation shows a different transformation of Pre-Group-Post PFnets by the type of problem-solver. As for the less successful group, all of their Pre-Group-Post maps were near the Problem space. As for the successful group, their Pre maps were moving from the problem space toward the solution space at the Group and still stayed near the solution space at the Post. This MDS analysis visually depicts the similarity of PFnet to one another (see Table 2) and similarity of PFnet to referents (see Table 3), supporting the less successful problem-solvers' problem-focused information sharing during collaboration that resulted in their prior knowledge-oriented knowledge structure after collaboration vs. the successful problem-solvers' solution-focused information sharing during collaboration that led to their group knowledge-oriented knowledge structure after collaboration.
Learning strategies employed in STEM often pose ill-structured problems to learners, which requires them to apply specialized domain knowledge. According to CBR, it is important to specify not only the knowledge content, but also how this knowledge is organized and structured to facilitate efficient retrieval of relevant information and future problem solving (Reif & Heller, 1982). Wang et al. (2013) assert that 'many existing studies in the field have tackled problem-solving and knowledge construction separately, failing to see them as an integrated two-way process'' (p. 294). Whereas many studies convey post-hoc measures of learning to understand the CBR benefits of PBL in classroom contexts (Tawfik, 2017), this study provides multiple measures before, during, and after the problem-solving process, which explores the temporal aspect of knowledge structure development. The results of this study thus advance the field's understanding of knowledge structures in two ways. First, it provides empirical evidence regarding the problem solving and development of knowledge structures at multiple instances (before, during, after) and under different conditions (individual, group), which has been previously documented as a known gap (Supanc et al., 2017). The second contribution specifically underscores the role of collaboration in knowledge structure development across different groups of learners, namely successful and learners that struggled.
An important finding specifically details the role of groups toward knowledge structure development at different instances of problem-solving. Those that struggled (the less successful group) had a strong relationship with their pre-task knowledge score, which is a finding documented in the literature (Asterhan & Dotan, 2018). That is, little conceptual change and knowledge strucgrowth was found when compared with their initial understanding of the concepts. This suggests that learners were more focused on sharing elements of the problem rather than being able to move toward areas where their ideas diverged or needed refinement. Because of this, the data suggests that learners were not able to build on their ideas, so their discourse continued to be focused on the core problem rather than solutions. Alternatively, the networks of those in the successful condition included additional concepts from their initial networks, and especially adopted concepts that were similar to those of their peers. Although studies have been done to understand individual CBR knowledge structure development, this study further underscores the role of collaboration in learning (Chen et al., 2018;Sharan et al., 2013) and contextualizes it in outcomes that measure complex and interrelated knowledge structures.
There may be multiple interpretations to the findings. One interpretation is that less successful learners exhibited a 'my-side bias' that was especially rooted in their own seminal understanding and overreliance on their own prior knowledge structures, which may describe why their soluwere more focused on the initial set of their own concepts (i.e., prior knowledge-oriented; see Table 2) and those described in the problem referent map (i.e., problem-focused; see Table 3). Indeed, the literature has described how learners often tend to heavily rely on their prior knowledge and limited internal case library (Delahunty et al., 2020;Oh & Jonassen, 2007); hence, they struggle to align new information as they engage in ill-structure problem-solving (Ge et al., 2016;Tawfik et al., 2019). Another interpretation is that learners shared ideas, but were unable (rather than unwilling) to recognize the merit among their peers or identify common ground, which impacted their ability to engage in meaning-making and a shared understanding of the topic. This could be due to how the ideas were presented among peers in terms of language or other forms of knowledge representation, which impacted their collaborative knowledge structure development. An implication thus relates to how learners are scaffolded in collaborative contexts. Whereas other scaffolding strategies may focus on retainment of information, it may be more apt to focus on equally address learners' prior assumptions and target ways to induce conceptual change, especially as they collaborate with their peers around divergent ideas.
Another central finding was that those in the successful group were more aligned with the solution referent map (see Table 3). As noted by Ge et al. (2016), problem-solving can be generally conceived as the problem representation and solution generation stage. Whereas those in the successful group were able to work together toward a solution, the data showed stark contrasts for those that were in the other condition. This is noteworthy in light of the differences that emerged in the problem representation phase of CBR. A precursor toward a collective successful solution is being able to identify a common understanding of the problem. Other related literature details how learners often struggle to progress beyond sharing ideas with their peers in collaborative settings. This study extends prior research by exploring how learners migrated their initial understanding toward a solution. Specifically, learners in the less successful condition in the current study were not able to come to a consensus and failed to progress in their knowledge structures beyond their initial conceptualization. Given that this study explored the temporal aspect of problem-solving, the results indicate that if learners are not able to develop a consensus in the early stages, the data suggests that it unlikely they will recover as they move toward other problem-solving competencies. One implication may be to consider the role of intermittent reflection during problem-solving. In studies, supporting problem-solving has often been through distinct cognitive phases (i.e. sharing ideas, hypothesis generation), with reflection serving as a form of culminating activity. Because the initial consensus building was key to the subsequent interaction, it stands that educators and learners may take additional time solidify their shared perspectives and address outlining issues.
While the study adds to the empirical literature about collaborative problem-solving, there are future studies that could build on this research. To begin, the current study was conducted with certain population (Grade 9 high school students). As a result, some caution is called for in generalizing these findings to other populations, especially those possess advanced domain knowledge and higher-order thinking skills. Along similar lines it should also be noted that all these findings were observed on one type of problem-solving task (i.e., ill-structured problems).
As evidenced by our earlier study (Kim et al, in press), there are distinctive cognitive differences required to solve ill-structured and well-structured problems; for example, a focused convergence needed for well-structured problem and extending divergence needed for ill-structured problem. Thus, the findings from the current study with ill-structured problems might not be applied to well-structured problems solving. Other studies could thus explore the degree to which these findings from the current study are maintained across various domain areas, since it is possible that the results are a byproduct of the problem situated within a context.
All of the data are available by individual application directly to the first author. Ethic permissions were obtained to collect the data from the institution. All participants' data were treated and stored confidentially and anonymously.
We declare that no conflict of interest concerning this study
Kyung Kim is an Assistant Professor of School of Energy Technology and Director of Center for Education Reform at Korea Institute of Energy Technology. He is interested in looking at how knowledge is structured in the mind of individuals and visualizing the knowledge structure using the network technologies. So far, he has designed and developed various network analytic systems that can capture, visually represent, and compare knowledge structure in learners' multi-modal data in their learning process so that he can assess individual students' learning trajectories for more personalized learning environment.
Andrew A. Tawfik is an Assistant Professor of Instructional Design & Technology at the University of Memphis, where he also serves as the director for the Instructional Design & Technology Studio. His research interests include problem-based learning, case-based reasoning, case library instructional design, and computer-supported collaborative learning.
or → proposes at least two speci c scientists to cover all scienti c perspectives; → gives general job description and may not describe the relationships to the situations; → uses nearly appropriate astronomical concepts and terminology.
2) Astronomers to help with all their telescope as far as how big it is, how fast it's coming. Geologists, what areas could take the impact.
2 points → lists at least two team names; may cover only one scienti c perspective; → does not give any explanations: -uses some astronomical concepts and terminology, Example: 1) Astronomer, Physicist, Geologist or → describes general role of at least one team member may not list speci c scientists or may not correctly match the job descriptions and scientists; → uses some astronomical concepts and principles.
2) Scientists will help where the asteroid would hit, how large it is, and the speed of the asteroid to determine the force of the explosions.
1 point → shows consideration of the perspectives; → uses limited astronomical terminology Example: Astronomers that are strictly the best 0 point → no consideration of the perspectives; → fails to use astronomical terminology.
Example:
The affects of the asteroid will be very strong States selection procedures 4 3 2 1 0 4 points → Shows logical procedures for selecting members, including at least three of the following elements; → 1) Con rm the prediction, 2) if yes, where, how large, 3) what impact on global population, 4) possible defense methods and ways to protect populationGives complete and clear responses of the selecting procedures with logically sound and systematic explanations; → Explanations focus on scienti c perspectives Example: the first step would be to confirm the claims of the asteroid, and if confirmed, it would need to be further studied to learn about where it might hit, how large the explosion would be, and its after-effects on local and global population. Finally, I will evacuate people and find the possibility to destroy the asteroid 3 points → Presents some logical procedures for selecting members including at least two elements → May give general responses with logically sound selecting procedures; → Explanations focus on scienti c perspectives Example: Astronomers tell how big it is, how fast. Geologist could take the impact. Oceanographers will be needed if the asteroid goes in the ocean…… Depending on where the asteroid hits, the ozone, or pollution, or something else may bum up. 1 will also have law enforcement officials to control any large crowds and get people evacuated if necessary 2 points → Shows some logical procedures for selecting members including at least two elements or only scienti c perspectives; → May not give any responses of the selecting procedures; → Explanations may focus on defense methods or ways to protect population; or focus on only scienti c perspectives Example: I will need a mathematics to pin point the exact spot where the asteroid will hit. I will need an expert army guy to fine a missile on the trajectory of the course of the asteroid. I will also need pilots to fly the people to out. 1 will also need someone to find out how big the asteroid to decide how big of the missile to use.
1 point → Shows logically unsound procedures for selecting members; or no scienti c ideas; → May give incomplete, ambiguous response with logically unsound explanations of the selecting procedures; → Procedures are di cult to follow Example: If 1 was putting together a team to see if Dr.
Smith was right about the asteroid. 1 would pick people that had been part of something like this before. I would need someone that new a lot about asteroids. Some one that was into weather and etc. Someone that was into astronomy. 0 point → Does not show any procedures; or simply provides a list of team members; no explanation Example: Geologist, Physicists, Astronomers, etc.