Machine Learning (ML) has become a powerful tool for scientic discovery, enabling new problem-solving approaches in several scientic disciplines [2,24,35,37]. Techniques such as clustering and classication uncover patterns in large datasets that were previously intractable, accelerating data-driven knowledge discovery [6,16,51]. Scientic inquiry (SI) learning advocates for K-12 students to adopt practices akin to professional scientists for knowledge construction [25]. In the era of big data and ML, it is therefore important to explore the design of ML-supported SI activities for K-12 STEM education [34]. The aspiration is to cultivate authentic data and computational literacy at an early age, thereby equipping the next generation with the expertise needed in a data-centric world [49].
There remains a gap between ML concepts and K-12 STEM pedagogy [20,45,57]. To bridge the gap with meaningful learning experiences for students from diverse backgrounds, it is crucial to involve K-12 teachers in creating learning activities [30,38,54]. Teachers, however, often have limited ML expertise [30,44] to craft pedagogically sound ML practices that captivate students' interests [31,45]. To address these challenges, our ongoing work makes two contributions:
(1) We present ML4SI, a web-based tool for K-12 teachers to design ML-supported SI learning activities. It allows teachers to construct learning activities by arranging pre-designed ML & SI components in a side-by-side layout. (2) With the data collected from ML4SI, we can model how K-12 teachers and students apply dierent ML methods along with various SI behaviors, such as questioning and formulating hypotheses [34].
Preliminary ndings from a pilot study with three K-12 teachers suggest strengths and opportunities for design for ML4SI, and a potential interplay between ML techniques and SI learning behaviors.
Various guidelines and frameworks have been proposed for designing programming environments to support non-expert end-users [4]. The cognitive dimensions of notations [18], for instance, provide a vocabulary by which notation designers can discuss the tradeos made by design choices. These dimensions provide insights into the user interface's navigability, consistency, and error-proneness, making them a valuable tool for assessing and improving the design of systems intended for non-programmers. Building upon these dimensions, Repenning and Ioannidou's 13 design guidelines are more implementation-focused [36]. For instance, strategies like minimizing the possibility of syntactic errors, incorporating objects as language elements, and fostering incremental development are directly applicable during the design phase, ensuring a more accessible and intuitive user experience. Sarkar [39] proposes four design principles for non-expert data analysis through ML: (1) start the abstraction gradient at zero, (2) abstract complex processes through heuristic automation, (3) build expertise through iteration on multiple representations, and (4) support dialogue through metamodels. These design dimensions and guidelines oer insights into the ML4SI interface and our pilot study with K-12 teachers.
Inquiry-based learning serves as an educational approach designed to engage students in authentic SI practices [25]. It emphasizes students' active engagement in the SI learning process and their responsibility for uncovering novel knowledge through a dual process of inductive and deductive reasoning [9,50].
In ML4SI, we adopt a well-recognized framework for inquirybased learning [34] to extract the essential SI learning behaviors. From the ve key SI learning phases, we primarily focus on conceptualization, investigation, and conclusion-since they have a direct connection with data.
Conceptualization illustrates the process where students actively propose research questions to be explored or hypotheses to be tested. Investigation is where students investigate research questions or hypotheses through exploration, experimentation, analysis, and data interpretation-exploration emphasizes exploratory data analysis and observation in which students make discoveries related to their questions without a predetermined hypothesis; students, during experimentation, test specic hypotheses by designing and executing experiments; analysis is systematically analyzing data to identify patterns and draw meaningful inferences; data interpretation derives meanings from patterns revealed by data analysis. Conclusion is where students address their original research questions or hypotheses and determine whether they have been eectively answered or supported by the results obtained from their study.
ML techniques, known for their ability to learn from experience and identify complex patterns within data, have been rapidly adopted by scientists across various disciplines in the pursuit of SI [1-3, 35, 37]. ML equips scientists with a powerful toolkit for automating the analysis of large datasets, accelerating systematic investigations and explorations [16,32], and uncovering novel patterns that are often hidden from conventional techniques [5]. Moreover, it can guide the design of experiments and shape future data collection strategies [27].
Despite the demonstrated potential of ML in SI, its integration into K-12 STEM education has been slow. Two pioneering studies shed light on this matter. One study observed early signs of SI learning behaviors, such as questioning and explaining, from students during a hands-on data analysis experience that teaches Kmeans clustering [46]. Another study used a predened SI model to guide K-12 teachers to create conceptual designs of ML-supported SI learning activities [44,55]. However, the structured nature of the predened model hindered the delity of the designs in real-world teaching scenarios, compared with an authentic ML tool.
To address this research gap, we develop ML4SI, an authoring tool that provides teachers with exibility in integrating ML into SI practices, focusing on two ML algorithms: K-means clustering and K-nearest neighbors (KNN). K-means clustering, an unsupervised ML algorithm, is employed to discover patterns among similar objects and create taxonomies [13,14,47], making it suitable for exploratory data analysis [5]. K-nearest neighbors, a supervised ML algorithm for the classication task, with its primary objective being to predict the class of a data point based on known examples.
The interpretability of ML-revealed patterns is a signicant concern, particularly for K-12 teachers and students with limited skills in reading and interpreting multidimensional data [14,29,41].
There are two main types of multidimensional data visualization: geometric methods and iconographic displays [12]. Geometric methods portray multidimensional data using the axes of selected shapes [12]. Parallel coordinates, a widely-used geometric method [12,22], represent attributes by parallel vertical axes linearly scaled within respective data ranges and depict each data point by a polygonal line that intersects each axis at the corresponding value. Iconographic displays, also known as glyphs, encode attributes and values into visual features, such as size, shape, and color [12,48]. It makes multidimensional information more accessible, particularly for novice learners, compared with geometric methods [12,48]. Face glyphs, star glyphs, and prole glyphs are three common glyph methods [7,11,12]. Face glyphs map attributes and values to facial features, such as eye size and nose width [8]. Star glyphs represent attributes as spokes of a circular wheel, with values encoded by the length from outer points to a central point [15,48]. Prole glyphs use linear position/length to encode values [7,11].
Each aforementioned visualization technique oers distinct advantages for multidimensional data analysis. For example, parallel coordinates are particularly eective for discerning data distribution and functional dependencies [52]. Face glyphs leverage human familiarity with facial elements, facilitating more eective data integration and stimulating greater interest in engaging with data analysis [8,23,28]. Star glyphs and prole glyphs resemble conventional visualization tools familiar to K-12 students, such as pie graphs and bar charts. We incorporate dierent forms of visualizations into ML4SI, allowing teachers to decide the most suitable visualization for specic inquiry activities based on their expertise.
We leverage design implications derived from a co-design study with K-12 teachers [44,56] and existing design guidelines for enduser programming environments [4,36] for developing ML4SI.
First, "abstraction gradient" assesses how the tool abstracts complex ML processes into user-friendly interfaces. ML4SI simplies ML algorithms like K-means clustering into drag-and-drop components (Section 3.2.1), making advanced data analysis accessible to teachers with limited ML expertise. Second, "closeness of mapping" evaluates how well the tool's interface corresponds to the real-world tasks of K-12 teachers. Our system's side-by-side layout of ML and SI components (Section 3.2.2) mirrors the natural workow of teachers planning and executing lessons, thus aligning closely with their educational objectives and problem-solving domain. Third, to support non-experts' incremental development of ML-supported analysis, ML4SI enables teachers to easily customize input for individual ML components (Section 3.2.4) and execute before the completion of the entire process. Such step-by-step creation with ML components that gradually reveal the computational complexity (Section 3.2.1) avoids huge leaps in the challenge for novices. Finally, ML4SI allows immersion. Teachers are immersed in ML-supported tasks and actively experience the results through directly dragging & dropping data features onto glyph visual features (Fig. 1.2, left column) along with real-time updates of results in the consecutive steps (Section 3.2.5). Glyph visualizations (Section 3.2.3) also have unique learning benets for end users by aording direct manipulation and interaction with multidimensional data points.
Jeremy, a middle-school biology teacher, aims to demonstrate the dynamic interactions between various ecological features. His objective is to create an SI learning activity for students to uncover relationships among temperature, precipitation, canopy height, mammal richness, and beetle richness. This task involves applying ML algorithms to a multidimensional dataset gathered from extensive eld sites. Despite his high motivation, Jeremy has not yet acquired sucient expertise in ML to construct an eective lesson for his students. ML4SI is designed to assist teachers like Jeremy in realizing such educational objectives.
Blocks to Initiate ML/SI Components. The top bar of ML4SI has draggable components (Fig. 1.1). When Jeremy drags and drops ML/SI components into the main workspace (Fig. 1.2), the corresponding ML methods or SI behaviors are initiated. This enables Jeremy to begin exploring the dataset, interpreting patterns, and creating specic steps in the learning activity.
ML components (Fig. 1.1, left column) allow the application of ML techniques. For example, the pairwise comparison component computes the similarity between two data points. Manual clustering [46] reveals patterns in a subset by allowing users to manually overlay glyphs for similarity comparison through superposition comparative visualization [17]. Automatic clustering applies k-means clustering on input data and visualizes the clusters. Prediction with classiers guides users to predict unlabeled data with KNN and evaluate the prediction result.
SI components (Fig. 1.1, right column) represent established SI learning behaviors (e.g., questioning, hypothesis generation, data analysis, conclusions) [34] as introduced in Section 2.2. Since inquirybased learning is self-directed, students may encounter challenges in the learning process if they lack the necessary self-regulation skills [53]. ML4SI therefore includes an instruction component that teachers can use to create scaolds to keep students metacognitively, motivationally, and behaviorally engaged.
Layout. This layout reects the visual presentation of the nal design (Fig. 1.2). Users are able to add, reorder, and remove individual or pairs of ML and SI components to tailor the learning design. Jeremy experiments with dierent pairs of ML and SI components, by considering which ML components would best support a particular SI learning behavior, or which SI behaviors students are most likely to demonstrate while analyzing with a specic ML technique.
Jeremy can select from multiple data visualizations for each ML component (Fig. 1.3). The choice of visualization in one component does not aect the others, allowing for independent selection across dierent inquiry phases. More visualization options can be added for dierent ML techniques.
For instance, Jeremy might opt for face glyphs in early data exploration to spark students' interest in the intriguing patterns revealed by the "smileys". Later, he could switch to parallel coordinates for his students to examine more detailed numerical patterns.
Input and Output for Individual ML Components. In each ML component, Jeremy can input dierent data, ranging from the entire dataset to a cluster formed by a previous clustering component, or a manually selected subset (Fig. 2.1 & 2.4). Additionally, Jeremy can save the outputs of ML components. For instance, he might preserve the clusters created by K-means clustering (Fig. 2.2 & 2.3) for another round of clustering.
Updates for Feature Selection. In feature selection (Fig. 1.2, left column), any edits also trigger updates of subsequent ML components. For example, Jeremy creates parallel coordinates using all ve data features from a ve-dimensional dataset, followed by automatic clustering. Clusters will then be generated out of the ve-dimensional data. Through cluster analysis, Jeremy discovers that there are still large variations in the last data feature "mammal richness" in all clusters (Fig. 2.2). He hypothesizes that this is not an important feature for clustering. He tests this by removing the feature in the feature selection, and the change is immediately reected in the clustering results. This allows users to rene their hypotheses and explore the impact of feature selection on ML outputs, encouraging experimentation and trial & error.
As an initial investigation, we invited three K-12 STEM teachers (Table 1) to try ML4SI. We wished to evaluate ML4SI with K-12 teachers in a co-design setting for three reasons: (1) teachers' expertise is crucial due to the limited pedagogical theories on integrating ML methods into K-12 scientic inquiry [44,45]; (2) teachers often underestimate their ability to teach with ML, leading to low selfecacy; adequate support can mitigate this issue [21,26,30,43]; and (3) involving teachers and ML experts helps balance between user-centered and learner-centered methodologies; teachers can engage with complex data and uncover meaningful patterns with the support of experts [10,19].
The pilot study consists of two two-hour sessions. The study was approved by the institutional Research Subjects Review Board.
Session 1: Teacher-as-Learner. Teachers familiarised themselves with the ML4SI interface by analyzing an example dataset under researcher guidance, including (1) dierent data visualizations, (2) ML components, (3) SI components, and (4) the ow of designing the learning activity in ML4SI. Participants asked questions and provided feedback after interacting with each ML/SI component. At the end of the rst session, the teacher and the researcher discussed the types of datasets or learning activities the teacher wished to create during the second session.
A minimum of two days separated the rst and second sessions. In this interval, either the researcher or the participant acquired a new dataset as per the discussion in the rst session. The researcher pre-processed the new dataset by eliminating data points with missing values and non-numeric features. This ensured the dataset's compatibility with the ML algorithms implemented in ML4SI.
Session 2: Teacher-as-Designer. Teachers explored the dataset of their choice with ML4SI. A researcher and an ML expert provided facilitation when needed. Teachers asked questions that interested them at each step, answered the inquiries based on the data visualization, and planned subsequent actions. Following each step, the researcher prompted teachers to consider any necessary adjustments to the pairing of ML and SI components. Upon completion, teachers were asked to review their process and contemplate modications to the learning activity they created. Finally, teachers provided feedback on the system design.
We collect video recordings of the study sessions and the log data of teachers' interaction behaviors with ML4SI. We had two aims. First, to investigate the analysis process of teachers applying ML methods to explore the datasets of their interest. Second, to understand how dierent ML components support dierent SI behaviors in the nal designed learning activity.
This section describes a lesson designed by P2, a CS teacher in a city school where students have low digital literacy and are from underrepresented backgrounds in STEM. The learning objective is to investigate what impacts a person's income level. The essential question for students to answer in the end is how college and career planning in high school aect your life.
First, P2 began with introducing students to the key concepts involved in the dataset by viewing data attributes' denitions. Second, P2 guided students to select specic data attributes -working hours, age, gender, and education level -aligning with the target learning objective. Third, P2 asked students to compare income predictions, aiming to inspire students to question: "why some individuals are predicted to have high income and others low?" After applying KNN a few times, she hypothesized positive correlations between higher education, longer working hours, and higher income. P2 expected her students would be able to achieve the same SI learning behavior while interacting with KNN. Fourth, P2 directed students to validate initial hypotheses with the patterns revealed by applying k-means clustering on the entire dataset. P2 discovered trends that clusters with high incomes have higher education, longer working hours, and medium ages, while the low-income cluster has lower education and fewer working hours (Fig. 3). Lastly, P2 added a step for students to answer the essential question by contextualizing the patterns in social, economic, and literacy development domains.
A structured and exible way to create SI learning activities. All teachers appreciated how the side-by-side layout aords a structured way of conducting the open-ended exploration and iterating the SI learning design after reection. Edit-triggered updates of feature selection and the ability to investigate any data subset helped teachers experiment with ideas, and zoom in and out on dierent parts of the data. This enabled a deeper understanding of the patterns and the ML algorithms through trial & error. After reviewing their analysis process, teachers tended to create the nal design to incrementally build up the complexity of hypotheses, in an attempt to scaold for students to apprehend the challenges of iterating hypotheses with dierent analysis methods applied to dierent data. Furthermore, teachers recognized ML4SI's adaptability to incorporate existing teaching techniques, such as the graphic organizer, the Three-Two-One technique (i.e., identify three interesting observations, ask two questions, raise one potential solution), self-directed learning for small group activities, etc.
Integrating ML literacy into K-12 STEM contexts. All teachers noted close connections between ML practices to K-12 science objectives, data/computational literacy, and digital uency required by K-12 curriculum standards [33,40,42]. For example, comparing the classication results with the ground truth and tinkering with the ML parameter supports the development of debugging skills. Visualizations can make a range of patterns accessible for children to analyze, such as intra-cluster patterns, outliers, variations, etc. One math teacher pointed out that analyzing centroid, intra-cluster similarity, and variation can create the opportunity for his students to discuss the issues of using the average to represent a group of data points with a large variation. Participants suggested a range of K-12 STEM topics that can be taught with ML4SI, including career readiness, nutrition, college applications, data-driven strategy for football drafting, digital citizenship, bias and privacy issues in data, AI recommendation systems, etc.
Introducing automated assistance. We noted opportunities for adding intelligent assistance. One teacher suggested that students' hypothesis generation and iteration can be automatically identied, extracted, and tracked throughout the analysis, and thus, scaolding can be personalized for dierent learners at dierent SI phases. After manually selecting a few data points of his interest as the input for manual clustering, one teacher suggested having the system suggest input for an ML component to generate the optimal or insightful results, based on the existing steps.
Opening the "black box". Teachers asked for more technical details underlying the clustering and classication algorithms, and more advanced statistical information about the data. For example, beyond learning about how KNN predicts an unknown value for a data point based on its nearest neighbors, one teacher was also curious about how to improve the labeled data to increase prediction accuracy. Another teacher requested a more advanced and detailed statistical summary for a specic cluster.
Benets and Limitations of Glyphs. Glyphs encode data at the level of individual data points, as opposed to chart types that aggregate data (e.g., bar charts). Teachers identied glyphs' unique benet in directly visualizing individual data points for students to manipulate and reason with, such as (1) the representation and calculation of centroid, (2) the relationship between average, variation, and individual values in a data group, (3) how global patterns such as trends and correlations emerge from individual data points, and (4) the algorithmic mechanism underlying KNN. This builds upon Sarkar's design principle of directly representing individual data points for non-experts doing data analysis through ML, i.e., starting the abstraction gradient at "zero" [39]. With the aforementioned benets of using glyphs for data exploration in K-12 classrooms, it is important to note that the scalability of glyphs is limited by the number of data features and the quantity of data points to be visualized.
Aordances of dierent visualizations. Teachers appreciated the ability to choose from multiple visualization types. Two teachers whose students have lower data and computational literacy preferred face glyphs, compared to the traditional graphs introduced in K-12 classrooms. They suggested that face glyphs could engage students who are not procient in reading data graphs. Teachers emphasized that if there are positive or negative implications behind some data attributes, face glyphs can establish semantic meaning between the visual representation and the patterns, and thus promote students' analysis. One teacher envisioned that his students might tell a compelling story about how to win baseball games by interpreting the patterns visualized in happy faces.
The preliminary ndings indicate that the integration of ML into K-12 education can serve as an opportunity to enhance data literacy and SI skills for young students. ML4SI is a step towards bridging the gap between advanced ML tools and their use in K-12 STEM classrooms, and demystifying ML. Furthermore, the positive feedback and synergy observed between ML elements and SI learning behaviors suggest that our approach to designing ML-supported SI activities shows potential.
Future research will involve more detailed evaluations of ML4SI with teachers and ML experts. Iterative design improvements will be made based on ongoing feedback collection, with an emphasis on enhancing the user experience and enhancing the integration with educational contexts.
Another essential next step is to analyze the patterns of how teachers and students utilize dierent ML methods along with SI behaviors. This may involve network analysis, lag sequential analysis, and frequent pattern mining.
We also plan to assess the eects of ML-supported SI learning activities designed by teachers, on students' engagement and development of data/computational literacy. Future work may also analyze the long-term eects of integrating ML into K-12 education, including tracking the development of student's critical thinking and problem-solving skills over time, as well as their readiness for advanced studies or careers in data-driven elds.