Search for: All records

Online reinforcement learning (RL) enhances policies through direct interactions with the environment, but faces challenges related to sample efficiency. In contrast, offline RL leverages extensive pre-collected data to learn policies, but often produces suboptimal results due to limited data coverage. Recent efforts integrate offline and online RL in order to harness the advantages of both approaches. However, effectively combining online and offline RL remains challenging due to issues that include catastrophic forgetting, lack of robustness to data quality and limited sample efficiency in data utilization. In an effort to address these challenges, we introduce A3RL, which incorporates a novel confidence aware Active Advantage Aligned (A3) sampling strategy that dynamically prioritizes data aligned with the policy's evolving needs from both online and offline sources, optimizing policy improvement. Moreover, we provide theoretical insights into the effectiveness of our active sampling strategy and conduct diverse empirical experiments and ablation studies, demonstrating that our method outperforms competing online RL techniques that leverage offline data. Our code will be publicly available at:this https URL. 

As the semiconductor and photonics industries grapple with mounting business pressures, weaving resourceefficiency into engineering education has evolved from a priority to an imperative. Under the umbrella of FUTUR-IC, this paper highlights novel pedagogical strategies at Bridgewater State University (BSU) aimed at equipping photonics and optical engineers to address today’s ecological challenges. We detail two complementary approaches that together form a cohesive educational framework. The first involves a newly introduced fresh year-level seminar on Resource Efficient Microchip Manufacturing, which immerses students in resource-efficiency metrics such as Life Cycle Intelligence and “design for resourceefficiency” principles. By interlinking photonic integration concepts with tangible business impact assessments, this course fosters an early appreciation of how advanced technologies can be developed responsibly, with reduced energy consumption and minimized waste. The second approach redefines senior-level engineering design courses to embed multifaceted resourceefficiency criteria in the design process. Through project-based learning and collaboration with industry partners, students integrate photonic solutions with data-driven metrics, refining their ability to propose holistic prototypes. These initiatives go beyond technical mastery to cultivate interdisciplinary collaboration and critical thinking. This work illustrates how an integrated approach to engineering education can spark the next generation of practitioners to design for both technological excellence and business viability. 

Students bring different levels of interest to learning experiences, which impacts how they engage with learning materials. This study aims to understand the relationship between student's interest levels and their scientific observation behaviors within a Minecraft-based learning system. Motivated by the growing interest in integrating human-AI collaboration within educational research, we combine the capabilities of Large Language Models (LLMs) with the expertise of human researchers to capture the emerging themes within students’ observations. Using epistemic network analysis, we then visualized and compared the observational patterns of students with high and low situational interest. Our findings indicate that students with higher situational interest tend to make observations across a broader range of topics, with a particular emphasis on scientific content. These results highlight the potential for developing timely interventions to support students with low situational interest. 

While training models and labeling data are resource-intensive, a wealth of pre-trained models and unlabeled data exists. To effectively utilize these resources, we present an approach to actively select pre-trained models while minimizing labeling costs. We frame this as an online contextual active model selection problem: At each round, the learner receives an unlabeled data point as a context. The objective is to adaptively select the best model to make a prediction while limiting label requests. To tackle this problem, we propose CAMS, a contextual active model selection algorithm that relies on two novel components: (1) a contextual model selection mechanism, which leverages context information to make informed decisions about which model is likely to perform best for a given context, and (2) an active query component, which strategically chooses when to request labels for data points, minimizing the overall labeling cost. We provide rigorous theoretical analysis for the regret and query complexity under both adversarial and stochastic settings. Furthermore, we demonstrate the effectiveness of our algorithm on a diverse collection of benchmark classification tasks. Notably, CAMS requires substantially less labeling effort (less than 10%) compared to existing methods on CIFAR10 and DRIFT benchmarks, while achieving similar or better accuracy.