Search for: All records

We consider the problem of offline reinforcement learning (RL) -- a well-motivated setting of RL that aims at policy optimization using only historical data. Despite its wide applicability, theoretical understandings of offline RL, such as its optimal sample complexity, remain largely open even in basic settings such as \emph{tabular} Markov Decision Processes (MDPs). In this paper, we propose Off-Policy Double Variance Reduction (OPDVR), a new variance reduction based algorithm for offline RL. Our main result shows that OPDVR provably identifies an ϵ-optimal policy with O˜(H2/dmϵ2) episodes of offline data in the finite-horizon stationary transition setting, where H is the horizon length and dm is the minimal marginal state-action distribution induced by the behavior policy. This improves over the best known upper bound by a factor of H. Moreover, we establish an information-theoretic lower bound of Ω(H2/dmϵ2) which certifies that OPDVR is optimal up to logarithmic factors. Lastly, we show that OPDVR also achieves rate-optimal sample complexity under alternative settings such as the finite-horizon MDPs with non-stationary transitions and the infinite horizon MDPs with discounted rewards. 

Some of ultrafine-grained (UFG) metals including UFG twinning induced plasticity (TWIP) steels have been found to overcome the paradox of strength and ductility in metals benefiting from their unique deformation modes. Here, this study provides insights into the atomistic process of deformation twin nucleation at Σ3{111} twin boundaries, the dominant type of grain boundary in this UFG high manganese TWIP steel. In response to the applied tensile stresses, grain boundary sliding takes place which changes the structure of coherent Σ3{111} twin boundary from atomistically smooth to partly defective. High resolution transmission electron microscopy demonstrates that the formation of disconnection on Σ3{111} twin boundaries is associated with the motion of Shockley partial dislocations on the boundaries. The twin boundary disconnections act as preferential nucleation sites for deformation twin that is a characteristic difference from the coarse-grained counterpart, and is likely correlated with the lethargy of grain interior dislocation activities, frequently seen in UFG metals. The deformation twin nucleation behavior will be discussed based on in-situ TEM deformation experiments and nanoscale strain distribution analyses results.

 

In polycrystalline materials, grain boundaries are known to be a critical microstructural component controlling material’s mechanical properties, and their characters such as misorientation and crystallographic boundary planes would also influence the dislocation dynamics. Nevertheless, many of generally used mechanistic models for deformation twin nucleation in fcc metal do not take considerable care of the role of grain boundary characters. Here, we experimentally reveal that deformation twin nucleation occurs at an annealing twin (Σ3{111}) boundary in a high-Mn austenitic steel when dislocation pile-up at Σ3{111} boundary produced a local stress exceeding the twining stress, while no obvious local stress concentration was required at relatively high-energy grain boundaries such as Σ21 or Σ31. A periodic contrast reversal associated with a sequential stacking faults emission from Σ3{111} boundary was observed by in-situ transmission electron microscopy (TEM) deformation experiments, proving the successive layer-by-layer stacking fault emission was the deformation twin nucleation mechanism, different from the previously reported observations in the high-Mn steels. Since this is also true for the observed high Σ-value boundaries in this study, our observation demonstrates the practical importance of taking grain boundary characters into account to understand the deformation twin nucleation mechanism besides well-known factors such as stacking fault energy and grain size.

 

Despite national efforts in increasing representation of minority students in STEM disciplines, disparities prevail. Hispanics account for 17.4% of the U.S. population, and nearly 20% of the youth population (21 years and below) in the U.S. is Hispanic, yet they account for just 7% of the STEM workforce. To tackle these challenges, the National Science Foundation (NSF) has granted a 5-year project – ASSURE-US, that seeks to improve undergraduate education in Engineering and Computer Science (ECS) at California State University, Fullerton. The project seeks to advance student success during the first two years of college for ECS students. Towards that goal, the project incorporates a very diverse set of approaches, such as socio-cultural and academic interventions. Multiple strategies including developing early intervention strategies in gateway STEM courses, creating a nurturing faculty-student interaction and collaborative learning environment, providing relevant, contextual-based learning experiences, integrating project-based learning with engineering design in lower-division courses, exposing lower-division students to research to sustain student interests, and helping students develop career-readiness skills. The project also seeks to develop an understanding of the personal, social, cognitive, and contextual factors contributing to student persistence in STEM learning that can be used by STEM faculty to improve their pedagogical and student-interaction approaches. This paper summarizes the major approaches the ASSURE-US project plans to implement to reduce the achievement gap and motivate ECS students to remain in the program. Preliminary findings from the first-year implementation of the project including pre- and post- data were collected and analyzed from about one hundred freshmen and sophomore ECS students regarding their academic experience in lower-division classes and their feedback for various social support events held by the ASSURE-US project during the academic year 2018-19. The preliminary results obtained during the first year of ASSURE-US project suggests that among the different ASSURE-US activities implemented in the first year, both the informal faculty-student interactions and summer research experiences helped students commit more to their major during their lower-division years. The pre-post surveys also show improvements in terms of awareness among ASSURE-US students for obtaining academic support services, understanding career options and pathways, and obtaining personal counseling services. 

Logic-in-Memory (LIM) architectures offer potential approaches to attaining such throughput goals within area and energy constraints starting with the lowest layers of the hardware stack. In this paper, we develop a Spintronic Logic-in-Memory (S-LIM) XNOR neural network (S-LIM XNN) which can perform binary convolution with reconfigurable in-memory logic without supplementing distinct logic circuits for computation within the memory module itself. Results indicate that the proposed S-LIM XNN designs achieve 1.2-fold energy reduction, 1.26-fold throughput increase, and 1.4-fold accuracy improvement compared to the state-of-the-art binarized convolutional neural network hardware. Design considerations, architectural approaches, and the impact of process variation on the proposed hybrid spin-CMOS design are identified and assessed, including comparisons and recommendations for future directions with respect to LIM approaches for neuromorphic computing.