Search for: All records

Introduces OAT (Offline with Augmented Trajectories), a generative sub-trajectory augmentation method designed to enhance off-policy evaluation accuracy. Experiments across robotics, healthcare, and e-learning show substantial performance gains over baselines. 

In the realm of reinforcement learning (RL), off-policy evaluation (OPE) holds a pivotal position, especially in high-stake human-centric scenarios such as e-learning and healthcare. Applying OPE to these domains is often challenging with scarce and underrepresentative offline training trajectories. Data augmentation has been a successful technique to enrich training data. However, directly employing existing data augmentation methods to OPE may not be feasible, due to the Markovian nature within the offline trajectories and the desire for generalizability across diverse target policies. In this work, we propose an offline trajectory augmentation approach, named \textbf{OAT}, to specifically facilitate OPE in human-involved scenarios. We propose sub-trajectory mining to extract potentially valuable sub-trajectories from offline data, and diversify the behaviors within those sub-trajectories by varying coverage of the state-action space. Our work was empirically evaluated in a wide array of environments, encompassing both simulated scenarios and real-world domains like robotic control, healthcare, and e-learning, where the training trajectories include varying levels of coverage of the state-action space. By enhancing the performance of a variety of OPE methods, our work offers a promising path forward for tackling OPE challenges in situations where human-centric data may be limited or underrepresentative. 

Quasicrystals are characterized by atomic arrangements possessing long-range order without periodicity. Van der Waals (vdW) bilayers provide a unique opportunity to controllably vary atomic alignment between two layers from a periodic moir´e crystal to an aperiodic quasicrystal. Here, we reveal a remarkable consequence of the unique atomic arrangement in a dodecagonal WSe2 quasicrystal: the K and Q valleys in separate layers are brought arbitrarily close in momentum space via higher-order Umklapp scatterings. A modest perpendicular electric field is sufficient to induce strong interlayer K − Q hybridization, manifested as a new hybrid excitonic doublet. Concurrently, we observe the disappearance of the trion resonance and attribute it to quasicrystal potential driven localization. Our findings highlight the remarkable attribute of incommensurate systems to bring any pair of momenta into close proximity, thereby introducing a novel aspect to valley engineering. 

We develop a consistent adaptive framework in a multilevel collocated grid layout for simulating two-phase flows with adaptive mesh refinement (AMR). The conservative mo-mentum equations and the mass equation are solved in the present consistent framework. This consistent mass and momentum transport treatment greatly improves the accuracy and robustness for simulating two-phase flows with a high density ratio and high Reynolds number. The interface capturing level set method is coupled with the conservative form of the Navier–Stokes equations, and the multilevel reinitialization technique is applied for mass conservation. This adaptive framework allows us to advance all variables level by level using either the subcycling or the non-subcycling method to decouple the data ad-vancement on each level. The accuracy and robustness of the framework are validated by a variety of canonical two-phase flow problems. We demonstrate that the consistent scheme results in a numerically stable solution in flows with high density ratios(up to 106) and high Reynolds numbers(up to 106), while the inconsistent scheme exhibits non-physical fluid behaviors in these tests. Furthermore, it is shown that the subcycling and non-subcycling methods provide consistent results and that both of them can accurately resolve the interfaces of the two-phase flows with surface tension effects. Finally, a 3D breaking wave problem is simulated to show the efficiency and significant speedup of the proposed framework using AMR. 

We develop a consistent adaptive framework in a multilevel collocated grid layout for simulating two-phase flows with adaptive mesh refinement (AMR). The conservative mo-mentum equations and the mass equation are solved in the present consistent framework. This consistent mass and momentum transport treatment greatly improves the accuracy and robustness for simulating two-phase flows with a high density ratio and high Reynolds number. The interface capturing level set method is coupled with the conservative form of the Navier–Stokes equations, and the multilevel reinitialization technique is applied for mass conservation. This adaptive framework allows us to advance all variables level by level using either the subcycling or the non-subcycling method to decouple the data ad-vancement on each level. The accuracy and robustness of the framework are validated by a variety of canonical two-phase flow problems. We demonstrate that the consistent scheme results in a numerically stable solution in flows with high density ratios(up to 106) and high Reynolds numbers(up to 106), while the inconsistent scheme exhibits non-physical fluid behaviors in these tests. Furthermore, it is shown that the subcycling and non-subcycling methods provide consistent results and that both of them can accurately resolve the interfaces of the two-phase flows with surface tension effects. Finally, a 3D breaking wave problem is simulated to show the efficiency and significant speedup of the proposed framework using AMR.