Search for: All records

Online reinforcement learning (RL) enhances policies through direct interactions with the environment, but faces challenges related to sample efficiency. In contrast, offline RL leverages extensive pre-collected data to learn policies, but often produces suboptimal results due to limited data coverage. Recent efforts integrate offline and online RL in order to harness the advantages of both approaches. However, effectively combining online and offline RL remains challenging due to issues that include catastrophic forgetting, lack of robustness to data quality and limited sample efficiency in data utilization. In an effort to address these challenges, we introduce A3RL, which incorporates a novel confidence aware Active Advantage Aligned (A3) sampling strategy that dynamically prioritizes data aligned with the policy's evolving needs from both online and offline sources, optimizing policy improvement. Moreover, we provide theoretical insights into the effectiveness of our active sampling strategy and conduct diverse empirical experiments and ablation studies, demonstrating that our method outperforms competing online RL techniques that leverage offline data. Our code will be publicly available at:this https URL. 

Diffusion models (DMs) create samples from a data distribution by starting from random noise and iteratively solving a reverse-time ordinary differential equation (ODE). Because each step in the iterative solution requires an expensive neural function evaluation (NFE), there has been significant interest in approximately solving these diffusion ODEs with only a few NFEs without modifying the underlying model. However, in the few NFE regime, we observe that tracking the true ODE evolution is fundamentally impossible using traditional ODE solvers. In this work, we propose a new method that learns a good solver for the DM, which we call Solving for the Solver (S4S). S4S directly optimizes a solver to obtain good generation quality by learning to match the output of a strong teacher solver. We evaluate S4S on six different pre-trained DMs, including pixel-space and latent-space DMs for both conditional and unconditional sampling. In all settings, S4S uniformly improves the sample quality relative to traditional ODE solvers. Moreover, our method is lightweight, data-free, and can be plugged in black-box on top of any discretization schedule or architecture to improve performance. Building on top of this, we also propose S4S-Alt, which optimizes both the solver and the discretization schedule. By exploiting the full design space of DM solvers, with 5 NFEs, we achieve an FID of 3.73 on CIFAR10 and 13.26 on MS-COCO, representing a 1.5× improvement over previous training-free ODE methods. 

Static typing and dynamic typing have respective strengths and weaknesses, and a language often commits to one typing discipline and inherits the qualities, good or bad. Gradual typing has been developed to reconcile these typing disciplines, allowing a single program to mix both static and dynamic typing. It protects soundness of typed regions with runtime checks when values flown into them do not have required static types. One issue with gradual typing is that such checks can incur significant performance overhead. Previous work on performance has focused on coarse-grained gradual typing where each module (file) has to be fully typed or untyped. In contrast, the performance of fine-grained gradual typing where each single parameter can be partially-typed (such as specifying the parameter as a list without giving element type) has not been investigated. Motivated by this situation, this paper systematically investigates performance of fine-grained gradual typing by studying the performance of more than 1 million programs. These programs are drawn from seven commonly-used benchmarks with different types for parameters: some parameters are untyped, some are statically typed, and others are partially statically typed. The paper observes many interesting phenomena that were previously unknown to the research community. They provide insights into future research directions of understanding, predicting, and optimizing gradual typing performance as well as migrating gradual programs towards more static 

The Fearless Steps Apollo (FS-APOLLO) resource is a collection of over 150,000 hours of audio, associated meta-data, and supplemental technological toolkit intended to benefit the (i) speech processing technology, (ii) communication science, team-based psychology, and history, and (iii) education/STEM, preservation/archival communities. The FSAPOLLO initiative which started in 2014 has since resulted in the preservation of over 75,000 hours of NASA Apollo Missions audio. Systems created for this audio collection have led to the emergence of several new Speech and Language Technologies (SLT). This paper seeks to provide an overview of the latest advancements in the FS-Apollo effort and explore upcoming strategies in big-data deployment, outreach, and novel avenues of K-12 and STEM education facilitated through this resource. 

Abstract We examined three descending positive leaders at distances of 5–11 km and three descending negative leaders at distances of 6–7 km, all simultaneously imaged by high‐speed framing cameras operating in the visible and UV ranges. UV images (290–370 nm) of the positive leaders each exhibited a strong embellishment at the lower channel end, which was not observed in the corresponding visible images (480–800 nm). In contrast, none of the negative leaders exhibited channel embellishment in the UV range and their morphology in UV was similar to that in the visible. Additionally, no embellishment was seen in four negative leaders imaged in UV only. The observed UV embellishment, which is likely to be the streamer zone at the positive‐leader tip, appeared to undergo expansion‐contraction cycles. We attributed the lack of detectable streamer‐zone emission in the UV range in negative leaders to a much lower streamer generation rate compared to positive leaders. 

We report on a nano-infrared (IR) imaging and spectroscopy study of epitaxial graphene on silicon carbide (SiC) by using scattering-type scanning near-field optical microscopy (s-SNOM). With nano-IR imaging, we reveal in real space microscopic domains with distinct IR contrasts. By analyzing the nano-IR, atomic force microscopy, and scanning tunneling microscopy imaging data, we conclude that the imaged domains correspond to single-layer graphene, bilayer graphene (BLG), and higher-doped BLG. With nano-IR spectroscopy, we find that graphene can screen the SiC phonon resonance, and the screening is stronger at more conductive sample regions. Our work offers insights into the rich surface properties of epitaxial graphene and demonstrates s-SNOM as an efficient and effective tool in characterizing graphene and possibly other two-dimensional materials.