Search for: All records

In this paper we consider the training stability of recurrent neural networks (RNNs) and propose a family of RNNs, namely SBO-RNN, that can be formulated using stochastic bilevel optimization (SBO). With the help of stochastic gradient descent (SGD), we manage to convert the SBO problem into an RNN where the feedforward and backpropagation solve the lower and upper-level optimization for learning hidden states and their hyperparameters, respectively. We prove that under mild conditions there is no vanishing or exploding gradient in training SBO-RNN. Empirically we demonstrate our approach with superior performance on several benchmark datasets, with fewer parameters, less training data, and much faster convergence. Code is available at https://zhang-vislab.github.io. 

Abstract Magnetic fields have an important role in the evolution of interstellar medium and star formation 1,2 . As the only direct probe of interstellar field strength, credible Zeeman measurements remain sparse owing to the lack of suitable Zeeman probes, particularly for cold, molecular gas 3 . Here we report the detection of a magnetic field of +3.8 ± 0.3 microgauss through the H  I narrow self-absorption (HINSA) 4,5 towards L1544 6,7 —a well-studied prototypical prestellar core in an early transition between starless and protostellar phases 8–10 characterized by a high central number density 11 and a low central temperature 12 . A combined analysis of the Zeeman measurements of quasar H  I absorption, H  I emission, OH emission and HINSA reveals a coherent magnetic field from the atomic cold neutral medium (CNM) to the molecular envelope. The molecular envelope traced by the HINSA is found to be magnetically supercritical, with a field strength comparable to that of the surrounding diffuse, magnetically subcritical CNM despite a large increase in density. The reduction of the magnetic flux relative to the mass, which is necessary for star formation, thus seems to have already happened during the transition from the diffuse CNM to the molecular gas traced by the HINSA. This is earlier than envisioned in the classical picture where magnetically supercritical cores capable of collapsing into stars form out of magnetically subcritical envelopes 13,14 . 

Many sequential decision making tasks can be viewed as combinatorial optimiza- tion problems over a large number of actions. When the cost of evaluating an ac- tion is high, even a greedy algorithm, which iteratively picks the best action given the history, is prohibitive to run. In this paper, we aim to learn a greedy heuris- tic for sequentially selecting actions as a surrogate for invoking the expensive oracle when evaluating an action. In particular, we focus on a class of combinato- rial problems that can be solved via submodular maximization (either directly on the objective function or via submodular surrogates). We introduce a data-driven optimization framework based on the submodular-norm loss, a novel loss func- tion that encourages the resulting objective to exhibit diminishing returns. Our framework outputs a surrogate objective that is efficient to train, approximately submodular, and can be made permutation-invariant. The latter two properties al- low us to prove strong approximation guarantees for the learned greedy heuristic. Furthermore, our model is easily integrated with modern deep imitation learning pipelines for sequential prediction tasks. We demonstrate the performance of our algorithm on a variety of batched and sequential optimization tasks, including set cover, active learning, and data-driven protein engineering. 

Abstract The dispersive sweep of fast radio bursts (FRBs) has been used to probe the ionized baryon content of the intergalactic medium 1 , which is assumed to dominate the total extragalactic dispersion. Although the host-galaxy contributions to the dispersion measure appear to be small for most FRBs 2 , in at least one case there is evidence for an extreme magneto-ionic local environment 3,4 and a compact persistent radio source 5 . Here we report the detection and localization of the repeating FRB 20190520B, which is co-located with a compact, persistent radio source and associated with a dwarf host galaxy of high specific-star-formation rate at a redshift of 0.241 ± 0.001. The estimated host-galaxy dispersion measure of approximately $${903}_{-111}^{+72}$$ 903 − 111 + 72 parsecs per cubic centimetre, which is nearly an order of magnitude higher than the average of FRB host galaxies 2,6 , far exceeds the dispersion-measure contribution of the intergalactic medium. Caution is thus warranted in inferring redshifts for FRBs without accurate host-galaxy identifications. 

We study the problem of learning sequential decision-making policies in settings with multiple state-action representations. Such settings naturally arise in many domains, such as planning (e.g., multiple integer programming formulations) and various combinatorial optimization problems (e.g., those with both integer programming and graph-based formulations). Inspired by the classical co-training framework for classification, we study the problem of co-training for policy learning. We present sufficient conditions under which learning from two views can improve upon learning from a single view alone. Motivated by these theoretical insights, we present a meta-algorithm for co-training for sequential decision making. Our framework is compatible with both reinforcement learning and imitation learning. We validate the effectiveness of our approach across a wide range of tasks, including discrete/continuous control and combinatorial optimization. 

We report the phase-connected timing ephemeris, polarization pulse profiles, Faraday rotation measurements, and Rotating-Vector-Model (RVM) fitting results of 12 millisecond pulsars (MSPs) discovered with the Five-hundred-meter Aperture Spherical radio Telescope (FAST) in the Commensal Radio Astronomy FAST survey (CRAFTS). The timing campaigns were carried out with FAST and Arecibo over 3 yr. 11 of the 12 pulsars are in neutron star–white dwarf binary systems, with orbital periods between 2.4 and 100 d. 10 of them have spin periods, companion masses, and orbital eccentricities that are consistent with the theoretical expectations for MSP–Helium white dwarf (He WD) systems. The last binary pulsar (PSR J1912−0952) has a significantly smaller spin frequency and a smaller companion mass, the latter could be caused by a low orbital inclination for the system. Its orbital period of 29 d is well within the range of orbital periods where some MSP–He WD systems have shown anomalous eccentricities, however, the eccentricity of PSR J1912−0952 is typical of what one finds for the remaining MSP–He WD systems.