More Like this

1. Practical Quasi-Newton Methods for Training Deep Neural Networks

   Goldfarb, D. ; Ren, Y. ; Bahamou, A. ( January 2020 , Advances in neural information processing systems)

   We consider the development of practical stochastic quasi-Newton, and in particular Kronecker-factored block diagonal BFGS and L-BFGS methods, for training deep neural networks (DNNs). In DNN training, the number of variables and components of the gradient n is often of the order of tens of millions and the Hessian has n^ 2 elements. Consequently, computing and storing a full n times n BFGS approximation or storing a modest number of (step, change in gradient) vector pairs for use in an L-BFGS implementation is out of the question. In our proposed methods, we approximate the Hessian by a block-diagonal matrix and use the structure of the gradient and Hessian to further approximate these blocks, each of which corresponds to a layer, as the Kronecker product of two much smaller matrices. This is analogous to the approach in KFAC, which computes a Kronecker-factored block diagonal approximation to the Fisher matrix in a stochastic natural gradient method. Because the indefinite and highly variable nature of the Hessian in a DNN, we also propose a new damping approach to keep the upper as well as the lower bounds of the BFGS and L-BFGS approximations bounded. In tests on autoencoder feed-forward network models with either nine or thirteen layers applied to three datasets, our methods outperformed or performed comparably to KFAC and state-of-the-art first-order stochastic methods. 

   more » « less
2. Multi-scale analysis of the Monoceros OB 1 star-forming region: II. Colliding filaments in the Monoceros OB1 molecular cloud

   https://doi.org/10.1051/0004-6361/201834903  

   Montillaud, Julien ; Juvela, Mika ; Vastel, Charlotte ; He, Jinhua ; Liu, Tie ; Ristorcelli, Isabelle ; Eden, David J. ; Kang, Sung-ju ; Kim, Kee-Tae ; Koch, Patrick M. ; et al ( November 2019 , Astronomy & Astrophysics)

   Context. We started a multi-scale analysis of star formation in G202.3+2.5, an intertwined filamentary sub-region of the Monoceros OB1 molecular complex, in order to provide observational constraints on current theories and models that attempt to explain star formation globally. In the first paper (Paper I), we examined the distributions of dense cores and protostars and found enhanced star formation activity in the junction region of the filaments. Aims. In this second paper, we aim to unveil the connections between the core and filament evolutions, and between the filament dynamics and the global evolution of the cloud. Methods. We characterise the gas dynamics and energy balance in different parts of G202.3+2.5 using infrared observations from the Herschel and WISE telescopes and molecular tracers observed with the IRAM 30-m and TRAO 14-m telescopes. The velocity field of the cloud is examined and velocity-coherent structures are identified, characterised, and put in perspective with the cloud environment. Results. Two main velocity components are revealed, well separated in radial velocities in the north and merged around the location of intense N 2 H + emission in the centre of G202.3+2.5 where Paper I found the peak of star formation activity. We show that the relative position of the two components along the sightline, and the velocity gradient of the N 2 H + emission imply that the components have been undergoing collision for ~10 5 yr, although it remains unclear whether the gas moves mainly along or across the filament axes. The dense gas where N 2 H + is detected is interpreted as the compressed region between the two filaments, which corresponds to a high mass inflow rate of ~1 × 10 −3 M ⊙ yr −1 and possibly leads to a significant increase in its star formation efficiency. We identify a protostellar source in the junction region that possibly powers two crossed intermittent outflows. We show that the H  II region around the nearby cluster NCG 2264 is still expanding and its role in the collision is examined. However, we cannot rule out the idea that the collision arises mostly from the global collapse of the cloud. Conclusions. The (sub-)filament-scale observables examined in this paper reveal a collision between G202.3+2.5 sub-structures and its probable role in feeding the cores in the junction region. To shed more light on this link between core and filament evolutions, one must characterise the cloud morphology, its fragmentation, and magnetic field, all at high resolution. We consider the role of the environment in this paper, but a larger-scale study of this region is now necessary to investigate the scenario of a global cloud collapse. 

   more » « less
3. Global analysis of synchronization performance for power systems: bridging the theory-practice gap

   https://doi.org/10.1109/TAC.2019.2942536  

   Paganini, Fernando ; Mallada, Enrique ( September 2019 , IEEE Transactions on Automatic Control)

   The issue of synchronization in the power grid is receiving renewed attention, as new energy sources with different dynamics enter the picture. Global metrics have been proposed to evaluate performance, and analyzed under highly simplified assumptions. In this work we extend this approach to more realistic network scenarios, and more closely connect it with metrics used in power engineering practice. In particular, our analysis covers networks with generators of heterogeneous ratings and richer dynamic models of machines. Under a suitable proportionality assumption in the parameters, we show that the step response of bus frequencies can be decomposed in two components. The first component is a system-wide frequency that captures the aggregate grid behavior, and the residual component represents the individual bus frequency deviations from the aggregate. Using this decomposition, we define --and compute in closed form-- several metrics that capture dynamic behaviors that are of relevance for power engineers. In particular, using the system frequency, we define industry-style metrics (Nadir, RoCoF) that are evaluated through a representative machine. We further use the norm of the residual component to define a synchronization cost that can appropriately quantify inter-area oscillations. Finally, we employ robustness analysis tools to evaluate deviations from our proportionality assumption. We show that the system frequency still captures the grid steady-state deviation, and becomes an accurate reduced-order model of the grid as the network connectivity grows. Simulation studies with practically relevant data are included to validate the theory and further illustrate the impact of network structure and parameters on synchronization. Our analysis gives conclusions of practical interest, sometimes challenging the conventional wisdom in the field. 

   more » « less
4. Shifts in the Isotopic Composition of Nitrous Oxide Between El Niño and La Niña in the Eastern Tropical South Pacific

   https://doi.org/10.1029/2023GB007959  

   Gluschankoff, Noah ; Santoro, Alyson E. ; Buchwald, Carolyn ; Casciotti, Karen L. ( October 2023 , Global Biogeochemical Cycles)

   The El Niño‐Southern Oscillation (ENSO) is a natural climate phenomenon that alters the biogeochemical and physical dynamics of the Eastern Tropical Pacific Ocean. Its two phases, El Niño and La Niña, are characterized by decreased and increased coastal upwelling, respectively, which have cascading effects on primary productivity, organic matter supply, and ocean‐atmosphere interactions. The Eastern Tropical South Pacific oxygen minimum zone is a source of nitrous oxide (N₂O), a potent greenhouse gas, to the atmosphere. Here, we present the first study to directly compare N₂O sources during opposing ENSO phases using N₂O isotopocule analyses. Our data show that during La Niña, N₂O accumulation increased six‐fold in the upper 100 m of the water column, and N₂O fluxes to the atmosphere increased up to 20‐fold. N₂O isotopocule data demonstrated substantial increases in δ¹⁸O up to 60.5‰ and decreases in δ¹⁵N^βdown to −10.3‰ in the oxycline, signaling a shift in N₂O cycling during La Niña compared to El Niño. During El Niño, N₂O production was primarily due to ammonia‐oxidizing archaea, whereas during La Niña, N₂O production by incomplete denitrification supplemented that from ammonia‐oxidation, with N₂O consumption likely maintaining the high site preference values (up to 26.7‰). Ultimately, our results illustrate a strong connection between upwelling intensity, biogeochemistry, and N₂O flux to the atmosphere. Additionally, they highlight the combined power of N₂O isotopocule analysis and repeat measurements in the same region to constrain N₂O interannual variability and cycling dynamics under different climate scenarios.

    

   more » « less
5. On comparison of luminescence properties of La 2 Zr 2 O 7 and La 2 Hf 2 O 7 nanoparticles

   https://doi.org/10.1111/jace.16693  

   Gupta, Santosh K. ; Abdou, Maya ; Ghosh, Partha S. ; Zuniga, Jose P. ; Manoharan, Ezhilarasan ; Kim, HyeongJun ; Mao, Yuanbing ( July 2019 , Journal of the American Ceramic Society)

   Unveiling the underlying mechanisms of properties of functional materials, including the luminescence differences among similar pyrochlores A₂B₂O₇, opens new gateways to select proper hosts for various optoelectronic applications by scientists and engineers. For example, although La₂Zr₂O₇(LZO) and La₂Hf₂O₇(LHO) pyrochlores have similar chemical compositional and crystallographic structural features, they demonstrate different luminescence properties both before and after doped with Eu³⁺ions. Based on our earlier work, LHO‐based nanophosphors display higher photo‐ and radioluminescence intensity, higher quantum efficiency, and longer excited state lifetime compared to LZO‐based nanophosphors. Moreover, under electronic O²⁻→Zr⁴⁺/Hf⁴⁺transition excitation at 306 nm, undoped LHO nanoparticles (NPs) have only violet blue emission, whereas LZO NPs show violet blue and red emissions. In this study, we have combined experimental and density functional theory (DFT) based theoretical calculation to explain the observed results. First, we calculated the density of state (DOS) based on DFT and studied the energetics of ionized oxygen vacancies in the band gaps of LZO and LHO theoretically, which explain their underlying luminescence difference. For Eu³⁺‐doped NPs, we performed emission intensity and lifetime calculations and found that the LHOE NPs have higher host to dopant energy transfer efficiency than the LZOE NPs (59.3% vs 24.6%), which accounts for the optical performance superiority of the former over the latter. Moreover, by corroborating our experimental data with the DFT calculations, we suggest that the Eu³⁺doping states in LHO present at exact energy position (both in majority and minority spin components) where oxygen defect states are located unlike those in LZO. Lastly, both the NPs show negligible photobleaching highlighting their potential for bioimaging applications. This current report provides a deeper understanding of the advantages of LHO over LZO as an advanced host for phosphors, scintillators, and fluoroimmunoassays.

    

   more » « less