skip to main content

This content will become publicly available on January 1, 2024

Title: Optimal Horizon-Free Reward-Free Exploration for Linear Mixture MDPs
Award ID(s):
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
International Conference on Machine Learning (ICML)
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Accurate estimation of solvation free energy (SFE) lays the foundation for accurate prediction of binding free energy. The Poisson‐Boltzmann (PB) or generalized Born (GB) combined with surface area (SA) continuum solvation method (PBSA and GBSA) have been widely used in SFE calculations because they can achieve good balance between accuracy and efficiency. However, the accuracy of these methods can be affected by several factors such as the charge models, polar and nonpolar SFE calculation methods and the atom radii used in the calculation. In this work, the performance of the ABCG2 (AM1‐BCC‐GAFF2) charge model as well as other two charge models, that is, RESP (Restrained Electrostatic Potential) and AM1‐BCC (Austin Model 1‐bond charge corrections), on the SFE prediction of 544 small molecules in water by PBSA/GBSA was evaluated. In order to improve the performance of the PBSA prediction based on the ABCG2 charge, we further explored the influence of atom radii on the prediction accuracy and yielded a set of atom radius parameters for more accurate SFE prediction using PBSA based on the ABCG2/GAFF2 by reproducing the thermodynamic integration (TI) calculation results. The PB radius parameters of carbon, oxygen, sulfur, phosphorus, chloride, bromide and iodine, were adjusted. New atom types,on,oi,hn1,hn2,hn3, were introduced to further improve the fitting performance. Then, we tuned the parameters in the nonpolar SFE model using the experimental SFE data and the PB calculation results. By adopting the new radius parameters and new nonpolar SFE model, the root mean square error (RMSE) of the SFE calculation for the 544 molecules decreased from 2.38 to 1.05 kcal/mol. Finally, the new radius parameters were applied in the prediction of protein‐ligand binding free energies using the MM‐PBSA method. For the eight systems tested, we could observe higher correlation between the experiment data and calculation results and smaller prediction errors for the absolute binding free energies, demonstrating that our new radius parameters can improve the free energy calculation using the MM‐PBSA method.

    more » « less
  2. Abstract Radio free–free emission is considered to be one of the most reliable tracers of star formation in galaxies. However, as it constitutes the faintest part of the radio spectrum—being roughly an order of magnitude less luminous than radio synchrotron emission at the GHz frequencies typically targeted in radio surveys—the usage of free–free emission as a star formation rate tracer has mostly remained limited to the local universe. Here, we perform a multifrequency radio stacking analysis using deep Karl G. Jansky Very Large Array observations at 1.4, 3, 5, 10, and 34 GHz in the COSMOS and GOODS-North fields to probe free–free emission in typical galaxies at the peak of cosmic star formation. We find that z ∼ 0.5–3 star-forming galaxies exhibit radio emission at rest-frame frequencies of ∼65–90 GHz that is ∼1.5–2 times fainter than would be expected from a simple combination of free–free and synchrotron emission, as in the prototypical starburst galaxy M82. We interpret this as a deficit in high-frequency synchrotron emission, while the level of free–free emission is as expected from M82. We additionally provide the first constraints on the cosmic star formation history using free–free emission at 0.5 ≲ z ≲ 3, which are in good agreement with more established tracers at high redshift. In the future, deep multifrequency radio surveys will be crucial in order to accurately determine the shape of the radio spectrum of faint star-forming galaxies, and to further establish radio free–free emission as a tracer of high-redshift star formation. 
    more » « less
  3. Free chlorine and free bromine ( e.g. , HOCl and HOBr) are employed as disinfectants in a variety of aqueous systems, including drinking water, wastewater, ballast water, recreational waters, and cleaning products. Yet, the most widely used methods for quantifying free halogens, including those employing N , N -diethyl- p -phenylenediamine (DPD), cannot distinguish between HOCl and HOBr. Herein, we report methods for selectively quantifying free halogens in a variety of aqueous systems using 1,3,5-trimethoxybenzene (TMB). At near-neutral pH, TMB reacted on the order of seconds with HOCl, HOBr, and inorganic bromamines to yield halogenated products that were readily quantified by liquid chromatography or, following liquid–liquid extraction, gas chromatography-mass spectrometry (GC-MS). The chlorinated and brominated products of TMB were stable, and their molar concentrations were used to calculate the original concentrations of HOCl (method quantitation limit (MQL) by GC-MS = 15 nmol L −1 = 1.1 μg L −1 as Cl 2 ) and HOBr (MQL by GC-MS = 30 nmol L −1 = 2 μg L −1 as Cl 2 ), respectively. Moreover, TMB derivatization was efficacious for quantifying active halogenating agents in drinking water, pool water, chlorinated surface waters, and simulated spa waters treated with 1-bromo-3-chloro-5,5-dimethylhydantoin. TMB was also used to quantify bromide as a trace impurity in 20 nominally bromide-free reagents (following oxidation of bromide by HOCl to HOBr). Several possible interferents were tested, and iodide was identified as impeding accurate quantitation of HOCl and HOBr. Overall, compared to the DPD method, TMB can provide lower MQLs, larger linear ranges, and selectivity between HOCl and HOBr. 
    more » « less