Search for: All records

We initiate the study of federated reinforcement learning under environmental heterogeneity by considering a policy evaluation problem. Our setup involves agents interacting with environments that share the same state and action space but differ in their reward functions and state transition kernels. Assuming agents can communicate via a central server, we ask: Does exchanging information expedite the process of evaluating a common policy? To answer this question, we provide the first comprehensive finite-time analysis of a federated temporal difference (TD) learning algorithm with linear function approximation, while accounting for Markovian sampling, heterogeneity in the agents' environments, and multiple local updates to save communication. Our analysis crucially relies on several novel ingredients: (i) deriving perturbation bounds on TD fixed points as a function of the heterogeneity in the agents' underlying Markov decision processes (MDPs); (ii) introducing a virtual MDP to closely approximate the dynamics of the federated TD algorithm; and (iii) using the virtual MDP to make explicit connections to federated optimization. Putting these pieces together, we rigorously prove that in a low-heterogeneity regime, exchanging model estimates leads to linear convergence speedups in the number of agents. 

We consider Shimura varieties for orthogonal or spin groups acting on hermitian symmetric domains of type IV. We give regular $$p$$ -adic integral models for these varieties over odd primes $$p$$ at which the level subgroup is the connected stabilizer of a vertex lattice in the orthogonal space. Our construction is obtained by combining results of Kisin and the first author with an explicit presentation and resolution of a corresponding local model. 

Let $$(G,\unicode[STIX]{x1D707})$$ be a pair of a reductive group $$G$$ over the $$p$$ -adic integers and a minuscule cocharacter $$\unicode[STIX]{x1D707}$$ of $$G$$ defined over an unramified extension. We introduce and study ‘ $$(G,\unicode[STIX]{x1D707})$$ -displays’ which generalize Zink’s Witt vector displays. We use these to define certain Rapoport–Zink formal schemes purely group theoretically, i.e. without $$p$$ -divisible groups. 

An integrated urban transportation system usually consists of multiple transport modes that have complementary characteristics of capacities, speeds, and costs, facilitating smooth passenger transfers according to planned schedules. However, such an integration is not designed to operate under disruptive events, e.g., a signal failure at a subway station or a breakdown of a bus, which have rippling effects on passenger demand and significantly increase delays. To address these disruptive events, current solutions mainly rely on a substitute service to transport passengers from and to affected areas using adhoc schedules. To fully utilize heterogeneous transportation systems under disruptive events, we design a service called eRoute based on a hierarchical receding horizon control framework to automatically reroute, reschedule, and reallocate multi-mode transportation systems based on real-time and predicted demand and supply. Focusing on an integration of subway and bus, we implement and evaluate eRoute with large datasets including (i) a bus system with 13,000 buses, (ii) a subway system with 127 subway stations, (iii) an automatic fare collection system with a total of 16,840 readers and 8 million card users from a metropolitan city. The data-driven evaluation results show that our solution improves the ratio of served passengers (RSP) by up to 11.5 times and reduces the average traveling time by up to 82.1% compared with existing solutions. 

Abstract Despite the growing number of binary black hole coalescences confidently observed through gravitational waves so far, the astrophysical origin of these binaries remains uncertain. Orbital eccentricity is one of the clearest tracers of binary formation channels. Identifying binary eccentricity, however, remains challenging due to the limited availability of gravitational waveforms that include the effects of eccentricity. Here, we present observational results for a waveform-independent search sensitive to eccentric black hole coalescences, covering the third observing run (O3) of the LIGO and Virgo detectors. We identified no new high-significance candidates beyond those that have already been identified with searches focusing on quasi-circular binaries. We determine the sensitivity of our search to high-mass (total source-frame massM> 70M_⊙) binaries covering eccentricities up to 0.3 at 15 Hz emitted gravitational-wave frequency, and use this to compare model predictions to search results. Assuming all detections are indeed quasi-circular, for our fiducial population model, we place a conservative upper limit for the merger rate density of high-mass binaries with eccentricities 0 <e≤ 0.3 at 16.9 Gpc⁻³yr⁻¹at the 90% confidence level. 

Abstract Gravitational lensing by massive objects along the line of sight to the source causes distortions to gravitational wave (GW) signals; such distortions may reveal information about fundamental physics, cosmology, and astrophysics. In this work, we have extended the search for lensing signatures to all binary black hole events from the third observing run of the LIGO-Virgo network. We search for repeated signals from strong lensing by (1) performing targeted searches for subthreshold signals, (2) calculating the degree of overlap among the intrinsic parameters and sky location of pairs of signals, (3) comparing the similarities of the spectrograms among pairs of signals, and (4) performing dual-signal Bayesian analysis that takes into account selection effects and astrophysical knowledge. We also search for distortions to the gravitational waveform caused by (1) frequency-independent phase shifts in strongly lensed images, and (2) frequency-dependent modulation of the amplitude and phase due to point masses. None of these searches yields significant evidence for lensing. Finally, we use the nondetection of GW lensing to constrain the lensing rate based on the latest merger-rate estimates and the fraction of dark matter composed of compact objects. 

Abstract We present Fermi Gamma-ray Burst Monitor (Fermi-GBM) and Swift Burst Alert Telescope (Swift-BAT) searches for gamma-ray/X-ray counterparts to gravitational-wave (GW) candidate events identified during the third observing run of the Advanced LIGO and Advanced Virgo detectors. Using Fermi-GBM onboard triggers and subthreshold gamma-ray burst (GRB) candidates found in the Fermi-GBM ground analyses, the Targeted Search and the Untargeted Search, we investigate whether there are any coincident GRBs associated with the GWs. We also search the Swift-BAT rate data around the GW times to determine whether a GRB counterpart is present. No counterparts are found. Using both the Fermi-GBM Targeted Search and the Swift-BAT search, we calculate flux upper limits and present joint upper limits on the gamma-ray luminosity of each GW. Given these limits, we constrain theoretical models for the emission of gamma rays from binary black hole mergers.