Search for: All records

We study the multi-agent multi-armed bandit (MAMAB) problem, where agents are factored into overlapping groups. Each group represents a hyperedge, forming a hypergraph over the agents. At each round of interaction, the learner pulls a joint arm (composed of individual arms for each agent) and receives a reward according to the hypergraph structure. Specifically, we assume there is a local reward for each hyperedge, and the reward of the joint arm is the sum of these local rewards. Previous work introduced the multi-agent Thompson sampling (MATS) algorithm and derived a Bayesian regret bound. However, it remains an open problem how to derive a frequentist regret bound for Thompson sampling in this multi-agent setting. To address these issues, we propose an efficient variant of MATS, the epsilon-exploring Multi-Agent Thompson Sampling (eps-MATS) algorithm, which performs MATS exploration with probability epsilon while adopts a greedy policy otherwise. We prove that eps-MATS achieves a worst-case frequentist regret bound that is sublinear in both the time horizon and the local arm size. We also derive a lower bound for this setting, which implies our frequentist regret upper bound is optimal up to constant and logarithm terms, when the hypergraph is sufficiently sparse. Thorough experiments on standard MAMAB problems demonstrate the superior performance and the improved computational efficiency of eps-MATS compared with existing algorithms in the same setting. 

We study the multi-agent multi-armed bandit (MAMAB) problem, where agents are factored into overlapping groups. Each group represents a hyperedge, forming a hypergraph over the agents. At each round of interaction, the learner pulls a joint arm (composed of individual arms for each agent) and receives a reward according to the hypergraph structure. Specifically, we assume there is a local reward for each hyperedge, and the reward of the joint arm is the sum of these local rewards. Previous work introduced the multi-agent Thompson sampling (MATS) algorithm and derived a Bayesian regret bound. However, it remains an open problem how to derive a frequentist regret bound for Thompson sampling in this multi-agent setting. To address these issues, we propose an efficient variant of MATS, the epsilon-exploring Multi-Agent Thompson Sampling (eps-MATS) algorithm, which performs MATS exploration with probability epsilon while adopts a greedy policy otherwise. We prove that eps-MATS achieves a worst-case frequentist regret bound that is sublinear in both the time horizon and the local arm size. We also derive a lower bound for this setting, which implies our frequentist regret upper bound is optimal up to constant and logarithm terms, when the hypergraph is sufficiently sparse. Thorough experiments on standard MAMAB problems demonstrate the superior performance and the improved computational efficiency of eps-MATS compared with existing algorithms in the same setting. 

Biomineralizing cells concentrate dissolved inorganic carbon (DIC) and remove protons from the site of mineral precipitation. However, the molecular regulatory mechanisms that orchestrate pH homeostasis and biomineralization of calcifying cells are poorly understood. Here, we report that the acid-base sensing enzyme soluble adenylyl cyclase (sAC) coordinates intracellular pH (pH i ) regulation in the calcifying primary mesenchyme cells (PMCs) of sea urchin larvae. Single-cell transcriptomics, in situ hybridization, and immunocytochemistry elucidated the spatiotemporal expression of sAC during skeletogenesis. Live pH i imaging of PMCs revealed that the downregulation of sAC activity with two structurally unrelated small molecules inhibited pH i regulation of PMCs, an effect that was rescued by the addition of cell-permeable cAMP. Pharmacological sAC inhibition also significantly reduced normal spicule growth and spicule regeneration, establishing a link between PMC pH i regulation and biomineralization. Finally, increased expression of sAC mRNA was detected during skeleton remineralization and exposure to CO 2 -induced acidification. These findings suggest that transcriptional regulation of sAC is required to promote remineralization and to compensate for acidic stress. This work highlights the central role of sAC in coordinating acid-base regulation and biomineralization in calcifying cells of a marine animal.