skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Bounded Rationality in Las Vegas: Probabilistic Finite Automata Play Multi-Armed Bandits
While traditional economics assumes that humans are fully rational agents who always maximize their expected utility, in practice, we constantly observe apparently irrational behavior. One explanation is that people have limited computational power, so that they are, quite rationally, making the best decisions they can, given their computational limitations. To test this hypothesis, we consider the multi-armed bandit (MAB) problem. We examine a simple strategy for playing an MAB that can be implemented easily by a probabilistic finite automaton (PFA). Roughly speaking, the PFA sets certain expectations, and plays an arm as long as it meets them. If the PFA has sufficiently many states, it performs near-optimally. Its performance degrades gracefully as the number of states decreases. Moreover, the PFA acts in a “human-like” way, exhibiting a number of standard human biases, like an optimism bias and a negativity bias.  more » « less
Award ID(s):
1703846
PAR ID:
10165587
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Proceedings of the 36th Conference on Uncertainty in AI (UAI 2020)
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    While traditional economics assumes that humans are fully rational agents who always maximize their expected utility, in practice, we constantly observe apparently irrational behavior. One explanation is that people have limited computational power, so that they are, quite rationally, making the best decisions they can, given their computational limitations. To test this hypothesis, we consider the multi-armed bandit (MAB) problem. We examine a simple strategy for playing an MAB that can be implemented easily by a probabilistic finite automaton (PFA). Roughly speaking, the PFA sets certain expectations, and plays an arm as long as it meets them. If the PFA has sufficiently many states, it performs near-optimally. Its performance degrades gracefully as the number of states decreases. Moreover, the PFA acts in a ``human-like'' way, exhibiting a number of standard human biases, like an optimism bias and a negativity bias. 
    more » « less
  2. Many studies have shown that humans are ``predictably irrational'': they do not act in a fully rational way, but their deviations from rational behavior are quite systematic. Our goal is to see the extent to which we can explain and justify these deviations as the outcome of rational but resource-bounded agents doing as well as they can, given their limitations. We focus on the well-studied ranger-poacher game, where rangers are trying to protect a number of sites from poaching. We capture the computational limitations by modeling the poacher and the ranger as probabilistic finite automata (PFAs). We show that, with sufficiently large memory, PFAs learn to play the Nash equilibrium (NE) strategies of the game and achieve the NE utility. However, if we restrict the memory, we get more ``human-like'' behaviors, such as probability matching (i.e., visiting sites in proportion to the probability of a rhino being there), and avoiding sites where there was a bad outcome (e.g., the poacher was caught by the ranger), that we also observed in experiments conducted on Amazon Mechanical Turk. Interestingly, we find that adding human-like behaviors such as probability matching and overweighting significant events (like getting caught) actually improves performance, showing that this seemingly irrational behavior can be quite rational. 
    more » « less
  3. Abstract Antibody–drug conjugates (ADCs) are antibody‐based therapeutics that have proven to be highly effective cancer treatment platforms. They are composed of monoclonal antibodies conjugated with highly potent drugs via chemical linkers. Compared to cysteine‐targeted chemistries, conjugation at native lysine residues can lead to a higher degree of structural heterogeneity, and thus it is important to evaluate the impact of conjugation on antibody conformation. Here, we present a workflow involving native ion mobility (IM)‐MS and gas‐phase unfolding for the structural characterization of lysine‐linked monoclonal antibody (mAb)–biotin conjugates. Following the determination of conjugation states via denaturing Liquid Chromatography‐Mass Spectrometry (LC–MS) measurements, we performed both size exclusion chromatography (SEC) and native IM‐MS measurements in order to compare the structures of biotinylated and unmodified IgG1 molecules. Hydrodynamic radii (Rh) and collision cross‐sectional (CCS) values were insufficient to distinguish the conformational changes in these antibody–biotin conjugates owing to their flexible structures and limited instrument resolution. In contrast, collision induced unfolding (CIU) analyses were able to detect subtle structural and stability differences in the mAb upon biotin conjugation, exhibiting a sensitivity to mAb conjugation that exceeds native MS analysis alone. Destabilization of mAb–biotin conjugates was detected by both CIU and differential scanning calorimetry (DSC) data, suggesting a previously unknown correlation between the two measurement tools. We conclude by discussing the impact of IM‐MS and CIU technologies on the future of ADC development pipelines. 
    more » « less
  4. null (Ed.)
    The Georgia Tech Miniature Autonomous Blimp (GT-MAB) needs localization algorithms to navigate to way-points in an indoor environment without leveraging an external motion capture system. Indoor aerial robots often require a motion capture system for localization or employ simultaneous localization and mapping (SLAM) algorithms for navigation. The proposed strategy for GT-MAB localization can be accomplished using lightweight sensors on a weight-constrained platform like the GT-MAB. We train an end-to-end convolutional neural network (CNN) that predicts the horizontal position and heading of the GT-MAB using video collected by an onboard monocular RGB camera. On the other hand, the height of the GT-MAB is estimated from measurements through a time-of-flight (ToF) single-beam laser sensor. The monocular camera and the single-beam laser sensor are sufficient for the localization algorithm to localize the GT-MAB in real time, achieving the averaged 3D positioning errors to be less than 20 cm, and the averaged heading errors to be less than 3 degrees. With the accuracy of our proposed localization method, we are able to use simple proportional-integral-derivative controllers to control the GT-MAB for waypoint navigation. Experimental results on the waypoint following are provided, which demonstrates the use of a CNN as the primary localization method for estimating the pose of an indoor robot that successfully enables navigation to specified waypoints. 
    more » « less
  5. Abstract. Previous studies have demonstrated volatility-dependent absorption of gas-phase volatile organic compounds (VOCs) to Teflon and other polymers. Polymer–VOC interactions are relevant for atmospheric chemistry sampling, as gas–wall partitioning in polymer tubing can cause delays and biases during measurements. They are also relevant to the study of indoor chemistry, where polymer-based materials are abundant (e.g., carpets and paints). In this work, we quantify the absorptive capacities of multiple tubing materials, including four nonconductive polymers (important for gas sampling and indoor air quality), four electrically conductive polymers and two commercial steel coatings (for gas and particle sampling). We compare their performance to previously characterized materials. To quantify the absorptive capacities, we expose the tubing to a series of ketones in the volatility range 104–109 µg m−3 and monitor transmission. For slow-diffusion polymers (e.g., perfluoroalkoxy alkane (PFA) Teflon and nylon), absorption is limited to a thin surface layer, and a single-layer absorption model can fit the data well. For fast-diffusion polymers (e.g., polyethylene and conductive silicone), a larger depth of the polymer is available for diffusion, and a multilayer absorption model is needed. The multilayer model allows fitting solid-phase diffusion coefficients for different materials, which range from 4×10-9 to 4×10-7 cm2 s−1. These diffusion coefficients are ∼ 8 orders of magnitude larger than literature values for fluorinated ethylene propylene (FEP) Teflon film. This enormous difference explains the differences in VOC absorption measured here. We fit an equivalent absorptive mass (CW, µg m−3) for each absorptive material. We found PFA to be the least absorptive, with CW ∼ 105 µg m−3, and conductive silicone to be the most absorptive, with CW ∼ 1013 µg m−3. PFA transmits VOCs easily and intermediate-volatility species (IVOCs) with quantifiable delays. In contrast, conductive silicone tubing transmits only the most volatile VOCs, denuding all lower-volatility species. Semi-volatile species (SVOCs) are very difficult to sample quantitatively through any tubing material. We demonstrate a system combining several slow- and fast-diffusion tubing materials that can be used to separate a mixture of VOCs into volatility classes. New conductive silicone tubing contaminated the gas stream with siloxanes, but this effect was reduced 10 000-fold for aged tubing, while maintaining the same absorptive properties. SilcoNert (tested in this work) and Silonite (tested in previous work) steel coatings showed gas transmission that was almost as good as PFA, but since they undergo adsorption, their delay times may be humidity- and concentration-dependent. 
    more » « less