More Like this

1. Computational mechanisms of distributed value representations and mixed learning strategies

   https://doi.org/10.1038/s41467-021-27413-2  

   Farashahi, Shiva; Soltani, Alireza (December 2021, Nature Communications)

   Abstract Learning appropriate representations of the reward environment is challenging in the real world where there are many options, each with multiple attributes or features. Despite existence of alternative solutions for this challenge, neural mechanisms underlying emergence and adoption of value representations and learning strategies remain unknown. To address this, we measure learning and choice during a multi-dimensional probabilistic learning task in humans and trained recurrent neural networks (RNNs) to capture our experimental observations. We find that human participants estimate stimulus-outcome associations by learning and combining estimates of reward probabilities associated with the informative feature followed by those of informative conjunctions. Through analyzing representations, connectivity, and lesioning of the RNNs, we demonstrate this mixed learning strategy relies on a distributed neural code and opponency between excitatory and inhibitory neurons through value-dependent disinhibition. Together, our results suggest computational and neural mechanisms underlying emergence of complex learning strategies in naturalistic settings. 

   more » « less
2. Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning

   Yang, Zhibo; Huang, Lihan; Chen, Yupei; Wei, Zijun; Ahn, S.; Samaras, Dimitris; Hoai, Minh (June 2020, IEEE Conference on Computer Vision and Pattern Recognition (CVPR))

   Human gaze behavior prediction is important for behavioral vision and for computer vision applications. Most models mainly focus on predicting free-viewing behavior using saliency maps, but do not generalize to goal-directed behavior, such as when a person searches for a visual target object. We propose the first inverse reinforcement learning (IRL) model to learn the internal reward function and policy used by humans during visual search. We modeled the viewer’s internal belief states as dynamic contextual belief maps of object locations. These maps were learned and then used to predict behavioral scanpaths for multiple target categories. To train and evaluate our IRL model we created COCO-Search18, which is now the largest dataset of highquality search fixations in existence. COCO-Search18 has 10 participants searching for each of 18 target-object categories in 6202 images, making about 300,000 goal-directed fixations. When trained and evaluated on COCO-Search18, the IRL model outperformed baseline models in predicting search fixation scanpaths, both in terms of similarity to human search behavior and search efficiency. Finally, reward maps recovered by the IRL model reveal distinctive targetdependent patterns of object prioritization, which we interpret as a learned object context. 

   more » « less
3. Information-driven attentional capture

   https://doi.org/10.3758/s13414-024-03008-z  

   Doyle, Alenka; Volkova, Kamilla; Crotty, Nicholas; Massa, Nicole; Grubb, Michael_A (February 2025, Attention, Perception, & Psychophysics)

   Abstract Visual attention, the selective prioritization of sensory information, is crucial in dynamic, information-rich environments. That both internal goals and external salience modulate the allocation of attention is well established. However, recent empirical work has found instances of experience-driven attention, wherein task-irrelevant, physically non-salient stimuli reflexively capture attention in ways that are contingent on an observer’s unique history. The prototypical example of experience-driven attention relies on a history of reward associations, with evidence attributing the phenomenon to reward-prediction errors. However, a mechanistic account, differing from the reward-prediction error hypothesis, is needed to explain how, in the absence of monetary reward, a history of target-seeking leads to attentional capture. Here we propose that what drives attentional capture in such cases is not target-seeking, but an association with instrumental information. To test this hypothesis, we used pre-cues to render the information provided by a search target either instrumental or redundant. We found that task-irrelevant, physically non-salient distractors associated with instrumental information were more likely to draw eye movements (a sensitive metric of information sampling) than were distractors associated with redundant information. Furthermore, saccading to an instrumental-information-associated distractor led to a greater behavioral cost: response times were slowed more severely. Crucially, the distractors had equivalent histories as sought targets, so any attentional differences between them must be due to different information histories resulting from our experimental manipulation. These findings provide strong evidence for the information history hypothesis and offer a method for exploring the neural signature of information-driven attentional capture. 

   more » « less
4. Learning from Demonstration Framework for Multi-Robot Systems Using Interaction Keypoints and Soft Actor-Critic Methods

   https://doi.org/10.1109/IROS58592.2024.10802324  

   Venkatesh, Vishnunandan_L N; Min, Byung-Cheol (October 2024, IEEE)

   Learning from Demonstration (LfD) is a promising approach to enable Multi-Robot Systems (MRS) to acquire complex skills and behaviors. However, the intricate interactions and coordination challenges in MRS pose significant hurdles for effective LfD. In this paper, we present a novel LfD framework specifically designed for MRS, which leverages visual demonstrations to capture and learn from robot-robot and robot-object interactions. Our framework introduces the concept of Interaction Keypoints (IKs) to transform the visual demonstrations into a representation that facilitates the inference of various skills necessary for the task. The robots then execute the task using sensorimotor actions and reinforcement learning (RL) policies when required. A key feature of our approach is the ability to handle unseen contact-based skills that emerge during the demonstration. In such cases, RL is employed to learn the skill using a classifier-based reward function, eliminating the need for manual reward engineering and ensuring adaptability to environmental changes. We evaluate our framework across a range of mobile robot tasks, covering both behavior-based and contact-based domains. The results demonstrate the effectiveness of our approach in enabling robots to learn complex multi-robot tasks and behaviors from visual demonstrations. 

   more » « less
5. Deep Deterministic Policy Gradients with Transfer Learning Framework in StarCraft Micromanagement

   https://doi.org/10.1109/EIT.2019.8833742  

   Xie, Dong; Zhong, Xiangnan (May 2019, 2019 IEEE International Conference on Electro Information Technology (EIT))

   This paper proposes an intelligent multi-agent approach in a real-time strategy game, StarCraft, based on the deep deterministic policy gradients (DDPG) techniques. An actor and a critic network are established to estimate the optimal control actions and corresponding value functions, respectively. A special reward function is designed based on the agents' own condition and enemies' information to help agents make intelligent control in the game. Furthermore, in order to accelerate the learning process, the transfer learning techniques are integrated into the training process. Specifically, the agents are trained initially in a simple task to learn the basic concept for the combat, such as detouring moving, avoiding and joining attacking. Then, we transfer this experience to the target task with a complex and difficult scenario. From the experiment, it is shown that our proposed algorithm with transfer learning can achieve better performance. 

   more » « less