Human gaze behavior prediction is important for behavioral vision and for computer vision applications. Most models mainly focus on predicting free-viewing behavior using saliency maps, but do not generalize to goal-directed behavior, such as when a person searches for a visual target object. We propose the first inverse reinforcement learning (IRL) model to learn the internal reward function and policy used by humans during visual search. We modeled the viewer’s internal belief states as dynamic contextual belief maps of object locations. These maps were learned and then used to predict behavioral scanpaths for multiple target categories. To train and evaluate our IRL model we created COCO-Search18, which is now the largest dataset of highquality search fixations in existence. COCO-Search18 has 10 participants searching for each of 18 target-object categories in 6202 images, making about 300,000 goal-directed fixations. When trained and evaluated on COCO-Search18, the IRL model outperformed baseline models in predicting search fixation scanpaths, both in terms of similarity to human search behavior and search efficiency. Finally, reward maps recovered by the IRL model reveal distinctive targetdependent patterns of object prioritization, which we interpret as a learned object context.
more »
« less
COCO-Search18 fixation dataset for predicting goal-directed attention control
Attention control is a basic behavioral process that has been studied for decades. The currently best models of attention control are deep networks trained on free-viewing behavior to predict bottom-up attention control – saliency. We introduce COCO-Search18, the first dataset of laboratory-quality
- PAR ID:
- 10223276
- Publisher / Repository:
- Nature Publishing Group
- Date Published:
- Journal Name:
- Scientific Reports
- Volume:
- 11
- Issue:
- 1
- ISSN:
- 2045-2322
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
Abstract This paper presents a search for dark matter,
$$\chi $$ W boson. The analysis is based on proton–proton collision data collected with the ATLAS experiment at$$\sqrt{s}=$$ $$^{-1}$$ b -jet and large missing transverse momentum. In addition, a result from a previous search considering two-charged-lepton final states is included in the interpretation of the results. The data are found to be in good agreement with the Standard Model predictions and the results are interpreted in terms of 95% confidence-level exclusion limits in the context of a class of dark matter models involving an extended two-Higgs-doublet sector together with a pseudoscalar mediator particle. The search is particularly sensitive to on-shell production of the charged Higgs boson state,$$H^{\pm }$$ a :$$H^{\pm } \rightarrow W^\pm a (\rightarrow \chi \chi )$$ $$H^{\pm }$$ a masses up to 350 GeV are excluded assuming a$$\tan \beta $$ a of 150 (250) GeV,$$\tan \beta $$ $$H^{\pm }$$ $$\tan \beta $$ $$H^{\pm }$$ -
more » « less
A<sc>bstract</sc> A search for supersymmetry is presented in events with a single charged lepton, electron or muon, and multiple hadronic jets. The data correspond to an integrated luminosity of 138 fb
− 1of proton-proton collisions at a center-of-mass energy of 13 TeV, recorded by the CMS experiment at the CERN LHC. The search targets gluino pair production, where the gluinos decay into final states with the lightest supersymmetric particle (LSP) and either a top quark-antiquark ($$ \textrm{t}\overline{\textrm{t}} $$ $$ \textrm{q}\overline{\textrm{q}} $$ $$ \textrm{t}\overline{\textrm{t}} $$ $$ \textrm{t}\overline{\textrm{t}} $$ $$ \textrm{q}\overline{\textrm{q}} $$ -
more » « less
Abstract A search for supersymmetry involving the pair production of gluinos decaying via off-shell third-generation squarks into the lightest neutralino
$$(\tilde{\chi }^0_1)$$ $$\sqrt{s} = 13$$ $$^{-1}$$ b -hadrons. Both a simple kinematic event selection and an event selection based upon a deep neural-network are used. No significant excess above the predicted background is found. In simplified models involving the pair production of gluinos that decay via off-shell top (bottom) squarks, gluino masses less than 2.44 TeV (2.35 TeV) are excluded at 95% CL for a massless$$\tilde{\chi }^0_1.$$ $$b\bar{b}\tilde{\chi }^0_1,$$ $$t\bar{t}\tilde{\chi }^0_1$$ $$t\bar{b}\tilde{\chi }^-_1/\bar{t}b\tilde{\chi }^+_1.$$ -
more » « less
Abstract The need to determine the structure of a graph arises in many applications. This paper studies directed graphs and defines the notions of
$$\ell$$ $$\{\ell ,k\}$$
