skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: From the Lab to the Wild: Examining Generalizability of Video-based Mind Wandering Detection
Abstract Student’s shift of attention away from a current learning task to task-unrelated thought, also called mind wandering, occurs about 30% of the time spent on education-related activities. Its frequent occurrence has a negative effect on learning outcomes across learning tasks. Automated detection of mind wandering might offer an opportunity to assess the attentional state continuously and non-intrusively over time and hence enable large-scale research on learning materials and responding to inattention with targeted interventions. To achieve this, an accessible detection approach that performs well for various systems and settings is required. In this work, we explore a new, generalizable approach to video-based mind wandering detection that can be transferred to naturalistic settings across learning tasks. Therefore, we leverage two datasets, consisting of facial videos during reading in the lab (N = 135) and lecture viewing in-the-wild (N = 15). When predicting mind wandering, deep neural networks (DNN) and long short-term memory networks (LSTMs) achieve F$$_{1}$$ 1 scores of 0.44 (AUC-PR = 0.40) and 0.459 (AUC-PR = 0.39), above chance level, with latent features based on transfer-learning on the lab data. When exploring generalizability by training on the lab dataset and predicting on the in-the-wild dataset, BiLSTMs on latent features perform comparably to the state-of-the-art with an F$$_{1}$$ 1 score of 0.352 (AUC-PR = 0.26). Moreover, we investigate the fairness of predictive models across gender and show based on post-hoc explainability methods that employed latent features mainly encode information on eye and mouth areas. We discuss the benefits of generalizability and possible applications.  more » « less
Award ID(s):
1920510 2019805
PAR ID:
10556464
Author(s) / Creator(s):
; ; ; ; ; ; ;
Publisher / Repository:
Springer
Date Published:
Journal Name:
International Journal of Artificial Intelligence in Education
ISSN:
1560-4292
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract We construct an example of a group$$G = \mathbb {Z}^2 \times G_0$$ G = Z 2 × G 0 for a finite abelian group $$G_0$$ G 0 , a subsetEof $$G_0$$ G 0 , and two finite subsets$$F_1,F_2$$ F 1 , F 2 of G, such that it is undecidable in ZFC whether$$\mathbb {Z}^2\times E$$ Z 2 × E can be tiled by translations of$$F_1,F_2$$ F 1 , F 2 . In particular, this implies that this tiling problem isaperiodic, in the sense that (in the standard universe of ZFC) there exist translational tilings ofEby the tiles$$F_1,F_2$$ F 1 , F 2 , but no periodic tilings. Previously, such aperiodic or undecidable translational tilings were only constructed for sets of eleven or more tiles (mostly in $$\mathbb {Z}^2$$ Z 2 ). A similar construction also applies for$$G=\mathbb {Z}^d$$ G = Z d for sufficiently large d. If one allows the group$$G_0$$ G 0 to be non-abelian, a variant of the construction produces an undecidable translational tiling with only one tile F. The argument proceeds by first observing that a single tiling equation is able to encode an arbitrary system of tiling equations, which in turn can encode an arbitrary system of certain functional equations once one has two or more tiles. In particular, one can use two tiles to encode tiling problems for an arbitrary number of tiles. 
    more » « less
  2. Abstract Let$$\mathbb {F}_q^d$$ F q d be thed-dimensional vector space over the finite field withqelements. For a subset$$E\subseteq \mathbb {F}_q^d$$ E F q d and a fixed nonzero$$t\in \mathbb {F}_q$$ t F q , let$$\mathcal {H}_t(E)=\{h_y: y\in E\}$$ H t ( E ) = { h y : y E } , where$$h_y:E\rightarrow \{0,1\}$$ h y : E { 0 , 1 } is the indicator function of the set$$\{x\in E: x\cdot y=t\}$$ { x E : x · y = t } . Two of the authors, with Maxwell Sun, showed in the case$$d=3$$ d = 3 that if$$|E|\ge Cq^{\frac{11}{4}}$$ | E | C q 11 4 andqis sufficiently large, then the VC-dimension of$$\mathcal {H}_t(E)$$ H t ( E ) is 3. In this paper, we generalize the result to arbitrary dimension by showing that the VC-dimension of$$\mathcal {H}_t(E)$$ H t ( E ) isdwhenever$$E\subseteq \mathbb {F}_q^d$$ E F q d with$$|E|\ge C_d q^{d-\frac{1}{d-1}}$$ | E | C d q d - 1 d - 1
    more » « less
  3. Abstract The minimum linear ordering problem (MLOP) generalizes well-known combinatorial optimization problems such as minimum linear arrangement and minimum sum set cover. MLOP seeks to minimize an aggregated cost$$f(\cdot )$$ f ( · ) due to an ordering$$\sigma $$ σ of the items (say [n]), i.e.,$$\min _{\sigma } \sum _{i\in [n]} f(E_{i,\sigma })$$ min σ i [ n ] f ( E i , σ ) , where$$E_{i,\sigma }$$ E i , σ is the set of items mapped by$$\sigma $$ σ to indices [i]. Despite an extensive literature on MLOP variants and approximations for these, it was unclear whether the graphic matroid MLOP was NP-hard. We settle this question through non-trivial reductions from mininimum latency vertex cover and minimum sum vertex cover problems. We further propose a new combinatorial algorithm for approximating monotone submodular MLOP, using the theory of principal partitions. This is in contrast to the rounding algorithm by Iwata et al. (in: APPROX, 2012), using Lovász extension of submodular functions. We show a$$(2-\frac{1+\ell _{f}}{1+|E|})$$ ( 2 - 1 + f 1 + | E | ) -approximation for monotone submodular MLOP where$$\ell _{f}=\frac{f(E)}{\max _{x\in E}f(\{x\})}$$ f = f ( E ) max x E f ( { x } ) satisfies$$1 \le \ell _f \le |E|$$ 1 f | E | . Our theory provides new approximation bounds for special cases of the problem, in particular a$$(2-\frac{1+r(E)}{1+|E|})$$ ( 2 - 1 + r ( E ) 1 + | E | ) -approximation for the matroid MLOP, where$$f = r$$ f = r is the rank function of a matroid. We further show that minimum latency vertex cover is$$\frac{4}{3}$$ 4 3 -approximable, by which we also lower bound the integrality gap of its natural LP relaxation, which might be of independent interest. 
    more » « less
  4. Abstract Given a prime powerqand$$n \gg 1$$ n 1 , we prove that every integer in a large subinterval of the Hasse–Weil interval$$[(\sqrt{q}-1)^{2n},(\sqrt{q}+1)^{2n}]$$ [ ( q - 1 ) 2 n , ( q + 1 ) 2 n ] is$$\#A({\mathbb {F}}_q)$$ # A ( F q ) for some ordinary geometrically simple principally polarized abelian varietyAof dimensionnover$${\mathbb {F}}_q$$ F q . As a consequence, we generalize a result of Howe and Kedlaya for$${\mathbb {F}}_2$$ F 2 to show that for each prime powerq, every sufficiently large positive integer is realizable, i.e.,$$\#A({\mathbb {F}}_q)$$ # A ( F q ) for some abelian varietyAover$${\mathbb {F}}_q$$ F q . Our result also improves upon the best known constructions of sequences of simple abelian varieties with point counts towards the extremes of the Hasse–Weil interval. A separate argument determines, for fixedn, the largest subinterval of the Hasse–Weil interval consisting of realizable integers, asymptotically as$$q \rightarrow \infty $$ q ; this gives an asymptotically optimal improvement of a 1998 theorem of DiPippo and Howe. Our methods are effective: We prove that if$$q \le 5$$ q 5 , then every positive integer is realizable, and for arbitraryq, every positive integer$$\ge q^{3 \sqrt{q} \log q}$$ q 3 q log q is realizable. 
    more » « less
  5. Abstract We introduce a family of Finsler metrics, called the$$L^p$$ L p -Fisher–Rao metrics$$F_p$$ F p , for$$p\in (1,\infty )$$ p ( 1 , ) , which generalizes the classical Fisher–Rao metric$$F_2$$ F 2 , both on the space of densities$${\text {Dens}}_+(M)$$ Dens + ( M ) and probability densities$${\text {Prob}}(M)$$ Prob ( M ) . We then study their relations to the Amari–C̆encov$$\alpha $$ α -connections$$\nabla ^{(\alpha )}$$ ( α ) from information geometry: on$${\text {Dens}}_+(M)$$ Dens + ( M ) , the geodesic equations of$$F_p$$ F p and$$\nabla ^{(\alpha )}$$ ( α ) coincide, for$$p = 2/(1-\alpha )$$ p = 2 / ( 1 - α ) . Both are pullbacks of canonical constructions on$$L^p(M)$$ L p ( M ) , in which geodesics are simply straight lines. In particular, this gives a new variational interpretation of$$\alpha $$ α -geodesics as being energy minimizing curves. On$${\text {Prob}}(M)$$ Prob ( M ) , the$$F_p$$ F p and$$\nabla ^{(\alpha )}$$ ( α ) geodesics can still be thought as pullbacks of natural operations on the unit sphere in$$L^p(M)$$ L p ( M ) , but in this case they no longer coincide unless$$p=2$$ p = 2 . Using this transformation, we solve the geodesic equation of the$$\alpha $$ α -connection by showing that the geodesic are pullbacks of projections of straight lines onto the unit sphere, and they always cease to exists after finite time when they leave the positive part of the sphere. This unveils the geometric structure of solutions to the generalized Proudman–Johnson equations, and generalizes them to higher dimensions. In addition, we calculate the associate tensors of$$F_p$$ F p , and study their relation to$$\nabla ^{(\alpha )}$$ ( α )
    more » « less