skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: From the Lab to the Wild: Examining Generalizability of Video-based Mind Wandering Detection
Abstract Student’s shift of attention away from a current learning task to task-unrelated thought, also called mind wandering, occurs about 30% of the time spent on education-related activities. Its frequent occurrence has a negative effect on learning outcomes across learning tasks. Automated detection of mind wandering might offer an opportunity to assess the attentional state continuously and non-intrusively over time and hence enable large-scale research on learning materials and responding to inattention with targeted interventions. To achieve this, an accessible detection approach that performs well for various systems and settings is required. In this work, we explore a new, generalizable approach to video-based mind wandering detection that can be transferred to naturalistic settings across learning tasks. Therefore, we leverage two datasets, consisting of facial videos during reading in the lab (N = 135) and lecture viewing in-the-wild (N = 15). When predicting mind wandering, deep neural networks (DNN) and long short-term memory networks (LSTMs) achieve F$$_{1}$$ 1 scores of 0.44 (AUC-PR = 0.40) and 0.459 (AUC-PR = 0.39), above chance level, with latent features based on transfer-learning on the lab data. When exploring generalizability by training on the lab dataset and predicting on the in-the-wild dataset, BiLSTMs on latent features perform comparably to the state-of-the-art with an F$$_{1}$$ 1 score of 0.352 (AUC-PR = 0.26). Moreover, we investigate the fairness of predictive models across gender and show based on post-hoc explainability methods that employed latent features mainly encode information on eye and mouth areas. We discuss the benefits of generalizability and possible applications.  more » « less
Award ID(s):
1920510 2019805
PAR ID:
10515393
Author(s) / Creator(s):
; ; ; ; ; ; ;
Publisher / Repository:
Springer Science + Business Media
Date Published:
Journal Name:
International Journal of Artificial Intelligence in Education
Volume:
35
Issue:
2
ISSN:
1560-4292
Format(s):
Medium: X Size: p. 823-857
Size(s):
p. 823-857
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract We construct an example of a group$$G = \mathbb {Z}^2 \times G_0$$ G = Z 2 × G 0 for a finite abelian group $$G_0$$ G 0 , a subsetEof $$G_0$$ G 0 , and two finite subsets$$F_1,F_2$$ F 1 , F 2 of G, such that it is undecidable in ZFC whether$$\mathbb {Z}^2\times E$$ Z 2 × E can be tiled by translations of$$F_1,F_2$$ F 1 , F 2 . In particular, this implies that this tiling problem isaperiodic, in the sense that (in the standard universe of ZFC) there exist translational tilings ofEby the tiles$$F_1,F_2$$ F 1 , F 2 , but no periodic tilings. Previously, such aperiodic or undecidable translational tilings were only constructed for sets of eleven or more tiles (mostly in $$\mathbb {Z}^2$$ Z 2 ). A similar construction also applies for$$G=\mathbb {Z}^d$$ G = Z d for sufficiently large d. If one allows the group$$G_0$$ G 0 to be non-abelian, a variant of the construction produces an undecidable translational tiling with only one tile F. The argument proceeds by first observing that a single tiling equation is able to encode an arbitrary system of tiling equations, which in turn can encode an arbitrary system of certain functional equations once one has two or more tiles. In particular, one can use two tiles to encode tiling problems for an arbitrary number of tiles. 
    more » « less
  2. Abstract The minimum linear ordering problem (MLOP) generalizes well-known combinatorial optimization problems such as minimum linear arrangement and minimum sum set cover. MLOP seeks to minimize an aggregated cost$$f(\cdot )$$ f ( · ) due to an ordering$$\sigma $$ σ of the items (say [n]), i.e.,$$\min _{\sigma } \sum _{i\in [n]} f(E_{i,\sigma })$$ min σ i [ n ] f ( E i , σ ) , where$$E_{i,\sigma }$$ E i , σ is the set of items mapped by$$\sigma $$ σ to indices [i]. Despite an extensive literature on MLOP variants and approximations for these, it was unclear whether the graphic matroid MLOP was NP-hard. We settle this question through non-trivial reductions from mininimum latency vertex cover and minimum sum vertex cover problems. We further propose a new combinatorial algorithm for approximating monotone submodular MLOP, using the theory of principal partitions. This is in contrast to the rounding algorithm by Iwata et al. (in: APPROX, 2012), using Lovász extension of submodular functions. We show a$$(2-\frac{1+\ell _{f}}{1+|E|})$$ ( 2 - 1 + f 1 + | E | ) -approximation for monotone submodular MLOP where$$\ell _{f}=\frac{f(E)}{\max _{x\in E}f(\{x\})}$$ f = f ( E ) max x E f ( { x } ) satisfies$$1 \le \ell _f \le |E|$$ 1 f | E | . Our theory provides new approximation bounds for special cases of the problem, in particular a$$(2-\frac{1+r(E)}{1+|E|})$$ ( 2 - 1 + r ( E ) 1 + | E | ) -approximation for the matroid MLOP, where$$f = r$$ f = r is the rank function of a matroid. We further show that minimum latency vertex cover is$$\frac{4}{3}$$ 4 3 -approximable, by which we also lower bound the integrality gap of its natural LP relaxation, which might be of independent interest. 
    more » « less
  3. Abstract Let$$\mathbb {F}_q^d$$ F q d be thed-dimensional vector space over the finite field withqelements. For a subset$$E\subseteq \mathbb {F}_q^d$$ E F q d and a fixed nonzero$$t\in \mathbb {F}_q$$ t F q , let$$\mathcal {H}_t(E)=\{h_y: y\in E\}$$ H t ( E ) = { h y : y E } , where$$h_y:E\rightarrow \{0,1\}$$ h y : E { 0 , 1 } is the indicator function of the set$$\{x\in E: x\cdot y=t\}$$ { x E : x · y = t } . Two of the authors, with Maxwell Sun, showed in the case$$d=3$$ d = 3 that if$$|E|\ge Cq^{\frac{11}{4}}$$ | E | C q 11 4 andqis sufficiently large, then the VC-dimension of$$\mathcal {H}_t(E)$$ H t ( E ) is 3. In this paper, we generalize the result to arbitrary dimension by showing that the VC-dimension of$$\mathcal {H}_t(E)$$ H t ( E ) isdwhenever$$E\subseteq \mathbb {F}_q^d$$ E F q d with$$|E|\ge C_d q^{d-\frac{1}{d-1}}$$ | E | C d q d - 1 d - 1
    more » « less
  4. Abstract Given a prime powerqand$$n \gg 1$$ n 1 , we prove that every integer in a large subinterval of the Hasse–Weil interval$$[(\sqrt{q}-1)^{2n},(\sqrt{q}+1)^{2n}]$$ [ ( q - 1 ) 2 n , ( q + 1 ) 2 n ] is$$\#A({\mathbb {F}}_q)$$ # A ( F q ) for some ordinary geometrically simple principally polarized abelian varietyAof dimensionnover$${\mathbb {F}}_q$$ F q . As a consequence, we generalize a result of Howe and Kedlaya for$${\mathbb {F}}_2$$ F 2 to show that for each prime powerq, every sufficiently large positive integer is realizable, i.e.,$$\#A({\mathbb {F}}_q)$$ # A ( F q ) for some abelian varietyAover$${\mathbb {F}}_q$$ F q . Our result also improves upon the best known constructions of sequences of simple abelian varieties with point counts towards the extremes of the Hasse–Weil interval. A separate argument determines, for fixedn, the largest subinterval of the Hasse–Weil interval consisting of realizable integers, asymptotically as$$q \rightarrow \infty $$ q ; this gives an asymptotically optimal improvement of a 1998 theorem of DiPippo and Howe. Our methods are effective: We prove that if$$q \le 5$$ q 5 , then every positive integer is realizable, and for arbitraryq, every positive integer$$\ge q^{3 \sqrt{q} \log q}$$ q 3 q log q is realizable. 
    more » « less
  5. Abstract Let us fix a primepand a homogeneous system ofmlinear equations$$a_{j,1}x_1+\dots +a_{j,k}x_k=0$$ a j , 1 x 1 + + a j , k x k = 0 for$$j=1,\dots ,m$$ j = 1 , , m with coefficients$$a_{j,i}\in \mathbb {F}_p$$ a j , i F p . Suppose that$$k\ge 3m$$ k 3 m , that$$a_{j,1}+\dots +a_{j,k}=0$$ a j , 1 + + a j , k = 0 for$$j=1,\dots ,m$$ j = 1 , , m and that every$$m\times m$$ m × m minor of the$$m\times k$$ m × k matrix$$(a_{j,i})_{j,i}$$ ( a j , i ) j , i is non-singular. Then we prove that for any (large)n, any subset$$A\subseteq \mathbb {F}_p^n$$ A F p n of size$$|A|> C\cdot \Gamma ^n$$ | A | > C · Γ n contains a solution$$(x_1,\dots ,x_k)\in A^k$$ ( x 1 , , x k ) A k to the given system of equations such that the vectors$$x_1,\dots ,x_k\in A$$ x 1 , , x k A are all distinct. Here,Cand$$\Gamma $$ Γ are constants only depending onp,mandksuch that$$\Gamma Γ < p . The crucial point here is the condition for the vectors$$x_1,\dots ,x_k$$ x 1 , , x k in the solution$$(x_1,\dots ,x_k)\in A^k$$ ( x 1 , , x k ) A k to be distinct. If we relax this condition and only demand that$$x_1,\dots ,x_k$$ x 1 , , x k are not all equal, then the statement would follow easily from Tao’s slice rank polynomial method. However, handling the distinctness condition is much harder, and requires a new approach. While all previous combinatorial applications of the slice rank polynomial method have relied on the slice rank of diagonal tensors, we use a slice rank argument for a non-diagonal tensor in combination with combinatorial and probabilistic arguments. 
    more » « less