The PoseASL dataset consists of color and depth videos collected from ASL signers at the Linguistic and Assistive Technologies Laboratory under the direction of Matt Huenerfauth, as part of a collaborative research project with researchers at the Rochester Institute of Technology, Boston University, and the University of Pennsylvania. Access: After becoming an authorized user of Databrary, please contact Matt Huenerfauth if you have difficulty accessing this volume. We have collected a new dataset consisting of color and depth videos of fluent American Sign Language signers performing sequences ASL signs and sentences. Given interest among sign-recognition and other computer-vision researchers in red-green-blue-depth (RBGD) video, we release this dataset for use by the research community. In addition to the video files, we share depth data files from a Kinect v2 sensor, as well as additional motion-tracking files produced through post-processing of this data. Organization of the Dataset: The dataset is organized into sub-folders, with codenames such as "P01" or "P16" etc. These codenames refer to specific human signers who were recorded in this dataset. Please note that there was no participant P11 nor P14; those numbers were accidentally skipped during the process of making appointments to collect video stimuli. Task: During the recording session, the participant was met by a member of our research team who was a native ASL signer. No other individuals were present during the data collection session. After signing the informed consent and video release document, participants responded to a demographic questionnaire. Next, the data-collection session consisted of English word stimuli and cartoon videos. The recording session began with showing participants stimuli consisting of slides that displayed English word and photos of items, and participants were asked to produce the sign for each (PDF included in materials subfolder). Next, participants viewed three videos of short animated cartoons, which they were asked to recount in ASL: - Canary Row, Warner Brothers Merrie Melodies 1950 (the 7-minute video divided into seven parts) - Mr. Koumal Flies Like a Bird, Studio Animovaneho Filmu 1969 - Mr. Koumal Battles his Conscience, Studio Animovaneho Filmu 1971 The word list and cartoons were selected as they are identical to the stimuli used in the collection of the Nicaraguan Sign Language video corpora - see: Senghas, A. (1995). Children’s Contribution to the Birth of Nicaraguan Sign Language. Doctoral dissertation, Department of Brain and Cognitive Sciences, MIT. Demographics: All 14 of our participants were fluent ASL signers. As screening, we asked our participants: Did you use ASL at home growing up, or did you attend a school as a very young child where you used ASL? All the participants responded affirmatively to this question. A total of 14 DHH participants were recruited on the Rochester Institute of Technology campus. Participants included 7 men and 7 women, aged 21 to 35 (median = 23.5). All of our participants reported that they began using ASL when they were 5 years old or younger, with 8 reporting ASL use since birth, and 3 others reporting ASL use since age 18 months. Filetypes: *.avi, *_dep.bin: The PoseASL dataset has been captured by using a Kinect 2.0 RGBD camera. The output of this camera system includes multiple channels which include RGB, depth, skeleton joints (25 joints for every video frame), and HD face (1,347 points). The video resolution produced in 1920 x 1080 pixels for the RGB channel and 512 x 424 pixels for the depth channels respectively. Due to limitations in the acceptable filetypes for sharing on Databrary, it was not permitted to share binary *_dep.bin files directly produced by the Kinect v2 camera system on the Databrary platform. If your research requires the original binary *_dep.bin files, then please contact Matt Huenerfauth. *_face.txt, *_HDface.txt, *_skl.txt: To make it easier for future researchers to make use of this dataset, we have also performed some post-processing of the Kinect data. To extract the skeleton coordinates of the RGB videos, we used the Openpose system, which is capable of detecting body, hand, facial, and foot keypoints of multiple people on single images in real time. The output of Openpose includes estimation of 70 keypoints for the face including eyes, eyebrows, nose, mouth and face contour. The software also estimates 21 keypoints for each of the hands (Simon et al, 2017), including 3 keypoints for each finger, as shown in Figure 2. Additionally, there are 25 keypoints estimated for the body pose (and feet) (Cao et al, 2017; Wei et al, 2016). Reporting Bugs or Errors: Please contact Matt Huenerfauth to report any bugs or errors that you identify in the corpus. We appreciate your help in improving the quality of the corpus over time by identifying any errors. Acknowledgement: This material is based upon work supported by the National Science Foundation under award 1749376: "Collaborative Research: Multimethod Investigation of Articulatory and Perceptual Constraints on Natural Language Evolution."
more »
« less
Brief Interventions Influence the Quantity and Quality of Caregiver-Child Conversations in an Everyday Context
Reading and arithmetic are difficult cognitive feats for children to master and youth from low-income communities are often less “school ready” in terms of letter and number recognition skills ( Lee and Burkam, 2002 ). One way to prepare children for school is by encouraging caregivers to engage children in conversations about academically-relevant concepts by using numbers, recognizing shapes, and naming colors ( Levine et al., 2010 ; Fisher et al., 2013 ). Previous research shows that caregiver-child conversations about these topics rarely take place in everyday contexts ( Hassinger-Das et al., 2018 ), but interventions designed to encourage such conversations, like displaying signs in a grocery store, have resulted in significant increases in caregiver-child conversations ( Ridge et al., 2015 ; Hanner et al., 2019 ). We investigated whether a similar brief intervention could change caregiver-child conversations in an everyday context. We observed 212 families in a volunteer-run facility where people who are food-insecure can select food from available donations. Volunteers greet all the clients as they pass through the aisles, offer food, and restock the shelves as needed. About 25% of the clients have children with them and our data consist of observations of the caregiver-child conversations with 2- to 10-year-old children. Half of the observation days consisted of a baseline condition in which the quantity and quality of caregiver-child conversation was observed as the client went through aisles where no signs were displayed, and volunteers merely greeted the clients. The other half of the observation days consisted of a brief intervention where signs were displayed (signs-up condition), where, volunteers greeted the clients and pointed out that there were signs displayed to entertain the children if they were interested. In addition, there was a within-subject manipulation for the intervention condition where each family interacted with two different categories of signs. Half of the signs had academically-relevant content and the other half had non-academically-relevant content. The results demonstrate that the brief intervention used in the signs-up condition increases the quantity of conversation between a caregiver and child. In addition, signs with academically-relevant content increases the quality of the conversation. These findings provide further evidence that brief interventions in an everyday context can change the caregiver-child conversation. Specifically, signs with academically-relevant content may promote school readiness.
more »
« less
- Award ID(s):
- 1729720
- NSF-PAR ID:
- 10278355
- Date Published:
- Journal Name:
- Frontiers in Psychology
- Volume:
- 12
- ISSN:
- 1664-1078
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
ABSTRACT Young children have better math abilities when their parents engage in more math‐related conversations with them. Yet, previous studies have found that math talk occurs only very infrequently in everyday interactions. In the present study, we sought to promote adult–child conversations about math in a naturalistic context using minimal instructions. We observed 179 adult–child dyads while they shopped in grocery stores with signs prompting them to engage in math‐related conversations (math condition), signs prompting them to talk about other topics (general language condition), or without any signs (baseline condition). In the math condition, more adults talked about math compared to the general language or the baseline condition, and this finding could not be explained by demographic characteristics of the dyad or the overall amount of conversations. This study demonstrates that cost‐effective signs placed in everyday contexts can promote math‐related conversations and potentially provide math learning opportunities for children.
-
Digital Behavioral Phenotyping Detects Atypical Pattern of Facial Expression in Toddlers with Autismmore » « less
Commonly used screening tools for autism spectrum disorder (ASD) generally rely on subjective caregiver questionnaires. While behavioral observation is more objective, it is also expensive, time‐consuming, and requires significant expertise to perform. As such, there remains a critical need to develop feasible, scalable, and reliable tools that can characterize ASD risk behaviors. This study assessed the utility of a tablet‐based behavioral assessment for eliciting and detecting one type of risk behavior, namely, patterns of facial expression, in 104 toddlers (ASD
N = 22) and evaluated whether such patterns differentiated toddlers with and without ASD. The assessment consisted of the child sitting on his/her caregiver's lap and watching brief movies shown on a smart tablet while the embedded camera recorded the child's facial expressions. Computer vision analysis (CVA) automatically detected and tracked facial landmarks, which were used to estimate head position and facial expressions (Positive, Neutral, All Other). Using CVA, specific points throughout the movies were identified that reliably differentiate between children with and without ASD based on their patterns of facial movement and expressions (area under the curves for individual movies ranging from 0.62 to 0.73). During these instances, children with ASD more frequently displayed Neutral expressions compared to children without ASD, who had more All Other expressions. The frequency of All Other expressions was driven by non‐ASD children more often displaying raised eyebrows and an open mouth, characteristic of engagement/interest. Preliminary results suggest computational coding of facial movements and expressions via a tablet‐based assessment can detect differences in affective expression, one of the early, core features of ASD.Lay Summary This study tested the use of a tablet in the behavioral assessment of young children with autism. Children watched a series of developmentally appropriate movies and their facial expressions were recorded using the camera embedded in the tablet. Results suggest that computational assessments of facial expressions may be useful in early detection of symptoms of autism.
-
The culture within engineering colleges and departments has been historically quiet when considering social justice issues. Often the faculty in those departments are less concerned with social issues and are primarily focused on their disciplines and the concrete ways that they can make impacts academically and professionally in their respective arena’s. However, with the social climate of the United States shifting ever more towards a politically charged climate, and current events, particularly the protests against police brutality in recent years, faculty and students are constantly inundated with news of injustices happening in our society. The murder of George Floyd on May 25th 2020 sent shockwaves across the United States and the world. The video captured of his death shared across the globe brought everyone’s attention to the glaringly ugly problem of police brutality, paired with the COVID-19 pandemic, and US election year, the conditions were just right for a social activist movement to grow to a size that no one could ignore. Emmanuel Acho spoke out, motivated by injustices seen in the George Floyd murder, initially with podcasts and then by writing his book “Uncomfortable Converstations with a Black Man” [1]. In his book he touched on various social justice issues such as: racial terminology (i.e., Black or African American), implicit biases, white privilege, cultural appropriation, stereotypes (e.g., the “angry black man”), racial slurs (particularly the n-word), systemic racism, the myth of reverse racism, the criminal justice system, the struggles faced by black families, interracial families, allyship, and anti-racism. Students and faculty at Anonymous University felt compelled to set aside the time to meet and discuss this book in depth through the video conferencing client Zoom. In these meetings diverse facilitators were tasked with bringing the topics discussed by Acho in his book into conversation and pushing attendees of these meetings to consider those topics critically and personally. In an effort to avoid tasking attendees with reading homework to be able to participate in these discussions, the discussed chapter of the audiobook version of Acho’s book was played at the beginning of each meeting. Each audiobook chapter lasted between fifteen and twenty minutes, after which forty to forty-five minutes were left in the hour-long meetings to discuss the content of the chapter in question. Efforts by students and faculty were made to examine how some of the teachings of the book could be implemented into their lives and at Anonymous University. For broader topics, they would relate the content back to their personal lives (e.g., raising their children to be anti-racist and their experiences with racism in American and international cultures). Each meeting was recorded for posterity in the event that those conversations would be used in a paper such as this. Each meeting had at least one facilitator whose main role was to provide discussion prompts based on the chapter and ensure that the meeting environment was safe and inclusive. Naturally, some chapters address topics that are highly personal to some participants, so it was vital that all participants felt comfortable and supported to share their thoughts and experiences. The facilitator would intervene if the conversation veered in an aggressive direction. For example, if a participant starts an argument with another participant in a non-constructive manner, e.g., arguing over the definition of ethnicity, then the facilitator will interrupt, clear the air to bring the group back to a common ground, and then continue the discussion. Otherwise, participants were allowed to steer the direction of the conversation as new avenues of discussion popped up. These meetings were recorded with the goal of returning to these conversations and analyzing the conversations between attendees. Grounded theory will be used to first assess the most prominent themes of discussion between attendees for each meeting [2]. Attendees will be contacted to expressly ask their permission to have their words and thoughts used in this work, and upon agreement that data will begin to be processed. Select attendees will be asked to participate in focus group discussions, which will also be recorded via Zoom. These discussions will focus around the themes pulled from general discussion and will aim to dive deeper into the impact that this experience has had on them as either students or faculty members. A set of questions will be developed as prompts, but conversation is expected to evolve organically as these focus groups interact. These sessions will be scheduled for an hour, and a set of four focus groups with four participants are expected to participate for a total of sixteen total focus group participants. We hope to uncover how this experience changed the lives of the participants and present a model of how conversations such as this can promote diversity, equity, inclusion, and access activities amongst faculty and students outside of formal programs and strategic plans that are implemented at university, college, or departmental levels.more » « less
-
more » « less
Abstract Objectives The objectives of this work were to evaluate (i) the prevalence and frequency of caregiver‐reported soil ingestion by children, (ii) whether household flooring material in the bedroom (earth
vs . concrete) affected caregiver‐reported soil ingestion, (iii) whether caregiver‐reported soil ingestion was associated with caregiver‐reported diarrhoea and (iv) caregivers’ perceptions of their children ingesting soil.Methods We conducted 309 household surveys in northern Ghana, including 529 children under five (249 children aged 6–36 months), and measured faecal contamination in soil from 31 households.
Results Among all children, 15% were reported to have directly ingested soil in the past week, including 28% of children aged 6–36 months. Among children reported to have ingested soil, the median frequency was 14 times in the past week, and the median amount of soil ingested each time was half a handful. There was no association between household floor material and whether the caregiver observed a child directly ingesting soil. After adjusting for household floor material and other potential confounding variables, caregiver‐reported soil ingestion was associated with caregiver‐reported diarrhoea for children under five [adjusted odds ratio (adj.
OR ) = 3.13, 95% confidence interval (CI ) 2.76–3.55] and children aged 6–36 months (adj.OR = 2.61, 95%CI 2.01–3.39). Approximately 83% of caregivers whose children ingested soil reported they thought it was unsafe and were more likely to report stopping their child from ingesting soil, but these responses did not affect the quantity of soil ingested.Conclusions Our results suggest direct soil ingestion is associated with diarrhoea independent of household floor material, and separate interventions may be necessary to prevent exploratory soil ingestion.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3389/fpsyg.2021.645788
