Software engineering candidates commonly participate in whiteboard technical interviews as part of a hiring assessment. During these sessions, candidates write code while thinking aloud as they work towards a solution, under the watchful eye of an interviewer. While technical interviews should allow for an unbiased and inclusive assessment of problem-solving ability, surprisingly, technical interviews may be instead a procedure for identifying candidates who best handle and migrate stress solely caused by being examined by an interviewer (performance anxiety).
To understand if coding interviews---as administered today---can induce stress that significantly hinders performance, we conducted a randomized controlled trial with 48 Computer Science students, comparing them in private and public whiteboard settings. We found that performance is reduced by more than half, by simply being watched by an interviewer. We also observed that stress and cognitive load were significantly higher in a traditional technical interview when compared with our private interview. Consequently, interviewers may be filtering out qualified candidates by confounding assessment of problem-solving ability with unnecessary stress. We propose interview modifications to make problem-solving assessment more equitable and inclusive, such as through private focus sessions and retrospective think-aloud, allowing companies to hire from a larger and diverse pool of talent.
more »
« less
Can we predict stressful technical interview settings through eye-tracking?
Recently, eye-tracking analysis for finding the cognitive load and stress while problem-solving on the whiteboard during a technical interview is finding its way in software engineering society. However, there is no empirical study on analyzing how much the interview setting characteristics affect the eye-movement measurements. Without knowing that, the results of a research on eye-movement measurements analysis for stress detection will not be reliable. In this paper, we analyzed the eye-movements of 11 participants in two interview settings, one on the whiteboard and the other on the paper, to find out if the characteristics of the interview settings affect the analysis of participants' stress. To this end, we applied 7 Machine Learning classification algorithms on three different labeling strategies of the data to suggest researchers of the domain a useful practice of checking the reliability of the eye-measurements before reporting any results.
more »
« less
- Award ID(s):
- 1755762
- NSF-PAR ID:
- 10085744
- Date Published:
- Journal Name:
- Proceedings of the Workshop on Eye Movements in Programming (EMIP'18)
- Page Range / eLocation ID:
- 1 to 5
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Background With nearly 20% of the US adult population using fitness trackers, there is an increasing focus on how physiological data from these devices can provide actionable insights about workplace performance. However, in-the-wild studies that understand how these metrics correlate with cognitive performance measures across a diverse population are lacking, and claims made by device manufacturers are vague. While there has been extensive research leading to a variety of theories on how physiological measures affect cognitive performance, virtually all such studies have been conducted in highly controlled settings and their validity in the real world is poorly understood. Objective We seek to bridge this gap by evaluating prevailing theories on the effects of a variety of sleep, activity, and heart rate parameters on cognitive performance against data collected in real-world settings. Methods We used a Fitbit Charge 3 and a smartphone app to collect different physiological and neurobehavioral task data, respectively, as part of our 6-week-long in-the-wild study. We collected data from 24 participants across multiple population groups (shift workers, regular workers, and graduate students) on different performance measures (vigilant attention and cognitive throughput). Simultaneously, we used a fitness tracker to unobtrusively obtain physiological measures that could influence these performance measures, including over 900 nights of sleep and over 1 million minutes of heart rate and physical activity metrics. We performed a repeated measures correlation (rrm) analysis to investigate which sleep and physiological markers show association with each performance measure. We also report how our findings relate to existing theories and previous observations from controlled studies. Results Daytime alertness was found to be significantly correlated with total sleep duration on the previous night (rrm=0.17, P<.001) as well as the duration of rapid eye movement (rrm=0.12, P<.001) and light sleep (rrm=0.15, P<.001). Cognitive throughput, by contrast, was not found to be significantly correlated with sleep duration but with sleep timing—a circadian phase shift toward a later sleep time corresponded with lower cognitive throughput on the following day (rrm=–0.13, P<.001). Both measures show circadian variations, but only alertness showed a decline (rrm=–0.1, P<.001) as a result of homeostatic pressure. Both heart rate and physical activity correlate positively with alertness as well as cognitive throughput. Conclusions Our findings reveal that there are significant differences in terms of which sleep-related physiological metrics influence each of the 2 performance measures. This makes the case for more targeted in-the-wild studies investigating how physiological measures from self-tracking data influence, or can be used to predict, specific aspects of cognitive performance.more » « less
-
more » « less
Abstract Sleep and stress independently enhance emotional memory consolidation. In particular, theta oscillations (4–7 Hz) during rapid eye movement (REM) sleep increase coherence in an emotional memory network (i.e., hippocampus, amygdala, and prefrontal cortex) and enhance emotional memory. However, little is known about how stress during learning might
interact with subsequent REM theta activity to affect emotional memory. In the current study, we examined whether the relationship between REM theta activity and emotional memory differs as a function of pre‐encoding stress exposure and reactivity. Participants underwent a psychosocial stressor (the Trier Social Stress Task;n = 32) or a comparable control task (n = 32) prior to encoding. Task‐evoked cortisol reactivity was assessed by salivary cortisol rise from pre‐ to post‐stressor, and participants in the stress condition were additionally categorized as high or low cortisol responders via a median split. During incidental encoding, participants studied 150 line drawings of negative, neutral, and positive images, followed by the complete color photo. All participants then slept overnight in the lab with polysomnographic recording. The next day, they were given a surprise recognition memory task. Results showed that memory was better for emotional relative to neutral information. Critically, these findings were observed only in the stress condition. No emotional memory benefit was observed in the control condition. In stressed participants, REM theta power significantly predicted memory for emotional information, specifically for positive items. This relationship was observed only in high cortisol responders. For low responders and controls, there was no relationship between REM theta and memory of any valence. These findings provide evidence that elevated stress at encoding, and accompanying changes in neuromodulators such as cortisol, may interact with theta activity during REM sleep to promote selective consolidation of emotional information. -
An ongoing focus of engineering education research is on increasing the number of women in engineering. Previous studies have primarily focused on examining why the number of women enrolled in engineering colleges remains persistent low. In doing so, while we have gained better understanding of the challenges and barriers women encountered and factors that contribute to such negative experiences, it also, as some scholars have pointed out, has cast a deficit frame to such matters. In this study, we take on a positive stand where we focus on women undergraduate students who not only “stay” but also succeed in engineering programs (that is, our definition of thriving) as a way to locate the personal and institutional factors that facilitate such positive outcomes. Our initial pilot study involved two female engineering undergraduate students at an R1 university. Each student was interviewed three times. While each of the interviews in the sequence had slightly different focus, the overall goal was to understand the women’s autobiographic and educational experiences leading to their paths to engineering and participation in the engineering project teams. The inductive thematic analysis revealed several primary findings which subsequently played a major role in developing a codebook for the current study. Building upon what is learned from the pilot study, the current study uses a layered multi-case study design involving three institutions: a public/private Ivy League and statutory land-grand research university in the Northeast, a public land-grant research university in the Midwest, and a public land-grant research university in the Southwest which is also designated as MSI/HSI. In addition to the interview method, data collection also contains documents and artifacts. For the purpose of this paper, we zone in onto the data collected in the first interviews, known as the “life history” where we mainly learn about the women undergraduate participants’ personal-familial contexts that contribute their entry to majoring in engineering as identified by the women themselves. Preliminary findings indicate that: (1) our participants tend to have supportive families; (2) while all experienced gender biases, not everyone has formed a critical consciousness of sexism; and (3) being able to actually engage “doing” something and creating a product is key to the women’s finding joy in engineering and associating self with the field/profession. It is important to note that the second interviews are underway which focuses on the educational journey of the participants in relation to engineering identity development and project team experiences. The ultimate goal for the study is to develop a theoretical framework speaking to a multifaceted model of forces (micro as autobiographic, macro as institutional, and in-between or middle-level as team-based) in shaping women’s entry and advance in engineering programs – one that recognizes the variations in institutional type, resource availability, and structural and cultural characteristics and traditions in teams, but uses such differences to show possibilities of more versatile ways for diversifying pathways for women and other minoritized groups to thrive in engineering.more » « less
-
An ongoing focus of engineering education research is on increasing the number of women in engineering. Previous studies have primarily focused on examining why the number of women enrolled in engineering colleges remains persistently low. In doing so, while we have gained better understanding of the challenges and barriers that women encountered and factors that contribute to such negative experiences, it also, as some scholars have pointed out, has cast a deficit frame on such matters. In this study, we take on a positive stand where we focus on women undergraduate students who not only “stay” but also succeed in engineering programs (that is, our definition of thriving) as a way to locate the personal and institutional factors that facilitate such positive outcomes. Our initial pilot study involved two female engineering undergraduate students at an R1 university. Each student was interviewed three times. While each of the interviews in the sequence had a slightly different focus, the overall goal was to understand the women’s autobiographic and educational experiences leading to their paths to engineering and participation in the engineering project teams. The inductive thematic analysis revealed several primary findings which subsequently played a major role in developing a codebook for the current study. Building upon what is learned from the pilot study, the current study uses a layered multi-case study design involving three institutions: a public/private Ivy League and statutory land-grand research university in the Northeast, a public land-grant research university in the Midwest, and a public land-grant research university in the Southwest which is also designated as MSI/HSI. In addition to the interview method, data collection also contains documents and artifacts. For this paper, we zone in onto the data collected in the first interviews, known as the “life history” where we mainly learn about the women undergraduate participants’ personal-familial contexts that contribute to their entry to majoring in engineering as identified by the women themselves. Preliminary findings indicate that: (1) our participants tend to have supportive families; (2) while all experienced gender biases, not everyone has formed a critical consciousness of sexism; and (3) being able to actually engage by “doing” something and creating a product is key to the women’s finding joy in engineering and associating themself with the field/profession. It is important to note that the second interviews, which focus on the educational journey of the participants in relation to engineering identity development and project team experiences, are underway. The ultimate goal for the study is to develop a theoretical framework speaking to a multifaceted model of forces (micro as autobiographic, macro as institutional, and in-between or middle-level as team-based) in shaping women’s entry and advance in engineering programs. This framework will recognize the variations in institutional type, resource availability, and structural and cultural characteristics and traditions in teams. It will also use such differences to show possibilities of more versatile ways for diversifying pathways for women and other minoritized groups to thrive in engineering.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3216723.3216729
