Sharma, Aakash; Bhasi, Vivek; Singh, Sonali; Jain, Rishabh; Raj, Jashwant; Mitra, Subrata; Kandemir, Mahmut Taylan; Kesidis, George; Das, Chita(
, Proceedings of the International Conference on Distributed Computing Systems)
Deep neural networks (DNNs) are increasingly popular
owing to their ability to solve complex problems such as
image recognition, autonomous driving, and natural language
processing. Their growing complexity coupled with the use of
larger volumes of training data (to achieve acceptable accuracy)
has warranted the use of GPUs and other accelerators. Such
accelerators are typically expensive, with users having to pay a
high upfront cost to acquire them. For infrequent use, users can,
instead, leverage the public cloud to mitigate the high acquisition
cost. However, with the wide diversity of hardware instances
(particularly GPU instances) available in public cloud, it becomes
challenging for a user to make an appropriate choice from a
cost/performance standpoint.
In this work, we try to address this problem by (i) introducing
a comprehensive distributed deep learning (DDL) profiler Stash,
which determines the various execution stalls that DDL suffers
from, and (ii) using Stash to extensively characterize various
public cloud GPU instances by running popular DNN models
on them. Specifically, it estimates two types of communication
stalls, namely, interconnect and network stalls, that play a
dominant role in DDL execution time. Stash is implemented
on top of prior work, DS-analyzer, that computes only the
CPU and disk stalls. Using our detailed stall characterization,
we list the advantages and shortcomings of public cloud GPU
instances for users to help them make an informed decision(s).
Our characterization results indicate that the more expensive
GPU instances may not be the most performant for all DNN
models and that AWS can sometimes sub-optimally allocate
hardware interconnect resources. Specifically, the intra-machine
interconnect can introduce communication overheads of up to
90% of DNN training time and the network-connected instances
can suffer from up to 5× slowdown compared to training on a
single instance. Furthermore, (iii) we also model the impact of
DNN macroscopic features such as the number of layers and the
number of gradients on communication stalls, and finally, (iv)
we briefly discuss a cost comparison with existing work.
Sujatha Ravindran, Akshay; Mobiny, Aryan; Cruz-Garza, Jesus G.; Paek, Andrew; Kopteva, Anastasiya; Contreras Vidal, José L.(
, Journal of Neural Engineering)
Abstract
Objective. Understanding neural activity patterns in the developing brain remains one of the grand challenges in neuroscience. Developing neural networks are likely to be endowed with functionally important variability associated with the environmental context, age, gender, and other variables. Therefore, we conducted experiments with typically developing children in a stimulating museum setting and tested the feasibility of using deep learning techniques to help identify patterns of brain activity associated with different conditions.Approach. A four-channel dry EEG-based Mobile brain-body imaging data of children at rest and during videogame play (VGP) was acquired at the Children’s Museum of Houston. A data-driven approach based on convolutional neural networks (CNN) was used to describe underlying feature representations in the EEG and their ability to discern task and gender. The variability of the spectral features of EEG during the rest condition as a function of age was also analyzed.Main results. Alpha power (7–13 Hz) was higher during rest whereas theta power (4–7 Hz) was higher during VGP. Beta (13–18 Hz) power was the most significant feature, higher in females, when differentiating between males and females. Using data from both temporoparietal channels to classify between VGP and rest condition, leave-one-subject-out cross-validation accuracy of 67% was obtained. Age-related changes in EEG spectral content during rest were consistent with previous developmental studies conducted in laboratory settings showing an inverse relationship between age and EEG power.Significance. These findings are the first to acquire, quantify and explain brain patterns observed during VGP and rest in freely behaving children in a museum setting using a deep learning framework. The study shows how deep learning can be used as a data driven approach to identify patterns in the data and explores the issues and the potential of conducting experiments involving children in a natural and engaging environment.
Darr, Katherine D., East, Jennifer L., Seabrook, Sarah, Dundas, Steven J., and Thurber, Andrew R. The Deep Sea and Me: Using a Science Center Exhibit to Promote Lasting Public Literacy and Elucidate Public Perception of the Deep Sea. Retrieved from https://par.nsf.gov/biblio/10206850. Frontiers in Marine Science 7. Web. doi:10.3389/fmars.2020.00159.
Darr, Katherine D., East, Jennifer L., Seabrook, Sarah, Dundas, Steven J., & Thurber, Andrew R. The Deep Sea and Me: Using a Science Center Exhibit to Promote Lasting Public Literacy and Elucidate Public Perception of the Deep Sea. Frontiers in Marine Science, 7 (). Retrieved from https://par.nsf.gov/biblio/10206850. https://doi.org/10.3389/fmars.2020.00159
Darr, Katherine D., East, Jennifer L., Seabrook, Sarah, Dundas, Steven J., and Thurber, Andrew R.
"The Deep Sea and Me: Using a Science Center Exhibit to Promote Lasting Public Literacy and Elucidate Public Perception of the Deep Sea". Frontiers in Marine Science 7 (). Country unknown/Code not available. https://doi.org/10.3389/fmars.2020.00159.https://par.nsf.gov/biblio/10206850.
@article{osti_10206850,
place = {Country unknown/Code not available},
title = {The Deep Sea and Me: Using a Science Center Exhibit to Promote Lasting Public Literacy and Elucidate Public Perception of the Deep Sea},
url = {https://par.nsf.gov/biblio/10206850},
DOI = {10.3389/fmars.2020.00159},
abstractNote = {},
journal = {Frontiers in Marine Science},
volume = {7},
author = {Darr, Katherine D. and East, Jennifer L. and Seabrook, Sarah and Dundas, Steven J. and Thurber, Andrew R.},
editor = {null}
}
Warning: Leaving National Science Foundation Website
You are now leaving the National Science Foundation website to go to a non-government website.
Website:
NSF takes no responsibility for and exercises no control over the views expressed or the accuracy of
the information contained on this site. Also be aware that NSF's privacy policy does not apply to this site.