Title: Accuracy of Automated Machine Learning Software in Identifying EEGs with Prolonged Seizures
Objective: To demonstrate that combining automatic processing of EEG data using high performance machine learning algorithms with manual review by expert annotators can quickly identify subjects with prolonged seizures. Background: Prolonged seizures are markers of seizure severity, risk of transformation into status epilepticus, and medical morbidity. Early recognition of prolonged seizures permits intervention and reduces morbidity. Design/Methods: We triaged the TUH EEG Corpus, an open source database of EEGs, by running a state-of-the-art hybrid LSTM-based deep learning system. Then, we postprocessed the output to identify high confidence hypotheses for seizures that were greater than three minutes in duration. Results: The triaging method selected 25 subjects for further review. 17 subjects had seizures; only 5 met criteria for seizures greater than 3 minutes. 11 subjects did not have a prior diagnosis of epilepsy. Among these, 63% had acute respiratory failure and 36% had cardiac arrest leading to seizures secondary to anoxic brain injury. 18 (72%) EEGs were obtained in long-term monitoring (LTM), 1 (4%) in the epilepsy monitoring unit (EMU), and 6 (24%) as a routine EEG (rEEG). 72.2% of seizures in LTM were identified correctly versus 66.7% in rEEGs. Of the 9 subjects who were deceased, 7 (78%) had been on LTM. The seizure detection algorithm misidentified seizures in 7 subjects (28%). A total of 22 (88%) subjects had some ictal pattern. Patterns mistaken for seizure activity included muscle artifact, generalized periodic discharges, generalized spike-and-wave, triphasic waves, and interestingly, an EEG recording captured during CPR. Conclusions: This hybrid approach, which combines state-of-the-art machine learning seizure detection software with human annotation, successfully identified prolonged seizures in 72% of subjects; 88% had ictal patterns. Prolonged seizures were more common in LTM subjects than the EMU and were associated with acute cardiac or pulmonary insult. more »« less
Golmohammadi, Meysam; Shah, Vinit; Obeid, Iyad; Picone, Joseph
(, Signal Processing in Medicine and Biology: Emerging Trends in Research and Applications)
Obeid, Iyad; Selesnick, Ivan; Picone, Joseph
(Ed.)
Scalp electroencephalograms (EEGs) are the primary means by which phy-sicians diagnose brain-related illnesses such as epilepsy and seizures. Au-tomated seizure detection using clinical EEGs is a very difficult machine learning problem due to the low fidelity of a scalp EEG signal. Neverthe-less, despite the poor signal quality, clinicians can reliably diagnose ill-nesses from visual inspection of the signal waveform. Commercially avail-able automated seizure detection systems, however, suffer from unaccepta-bly high false alarm rates. Deep learning algorithms that require large amounts of training data have not previously been effective on this task due to the lack of big data resources necessary for building such models and the complexity of the signals involved. The evolution of big data science, most notably the release of the Temple University EEG (TUEG) Corpus, has mo-tivated renewed interest in this problem. In this chapter, we discuss the application of a variety of deep learning ar-chitectures to automated seizure detection. Architectures explored include multilayer perceptrons, convolutional neural networks (CNNs), long short-term memory networks (LSTMs), gated recurrent units and residual neural networks. We use the TUEG Corpus, supplemented with data from Duke University, to evaluate the performance of these hybrid deep structures. Since TUEG contains a significant amount of unlabeled data, we also dis-cuss unsupervised pre-training methods used prior to training these com-plex recurrent networks. Exploiting spatial and temporal context is critical for accurate disambigua-tion of seizures from artifacts. We explore how effectively several conven-tional architectures are able to model context and introduce a hybrid system that integrates CNNs and LSTMs. The primary error modalities observed by this state-of-the-art system were false alarms generated during brief delta range slowing patterns such as intermittent rhythmic delta activity. A varie-ty of these types of events have been observed during inter-ictal and post-ictal stages. Training models on such events with diverse morphologies has the potential to significantly reduce the remaining false alarms. This is one reason we are continuing our efforts to annotate a larger portion of TUEG. Increasing the data set size significantly allows us to leverage more ad-vanced machine learning methodologies.
Nasseri, Mona; Stirling, Rachel E; Viana, Pedro F; Cui, Jie; Nurse, Ewan; Karoly, Philippa J; Kremen, Vaclav; Dümpelmann, Matthias; Worrell, Gregory A; Freestone, Dean R; et al
(, Epilepsia)
Abstract ObjectiveSeizure unpredictability can be debilitating and dangerous for people with epilepsy. Accurate seizure forecasters could improve quality of life for those with epilepsy but must be practical for long‐term use. This study presents the first validation of a seizure‐forecasting system using ultra‐long‐term, non‐invasive wearable data. MethodsEleven participants with epilepsy were recruited for continuous monitoring, capturing heart rate and step count via wrist‐worn devices and seizures via electroencephalography (average recording duration of 337 days). Two hybrid models—combining machine learning and cycle‐based methods—were proposed to forecast seizures at both short (minutes) and long (up to 44 days) horizons. ResultsThe Seizure Warning System (SWS), designed for forecasting near‐term seizures, and the Seizure Risk System (SRS), designed for forecasting long‐term risk, both outperformed traditional models. In addition, the SRS reduced high‐risk time by 29% while increasing sensitivity by 11%. SignificanceThese improvements mark a significant advancement in making seizure forecasting more practical and effective.
Gregg, Nicholas M.; Pal Attia, Tal; Nasseri, Mona; Joseph, Boney; Karoly, Philippa; Cui, Jie; Stirling, Rachel E.; Viana, Pedro F.; Richner, Thomas J.; Nurse, Ewan S.; et al
(, Epilepsia)
Abstract ObjectiveThe factors that influence seizure timing are poorly understood, and seizure unpredictability remains a major cause of disability. Work in chronobiology has shown that cyclical physiological phenomena are ubiquitous, with daily and multiday cycles evident in immune, endocrine, metabolic, neurological, and cardiovascular function. Additionally, work with chronic brain recordings has identified that seizure risk is linked to daily and multiday cycles in brain activity. Here, we provide the first characterization of the relationships between the cyclical modulation of a diverse set of physiological signals, brain activity, and seizure timing. MethodsIn this cohort study, 14 subjects underwent chronic ambulatory monitoring with a multimodal wrist‐worn sensor (recording heart rate, accelerometry, electrodermal activity, and temperature) and an implanted responsive neurostimulation system (recording interictal epileptiform abnormalities and electrographic seizures). Wavelet and filter–Hilbert spectral analyses characterized circadian and multiday cycles in brain and wearable recordings. Circular statistics assessed electrographic seizure timing and cycles in physiology. ResultsTen subjects met inclusion criteria. The mean recording duration was 232 days. Seven subjects had reliable electroencephalographic seizure detections (mean = 76 seizures). Multiday cycles were present in all wearable device signals across all subjects. Seizure timing was phase locked to multiday cycles in five (temperature), four (heart rate, phasic electrodermal activity), and three (accelerometry, heart rate variability, tonic electrodermal activity) subjects. Notably, after regression of behavioral covariates from heart rate, six of seven subjects had seizure phase locking to the residual heart rate signal. SignificanceSeizure timing is associated with daily and multiday cycles in multiple physiological processes. Chronic multimodal wearable device recordings can situate rare paroxysmal events, like seizures, within a broader chronobiology context of the individual. Wearable devices may advance the understanding of factors that influence seizure risk and enable personalized time‐varying approaches to epilepsy care.
Rahman, Safwanur
Hamid
(, IEEE Signal Processing in Medicine and Biology Symposium (SPMB))
Obeid, Iyad; Selesnick, Ivan; Picone, Joseph
(Ed.)
The Temple University Hospital Seizure Detection Corpus (TUSZ) [1] has been in distribution since April 2017. It is a subset of the TUH EEG Corpus (TUEG) [2] and the most frequently requested corpus from our 3,000+ subscribers. It was recently featured as the challenge task in the Neureka 2020 Epilepsy Challenge [3]. A summary of the development of the corpus is shown below in Table 1. The TUSZ Corpus is a fully annotated corpus, which means every seizure event that occurs within its files has been annotated. The data is selected from TUEG using a screening process that identifies files most likely to contain seizures [1]. Approximately 7% of the TUEG data contains a seizure event, so it is important we triage TUEG for high yield data. One hour of EEG data requires approximately one hour of human labor to complete annotation using the pipeline described below, so it is important from a financial standpoint that we accurately triage data. A summary of the labels being used to annotate the data is shown in Table 2. Certain standards are put into place to optimize the annotation process while not sacrificing consistency. Due to the nature of EEG recordings, some records start off with a segment of calibration. This portion of the EEG is instantly recognizable and transitions from what resembles lead artifact to a flat line on all the channels. For the sake of seizure annotation, the calibration is ignored, and no time is wasted on it. During the identification of seizure events, a hard “3 second rule” is used to determine whether two events should be combined into a single larger event. This greatly reduces the time that it takes to annotate a file with multiple events occurring in succession. In addition to the required minimum 3 second gap between seizures, part of our standard dictates that no seizure less than 3 seconds be annotated. Although there is no universally accepted definition for how long a seizure must be, we find that it is difficult to discern with confidence between burst suppression or other morphologically similar impressions when the event is only a couple seconds long. This is due to several reasons, the most notable being the lack of evolution which is oftentimes crucial for the determination of a seizure. After the EEG files have been triaged, a team of annotators at NEDC is provided with the files to begin data annotation. An example of an annotation is shown in Figure 1. A summary of the workflow for our annotation process is shown in Figure 2. Several passes are performed over the data to ensure the annotations are accurate. Each file undergoes three passes to ensure that no seizures were missed or misidentified. The first pass of TUSZ involves identifying which files contain seizures and annotating them using our annotation tool. The time it takes to fully annotate a file can vary drastically depending on the specific characteristics of each file; however, on average a file containing multiple seizures takes 7 minutes to fully annotate. This includes the time that it takes to read the patient report as well as traverse through the entire file. Once an event has been identified, the start and stop time for the seizure is stored in our annotation tool. This is done on a channel by channel basis resulting in an accurate representation of the seizure spreading across different parts of the brain. Files that do not contain any seizures take approximately 3 minutes to complete. Even though there is no annotation being made, the file is still carefully examined to make sure that nothing was overlooked. In addition to solely scrolling through a file from start to finish, a file is often examined through different lenses. Depending on the situation, low pass filters are used, as well as increasing the amplitude of certain channels. These techniques are never used in isolation and are meant to further increase our confidence that nothing was missed. Once each file in a given set has been looked at once, the annotators start the review process. The reviewer checks a file and comments any changes that they recommend. This takes about 3 minutes per seizure containing file, which is significantly less time than the first pass. After each file has been commented on, the third pass commences. This step takes about 5 minutes per seizure file and requires the reviewer to accept or reject the changes that the second reviewer suggested. Since tangible changes are made to the annotation using the annotation tool, this step takes a bit longer than the previous one. Assuming 18% of the files contain seizures, a set of 1,000 files takes roughly 127 work hours to annotate. Before an annotator contributes to the data interpretation pipeline, they are trained for several weeks on previous datasets. A new annotator is able to be trained using data that resembles what they would see under normal circumstances. An additional benefit of using released data to train is that it serves as a means of constantly checking our work. If a trainee stumbles across an event that was not previously annotated, it is promptly added, and the data release is updated. It takes about three months to train an annotator to a point where their annotations can be trusted. Even though we carefully screen potential annotators during the hiring process, only about 25% of the annotators we hire survive more than one year doing this work. To ensure that the annotators are consistent in their annotations, the team conducts an interrater agreement evaluation periodically to ensure that there is a consensus within the team. The annotation standards are discussed in Ochal et al. [4]. An extended discussion of interrater agreement can be found in Shah et al. [5]. The most recent release of TUSZ, v1.5.2, represents our efforts to review the quality of the annotations for two upcoming challenges we hosted: an internal deep learning challenge at IBM [6] and the Neureka 2020 Epilepsy Challenge [3]. One of the biggest changes that was made to the annotations was the imposition of a stricter standard for determining the start and stop time of a seizure. Although evolution is still included in the annotations, the start times were altered to start when the spike-wave pattern becomes distinct as opposed to merely when the signal starts to shift from background. This cuts down on background that was mislabeled as a seizure. For seizure end times, all post ictal slowing that was included was removed. The recent release of v1.5.2 did not include any additional data files. Two EEG files had been added because, originally, they were corrupted in v1.5.1 but were able to be retrieved and added for the latest release. The progression from v1.5.0 to v1.5.1 and later to v1.5.2, included the re-annotation of all of the EEG files in order to develop a confident dataset regarding seizure identification. Starting with v1.4.0, we have also developed a blind evaluation set that is withheld for use in competitions. The annotation team is currently working on the next release for TUSZ, v1.6.0, which is expected to occur in August 2020. It will include new data from 2016 to mid-2019. This release will contain 2,296 files from 2016 as well as several thousand files representing the remaining data through mid-2019. In addition to files that were obtained with our standard triaging process, a part of this release consists of EEG files that do not have associated patient reports. Since actual seizure events are in short supply, we are mining a large chunk of data for which we have EEG recordings but no reports. Some of this data contains interesting seizure events collected during long-term EEG sessions or data collected from patients with a history of frequent seizures. It is being mined to increase the number of files in the corpus that have at least one seizure event. We expect v1.6.0 to be released before IEEE SPMB 2020. The TUAR Corpus is an open-source database that is currently available for use by any registered member of our consortium. To register and receive access, please follow the instructions provided at this web page: https://www.isip.piconepress.com/projects/tuh_eeg/html/downloads.shtml. The data is located here: https://www.isip.piconepress.com/projects/tuh_eeg/downloads/tuh_eeg_artifact/v2.0.0/.
Abstract Seizure clusters are seizures that occur in rapid succession during periods of heightened seizure risk and are associated with substantial morbidity and sudden unexpected death in epilepsy. The objective of this feasibility study was to evaluate the performance of a novel seizure cluster forecasting algorithm. Chronic ambulatory electrocorticography recorded over an average of 38 months in 10 subjects with drug‐resistant epilepsies was analyzed pseudoprospectively by dividing data into training (first 85%) and validation periods. For each subject, the probability of seizure clustering, derived from the Kolmogorov–Smirnov statistic using a novel algorithm, was forecasted in the validation period using individualized autoregressive models that were optimized from training data. The primary outcome of this study was the mean absolute scaled error (MASE) of 1‐day horizon forecasts. From 10 subjects, 394 ± 142 (mean ± SD) electrocorticography‐based seizure events were extracted for analysis, representing a span of 38 ± 27 months of recording. MASE across all subjects was .74 ± .09, .78 ± .09, and .83 ± .07 at .5‐, 1‐, and 2‐day horizons. The feasibility study demonstrates that seizure clusters are quasiperiodic and can be forecasted to clinically meaningful horizons. Pending validation in larger cohorts, the forecasting approach described herein may herald chronotherapy during imminent heightened seizure vulnerability.
Lin, Rebecca, Marquez, Destiny, Jacobson, Mercedes, Castaldi, Hannah, Buckman, Samuel, Shah, Vinit, and Picone, Joseph. Accuracy of Automated Machine Learning Software in Identifying EEGs with Prolonged Seizures. Retrieved from https://par.nsf.gov/biblio/10199665. Annual Meeting of the American Academy of Neurology (AAN) 1.1
Lin, Rebecca, Marquez, Destiny, Jacobson, Mercedes, Castaldi, Hannah, Buckman, Samuel, Shah, Vinit, & Picone, Joseph. Accuracy of Automated Machine Learning Software in Identifying EEGs with Prolonged Seizures. Annual Meeting of the American Academy of Neurology (AAN), 1 (1). Retrieved from https://par.nsf.gov/biblio/10199665.
Lin, Rebecca, Marquez, Destiny, Jacobson, Mercedes, Castaldi, Hannah, Buckman, Samuel, Shah, Vinit, and Picone, Joseph.
"Accuracy of Automated Machine Learning Software in Identifying EEGs with Prolonged Seizures". Annual Meeting of the American Academy of Neurology (AAN) 1 (1). Country unknown/Code not available. https://par.nsf.gov/biblio/10199665.
@article{osti_10199665,
place = {Country unknown/Code not available},
title = {Accuracy of Automated Machine Learning Software in Identifying EEGs with Prolonged Seizures},
url = {https://par.nsf.gov/biblio/10199665},
abstractNote = {Objective: To demonstrate that combining automatic processing of EEG data using high performance machine learning algorithms with manual review by expert annotators can quickly identify subjects with prolonged seizures. Background: Prolonged seizures are markers of seizure severity, risk of transformation into status epilepticus, and medical morbidity. Early recognition of prolonged seizures permits intervention and reduces morbidity. Design/Methods: We triaged the TUH EEG Corpus, an open source database of EEGs, by running a state-of-the-art hybrid LSTM-based deep learning system. Then, we postprocessed the output to identify high confidence hypotheses for seizures that were greater than three minutes in duration. Results: The triaging method selected 25 subjects for further review. 17 subjects had seizures; only 5 met criteria for seizures greater than 3 minutes. 11 subjects did not have a prior diagnosis of epilepsy. Among these, 63% had acute respiratory failure and 36% had cardiac arrest leading to seizures secondary to anoxic brain injury. 18 (72%) EEGs were obtained in long-term monitoring (LTM), 1 (4%) in the epilepsy monitoring unit (EMU), and 6 (24%) as a routine EEG (rEEG). 72.2% of seizures in LTM were identified correctly versus 66.7% in rEEGs. Of the 9 subjects who were deceased, 7 (78%) had been on LTM. The seizure detection algorithm misidentified seizures in 7 subjects (28%). A total of 22 (88%) subjects had some ictal pattern. Patterns mistaken for seizure activity included muscle artifact, generalized periodic discharges, generalized spike-and-wave, triphasic waves, and interestingly, an EEG recording captured during CPR. Conclusions: This hybrid approach, which combines state-of-the-art machine learning seizure detection software with human annotation, successfully identified prolonged seizures in 72% of subjects; 88% had ictal patterns. Prolonged seizures were more common in LTM subjects than the EMU and were associated with acute cardiac or pulmonary insult.},
journal = {Annual Meeting of the American Academy of Neurology (AAN)},
volume = {1},
number = {1},
author = {Lin, Rebecca and Marquez, Destiny and Jacobson, Mercedes and Castaldi, Hannah and Buckman, Samuel and Shah, Vinit and Picone, Joseph},
editor = {null}
}
Warning: Leaving National Science Foundation Website
You are now leaving the National Science Foundation website to go to a non-government website.
Website:
NSF takes no responsibility for and exercises no control over the views expressed or the accuracy of
the information contained on this site. Also be aware that NSF's privacy policy does not apply to this site.
An official website of the United States government
