skip to main content

Title: Robust Small Area Estimation: An Overview
A small area typically refers to a subpopulation or domain of interest for which a reliable direct estimate, based only on the domain-specific sample, cannot be produced due to small sample size in the domain. While traditional small area methods and models are widely used nowadays, there have also been much work and interest in robust statistical inference for small area estimation (SAE). We survey this work and provide a comprehensive review here.We begin with a brief review of the traditional SAE methods. We then discuss SAEmethods that are developed under weaker assumptions and SAE methods that are robust in certain ways, such as in terms of outliers or model failure. Our discussion also includes topics such as nonparametric SAE methods, Bayesian approaches, model selection and diagnostics, and missing data. A brief review of software packages available for implementing robust SAE methods is also given.
Award ID(s):
Publication Date:
Journal Name:
Annual review of statistics and its application
Page Range or eLocation-ID:
Sponsoring Org:
National Science Foundation
More Like this
  1. Obeid, Iyad Selesnick (Ed.)
    Electroencephalography (EEG) is a popular clinical monitoring tool used for diagnosing brain-related disorders such as epilepsy [1]. As monitoring EEGs in a critical-care setting is an expensive and tedious task, there is a great interest in developing real-time EEG monitoring tools to improve patient care quality and efficiency [2]. However, clinicians require automatic seizure detection tools that provide decisions with at least 75% sensitivity and less than 1 false alarm (FA) per 24 hours [3]. Some commercial tools recently claim to reach such performance levels, including the Olympic Brainz Monitor [4] and Persyst 14 [5]. In this abstract, we describemore »our efforts to transform a high-performance offline seizure detection system [3] into a low latency real-time or online seizure detection system. An overview of the system is shown in Figure 1. The main difference between an online versus offline system is that an online system should always be causal and has minimum latency which is often defined by domain experts. The offline system, shown in Figure 2, uses two phases of deep learning models with postprocessing [3]. The channel-based long short term memory (LSTM) model (Phase 1 or P1) processes linear frequency cepstral coefficients (LFCC) [6] features from each EEG channel separately. We use the hypotheses generated by the P1 model and create additional features that carry information about the detected events and their confidence. The P2 model uses these additional features and the LFCC features to learn the temporal and spatial aspects of the EEG signals using a hybrid convolutional neural network (CNN) and LSTM model. Finally, Phase 3 aggregates the results from both P1 and P2 before applying a final postprocessing step. The online system implements Phase 1 by taking advantage of the Linux piping mechanism, multithreading techniques, and multi-core processors. To convert Phase 1 into an online system, we divide the system into five major modules: signal preprocessor, feature extractor, event decoder, postprocessor, and visualizer. The system reads 0.1-second frames from each EEG channel and sends them to the feature extractor and the visualizer. The feature extractor generates LFCC features in real time from the streaming EEG signal. Next, the system computes seizure and background probabilities using a channel-based LSTM model and applies a postprocessor to aggregate the detected events across channels. The system then displays the EEG signal and the decisions simultaneously using a visualization module. The online system uses C++, Python, TensorFlow, and PyQtGraph in its implementation. The online system accepts streamed EEG data sampled at 250 Hz as input. The system begins processing the EEG signal by applying a TCP montage [8]. Depending on the type of the montage, the EEG signal can have either 22 or 20 channels. To enable the online operation, we send 0.1-second (25 samples) length frames from each channel of the streamed EEG signal to the feature extractor and the visualizer. Feature extraction is performed sequentially on each channel. The signal preprocessor writes the sample frames into two streams to facilitate these modules. In the first stream, the feature extractor receives the signals using stdin. In parallel, as a second stream, the visualizer shares a user-defined file with the signal preprocessor. This user-defined file holds raw signal information as a buffer for the visualizer. The signal preprocessor writes into the file while the visualizer reads from it. Reading and writing into the same file poses a challenge. The visualizer can start reading while the signal preprocessor is writing into it. To resolve this issue, we utilize a file locking mechanism in the signal preprocessor and visualizer. Each of the processes temporarily locks the file, performs its operation, releases the lock, and tries to obtain the lock after a waiting period. The file locking mechanism ensures that only one process can access the file by prohibiting other processes from reading or writing while one process is modifying the file [9]. The feature extractor uses circular buffers to save 0.3 seconds or 75 samples from each channel for extracting 0.2-second or 50-sample long center-aligned windows. The module generates 8 absolute LFCC features where the zeroth cepstral coefficient is replaced by a temporal domain energy term. For extracting the rest of the features, three pipelines are used. The differential energy feature is calculated in a 0.9-second absolute feature window with a frame size of 0.1 seconds. The difference between the maximum and minimum temporal energy terms is calculated in this range. Then, the first derivative or the delta features are calculated using another 0.9-second window. Finally, the second derivative or delta-delta features are calculated using a 0.3-second window [6]. The differential energy for the delta-delta features is not included. In total, we extract 26 features from the raw sample windows which add 1.1 seconds of delay to the system. We used the Temple University Hospital Seizure Database (TUSZ) v1.2.1 for developing the online system [10]. The statistics for this dataset are shown in Table 1. A channel-based LSTM model was trained using the features derived from the train set using the online feature extractor module. A window-based normalization technique was applied to those features. In the offline model, we scale features by normalizing using the maximum absolute value of a channel [11] before applying a sliding window approach. Since the online system has access to a limited amount of data, we normalize based on the observed window. The model uses the feature vectors with a frame size of 1 second and a window size of 7 seconds. We evaluated the model using the offline P1 postprocessor to determine the efficacy of the delayed features and the window-based normalization technique. As shown by the results of experiments 1 and 4 in Table 2, these changes give us a comparable performance to the offline model. The online event decoder module utilizes this trained model for computing probabilities for the seizure and background classes. These posteriors are then postprocessed to remove spurious detections. The online postprocessor receives and saves 8 seconds of class posteriors in a buffer for further processing. It applies multiple heuristic filters (e.g., probability threshold) to make an overall decision by combining events across the channels. These filters evaluate the average confidence, the duration of a seizure, and the channels where the seizures were observed. The postprocessor delivers the label and confidence to the visualizer. The visualizer starts to display the signal as soon as it gets access to the signal file, as shown in Figure 1 using the “Signal File” and “Visualizer” blocks. Once the visualizer receives the label and confidence for the latest epoch from the postprocessor, it overlays the decision and color codes that epoch. The visualizer uses red for seizure with the label SEIZ and green for the background class with the label BCKG. Once the streaming finishes, the system saves three files: a signal file in which the sample frames are saved in the order they were streamed, a time segmented event (TSE) file with the overall decisions and confidences, and a hypotheses (HYP) file that saves the label and confidence for each epoch. The user can plot the signal and decisions using the signal and HYP files with only the visualizer by enabling appropriate options. For comparing the performance of different stages of development, we used the test set of TUSZ v1.2.1 database. It contains 1015 EEG records of varying duration. The any-overlap performance [12] of the overall system shown in Figure 2 is 40.29% sensitivity with 5.77 FAs per 24 hours. For comparison, the previous state-of-the-art model developed on this database performed at 30.71% sensitivity with 6.77 FAs per 24 hours [3]. The individual performances of the deep learning phases are as follows: Phase 1’s (P1) performance is 39.46% sensitivity and 11.62 FAs per 24 hours, and Phase 2 detects seizures with 41.16% sensitivity and 11.69 FAs per 24 hours. We trained an LSTM model with the delayed features and the window-based normalization technique for developing the online system. Using the offline decoder and postprocessor, the model performed at 36.23% sensitivity with 9.52 FAs per 24 hours. The trained model was then evaluated with the online modules. The current performance of the overall online system is 45.80% sensitivity with 28.14 FAs per 24 hours. Table 2 summarizes the performances of these systems. The performance of the online system deviates from the offline P1 model because the online postprocessor fails to combine the events as the seizure probability fluctuates during an event. The modules in the online system add a total of 11.1 seconds of delay for processing each second of the data, as shown in Figure 3. In practice, we also count the time for loading the model and starting the visualizer block. When we consider these facts, the system consumes 15 seconds to display the first hypothesis. The system detects seizure onsets with an average latency of 15 seconds. Implementing an automatic seizure detection model in real time is not trivial. We used a variety of techniques such as the file locking mechanism, multithreading, circular buffers, real-time event decoding, and signal-decision plotting to realize the system. A video demonstrating the system is available at: The final conference submission will include a more detailed analysis of the online performance of each module. ACKNOWLEDGMENTS Research reported in this publication was most recently supported by the National Science Foundation Partnership for Innovation award number IIP-1827565 and the Pennsylvania Commonwealth Universal Research Enhancement Program (PA CURE). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the official views of any of these organizations. REFERENCES [1] A. Craik, Y. He, and J. L. Contreras-Vidal, “Deep learning for electroencephalogram (EEG) classification tasks: a review,” J. Neural Eng., vol. 16, no. 3, p. 031001, 2019. [2] A. C. Bridi, T. Q. Louro, and R. C. L. Da Silva, “Clinical Alarms in intensive care: implications of alarm fatigue for the safety of patients,” Rev. Lat. Am. Enfermagem, vol. 22, no. 6, p. 1034, 2014. [3] M. Golmohammadi, V. Shah, I. Obeid, and J. Picone, “Deep Learning Approaches for Automatic Seizure Detection from Scalp Electroencephalograms,” in Signal Processing in Medicine and Biology: Emerging Trends in Research and Applications, 1st ed., I. Obeid, I. Selesnick, and J. Picone, Eds. New York, New York, USA: Springer, 2020, pp. 233–274. [4] “CFM Olympic Brainz Monitor.” [Online]. Available: [Accessed: 17-Jul-2020]. [5] M. L. Scheuer, S. B. Wilson, A. Antony, G. Ghearing, A. Urban, and A. I. Bagic, “Seizure Detection: Interreader Agreement and Detection Algorithm Assessments Using a Large Dataset,” J. Clin. Neurophysiol., 2020. [6] A. Harati, M. Golmohammadi, S. Lopez, I. Obeid, and J. Picone, “Improved EEG Event Classification Using Differential Energy,” in Proceedings of the IEEE Signal Processing in Medicine and Biology Symposium, 2015, pp. 1–4. [7] V. Shah, C. Campbell, I. Obeid, and J. Picone, “Improved Spatio-Temporal Modeling in Automated Seizure Detection using Channel-Dependent Posteriors,” Neurocomputing, 2021. [8] W. Tatum, A. Husain, S. Benbadis, and P. Kaplan, Handbook of EEG Interpretation. New York City, New York, USA: Demos Medical Publishing, 2007. [9] D. P. Bovet and C. Marco, Understanding the Linux Kernel, 3rd ed. O’Reilly Media, Inc., 2005. [10] V. Shah et al., “The Temple University Hospital Seizure Detection Corpus,” Front. Neuroinform., vol. 12, pp. 1–6, 2018. [11] F. Pedregosa et al., “Scikit-learn: Machine Learning in Python,” J. Mach. Learn. Res., vol. 12, pp. 2825–2830, 2011. [12] J. Gotman, D. Flanagan, J. Zhang, and B. Rosenblatt, “Automatic seizure detection in the newborn: Methods and initial evaluation,” Electroencephalogr. Clin. Neurophysiol., vol. 103, no. 3, pp. 356–362, 1997.« less
  2. Abstract. Robust, proxy-based reconstructions of relative sea-level (RSL) change are critical to distinguishing the processes that drive spatial and temporal sea-level variability. The relationships between individual proxies and RSL can be complex and are often poorly represented by traditional methods that assume Gaussian likelihood distributions. We develop a new statistical framework to estimate past RSL change based on nonparametric, empirical modern distributions of proxies in relation to RSL, applying the framework to corals and mangroves as an illustrative example. We validate our model by comparing its skill in reconstructing RSL and rates of change to two previous RSL models usingmore »synthetic time-series datasets based on Holocene sea-level data from South Florida. The new framework results in lower bias, better model fit, and greater accuracy and precision than the two previous RSL models. We also perform sensitivity tests using sea-level scenarios based on two periods of interest – meltwater pulses (MWPs) and the Holocene – to analyze the sensitivity of the statistical reconstructions to the quantity and precision of proxy data; we define high-precision indicators, such as mangroves and the reef-crest coral Acropora palmata, with 2σ vertical uncertainties within ± 3 m and lower-precision indicators, such as Orbicella spp., with 2σ vertical uncertainties within ± 10 m. For reconstructing rapid rates of change in RSL of up to ∼ 40 m kyr−1, such as those that may have characterized MWPs during deglacial periods, we find that employing the nonparametric model with 5 to 10 high-precision data points per kiloyear enables us to constrain rates to within ± 3 m kyr−1 (1σ). For reconstructing RSL with rates of up to ∼ 15 m kyr−1, as observed during the Holocene, we conclude that employing the model with 5 to 10 high-precision (or a combination of high- and low-precision) data points per kiloyear enables precise estimates of RSL within ±∼ 2 m (2σ) and accurate RSL reconstructions with errors ≲ 0.7 m. Employing the nonparametric model with only lower-precision indicators also produces fairly accurate estimates of RSL with errors ≲1.50 m, although with less precision, only constraining RSL to ±∼ 3–4 m (2σ). Although the model performs better than previous models in terms of bias, model fit, accuracy, and precision, it is computationally expensive to run because it requires inverting large matrices for every sample. The new model also provides minimal gains over similar models when a large quantity of high-precision data are available. Therefore, we recommend incorporating the nonparametric likelihood distributions when no other information (e.g., reef facies or epibionts indicative of shallow-water environments to refine coral elevational uncertainties) or no high-precision data are available at a location or during a given time period of interest.« less
  3. Abstract Objective Machine learning is used to understand and track influenza-related content on social media. Because these systems are used at scale, they have the potential to adversely impact the people they are built to help. In this study, we explore the biases of different machine learning methods for the specific task of detecting influenza-related content. We compare the performance of each model on tweets written in Standard American English (SAE) vs African American English (AAE). Materials and Methods Two influenza-related datasets are used to train 3 text classification models (support vector machine, convolutional neural network, bidirectional long short-term memory)more »with different feature sets. The datasets match real-world scenarios in which there is a large imbalance between SAE and AAE examples. The number of AAE examples for each class ranges from 2% to 5% in both datasets. We also evaluate each model's performance using a balanced dataset via undersampling. Results We find that all of the tested machine learning methods are biased on both datasets. The difference in false positive rates between SAE and AAE examples ranges from 0.01 to 0.35. The difference in the false negative rates ranges from 0.01 to 0.23. We also find that the neural network methods generally has more unfair results than the linear support vector machine on the chosen datasets. Conclusions The models that result in the most unfair predictions may vary from dataset to dataset. Practitioners should be aware of the potential harms related to applying machine learning to health-related social media data. At a minimum, we recommend evaluating fairness along with traditional evaluation metrics.« less
  4. Cadmium telluride (CdTe) solar cells are a promising photovoltaic (PV) technology for producing power in space owing to their high-efficiency (> 22.1 %), potential for specific power, and cost-effective manufacturing processes. In contrast to traditional space PVs, the high-Z (atomic number) CdTe absorbers can be intrinsically robust under extreme space radiation, offering long-term stability. Despite these advantages, the performance assessment of CdTe solar cells under high-energy particle irradiation (e.g., photons, neutrons, charged particles) is limited in the literature, and their stability is not comprehensively studied. In this work, we present the PV response of n-CdS / p-CdTe PVs under acceleratedmore »neutron irradiation. We measure PV properties of the devices at different neutron/photon doses. The equivalent dose deposited in the CdTe samples is simulated with deterministic and Monte Carlo radiation transport methods. Thin-film CdTe solar cells were synthesized on a fluorine-doped tin oxide (FTO) coated glass substrate (≈ 4 cm × 4 cm). CdS:O (≈ 100 nm) was reactively RF sputtered in an oxygen/argon ambient followed by a close-spaced sublimation deposition of CdTe (≈ 3.5 μm) in an oxygen/helium ambient. The sample was exposed to a 10 min vapor CdCl2 in oxygen/helium ambient at 430˚C. The samples were exposed to a wet CuCl2 solution prior to anneal 200ºC. A gold back-contact was formed on CdTe via thermal evaporation. The final sample contains 16 CdTe devices. For neutron irradiation, we cleaved the CdTe substrate into four samples and exposed two samples to ≈ 90 kW reactor power neutron radiation for 5.5 hours and 8.2 hours, respectively, in our TRIGA (Training, Research, Isotopes, General Atomics) reactor. We observed a noticeable color change of the glass substrates to brown after the neutron/gamma reactor exposure. Presumably, the injected high-energy neutrons caused the breaking of chemical bonds and the displacement of atoms in the glass substrates, creating point defects and color centers. The I-V characteristics showed noticeable deterioration with over 8 hour radiations. Specifically, the saturation current of the control devices was ≈ 25 nA increasing to 1 μA and 10 μA for the 5.5-hour and 8.2-hour radiated samples, respectively. The turn-on voltage of the control devices (≈ 0.85 V) decreased with the irradiated sample (≈ 0.75 V for 5.5-hour and ≈ 0.5 V for 8.2-hour exposures), implying noticeable radiation damage occurred at the heterojunction. The higher values of the ideality factor for irradiated devices (n > 2.2) compared to that of the control devices (n ≈ 1.3) also support the deterioration of the p-n junction. We observed the notable decrease in shunt resistance (RSH) and the increase in series resistance (Rs) with the neutron dose. It is possible that Cu ions introduced during the CuCl2 treatment may migrate into CdTe grain boundaries (GBs). The presence of Cu ions at GBs can create additional leakage paths for photocarrier transport, deteriorating the overall PV performance. We estimated the radiation dose of CdTe in comparison to Si (conventional PV) using a UUTR model (e.g., MCNP6 2D UTR Reactor simulations). In this model, we simulated Si and CdTe at the center point of the triangular fuel lattice and used an “unperturbed flux” tally in the water. Our simulations yielded a dose rate of 6916 Gy/s of neutrons and 16 Gy/s of photons for CdTe, and 1 Gy/s of neutrons and 21 Gy/s of photons for Si (doses +/- <1%). The large dose rate of neutrons in CdTe is mainly attributed to the large thermal neutron absorption cross-section of 113Cd. Based on this estimation, we calculate that the exposure of our CdTe PVs is equivalent to several million years in LEO (Low-Earth Orbit), or about 10,000 years for Si in LEO. Currently, we are working on a low-dose neutron/photon radiation on CdTe PVs and their light I-Vs and microstructural characterizations to gain better understanding on the degradation of CdTe PVs.« less
  5. Great progress has been made in recent years towards understanding the properties of disordered electronic systems. In part, this is made possible by recent advances in quantum effective medium methods which enable the study of disorder and electron-electronic interactions on equal footing. They include dynamical mean-field theory and the Coherent Potential Approximation, and their cluster extension, the dynamical cluster approximation. Despite their successes, these methods do not enable the first-principles study of the strongly disordered regime, including the effects of electronic localization. The main focus of this review is the recently developed typical medium dynamical cluster approximation for disordered electronicmore »systems. This method has been constructed to capture disorder-induced localization and is based on a mapping of a lattice onto a quantum cluster embedded in an effective typical medium, which is determined self-consistently. Unlike the average effective medium-based methods mentioned above, typical medium-based methods properly capture the states localized by disorder. The typical medium dynamical cluster approximation not only provides the proper order parameter for Anderson localized states, but it can also incorporate the full complexity of Density-Functional Theory (DFT)-derived potentials into the analysis, including the effect of multiple bands, non-local disorder, and electron-electron interactions. After a brief historical review of other numerical methods for disordered systems, we discuss coarse-graining as a unifying principle for the development of translationally invariant quantum cluster methods. Together, the Coherent Potential Approximation, the Dynamical Mean-Field Theory and the Dynamical Cluster Approximation may be viewed as a single class of approximations with a much-needed small parameter of the inverse cluster size which may be used to control the approximation. We then present an overview of various recent applications of the typical medium dynamical cluster approximation to a variety of models and systems, including single and multiband Anderson model, and models with local and off-diagonal disorder. We then present the application of the method to realistic systems in the framework of the DFT and demonstrate that the resulting method can provide a systematic first-principles method validated by experiment and capable of making experimentally relevant predictions. We also discuss the application of the typical medium dynamical cluster approximation to systems with disorder and electron-electron interactions. Most significantly, we show that in the limits of strong disorder and weak interactions treated perturbatively, that the phenomena of 3D localization, including a mobility edge, remains intact. However, the metal-insulator transition is pushed to larger disorder values by the local interactions. We also study the limits of strong disorder and strong interactions capable of producing moment formation and screening, with a non-perturbative local approximation. Here, we find that the Anderson localization quantum phase transition is accompanied by a quantum-critical fan in the energy-disorder phase diagram.« less