skip to main content

Title: Exploring Blob Detection to Determine Atomic Column Positions and Intensities in Time-Resolved TEM Images with Ultra-Low Signal-to-Noise

Spatially resolved in situ transmission electron microscopy (TEM), equipped with direct electron detection systems, is a suitable technique to record information about the atom-scale dynamics with millisecond temporal resolution from materials. However, characterizing dynamics or fluxional behavior requires processing short time exposure images which usually have severely degraded signal-to-noise ratios. The poor signal-to-noise associated with high temporal resolution makes it challenging to determine the position and intensity of atomic columns in materials undergoing structural dynamics. To address this challenge, we propose a noise-robust, processing approach based on blob detection, which has been previously established for identifying objects in images in the community of computer vision. In particular, a blob detection algorithm has been tailored to deal with noisy TEM image series from nanoparticle systems. In the presence of high noise content, our blob detection approach is demonstrated to outperform the results of other algorithms, enabling the determination of atomic column position and its intensity with a higher degree of precision.

; ; ; ; ;
Award ID(s):
1940179 1940263 1604971
Publication Date:
Journal Name:
Microscopy and Microanalysis
Page Range or eLocation-ID:
p. 1917-1930
Oxford University Press
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Dislocations, linear defects in a crystalline lattice characterized by their slip systems, can provide a record of grain internal deformation. Comprehensive examination of this record has been limited by intrinsic limitations of the observational methods. Transmission electron microscopy reveals individual dislocations, but images only a few square$$\upmu$$μm of sample. Oxidative decoration requires involved sample preparation and has uncertainties in detection of all dislocations and their types. The possibility of mapping dislocation density and slip systems by conventional (Hough-transform based) EBSD is investigated here with naturally and experimentally deformed San Carlos olivine single crystals. Geometry and dislocation structures of crystals deformed in orientations designed to activate particular slip systems were previously analyzed by TEM and oxidative decoration. A curvature tensor is calculated from changes in orientation of the crystal lattice, which is inverted to calculate density of geometrically necessary dislocations with the Matlab Toolbox MTEX. Densities of individual dislocation types along with misorientation axes are compared to orientation change measured on the deformed crystals. After filtering (denoising), noise floor and calculated dislocation densities are comparable to those reported from high resolution EBSD mapping. For samples deformed in [110]c and [011]c orientations EBSD mapping confirms [100](010) and [001](010), respectively, as themore »dominant slip systems. EBSD mapping thus enables relatively efficient observation of dislocation structures associated with intracrystalline deformation, both distributed, and localized at sub-boundaries, over substantially larger areas than has previously been possible. This will enable mapping of dislocation structures in both naturally and experimentally deformed polycrystals, with potentially new insights into deformation processes in Earth’s upper mantle.

    « less
  2. Denoising is a fundamental challenge in scientific imaging. Deep convolutional neural networks (CNNs) provide the current state of the art in denoising natural images, where they produce impressive results. However, their potential has been inadequately explored in the context of scientific imaging. Denoising CNNs are typically trained on real natural images artificially corrupted with simulated noise. In contrast, in scientific applications, noiseless ground-truth images are usually not available. To address this issue, we propose a simulation-based denoising (SBD) framework, in which CNNs are trained on simulated images. We test the framework on data obtained from transmission electron microscopy (TEM), an imaging technique with widespread applications in material science, biology, and medicine. SBD outperforms existing techniques by a wide margin on a simulated benchmark dataset, as well as on real data. We analyze the generalization capability of SBD, demonstrating that the trained networks are robust to variations of imaging parameters and of the underlying signal structure. Our results reveal that state-of-the-art architectures for denoising photographic images may not be well adapted to scientific-imaging data. For instance, substantially increasing their field-of-view dramatically improves their performance on TEM images acquired at low signal-to-noise ratios. We also demonstrate that standard performance metrics for photographsmore »(such as PSNR and SSIM) may fail to produce scientifically meaningful evaluation. We propose several metrics to remedy this issue for the case of atomic resolution electron microscope images. In addition, we propose a technique, based on likelihood computations, to visualize the agreement between the structure of the denoised images and the observed data. Finally, we release a publicly available benchmark dataset of TEM images, containing 18,000 examples.« less
  3. Investigating the earliest stages of crystallization requires the transmission electron microscope (TEM) and is particularly challenging for materials which can be affected by the electron beam. Typically, when imaging at magnifications high enough to observe local crystallinity, the electron beam's current density must be high to produce adequate image contrast. Yet, minimizing the electron dose is necessary to reduce the changes caused by the beam. With the advent of a sensitive, high-speed, direct-detection camera for a TEM that is corrected for spherical aberration, it is possible to probe the early stages of crystallization at the atomic scale. High-quality images with low contrast can now be analyzed using new computing methods. In the present paper, this approach is illustrated for crystallization in a Ge 2 Sb 2 Te 5 (GST-225) phase-change material which can undergo particularly rapid phase transformations and is sensitive to the electron beam. A thin (20 nm) film of GST-225 has been directly imaged in the TEM and the low-dose images processed using Python scripting to extract details of the nanoscale nuclei. Quantitative analysis of the processed images in a video sequence also allows the growth of such nuclei to be followed.

    Infrasound sensors are deployed in a variety of spatial configurations and scales for geophysical monitoring, including networks of single sensors and networks of multisensor infrasound arrays. Infrasound signal detection strategies exploiting these data commonly make use of intersensor correlation and coherence (array processing, multichannel correlation); network-based tracking of signal features (e.g. reverse time migration); or a combination of these such as backazimuth cross-bearings for multiple arrays. Single-sensor trace-based denoising techniques offer significant potential to improve all of these various infrasound data processing strategies, but have not previously been investigated in detail. Single-sensor denoising represents a pre-processing step that could reduce the effects of ambient infrasound and wind noise in infrasound signal association and location workflows. We systematically investigate the utility of a range of single-sensor denoising methods for infrasound data processing, including noise gating, non-negative matrix factorization, and data-adaptive Wiener filtering. For the data testbed, we use the relatively dense regional infrasound network in Alaska, which records a high rate of volcanic eruptions with signals varying in power, duration, and waveform and spectral character. We primarily use data from the 2016–2017 Bogoslof volcanic eruption, which included multiple explosions, and synthetics. The Bogoslof volcanic sequence provides an opportunity to investigatemore »regional infrasound detection, association, and location for a set of real sources with varying source spectra subject to anisotropic atmospheric propagation and varying noise levels (both incoherent wind noise and coherent ambient infrasound, primarily microbaroms). We illustrate the advantages and disadvantages of the different denoising methods in categories such as event detection, waveform distortion, the need for manual data labelling, and computational cost. For all approaches, denoising generally performs better for signals with higher signal-to-noise ratios and with less spectral and temporal overlap between signals and noise. Microbaroms are the most globally pervasive and repetitive coherent ambient infrasound noise source, with such noise often referred to as clutter or interference. We find that denoising offers significant potential for microbarom clutter reduction. Single-channel denoising of microbaroms prior to standard array processing enhances both the quantity and bandwidth of detectable volcanic events. We find that reduction of incoherent wind noise is more challenging using the denoising methods we investigate; thus, station hardware (wind noise reduction systems) and site selection remain critical and cannot be replaced by currently available digital denoising methodologies. Overall, we find that adding single-channel denoising as a component in the processing workflow can benefit a variety of infrasound signal detection, association, and location schemes. The denoising methods can also isolate the noise itself, with utility in statistically characterizing ambient infrasound noise.

    « less
  5. Abstract

    In computer vision, single-image super-resolution (SISR) has been extensively explored using convolutional neural networks (CNNs) on optical images, but images outside this domain, such as those from scientific experiments, are not well investigated. Experimental data is often gathered using non-optical methods, which alters the metrics for image quality. One such example is electron backscatter diffraction (EBSD), a materials characterization technique that maps crystal arrangement in solid materials, which provides insight into processing, structure, and property relationships. We present a broadly adaptable approach for applying state-of-art SISR networks to generate super-resolved EBSD orientation maps. This approach includes quaternion-based orientation recognition, loss functions that consider rotational effects and crystallographic symmetry, and an inference pipeline to convert network output into established visualization formats for EBSD maps. The ability to generate physically accurate, high-resolution EBSD maps with super-resolution enables high-throughput characterization and broadens the capture capabilities for three-dimensional experimental EBSD datasets.