An official website of the United States government
Abstract Highly-directional image artifacts such as ion mill curtaining, mechanical scratches, or image striping from beam instability degrade the interpretability of micrographs. These unwanted, aperiodic features extend the image along a primary direction and occupy a small wedge of information in Fourier space. Deleting this wedge of data replaces stripes, scratches, or curtaining, with more complex streaking and blurring artifacts—known within the tomography community as “missing wedge” artifacts. Here, we overcome this problem by recovering the missing region using total variation minimization, which leverages image sparsity-based reconstruction techniques—colloquially referred to as compressed sensing (CS)—to reliably restore images corrupted by stripe-like features. Our approach removes beam instability, ion mill curtaining, mechanical scratches, or any stripe features and remains robust at low signal-to-noise. The success of this approach is achieved by exploiting CS's inability to recover directional structures that are highly localized and missing in Fourier Space.
more »
« less
Dictionary learning in Fourier-transform scanning tunneling spectroscopy
Abstract Modern high-resolution microscopes are commonly used to study specimens that have dense and aperiodic spatial structure. Extracting meaningful information from images obtained from such microscopes remains a formidable challenge. Fourier analysis is commonly used to analyze the structure of such images. However, the Fourier transform fundamentally suffers from severe phase noise when applied to aperiodic images. Here, we report the development of an algorithm based on nonconvex optimization that directly uncovers the fundamental motifs present in a real-space image. Apart from being quantitatively superior to traditional Fourier analysis, we show that this algorithm also uncovers phase sensitive information about the underlying motif structure. We demonstrate its usefulness by studying scanning tunneling microscopy images of a Co-doped iron arsenide superconductor and prove that the application of the algorithm allows for the complete recovery of quasiparticle interference in this material.
more »
« less
- Award ID(s):
- 1742716
- PAR ID:
- 10153445
- Publisher / Repository:
- Nature Publishing Group
- Date Published:
- Journal Name:
- Nature Communications
- Volume:
- 11
- Issue:
- 1
- ISSN:
- 2041-1723
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
As a real-space technique, atomic-resolution STEM imaging contains both amplitude and geometric phase information about structural order in materials, with the latter encoding important information about local variations and heterogeneities present in crystalline lattices. Such phase information can be extracted using geometric phase analysis (GPA), a method which has generally focused on spatially mapping elastic strain. Here we demonstrate an alternative phase demodulation technique and its application to reveal complex structural phenomena in correlated quantum materials. As with other methods of image phase analysis, the phase lock-in approach can be implemented to extract detailed information about structural order and disorder, including dislocations and compound defects in crystals. Extending the application of this phase analysis to Fourier components that encode periodic modulations of the crystalline lattice, such as superlattice or secondary frequency peaks, we extract the behavior of multiple distinct order parameters within the same image, yielding insights into not only the crystalline heterogeneity but also subtle emergent order parameters such as antipolar displacements. When applied to atomic-resolution images spanning large (~0.5 × 0.5 μ m 2 ) fields of view, this approach enables vivid visualizations of the spatial interplay between various structural orders in novel materials.more » « less
-
Abstract A new image-reconstruction algorithm, Principal-component Interferometric Modeling (PRIMO), applied to the interferometric data of the M87 black hole collected with the Event Horizon Telescope (EHT), resulted in an image that reached the native resolution of the telescope array.PRIMOis based on learning a compact set of image building blocks obtained from a large library of high-fidelity, physics-based simulations of black hole images. It uses these building blocks to fill the sparse Fourier coverage of the data that results from the small number of telescopes in the array. In this paper, we show that this approach is readily justified. Since the angular extent of the image of the black hole and of its inner accretion flow is finite, the Fourier space domain is heavily smoothed, with a correlation scale that is at most comparable to the sizes of the data gaps in the coverage of Fourier space with the EHT. Consequently,PRIMOor other machine learning algorithms can faithfully reconstruct the images without the need to generate information that is unconstrained by the data within the resolution of the array. We also address the completeness of the eigenimages and the compactness of the resulting representation. We show thatPRIMOprovides a compact set of eigenimages that have sufficient complexity to recreate a broad set of images well beyond those in the training set.more » « less
-
null (Ed.)Image data remains an important tool for post-event building assessment and documentation. After each natural hazard event, significant efforts are made by teams of engineers to visit the affected regions and collect useful image data. In general, a global positioning system (GPS) can provide useful spatial information for localizing image data. However, it is challenging to collect such information when images are captured in places where GPS signals are weak or interrupted, such as the indoor spaces of buildings. The inability to document the images’ locations hinders the analysis, organization, and documentation of these images as they lack sufficient spatial context. In this work, we develop a methodology to localize images and link them to locations on a structural drawing. A stream of images can readily be gathered along the path taken through a building using a compact camera. These images may be used to compute a relative location of each image in a 3D point cloud model, which is reconstructed using a visual odometry algorithm. The images may also be used to create local 3D textured models for building-components-of-interest using a structure-from-motion algorithm. A parallel set of images that are collected for building assessment is linked to the image stream using time information. By projecting the point cloud model to the structural drawing, the images can be overlaid onto the drawing, providing clear context information necessary to make use of those images. Additionally, components- or damage-of-interest captured in these images can be reconstructed in 3D, enabling detailed assessments having sufficient geospatial context. The technique is demonstrated by emulating post-event building assessment and data collection in a real building.more » « less
-
Traditionally, a high-performance microscope with a large numerical aperture is required to acquire high-resolution images. However, the images’ size is typically tremendous. Therefore, they are not conveniently managed and transferred across a computer network or stored in a limited computer storage system. As a result, image compression is commonly used to reduce image size resulting in poor image resolution. Here, we demonstrate custom convolution neural networks (CNNs) for both super-resolution image enhancement from low-resolution images and characterization of both cells and nuclei from hematoxylin and eosin (H&E) stained breast cancer histopathological images by using a combination of generator and discriminator networks so-called super-resolution generative adversarial network-based on aggregated residual transformation (SRGAN-ResNeXt) to facilitate cancer diagnosis in low resource settings. The results provide high enhancement in image quality where the peak signal-to-noise ratio and structural similarity of our network results are over 30 dB and 0.93, respectively. The derived performance is superior to the results obtained from both the bicubic interpolation and the well-known SRGAN deep-learning methods. In addition, another custom CNN is used to perform image segmentation from the generated high-resolution breast cancer images derived with our model with an average Intersection over Union of 0.869 and an average dice similarity coefficient of 0.893 for the H&E image segmentation results. Finally, we propose the jointly trained SRGAN-ResNeXt and Inception U-net Models, which applied the weights from the individually trained SRGAN-ResNeXt and inception U-net models as the pre-trained weights for transfer learning. The jointly trained model’s results are progressively improved and promising. We anticipate these custom CNNs can help resolve the inaccessibility of advanced microscopes or whole slide imaging (WSI) systems to acquire high-resolution images from low-performance microscopes located in remote-constraint settings.more » « less
