Super-resolution fluorescence microscopy methods enable the characterization of nanostructures in living and fixed biological tissues. However, they require the adjustment of multiple imaging parameters while attempting to satisfy conflicting objectives, such as maximizing spatial and temporal resolution while minimizing light exposure. To overcome the limitations imposed by these trade-offs, post-acquisition algorithmic approaches have been proposed for resolution enhancement and image-quality improvement. Here we introduce the task-assisted generative adversarial network (TA-GAN), which incorporates an auxiliary task (for example, segmentation, localization) closely related to the observed biological nanostructure characterization. We evaluate how the TA-GAN improves generative accuracy over unassisted methods, using images acquired with different modalities such as confocal, bright-field, stimulated emission depletion and structured illumination microscopy. The TA-GAN is incorporated directly into the acquisition pipeline of the microscope to predict the nanometric content of the field of view without requiring the acquisition of a super-resolved image. This information is used to automatically select the imaging modality and regions of interest, optimizing the acquisition sequence by reducing light exposure. Data-driven microscopy methods like the TA-GAN will enable the observation of dynamic molecular processes with spatial and temporal resolutions that surpass the limits currently imposed by the trade-offs constraining super-resolution microscopy.
The resolution of fluorescence microscopy images is limited by the physical properties of light. In the last decade, numerous super-resolution microscopy (SRM) approaches have been proposed to deal with such hindrance. Here we present Mean-Shift Super Resolution (MSSR), a new SRM algorithm based on the Mean Shift theory, which extends spatial resolution of single fluorescence images beyond the diffraction limit of light. MSSR works on low and high fluorophore densities, is not limited by the architecture of the optical setup and is applicable to single images as well as temporal series. The theoretical limit of spatial resolution, based on optimized real-world imaging conditions and analysis of temporal image stacks, has been measured to be 40 nm. Furthermore, MSSR has denoising capabilities that outperform other SRM approaches. Along with its wide accessibility, MSSR is a powerful, flexible, and generic tool for multidimensional and live cell imaging applications.
more » « less- Award ID(s):
- 2102832
- NSF-PAR ID:
- 10383733
- Author(s) / Creator(s):
- ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; more »
- Publisher / Repository:
- Nature Publishing Group
- Date Published:
- Journal Name:
- Nature Communications
- Volume:
- 13
- Issue:
- 1
- ISSN:
- 2041-1723
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract -
more » « less
Visualizing fine neuronal structures deep inside strongly light‐scattering brain tissue remains a challenge in neuroscience. Recent nanoscopy techniques have reached the necessary resolution but often suffer from limited imaging depth, long imaging time or high light fluence requirements. Here, we present two‐photon super‐resolution patterned excitation reconstruction (2P‐SuPER) microscopy for 3‐dimensional imaging of dendritic spine dynamics at a maximum demonstrated imaging depth of 130 μm in living brain tissue with approximately 100 nm spatial resolution. We confirmed 2P‐SuPER resolution using fluorescence nanoparticle and quantum dot phantoms and imaged spiny neurons in acute brain slices. We induced hippocampal plasticity and showed that 2P‐SuPER can resolve increases in dendritic spine head sizes on CA1 pyramidal neurons following theta‐burst stimulation of Schaffer collateral axons. 2P‐SuPER further revealed nanoscopic increases in dendritic spine neck widths, a feature of synaptic plasticity that has not been thoroughly investigated due to the combined limit of resolution and penetration depth in existing imaging technologies.
-
more » « less
Abstract Single-molecule localization microscopy (SMLM) breaks the optical diffraction limit by numerically localizing sparse fluorescence emitters to achieve super-resolution imaging. Spectroscopic SMLM or sSMLM further allows simultaneous spectroscopy and super-resolution imaging of fluorescence molecules. Hence, sSMLM can extract spectral features with single-molecule sensitivity, higher precision, and higher multiplexity than traditional multicolor microscopy modalities. These new capabilities enabled advanced multiplexed and functional cellular imaging applications. While sSMLM suffers from reduced spatial precision compared to conventional SMLM due to splitting photons to form spatial and spectral images, several methods have been reported to mitigate these weaknesses through innovative optical design and image processing techniques. This review summarizes the recent progress in sSMLM, its applications, and our perspective on future work.
Graphical Abstract -
more » « less
Abstract Interactions between light and matter serve as the basis of many technologies, but the quality of these devices is inherently limited by the optical properties of their constituents. Plasmonic nanoparticles are a highly versatile and tunable platform for the enhancement of such optical properties. However, the near‐field nature of these effects has made thorough study and understanding of these mechanisms difficult. In this work, we introduce a fully confocal technique combining photoswitching super‐resolution microscopy with fluorescence lifetime imaging microscopy to study single‐molecule decay rate enhancement. We demonstrate that the technique combines a spatial resolution better than 20 nm, and a 16 ps temporal resolution. Simultaneously, an autocorrelation measurement is also performed to confirm that the data indeed originates from single molecules. This work provides insight into the various mechanisms of plasmon‐enhanced emission, and allows the study of the correlation between emission intensity and lifetime enhancement. This complicated relationship is shown to be dependent upon the relative influence of various radiative and nonradiative decay pathways. Here, we provide a platform for further study of emission mislocalization, the position‐dependent prominence of different decay pathways, and the direct super‐resolved measurement of the local density of states.
-
more » « less
Summary Structured‐illumination microscopy allows widefield fluorescence imaging with resolution beyond the classical diffraction limit. Its linear form extends resolution by a factor of two, and its nonlinear form by an in‐principle infinite factor, the effective resolution in practice being determined by noise. In this paper, we analyse the noise properties and achievable resolution of linear and nonlinear 1D and 2D patterned SIM from a frequency–space perspective. We develop an analytical theory for a general case of linear or nonlinear fluorescent imaging, and verify the analytical calculations with numerical simulation for a special case where nonlinearity is produced by photoswitching of fluorescent labels. We compare the performance of two alternative implementations, using either two‐dimensional (2D) illumination patterns or sequentially rotated one‐dimensional (ID) patterns. We show that 1D patterns are advantageous in the linear case, and that in the nonlinear case 2D patterns provide a slight signal‐to‐noise advantage under idealised conditions, but perform worse than 1D patterns in the presence of nonswitchable fluorescent background.
Lay Description Structured‐illumination microscopy (SIM) is a high‐resolution light microscopy technique that allows imaging of fluorescence at a resolution about twice the classical diffraction limit. There are various ways that the illumination can be structured, but it is not obvious how the choice of illumination pattern affects the final image quality, especially in view of the noise. We present a detailed performance analysis considering two illumination techniques: sequential illumination with line‐gratings that are shifted and rotated during image acquisition and two‐dimensional (2D) illumination structures requiring only shift operations. Our analysis is based on analytical theory, supported by simulations of images considering noise. We also extend our analysis to a nonlinear variant of SIM, with which enhanced resolution can be achieved, limited only by noise. This includes nonlinear SIM based on the light‐induced switching of the fluorescent molecules between a bright and a dark state. We find sequential illumination with line‐gratings to be advantageous in ordinary (linear) SIM, whereas 2D patterns provides a slight signal‐to‐noise advantage under idealised conditions in nonlinear SIM if there is no nonswitching background.
