Structured illumination microscopy (SIM) is a popular super-resolution imaging technique that can achieve resolution improvements of 2× and greater depending on the illumination patterns used. Traditionally, images are reconstructed using the linear SIM reconstruction algorithm. However, this algorithm has hand-tuned parameters which can often lead to artifacts, and it cannot be used with more complex illumination patterns. Recently, deep neural networks have been used for SIM reconstruction, yet they require training sets that are difficult to capture experimentally. We demonstrate that we can combine a deep neural network with the forward model of the structured illumination process to reconstruct sub-diffraction images without training data. The resulting physics-informed neural network (PINN) can be optimized on a single set of diffraction-limited sub-images and thus does not require any training set. We show, with simulated and experimental data, that this PINN can be applied to a wide variety of SIM illumination methods by simply changing the known illumination patterns used in the loss function and can achieve resolution improvements that match theoretical expectations.
more » « less- PAR ID:
- 10398542
- Publisher / Repository:
- Optical Society of America
- Date Published:
- Journal Name:
- Optics Express
- Volume:
- 31
- Issue:
- 5
- ISSN:
- 1094-4087; OPEXFF
- Format(s):
- Medium: X Size: Article No. 8714
- Size(s):
- Article No. 8714
- Sponsoring Org:
- National Science Foundation
More Like this
-
Abstract Background Structured illumination microscopy (SIM) is a method that can be used to image biological samples and can achieve both optical sectioning and super-resolution effects. Optimization of the imaging set-up and data-processing methods results in high-quality images without artifacts due to mosaicking or due to the use of SIM methods. Reconstruction methods based on Bayesian estimation can be used to produce images with a resolution beyond that dictated by the optical system. Findings Five complete datasets are presented including large panoramic SIM images of human tissues in pathophysiological conditions. Cancers of the prostate, skin, ovary, and breast, as well as tuberculosis of the lung, were imaged using SIM. The samples are available commercially and are standard histological preparations stained with hematoxylin-eosin. Conclusion The use of fluorescence microscopy is increasing in histopathology. There is a need for methods that reduce artifacts caused by the use of image-stitching methods or optical sectioning methods such as SIM. Stitched SIM images produce results that may be useful for intraoperative histology. Releasing high-quality, full-slide images and related data will aid researchers in furthering the field of fluorescent histopathology.more » « less
-
Summary Structured‐illumination microscopy allows widefield fluorescence imaging with resolution beyond the classical diffraction limit. Its linear form extends resolution by a factor of two, and its nonlinear form by an in‐principle infinite factor, the effective resolution in practice being determined by noise. In this paper, we analyse the noise properties and achievable resolution of linear and nonlinear 1D and 2D patterned SIM from a frequency–space perspective. We develop an analytical theory for a general case of linear or nonlinear fluorescent imaging, and verify the analytical calculations with numerical simulation for a special case where nonlinearity is produced by photoswitching of fluorescent labels. We compare the performance of two alternative implementations, using either two‐dimensional (2D) illumination patterns or sequentially rotated one‐dimensional (ID) patterns. We show that 1D patterns are advantageous in the linear case, and that in the nonlinear case 2D patterns provide a slight signal‐to‐noise advantage under idealised conditions, but perform worse than 1D patterns in the presence of nonswitchable fluorescent background.
Lay Description Structured‐illumination microscopy (SIM) is a high‐resolution light microscopy technique that allows imaging of fluorescence at a resolution about twice the classical diffraction limit. There are various ways that the illumination can be structured, but it is not obvious how the choice of illumination pattern affects the final image quality, especially in view of the noise. We present a detailed performance analysis considering two illumination techniques: sequential illumination with line‐gratings that are shifted and rotated during image acquisition and two‐dimensional (2D) illumination structures requiring only shift operations. Our analysis is based on analytical theory, supported by simulations of images considering noise. We also extend our analysis to a nonlinear variant of SIM, with which enhanced resolution can be achieved, limited only by noise. This includes nonlinear SIM based on the light‐induced switching of the fluorescent molecules between a bright and a dark state. We find sequential illumination with line‐gratings to be advantageous in ordinary (linear) SIM, whereas 2D patterns provides a slight signal‐to‐noise advantage under idealised conditions in nonlinear SIM if there is no nonswitching background.
-
We present a tomographic imaging technique, termed Deep Prior Diffraction Tomography (DP-DT), to reconstruct the 3D refractive index (RI) of thick biological samples at high resolution from a sequence of low-resolution images collected under angularly varying illumination. DP-DT processes the multi-angle data using a phase retrieval algorithm that is extended by a deep image prior (DIP), which reparameterizes the 3D sample reconstruction with an untrained, deep generative 3D convolutional neural network (CNN). We show that DP-DT effectively addresses the missing cone problem, which otherwise degrades the resolution and quality of standard 3D reconstruction algorithms. As DP-DT does not require pre-captured data or pre-training, it is not biased towards any particular dataset. Hence, it is a general technique that can be applied to a wide variety of 3D samples, including scenarios in which large datasets for supervised training would be infeasible or expensive. We applied DP-DT to obtain 3D RI maps of bead phantoms and complex biological specimens, both in simulation and experiment, and show that DP-DT produces higher-quality results than standard regularization techniques. We further demonstrate the generality of DP-DT, using two different scattering models, the first Born and multi-slice models. Our results point to the potential benefits of DP-DT for other 3D imaging modalities, including X-ray computed tomography, magnetic resonance imaging, and electron microscopy.
-
Abstract Mapping 3D plasma membrane topology in live cells can bring unprecedented insights into cell biology. Widefield-based super-resolution methods such as 3D-structured illumination microscopy (3D-SIM) can achieve twice the axial ( ~ 300 nm) and lateral ( ~ 100 nm) resolution of widefield microscopy in real time in live cells. However, twice-resolution enhancement cannot sufficiently visualize nanoscale fine structures of the plasma membrane. Axial interferometry methods including fluorescence light interference contrast microscopy and its derivatives (e.g., scanning angle interference microscopy) can determine nanoscale axial locations of proteins on and near the plasma membrane. Thus, by combining super-resolution lateral imaging of 2D-SIM with axial interferometry, we developed multi-angle-crossing structured illumination microscopy (MAxSIM) to generate multiple incident angles by fast, optoelectronic creation of diffraction patterns. Axial localization accuracy can be enhanced by placing cells on a bottom glass substrate, locating a custom height-controlled mirror (HCM) at a fixed axial position above the glass substrate, and optimizing the height reconstruction algorithm for noisy experimental data. The HCM also enables imaging of both the apical and basal surfaces of a cell. MAxSIM with HCM offers high-fidelity nanoscale 3D topological mapping of cell plasma membranes with near-real-time ( ~ 0.5 Hz) imaging of live cells and 3D single-molecule tracking.
-
Background Structured illumination microscopy (SIM) is a family of methods in optical fluorescence microscopy that can achieve both optical sectioning and super-resolution effects. SIM is a valuable method for high-resolution imaging of fixed cells or tissues labeled with conventional fluorophores, as well as for imaging the dynamics of live cells expressing fluorescent protein constructs. In SIM, one acquires a set of images with shifting illumination patterns. This set of images is subsequently treated with image analysis algorithms to produce an image with reduced out-of-focus light (optical sectioning) and/or with improved resolution (super-resolution). Findings Five complete, freely available SIM datasets are presented including raw and analyzed data. We report methods for image acquisition and analysis using open-source software along with examples of the resulting images when processed with different methods. We processed the data using established optical sectioning SIM and super-resolution SIM methods and with newer Bayesian restoration approaches that we are developing. Conclusions Various methods for SIM data acquisition and processing are actively being developed, but complete raw data from SIM experiments are not typically published. Publically available, high-quality raw data with examples of processed results will aid researchers when developing new methods in SIM. Biologists will also find interest in the high-resolution images of animal tissues and cells we acquired. All of the data were processed with SIMToolbox, an open-source and freely available software solution for SIM.more » « less