More Like this

1. PSSR2: a user-friendly Python package for democratizing deep learning-based point-scanning super-resolution microscopy

   https://doi.org/10.1186/s44330-024-00020-5  

   Stites, Hayden_C; Manor, Uri (January 2025, BMC Methods)

   Abstract BackgroundTo address the limitations of large-scale high quality microscopy image acquisition, PSSR (Point-Scanning Super-Resolution) was introduced to enhance easily acquired low quality microscopy data to a higher quality using deep learning-based methods. However, while PSSR was released as open-source, it was difficult for users to implement into their workflows due to an outdated codebase, limiting its usage by prospective users. Additionally, while the data enhancements provided by PSSR were significant, there was still potential for further improvement. MethodsTo overcome this, we introduce PSSR2, a redesigned implementation of PSSR workflows and methods built to put state-of-the-art technology into the hands of the general microscopy and biology research community. PSSR2 enables user-friendly implementation of super-resolution workflows for simultaneous super-resolution and denoising of undersampled microscopy data, especially through its integrated Command Line Interface and Napari plugin. PSSR2 improves and expands upon previously established PSSR algorithms, mainly through improvements in the semi-synthetic data generation (“crappification”) and training processes. ResultsIn benchmarking PSSR2 on a test dataset of paired high and low resolution electron microscopy images, PSSR2 super-resolves high-resolution images from low-resolution images to a significantly higher accuracy than PSSR. The super-resolved images are also more visually representative of real-world high-resolution images. DiscussionThe improvements in PSSR2, in providing higher quality images, should improve the performance of downstream analyses. We note that for accurate super-resolution, PSSR2 models should only be applied to super-resolve data sufficiently similar to training data and should be validated against real-world ground truth data. 

   more » « less
2. Deep learning 2D and 3D optical sectioning microscopy using cross-modality Pix2Pix cGAN image translation

   https://doi.org/10.1364/BOE.439894  

   Zhuge, Huimin; Summa, Brian; Hamm, Jihun; Brown, J_Quincy (November 2021, Biomedical Optics Express)

   Structured illumination microscopy (SIM) reconstructs optically-sectioned images of a sample from multiple spatially-patterned wide-field images, but the traditional single non-patterned wide-field images are more inexpensively obtained since they do not require generation of specialized illumination patterns. In this work, we translated wide-field fluorescence microscopy images to optically-sectioned SIM images by a Pix2Pix conditional generative adversarial network (cGAN). Our model shows the capability of both 2D cross-modality image translation from wide-field images to optical sections, and further demonstrates potential to recover 3D optically-sectioned volumes from wide-field image stacks. The utility of the model was tested on a variety of samples including fluorescent beads and fresh human tissue samples. 

   more » « less
3. Rotation equivariant and invariant neural networks for microscopy image analysis

   https://doi.org/10.1093/bioinformatics/btz353  

   Chidester, Benjamin; Zhou, Tianming; Do, Minh_N; Ma, Jian (July 2019, Bioinformatics)

   Abstract MotivationNeural networks have been widely used to analyze high-throughput microscopy images. However, the performance of neural networks can be significantly improved by encoding known invariance for particular tasks. Highly relevant to the goal of automated cell phenotyping from microscopy image data is rotation invariance. Here we consider the application of two schemes for encoding rotation equivariance and invariance in a convolutional neural network, namely, the group-equivariant CNN (G-CNN), and a new architecture with simple, efficient conic convolution, for classifying microscopy images. We additionally integrate the 2D-discrete-Fourier transform (2D-DFT) as an effective means for encoding global rotational invariance. We call our new method the Conic Convolution and DFT Network (CFNet). ResultsWe evaluated the efficacy of CFNet and G-CNN as compared to a standard CNN for several different image classification tasks, including simulated and real microscopy images of subcellular protein localization, and demonstrated improved performance. We believe CFNet has the potential to improve many high-throughput microscopy image analysis applications. Availability and implementationSource code of CFNet is available at: https://github.com/bchidest/CFNet. Supplementary informationSupplementary data are available at Bioinformatics online. 

   more » « less
4. Community-developed checklists for publishing images and image analyses

   https://doi.org/10.1038/s41592-023-01987-9  

   Schmied, Christopher; Nelson, Michael S.; Avilov, Sergiy; Bakker, Gert-Jan; Bertocchi, Cristina; Bischof, Johanna; Boehm, Ulrike; Brocher, Jan; Carvalho, Mariana T.; Chiritescu, Catalin; et al (September 2023, Nature Methods)

   Images document scientific discoveries and are prevalent in modern biomedical research. Microscopy imaging in particular is currently undergoing rapid technological advancements. However, for scientists wishing to publish obtained images and image-analysis results, there are currently no unified guidelines for best practices. Consequently, microscopy images and image data in publications may be unclear or difficult to interpret. Here, we present community-developed checklists for preparing light microscopy images and describing image analyses for publications. These checklists offer authors, readers and publishers key recommendations for image formatting and annotation, color selection, data availability and reporting image-analysis workflows. The goal of our guidelines is to increase the clarity and reproducibility of image figures and thereby to heighten the quality and explanatory power of microscopy data. 

   more » « less
5. Automated Registration and Clustering for Enhanced Localization Atomic Force Microscopy of Flexible Membrane Proteins

   https://doi.org/10.1101/2025.06.30.662259  

   Lisowski, Creighton M; King, Gavin M; Kosztin, Ioan (July 2025, bioRxiv)

   Atomic Force Microscopy (AFM) can create images of biomolecules under near-native conditions but suffers from limited lateral resolution due to the finite AFM tip size and recording frequency. The recently developed Localization Atomic Force Microscopy or LAFM (Heath et al., Nature 594, 385 (2021)) enhances lateral resolution by reconstructing peak positions in AFM image stacks, but it is less effective for flexible proteins with multiple conformations. Here we introduce an unsupervised deep learning algorithm that simultaneously registers and clusters images by protein conformation, thus making LAFM applicable to more flexible proteins. Using simulated AFM images from molecular dynamics simulations of the SecYEG translocon as a model membrane protein system, we demonstrate improved resolution for individual protein conformations. This work represents a step towards a more general LAFM algorithm that can handle biological macromolecules with multiple distinct conformational states such as SecYEG. Author summaryAtomic Force Microscopy (AFM) enables high-resolution imaging of biomolecules under near-native conditions but faces lateral resolution limits due to the finite AFM tip size and recording frequency. The recently developed Localization Atomic Force Microscopy (LAFM) method addresses this by reconstructing peak positions from AFM image stacks, achieving almost atomic resolution for rigid proteins like bacteriorhodopsin (Heath et al., Nature 594, 385 (2021)). However, flexible membrane proteins with dynamic conformations, such as the SecYEG translocon, which exhibits large and highly mobile cytoplasmic loops, lead to non-physical smearing in standard LAFM reconstructions. Here, we present a computational framework combining unsupervised deep clustering and LAFM to enhance the lateral resolution of AFM images of flexible membrane proteins. Our neural network algorithm (i) groups AFM images into conformationally homogeneous clusters and (ii) registers images within each cluster. Applying LAFM separately to these clusters minimizes smearing artifacts, yielding high-resolution reconstructions for distinct conformations. We validate this approach using synthetic AFM images generated from all-atom molecular dynamics simulations of SecYEG in a solvated POPE lipid bilayer. This advancement extends LAFM’s utility to encompass conformationally diverse membrane proteins. 

   more » « less