More Like this

1. Synchronized strobed phase contrast and fluorescence microscopy: the interlaced standard reimagined

   https://doi.org/10.1364/OE.474045  

   Hosu, Basarab G.; Hill, Winfield; Samuel, Aravinthan D.; Berg, Howard C. (January 2023, Optics Express)

   We propose a simple, cost-effective method for synchronized phase contrast and fluorescence video acquisition in live samples. Counter-phased pulses of phase contrast illumination and fluorescence excitation light are synchronized with the exposure of the two fields of an interlaced camera sensor. This results in a video sequence in which each frame contains both exposure modes, each in half of its pixels. The method allows real-time acquisition and display of synchronized and spatially aligned phase contrast and fluorescence image sequences that can be separated by de-interlacing in two independent videos. The method can be implemented on any fluorescence microscope with a camera port without needing to modify the optical path. 

   more » « less
2. ADARE-HD: Adaptive-Resolution Framework for Efficient Object Detection and Tracking via HD-Computing

   Mejri, M; Amarnath, C; Chatterjee, A (August 2024, IEEE)

   Efficient and low-energy camera signal processing is critical for battery-supported sensing and surveillance applications. In this research, we develop a video object detection and tracking framework which adaptively down-samples frame pixels to minimize computation and memory costs, and thereby the energy consumed, while maintaining a high level of accuracy. Instead of always operating with the highest sensor pixel resolution (compute-intensive), video frame (pixel) content is down-sampled spatially, to adapt to changing camera environments (size of object tracked, peak-signal-tonoise- ratio (i.e, PSNR) of video frames). Object detection and tracking is supported by a novel video resolution-aware adaptive hyperdimensional computing framework. This leverages a low memory overhead non-linear hypervector encoding scheme specifically tailored for handling multiple degrees of resolution. Previous classification decisions of a moving object based on its tracking label are used to improve tracking robustness. Energy savings of up to 1.6 orders of magnitude and up to an order of magnitude compute speedup is obtained on a range of experiments performed on benchmark systems. 

   more » « less
3. Comparison of super-resolution and noise reduction for passive single-photon imaging

   https://doi.org/10.1117/1.JEI.31.3.033042  

   Laurenzis, Martin; Seets, Trevor; Bacher, Emmanuel; Ingle, Atul; Velten, Andreas (May 2022, Journal of Electronic Imaging)

   Single-photon sensitive image sensors have recently gained popularity in passive imaging applications where the goal is to capture photon flux (brightness) values of different scene points in the presence of challenging lighting conditions and scene motion. Recent work has shown that high-speed bursts of single-photon timestamp information captured using a single-photon avalanche diode camera can be used to estimate and correct for scene motion thereby improving signal-to-noise ratio and reducing motion blur artifacts. We perform a comparison of various design choices in the processing pipeline used for noise reduction, motion compensation, and upsampling of single-photon timestamp frames. We consider various pixelwise noise reduction techniques in combination with state-of-the-art deep neural network upscaling algorithms to super-resolve intensity images formed with single-photon timestamp data. We explore the trade space of motion blur and signal noise in various scenes with different motion content. Using real data captured with a hardware prototype, we achieved superresolution reconstruction at frame rates up to 65.8 kHz (native sampling rate of the sensor) and captured videos of fast-moving objects. The best reconstruction is obtained with the motion compensation approach, which achieves a structural similarity (SSIM) of about 0.67 for fast moving rigid objects. We are able to reconstruct subpixel resolution. These results show the relative superiority of our motion compensation compared to other approaches that do not exceed an SSIM of 0.5. 

   more » « less
4. Real-time machine learning–enhanced hyperspectro-polarimetric imaging via an encoding metasurface

   https://doi.org/10.1126/sciadv.adp5192  

   Zhang, Lidan; Zhou, Chen; Liu, Bofeng; Ding, Yimin; Ahn, Hyun-Ju; Chang, Shengyuan; Duan, Yao; Rahman, Md Tarek; Xia, Tunan; Chen, Xi; et al (September 2024, Science Advances)

   Light fields carry a wealth of information, including intensity, spectrum, and polarization. However, standard cameras capture only the intensity, disregarding other valuable information. While hyperspectral and polarimetric imaging systems capture spectral and polarization information, respectively, in addition to intensity, they are often bulky, slow, and costly. Here, we have developed an encoding metasurface paired with a neural network enabling a normal camera to acquire hyperspectro-polarimetric images from a single snapshot. Our experimental results demonstrate that this metasurface-enhanced camera can accurately resolve full-Stokes polarization across a broad spectral range (700 to 1150 nanometer) from a single snapshot, achieving a spectral sensitivity as high as 0.23 nanometer. In addition, our system captures full-Stokes hyperspectro-polarimetric video in real time at a rate of 28 frames per second, primarily limited by the camera’s readout rate. Our encoding metasurface offers a compact, fast, and cost-effective solution for multidimensional imaging that effectively uses information within light fields. 

   more » « less
5. Passive Inter-Photon Imaging

   https://doi.org/10.1109/CVPR46437.2021.00848  

   Ingle, Atul; Seets, Trevor; Buttafava, Mauro; Gupta, Shantanu; Tosi, Alberto; Gupta, Mohit; Velten, Andreas (June 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR))

   Digital camera pixels measure image intensities by converting incident light energy into an analog electrical current, and then digitizing it into a fixed-width binary representation. This direct measurement method, while conceptually simple, suffers from limited dynamic range and poor performance under extreme illumination — electronic noise dominates under low illumination, and pixel full-well capacity results in saturation under bright illumination. We propose a novel intensity cue based on measuring inter-photon timing, defined as the time delay between detection of successive photons. Based on the statistics of inter-photon times measured by a time-resolved single-photon sensor, we develop theory and algorithms for a scene brightness estimator which works over extreme dynamic range; we experimentally demonstrate imaging scenes with a dynamic range of over ten million to one. The proposed techniques, aided by the emergence of single-photon sensors such as single-photon avalanche diodes (SPADs) with picosecond timing resolution, will have implications for a wide range of imaging applications: robotics, consumer photography, astronomy, microscopy and biomedical imaging. 

   more » « less