More Like this

1. Transferring Implicit Knowledge of Non-Visual Object Properties Across Heterogeneous Robot Morphologies

   https://doi.org/10.1109/ICRA48891.2023.10160811  

   Tatiya G.; Francis J.; and Sinapov, J. (May 2023, IEEE International Conference on Robotics and Automation)

   Abstract— Humans leverage multiple sensor modalities when interacting with objects and discovering their intrinsic properties. Using the visual modality alone is insufficient for deriving intuition behind object properties (e.g., which of two boxes is heavier), making it essential to consider non-visual modalities as well, such as the tactile and auditory. Whereas robots may leverage various modalities to obtain object property understanding via learned exploratory interactions with objects (e.g., grasping, lifting, and shaking behaviors), challenges remain: the implicit knowledge acquired by one robot via object exploration cannot be directly leveraged by another robot with different morphology, because the sensor models, observed data distributions, and interaction capabilities are different across these different robot configurations. To avoid the costly process of learning interactive object perception tasks from scratch, we propose a multi-stage projection framework for each new robot for transferring implicit knowledge of object properties across heterogeneous robot morphologies. We evaluate our approach on the object-property recognition and object-identity recognition tasks, using a dataset containing two heterogeneous robots that perform 7,600 object interactions. Results indicate that knowledge can be transferred across robots, such that a newly-deployed robot can bootstrap its recognition models without exhaustively exploring all objects. We also propose a data augmentation technique and show that this technique improves the generalization of models. We release code, datasets, and additional results, here: https: //github.com/gtatiya/Implicit-Knowledge-Transfer. 

   more » « less
2. Acousto-optic ptychography

   https://doi.org/10.1364/OPTICA.424828  

   Rosenfeld, Moriya; Weinberg, Gil; Doktofsky, Daniel; Li, Yunzhe; Tian, Lei; Katz, Ori (June 2021, Optica)

   Acousto-optic imaging (AOI) enables optical-contrast imaging deep inside scattering samples via localized ultrasound-modulation of scattered light. While AOI allows optical investigations at depths, its imaging resolution is inherently limited by the ultrasound wavelength, prohibiting microscopic investigations. Here, we propose a computational imaging approach that allows optical diffraction-limited imaging using a conventional AOI system. We achieve this by extracting diffraction-limited imaging information from speckle correlations in the conventionally detected ultrasound-modulated scattered-light fields. Specifically, we identify that since “memory-effect” speckle correlations allow estimation of the Fourier magnitude of the field inside the ultrasound focus, scanning the ultrasound focus enables robust diffraction-limited reconstruction of extended objects using ptychography (i.e., we exploit the ultrasound focus as the scanned spatial-gate probe required for ptychographic phase retrieval). Moreover, we exploit the short speckle decorrelation-time in dynamic media, which is usually considered a hurdle for wavefront-shaping- based approaches, for improved ptychographic reconstruction. We experimentally demonstrate noninvasive imaging of targets that extend well beyond the memory-effect range, with a 40-times resolution improvement over conventional AOI. 

   more » « less
3. A framework for glass-box physics rule learner and its application to nano-scale phenomena

   https://doi.org/10.1038/s42005-020-0339-x  

   Cho, In Ho; Li, Qiang; Biswas, Rana; Kim, Jaeyoun (May 2020, Communications Physics)

   Abstract Attempts to use machine learning to discover hidden physical rules are in their infancy, and such attempts confront more challenges when experiments involve multifaceted measurements over three-dimensional objects. Here we propose a framework that can infuse scientists’ basic knowledge into a glass-box rule learner to extract hidden physical rules behind complex physics phenomena. A “convolved information index” is proposed to handle physical measurements over three-dimensional nano-scale specimens, and the multi-layered convolutions are “externalized” over multiple depths at the information level, not in the opaque networks. A transparent, flexible link function is proposed as a mathematical expression generator, thereby pursuing “glass-box” prediction. Consistent evolution is realized by integrating a Bayesian update and evolutionary algorithms. The framework is applied to nano-scale contact electrification phenomena, and results show promising performances in unraveling transparent expressions of a hidden physical rule. The proposed approach will catalyze a synergistic machine learning-physics partnership. 

   more » « less
4. Optofluidic ptychography on a chip

   https://doi.org/10.1039/D1LC00719J  

   Song, Pengming; Guo, Chengfei; Jiang, Shaowei; Wang, Tianbo; Hu, Patrick; Hu, Derek; Zhang, Zibang; Feng, Bin; Zheng, Guoan (November 2021, Lab on a Chip)

   We report the implementation of a fully on-chip, lensless microscopy technique termed optofluidic ptychography. This imaging modality complements the miniaturization provided by microfluidics and allows the integration of ptychographic microscopy into various lab-on-a-chip devices. In our prototype, we place a microfluidic channel on the top surface of a coverslip and coat the bottom surface with a scattering layer. The channel and the coated coverslip substrate are then placed on top of an image sensor for diffraction data acquisition. Similar to the operation of a flow cytometer, the device utilizes microfluidic flow to deliver specimens across the channel. The diffracted light from the flowing objects is modulated by the scattering layer and recorded by the image sensor for ptychographic reconstruction, where high-resolution quantitative complex images are recovered from the diffraction measurements. By using an image sensor with a 1.85 μm pixel size, our device can resolve the 550 nm linewidth on the resolution target. We validate the device by imaging different types of biospecimens, including C. elegans , yeast cells, paramecium , and closterium sp . We also demonstrate a high-resolution ptychographic reconstruction at a video framerate of 30 frames per second. The reported technique can address a wide range of biomedical needs and engenders new ptychographic imaging innovations in a flow cytometer configuration. 

   more » « less
5. Localization and coherent imaging of hidden moving objects using laser speckle

   https://doi.org/10.1364/OL.549456  

   Hastings, Ryan_L; Webb, Kevin_J (February 2025, Optics Letters)

   Imaging and sensing of moving objects through opaque scattering media is a challenging but important problem in a variety of applications, including environmental sensing, biomedical imaging, and material inspection. We have previously demonstrated a technique to coherently image a moving object through thick, heavily scattering random media using correlations of speckle images as a function of the object’s spatial translation. Here, we demonstrate that this technique can be combined with localization to achieve imaging without prior knowledge of the object’s motion, greatly extending the application domain. This method is effective beyond the thin or weakly scattering regime and, rather than motion being deleterious, exploits the information available when the hidden object is moving, as could be the case in a cluttered terrestrial environment or through substantial levels of biological tissue scatter. 

   more » « less