skip to main content

This content will become publicly available on May 14, 2023

Title: Real-Time Camera Localization during Robot-Assisted Telecystoscopy for Bladder Cancer Surveillance
Telecystoscopy can lower the barrier to access critical urologic diagnostics for patients around the world. A major challenge for robotic control of flexible cystoscopes and intuitive teleoperation is the pose estimation of the scope tip. We propose a novel real-time camera localization method using video recordings from a prior cystoscopy and 3D bladder reconstruction to estimate cystoscope pose within the bladder during follow-up telecystoscopy. We map prior video frames into a low-dimensional space as a dictionary so that a new image can be likewise mapped to efficiently retrieve its nearest neighbor among the dictionary images. The cystoscope pose is then estimated by the correspondence among the new image, its nearest dictionary image, and the prior model from 3D reconstruction. We demonstrate performance of our methods using bladder phantoms with varying fidelity and a servo-controlled cystoscope to simulate the use case of bladder surveillance through telecystoscopy. The servo-controlled cystoscope with 3 degrees of freedom (angulation, roll, and insertion axes) was developed for collecting cystoscope videos from bladder phantoms. Cystoscope videos were acquired in a 2.5D bladder phantom (bladder-shape cross-section plus height) with a panorama of a urothelium attached to the inner surface. Scans of the 2.5D phantom were performed in separate more » arc trajectories each of which is generated by actuation on the angulation with a fixed roll and insertion length. We further included variance in moving speed, imaging distance and existence of bladder tumors. Cystoscope videos were also acquired in a water-filled 3D silicone bladder phantom with hand-painted vasculature. Scans of the 3D phantom were performed in separate circle trajectories each of which is generated by actuation on the roll axis under a fixed angulation and insertion length. These videos were used to create 3D reconstructions, dictionary sets, and test data sets for evaluating the computational efficiency and accuracy of our proposed method in comparison with a method based on global Scale-Invariant Feature Transform (SIFT) features, named SIFT-only. Our method can retrieve the nearest dictionary image for 94–100% of test frames in under 55[Formula: see text]ms per image, whereas the SIFT-only method can only find the image match for 56–100% of test frames in 6000–40000[Formula: see text]ms per image depending on size of the dictionary set and richness of SIFT features in the images. Our method, with a speed of around 20 Hz for the retrieval stage, is a promising tool for real-time image-based scope localization in robotic cystoscopy when prior cystoscopy images are available. « less
Authors:
; ; ; ; ; ; ;
Award ID(s):
1631146
Publication Date:
NSF-PAR ID:
10331716
Journal Name:
Journal of Medical Robotics Research
ISSN:
2424-905X
Sponsoring Org:
National Science Foundation
More Like this
  1. A body of studies has proposed to obtain high-quality images from low-dose and noisy Computed Tomography (CT) scans for radiation reduction. However, these studies are designed for population-level data without considering the variation in CT devices and individuals, limiting the current approaches' performance, especially for ultra-low-dose CT imaging. Here, we proposed PIMA-CT, a physical anthropomorphic phantom model integrating an unsupervised learning framework, using a novel deep learning technique called Cyclic Simulation and Denoising (CSD), to address these limitations. We first acquired paired low-dose and standard-dose CT scans of the phantom and then developed two generative neural networks: noise simulator and denoiser. The simulator extracts real low-dose noise and tissue features from two separate image spaces (e.g., low-dose phantom model scans and standard-dose patient scans) into a unified feature space. Meanwhile, the denoiser provides feedback to the simulator on the quality of the generated noise. In this way, the simulator and denoiser cyclically interact to optimize network learning and ease the denoiser to simultaneously remove noise and restore tissue features. We thoroughly evaluate our method for removing both real low-dose noise and Gaussian simulated low-dose noise. The results show that CSD outperforms one of the state-of-the-art denoising algorithms without using anymore »labeled data (actual patients' low-dose CT scans) nor simulated low-dose CT scans. This study may shed light on incorporating physical models in medical imaging, especially for ultra-low level dose CT scans restoration.« less
  2. The instability and transition to turbulence and its evolution in pulsatile flows, which involve reverse flows and unsteady flow separations, is the primary focus of this experimental work. A piston driven by a programmable DC servo motor was used to set-up a water flow system and provide the pulsation characteristics. Time-resolved particle image velocimetry data were acquired in a refractive index matching set-up by using a continuous wave laser and a high-frame-rate digital camera. The position of the piston was continuously recorded by a laser proximity sensor. Five different experiments were carried out with Reynolds numbers in the range of 535–4825 and Womersley numbers from 11.91 to 23.82. The non-stationarity of the data was addressed by incorporating trend removal methods involving low- and high-pass filtering of the data, and using empirical mode decomposition together with the relevant Hilbert–Huang transform to determine the intrinsic mode functions. This latter method is more appropriate for nonlinear and non-stationary cases, for which traditional analysis involving classical Fourier decomposition is not directly applicable. It was found that transition to turbulence is a spontaneous event covering the whole near-wall region. The instantaneous vorticity profiles show the development of a large-scale ring-like attached wall vortical layer (WVL)more »with smaller vortices of higher frequencies than the pulsation frequency superimposed, which point to a shear layer Kelvin–Helmholtz (K–H) type of instability. Inflectional instability leads to flow separation and the formation of a major roll-up structure with the K–H vortices superimposed. This structure breaks down in the azimuthal direction into smaller turbulence patches with vortical content, which appears to be the prevailing structural content of the flow at each investigated Reynolds number ( Re ). At higher Re numbers, the strength and extent of the vortices are larger and substantial disturbances appear in the free stream region of the flow, which are typical of pipe flows at transitional Re numbers. Turbulence appears to be produced at the locations of maximum or minimum vorticity within the attached WVL, in the ridges between the K–H vortices around the separated WVL and the upstream side of the secondary vortex where the flow impinges on the wall. This wall turbulence breaks away into the middle section of the pipe, at approximately $Re \ge 2200$ , by strong eruptions of the K–H vortices.« less
  3. The instability and transition to turbulence and its evolution in pulsatile flows, which involve reverse flows and unsteady flow separations, is the primary focus of this experimental work. A piston driven by a programmable DC servo motor was used to set-up a water flow system and provide the pulsation characteristics. Time-resolved particle image velocimetry data were acquired in a refractive index matching set-up by using a continuous wave laser and a high-frame-rate digital camera. The position of the piston was continuously recorded by a laser proximity sensor. Five different experiments were carried out with Reynolds numbers in the range of 535–4825 and Womersley numbers from 11.91 to 23.82. The non-stationarity of the data was addressed by incorporating trend removal methods involving low- and high-pass filtering of the data, and using empirical mode decomposition together with the relevant Hilbert–Huang transform to determine the intrinsic mode functions. This latter method is more appropriate for nonlinear and non-stationary cases, for which traditional analysis involving classical Fourier decomposition is not directly applicable. It was found that transition to turbulence is a spontaneous event covering the whole near-wall region. The instantaneous vorticity profiles show the development of a large-scale ring-like attached wall vortical layer (WVL)more »with smaller vortices of higher frequencies than the pulsation frequency superimposed, which point to a shear layer Kelvin–Helmholtz (K–H) type of instability. Inflectional instability leads to flow separation and the formation of a major roll-up structure with the K–H vortices superimposed. This structure breaks down in the azimuthal direction into smaller turbulence patches with vortical content, which appears to be the prevailing structural content of the flow at each investigated Reynolds number (Re). At higher Re numbers, the strength and extent of the vortices are larger and substantial disturbances appear in the free stream region of the flow, which are typical of pipe flows at transitional Re numbers. Turbulence appears to be produced at the locations of maximum or minimum vorticity within the attached WVL, in the ridges between the K–H vortices around the separated WVL and the upstream side of the secondary vortex where the flow impinges on the wall. This wall turbulence breaks away into the middle section of the pipe, at approximately Re ≥ 2200, by strong eruptions of the K–H vortices.« less
  4. SUMMARY An efficient method for tracking a target using a single Pan-Tilt-Zoom (PTZ) camera is proposed. The proposed Scale-Invariant Optical Flow (SIOF) method estimates the motion of the target and rotates the camera accordingly to keep the target at the center of the image. Also, SIOF estimates the scale of the target and changes the focal length relatively to adjust the Field of View (FoV) and keep the target appear in the same size in all captured frames. SIOF is a feature-based tracking method. Feature points used are extracted and tracked using Optical Flow (OF) and Scale-Invariant Feature Transform (SIFT). They are combined in groups and used to achieve robust tracking. The feature points in these groups are used within a twist model to recover the 3D free motion of the target. The merits of this proposed method are (i) building an efficient scale-invariant tracking method that tracks the target and keep it in the FoV of the camera with the same size, and (ii) using tracking with prediction and correction to speed up the PTZ control and achieve smooth camera control. Experimental results were performed on online video streams and validated the efficiency of the proposed method SIOF, comparingmore »with OF, SIFT, and other tracking methods. The proposed SIOF has around 36% less average tracking error and around 70% less tracking overshoot than OF.« less
  5. In this paper, we propose a novel strep throat detection method using a smartphone with an add-on gadget. Our smartphone-based strep throat detection method is based on the use of camera and flashlight embedded in a smartphone. The proposed algorithm acquires throat image using a smartphone with a gadget, processes the acquired images using color transformation and color correction algorithms, and finally classifies streptococcal pharyngitis (or strep) throat from healthy throat using machine learning techniques. Our developed gadget was designed to minimize the reflection of light entering the camera sensor. The scope of this paper is confined to binary classification between strep and healthy throats. Specifically, we adopted k-fold validation technique for classification, which finds the best decision boundary from training and validation sets and applies the acquired best decision boundary to the test sets. Experimental results show that our proposed detection method detects strep throats with 93.75% accuracy, 88% specificity, and 87.5% sensitivity on average.