An official website of the United States government
The pixel-wise code exposure (PCE) camera is a compressive sensing camera that has several advantages, such as low power consumption and high compression ratio.Moreover, one notable advantage is the capability to control individual pixel exposure time. Conventional approaches of using PCE cameras involve a time-consuming and lossy process to reconstruct the original frames and then use those frames for target tracking and classification. Otherwise, conventional approaches will fail if compressive measurements are used. In this paper, we present a deep learning approach that directly performs target tracking and classification in the compressive measurement domain without any frame reconstruction. Our approach has two parts: tracking and classification. The tracking has been done via detection using You Only Look Once (YOLO), and the classification is achieved using residual network (ResNet). Extensive simulations using short-wave infrared (SWIR) videos demonstrated the efficacy of our proposed approach.
more »
« less
Detection and Confirmation of Multiple Human Targets Using Pixel-Wise Code Aperture Measurements
Compressive video measurements can save bandwidth and data storage. However, conventional approaches to target detection require the compressive measurements to be reconstructed before any detectors are applied. This is not only time consuming but also may lose information in the reconstruction process. In this paper, we summarized the application of a recent approach to vehicle detection and classification directly in the compressive measurement domain to human targets. The raw videos were collected using a pixel-wise code exposure (PCE) camera, which condensed multiple frames into one frame. A combination of two deep learning-based algorithms (you only look once (YOLO) and residual network (ResNet)) was used for detection and confirmation. Optical and mid-wave infrared (MWIR) videos from a well-known database (SENSIAC) were used in our experiments. Extensive experiments demonstrated that the proposed framework was feasible for target detection up to 1500 m, but target confirmation needs more research.
more »
« less
- Award ID(s):
- 1824198
- PAR ID:
- 10189738
- Date Published:
- Journal Name:
- Journal of Imaging
- Volume:
- 6
- Issue:
- 6
- ISSN:
- 2313-433X
- Page Range / eLocation ID:
- 40
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
This paper develops a cost-effective vehicle detection and tracking system based on fusion of a 2-D LIDAR and a monocular camera to protect electric micromobility devices, especially e-scooters, by predicting the real- time danger of a car- scooter collision. The cost and size disadvantages of 3-D LIDAR sensors make them an unsuitable choice for micromobility devices. Therefore, a 2-D RPLIDAR Mapper sensor is used. Although low-cost, this sensor comes with major shortcomings such as the narrow vertical field of view and its low density of data points. Due to these factors, the sensor does not have a robust output in outdoor applications, and the measurements keep jumping and sliding on the vehicle surface. To improve the performance of the LIDAR, a single monocular camera is fused with the LIDAR data not only to detect vehicles, but also to separately detect the front and side of a target vehicle and to find its corner. It is shown that this corner detection method is more accurate than strategies that are only based on the LIDAR data. The corner measurements are used in a high-gain observer to estimate the location, velocity, and orientation of the target vehicle. The developed system is implemented on a Ninebot e-scooter platform, and multiple experiments are performed to evaluate the performance of the algorithm.more » « less
-
Refracto-vibrometry is a technique that uses a laser Doppler vibrometer to measure acoustic pressure fields. The vibrometer laser is directed through a medium towards a stationary retroreflective surface. Acoustic waves (density variations) for which the wavefronts pass through the laser, as the beam travels from the vibrometer to the retroreflector and back, cause variations in the integrated optical path length. This results in a time-varying modulation of the laser signal returning to the vibrometer, enabling optical detection of the acoustic wavefronts. In the current experiment, a Polytec PSV-400 scanning laser Doppler vibrometer, sampled at 100 MHz, monitored the waves emitted by a 1 MHz Panametrics V303 ultrasound transducer immersed in a water tank. The time-varying signal detected by the vibrometer at numerous scan points was used to generate videos of the time evolution of acoustic wavefronts; these videos will be presented. Refracto-vibrometry was also used for optical measurements of the time of flight of ultrasonic waves through different materials, including samples of lead and fabricated bone. This enabled determination of wave propagation speeds. The wave speeds obtained with optical detection using refracto-vibrometry were in agreement with measurements using a conventional ultrasonic transducer to detect the wavefronts.more » « less
-
Many online learning platforms and MOOCs incorporate some amount of video-based content into their platform, but there are few randomized controlled experiments that evaluate the effective- ness of the different methods of video integration. Given the large amount of publicly available educational videos, an investigation into this content’s impact on students could help lead to more ef- fective and accessible video integration within learning platforms. In this work, a new feature was added into an existing online learn- ing platform that allowed students to request skill-related videos while completing their online middle-school mathematics assign- ments. A total of 18,535 students participated in two large-scale randomized controlled experiments related to providing students with publicly available educational videos. The first experiment investigated the effect of providing students with the opportunity to request these videos, and the second experiment investigated the effect of using a multi-armed bandit algorithm to recommend relevant videos. Additionally, this work investigated which features of the videos were significantly predictive of students’ performance and which features could be used to personalize students’ learning. Ultimately, students were mostly disinterested in the skill-related videos, preferring instead to use the platforms existing problem- specific support, and there was no statistically significant findings in either experiment. Additionally, while no video features were significantly predictive of students’ performance, two video fea- tures had significant qualitative interactions with students’ prior knowledge, which showed that different content creators were more effective for different groups of students. These findings can be used to inform the design of future video-based features within online learning platforms and the creation of different educational videos specifically targeting higher or lower knowledge students. The data and code used in this work is hosted by the Open Science Foundation.more » « less
-
Many online learning platforms and MOOCs incorporate some amount of video-based content into their platform, but there are few randomized controlled experiments that evaluate the effectiveness of the different methods of video integration. Given the large amount of publicly available educational videos, an investigation into this content's impact on students could help lead to more effective and accessible video integration within learning platforms. In this work, a new feature was added into an existing online learning platform that allowed students to request skill-related videos while completing their online middle-school mathematics assignments. A total of 18,535 students participated in two large-scale randomized controlled experiments related to providing students with publicly available educational videos. The first experiment investigated the effect of providing students with the opportunity to request these videos, and the second experiment investigated the effect of using a multi-armed bandit algorithm to recommend relevant videos. Additionally, this work investigated which features of the videos were significantly predictive of students' performance and which features could be used to personalize students' learning. Ultimately, students were mostly disinterested in the skill-related videos, preferring instead to use the platforms existing problem-specific support, and there was no statistically significant findings in either experiment. Additionally, while no video features were significantly predictive of students' performance, two video features had significant qualitative interactions with students' prior knowledge, which showed that different content creators were more effective for different groups of students. These findings can be used to inform the design of future video-based features within online learning platforms and the creation of different educational videos specifically targeting higher or lower knowledge students. The data and code used in this work can be found at https://osf.io/cxkzf/.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3390/jimaging6060040
