Cameras are increasingly being deployed in cities,
enterprises and roads world-wide to enable many applications
in public safety, intelligent transportation, retail, healthcare and
manufacturing. Often, after initial deployment of the cameras, the
environmental conditions and the scenes around these cameras
change, and our experiments show that these changes can
adversely impact the accuracy of insights from video analytics.
This is because the camera parameter settings, though optimal
at deployment time, are not the best settings for good-quality
video capture as the environmental conditions and scenes around
a camera change during operation. Capturing poor-quality video
adversely affects the accuracy of analytics. To mitigate the loss in
accuracy of insights, we propose a novel, reinforcement-learning
based system APT that dynamically, and remotely (over 5G
networks), tunes the camera parameters, to ensure a high-quality
video capture, which mitigates any loss in accuracy of video
analytics. As a result, such tuning restores the accuracy of
insights when environmental conditions or scene content change.
APT uses reinforcement learning, with no-reference perceptual
quality estimation as the reward function. We conducted extensive
real-world experiments, where we simultaneously deployed two
cameras side-by-side overlooking an enterprise parking lot (one
camera only has manufacturer-suggested default setting, while the
other camera is dynamically tuned by APT during operation). Our
experiments demonstrated that due to dynamic tuning by APT,
the analytics insights are consistently better at all times of the
day: the accuracy of object detection video analytics application
was improved on average by ∼ 42%. Since our reward function
is independent of any analytics task, APT can be readily used
for different video analytics tasks.
more »
« less
Good-Eye: A Combined Computer-Vision and Physiological-Sensor based Device for Full-Proof Prediction and Detection of Fall of Adults
It is imperative to find the most accurate way to detect falls in elders to help mitigate the disastrous effects of such unfortunate injuries. In order to mitigate fall related accidents, we propose the Good-Eye System, an Internet of Things (IoT) enabled Edge Level Device which works when there is an orientation change detected by camera, and monitors physiological signal parameters. If the observed change is greater than the set threshold, the user is notified with information regarding a prediction of fall or a detection of fall, using LED lights. The Good-Eye System has a remote wall attached camera to monitor continuously the subject as long as the person is in a room along with a camera attached to a wearable to increase the accuracy of the model. The observed accuracy of the Good-Eye System as a whole is approximately 95%.
more »
« less
- Award ID(s):
- 1924112
- NSF-PAR ID:
- 10158167
- Date Published:
- Journal Name:
- Proceedings of the 2nd IFIP International Internet of Things (IoT) Conference (IFIP-IoT)
- Page Range / eLocation ID:
- 273-288
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
We present a first-of-its-kind ultra-compact intelligent camera system, dubbed i-FlatCam, including a lensless camera with a computational (Comp.) chip. It highlights (1) a predict-then-focus eye tracking pipeline for boosted efficiency without compromising the accuracy, (2) a unified compression scheme for single-chip processing and improved frame rate per second (FPS), and (3) dedicated intra-channel reuse design for depth-wise convolutional layers (DW-CONV) to increase utilization. i-FlatCam demonstrates the first eye tracking pipeline with a lensless camera and achieves 3.16 degrees of accuracy, 253 FPS, 91.49 µJ/Frame, and 6.7mm×8.9mm×1.2mm camera form factor, paving the way for next-generation Augmented Reality (AR) and Virtual Reality (VR) devices.more » « less
-
more » « less
This paper presents a novel technique to achieve autofocusing for a three-dimensional (3D) profilometry system with dual projectors. The proposed system uses a camera that is attached with an electronically focus-tunable lens (ETL) that allows dynamic change of camera’s focal plane such that the camera can focus on the object; the camera captures fringe patterns projected by each projector to establish corresponding points between two projectors, and two pre-calibrated projectors form triangulation for 3D reconstruction. We pre-calibrate the relationship between the depth and the current being used for each focal plane, perform a 3D shape measurement with an unknown focus level, and calculate the desired current value based on the initial 3D result. We developed a prototype system that can automatically focus on an object positioned between 450 mm to 850 mm.
-
Algorithms for the estimation of gaze direction from mobile and video-based eye trackers typically involve tracking a feature of the eye that moves through the eye camera image in a way that covaries with the shifting gaze direction, such as the center or boundaries of the pupil. Tracking these features using traditional computer vision techniques can be difficult due to partial occlusion and environmental reflections. Although recent efforts to use machine learning (ML) for pupil tracking have demonstrated superior results when evaluated using standard measures of segmentation performance, little is known of how these networks may affect the quality of the final gaze estimate. This work provides an objective assessment of the impact of several contemporary ML-based methods for eye feature tracking when the subsequent gaze estimate is produced using either feature-based or model-based methods. Metrics include the accuracy and precision of the gaze estimate, as well as drop-out rate.more » « less
-
more » « less
Active feedback control in magnetic confinement fusion devices is desirable to mitigate plasma instabilities and enable robust operation. Optical high-speed cameras provide a powerful, non-invasive diagnostic and can be suitable for these applications. In this study, we process high-speed camera data, at rates exceeding 100 kfps, on in situ field-programmable gate array (FPGA) hardware to track magnetohydrodynamic (MHD) mode evolution and generate control signals in real time. Our system utilizes a convolutional neural network (CNN) model, which predicts the n = 1 MHD mode amplitude and phase using camera images with better accuracy than other tested non-deep-learning-based methods. By implementing this model directly within the standard FPGA readout hardware of the high-speed camera diagnostic, our mode tracking system achieves a total trigger-to-output latency of 17.6 μs and a throughput of up to 120 kfps. This study at the High Beta Tokamak-Extended Pulse (HBT-EP) experiment demonstrates an FPGA-based high-speed camera data acquisition and processing system, enabling application in real-time machine-learning-based tokamak diagnostic and control as well as potential applications in other scientific domains.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/https://doi.org/10.1007/978-3-030-43605-6_16
