Cameras are increasingly being deployed in cities,
enterprises and roads world-wide to enable many applications
in public safety, intelligent transportation, retail, healthcare and
manufacturing. Often, after initial deployment of the cameras, the
environmental conditions and the scenes around these cameras
change, and our experiments show that these changes can
adversely impact the accuracy of insights from video analytics.
This is because the camera parameter settings, though optimal
at deployment time, are not the best settings for good-quality
video capture as the environmental conditions and scenes around
a camera change during operation. Capturing poor-quality video
adversely affects the accuracy of analytics. To mitigate the loss in
accuracy of insights, we propose a novel, reinforcement-learning
based system APT that dynamically, and remotely (over 5G
networks), tunes the camera parameters, to ensure a high-quality
video capture, which mitigates any loss in accuracy of video
analytics. As a result, such tuning restores the accuracy of
insights when environmental conditions or scene content change.
APT uses reinforcement learning, with no-reference perceptual
quality estimation as the reward function. We conducted extensive
real-world experiments, where we simultaneously deployed two
cameras side-by-side overlooking an enterprise parking lot (one
camera only has manufacturer-suggested default setting, while the
other camera is dynamically tuned by APT during operation). Our
experiments demonstrated that due to dynamic tuning by APT,
the analytics insights are consistently better at all times of the
day: the accuracy of object detection video analytics application
was improved on average by ∼ 42%. Since our reward function
is independent of any analytics task, APT can be readily used
for different video analytics tasks.
more »
« less
Good-Eye: A Combined Computer-Vision and Physiological-Sensor based Device for Full-Proof Prediction and Detection of Fall of Adults
It is imperative to find the most accurate way to detect falls in elders to help mitigate the disastrous effects of such unfortunate injuries. In order to mitigate fall related accidents, we propose the Good-Eye System, an Internet of Things (IoT) enabled Edge Level Device which works when there is an orientation change detected by camera, and monitors physiological signal parameters. If the observed change is greater than the set threshold, the user is notified with information regarding a prediction of fall or a detection of fall, using LED lights. The Good-Eye System has a remote wall attached camera to monitor continuously the subject as long as the person is in a room along with a camera attached to a wearable to increase the accuracy of the model. The observed accuracy of the Good-Eye System as a whole is approximately 95%.
more »
« less
- Award ID(s):
- 1924112
- NSF-PAR ID:
- 10158167
- Date Published:
- Journal Name:
- Proceedings of the 2nd IFIP International Internet of Things (IoT) Conference (IFIP-IoT)
- Page Range / eLocation ID:
- 273-288
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Eye tracking has become an essential human-machine interaction modality for providing immersive experience in numerous virtual and augmented reality (VR/AR) applications desiring high throughput (e.g., 240 FPS), small-form, and enhanced visual privacy. However, existing eye tracking systems are still limited by their: (1) large form-factor largely due to the adopted bulky lens-based cameras; (2) high communication cost required between the camera and backend processor; and (3) potentially concerned low visual privacy, thus prohibiting their more extensive applications. To this end, we propose, develop, and validate a lensless FlatCambased eye tracking algorithm and accelerator co-design framework dubbed EyeCoD to enable eye tracking systems with a much reduced form-factor and boosted system efficiency without sacrificing the tracking accuracy, paving the way for next-generation eye tracking solutions. On the system level, we advocate the use of lensless FlatCams instead of lens-based cameras to facilitate the small form-factor need in mobile eye tracking systems, which also leaves rooms for a dedicated sensing-processor co-design to reduce the required camera-processor communication latency. On the algorithm level, EyeCoD integrates a predict-then-focus pipeline that first predicts the region-of-interest (ROI) via segmentation and then only focuses on the ROI parts to estimate gaze directions, greatly reducing redundant computations and data movements. On the hardware level, we further develop a dedicated accelerator that (1) integrates a novel workload orchestration between the aforementioned segmentation and gaze estimation models, (2) leverages intra-channel reuse opportunities for depth-wise layers, (3) utilizes input feature-wise partition to save activation memory size, and (4) develops a sequential-write-parallel-read input buffer to alleviate the bandwidth requirement for the activation global buffer. On-silicon measurement and extensive experiments validate that our EyeCoD consistently reduces both the communication and computation costs, leading to an overall system speedup of 10.95×, 3.21×, and 12.85× over general computing platforms including CPUs and GPUs, and a prior-art eye tracking processor called CIS-GEP, respectively, while maintaining the tracking accuracy. Codes are available at https://github.com/RICE-EIC/EyeCoD.more » « less
-
We present the design, implementation, and evaluation of RF-Grasp, a robotic system that can grasp fully-occluded objects in unknown and unstructured environments. Unlike prior systems that are constrained by the line-of-sight perception of vision and infrared sensors, RF-Grasp employs RF (Radio Frequency) perception to identify and locate target objects through occlusions, and perform efficient exploration and complex manipulation tasks in non-line-of-sight settings.RF-Grasp relies on an eye-in-hand camera and batteryless RFID tags attached to objects of interest. It introduces two main innovations: (1) an RF-visual servoing controller that uses the RFID’s location to selectively explore the environment and plan an efficient trajectory toward an occluded target, and (2) an RF-visual deep reinforcement learning network that can learn and execute efficient, complex policies for decluttering and grasping.We implemented and evaluated an end-to-end physical prototype of RF-Grasp. We demonstrate it improves success rate and efficiency by up to 40-50% over a state-of-the-art baseline. We also demonstrate RF-Grasp in novel tasks such mechanical search of fully-occluded objects behind obstacles, opening up new possibilities for robotic manipulation. Qualitative results (videos) available at rfgrasp.media.mit.edumore » « less
-
We present a first-of-its-kind ultra-compact intelligent camera system, dubbed i-FlatCam, including a lensless camera with a computational (Comp.) chip. It highlights (1) a predict-then-focus eye tracking pipeline for boosted efficiency without compromising the accuracy, (2) a unified compression scheme for single-chip processing and improved frame rate per second (FPS), and (3) dedicated intra-channel reuse design for depth-wise convolutional layers (DW-CONV) to increase utilization. i-FlatCam demonstrates the first eye tracking pipeline with a lensless camera and achieves 3.16 degrees of accuracy, 253 FPS, 91.49 µJ/Frame, and 6.7mm×8.9mm×1.2mm camera form factor, paving the way for next-generation Augmented Reality (AR) and Virtual Reality (VR) devices.more » « less
-
A leading cause of physical injury sustained by elderly persons is the event of unintentionally falling. A delay between the time of fall and the time of medical attention can exacerbate injury if the fall resulted in a concussion, traumatic brain injury, or bone fracture. The authors present a solution capable of finding and tracking, in real-time, the location of an elderly person within an indoor facility, using only existing Wi-Fi infrastructure. This paper discusses the development of an open source software framework capable of finding the location of an individual within 3m accuracy using 802.11 Wi-Fi in good coverage areas. This framework is comprised of an embedded software layer, a Web Services layer, and a mobile application for monitoring the location of individuals, calculated using trilateration, with Kalman filtering employed to reduce the effect of multipath interference. The solution provides a real-time, low cost, extendible solution to the problem of indoor geolocation to mitigate potential harm to elderly persons who have fallen and require immediate medical help.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/https://doi.org/10.1007/978-3-030-43605-6_16
