More Like this

1. APT: Adaptive Perceptual quality based camera Tuning using reinforcement learning

   https://doi.org/10.1109/IOTSMS58070.2022.10062226  

   Paul, Sibendu; Rao, Kunal; Coviello, Giuseppe; Sankaradas, Murugan; Po, Oliver; Hu, Y. Charlie; Chakradhar, Srimat (November 2022, Proc. of 9th International Conference on Internet of Things: Systems, Management and Security (IOTSMS 2022))

   Cameras are increasingly being deployed in cities, enterprises and roads world-wide to enable many applications in public safety, intelligent transportation, retail, healthcare and manufacturing. Often, after initial deployment of the cameras, the environmental conditions and the scenes around these cameras change, and our experiments show that these changes can adversely impact the accuracy of insights from video analytics. This is because the camera parameter settings, though optimal at deployment time, are not the best settings for good-quality video capture as the environmental conditions and scenes around a camera change during operation. Capturing poor-quality video adversely affects the accuracy of analytics. To mitigate the loss in accuracy of insights, we propose a novel, reinforcement-learning based system APT that dynamically, and remotely (over 5G networks), tunes the camera parameters, to ensure a high-quality video capture, which mitigates any loss in accuracy of video analytics. As a result, such tuning restores the accuracy of insights when environmental conditions or scene content change. APT uses reinforcement learning, with no-reference perceptual quality estimation as the reward function. We conducted extensive real-world experiments, where we simultaneously deployed two cameras side-by-side overlooking an enterprise parking lot (one camera only has manufacturer-suggested default setting, while the other camera is dynamically tuned by APT during operation). Our experiments demonstrated that due to dynamic tuning by APT, the analytics insights are consistently better at all times of the day: the accuracy of object detection video analytics application was improved on average by ∼ 42%. Since our reward function is independent of any analytics task, APT can be readily used for different video analytics tasks. 

   more » « less
2. RadEye: Tracking Eye Motion Using FMCW Radar

   https://doi.org/10.1145/3706598.3713775  

   Zhang, Shichen; Wang, Qijun; Song, Kunzhe; Yan, Qiben; Zeng, Huacheng (April 2025, ACM)

   Eye motion tracking plays a vital role in many applications such as human-computer interaction (HCI), virtual reality, and disease detection. Camera-based eye tracking, albeit accurate and easy to use, may raise privacy concerns and appear to be unreliable in poor lighting conditions. In this paper, we present RadEye, a radar system capable of detecting fine-grained human eye motions from a distance. RadEye is realized through an integrated hardware and software design. It customizes a sub-6GHz FMCW radar so as to detect millimeter-level eye movement while extending its detection range using low frequency. It further employs a deep neural network (DNN) to refine the detection accuracy through camera-guided supervisory training. We have built a prototype of RadEye. Extensive experimental results show that it achieves 90% accuracy when detecting human eye rotation directions (up, down, left, and right) in various scenarios. 

   more » « less
3. i-FlatCam: A 253 FPS, 91.49 µJ/Frame Ultra-Compact Intelligent Lensless Camera for Real-Time and Efficient Eye Tracking in VR/AR

   https://doi.org/10.1109/VLSITechnologyandCir46769.2022.9830458  

   Zhao, Yang; Li, Ziyun; Fu, Yonggan; Zhang, Yongan; Li, Chaojian; Wan, Cheng; You, Haoran; Wu, Shang; Ouyang, Xu; Boominathan, Vivek; et al (June 2022, 2022 Symposium on VLSI Technology & Circuits Digest of Technical Papers)

   We present a first-of-its-kind ultra-compact intelligent camera system, dubbed i-FlatCam, including a lensless camera with a computational (Comp.) chip. It highlights (1) a predict-then-focus eye tracking pipeline for boosted efficiency without compromising the accuracy, (2) a unified compression scheme for single-chip processing and improved frame rate per second (FPS), and (3) dedicated intra-channel reuse design for depth-wise convolutional layers (DW-CONV) to increase utilization. i-FlatCam demonstrates the first eye tracking pipeline with a lensless camera and achieves 3.16 degrees of accuracy, 253 FPS, 91.49 µJ/Frame, and 6.7mm×8.9mm×1.2mm camera form factor, paving the way for next-generation Augmented Reality (AR) and Virtual Reality (VR) devices. 

   more » « less
4. Autofocusing method for a digital fringe projection system with dual projectors

   https://doi.org/10.1364/OE.392006  

   Zhong, Min; Hu, Xiaowei; Chen, Feng; Xiao, Chao; Peng, Duan; Zhang, Song (April 2020, Optics Express)

   This paper presents a novel technique to achieve autofocusing for a three-dimensional (3D) profilometry system with dual projectors. The proposed system uses a camera that is attached with an electronically focus-tunable lens (ETL) that allows dynamic change of camera’s focal plane such that the camera can focus on the object; the camera captures fringe patterns projected by each projector to establish corresponding points between two projectors, and two pre-calibrated projectors form triangulation for 3D reconstruction. We pre-calibrate the relationship between the depth and the current being used for each focal plane, perform a 3D shape measurement with an unknown focus level, and calculate the desired current value based on the initial 3D result. We developed a prototype system that can automatically focus on an object positioned between 450 mm to 850 mm. 

   more » « less
5. Precise and Effective Robotic Tool Change Strategy Using Visual Servoing With RGB-D Camera

   https://doi.org/10.1115/DETC2021-72123  

   Wei, Danming; Trombley, Christopher M.; Sherehiy, Andriy; Popa, Dan O. (August 2021, 45th Mechanisms and Robotics Conference (MR))

   In modern industrial manufacturing processes, robotic manipulators are routinely used in the assembly, packaging, and material handling operations. During production, changing end-of-arm tooling is frequently necessary for process flexibility and reuse of robotic resources. In conventional operation, a tool changer is sometimes employed to load and unload end-effectors, however, the robot must be manually taught to locate the tool changers by operators via a teach pendant. During tool change teaching, the operator takes considerable effort and time to align the master and tool side of the coupler by adjusting the motion speed of the robotic arm and observing the alignment from different viewpoints. In this paper, a custom robotic system, the NeXus, was programmed to locate and change tools automatically via an RGB-D camera. The NeXus was configured as a multi-robot system for multiple tasks including assembly, bonding, and 3D printing of sensor arrays, solar cells, and microrobot prototypes. Thus, different tools are employed by an industrial robotic arm to position grippers, printers, and other types of end-effectors in the workspace. To improve the precision and cycle-time of the robotic tool change, we mounted an eye-in-hand RGB-D camera and employed visual servoing to automate the tool change process. We then compared the teaching time of the tool location using this system and compared the cycle time with those of 6 human operators in the manual mode. We concluded that the tool location time in automated mode, on average, more than two times lower than the expert human operators. 

   more » « less