In this paper we proposed a real-time face mask detection and recognition for CCTV surveillance camera videos. The proposed work consists of six steps: video acquisition and keyframes selection, data augmentation, facial parts segmentation, pixel-based feature extraction, Bag of Visual Words (BoVW) generation, face mask detection, and face recognition. In the first step, a set of keyframes are selected using histogram of gradient (HoG) algorithm. Secondly, data augmentation is involved with three steps as color normalization, illumination correction (CLAHE), and poses normalization (Angular Affine Transformation). In third step, facial parts are segmented using clustering approach i.e. Expectation Maximization with Gaussian Mixture Model (EM-GMM), in which facial regions are segmented into Eyes, Nose, Mouth, Chin, and Forehead. Then, Pixel-based Feature Extraction is performed using Yolo Nano approach, which performance is higher and lightweight model than the Yolo Tiny V2 and Yolo Tiny V3, and extracted features are constructed into Codebook by Hassanat Similarity with K-Nearest neighbor (H-M with KNN) algorithm. For mask detection, L2 distance function is used. The final step is face recognition which is implemented by a Kernel-based Extreme Learning Machine with Slime Mould Optimization (SMO). Experiments conducted using Python IDLE 3.8 for the proposed Yolo Nano model and also previous works as GMM with Deep learning (GMM+DL), Convolutional Neural Network (CNN) with VGGF, Yolo Tiny V2, and Yolo Tiny V3 in terms of various performance metrics.
more »
« less
Measuring and Visualizing Social Distancing Using Deep Learning and 3D Computer Vision
Social Distancing has proved a necessary measure in con- trolling the spread of Coronavirus. The CDC (Center for dis- ease control and prevention) in the United States recommends 6 feet as a safe distance between individuals. Therefore, a surveillance system capable of measuring distances between individuals can prove beneficial in limiting the spread. Video surveillance systems have already been introduced in various fields of our daily life to enhance security and protect individuals and sensitive infrastructure. In this work we present a way to use the existing video surveillance infrastructure to measure and monitor social distancing. We present a practical approach to monitor this distance using artificial intelligence and projective geometry techniques. Our approach, after initial setup, works in real-time requiring only a monocular surveillance camera feed. The proposed approach utilizes YOLO v4 neural network object detector for detecting pedestrians in the camera’s view. Projective transformation is used to localize the pedestrians on the ground. Finally, the real world distances between pedestrians is calculated and visualized with the right perspective and occlusion relations as if the distance marks are actually on the ground. All the implementation is in real-time, and is performed on python using the OPENCV libraries and the YOLO v4 neural net with pre- trained weights. Experimental results are provided to validate our approach. The code of this work will be made publicly available at GitHub upon acceptance.
more »
« less
- Award ID(s):
- 1827505
- NSF-PAR ID:
- 10286825
- Date Published:
- Journal Name:
- AI for Social Good - AAAI Fall Symposium 2020
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Social distancing is an effective public health tool to reduce the spread of respiratory pandemics such as COVID-19. To analyze compliance with social distancing policies, we design two video-based pipelines for social distancing analysis, namely, Auto-SDA and B-SDA. Auto-SDA (Automated video-based Social Distancing Analyzer) is designed to measure social distancing using street-level cameras. To avoid privacy concerns of using street-level cameras, we further develop B-SDA (Bird’s eye view Social Distancing Analyzer), which uses bird’s eye view cameras, thereby preserving pedestrian’s privacy. We used the COSMOS testbed deployed in West Harlem, New York City, to evaluate both pipelines. In particular, Auto-SDA and B-SDA are applied on videos recorded by two of COSMOS cameras deployed on the 2nd floor (street-level) and 12th floor (bird’s eye view) of Columbia University’s Mudd building, looking at 120th St. and Amsterdam Ave. intersection, New York City. Videos are recorded before and during the peak of the pandemic, as well as after the vaccines became broadly available. The results represent the impact of social distancing policies on pedestrians’ social behavior. For example, the analysis shows that after the lockdown, less than 55% of the pedestrians failed to adhere to the social distancing policies, whereas this percentage increased to 65% after the vaccines’ availability. Moreover, after the lockdown, 0-20% of the pedestrians were affiliated with a social group, compared to 10-45% once the vaccines became available. The results also show that the percentage of face-to-face failures has decreased from 42.3% (pre-pandemic) to 20.7%(after the lockdown).more » « less
-
Social distancing can reduce infection rates in respiratory pandemics such as COVID-19, especially in dense urban areas. To assess pedestrians’ compliance with social distancing policies, we use the pilot site of the PAWR COSMOS wireless edge-cloud testbed in New York City to design and evaluate an Automated video-based Social Distancing Analyzer (Auto-SDA) pipeline. Auto-SDA derives pedestrians’ trajectories and measures the duration of close proximity events. It relies on an object detector and a tracker, however, to achieve highly accurate social distancing analysis, we design and incorporate 3 modules into Auto-SDA: (i) a calibration module that converts 2D pixel distances to 3D on-ground distances with less than 10 cm error, (ii) a correction module that identifies pedestrians who were missed or assigned duplicate IDs by the object detectiontracker and rectifies their IDs, and (iii) a group detection module that identifies affiliated pedestrians (i.e., pedestrians who walk together as a social group) and excludes them from the social distancing violation analysis. We applied Auto-SDA to videos recorded at the COSMOS pilot site before the pandemic, soon after the lockdown, and after the vaccines became broadly available, and analyzed the impacts of the social distancing protocols on pedestrians’ behaviors and their evolution. For example, the analysis shows that after the lockdown, less than 55% of the pedestrians violated the social distancing protocols, whereas this percentage increased to 65% after the vaccines became available. Moreover, after the lockdown, 0-20% of the pedestrians were affiliated with a social group, compared to 10-45% once the vaccines became available. Finally, following the lockdown, the density of the pedestrians at the intersection decreased by almost 50%.more » « less
-
null (Ed.)Background Human movement is one of the forces that drive the spatial spread of infectious diseases. To date, reducing and tracking human movement during the COVID-19 pandemic has proven effective in limiting the spread of the virus. Existing methods for monitoring and modeling the spatial spread of infectious diseases rely on various data sources as proxies of human movement, such as airline travel data, mobile phone data, and banknote tracking. However, intrinsic limitations of these data sources prevent us from systematic monitoring and analyses of human movement on different spatial scales (from local to global). Objective Big data from social media such as geotagged tweets have been widely used in human mobility studies, yet more research is needed to validate the capabilities and limitations of using such data for studying human movement at different geographic scales (eg, from local to global) in the context of global infectious disease transmission. This study aims to develop a novel data-driven public health approach using big data from Twitter coupled with other human mobility data sources and artificial intelligence to monitor and analyze human movement at different spatial scales (from global to regional to local). Methods We will first develop a database with optimized spatiotemporal indexing to store and manage the multisource data sets collected in this project. This database will be connected to our in-house Hadoop computing cluster for efficient big data computing and analytics. We will then develop innovative data models, predictive models, and computing algorithms to effectively extract and analyze human movement patterns using geotagged big data from Twitter and other human mobility data sources, with the goal of enhancing situational awareness and risk prediction in public health emergency response and disease surveillance systems. Results This project was funded as of May 2020. We have started the data collection, processing, and analysis for the project. Conclusions Research findings can help government officials, public health managers, emergency responders, and researchers answer critical questions during the pandemic regarding the current and future infectious risk of a state, county, or community and the effectiveness of social/physical distancing practices in curtailing the spread of the virus. International Registered Report Identifier (IRRID) DERR1-10.2196/24432more » « less
-
Social distancing can reduce the infection rates in respiratory pandemics such as COVID-19. Traffic intersections are particularly suitable for monitoring and evaluation of social distancing behavior in metropolises. Hence, in this paper, we propose and evaluate a real-time privacy-preserving social distancing analysis system (B-SDA), which uses bird’s-eye view video recordings of pedestrians who cross traffic intersections. We devise algorithms for video pre-processing, object detection, and tracking which are rooted in the known computer-vision and deep learning techniques, but modified to address the problem of detecting very small objects/pedestrians captured by a highly elevated camera. We propose a method for incorporating pedestrian grouping for detection of social distancing violations, which achieves 0.92 F1 score. B-SDA is used to compare pedestrian behavior in pre-pandemic and during-pandemic videos in uptown Manhattan, showing that the social distancing violation rate of 15.6% during the pandemic is notably lower than 31.4% prepandemic baseline. Keywords—Social distancing, Object detection, Smart city, Testbedsmore » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
