Python Based Pose and Skin Color Improved Face Recognition System
Date
2020-12
Authors
Mathew, Akku Merium
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Face recognition has been an active research topic in the field of computer vision. Face recognition is currently being used for applications such as law enforcement, security, and video surveillance. Variations in pose and skin color are the two main factors that affect the accuracy of a face recognition system. The aim of this research is to develop a pose invariant and skin color invariant face recognition system for surveillance and law enforcement applications. In this research, we tackle the problem of pose variations and skin color separately.
To understand the effect of pose on face recognition, we initially created a face recognition system for variation in head poses from -90 degrees yaw angle to 90 degrees yaw angle. From our study, we found that the accuracy of a face recognition system drops significantly from a yaw angle of 45 degrees to 90 degrees and -60 degrees to -90 degrees. To combat this problem, we identified two solutions. Firstly, we propose skin segmentation in HSV color space as a preprocessing step to improve accuracy for nonfrontal poses from 45 degrees to 90 degrees and -60 degrees to -90 degrees. Secondly, we propose using a Style Transfer Generative Adversarial Network (StyleGAN) generated frontal image for face recognition to improve accuracy for non-frontal poses from 45 degrees to 75 degrees and -60 degrees to -75 degrees. The experimental results demonstrate that both methods have significantly improved the accuracy of non-frontal poses.
To study the effect of skin color on face recognition, we created the ‘Celeb-Skin’ dataset. The dataset contains 480 images of male celebrities of different races such as White, Asian, and African American. From testing the face recognition system on the Celeb-Skin dataset, we found that the accuracy of White faces was comparatively higher than African American and Asian faces. In this research, we propose that creating a training dataset based on skin color and grouping the celebrities based on their skin color would improve the accuracy of dark-skinned faces. Experimental results show that our technique has improved the accuracy of dark-skinned faces by 7.38 percent.
Description
Keywords
face recognition, pose, skin color, machine learning, image processing
Citation
Mathew, A. M. (2020). Python based pose and skin color improved face recognition system (Unpublished thesis). Texas State University, San Marcos, Texas.