Liveness detection of occluded face based on dual-modality convolutional neural network

doi:10.19682/j.cnki.1005-8885.2021.2001

中国邮电高校学报(英文) ›› 2021, Vol. 28 ›› Issue (4): 1-12.doi: 10.19682/j.cnki.1005-8885.2021.2001

• Artificial Intelligence • 下一篇

Liveness detection of occluded face based on dual-modality convolutional neural network

Ming Yue, Li Wenmin, Xu Siya, Gao Lifang, Zhang Hua, Shao Sujie, Yang Huifeng

1. State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China
2. State Grid Hebei Information and Telecommunication Branch, Shijiazhuang 050013, China

收稿日期:2020-12-22 修回日期:2021-07-29 接受日期:2021-07-29 出版日期:2021-08-31 发布日期:2021-10-11
通讯作者: Corresponding author: Li Wenmin, E-mail: liwenmin@bupt.edu.cn E-mail:liwenmin@bupt.edu.cn
基金资助:
This work was supported by the Science and Technology Project of State Grid Corporation of China
(SGHEXT00YJJS1900050).

Liveness detection of occluded face based on dual-modality convolutional neural network

Ming Yue, Li Wenmin, Xu Siya, Gao Lifang, Zhang Hua, Shao Sujie, Yang Huifeng

1. State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China
2. State Grid Hebei Information and Telecommunication Branch, Shijiazhuang 050013, China

Received:2020-12-22 Revised:2021-07-29 Accepted:2021-07-29 Online:2021-08-31 Published:2021-10-11
Contact: Corresponding author: Li Wenmin, E-mail: liwenmin@bupt.edu.cn E-mail:liwenmin@bupt.edu.cn
Supported by:
This work was supported by the Science and Technology Project of State Grid Corporation of China
(SGHEXT00YJJS1900050).

摘要/Abstract

摘要： Facial recognition has become the most common identity authentication technologies. However, problems such as uneven light and occluded faces have increased the hardness of liveness detection. Nevertheless, there are a few pieces of research on face liveness detection under occlusion conditions. This paper designs a face recognition technique suitable for different degrees of facial occlusion, which employs the facial datasets of near-infrared (NIR) images and visible (VIS) light images to examine the single-modality detection accuracy rate (experimental control group) and the corresponding high-dimensional features through the residual network (ResNet). Based on the idea of data fusion, we propose two feature fusion methods. The two methods extract and fuse the data of one and two convolutional layers from two single-modality detectors respectively. The fusion of high-dimensional features apply a new ResNet to get the dual-modality detection accuracy. And then, a new ResNet is applied to test the accuracy of dual-modality detection. The experimental results show that the dual-modality face liveness detection model improves face live detection accuracy and robustness compared with the single-modality. The fusion of two-layer features from the single-modality detector can also improve face detection accuracy by utilizing the above-mentioned dual-modality detector, and it doesn't increase the algorithm's complexity.

关键词: face deception, face occlusion, near-infrared image, residual network, feature fusion

Abstract: Facial recognition has become the most common identity authentication technologies. However, problems such as uneven light and occluded faces have increased the hardness of liveness detection. Nevertheless, there are a few pieces of research on face liveness detection under occlusion conditions. This paper designs a face recognition technique suitable for different degrees of facial occlusion, which employs the facial datasets of near-infrared (NIR) images and visible (VIS) light images to examine the single-modality detection accuracy rate (experimental control group) and the corresponding high-dimensional features through the residual network (ResNet). Based on the idea of data fusion, we propose two feature fusion methods. The two methods extract and fuse the data of one and two convolutional layers from two single-modality detectors respectively. The fusion of high-dimensional features apply a new ResNet to get the dual-modality detection accuracy. And then, a new ResNet is applied to test the accuracy of dual-modality detection. The experimental results show that the dual-modality face liveness detection model improves face live detection accuracy and robustness compared with the single-modality. The fusion of two-layer features from the single-modality detector can also improve face detection accuracy by utilizing the above-mentioned dual-modality detector, and it doesn't increase the algorithm's complexity.

Key words: face deception, face occlusion, near-infrared image, residual network, feature fusion

中图分类号:

TP39

Ming Yue, Li Wenmin, Xu Siya, Gao Lifang, Zhang Hua, Shao Sujie, Yang Huifeng. Liveness detection of occluded face based on dual-modality convolutional neural network[J]. The Journal of China Universities of Posts and Telecommunications, 2021, 28(4): 1-12.

参考文献

1. Wayman J L. Biometrics in identity management systems. IEEE Security and Privacy, 2008, 6(2): 30 -37
2. Liu C X. The development trend of evaluating face-recognition technology. Proceedings of the 2014 International Conference on Mechatronics and Control (ICMC'14), 2014, Jul 3 - 5, Jinzhou, China. Piscataway, NJ, USA: IEEE, 2014: 1540 -1544
3. Zhu Y L, Chen S C. Sub-image method based on feature sampling and feature fusion for face recognition. Journal of Software, 2012, 23(12): 3209 -3220 (in Chinese)
4. Best-Rowden L, Han H, Otto C, et al. Unconstrained face recognition: identifying a person of interest from a media
collection. IEEE Transactions on Information Forensics and Security, 2014, 9(12): 2144 -2157
5. Klare B F, Klein B, Taborsky E, et al. Pushing the frontiers of unconstrained face detection and recognition: IARPA Janus Benchmark A. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR'15), 2015, Jun 7 -12, Boston, MA, USA. Piscataway, NJ, USA: IEEE, 2015: 1931 -1939
6. Boutellaa E, Boulkenafet Z, Komulainen J, et al. Audiovisual synchrony assessment for replay attack detection in talking face biometrics. Multimedia Tools and Applications, 2016, 75 (9): 5329 -5343
7. Zhang J, Yan Y, Lades M. Face recognition: eigenface, elastic matching, and neural nets. Proceedings of the IEEE, 1997, 85(9): 1423 -1435
8. Albiol A, Monzo D, Martin A, et al. Face recognition using HOG -EBGM. Pattern Recognition Letters, 2008, 29(10): 1537 -1543
9. Yan L, Hu X P. Face anti-spoofing based on context and OCSVM. Application of Electronic Technique, 2020, 46(6): 32 - 35 (in Chinese)
10. Chen H N, Chen Y W, Tian X, et al. A cascade face spoofing detector based on face anti-spoofing R - CNN and improved retinex LBP. IEEE Access, 2019, 7: 170116 -170133
11. Cao Y, Tu L, Wu L F. Face liveness detection using gray level co-occurrence matrix and wavelets analysis in identity authentication. Journal of Signal Processing, 2014, 30 (7): 830 - 835 (in Chinese)
12. Wen D, Han H, Jain A K. Face spoof detection with image distortion analysis. IEEE Transactions on Information Forensics and Security, 2015, 10(4): 746 -761
13. Boulkenafet Z, Komulainen J, Hadid A. Face spoofing detection using colour texture analysis. IEEE Transactions on Information Forensics and Security, 2016, 11(8): 1818 -1830
14. Tirunagari S, Poh N, Windridge D, et al. Detection of face spoofing using visual dynamics. IEEE Transactions on Information Forensics and Security, 2015, 10(4): 762 -777
15. Freitas P T, Komulainen J, Anjos A, et al. Face liveness detection using dynamic texture. Eurasip Journal on Image and Video Processing, 2014(1): 1 -2
16. Akbulut Y, Sengur A, Budak U, et al. Deep learning based face liveness detection in videos. Proceedings of the 2017 International Artificial Intelligence and Data Processing Symposium (IDAP'17), 2017, Sept 16 - 17, Malatya, Turkey. Piscataway, NJ, USA: IEEE, 2017: 1 -4
17. Garg S, Mittal S, Kumar P, et al. DeBNet: multilayer deep network for liveness detection in face recognition system.
proceedings of the 7th International Conference on Signal Processing and Integrated Networks (SPIN'20), 2020, Feb 27 -28, Noida, India. Piscataway, NJ, USA: IEEE, 2020: 1136 -1141
18. Sun W Y, Song Y, Chen C S, et al. Face spoofing detection based on local ternary label supervision in fully convolutional networks. IEEE Transactions on Information Forensics and Security, 2020, 15: 3181 -3196
19. Xu Z Q, Li S, Deng W H. Learning temporal features using LSTM-CNN architecture for face anti-spoofing. Proceedings of the 3rd IAPR Asian Conference on Pattern Recognition (ACPR'15), 2015, Nov 3 - 6, Kuala Lumpur, Malaysia. Piscataway, NJ, USA: IEEE, 2015: 141 -145
20. Atoum Y, Liu Y J, Jourabloo A, et al. Face anti-spoofing using patch and depth-based CNNs. Proceedings of the 2017 IEEE International Joint Conference on Biometrics (IJCB'17), 2017, Oct 1 - 4, Denver, CO, OSA. Piscataway, NJ, USA: IEEE, 2017: 319 -328
21. Ma Y K, Wu L F, Jian M, et al. Algorithm to generate adversarial examples for face-spoofing detection. Journal of Software, 2019, 30(2): 469 -480 (in Chinese)
22. Kuang H F, Ji R R, Liu H, et al. Multi-modal multi-layer fusion network with average binary center loss for face anti-spoofing. Proceedings of the 27th ACM International Conference on Multimedia (MM'19), 2019, Oct 21 -25, 2019, Nice, France. New York, NY, USA: ACM, 2019: 48 -56
23. Kim T Y, Lee K M, Lee S U, et al. Occlusion invariant face recognition using two-dimensional PCA. Advances in Computer Graphics and Computer Vision, 2007, 4: 305 -315
24. Song L X, Gong D H, Li Z F, et al. Occlusion robust face recognition based on mask learning with pairwise differential siamese network. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV'19), 2019, Oct 27-Nov 2, Seoul, Republic of Korea. Piscataway, NJ, USA: IEEE, 2019: 773 -782
25. Jongsun K, Jongmoo C, Juneho Y, et al. Effective representation using ICA for face recognition robust to local distortion and partial occlusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(12): 1977 -1981
26. He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR'16), 2016, Jun 27 - 30, Las Vegas, NV, USA. Piscataway, NJ, USA: IEEE, 2016: 770 -778
27. Hinton G E, Osindero S, Teh Y W. A fast learning algorithm for deep belief nets. Neural Computation, 2006, 18(7): 1527 -1554
28. Pang J M, Chen K, Shi J P, et al, Libra R - CNN: towards balanced learning for object detection. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'19), 2019, Jun 15 - 20, Long Beach, CA,
USA. Piscataway, NJ, USA: IEEE, 2019: 821 -830

[1]	Wang Zixian, Zhang Miao, Yan Danfeng . DRO-SLAM: Real-time object-aware SLAM for navigation robots and autonomous driving in dynamic environments[J]. 中国邮电高校学报(英文版), 2023, 30(3): 14-24.
[2]	Zhang Luying, Liu Xiaokai, Li Zhao, Xu Fangmin, Zhao Chenglin. Prediction based dynamic resource allocation method for edge computing first networking [J]. 中国邮电高校学报(英文版), 2023, 30(3): 78-87.
[3]	Chen Hui, Jiang Xiaoling, Wu Tianting, Mou Xingyu. Crowd sensing data delivery based on tangle DAG network [J]. 中国邮电高校学报(英文版), 2023, 30(3): 88-98.
[4]	Guo Xiangbo, Wang Jian, Huang Mengjie, Wang Minghui, Yang Jian, Yu Yongtao. Deep knowledge tracking algorithm based on forgetting law[J]. 中国邮电高校学报(英文版), 2023, 30(1): 17-27.
[5]	Wang Xianlun, Wang Guangyu, Cui Yuxia. Facial expression recognition based on improved ResNet[J]. 中国邮电高校学报(英文版), 2023, 30(1): 28-38.
[6]	Wu Hongxin, Lin Zhijian, Chen Pingping, Chen Feng. Joint partial computation offloading and resource allocation in MEC-enable networks[J]. 中国邮电高校学报(英文版), 2023, 30(1): 80-86.
[7]	Zhu Ruijie, Li Gong, Wang Peisen, Zhang Wenchao. Reinforced virtual optical network embedding algorithm in EONs for edge computing [J]. 中国邮电高校学报(英文版), 2022, 29(6): 18-29.
[8]	Kong Chao, Ou Weihua, Gong Xiaofeng, Li Weian, Han Jie, Yao Yi, Xiong Jiahao. Face anti-spoofing based on multi-modal and multi-scale features fusion [J]. 中国邮电高校学报(英文版), 2022, 29(6): 73-82.
[9]	Yang Jingjing, Guo Yuchun, Zhao Yongxiang, Chen Yishuai. How time latency in navigation Apps affects traffic [J]. 中国邮电高校学报(英文版), 2022, 29(6): 53-63.
[10]	张函井音吉赵永利. FS-LSTM: sales forecasting in e-commerce on feature selection [J]. 中国邮电高校学报(英文版), 2022, 29(5): 92-98.
[11]	Shi Jinjing, Wang Wenxuan, Xiao Zimeng, Mu Shuai, Li Qin. Quantum classifier with parameterized quantum circuit based on the isolated quantum system[J]. 中国邮电高校学报(英文版), 2022, 29(4): 21-31.
[12]	Liu Hailing, Zhang Jie, Qin Sujuan, Gao Fei. Quantum algorithm for soft margin support vector machine with hinge loss function[J]. 中国邮电高校学报(英文版), 2022, 29(4): 32-41.
[13]	Wang Jian, Qiao Kuoyuan, Yuan Yanlei, Liu Xiaole, Yang Jian. Adaptive learning path recommendation model for examination-oriented education[J]. 中国邮电高校学报(英文版), 2022, 29(4): 77-88.
[14]	Meng Wei, Wang Liting, Lu Meng. Summary of research on recommendation system based on serendipity[J]. 中国邮电高校学报(英文版), 2022, 29(4): 89-105.
[15]	Jia Wei, Gong Chao. Precise and efficient Chinese license plate recognition in the real monitoring scene of intelligent transportation system[J]. 中国邮电高校学报(英文版), 2022, 29(3): 1-14.

Liveness detection of occluded face based on dual-modality convolutional neural network

Liveness detection of occluded face based on dual-modality convolutional neural network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价