中国邮电高校学报(英文版) ›› 2022, Vol. 29 ›› Issue (6): 73-82.doi: 10.19682/j.cnki.1005-8885.2022.1004

• Others • 上一篇    下一篇

Face anti-spoofing based on multi-modal and multi-scale features fusion

Kong Chao, Ou Weihua, Gong Xiaofeng, Li Weian, Han Jie, Yao Yi, Xiong Jiahao   

  1. 1. School of Big Data and Computer Science, Guizhou Normal University, Guiyang 550025, China 2. Special Key Laboratory of Artificial Intelligence and Intelligent Control of Guizhou Province, Guizhou Institute of Technology, Guiyang 550003, China 3. Guizhou Science and Technology Information Center, Department of Science and Technology of Guizhou Province, Guiyang 550002, China
  • 收稿日期:2021-02-19 修回日期:2021-07-10 出版日期:2022-12-30 发布日期:2022-12-30
  • 通讯作者: Ou Weihua E-mail:1305532268@qq.com
  • 基金资助:

    This work was supported by the National Natural Science Foundation of China (61962010, 62262005), and the Natural

    Science Foundation of Guizhou Priovince ( QianKeHeJichu [2019]1425).

Face anti-spoofing based on multi-modal and multi-scale features fusion

Kong Chao, Ou Weihua, Gong Xiaofeng, Li Weian, Han Jie, Yao Yi, Xiong Jiahao   

  1. 1. School of Big Data and Computer Science, Guizhou Normal University, Guiyang 550025, China 2. Special Key Laboratory of Artificial Intelligence and Intelligent Control of Guizhou Province, Guizhou Institute of Technology, Guiyang 550003, China 3. Guizhou Science and Technology Information Center, Department of Science and Technology of Guizhou Province, Guiyang 550002, China
  • Received:2021-02-19 Revised:2021-07-10 Online:2022-12-30 Published:2022-12-30
  • Contact: Ou Weihua E-mail:1305532268@qq.com
  • Supported by:

    This work was supported by the National Natural Science Foundation of China (61962010, 62262005), and the Natural

    Science Foundation of Guizhou Priovince ( QianKeHeJichu [2019]1425).

摘要:

Face anti-spoofing is used to assist face recognition system to judge whether the detected face is real face or fake face. In the traditional face anti-spoofing methods, features extracted by hand are used to describe the difference between living face and fraudulent face. But these handmade features do not apply to different variations in an unconstrained environment. The convolutional neural network (CNN) for face deceptions achieves considerable results. However, most existing neural network-based methods simply use neural networks to extract single-scale features from single-modal data, while ignoring multi-scale and multi-modal information. To address this problem, a novel face anti-spoofing method based on multi-modal and multi-scale features fusion ( MMFF) is proposed. Specifically, first residual network ( Resnet )-34 is adopted to extract features of different scales from each modality, then these features of different scales are fused by feature pyramid network (FPN), finally squeeze-and-excitation fusion ( SEF) module and self-attention network ( SAN) are combined to fuse features from different modalities for classification. Experiments on the CASIA-SURF dataset show that the new method based on MMFF achieves better performance compared with most existing methods.

关键词:

face anti-spoofing, multi-modal fusion, multi-scale fusion, self-attention network (SAN), feature pyramid network (FPN)

Abstract: Face anti-spoofing is used to assist face recognition system to judge whether the detected face is real face or fake face. In the traditional face anti-spoofing methods, features extracted by hand are used to describe the difference between living face and fraudulent face. But these handmade features do not apply to different variations in an unconstrained environment. The convolutional neural network (CNN) for face deceptions achieves considerable results. However, most existing neural network-based methods simply use neural networks to extract single-scale features from single-modal data, while ignoring multi-scale and multi-modal information. To address this problem, a novel face anti-spoofing method based on multi-modal and multi-scale features fusion ( MMFF) is proposed. Specifically, first residual network ( Resnet )-34 is adopted to extract features of different scales from each modality, then these features of different scales are fused by feature pyramid network (FPN), finally squeeze-and-excitation fusion ( SEF) module and self-attention network ( SAN) are combined to fuse features from different modalities for classification. Experiments on the CASIA-SURF dataset show that the new method based on MMFF achieves better performance compared with most existing methods.

Key words: face anti-spoofing, multi-modal fusion, multi-scale fusion, self-attention network (SAN), feature pyramid network (FPN)

中图分类号: