基于超分辨率重建的低质量视频人脸识别方法

doi:10.11896/jsjkx.201200159

计算机科学 ›› 2021, Vol. 48 ›› Issue (11A): 295-302.doi: 10.11896/jsjkx.201200159

• 图像处理& 多媒体技术 • 上一篇下一篇

基于超分辨率重建的低质量视频人脸识别方法

陆要要¹, 袁家斌^1,2, 何珊¹, 王天星¹

1 南京航空航天大学计算机科学与技术学院南京211106
2 南京航空航天大学信息化处(信息化技术中心) 南京211106

出版日期:2021-11-10 发布日期:2021-11-12
通讯作者: 袁家斌(jbyuan@nuaa.edu.cn)
作者简介:yaoyaolu@nuaa.edu.cn
基金资助:
国家重点研发计划课题(2017YFB0802303);国家自然科学基金项目(62076127,61571226)

Low-quality Video Face Recognition Method Based on Super-resolution Reconstruction

LU Yao-yao¹, YUAN Jia-bin^1,2, HE Shan¹, WANG Tian-xing¹

1 School of Computer Science and Technology,Nanjing University of Aeronautics and Astronautics,Nanjing 211106,China
2 Information Department(Informationization Technology Center),Nanjing University of Aeronautics and Astronautics,Nanjing 211106,China

Online:2021-11-10 Published:2021-11-12
About author:LU Yao-yao,born in 1995,postgraduate.Her main research interests include deep learning and face recognition.
YUAN Jia-bin,born in 1968,Ph.D,professor,is a senior member of China Computer Federation.His main research interests include deep learning,high performance computing and information security,etc.
Supported by:
National Key Research and Development Program of China(2017YFB0802303) and National Natural Science Foundation of China(62076127,61571226).

摘要/Abstract

摘要： 随着深度神经网络的兴起,人脸识别技术得到了飞速发展。但在光照条件差、低分辨率等情况下的低质量视频S2V(Still to Video)人脸识别由于存在低质量测试视频与样本库高清图像的异质匹配问题,仍然没有达到预期的效果。针对这个问题,提出一种基于超分辨率重建的低质量视频人脸识别方法。首先根据人脸姿态对低质量视频帧采用聚类算法和随机算法选取关键帧,然后建立一个面向低质量视频S2V人脸识别的超分辨率重建模型S2V-SR,对关键帧进行超分辨率重建,从而获得高分辨率且更多身份特征的超分辨率关键帧,最后使用视频人脸识别网络提取深度特征进行分类投票,得到最终的人脸识别结果。所提方法在COX视频人脸数据集上进行实验测试,在相对较高质量的cam1和cam3视频中获得了最好的识别准确率,即55.91%和70.85%,而在相对较低质量的cam2视频中获得了仅次于最好方法的识别准确率。实验结果证明,所提方法能够在一定程度上解决S2V人脸识别中异质匹配的问题,并且能够获得较高的识别准确性和稳定性。

关键词: 超分辨率重建, 低质量视频, 人脸识别, 深度特征

Abstract: With the rise of deep neural networks,face recognition technology has developed rapidly.However,S2V (Still to Video)face recognition for low-quality video that is poor lighting conditions and low resolution still does not achieve the expected results,because the heterogeneous matching problem between the test video of low-quality and the high-definition image of the sample library.To solve this problem,this paper proposes a face recognition method for low-quality video based on super-resolution reconstruction.First,it uses clustering algorithm and random algorithm to select key frames for low-quality video frames based on face pose.Then,it builds a super-resolution reconstruction model S2V-SR for low-quality video S2V face recognition,and performs super-resolution reconstruction on key frames to obtain super-resolution key frames with higher resolution and more identity features.Finally,it uses the video face recognition network to extract deep features for classification and voting to obtain the final result.The proposed method is experimentally tested on the COX video face data set,and the best recognition accuracy is 55.91% and 70.85% in the relatively high-quality cam1 and cam3 videos,while in the relatively low-quality cam2 video the re-cognition accuracy rate second only to the best method is obtained.Experiments show that the proposed method can solve the hete-rogeneous matching problem in S2V face recognition to a certain extent,and obtain higher recognition accuracy and stability.

Key words: Depth feature, Face recognition(FR), Llow-quality video, Super-resolution reconstruction(SR)

中图分类号:

TP183

陆要要, 袁家斌, 何珊, 王天星. 基于超分辨率重建的低质量视频人脸识别方法[J]. 计算机科学, 2021, 48(11A): 295-302. https://doi.org/10.11896/jsjkx.201200159

LU Yao-yao, YUAN Jia-bin, HE Shan, WANG Tian-xing. Low-quality Video Face Recognition Method Based on Super-resolution Reconstruction[J]. Computer Science, 2021, 48(11A): 295-302. https://doi.org/10.11896/jsjkx.201200159

参考文献

[1]ZHANG K P,ZHANG Z P,CHENG C W,et al.Super-Identity Convolutional Neural Network for Face Hallucination[C]//European Conference on Computer Vision(ECCV).Springer,Cham,2018:196-211.
[2]WANG R,GUO H,DAVIS L S,et al.Covariance discriminative learning:A natural and efficient approach to image set classification[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2012:2496-2503.
[3]LU J,WANG G,MOULIN P.Image Set Classification UsingHolistic Multiple Order Statistics Features and Localized Multi-kernel Metric Learning[C]//2013 IEEE International Conference on Computer Vision(ICCV).IEEE 2013:329-336.
[4]WANG W,WANG R,HUANG Z,et al.Discriminant Analysis on Riemannian Manifold of Gaussian Distributions for Face Recognition with ImageSets[J].IEEE Transactions on Image Processing,2018,27(1):151-163.
[5]CHIEN J T,WU C C.Discriminant Waveletfaces and NearestFeature Classifiers for Face Recognition[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2002(12):1644-1649.
[6]XIAO Q L.Research and application of face recognition methods in low-quality videos[D].Nanjing:Nanjing University,2019.
[7]SUN Y,WANG X,TANG X.Deep Learning Face Representation by Joint Identification-Verification[C]//Advances in Neural Information Processing Systems.2014:1988-1996.
[8]SCHROFF F,KALENICHENKO D,PHILBIN J.FaceNet:AUnified Embedding for Face Recognition and Clustering[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE 2015:815-823
[9]WEN Y,ZHANG K,LI Z,et al.A Discriminative FeatureLearning Approach for Deep Face Recognition[C]//European Conference on Computer Vision.Springer,Cham,2016:499-515.
[10]LIU W,WEN Y,YU Z,et al.SphereFace:Deep Hypersphere Embedding for Face Recognition[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2017:212-220.
[11]WANG F,XIANG X,CHENG J,et al.NormFace:L 2 Hypersphere Embedding for Face Verification[C]//ACM Conference on MultiMedia (MM) 2017.ACM,2017:1041-1049.
[12]DENG J,GUO J,ZAFEIRIOU S.ArcFace:Additive AngularMargin Loss for Deep Face Recognition[J].arXiv:1801.07698,2018.
[13]SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition[J].arXiv:1409.1556,2014.
[14]HE K,ZHANG X,REN S,et al.Deep Residual Learning for Image Recognition[C]//2016 IEEE Conference on Computer Vision & Pattern Recognition(CVPR).IEEE,2016:770-778.
[15]SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision & Pattern Recognition(CVPR).IEEE,2015.
[16]SZEGEDY C,IOFFE S,VANHOUCKE V.Inception-v4,inception-resnetandtheimpact of residual connections on learning[J].arXiv:1602.07261,2016.
[17]ZHANG K B,ZHENG D D,JING J F.Overview of low-resolution face recognition[J].Computer Engineering and Applications,2019,55(22):14-24.
[18]HERRMANN C,WILLERSINN D,JURGEN B.Low-resolution Convolutional Neural Networks for video face recognition[C]//IEEE International Conference on Advanced Video & Signal Based Surveillance(AVSS).IEEE,2016:221-227.
[19]ZANGENEH E,RAHMATI M,MOHSENZADEH Y.LowResolution Face Recognition Using a Two-Branch Deep Convolutional Neural Network Architecture[J].arXiv:1706.06247,2017.
[20]ELAZHARI A,AHMADI M.A neural network based humanface recognition of low resolution images[C]//2014 World Automation Congress (WAC).IEEE,2014:185-190.
[21]HUANG Z,SHAN S,WANG R,et al.A Benchmark and Comparative Study of Video-Based Face Recognition on COX Face Database[J].IEEE Transactions on Image Processing,2015,24(12):5967-5981.
[22]HUANG G,LIU Z,Maaten L V D,et al.Densely Connected Convolutional Networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2017.
[23]JOHNSON J,ALAHI A,FEI-FEI L.Perceptual Losses for Real-Time Style Transfer and Super-Resolution[C]//European Conference on Computer Vision(ECCV).Springer,Cham,2016:694-711.
[24]DONG Y,ZHEN L,LIAO S,et al.Learning face representation from scratch[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2014.
[25]CHAO D,CHEN C L,TANG X.Accelerating the Super-Resolution Convolutional Neural Network[C]//2016 IEEE Conference on Computer Vision & Pattern Recognition(CVPR).IEEE,2016.
[26]LAI W S,HUANG J B,AHUJA N,et al.Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution[C]//2017 IEEE Conference on Computer Vision & Pattern Recognition(CVPR).IEEE,2017.
[27]LIM B,SON S,KIM H,et al.Enhanced Deep Residual Networks for Single Image Super-Resolution[C]//2017 IEEE Conference on Computer Vision & Pattern Recognition(CVPR).IEEE,2017.
[28]LEDIG C,THEIS L,HUSZAR F,et al.Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network[C]//2016 IEEE Conference on Computer Vision & Pat-tern Recognition(CVPR).IEEE,2016.

相关文章 15

[1]	戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮. 基于DNGAN的磁共振图像超分辨率重建算法 Super-resolution Reconstruction of MRI Based on DNGAN 计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105
[2]	黄璞, 杜旭然, 沈阳阳, 杨章静. 基于局部正则二次线性重构表示的人脸识别 Face Recognition Based on Locality Regularized Double Linear Reconstruction Representation 计算机科学, 2022, 49(6A): 407-411. https://doi.org/10.11896/jsjkx.210700018
[3]	黄璞, 沈阳阳, 杜旭然, 杨章静. 基于局部约束特征线表示的人脸识别 Face Recognition Based on Locality Constrained Feature Line Representation 计算机科学, 2022, 49(6A): 429-433. https://doi.org/10.11896/jsjkx.210300169
[4]	程祥鸣, 邓春华. 基于无标签知识蒸馏的人脸识别模型的压缩算法 Compression Algorithm of Face Recognition Model Based on Unlabeled Knowledge Distillation 计算机科学, 2022, 49(6): 245-253. https://doi.org/10.11896/jsjkx.210400023
[5]	魏勤, 李瑛娇, 娄平, 严俊伟, 胡辑伟. 基于边云协同的人脸识别方法研究 Face Recognition Method Based on Edge-Cloud Collaboration 计算机科学, 2022, 49(5): 71-77. https://doi.org/10.11896/jsjkx.210300222
[6]	周颖, 常明新, 叶红, 张燕. 基于元迁移的太阳能电池板缺陷图像超分辨率重建方法 Super Resolution Reconstruction Method of Solar Panel Defect Images Based on Meta-transfer 计算机科学, 2022, 49(3): 185-191. https://doi.org/10.11896/jsjkx.210100234
[7]	陈贵强, 何军. 自然场景下遥感图像超分辨率重建算法研究 Study on Super-resolution Reconstruction Algorithm of Remote Sensing Images in Natural Scene 计算机科学, 2022, 49(2): 116-122. https://doi.org/10.11896/jsjkx.210700095
[8]	何嘉玉, 黄宏博, 张红艳, 孙牧野, 刘亚辉, 周哲海. 基于深度学习的单幅图像三维人脸重建研究综述 Review of 3D Face Reconstruction Based on Single Image 计算机科学, 2022, 49(2): 40-50. https://doi.org/10.11896/jsjkx.210500215
[9]	陈长伟, 周晓峰. 快速局部协同表示分类器及其在人脸识别中的应用 Fast Local Collaborative Representation Based Classifier and Its Applications in Face Recognition 计算机科学, 2021, 48(9): 208-215. https://doi.org/10.11896/jsjkx.200800155
[10]	温荷, 罗频捷. 基于改进脉冲耦合神经网络的动态人脸识别 Dynamic Face Recognition Based on Improved Pulse Coupled Neural Network 计算机科学, 2021, 48(6A): 85-88. https://doi.org/10.11896/jsjkx.200600172
[11]	白子轶, 毛懿荣, 王瑞平. 视频人脸识别进展综述 Survey on Video-based Face Recognition 计算机科学, 2021, 48(3): 50-59. https://doi.org/10.11896/jsjkx.210100210
[12]	杨章静, 王文博, 黄璞, 张凡龙, 王昕. 基于局部加权表示的线性回归分类器及人脸识别 Local Weighted Representation Based Linear Regression Classifier and Face Recognition 计算机科学, 2021, 48(11A): 351-359. https://doi.org/10.11896/jsjkx.210100173
[13]	栾晓, 李晓双. 基于多特征融合的人脸活体检测算法 Face Anti-spoofing Algorithm Based on Multi-feature Fusion 计算机科学, 2021, 48(11A): 409-415. https://doi.org/10.11896/jsjkx.210100181
[14]	宋娅菲, 谌雨章, 沈君凤, 曾张帆. 基于改进残差网络的水下图像重建方法 Underwater Image Reconstruction Based on Improved Residual Network 计算机科学, 2020, 47(6A): 500-504. https://doi.org/10.11896/JsJkx.200100084
[15]	吴庆洪, 高晓东. 稀疏表示和支持向量机相融合的非理想环境人脸识别 Face Recognition in Non-ideal Environment Based on Sparse Representation and Support Vector Machine 计算机科学, 2020, 47(6): 121-125. https://doi.org/10.11896/jsjkx.190500058

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于超分辨率重建的低质量视频人脸识别方法

Low-quality Video Face Recognition Method Based on Super-resolution Reconstruction

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

Metrics

本文评价

推荐阅读 0