我們提出透過三個子網路架構的生成對抗網路(Triple Component Generative Adversarial Nets, TC-GAN)進行姿態/年齡變數分離(Pose/Age Variance)促進跨角度與跨年齡人臉識別。TC-GAN藉由分離分類器與判別器於單一網路架構，令分類器提供更穩定的分類效果，以及判別器使用權重正規化進行Lipchitz限制確保訓練過程提供穩定梯度給予生成器，藉此穩定訓練過程以及提升生成影像品質與人臉識別效能。且考慮到跨年齡/偏航角資料不均衡問題，採用Conditional Adversarial AutoEncoder (CAAE)與Nonlinear 3D Morphable Model (3DMM)進行偏航角與跨年齡樣本生成，同時提升辨識效能以及任意角度人臉生成之影像品質。在與其他方法的比較上，TC-GAN在公開測試資料庫MORPH與Multi-PIE均展現具競爭力之辨識率。

We propose the Disentangled Representation Learning on a Triple Component Generative Adversarial Network (TC-GAN), for handling cross-pose and cross-age face recognition. The proposed TC-GAN has improved state-of-the-art DR-GAN with the Spectral Normalization considered in the discriminator so that the generative and the adversarial framework can be better trained, and divide semi-supervised discriminator into two weights-independence Classifier and Discriminator for better classification capability. We also highlight the influences of pose/age imbalanced training data on the disentangled facial representation learning, and point out the difficulty of generating faces of extreme poses. We have explored the recently proposed nonlinear 3D Morphable Model (3DMM) and Conditional Adversarial Autoencoder (CAAE) to augment the training data, and verify the contributions made by the learning about augmented data. Experiments on the Multi-PIE and MORPH demonstrate the superiority of TC-GAN over other state-of-the-art approaches.

[1] Hsu, Gee-Sern, and Cheng-Hua Hsieh. "Multi-dropout regression for wide-angle landmark localization." 2017 IEEE International Conference on Image Processing (ICIP). IEEE, 2017.
[2] Whitelam, Cameron, et al. "Iarpa janus benchmark-b face dataset." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2017.
[3] Yi, Dong, et al. "Learning face representation from scratch." arXiv preprint arXiv:1411.7923 (2014).
[4] Tran, Luan, Xi Yin, and Xiaoming Liu. "Disentangled representation learning gan for pose-invariant face recognition." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017.
[5] Tran, Luan Quoc, Xi Yin, and Xiaoming Liu. "Representation learning by rotating your faces." IEEE transactions on pattern analysis and machine intelligence (2018).
[6] Zhu, Xiangyu, et al. "High-fidelity pose and expression normalization for face recognition in the wild." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.
[7] Bulat, Adrian, and Georgios Tzimiropoulos. "How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks)." Proceedings of the IEEE International Conference on Computer Vision. 2017.
[8] Zhu, Xiangyu, et al. "Face alignment across large poses: A 3d solution." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.
[9] Blanz, Volker, and Thomas Vetter. "A morphable model for the synthesis of 3D faces." Siggraph. Vol. 99. No. 1999. 1999.
[10] Paysan, Pascal, et al. "A 3D face model for pose and illumination invariant face recognition." Advanced video and signal based surveillance, 2009. AVSS'09. Sixth IEEE International Conference on. Ieee, 2009.
[11] Cao, Chen, et al. "Facewarehouse: A 3d facial expression database for visual computing." IEEE Transactions on Visualization and Computer Graphics 20.3 (2014): 413-425.
[12] Gross, Ralph, et al. "Multi-pie." Image and Vision Computing 28.5 (2010): 807-813.
[13] Ricanek, Karl, and Tamirat Tesafaye. "Morph: A longitudinal image database of normal adult age-progression." 7th International Conference on Automatic Face and Gesture Recognition (FGR06). IEEE, 2006.
[14] Lanitis, Andreas, Christopher J. Taylor, and Timothy F. Cootes. "Toward automatic simulation of aging effects on face images." IEEE Transactions on pattern Analysis and machine Intelligence 24.4 (2002): 442-455.
[15] Chen, Bor-Chun, Chu-Song Chen, and Winston H. Hsu. "Cross-age reference coding for age-invariant face recognition and retrieval." European Conference on Computer Vision. Springer, Cham, 2014.
[16] Canny, John Francis. Finding edges and lines in images. No. AI-TR-720. MASSACHUSETTS INST OF TECH CAMBRIDGE ARTIFICIAL INTELLIGENCE LAB, 1983.
[17] Xie, Saining, and Zhuowen Tu. "Holistically-nested edge detection." Proceedings of the IEEE international conference on computer vision. 2015.
[18] Liu, Yun, et al. "Richer convolutional features for edge detection." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
[19] Chongxuan, L. I., et al. "Triple generative adversarial nets." Advances in neural information processing systems. 2017.
[20] Wu, Yuxin, and Kaiming He. "Group normalization." Proceedings of the European Conference on Computer Vision (ECCV). 2018.
[21] Miyato, Takeru, et al. "Spectral normalization for generative adversarial networks." arXiv preprint arXiv:1802.05957 (2018).
[22] Ioffe, Sergey, and Christian Szegedy. "Batch normalization: Accelerating deep network training by reducing internal covariate shift." arXiv preprint arXiv:1502.03167 (2015).
[23] Arjovsky, Martin, Soumith Chintala, and Léon Bottou. "Wasserstein generative adversarial networks." International Conference on Machine Learning. 2017.
[24] Gulrajani, Ishaan, et al. "Improved training of wasserstein gans." Advances in Neural Information Processing Systems. 2017.
[25] Deng, Jiankang, et al. "Arcface: Additive angular margin loss for deep face recognition." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019.
[26] Masi, Iacopo, et al. "Pose-aware face recognition in the wild." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
[27] Masi, Iacopo, et al. "Learning pose-aware models for pose-invariant face recognition in the wild." IEEE Transactions on Pattern Analysis and Machine Intelligence 41.2 (2018): 379-393.
[28] Sengupta, Soumyadip, et al. "Frontal to profile face verification in the wild." Applications of Computer Vision (WACV), 2016 IEEE Winter Conference on. IEEE, 2016.
[29] Maze, Brianna, et al. "IARPA janus benchmark-c: Face dataset and protocol." 2018 International Conference on Biometrics (ICB). IEEE, 2018.
[30] He, Kaiming, et al. "Deep residual learning for image recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
[31] Klare, Brendan F., et al. "Pushing the frontiers of unconstrained face detection and recognition: Iarpa janus benchmark a." Proceedings of the IEEE conference on computer vision and pattern recognition. 2015..
[32] Parkhi, Omkar M., Andrea Vedaldi, and Andrew Zisserman. "Deep face recognition." bmvc. Vol. 1. No. 3. 2015.
[33] Zhang, Zhifei, Yang Song, and Hairong Qi. "Age progression/regression by conditional adversarial autoencoder." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017.
[34] Johnson, Justin, Alexandre Alahi, and Li Fei-Fei. "Perceptual losses for real-time style transfer and super-resolution." European conference on computer vision. Springer, Cham, 2016.
[35] Kingma, Diederik P., and Jimmy Ba. "Adam: A method for stochastic optimization." ICLR, 2015.
[36] Ba, Jimmy Lei, Jamie Ryan Kiros, and Geoffrey E. Hinton. "Layer normalization." arXiv preprint arXiv:1607.06450 (2016).
[37] Rothe, Rasmus, Radu Timofte, and Luc Van Gool. "Dex: Deep expectation of apparent age from a single image." Proceedings of the IEEE International Conference on Computer Vision Workshops. 2015.
[38] A. Li, S. Shan, W. Gao, “Coupled Bias–Variance Tradeoff for Cross-Pose Face Recognition,” in TIP , vol.21, no.1, pp.305-315, Jan. 2012
[39] Zhou, Huiling, and Kin-Man Lam. "Age-invariant face recognition based on identity inference from appearance age." Pattern Recognition 76 (2018): 191-202.
[40] Goodfellow, Ian, et al. "Generative adversarial nets." Advances in neural information processing systems. 2014.
[41] Springenberg, Jost Tobias. "Unsupervised and semi-supervised learning with categorical generative adversarial networks." arXiv preprint arXiv:1511.06390 (2015).
[42] Salimans, Tim, et al. "Improved techniques for training gans." Advances in neural information processing systems. 2016.
[43] Yin, Xi, et al. "Towards large-pose face frontalization in the wild." Proceedings of the IEEE International Conference on Computer Vision. 2017.
[44] Yim, Junho, et al. "Rotating your face using multi-task deep neural network." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.
[45] Chen, Jun-Cheng, Vishal M. Patel, and Rama Chellappa. "Unconstrained face verification using deep cnn features." 2016 IEEE winter conference on applications of computer vision (WACV). IEEE, 2016.
[46] Shen, Yujun, et al. "Faceid-gan: Learning a symmetry three-player gan for identity-preserving face synthesis." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.
[47] Li, Zhifeng, et al. "Aging face recognition: A hierarchical learning model based on local patterns selection." IEEE Transactions on Image Processing 25.5 (2016): 2146-2154.
[48] Chen, Bor-Chun, Chu-Song Chen, and Winston H. Hsu. "Face recognition and retrieval using cross-age reference coding with cross-age celebrity dataset." IEEE Transactions on Multimedia17.6 (2015): 804-815.
[49] Deng, Jiankang, et al. "Uv-gan: Adversarial facial uv map completion for pose-invariant face recognition." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.
[50] Peng, Xi, et al. "Reconstruction-based disentanglement for pose-invariant face recognition." Proceedings of the IEEE international conference on computer vision. 2017.
[51] Liu, Weiyang, et al. "Sphereface: Deep hypersphere embedding for face recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
[52] Schroff, Florian, Dmitry Kalenichenko, and James Philbin. "Facenet: A unified embedding for face recognition and clustering." Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.
[53] Wen, Yandong, et al. "A discriminative feature learning approach for deep face recognition." European conference on computer vision. Springer, Cham, 2016.
[54] Cao, Qiong, et al. "Vggface2: A dataset for recognising faces across pose and age." 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). IEEE, 2018.