我們提出邊緣輔助之人臉地標點混合網路 (Hybrid Convolutional Neural Network (HCNN) for facial landmark detection)，所提出之架構能更精準預測人臉輪廓與五官之地標點。HCNN Framework對於人臉地標點偵測提出了有效的解決方案，主要分為兩個區塊，分別為LSN (Line-Segment Network)，以及CDN (Convolution-Dropout Network)。在一個訓練集中，首先我們連結每個人臉圖像中相鄰的地標點，並定義其地標點的相關性像作為LSN的輸出，經過訓練後的LSN被作為CDN的預處理器，其面部圖像與LSN生成的地標點相關之線段作為其輸入，而最終輸出其人臉地標點。CDN不同於以往將Dropout Layer置放Fully Connected Layers後的做法，而是將其Dropout Layer放置在Convolution Layer後，進一步的強化其地標的預測準確性與減緩其過擬合Over-fitting的現象。本研究旨在探索HCNN生成線段特徵對於人臉地標點之定位影響，藉此處理人臉地標點定位問題。HCNN提出下列優點與觀察：(1) LSN的輔助對於CDN產生更進一步的緩解其過擬合Over-fitting現象、(2) LSN生成之地標點相關之線段特徵圖增加其地標點定位的準確性。本研究在公開資料庫300W與AFLW中與近年方法進行比較，證實HCNN於跨視角人臉地標點偵測具有競爭力。

We propose the Hybrid Convolutional Neural Network (HCNN) for facial landmark detection. The HCNN considers the hierarchical facial features extracted from the multiple convolutional layers in a pre-trained VGG network, and exploits these hierarchical features for locating facial landmarks. The HCNN is composed of two subnetworks, namely the Line-Segment Network (LSN) and the Convolution-Dropout Network (CDN). Given a training set, we first connect the neighboring landmarks in each image to form the landmarked line segments, and define the landmarked line segments as the output of the LSN with the associated face image as the input. The LSN is trained to estimate the landmarked line segments for an input facial image. The trained LSN is then exploited as a preprocessor for the CDN, which takes the face image and the associated landmarked line segments, generated by the LSN, as inputs and the landmarks as output. The CDN has dropouts implemented at convolutional layers, instead of at the common fully-connected layers, to further enhance the network robustness against Over-fitting and improve the accuracy of landmark localization. Due to the hierarchical features embedded in the landmarked line segment learning, the proposed HCNN demonstrates state-of-the-art performance on several benchmark databases.

摘要 V
Abstract VI
誌謝 VII
圖目錄 XI
第一章 介紹 1
1.1 研究背景和動機 1
1.2 論文貢獻 3
1.3 論文架構 4
第二章 文獻回顧 5
2.1 泛物件偵測相關文獻 6
2.1.1 SSD 物件偵測器 6
2.1.2 YOLO9000 物件偵測器 8
2.2 邊緣檢測相關文獻 9
2.2.1 Holistically-Nested Edge Detection 9
2.3 人臉地標點偵測相關文獻 11
2.3.1 Cascaded Convolutional Neural Network 11
2.3.2 Multi-Task Convolutional Neural Network 12
2.3.3 CNN-based 3D Model Fitting for Landmark Localization 13
2.3.4 Shape Searching for Landmark Localization 15
2.3.5 Style Aggregated Network for Facial Landmark Detection 16
第三章 主要方法 18
3.1 Facial Detection Network 19
3.1.1 YOLO2 人臉偵測器 19
3.2 Line-Segment Network 21
3.2.1 Network Architecture 21
3.2.2 Comparison With HED 24
3.3 Hybrid Convolutional Network for Facial Landmark Detection 25
3.3.1 Convolution-Dropout Network 25
3.3.2 側面CDN 預測Landmark訓練階段 27
3.4 Facial Pose Classification Network 28
第四章 實驗設置與分析 29
4.1 人臉臉側之公開資料庫介紹 29
4.1.1 Wider Face 29
4.1.2 AFW 30
4.1.3 PASCAL Faces 30
4.2 人臉地標點之公開資料庫介紹 31
4.2.1 300W 31
4.2.2 300W-LPA 32
4.2.3 AFLW 33
4.3 實驗樣本規格 34
4.3.1 影像前處理 34
4.3.2 Data Augmentation 34
4.4 實驗結果與分析 34
4.4.1 人臉偵測器效能比較 35
4.4.2 最佳線條輔助網路之比較 36
4.4.3 正、側面相關人臉地標點與相關定位地標點演算法比較 37
第五章 結論與未來研究方向 41
第六章 參考文獻 42