Given a successive sequence of color and depth images of a face, this
research presents the development of a real-time 3D face reconstruc-
tion system. The core part of the system consists of three modules.
The rst module detects faces in the color images. The second module
extracts the facial features good for tracking the face across successive
color image frames. The third module exploits the matching across
successive depth image frames using point cloud models and iterative
closest points. Experiments show that the system can reconstruct 3D
face from various viewpoints, and distances.

Given a successive sequence of color and depth images of a face, this
research presents the development of a real-time 3D face reconstruc-
tion system. The core part of the system consists of three modules.
The rst module detects faces in the color images. The second module
extracts the facial features good for tracking the face across successive
color image frames. The third module exploits the matching across
successive depth image frames using point cloud models and iterative
closest points. Experiments show that the system can reconstruct 3D
face from various viewpoints, and distances.

[1] V. Blanz and T. Vetter. A morphable model for the synthesis
of 3D faces. In Proceedings of the 26th annual conference on Computer
graphics and interactive techniques, pages 187-194, New York, NY, USA,
1999.
[2] S. Roy and I.J. Cox. A maximum-flow formulation of the N-camera
stereo correspondence problem. In Computer Vision, 1998. Sixth In-
ternational Conference on, pages 492-499, Jan 1998.
[3] M. Dellepiane, A. Venturi, and R. Scopigno. Image guided re-
construction of un-sampled data: a coherent lling for uncomplete
Cultural Heritage models. In Computer Vision Workshops (ICCV Work-
shops), 2009 IEEE 12th International Conference on, pages 939-946, Oct. 4 2009.
[4] D. G. R. Bradski and A. Kaehler. Learning opencv, 1st edition. O'Reilly
Media, Inc., first edition, 2008.
[5] H. Bay, T. Tuytelaars, and L. Van Gool. SURF: Speeded Up
Robust Features. In Ale? Leonardis, Horst Bischof, and Axel
Pinz, editors, Computer Vision V ECCV 2006, 3951 of Lecture Notes in
Computer Science, pages 404-417. Springer Berlin / Heidelberg, 2006.
[6] B. Pinto and P.R. Anurenjan. Video stabilization using Speeded
Up Robust Features. In Communications and Signal Processing (ICCSP),
2011 International Conference, pages 527-531, Feb. 2011.
[7] Y. Hu, D. Jiang, S. Yan, L. Zhang, and H. Zhang. Automatic 3D
reconstruction for face recognition. In Automatic Face and Gesture
Recognition, 2004. Proceedings. Sixth IEEE International Conference,
pages 843-848, May 2004.
[8] J. Lee, B. Moghaddam, H. Pfister, and R. Machiraju. A bilinear
illumination model for robust face recognition. In Computer Vision,
2005. ICCV 2005. Tenth IEEE International Conference, pages 1177-1184 Vol. 2, Oct. 2005.
[9] Atick, J. Joseph, Griffin, A. Paul, A. Redlich, and Norman.
Statistical Approach to Shape from Shading: Reconstruction of
Three-Dimensional Face Surfaces from Single Two-Dimensional
Images. Neural Computation, 8(6):1321-1340, 1996.
[10] M. Reiter, R. Donner, G. Langs, and H. Bischof. Estimation of
face depth maps from color textures using canonical Correlation.
In Computer VisionWinter Workshop, 2006.
[11] Ph. Leclercq, J. Liu, A. Woodward, and P. Delmas. Which Stereo
Matching Algorithm for Accurate 3D Face Creation ? In Combina-
torial Image Analysis, 3322, pages 690-704. Springer Berlin / Heidelberg,
2005.
[12] Q. Chen and G. Medioni. Building 3-D Human Face Models from
Two Photographs. The Journal of VLSI Signal Processing, 27:127-140,
2001.
[13] G. Gimelfarb. Binocular Stereo by Maximizing the Likelihood
Ratio Relative to a Random Terrain. In Reinhard Klette, Shmuel
Peleg, and Gerald Sommer, editors, Robot Vision, 1998, pages 201-
209. Springer Berlin / Heidelberg, 2001.
[14] A.R. Chowdhury, R. Chellappa, S. Krishnamurthy, and T. Vo.
3D face reconstruction from video using a generic model. In Mul-
timedia and Expo, 2002. ICME '02. Proceedings. 2002 IEEE International
Conference, pages 449-452, 2002.
[15] P. J. Phillips, P. J. Flynn, T. Scruggs, K. W. Bowyer, J. Chang,
K. Hoffman, J. Marques, J. Min, and W. Worek. Overview of the
Face Recognition Grand Challenge. In In IEEE CVPR, pages 947-954,
2005.
[16] R. B. Rusu, Z. C. Marton, N. Blodow, M. Dolha, and M. Beetz.
Towards 3D Point cloud based object maps for household envi-
ronments. Robotics and Autonomous Systems, 56(11):927-941, 2008.
Semantic Knowledge in Robotics.
[17] A. Golovinskiy, V.G. Kim, and T. Funkhouser. Shape-based recog-
nition of 3D point clouds in urban environments. In Computer Vision,
2009 IEEE 12th International Conference, pages 2154-2161, Oct.
2 2009.
[18] N. Salman and M. Yvinec. High resolution surface reconstruction
from overlapping multiple-views. In Proceedings of the 25th annual
symposium on Computational geometry, SCG '09, pages 104-105, New York,
NY, USA, 2009.
[19] A. Lanitis and G. Stylianou. Evaluating the Performance of 3D
Face Reconstruction Algorithms. In Innovations and Advances in Com-
puter Sciences and Engineering, pages 153-158. Springer Netherlands, 2010.
[20] J. Oliensis. A Critique of Structure-from-Motion Algorithms. Com-
puter Vision and Image Understanding, 80(2):172-214, 2000.
[21] A. Lanitis and Stylianou. Reconstructing 3D faces in Cultural
Heritance Applications. In 14th International Conference on Virtual
Systems and Multimedia VSMM, 2008.
[22] B. Gorte and N. Pfeifer. Structuring Laser-Scanned Trees Using
3D Mathematical Morphology. pages 5-929, 2004.
[23] N. Pfeifer, B. Gorte, D. Winterhalder, R. Sensing, and
C. Range. Automatic reconstruction of single trees from terrestrial
laser scanner data. Current, XXXV(2001):1-6, 2004.
[24] G. Pavlidis, A. Koutsoudis, F. Arnaoutoglou, V. Tsioukas, and
C. Chamzas. Methods for 3D digitization of Cultural Heritage.
Journal of Cultural Heritage, 8(1):93-98, 2007.
[25] R. Li, T. Luo, and H. Zha. 3D Digitization and Its Applica-
tions in Cultural Heritage. In Marinos Ioannides, Dieter Fellner,
Andreas Georgopoulos, and Diofantos Hadjimitsis, editors, Digi-
tal Heritage, 6436 of Lecture Notes in Computer Science, pages 381-388.
Springer Berlin / Heidelberg, 2010.
[26] P. Viola and M. Jones. Rapid Object Detection using a Boosted
Cascade of Simple Features. Computer Vision and Pattern Recognition,
IEEE Computer Society Conference, 1:511, 2001.
[27] C. HARRIS. A combined corner and edge detector. Proc. Alvey
Vision Conf., 1988
[28] D. G. Lowe. Distinctive Image Features from Scale-Invariant Key-
points. International Journal of Computer Vision, 60:91-110, 2004.
[29] T. Tuytelaars and K. Mikolajczyk. Local invariant feature de-
tectors: a survey. Found. Trends. Comput. Graph. Vis., 3:177-280, July
2008.
[30] M. Brown and D. Lowe. Invariant Features from Interest Point
Groups. In In British Machine Vision Conference, pages 656-665, 2002. 18
[31] P. J. Besl and N. D. McKay. A Method for Registration of 3-D
Shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence,
14:239-256, 1992.