本論文提出了基於空間通道注意力機制與時序嵌入傳遞模組之深度網路於即時物件追蹤系統，是一種基於孿生區域候選網路架構進行改良之物件追蹤技術，並且擁有良好的處理效能。
本論文所提出的空間通道注意力機制是為了使得網路模型能夠具有良好的泛化能力，使得萃取物件模板資訊時能夠針對各別的通道賦予不同權重值來進行特徵微調，當追蹤場景或物件外觀跟著時間而變化時仍能夠自適應的取得重要特徵。另外，本論文所提出的時序嵌入傳遞模組是為了解決以往孿生網路在物件尺度變化以及物件形變等問題。由於過往研究方法中並未將時間域的資訊考慮進去，因此我們有效利用時間域資訊帶來的好處，將前一影格之物件資訊藉由RoI嵌入層轉換成一維編碼後傳遞至當前影格，提供與當前影格中的目標物件更相近的特徵，這對後續在進行相似度匹配時能提高其判別性。
在實驗結果方面，本論文使用公開測試基準庫OTB以及VOT競賽之測試資料進行測試並與前人技術比較，儘管測試資料中存在不受控制的因素，如光影變化、物件快速移動等，但從結果可看出相較於前人所提出的技術，本論文所提出的算法皆可獲得良好的準確率及即時性，因此有相當大的潛力可被應用於現實生活中，並仍保有一定的準確性。

This study proposed a spatial-aware channel attention mechanism and temporal embedding propagation module based on deep learning architecture for real-time visual object tracking system. The proposed method is an advanced research based on the SiamRPN, and it is an end-to-end off-line trained network, and can provide real-time efficiency.
For the proposed spatial-aware channel attention mechanism, we apply it to reweight the channels while extracting the template feature and enhance the generalization power of the model. The system can benefit from the mechanism and find the representative feature which can adapt to the time-varied appearance of an object or background. For the proposed temporal embedding propagation module, we design it to address the scale changes and deformation problem. In our method, the module can efficiently utilize the advantage of temporal information between the adjacent frames. The object information in the previous frame can be transformed into a single dimension embedding vector using our designed RoI embedding layer and propagate to the current frame. The operation can provide the current feature of the target object, and can also increase the discriminative power in the similarity stage.
This study conducts several of experiments on the public benchmark OTB and the VOT challenges, and is compared against previous works. Though there are uncontrolled factors in the videos, such as illumination changes and fast motion, the proposed method can achieve superior accuracy than the former schemes. Thus, the proposed method has considerable potential to be applied in the practical applications.

[1] B.Zhou, A.Lapedriza, J.Xiao, A.Torralba, andA.Oliva, “Learning deep features for scene recognition using places database,” in Advances in neural information processing systems, 2014, pp. 487–495.
[2] S. C.Park, M. K.Park, andM. G.Kang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag., vol. 20, no. 3, pp. 21–36, 2003.
[3] J.Redmon, S.Divvala, R.Girshick, andA.Farhadi, “You only look once: Unified, real-time object detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779–788.
[4] B.Zoph, V.Vasudevan, J.Shlens, andQ.VLe, “Learning transferable architectures for scalable image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 8697–8710.
[5] S.Ojha andS.Sakhare, “Image processing techniques for object tracking in video surveillance-A survey,” in 2015 International Conference on Pervasive Computing (ICPC), 2015, pp. 1–6.
[6] B. D.Lucas, T.Kanade, andothers, “An iterative image registration technique with an application to stereo vision,” 1981.
[7] A.Adam, E.Rivlin, andI.Shimshoni, “Robust fragments-based tracking using the integral histogram,” in 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), 2006, vol. 1, pp. 798–805.
[8] N.Alt, S.Hinterstoisser, andN.Navab, “Rapid selection of reliable templates for visual tracking,” in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010, pp. 1355–1362.
[9] G. D.Hager andP. N.Belhumeur, “Efficient region tracking with parametric models of geometry and illumination,” IEEE Trans. Pattern Anal. Mach. Intell., no. 10, pp. 1025–1039, 1998.
[10] D. A.Ross, J.Lim, R.-S.Lin, andM.-H.Yang, “Incremental learning for robust visual tracking,” Int. J. Comput. Vis., vol. 77, no. 1–3, pp. 125–141, 2008.
[11] P.Pérez, C.Hue, J.Vermaak, andM.Gangnet, “Color-based probabilistic tracking,” in European Conference on Computer Vision, 2002, pp. 661–675.
[12] D.Comaniciu, V.Ramesh, andP.Meer, “Kernel-based object tracking,” IEEE Trans. Pattern Anal. Mach. Intell., no. 5, pp. 564–575, 2003.
[13] T.Robert andothers, “Mean-shift blob tracking through scale space,” in IEEE Conference on Computer Vision and Pattern Recognition, 2003, vol. 2, pp. 234–240.
[14] S. T.Birchfield andS.Rangarajan, “Spatiograms versus histograms for region-based tracking,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), 2005, vol. 2, pp. 1158–1163.
[15] F.Tang, S.Brennan, Q.Zhao, andH.Tao, “Co-tracking using semi-supervised support vector machines,” in 2007 IEEE 11th International Conference on Computer Vision, 2007, pp. 1–8.
[16] N.Dalal andB.Triggs, “Histograms of oriented gradients for human detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2005, vol. 1, pp. 886–893.
[17] K.Zhang, L.Zhang, andM.-H.Yang, “Real-time compressive tracking,” in European conference on computer vision, 2012, pp. 864–877.
[18] B.Babenko, M.-H.Yang, andS.Belongie, “Robust object tracking with online multiple instance learning,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 8, pp. 1619–1632, 2010.
[19] H.Grabner, M.Grabner, andH.Bischof, “Real-time tracking via on-line boosting.,” in Bmvc, 2006, vol. 1, no. 5, p. 6.
[20] T.Ojala, M.Pietikäinen, andT.Mäenpää, “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns,” IEEE Trans. Pattern Anal. Mach. Intell., no. 7, pp. 971–987, 2002.
[21] P.Viola, M.Jones, andothers, “Rapid object detection using a boosted cascade of simple features,” CVPR, vol. 1, no. 511–518, p. 3, 2001.
[22] G. E.Hinton andR. R.Salakhutdinov, “Reducing the dimensionality of data with neural networks,” Science (80-. )., vol. 313, no. 5786, pp. 504–507, 2006.
[23] A.Krizhevsky, I.Sutskever, andG. E.Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 2012, pp. 1097–1105.
[24] J.Kwon andK. M.Lee, “Visual tracking decomposition,” in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010, pp. 1269–1276.
[25] W.Hu, X.Li, X.Zhang, X.Shi, S.Maybank, andZ.Zhang, “Incremental tensor subspace learning and its applications to foreground segmentation and tracking,” Int. J. Comput. Vis., vol. 91, no. 3, pp. 303–327, 2011.
[26] X.Li, W.Hu, Z.Zhang, X.Zhang, M.Zhu, andJ.Cheng, “Visual tracking via incremental log-euclidean riemannian subspace learning,” in 2008 IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp. 1–8.
[27] X.Mei andH.Ling, “Robust visual tracking using ℓ 1 minimization,” in 2009 IEEE 12th International Conference on Computer Vision, 2009, pp. 1436–1443.
[28] X.Mei, H.Ling, Y.Wu, E.Blasch, andL.Bai, “Minimum error bounded efficient ℓ 1 tracker with occlusion detection,” in CVPR 2011, 2011, pp. 1257–1264.
[29] C.Bao, Y.Wu, H.Ling, andH.Ji, “Real time robust l1 tracker using accelerated proximal gradient approach,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 1830–1837.
[30] H.Li, C.Shen, andQ.Shi, “Real-time visual tracking using compressive sensing,” in CVPR 2011, 2011, pp. 1305–1312.
[31] S.Avidan, “Support vector tracking,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 8, pp. 1064–1072, 2004.
[32] Y.Bai andM.Tang, “Robust tracking via weakly supervised ranking SVM,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 1854–1861.
[33] S.Hare et al., “Struck: Structured output tracking with kernels,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 10, pp. 2096–2109, 2015.
[34] D. S.Bolme, J. R.Beveridge, B. A.Draper, andY. M.Lui, “Visual object tracking using adaptive correlation filters,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010, pp. 2544–2550.
[35] Z.Chen, Z.Hong, andD.Tao, “An experimental survey on correlation filter-based tracking,” arXiv Prepr. arXiv1509.05520, 2015.
[36] J. F.Henriques, R.Caseiro, P.Martins, andJ.Batista, “High-speed tracking with kernelized correlation filters,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 3, pp. 583–596, 2014.
[37] J. F.Henriques, R.Caseiro, P.Martins, andJ.Batista, “Exploiting the circulant structure of tracking-by-detection with kernels,” in Proceedings of the European Conference on Computer Vision (ECCV), 2012, pp. 702–715.
[38] Y.Li andJ.Zhu, “A scale adaptive kernel correlation filter tracker with feature integration,” in Proceedings of the European Conference on Computer Vision (ECCV), 2014, pp. 254–265.
[39] M.Danelljan, G.Hager, F.Shahbaz Khan, andM.Felsberg, “Learning spatially regularized correlation filters for visual tracking,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2015, pp. 4310–4318.
[40] M.Danelljan, G.Häger, F.Khan, andM.Felsberg, “Accurate scale estimation for robust visual tracking,” in British Machine Vision Conference, Nottingham, September 1-5, 2014, 2014.
[41] J.Huang, X.Yang, andM.Yang, “Hierarchical Convolutional Features for Visual Tracking,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2015, pp. 3074–3082.
[42] M.Danelljan, A.Robinson, F. S.Khan, andM.Felsberg, “Beyond correlation filters: Learning continuous convolution operators for visual tracking,” in Proceedings of the European Conference on Computer Vision (ECCV), 2016, pp. 472–488.
[43] M.Danelljan, G.Bhat, F.Shahbaz Khan, andM.Felsberg, “ECO: efficient convolution operators for tracking,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 6638–6646.
[44] G.Bhat, J.Johnander, M.Danelljan, F.Shahbaz Khan, andM.Felsberg, “Unveiling the power of deep tracking,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 483–498.
[45] M.Danelljan, G.Hager, F.Shahbaz Khan, andM.Felsberg, “Convolutional features for correlation filter based visual tracking,” in Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV Workshops), 2015, pp. 58–66.
[46] Z.Zhu, W.Wu, W.Zou, andJ.Yan, “End-to-End Flow Correlation Tracking with Spatial-Temporal Attention,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 548–557, 2018.
[47] H.Nam andB.Han, “Learning Multi-domain Convolutional Neural Networks for Visual Tracking.,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 4293–4302.
[48] I.Jung, J.Son, M.Baek, andB.Han, “Real-time mdnet,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 83–98.
[49] R.Tao, E.Gavves, andA. W. M.Smeulders, “Siamese instance search for tracking,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 1420–1429.
[50] L.Bertinetto, J.Valmadre, J. F.Henriques, A.Vedaldi, andP. H. S.Torr, “Fully-Convolutional Siamese Networks for Object Tracking.,” in Proceedings of the European Conference on Computer Vision Workshops (ECCV Workshops), 2016, vol. 9914, pp. 850–865.
[51] J.Valmadre, L.Bertinetto, A.Vedaldi, andP. H. S.Torr, “End-to-end representation learning for Correlation Filter based tracking,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 2805–2813.
[52] D.Held, S.Thrun, andS.Savarese, “Learning to Track at 100 FPS with Deep Regression Networks.,” in Proceedings of the European Conference on Computer Vision (ECCV), 2016, vol. 9905, pp. 749–765.
[53] A.He, C.Luo, X.Tian, andW.Zeng, “A Twofold Siamese Network for Real-Time Object Tracking,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 4834–4843.
[54] B.Li, J.Yan, W.Wu, Z.Zhu, andX.Hu, “High Performance Visual Tracking with Siamese Region Proposal Network,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 8971–8980.
[55] Q.Wang, Z.Teng, J.Xing, J.Gao, W.Hu, andS. J.Maybank, “Learning Attentions: Residual Attentional Siamese Network for High Performance Online Visual Tracking.,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 4854–4863.
[56] Q.Guo, W.Feng, C.Zhou, R.Huang, L.Wan, andS.Wang, “Learning Dynamic Siamese Network for Visual Object Tracking,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017.
[57] T.Yang andA. B.Chan, “Learning dynamic memory networks for object tracking,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 152–167.
[58] Z.Zhu, Q.Wang, B.Li, W.Wu, J.Yan, andW.Hu, “Distractor-Aware Siamese Networks for Visual Object Tracking.,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, vol. 11213, pp. 103–119.
[59] C.Huang, S.Lucey, andD.Ramanan, “Learning policies for adaptive tracking with deep feature cascades,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 105–114.
[60] S.Ren, K.He, R.Girshick, andJ.Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” in Advances in neural information processing systems, C.Cortes, N. D.Lawrence, D. D.Lee, M.Sugiyama, andR.Garnett, Eds.Curran Associates, Inc., 2015, pp. 91–99.
[61] Y.Wu, J.Lim, andM.-H.Yang, “Online object tracking: A benchmark,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013, pp. 2411–2418.
[62] Y.Wu, J.Lim, andM.-H.Yang, “Object Tracking Benchmark.,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 9, pp. 1834–1848, 2015.
[63] M.Kristan et al., “The Visual Object Tracking VOT2015 Challenge Results.,” in Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV Workshops), 2015, pp. 564–586.
[64] M.Kristan et al., “The Visual Object Tracking VOT2016 Challenge Results,” in Proceedings of the European Conference on Computer Vision Workshops (ECCV Workshops), 2016, pp. 777–823.
[65] M.Kristan et al., “The Visual Object Tracking VOT2017 Challenge Results.,” in Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV Workshops), 2017, pp. 1949–1972.
[66] M.Kristan et al., “The Sixth Visual Object Tracking VOT2018 Challenge Results.,” in Proceedings of the European Conference on Computer Vision Workshops (ECCV Workshops), 2018, vol. 11129, pp. 3–53.
[67] K.Simonyan andA.Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv Prepr. arXiv1409.1556, 2014.
[68] C.Szegedy et al., “Going deeper with convolutions,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1–9.
[69] K.He, X.Zhang, S.Ren, andJ.Sun, “Deep Residual Learning for Image Recognition.,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778.
[70] N.Wang andD.-Y.Yeung, “Learning a deep compact image representation for visual tracking,” in Advances in neural information processing systems, 2013, pp. 809–817.
[71] N.Wang, S.Li, A.Gupta, andD.-Y.Yeung, “Transferring rich feature hierarchies for robust visual tracking,” arXiv Prepr. arXiv1501.04587, 2015.
[72] F.Schroff, D.Kalenichenko, andJ.Philbin, “Facenet: A unified embedding for face recognition and clustering,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 815–823.
[73] Y.Taigman, M.Yang, M.Ranzato, andL.Wolf, “Deepface: Closing the gap to human-level performance in face verification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 1701–1708.
[74] G.Koch, R.Zemel, andR.Salakhutdinov, “Siamese neural networks for one-shot image recognition,” in ICML deep learning workshop, 2015, vol. 2.
[75] J.Hu, L.Shen, andG.Sun, “Squeeze-and-Excitation Networks.,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 7132–7141.
[76] S.Woo, J.Park, J.-Y.Lee, andI. S.Kweon, “CBAM: Convolutional Block Attention Module.,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, vol. 11211, pp. 3–19.
[77] O.Russakovsky et al., “ImageNet Large Scale Visual Recognition Challenge.,” Int. J. Comput. Vis., vol. 115, no. 3, pp. 211–252, 2015.
[78] T.-Y.Lin et al., “Microsoft COCO: Common Objects in Context.,” in ECCV, 2014, vol. 8693, pp. 740–755.
[79] E.Real, J.Shlens, S.Mazzocchi, X.Pan, andV.Vanhoucke, “YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in Video.,” in CVPR, 2017, pp. 7464–7473.
[80] L.Huang, X.Zhao, andK.Huang, “Got-10k: A large high-diversity benchmark for generic object tracking in the wild,” arXiv Prepr. arXiv1810.11981, 2018.
[81] Y.LeCun et al., “Backpropagation applied to handwritten zip code recognition,” Neural Comput., vol. 1, no. 4, pp. 541–551, 1989.
[82] B.Chen, D.Wang, P.Li, S.Wang, andH.Lu, “Real-time’Actor-Critic’Tracking,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 318–334.
[83] H.Fan andH.Ling, “Parallel tracking and verifying: A framework for real-time and high accuracy visual tracking,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 5486–5494.
[84] H.Kiani Galoogahi, A.Fagg, andS.Lucey, “Learning background-aware correlation filters for visual tracking,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 1135–1143.
[85] Y.Song, C.Ma, L.Gong, J.Zhang, R. W. H.Lau, andM.-H.Yang, “Crest: Convolutional residual learning for visual tracking,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2555–2564.
[86] J.Choi et al., “Context-aware deep feature compression for high-speed visual tracking,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 479–488.
[87] L.Bertinetto, J.Valmadre, S.Golodetz, O.Miksik, andP. H. S.Torr, “Staple: Complementary learners for real-time tracking,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1401–1409.
[88] H.Nam, M.Baek, andB.Han, “Modeling and propagating cnns in a tree structure for visual tracking,” arXiv Prepr. arXiv1608.07242, 2016.
[89] A.He, C.Luo, X.Tian, andW.Zeng, “Towards a better match in siamese network based visual object tracker,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, p. 0.
[90] C.Sun, D.Wang, H.Lu, andM.-H.Yang, “Learning spatial-aware regressions for visual tracking,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 8962–8970.
[91] Z.He, Y.Fan, J.Zhuang, Y.Dong, andH.Bai, “Correlation filters with weighted convolution responses,” in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 1992–2000.