隨著網路的資源與服務不斷增加，電子商務與網路廣告也越來越普及化，同時影片分享網站也呈現爆發性成長，相對的網路使用者平均線上購物的次數以及觀看線上影片的時間也都隨之不斷增加。所以有許多廣告商透過影片分享網站來進行網路廣告的行銷，希望透過影片分享網站大量的使用者流量來替商品帶來更多收益；但是傳統隨機呈現的廣告機制已經無法滿足快速成長的電子商務與網路廣告市場需求。因此本研究將基於內容的圖像檢索技術應用在基於內容過濾的推薦系統，產生了基於網路影片內容的商品推薦系統，藉由分析影片內容的資訊來推薦更加接近目標興趣的商品。
本研究提出兩種方法來達到基於影片內容來推薦商品的目的：第一種方法為基於ASIFT的色彩區域比對方法，利用色彩以及區域兩種資訊來加強局部特徵ASIFT的圖像比對結果，藉由計算特徵點之間的區域色彩相似度、區域匹配密度以及區域幾何一致性三種區域特性來提升原本單點特徵比對的ASIFT所欠缺的物件特性，以減少語意鴻溝。第二種方法為基於內容相似度的推薦值計算方法，藉由影片的關鍵影格與商品圖像之間的關係所產生的商品相似度、商品出現率以及關鍵影格重要程度三種特性來計算商品的推薦值，藉此進行商品推薦。
最後本研究利用三種評估標準來分析與比較實驗結果在準確率與排名順序上的表現。從實驗結果可以發現，本研究的兩種方法之間相輔相成，有效提升整體的推薦結果，增加商品廣告與影片內容的相似度，提高使用者對於商品廣告的點閱興趣。

With the increasing network resource and services, e-commerce and online advertising are more and more popular and the number of video sharing websites also shows in explosive growth. Consequently, the frequency of online shopping and the time of watching online videos increase significantly too. Many advertisers do their advertisements through video sharing websites, hoping to get the maximum revenue due to large amount of website users. On the other hand, traditional advertising mechanism is unable to meet the rapid growth of e-commerce and online advertising markets. Therefore, this thesis proposes a content-based image retrieval approach for a product recommendation system by analyzing video contents so that the recommended products can be as matching as possible to a user’s needs.
Two methodologies are proposed by this thesis to improve the recommended result based on video content. Firstly, a color region matching that uses color and region information to enhance the local feature image matching result of ASIFT. By analyzing the color similarity degree, the density of matching points and geometric consistency of regions to overcome the drawbacks of lacking object characteristics and semantics. Secondly, a mechanism that calculates the recommendation value of a product based on the content similarity, the frequency of appearance and the importance of keyframes of the product. These features exist between the video keyframes and product images.
Finally, the study uses three kinds of evaluation standards to analyze and compare the performance on the precision and rank order of experimental results. The experimental results show that the proposed approach in this thesis have effectively improved the overall recommendation result on increasing the similarity between product advertising and video content and enhancing the users’ interest on viewing the product advertising.

[1] Imran Khan, "Nothing But Net: 2011 Internet Investment Guide," Global Equity Research , JP Morgan, 2011.
[2] P. Resnicka and H. R. Varian, "Recommender Systems," Communications of the ACM, vol. 40, no. 3, pp. 56-58, 1997.
[3] Alex Iskold, "The Art, Science and Business of Recommendation Engines," RWW(Read Write Web), 2007.
http://www.readwriteweb.com/archives/recommendation_engines.php
[4] G. Linden, B. Smith and J. York, "Amazon.com Recommendations Item-to-Item Collaborative Filtering," IEEE Internet Computing, vol. 7, no. 1, pp. 76-80, 2003.
[5] Matt Marshall, "Aggregate Knowledge raises $5M from Kleiner, on a roll," VentureBeat, 2006.
http://venturebeat.com/2006/12/10/aggregate-knowledge-raises-5m-from-kleiner-on-a-roll/
[6] J. Eakins and M. Graham, "Content-based Image Retrieval," Technical Report, University of Northumbria at Newcastle, 1999.
[7] G. Adomavicius and A. Tuzhilin, "Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions," IEEE Transactions on Knowledge and Data Engineering, vol.17, no.6, pp. 734-749, 2005.
[8] Q. Li and B. M. Kim, "Clustering Approach for Hybrid Recommender System," IEEE/WIC International Conference on Web Intelligence (WI), pp. 33-38, 2003.
[9] D. M. Nichols, "Implicit Rating and Filtering," DELOS Workshop on Filtering and Collaborative Filtering, pp. 31-36, 1997.
[10] Y. Liu, D. Zhang, G. Lu and W. Y. Ma, "A survey of content-based image retrieval with high-level semantics," Pattern Recognition, vol.40, no.1, pp. 262-282, 2007.
[11] J. Jeon, V. Lavrenko and R. Manmatha, "Automatic Image Annotation and Retrieval using Cross-Media Relevance Models," Annual International ACM SIGIR Conference on Research and development in information retrieval, pp. 119-126, 2003.
[12] Shih-Fu Chang, T. Sikora and A. Puri, "Overview of the MPEG-7 Standard," IEEE Transactions on Circuits and Systems for Video Technology, vol. 11, no. 6, pp. 688-695, 2001.
[13] A. R. Smith, "Color Gamut Transform Pairs," ACM SIGGRAPH Computer Graphics, vol.12, no.3, pp. 12-19, 1978.
[14] David G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[15] J. J. Koenderink, "The Structure of Images," Biological Cybernetics, vol. 50, no.5, 363-370, 1984.
[16] T. Lindeberg, "Scale-space theory: A basic tool for analyzing structures at different scales," Journal of Applied Statistics, vol. 21, no. 2, pp. 224-270, 1994.
[17] Yushi Jing and Shumeet Baluja, "PageRank for Product Image Search," International Conference on World Wide Web, pp. 307-316, 2008.
[18] J.M. Morel and G.Yu, "ASIFT: A New Framework for Fully Affine Invariant Image Comparison," SIAM Journal on Imaging Sciences, vol. 2, no. 2, 2009.
[19] G. Yu and J.M. Morel, "A Fully Affine Invariant Image Comparison Method," IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2009.
[20] J.M. Morel and G. Yu, "On the Consistency of the SIFT Method," Technical report, CMLA, ENS Cachan, Cachan, France, 2008.
[21] A.E. Abdel-Hakim and A.A. Farag, "CSIFT: A SIFT Descriptor with Color Invariant Characteristics," IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 1978-1983, 2006.
[22] J. M. Geusebroek, R. van den Boomgaard, A. W. M. Smeulders and H. Geerts, "Color invariance," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 12, pp. 1338-1350, 2001.
[23] G. J. Burghouts and J. M. Geusebroek, "Performance evaluation of local colour invariants," Computer Vision and Image Understanding (CVIU), vol. 113, no. 1, pp. 48-62, 2009.
[24] Koen E. A. van de Sande, Theo Gevers and Cees G. M. Snoek, "Evaluating Color Descriptors for Object and Scene Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1582-1596, 2010.
[25] K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir and L. Van Gool, "A comparison of affine region detectors," International Journal of Computer Vision, vol. 65, no. 1-2, pp. 43-72, 2005.
[26] J. Matas, O. Chum, M. Urban and T. Pajdla, "Robust Wide-Baseline Stereo from Maximally Stable Extremal Regions," Image and Vision Computing, vol. 22, no. 10, pp. 761–767, 2004.
[27] K. Mikolajczyk and C. Schmid, "Scale and Affine Invariant Interest Point Detectors," International Journal of Computer Vision, vol. 60, no. 1, pp. 63-86, 2004.
[28] K. Mikolajczyk, C. Schmid, "A performance evaluation of local descriptors," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 10, pp. 1615-1630, 2005.
[29] L. Lucchese and S. K. Mitra, "Color Image Segmentation: A State-of-The-Art Survey," Indian National Science Academy (INSA-A), vol. 67, A, no.2, pp. 207-221, 2001.
[30] Yong Rui, T. S. Huang and S. Mehrotra, "Exploring Video Structure Beyond the Shots," IEEE International Conference on Multimedia Computing and Systems, pp. 237-240, 1998.
[31] A. Hanjalic, "Shot-Boundary Detection: Unraveled and Resolved?," IEEE Transactions on Circuits and Systems for Video Technology, vol. 12, no. 2, pp. 90-105, 2002.
[32] Xiao Zhang, Gang Hua, Lei Zhang and Heung-Yeung Shum, "Interest Seam Image," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3296-3303, 2010.
[33] Xiaojun Guo and Fangxia Shi, "Quick Extracting Keyframes From Compressed Video," International Conference on Computer Engineering and Technology (ICCET), vol. 4, 163-165, 2010.
[34] M. Omidyeganeh, S. Ghaemmaghami and S. Shirmohammadi, "An event based approach to video analysis and keyframe selection," IEEE Workshop on Signal Processing Systems (SIPS), pp. 128-133, 2010.
[35] FFmpeg http://www.ffmpeg.org/
[36] Yan Ke and R. Sukthankar, "PCA-SIFT: A More Distinctive Representation for Local Image Descriptors," IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 506-513, 2004.
[37] H. Bay, A. Ess, T. Tuytelaars, L. V. Gool, "SURF: Speeded Up Robust Features," Computer Vision and Image Understanding (CVIU), vol. 110, no. 3, pp. 346-359, 2008.
[38] John R. Smith and Shih-Fu Chang, "VisualSEEk: a Fully Automated Content-Based Image Query System," ACM International Conference on Multimedia, pp. 87-98, 1996.
[39] John R. Smith, "Integrated Spatial and Feature Image Systems: Retrieval, Analysis and Compression," PhD Thesis Graduate School of Arts and Sciences, Columbia University, 1997.
[40] Zhong Wu Qifa Ke Isard and M. Jian Sun, "Bundling Features for Large Scale Partial- Duplicate Web Image Search," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 25-32, 2009.
[41] J. Kim and K. Grauman, "Asymmetric Region-to-Image Matching for Comparing Images with Generic Object Categories," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2344-2351, 2010.
[42] X. Ren and J. Malik. "Learning a classification model for segmentation," IEEE International Conference on Computer Vision (ICCV), vol. 1, pp. 10-17, 2003.
[43] G. Mori, X. Ren, A. Efros, J. Malik, "Recovering Human Body Configurations: Combining Segmentation and Recognition," IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 326-333, 2004.
[44] J. Shi, J. Malik, "Normalized Cuts and Image Segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, 2000.
[45] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, S. Susstrunk, "SLIC Superpixels," Technical Report, EPFL, 2010.
[46] Pedro F. Felzenszwalb and Daniel P. Huttenlocher, "Efficient Graph-Based Image Segmentation," International Journal of Computer Vision (IJCV), vol.59, no. 2, pp. 167-181, 2004.
[47] G. Mori, "Guiding Model Search Using Segmentation," IEEE International Conference on Computer Vision (ICCV), vol. 2, pp. 1417-1423, 2005.
[48] A. Levinshtein, A. Stere, K. Kutulakos, D. Fleet, S. Dickinson and K. Siddiqi, "TurboPixels: Fast Superpixels Using Geometric Flows," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 12, pp. 2290-2297 2009.
[49] A. Vedaldi and S. Soatto, "Quick Shift and Kernel Methods for Mode Seeking," European Conference on Computer Vision (ECCV), pp. 705-718, 2008.
[50] Tie-Yan Liu, Jun Xu, Tao Qin, Wenying Xiong and Hang Li, "LETOR: Benchmarking Learning to Rank for Information Retrieval," SIGIR Workshop on Learning to Rank for Information Retrieval (LR4IR), 2007.
[51] T. Qin, T. Liu, J. Xu, and H. Li, "LETOR: A Benchmark Collection for Research on Learning to Rank for Information Retrieval," Information Retrieval, vol. 13, no. 4, pp. 346-374, 2010.
[52] Lux Mathias and Savvas A. Chatzichristofis, "LIRe: Lucene Image Retrieval - An Extensible Java CBIR Library," ACM International Conference on Multimedia, pp. 1085-1088, 2008.
[53] H. Tamura, S. Mori, and T. Yamawaki, "Textural Features Corresponding to Visual Perception," IEEE Transactions on Systems, Man, and Cybernetics, vol. 8, no. 6, pp. 460–472, 1978.
[54] J. G. Daugman, "Complete Discrete 2-D Gabor Transforms by Neural Networks for Image Analysis and Compression," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 36, no. 7, pp. 1169-1179, 1988.
[55] J. Huang, S. R. Kumar, M. Mitra, W.-J. Zhu, and R. Zabih, "Image indexing using color correlograms," IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 762–768, 1997.
[56] S. A. Chatzichristofis and Y. S. Boutalis, "CEDD: Color and Edge Directivity Descriptor. A Compact Descriptor for Image Indexing and Retrieval," International conference on Computer vision systems (ICVS), vol. 5008, pp. 312–322, 2008.
[57] S. A. Chatzichristofis and Y. S. Boutalis, "FCTH: Fuzzy Color and Texture Histogram - A Low Level Feature for Accurate Image Retrieval," International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS), pp. 191–196, 2008.