近幾年眼動儀在各大領域上的應用大量興起，例如：行銷／廣告、注意力、圖像知覺、問題解決以及心理學等。透過眼動儀獲取到的各項特徵，例如：眼跳(saccade)、凝視(fixation)、平滑追視(smooth pursuit)、集散行為(vergence)及前庭眼反射(vestibular ocular reflex)，不僅可以分析操作者的熟練程度，也可以推斷使用者是否對某項產品感興趣。然而，眼動儀的軟體僅具備基本的眼動特徵並沒有後續的分析結果，並且在收集資料的同時，影像儘管也被收集卻沒有很好的利用到。因此，本研究提出結合眼動數據及影像資料的方法，藉由兩者的結合期望提升整體的分類效果。
本研究使用真實世界資料之樁轉移任務進行個案研究分析，並使用正確率評比指標衡量提出方法的結果。根據實驗結果證實，相較於只使用眼動數據及影像資料，兩者結合後的結果能獲得較優異的表現。

The applications of eye-tracking system have been widely used in various fields such as commercial/marketing, attention, scene perception, problem solving, etc. Through the information extracted from eyes such as saccade, fixation, smooth pursuit, vergence and vestibular ocular reflex, not only the skill level of operators can be evaluated, but can also infer whether the users are interesting in some product or not. However, the eye-tracking software only provides basic features of eye movements and follow-up analysis is not available. While acquiring the eye-tracking data, video data is also stored but not utilized sufficiently. Therefore, this study proposed a method, which combines eye-tracking data and video in order to achieve better classification results.
This study uses real-world data, which is generated from peg-transfer task for case
study. Accuracy indicator is applied to measure the performance of the proposed algorithm. According to the experimental results, combining both data is able to provide better results comparing to those only using individual one.

Abanda, A., Mori, U. and Lozano, J.A., A review on distance-based time series classification, Data Mining and Knowledge Discovery 33, 378-412, 2019.
Ahmad, I., Basheri, M., Iqbal, M.J. and Rahim, A., Performance comparison of support vector machine, random forest, and extreme learning machine for intrusion detection, IEEE Access 6, 33789-33795, 2018.
Bagnall, A., Lines, J., Bostrom, A. and Large, J., The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances, Data Mining and Knowledge Discovery 31, 606-660, 2017.
Breiman, L., Random forests, Machine Learning 45, 1, 5-32, 2001.
Brockwell, P.J. and Richard A. D., Introduction to Time Series and Forecasting. New York: Springer, 2002.
Castro, N. and Azevedo, P., Multiresolution motif discovery in time series, Proceedings of the 2010 SIAM International Conference on Data Mining, 665-676, 2010.
Chen, Y.X. and Wang, J., Image categorization by learning and reasoning with regions, Journal of Machine Learning Research 5, 913-939, 2004.
Cover, T. and Hart, P., Nearest neighbor pattern classification, IEEE Transactions on Information Theory 13, 1, 21-27, 1967.
Cortes, C. and Vapnik, V., Support-vector networks. Machine Learning, 20, 273-297, 1995.
Chen, T.Q. and Guestrin, C., Xgboost: A scalable tree boosting system. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 785-794. ACM, 2016.
Duro, D.C., Franklin, S.E. and Dubé, M.G., A comparison of pixel-based and object-based image analysis with selected machine learning algorithms for the classification of agricultural landscapes using SPOT-5 HRG imagery, Remote Sensing of Environment 118, 259-272, 2012.
Deng, H., Runger, G., Tuv, E. and Vladimir, M., A time series forest for classification and feature extraction, Information Sciences 239, 142-153, 2013.
Ding, H., Trajcevski, G., Scheuermann, P., Wang, X. and Keogh, E., Querying and mining of time series data: experimental comparison of representations and distance measures, Proc VLDB Very Large Database Endow 1, 2, 1542-1552, 2008.
Esling, P. and Agon, C., Time-series data mining, ACM Computing Surveys, Association for Computing Machinery 45, 1, 12, 1-34, 2012.
Freund, Y. and Schapire, R.E., A decision-theoretic generalization of on-line learning and an application to boosting, Journal of Computer and System Sciences 55, 1, 119-139, 1997.
Friedman, J., Greedy function approximation: A gradient boosting machine. Annals of Statistics 29, 1189-1232, 2001.
Gordon, D., Hendler, D. and Rokach, L., Fast randomized model generation for shapelet-based time series classification, 2012.
Gordon, D., Hendler, D. and Rokach, L., Fast and space-efficient shapelets-based time-series classification, Intelligent Data Analysis 19, 953-981, 2015.
Grabocka, J., Schilling, N., Wistuba, M. and Schmidt-Thieme, L., Learning time-series shapelets, Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (SIGKDD), 392-401, 2014.
Geurts, P., Ernst, D. and Wehenkel, L., Extremely randomized trees, Machine Learning 63, 3-42, 2006.
Gudmundsson, S., Runarsson, T.P. and Sigurdsson, S., Support vector machines and dynamic time warping for time series, Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), 2772-2776, 2008.
Hacibeyoglu, M., Human gender prediction on facial mobil images using convolutional neural networks, International Journal of Intelligent Systems and Applications in Engineering 3, 203-208, 2018.
Hastie, T., Tibshirani, R., and Friedman, J. H., The elements of statistical learning: data mining, inference, and prediction. 2nd ed. New York: Springer, 2009.
Hills, J., Lines, J., Baranauskas, E., Mapp, J., and Bagnall, A., Classification of time series by shapelet transformation, Data Mining and Knowledge Discovery 28, 4, 851-881, 2014.
Huang, G.B., Zhu, Q.Y. and Siew, C.K., Extreme learning machine: Theory and applications, Neurocomputing 70, 1-3, 489-501, 2006.
Hubel, D.H., and Wiesel, T.N., Receptive fields of single neurones in the cat's striate cortex, The Journal of Physiology 148, 574-591, 1959.
Jeffrey, H.P., Gerald, M.F., Lee, L.S., Nathaniel, J.S., Lelan, F.S., Bruce, S., Kaaren, H. and the SAGES FLS Committee., Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic Surgery, Surgery 135, 1, 21-27, 2004.
Kasten, E.P., McKinley, P.K. and Gage, S.H., Automated ensemble extraction and analysis of acoustic data streams, 27th International Conference on Distributed Computing Systems Workshops (ICDCSW'07), 66-66, 2007.
Keogh, E. and Kasetty, S., On the need for time series data mining benchmarks: A survey and empirical demonstration, Data Mining and Knowledge Discovery 7, 349-371, 2003.
Krizhevsky, A., Sutskever, I. and Hinton, G., Imagenet classification with deep convolutional neural networks, Neural Information Processing Systems, 2012.
Kamavisdar, P., Saluja, S. and Agrawal, S., A survey on image classification approaches and techniques. International Journal of Advanced Research in Computer and Communication Engineering 2, 1, 1005-1008, 2013.
Lin, J., Keogh, E., Wei, L. and Lonardi, S., Experiencing SAX: a novel symbolic representation of time series, Data Mining and Knowledge Discovery 15, 2, 107-144, 2007.
Lines, J., Taylor, S. and Bagnall, A., HIVE-COTE: The hierarchical vote collective of transformation-based ensembles for time series classification, 2016 IEEE 16th International Conference on Data Mining (ICDM), 1041-1046, 2016.
Lu, D. and Weng, Q., A survey of image classification methods and techniques for improving classification performance, International Journal of Remote Sensing 28, 5, 823-870, 2007.
Lecun, Y., Bottou, L., Bengio, Y. and Haffner, P., Gradient-based learning applied to document recognition, Proceedings of the IEEE 86, 11, 2278-2324, 1998.
Lewis, K.G., Teaching large classes (how to do it well and remain sane). In K.W. Prichard & R. M. Sawyer (Eds.), Handbook of College Teaching: Theory and Applications, 319-343, 1994.
Lin, M., Chen, Q., and Yan, S., Network in network, International Conference on Learning Representations, 2014.
Lines, J., Davis, L.M., Hills, J., Bagnall, A., A shapelet transform for time series classification, Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 289-297, 2012.
Löning, M., Bagnall, A., Ganesh, S., Kazakov, V., Lines, J. and Király, F., sktime: A unified interface for machine learning with time series, 2019.
Mueen, A., Keogh, E., and Young, N., Logical-shapelets: an expressive primitive for time series classification, In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'11). Association for Computing Machinery, 1154-1162, 2011.
Ma, L., Li, M., Ma, X.X., Cheng, L., Du, P.J. and Liu, Y.X., A review of supervised object-based land-cover image classification, ISPRS Journal of Photogrammetry and Remote Sensing 130, 277-293, 2017.
Nguyen, T., Gsponer, S., and Ifrim, G., Time series classification by sequence learning in all-subsequence space, 2017 IEEE 33rd International Conference on Data Engineering (ICDE), 947-958, 2017.
Oshiro, T.M., Perez, P.S., and Baranauskas, J.A., How many trees in a random forest? Proceedings of the International Workshop on Machine Learning and Data Mining in Pattern Recognition, 154-168, 2012.
Rosch, J.L., and Vogel-Walcutt, J.J., A review of eye-tracking applications as tools for training, Cognition, Technology & Work 15, 313-327, 2012.
Rakthanmanon, T., Keogh, E., Fast Shapelets: A scalable algorithm for discovering time series shapelets, Proceedings of the thirteenth SIAM Conference on Data Mining (SDM), 668-676, 2013.
Renard, X., Rifqi, M., Erray, W. and Detyniecki, M., Random-shapelet: An algorithm for fast shapelet discovery, IEEE International Conference on Data Science and Advanced Analytics (DSAA), 1-10, 2015.
Simonyan, K. and Zisserman, A., Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representations, 2014.
Senin, P. and Malinchik, S., SAX-VSM: Interpretable time series classification using SAX and vector space model, IEEE 13th International Conference on Data Mining, 1175-1180, 2013.
Shojaeizadeh, M., Djamasbi, S., Paffenroth, R.C. and Trapp, A.C., Detecting task demand via an eye tracking machine learning system, Decision Support Systems 116, 91-101, 2019.
Szegedy, C., Ioffe, S. and Vanhoucke, V., Inception-v4, inception-resnet and the impact of residual connections on learning, Proceedings of the AAAI Conference on Artificial Intelligence, 4278-4284, 2017.
Salminen, J., Nagpal, M., Kwak, H., An, J., Jung, S.G. and Jansen, B.J., Confusion prediction from eye-tracking data: Experiments with machine learning, Proceedings of the 9th International Conference on Information Systems and Technologies (ICIST 2019), 1-9, 2019.
Schäfer, P. and Högqvist, M., SFA: A symbolic fourier approximation and index for similarity search in high dimensional datasets, ACM International Conference Proceeding Series, 516-527, 2012.
Schäfer, P., The BOSS is concerned with time series classification in the presence of noise, Data Mining and Knowledge Discovery 29, 1505-1530, 2015.
Schäfer, P., Scalable time series classification. Data Mining and Knowledge Discovery 30, 1273-1298, 2016.
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J. and Wojna, Z., Rethinking the inception architecture for computer vision. Conference on Computer Vision and Pattern Recognition, 2016.
Tehrany, M.S., Pradhan, B. and Jebuv, M.N., A comparative assessment between object and pixel-based classification approaches for land use/land cover mapping using SPOT 5 imagery, Geocarto International 29, 4, 351-369, 2014.
Wolpert, D.H., Stacked generalization, Neural Networks 5, 2, 241-259, 1992.
Yang, Q. and Wu, X.D., 10 challenging problems in data mining research, International Journal of Information Technology & Decision Making (IJITDM) 5, 597-604, 2006.
Ye, L. and Keogh, E., Time series shapelets: A new primitive for data mining, In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 947-956, 2009.
Zemblys, R., Niehorster, D.C., Komogortsev, O. and Holmqvist, K., Using machine learning to detect events in eye-tracking data, Behavior Research Methods 50, 160-181, 2018.