A new task analysis methodology that combines a statistical approach had been adapted to develop a semi-automated task analysis tool based on machine learning application to help call canter agent during remote troubleshooting process (Lin and Lehto, 2009). Three Bayesian models had been chosen as statistical approach for identifying subtasks, such as the classic Bayes, fuzzy Bayes, and hybrid Bayes. By examining the preliminary results of Bayesian based tool's performance, it had been proven that the Bayesian based tool is able to learn subtask categories from the agent or customer narrative telephone conversations and to predict them as well. However, there is still necessary to prove the Bayesian based tool is better applied than human expert on many occasions. One of the key issue on previous research is the misclassified cases of discrepancies between human decision and Bayesian based tool's prediction when determines the subtask categories based on conversations or narratives. This issue will be appointed in this study to be a starting point for more in depth investigation of the application of task analysis tools to the naturalistic decision making environment.
The discrepancies had pointed out the bad result of Bayesian based tool when predict the subtask categories. Based on the signal detection theory, the error can happened in "false alarm" cases and "miss" cases. By learning the preliminary results of misclassified revealed that the tool not only identifies the hard to classify dialogs, but also actually helps uncover the misclassified pre-assigned subtask categories by human experts. This information denotes the misclassified cases can be caused by human decision or tool's prediction or even both of them.
This study focuses on further analysis of misclassified cases with emphasis to discrepancies of human decision and tool's prediction, especially in "false alarm" condition and "miss" condition. Through statistical proving will be done as approach in this study with exploit the abundant data and results obtained by Lin and Lehto's research to reveals from time to time that the tool is more accurate than human experts on many occasions. Furthermore, the phases of this study are arranged as follows: data collection, data identification, data preparation, data analysis, and the advanced investigation of misclassified cases.
In these results of this study point out there are influence of discrepancies caused by some treatments in the preliminary stage of Bayesian based tool development when the measuring of tool's performance, but not significant. Due to the bad results of the Bayesian based tool is very low, so the Bayesian based tool’s prediction can be proven better than human decision to identify the subtask categories. In more deep investigation, the discrepancies between human decision and tool’s prediction have been found the misclassified by the human is greater than the tool. These findings can confirm the Bayesian based tool had been proven to be able to uncover the misclassified pre-assigned by the human expert.

A new task analysis methodology that combines a statistical approach had been adapted to develop a semi-automated task analysis tool based on machine learning application to help call canter agent during remote troubleshooting process (Lin and Lehto, 2009). Three Bayesian models had been chosen as statistical approach for identifying subtasks, such as the classic Bayes, fuzzy Bayes, and hybrid Bayes. By examining the preliminary results of Bayesian based tool's performance, it had been proven that the Bayesian based tool is able to learn subtask categories from the agent or customer narrative telephone conversations and to predict them as well. However, there is still necessary to prove the Bayesian based tool is better applied than human expert on many occasions. One of the key issue on previous research is the misclassified cases of discrepancies between human decision and Bayesian based tool's prediction when determines the subtask categories based on conversations or narratives. This issue will be appointed in this study to be a starting point for more in depth investigation of the application of task analysis tools to the naturalistic decision making environment.
The discrepancies had pointed out the bad result of Bayesian based tool when predict the subtask categories. Based on the signal detection theory, the error can happened in "false alarm" cases and "miss" cases. By learning the preliminary results of misclassified revealed that the tool not only identifies the hard to classify dialogs, but also actually helps uncover the misclassified pre-assigned subtask categories by human experts. This information denotes the misclassified cases can be caused by human decision or tool's prediction or even both of them.
This study focuses on further analysis of misclassified cases with emphasis to discrepancies of human decision and tool's prediction, especially in "false alarm" condition and "miss" condition. Through statistical proving will be done as approach in this study with exploit the abundant data and results obtained by Lin and Lehto's research to reveals from time to time that the tool is more accurate than human experts on many occasions. Furthermore, the phases of this study are arranged as follows: data collection, data identification, data preparation, data analysis, and the advanced investigation of misclassified cases.
In these results of this study point out there are influence of discrepancies caused by some treatments in the preliminary stage of Bayesian based tool development when the measuring of tool's performance, but not significant. Due to the bad results of the Bayesian based tool is very low, so the Bayesian based tool’s prediction can be proven better than human decision to identify the subtask categories. In more deep investigation, the discrepancies between human decision and tool’s prediction have been found the misclassified by the human is greater than the tool. These findings can confirm the Bayesian based tool had been proven to be able to uncover the misclassified pre-assigned by the human expert.

Abdi, Herve. (2009). Signal Detection Theory. In McGaw, B., Peterson, P.L., Baker, E. (Eds.): Encyclopedia of Education (3rd Ed). New York: Elsevier.
Adams, B. D., Rehak, L., Brown, A. and Hall, C. D. T. (2009). Human decision-making biases. DRDC Toronto (in press).
Alberdi, E., Strigini, L., Povyakalo, A. A. and Ayton, P. (2009) Why are people's decisions sometimes worse with computer support?, in Proc. SAFECOMP 2009, The 28th International Conference on Computer Safety, Reliability and Security, (Bettina Buth, Gerd Rabe, Till Seyfarth, Eds.), pp. 18-31, Springer, Lecture Notes in Computer Science 5775, Hamburg, Germany, 2009.
Annett, J. and Stanton, N.A. (2000). Task Analysis. London: Taylor and Francis.
Annett, J., Duncan, K.D., Stammers, R.B. and Gray, M.J. (1971). Task Analysis. London: HMSO.
Balamurali, S. and Kalyanasundaram, M. (2010). An Investigation of the Misclassification Errors on the Analysis of Means. Tamsui Oxford Journal of Information and Mathematical Sciences 27(2) (2011) 117-136, Aletheia University.
Bassil, Y. (2012). Expert PC Troubleshooter with Fuzzy-Logic And Self-Learning Support. International Journal of Artificial Intelligence & Applications (IJAIA), Vol.3, No.2, March 2012.
Bridger, R.S. (2003). Introduction to Ergonomics (2nd Ed). London: Taylor and Francis.
Briggs, W., Pocernich, M. and Ruppert, D. (2005). Incorporating Misclassification Error in Skill Assessment. American Meteorological Society, November 2005.
Bruland, R.C. and Schnidt, M. (2009). Response Bias in Judging Deceptive Movement. Acta Psychologica 130 (2009) 235–240.
Chipman, S.F., Schraagen, J.M. and Shalin, V.L. (2000). Introduction to Cognitive Task Analysis.In Chipman, et al. (2000), Part I (page 3-24).
Crystal, A. and Ellington, B. (2004). Task Analysis and Human-Computer Interaction: Approaches, Techniques, and Levels of Analysis. Proceedings of the Tenth Americas Conference on Information Systems, New York.
Diaper, D. and Stanton, N.A. (2004). The Handbook of Task Analysis For Human-Computer Interaction. London: Lawrence Erlbaum Associates, Inc.
Embrey, D. (1990), Task Analysis Techniques. Lancashire: Human Reliability Associates.
Fields, R.E., Wright, P.C. and Harrison, M.D. (1995). A Task Centered Approach to Analyzing Human Error Tolerance Requirements. Second IEEE International Symposium on Requirements Engineering. York, 1995.
Gupta, V. and Lehal, G. S. (2009). A Survey of Text Mining Techniques and Applications. Journal of Emerging Technologies in Web Intelligence, Vol. 1, No. 1, August 2009.
Haines, L.A. (1993). Task Analysis as a Basis for the Development of A Training System. Proceedings of the IEEE 1993 National Aerospace and Electronics Conference, 1993. NAECON 1993.
Hayes, C.C. and Anderson, R.A. (2005). Differential Benefits Derived from Decision Support Tools For Domain Novices versus Experts. IEEE International Conference on System, Man and Cybernetics, October 2005.
Helander, M.G., Landauer, T.K. and Prabhu, P.V. (1997). Handbook of Human-Computer Interaction. (2nd Ed). Amsterdam: North-Holland.
Imran, H. and Joshi, M.L. (2008). A Trainable Document Summarizer Using Bayesian Classifier Approach. First International Conference on Emerging Trends in Engineering and Technology, 2008. ICETET '08.
Kim, S.K. and Carrington, D., (2002). Integrating Use-Case Analysis for Interactive Systems. Proceedings of The Ninth Asia-Pacific Software Engineering Conference (APSEC’02).
Lehto, M.R. and Buck, J.R. (2008). Introduction to Human Factors and Ergonomics for Engineer. New York: Lawrence Erlbaum Associates, Inc.
Lehto, M.R., Maruci-Wellman, H. and Corns, H., (2009). Bayesian Methods: A Useful Tool for Classifying Injury Narratives into Cause Groups. Injury Prevention Module 2009;000:0–7. doi:10.1136/ip.2008.021337.
Lin, S. and Lehto, M.R. (2009). A Bayesian Based Machine Learning Application to Task Analysis, Encyclopedia of Data Warehousing and Mining, Classification B, 1-7, Wang, John (Ed., 2nd Edition).
Lin, S. And Lehto, M.R. (2007). A Bayesian Methodology for Semi-automated Task Analysis.M.J. Smith, G. Salvendy (Eds.): Human Interface, Part I, HCII 2007, LNCS 4557, pp. 697 – 704, 2007.Springer-Verlag Berlin Heidelberg.
Lehto, M.R. and Sorock, G.S. (1996). Machine learning of motor vehicle accident categories from narrative data. Meth. Inform. Med., 35 (4/5), 309-316.
Mane, S., Srivastava, J., Hwang, S.Y. and Vayghan, J. (2004). Estimation of False Negatives in Classification. Proceedings of the Fourth IEEE International Conference on Data Mining (ICDM’04).
McNicol, D. (2005). A Primer of Signal Detection Theory. London: Lawrence Erlbaum Associates, Inc.
Militello, L. and Hutton, R. (2000). Applied Cognitive Task Analysis (ACTA): A Practitioner’s Toolkit for Understanding Cognitive Task Demands. In Annett and Stanton (2000): page 90-113.
Shneiderman, B., Plaisant, C., Cohen, M. and Jacobs, S., (2009). Designing the User Interface: Strategies for Effective Human Computer Interaction 5th edition. Chapter 8 (pp. 349-359). USA: Pearson Higher Education.

Watkins, R. (2007). Performance By Design: The selection, design, and development of performance technologies that achieve results. Amherst, MA: HRD Press, I.
Yusoff, N.M. and Yin, W.C. (2010). Multimedia Learning System (MMLS): Valuing the Significance of Cognitive Task Analysis Technique and User Interface Design. International Symposium in Information Technology (ITSim) 2010.
Zhou, L., (2005). Data Mining for Detecting Errors in Detection Speech Recognition. IEEE Transaction on Speech and Audio Processing, Vol.13, No.5, 2005.