Numerous books and papers have been discussed task analysis and its applications in various fields. Broadly speaking, task analysis is a technique that determines the inputs, tools, and skills or knowledge necessary for successful task performance. Questions about what tasks the users will perform with the system often arise in task analyst’s mind. Unfortunately, a task analysis of users' current activities is insufficient to guess what tasks the users will do following the previous tasks. The purpose of this research is to examine and further analyze the primitive results of Bayesian based task analysis model obtained by Lin and Lehto (2009) by comparing the prediction accuracy among all the word combinations from fuzzy Bayes model. This Bayesian based semi-automated task analysis tool was developed to help task analysts predict categories of tasks/subtasks performed by knowledge agents from telephone conversations where agents were trying to help customers to troubleshoot their problems.
A total of twelve prediction results recorded in twelve prediction tables were generated by the Bayesian based tool. A Textminer program was used to generate master word list containing more than 165,000 instances of words used in 5184 dialogs between agent-customer. After excluding repetitive words, a total of around 5400 single words were collected in the master word list. Once data parsing completed, the Textminer learning tool then used the revised master word list (single-word) to identify word combinations appearing in the narratives that could be candidate for subtask category predictors. The single-word frequency list was then used to create list of pair-word combination. As for single word, the pair-word had to appear together at least four times in the dataset to be included in the pair-word frequency list. By repeating the previous steps, list of triple-word combination was obtained. In terms of quadruple-word combination, we used stricter criteria with the frequency requirement set to be at least five.
This study investigates two factors, subtask category and word predictor, based on three responses: hit rate, false alarm rate, and sensitivity value. The ANOVA test results reveal that both factors have significant effect to the three responses. Our findings imply that predictors contained single word always have higher hit rate than others. In addition, triple-quadruple (TQ), triple word (TW), and quadruple word (QW) have low hit rates, low false alarm rates, and low sensitivity values. Moreover, predictors which have high hit rate tend to have high false alarm rate as well. Therefore, predictors with high hit rate do not always have high sensitivity value.
Since this study is interested in finding the most accurate predictor, our analysis focus primarily on word predictor than on subtask category. Although the ANOVA test results cannot specify the most accurate predictor, they provide information on group of predictors with the best and the worst performance. These findings support our hypothesis that the word combination affects the accuracy of predicted subtask category. Overall, the results show that the hit rate is significantly higher than the false alarm rate and the sensitivity value is greater than zero. Since the machine learning tool is still under developing, the findings in this study pave a way for further investigating relationships between word predictor and subtask category once the tool development is complete.

Numerous books and papers have been discussed task analysis and its applications in various fields. Broadly speaking, task analysis is a technique that determines the inputs, tools, and skills or knowledge necessary for successful task performance. Questions about what tasks the users will perform with the system often arise in task analyst’s mind. Unfortunately, a task analysis of users' current activities is insufficient to guess what tasks the users will do following the previous tasks. The purpose of this research is to examine and further analyze the primitive results of Bayesian based task analysis model obtained by Lin and Lehto (2009) by comparing the prediction accuracy among all the word combinations from fuzzy Bayes model. This Bayesian based semi-automated task analysis tool was developed to help task analysts predict categories of tasks/subtasks performed by knowledge agents from telephone conversations where agents were trying to help customers to troubleshoot their problems.
A total of twelve prediction results recorded in twelve prediction tables were generated by the Bayesian based tool. A Textminer program was used to generate master word list containing more than 165,000 instances of words used in 5184 dialogs between agent-customer. After excluding repetitive words, a total of around 5400 single words were collected in the master word list. Once data parsing completed, the Textminer learning tool then used the revised master word list (single-word) to identify word combinations appearing in the narratives that could be candidate for subtask category predictors. The single-word frequency list was then used to create list of pair-word combination. As for single word, the pair-word had to appear together at least four times in the dataset to be included in the pair-word frequency list. By repeating the previous steps, list of triple-word combination was obtained. In terms of quadruple-word combination, we used stricter criteria with the frequency requirement set to be at least five.
This study investigates two factors, subtask category and word predictor, based on three responses: hit rate, false alarm rate, and sensitivity value. The ANOVA test results reveal that both factors have significant effect to the three responses. Our findings imply that predictors contained single word always have higher hit rate than others. In addition, triple-quadruple (TQ), triple word (TW), and quadruple word (QW) have low hit rates, low false alarm rates, and low sensitivity values. Moreover, predictors which have high hit rate tend to have high false alarm rate as well. Therefore, predictors with high hit rate do not always have high sensitivity value.
Since this study is interested in finding the most accurate predictor, our analysis focus primarily on word predictor than on subtask category. Although the ANOVA test results cannot specify the most accurate predictor, they provide information on group of predictors with the best and the worst performance. These findings support our hypothesis that the word combination affects the accuracy of predicted subtask category. Overall, the results show that the hit rate is significantly higher than the false alarm rate and the sensitivity value is greater than zero. Since the machine learning tool is still under developing, the findings in this study pave a way for further investigating relationships between word predictor and subtask category once the tool development is complete.

Abdi, Herve. (2009). Signal Detection Theory. In McGaw, B., Peterson, P.L., Baker, E. (Eds.): Encyclopedia of Education (3rd Ed). New York: Elsevier.
Aghaie, A. and Saeedi, A. (2009). Using Bayesian Networks for Bankruptcy Prediction: Empirical Evidence from Iranian Companies. International Conference on Information Management and Engineering 2009, 450-455.
Annet, J. (2005). Hierarchical Task Analysis (HTA). In Stanton et al., (2005): Chapter 33 (pp. 33-1 – 33-7).
Annett, J. (1996). Recent Developments in Hierarchical Task Analysis. Contemporary Ergonomics: Selected Papers and an Overview of the Ergonomics Society Annual Conference 1984-2008, 373-378. New York: Taylor and Francis.
Annett, J., Duncan, K.D., Stammers, R.B. and Gray, M.J. (1971). Task Analysis. London: HMSO.
Annett, J. and Stanton, N.A. (2000). Task Analysis. London: Taylor and Francis.
Annett, J. and Stanton, N.A. (2000). Research and Developments in Task Analysis. In Annett and Stanton. (2000): Chapter 1 (pp. 1-8).
Anonymous. (2009). Hit Rate and False Alarm Rate. Retrieved June 1, 2011, from http://www.ecmwf.int/products/forecasts/guide/Hit_rate_and_False_alarm_rate.html
Ashcraft, M. H.,(2005). Cognition 3rd Edition. New Jersey: Prentice Hall.
Basu, S., Gupta, S., Mahajam, M., Nguyen, P., Platt, J. C., (2008). Scalable summaries of spoken conversation. Proceeding International Conference on Intelligent User Interface. 267-275. New York: ACM Press.
Cave, E. K., Balakrishna, M., and Moldovan, D., (2006). Efficient Grammar Generation and Tuning for Interactive Voice Response Applications. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. Vol. 1, I-1109 - I-1112.
Chatterjee, S. (1998). A Connectionist Approach for Classifying Accident Narratives. Unpublished Ph.D. Dissertation. Purdue University, West Lafayette, IN.Retrieved October 19, 2011, from http://search.proquest.com/docview/304450940
Chipman, S.F., Schraagen, J.M., and Shalin, V.L. (2000). Cognitive task analysis. Mahwah, NJ: Lawrence Erlbaum. Retrieved September 27, 2011, from http://www.questia.com/PM.qst?a=o&d=78569828
Chipman, S.F., Schraagen, J.M., and Shalin, V.L.(2000). Introduction to Cognitive Task Analysis. In Chipman, et al. (2000), Part I (page 3-24).
Crystal, A. and Ellington, B. (2004). Task Analysis and Human-Computer Interaction: Approaches, Techniques, and Levels of Analysis. Proceedings of the Tenth Americas Conference on Information Systems, New York.
Dubois, D. and Shalin, V. (2000). Describing Job Expertise using Cognitively Oriented Task Analyses (COTA). In Chipman, et al. (2000), Part II (page 41-56).
Evans, G.W., Wilhelm, M.R., and Karwowski, W. (1987). A layout design heuristic employing the theory of fuzzy sets. International Journal of Production Research, 25,1431-1450.
Green, D.M. and Swets, J.A. (1966). Signal Detection Theory and Psychophysics. NewYork: John Wiley & Sons.
Hancock, P.A., Parasuraman, R., 1992. Human factors and safety in the design of intelligent vehicle-highway systems (IVHS). Journal of Safety Research 23, 181-198.
Hatakeyama, N., Furuta, K., and Nakata, K. (2003). Model of Intention Inference Using Bayesian Network. In Stephanidis, C. and Jacko, J.A. (Eds.). Human-Centred Computing (v2). Human-computer interaction : proceedings of HCI International 2003. (pp. 390-394). Mahwah, N.J.: Lawrence Erlbaum.
He, Z.Y., Li, W., and Wu, J. (2008). Task Analysis Methods for Data Selection in Task Adaptation on Mandarin Isolated Word Recognition. Proceeding of 9th International Conference on Signal Processing (ICSP 2008),697-700.
Helmi, N., and Helmi, B. H., (2008). Speech Recognition with Fuzzy Neural Network for Discrete Words. Fourth International Conference on Natural Computation. Vol. 7, pp. 265 – 269.
Hoffman, R.R and Militello, L.G. (2008). Perspective on Cognitive Task Analysis: Historical Origins and Modern Communities of Practice. New York: Taylor and Francis.
Kirsch, H. and Kroschel, K. (1994). Applying Bayesian networks to fault diagnosis. Proceedings of the Third IEEE Conference on Control Applications, 2, 895-900.
Kirwan, B. and Ainsworth, L. K. (Eds.) (1992). A guide to Task Analysis. London: Taylor and Francis.
Legget, J. and Williams, G., (1984). An Empirical Investigation of voice as an Input Modality for Computer Programming. International Journal of Man-Machine Studies 21, 493-520.
Lehto, M.R., Maruci-Wellman, H., and Corns, H., (2009). Bayesian Methods: A Useful Tool for Classifying InjuryNarratives into Cause Groups. Injury Prevention Module 2009;000:0–7. doi:10.1136/ip.2008.021337.
Lehto, M.R., and Buck, J.R. (2008) Introduction to Human Factors and Ergonomics for Engineer. New York: Lawrence Erlbaum Associates.
Lehto, M.R., Papastavrou, J.D., Ranney, T.A. and Simmons, L.A. (2000). An experimental comparison of conservative versus optimal collision avoidance warning system thresholds. Safety Science 36 (2000), 185-209.
Lehto, M.R., and Sorock, G.S. (1996). Machine learning of motor vehicle accident categories from narrative data. Meth. Inform. Med., 35 (4/5), 309-316.
Leman, S. and Lehto, M.R. (2003). Interactive decision support system to predict print quality. Ergonomics, 46(1-3), 52-67.
Liang, L., Liu, X., Zhao, Y., Pi, X., and NeJian, A., (2002). Speaker independent audio-visual continuous speech recognition. Proceedings of IEEE International Conference on Multimedia and Expo. Vol. 2, 25 – 28.
Lin, S. and Lehto, M.R. (2009). A Bayesian Based Machine Learning Application to Task Analysis, Encyclopedia of Data Warehousing and Mining, Classification B, 1-7, Wang, John (Ed., 2nd Edition).
Lin, S. And Lehto, M.R. (2007). A Bayesian Methodology for Semi-automated Task Analysis.M.J. Smith, G. Salvendy (Eds.): Human Interface, Part I, HCII 2007, LNCS 4557, pp. 697 – 704, 2007.Springer-Verlag Berlin Heidelberg.
Michaelis, P. R., and Wiggins, R. H., (1982). A human factors engineer’s introduction to speech synthesizers. In Bandre, A., and Shneiderman, B. (Editors). Directions in Human-Computer Interaction. 149-178. Ablex, Norwood,New Jersey.
Militello, L. and Hutton, R. (2000). Applied Cognitive Task Analysis (ACTA): A Practitioner’s Toolkit for Understanding Cognitive Task Demands. In Annett and Stanton (2000): page 90-113.
Milton, N.R. (2007). Knowledge Acquisition in Practice: A Step-by-step Guide. London: Springer.
Montgomery, D.C., (2009). Design and Analysis of Experiment 7th Edition. New Jersey: John Wiley and Sons, Inc.
Morgan, N., and Bourlard, H., (1995). Continuous speech recognition: an introduction to the hybrid HMM/connectionist approach.IEEE Signal Processing Magazine. 1053-5888/1995 pp 25-42.
Neal, R.M. (1998). Philosophy of Bayesian Inference. Retrieved May 25, 2011, from http://www.cs.toronto.edu/~radford/res-bayes-ex.html
Nezafati, N., Khadivar, A., Afarideh, E. and Jalali, S.M.J. (2007). A Method for Human Driven Knowledge Acquisition (Case Study in a Petrochemical Company). Proceedings of the 2007 IEEE IEEM, 510-514.
Nielsen, Jakob. (1994). Goal Composition: Extending Task Analysis to Predict Things People May Want to Do. Retrieved May 25, 2011, from http://www.useit.com/ papers/goalcomposition.html
Olsson, J. S., and Oard, D. W., (2007). Improving text classification for oral history archive with temporal domain knowledge. Proceeding ACM SIGIR Conference: Research and Development in Information Retrieval. 623-630. New York: ACM Press.
Owens, D.A., Helmers, G., Sivak, M., 1993. Intelligent Vehicle Highway Systems: a call for user-centreddesign. Ergonomics 36, 363-369.
Papastavrou, J.D., Athans, M., 1995. The Team ROC curve in a binary hypothesis testing environment.IEEE Transactions in Aerospace and Electronic Systems 31 (1), 96-105.
Pausch, R., and Leatherby, J. H., (1991). An Empirical Study: Adding voice input to a graphical editor. Journal of the American Voice Input/Output Society. 9, 2, 55-66.
Pitt, I., and Edwards, A., (2003). Design of speech-based devices: A practical guide. London: Springer-Verlag.
Price, K., and Sears, A., (2005). Speech-based Text Entry for Mobile Handheld Devices: An Analysis of Efficacy and Error Correction Techniques for Server-Based Solution. International Journal of Human-Computer Interaction. 19, 3, 279-304.
Quchani, S. R.,and Rahbar, K.,(2006). Local Orthogonal Discriminate Bases to Hybrid SVM/Self-adaptive HMM Classifierfor Discrete Word Speech Recognition. IEEE International Symposium on Signal Processing and Information Technology. 370 – 373.
Rahbar, K., and Broumandnia, A.,(2010). Independent-speaker isolated word speech recognition based on mean-shift framing using hybrid HMM/SVM classifier. 18th Iranian Conference on Electrical Engineering (ICEE).156 – 161.
Salton, G. and McGill, M.J. (1983). Introduction to Modern Information Retrieval. New York: McGraw-Hill.
Sarker, S.K., Hutchinson, R., Chang, A., Vincent, C., and Darzi, A.W. (2006). Self-Appraisal Hierarchical Task Analysis of Laparoscopic Surgery Performed by Expert Surgeons. Springer Science+Business Media, Inc. Surg Endosc (2006) 20: 636–640.
Sears, A., Lin, M., and Karimullah, A. S., (2002). Speech-based cursor control: understanding the effects of target size, cursor speed, and comman selection. Universal Access in the Information Society.2, 30-43.
Shepherd, A. (2000). HTA as a framework for task analysis. In Annett and Stanton(2000): Chapter 2 (pp. 9-24).
Shepherd, A. (2001). Hierarchical Task Analysis. New York: Taylor & Francis.
Shi, H.B., Huang, H.K., and Wang, Z.H. (2003). The Stability of A Restricted Bayesian Network: An Empirical Investigation. Proceedings of the Second International Conference on Machine Learning and Cybernetics, 2-5 November 2003, 345-349.
Shimakawa, M. (2007). A Study on Particle Fuzzy Reasoning Method. Second International Conference on Innovative Computing, Information and Control, ICICIC '07.
Shneiderman, B., Plaisant, C., Cohen, M., and Jacobs, S., (2009). Designing the User Interface: Strategies for Effective Human Computer Interaction 5th edition. Chapter 8 (pp. 349-359). USA: Pearson Higher Education.
Smith, J.Q., (2010). Bayesian Decision Analysis: Principles and Practices. Chapter 5 (pp. 131-165). New York: Cambridge University Press.
Sorkin, R.D., (1988). Why are people turning off our alarms? Journal of Accoustical Society of America 84(3), 1107-1108.
Stanton, N.A., and Baber, C., (2005). Task Analysis for Error Identification. In Stanton et al., (2005): Chapter 38 (pp. 38-1 – 38-9).
Stanton, N.A., Hedge, A., Brookhuis, K., Salas, E., and Hendrick, H. (2005). Handbook of Human Factors and Ergonomics Methods. USA: CRC Press.
Stanton, N.A. and Young, M.S. (1999). A Guide to Methodology in Ergonomics: Designing for Human Use. London: Taylor and Francis.
Suhm, B., and Peterson, P., (2002). A data-driven methodology for evaluating and optimizing call center IVRs. International Journal of Speech Technology. 5, 1, 23-38.
Swets, J.A. (1996). Signal detection theory and ROC analysis in psychology and diagnostics: collected papers. N.J.: Erlbaum Associates. Retrieved October 19, 2011, from http://www.questia.com/PM.qst?a=o&d=91082370
Swezey, R.W. and Pearlstein, R.B. (2001). Selection, Training, and Development of Personnel. In Salvendy, G. (Eds). Handbook of Industrial Engineering: Technology and Operations Management (3rd Ed). Chapter 35 (pp. 920-947). New York: John Wiley & Sons.
Tan, J.T.C, Duan, F., Zhang, Y., and Arai, T. (2008). Extending Task Analysis in HTA to Model Man-Machine Collaboration in Cell Production. Proceedings of the 2008 IEEE International Conference on Robotics and Biomimetics. Bangkok, Thailand, 542-547.
Tanner JR, W.P., and Swets, J.A., (1954). A Decision-Making Theory of Visual Detection.Psychological Review Vol. 61, No. 6, 1954.
Tapias, D., Garcia, C., and Cazassus, C., (2002). On the characteristics and effects of loudness during utterance production in continuous speech recognition. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing. Vol. 1, 89 – 92.
Teixeira, L., Ferreira, C., and Santos, B.S. (2007). Using Task Analysis to Improve the Requirements Elicitation in Health Information System. Proceedings of the 29th Annual International Conference of the IEEE EMBS Cite InternationaleLyon, France, 3669-3672.
Wada,T., Motoda,H. and Washio, T. (2001). Knowledge Acquisition from Both Human Expert and Data. Proceedings of 5th Pacific-Asia Conference, PAKDD 2001 Hong Kong, 550-561.
Walker, M. A., Fromer, J., Di Fabbrizio, G., Mestel, C., and Hindle, D., (1998). What can I say? Evaluating a spoken language interface to email. Proceeding CHI’98 Conference: Human Factors in Computing Systems. 582-589. New York: ACM.
Wang, F., Pan, D., and Ding, J. (2008). A New Approach Combined Fuzzy Clustering and Bayesian Networks for Modeling Gene Regulatory Networks.International Conference on Bio Medical Engineering and Informatics 2008.
WBI Evaluation Group, (2007a). Task Analysis. Retrieved May 2, 2012, from http://siteresources.worldbank.org/WBI/Resources/213798-1194538727144/4Final-Task_Analysis.pdf
WBI Evaluation Group, (2007b). Cognitive Task Analysis. Retrieved May 2, 2012, from http://siteresources.worldbank.org/WBI/Resources/213798-1194538727144/3Final-Cog_Task_Analysis.pdf
Webb, G.I., Boughton, J.R., and Wang, Z., (2005). Not So Naive Bayes: Aggregating One-Dependence Estimators. Machine Learning, 58, 5–24, 2005. Springer Science + Business Media, Inc.: The Netherlands.
Westrenen, F. (2010). Cognitive Work Analysis and The Design of User Interfaces. Cogn Tech Work (2011) 13:31–42. Springer.
Wickens, C.D., (1992). Engineering Psychology and Human Performance, 2nd Edition. New York: Harper Collins.
Wickens, T.D., (2002). Elementary Signal Detection Theory. New York: Oxford University Press.
Yan, H., Jiang, Y., Zheng,J and Fu, B. (2002). Internet-Based Knowledge Acquisition and Management Method to Build Large-Scale Medical Expert Systems. Proceedings of the 2nd Joint EMBS/BMES Conference Houston, USA, 1885-1886.
Yao, X., Fu, J., and Chen, Z. (2008). Bayesian networks modeling for thermal error of numerical control machine tools.Journal of Zhejiang University SCIENCE A 2008 9(11):1524-1530.
Yusof, R.J.R and Hashim, K. (2009). Improving Usability of HTA Charts. Second Int’l Conferences on Advances in Computer-Human Interactions, 365-368.
Yusoff, N.M. and Yin, W.C. (2010). Multimedia Learning System (MMLS): Valuing the Significance of Cognitive Task Analysis Technique and User Interface Design. International Symposium in Information Technology (ITSim) 2010.
Zhu, W. and Lehto, M.R. (1999). Decision support for indexing and retrieval of information in hypertext system. International Journal of Human Computer Interaction, 11, 349-371.