傳統的防毒軟體技術，是藉由特定的特徵碼來判定是否為惡意程式。但程式一旦通過加殼處理，其內容已經改變所以透過特徵碼(pattern recognition) [9] 的比對也無法辨識。因此目前的惡意程式常會利用加殼軟體進行加殼的動作，使其可以有效躲過防毒軟體的偵測。而加殼其實就是利用特殊的演算法，對檔案進行壓縮但壓縮之後的檔案卻可以獨立執行，且脫殼過程可以全部在記憶體中獨立完成。由於現今電腦的CPU執行速度都很快，所以脫殼的過程，使用者並不會有機會察覺程式正在進行脫殼的動作，因此如何辨識檔案是否為加殼的病毒變得非常重要，在目前一般的通用工具PEiD有其使用上的限制的情況下，如何能更有效更快速的判斷是當前研究領域中的重點，因此本論文提出在沙盒 (sandbox) 的環境中利用動態偵測的方式取出DLL檔案裡面的API作為特徵值並將這些特徵匯入Agglomerative hierarchical clustering分叢器完成加殼種類跟病毒種類的分析，結果顯示使用這些資訊做為分析檔案的特徵十分有效，可以正確的分類檔案所使用的加殼軟體跟病毒特性。

Traditional antivirus technology determine malware by specific signatures, But the program once through parker and its contents have changed, Pattern recognition cannot recognize for it does or not. Therefore, The current malware often use software to packers, so that it can effectively escape detection by antivirus software. The packer is actually using special algorithms to compress the file, but file can execute independently after compression. Because modern computer's CPU execution speed is very quickly, during Unpack process the user will not have the opportunity to understand what going on action program. How to identify whether a file is packed that becomes very important. Current general purpose tools PEiD has restrictions on its use. How to more effectively and quickly determine which file is packed. The current research focus on this field, and this paper presents using the sandbox environment to implement DLL detection files by API as a characteristic value. Agglomerative hierarchical clustering use these features to determine packers types and virus analysis.

[1] Mirza Baig, Pavol Zavarsky, Ron Ruhl, Dale Lindskog ,“The Study of Evasion of Packed PE from Static Detection” World Congress on Internet Security (WorldCIS-2012),pp 99-104,2012.
[2] Liu Yu showed,” Packed Executables Detection System Based on Static Analysis”, National Taiwan University of Science and Technology,2011.
[3] Zhang Huiyu , ” Code Obfuscator Detection System Based on API calls” National Taiwan University of Science and Technology,2012.
[4] Li Lu, Liu Qiuju, Xu Tingrong ,” Research and Implementation of Compression Shell Unpacking Technology for PE File”, 2009 International Forum on Information Technology and Applications ,pp 438-442,2009.
[5] SANS ,” Use offense to inform defense.Find flaws before the bad guys do.”, the SANS Penetration Testing site. Reposting is not permited without express written permission, June 30th 2012.
[6] Sami, A., Rahimi, H., Yadegari, B., & Hashemi, S., “Malware Detection Based on Mining API Calls,” ACM Symposium on Applied Computing, April, pp. 1020-1025, 2010.
[7] www.syngress.com, “Malware Incident Response: Volatile Data Collection and Examination on a Live Windows System”,2011
[8] Igor Santos , Felix Brezo, Xabier Ugarte-Pedrero, Pablo G. Bringas,” Opcode sequences as representation of executables for data-mining-based unknown malware detection” , Information Sciences,2011
[9] Li Sun, Steven Versteeg, Serdar Boztas, and Trevor Yann, “Pattern Recognition Techniques for the Classification of Malware Packers”, ACISP 2010, LNCS 6168, pp. 370–390, 2010.
[10] Wei, Te-En and Chen, Zhi-Wei and Tien, Chin-Wei and Wu, Jain-Shing and Lee, Hahn-Ming and Jeng, Albert B, “RePEF – A System for Restoring Packed Executable File for Malware Analysis,” ICMLC, pp.519-527, 2011.
[11] Wei Yan, Zheng Zhang, Nirwan Ansari, “Revealing Packed Malware”, the IEEE Computer Society, PP 65-69, 2007
[12] Sagar Chaki, Cory Cohen, and Arie Gurfinkel,” Supervised Learning for Provenance-Similarity of Binaries”, KDD’11, August 21–24,PP 15-23, 2011.
[13] Xabier Ugarte-Pedrero, Igor Santos, and Pablo G. Bringas ,” Structural Feature based Anomaly Detection for Packed Executable Identi_cation”, CISIS2011-AnomalyDetectionPacked, 2011.
[14] Zahra Salehi, Mahboobeh Ghiasi, Ashkan Sami ,"A Miner for Malware Detection Based on API Function Calls and Their Arguments ",The 16th CSI International Symposium on Artificial Intelligence and Signal Processing (AISP 2012),pp 563-568,2012
[15] Zhang Xiaosong,Pan Xiaohui and Long Xiaoshu , “Analysis of Virtual Machine applied to Malware Detection System”, 2009 International Symposium on Information Engineering and Electronic Commerce ,pp 290-294,2009
[16] Mohamad Fadli Zolkipli,Penang,"A Framework for Malware Detection Using Combinati n Technique and Signature Generation",Second International Conference on Computer Research and Developme, PP 196-199,2010
[17] Ivan Firdausi, Charles Lim, Alva Erwin,"ANALYSIS OF MACHINE LEARNING TECHNIQUES USED IN BEHAVIOR-BASED MALWARE DETECTION",2010 Second International Conference on Advances in Computing, Control, and Telecommunication Technologies ,pp 201-203,2010
[18] Jonathan J. Blount, Daniel R. Tauritz,"Adaptive Rule-Based Malware Detection Employing Learning Classifier Systems:A Proof of Concept",2011 35th IEEE Annual Computer Software and Applications Conference Workshops,pp110-115,2011 IEEE.
[19] Hui Fang, Yongdong Wu, Shuhong Wang, and Yin Huang,"Multi-stage Binary Code Obfuscation Using Improved Virtual Machine",ISC 2011, LNCS 7001, pp. 168–181, 2011.
[20] Xabier Ugarte-Pedrero, Igor Santos, Pablo G. Bringas, Mikel Gastesi, Jos’e Miguel Esparza,"Semi-supervised Learning for Packed Executable Detection",pp 342-346,2011
[21] TZU-YEN WANG, CHIN-HSIUNG WU,"DETECTION OF PACKED EXECUTABLES USING SUPPORT VECTOR MACHINES",2011 International Conference on Machine Learning and Cybernetics,p717 -722,2011
[22] Philip O’Kane,Sakir Sezer,and Kieran McLaughlin,"Obfuscation: The Hidden Malware",COPUBLISHED BY THE IEEE COMPUTER AND RELIABILITY SOCIETIES,pp 41-47,2011.
[23] Chan Lee Yee, Lee Ling Chuan, Mahamod Ismail, Kasmiran Jumari,”Metaware - An Extensible Malware Detection and Removal Toolkit”,ICACT2011,pp 996-1000,2011
[24] Yuhei Kawakoya,Makoto Iwamura,Mitsutaka Itoh,"Memory Behavior-Based Automatic Malware Unpacking in Stealth Debugging Environment",2010 5th International Conference on Malicious and Unwanted Software,pp39-46 ,2010
[25] Guhyeon Jeong, Euijin Choo, Joosuk Lee, Munkhbayar Bat-Erdene, and Heejo Lee,"Generic Unpacking using Entropy Analysis",2010 5th International Conference on Malicious and Unwanted Software,pp 98-105,2010
[26] Lee Ling Chuan, Chan Lee Yee, Mahamod Ismail, Kasmiran Jumari,"Automating Uncompressing and Static Analysis of Conficker Worm",2009 IEEE 9th Malaysia International Conference on Communications 15 -17 December 2009 Kuala Lumpur Malaysia,PP 193-198,2009
[27] YU SAN-CHAO, LI YI-CHAO, LIU DAN,and YANG TING,"A UNPACKING AND RECONSTRUCTION SYSTEM-AGUNPACKER" Apperceiving Computing and Intelligence Analysis, 2009. ICACIA 2009,PP 440-443,2009
[28] Kevin Coogan,Saumya Debray,Tasneem Kaochar and Gregg Townsend,"Automatic Static Unpacking of Malware Binaries",2009 16th Working Conference on Reverse Engineering,PP 167-176,2009
[29] Lorenzo Martignoni,Mihai Christodorescu and Somesh a,OmniUnpack: Fast, Generic, and Safe Unpacking of Malware,23rd Annual Computer Security Applications Conference,PP 431-440,2007
[30] ROBERT LYDA ,JAMES HAMROCK,"Using Entropy Analysis to Find Encrypted and Packed Malware",IEEE COMPUTER SOCIETY IEEE SECURITY & PRIVACY,PP 40-45,2007
[31] Paul Royal, Mitch Halpin, David Dagon, Robert Edmonds, Wenke Lee,"PolyUnpack: Automating the Hidden-Code Extraction ofUnpack-Executing Malware",the 22nd Annual Computer Security Applications Conference (ACSAC'06),2006
[32] Mohd Fadzli Marhusin, Henry Larkin, Chris Lokan, David Cornforth,"An Evaluation of API Calls Hooking Performance",2008 International Conference on Computational Intelligence and Security,PP 315-319,2008
[33] Wen Fu,Jianmin Pang,Rongcai Zhao,Yichi Zhang,Bo Wei,"Static Detection of API-calling Behavior from Malicious Binary Executables",2008 International Conference on Computer and Electrical Engineering,PP 388-392,2008
[34] Mamoun Alazab,Sitalakshmi Venkataraman,Paul Watters,"Towards Understanding Malware Behaviour by the Extraction of API calls",2010 Second Cybercrime and Trustworthy Computing Workshop,PP 52-59,2010 IEEE
[35] Rafiqul Islam, Ronghua Tian and Lynn Batten,"Classification of Malware Based on String and Function Feature Selection",2010 Second Cybercrime and Trustworthy Computing Workshop,PP 9-17,2010
[36] Yoshiro Fukushimayz, Akihiro Sakaiy, Yoshiaki Horiyz, and Kouichi Sakuraiyz,"A Behavior Based Malware Detection Scheme forAvoiding False Positive",978-1-4244-8915-2/10/$26.00 c2010 IEEE
[37] J. Bergeron, M. Debbabi, J. Desharnais, M. M. Erhioui, Y. Lavoie and N. Tawbi,Static Detection of Malicious Code in Executable Programs,PP 1-9,2001