簡易檢索 / 詳目顯示

研究生: 盧燕玲
Yen-Ling Lu
論文名稱: 用於辨識高相似性大量資料的可適性結構類神經網路之研究
Study on Adaptive Structuring Neural Networks for Recognizing High Similar Large Data Sets
指導教授: 范欽雄
Chin-Shyurng Fahn
口試委員: 陳錫明
Shyi-Ming Chen
黃培華
none
江昭皚
Joe-Air Jiang
王永鐘
Yung-Chung Wang
吳偉賢
Wei-Hsien Wu
學位類別: 博士
Doctor
系所名稱: 電資學院 - 資訊工程系
Department of Computer Science and Information Engineering
論文出版年: 2011
畢業學年度: 99
語文別: 英文
論文頁數: 93
中文關鍵詞: 相似性搜尋自組織映射圖網路學習向量量化網路辨識動態結構類神經網路
外文關鍵詞: Similarity search, LVQ; Recognition; Dynamic Structural Neural Netw
相關次數: 點閱:284下載:3
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 由於許多應用系統的資料成長數量龐大,使得高維度空間之資料捜尋、辨識與分類的方法,在各種不同的應用領域廣受注意。從實際應用的觀點而言,若僅憑藉著目測的方式來分析或識別此類資料不但缺乏效率且相當耗費人力。事實上,許多具結構性的方法已被發表運用於解決高維度資料之分類問題。一般而言,傳統方法的效能瓶頸主要落在兩個地方。其一,如果指標不符合目標資料的特徵,則此類捜尋演算法無法充分展現效能,其中主要的問題來自必須從目標系統取得符合系統龐大資料的特徵指標,這並不容易。其二,因隨著特徵向量的維度之增加,捜尋空間將以指數的方式相對地增加,而使得計算量大增。

    現階段由於許多應用系統的資料量不斷地增加,發展能夠應用於大量資料之高效率與高精準度辨識技術儼然成為相當熱門的研究領域。一般從實際面而言,許多情況下我們所面對的系統其各子群資料之機率分佈型態並不明確,所以傳統的統計方法並不適用。相對地,類神經網路之具有學習歷史資料或案例的能力,就顯得相當適合用來解決這類問題。

    近來類神經網路分類器能展現出較令人滿意或接受之效能,其主要原因為類神經網路分類器可以逼近非線性之決策界線,無法分類的機率因此可大為降低至最小。然而,仍然可於實際應用中發現,類神經網路由於輸入向量的相似度太高,導致效能不佳或精準度不高的情形。因此本研究提出一可適性結構之類神經網路,以應用於大量而具高相似性資料的辨識或分類。

    本文提出一種「動態結構類神經網路」之新型類神經網路分類器,具有自動調整自身網路的結構,使之最佳化並具可適性學習之功能。此「動態結構類神經網路」之最大特色在於可重複使用性,換言之,當面對新知識庫時,可減少如傳統類神經網路因固定性網路結構而需重新設計網路架構並得重新訓練類神經網路來適應之情形。

    本研究應用一相當複雜案例來評估所提出之演算法效能,從實驗結果顯示此方法可應用於實際系統中,具有高相似度大量資料之分類能力,且成功率優於傳統方法。對於給定的複雜系統,高相似之大量資料分類,確實十分適合。


    This study proposes an adaptive structuring artificial neural network for recognizing high similar large data sets. It is usually required to classify large data sets with high similar characteristics in many applications. The high dimensions or enormous size of these data sets with high similarity arouse very challenging problems in recognizing the data sets via efficient processing. In many field applications, data sets are measured and recorded continuously using automatic monitoring equipment. Therefore, a large amount of data can be collected, and manual inspection has become an unsuitable approach to recognizing those data. The search, recognition or classification of similar data in a high dimensional space has generated a great deal of interest lately because of wide applications in various fields. From the point view of the practical application, analyzing and identifying those data is a laborious task when the methods adopted are primarily based on visual inspection. Many structure methods have been proposed to classify high dimensional data. In general, the performance bottleneck of the traditional method lies in the two categories. In the first step, if the index structure does not fit in the characteristics of the interesting data sets and the search algorithm is inefficient, a large portion of the index structure must be fetched from the objective system. In the second step, the size of the search space grows exponentially with the size of the problem.

    The development of highly efficient and accurate recognition techniques applied to high dimensional data access has become an active research area. Generally, there are not to know the statistical distribution of each class of disturbance signals for many practical applications. Therefore, statistical approaches are not appropriate. Instead, the neural network approach is more suitable for this purpose since the network learns by generalizing presented examples. It has been recently shown that the performance of neural classifiers is well or acceptable because the neural classifiers are able to approximate the decision boundary, so that the probability of misclassification is minimized. In some applications, the neural network approach is lack of accuracy or not performed well due to the input vectors with high similarity. Hence, this study proposes an adaptive structuring artificial neural network for recognizing the enormous size of datasets with similarity.

    A brand-new neural network model, called dynamic structural neural network (DSNN), is developed to perform automatic learning and optimal structure reconfiguration of the proposed neural classifier. The most significant feature of DSNN is its reusability such that we can retrain the proposed neural classifier using a newly collected knowledge database without worrying about the limitation caused by the fixed structure in a traditional network model.

    A comprehensive performance evaluation of the proposed algorithm is conducted, and the result shows that the proposed algorithm is capable of recognizing high similar large data sets in a real system at an excellent success rate. Therefore, the proposed classifier is quite suitable for high similar large data sets in a given complex system.

    中文摘要...................I Abstract..................II Content...................IV List of Tables ......... VII List of Figures ........ VIII Chapter 1 Introdution ........ 1 1.1 Background and Motivation ............. 1 1.2 Dissertation Organization ............. 5 Chapter 2 Related Work ......... 6 2.1 Introduction ............... 6 2.2 Wavelet Transform .......... 7 2.3 Neural Network ............. 10 2.3.1 The Biological Model ..... 14 2.3.2 The Mathematical Model of ANN ................... 16 Chapter 3 Dynamic Structural Neural Network ........... 20 3.1 Introduction ............... 20 3.2 Architecture of the DSNN .......... 24 3.3 Models of Neurons ................. 26 3.4 Supervised Training of Output Neurons .............. 27 3.5 Supervised Training of Hidden Neurons .............. 29 3.6 Testing Results .......... 34 Chapter 4 Development of Signal Classifier to Recognize the Type of Power Quality .. 43 4.1 Introduction ............. 43 4.2 Characteristics of Power Quality ..... 45 4.3 Types of Power Quality Disturbance ... 50 4.4 Amplitude Estimation ................. 56 4.5 Wavelet Transform .................... 59 4.6 Dynamic Structural Neural Network .... 62 Chapter 5 Simulation Results ............. 65 5.1 Problem Description .................. 65 5.2 Single-Disturbance Case............... 70 5.3 Dual-Disturbance Case ................ 71 5.4 Multiple-Disturbance Case ............ 72 Chapter 6 Conclusions .................... 74 6.1 Conclusions .......................... 74 6.2 Possible Future Works ................ 76 References..................77 Vita........................85 Publication List ........... 2

    1. C. Li, E. Chang, H. Garcia-Molina, and G.Wiederhold, “Clustering for Approximate Similarity Search in High-Dimensional Spaces,” IEEE Transactions on Knowledge and Data Engineering, vol. 14, no. 4, July/August 2002, pp. 792-808.

    2. E. Chang, J. Wang, C. Li, and G. Wiederhold, “RIME, A Replicated Image Detector for the WWW,” in Proc. Int'l Soc. for Optical Eng. (SPIE) Symp. Voice, Video, and Data Comm., Nov. 1998.

    3. A. Gupta and R. Jain, “Visual Information Retrieval,” Comm. ACM, vol. 40, no. 5, pp. 69-79, 1997.

    4. S. Arya, D. Mount, N. Netanyahu, R. Silverman, and A. Wu, “An Optimal Algorithm for Approximate Nearest Neighbor Searching in Fixed Dimensions,” in Proc. Fifth Symp. Discrete Algorithms (SODA), pp. 573-582, 1994.

    5. P. Ciaccia and M. Patella, “Pac Nearest Neighbor Queries: Approximate and Controlled Search in High-Dimensional and Metric Spaces,” in Proc. Int'l Conf. Data Eng. (ICDE), pp. 244-255, 2000.

    6. K. Clarkson, “An Algorithm for Approximate Closest-Point Queries,” in Proc. 10th Software Consulting Group (SCG), pp. 160-164, 1994.

    7. P. Indyk and R. Motwani, “Approximate Nearest Neighbors: Towards Removing the Curse of Dimensionality,” in Proc. 30th Ann. ACM Symp. Theory of Computers (STOC), pp. 604-613, 1998.

    8. J.M. Kleinberg, “Two Algorithms for Nearest-Neighbor Search in High Dimensions,” in Proc. 29th Ann. ACM Symp. Theory of Computers (STOC), 1997.

    9. E. Kushilevitz, R. Ostrovsky, and Y. Rabani, “Efficient Search for Approximate Nearest Neighbor in High Dimensional Spaces,” in Proc. 30th Symp. Theory of Computers (STOC), pp. 614-623, 1998.

    10. E. Chang, K.-T. Cheng, and L. Chang, “PBIR-Perception-Based Image Retrieval,” in Proc. ACM Sigmond, May, 2001.

    11. M. Flickner, H. Sawhney, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele, and P. Yanker, “Query by Image and Video Content: The QBIC System,” Computer, vol. 28, no. 9, pp. 23-32, 1995.

    12. J.R. Smith and S.-F. Chang, “Visualseek: A Fully Automated Content-Based Image Query System,” in Proc. ACM Multimedia Conf., 1996.

    13. S. Brin and H. Garcia-Molina, “Copy Detection Mechanisms for a Digital Document,” in Proc. ACM SIGMOD, May 1995.

    14. H. Garcia-Molina, S. Ketchpel, and N. Shivakumar, “Safeguarding and Charging for Information on the Internet,” in Proc. Int'l Conf. Data Eng. (ICDE), 1998.

    15. N. Beckmann, H.-P. Kriegel, R. Schneider, and B. Seeger, “The R*-Tree: An Efficient and Robust Access Method for Points and Rectangles,” in Proc. ACM SIGMOD, May 1990.

    16. A. Guttman, “R-Trees: A Dynamic Index Structure for Spatial Searching,” in Proc. ACM SIGMOD, June 1984.

    17. H. Garcia-Molina, J. Ullman, and J. Widom, Database System Implementation. Prentice Hall, 1999.

    18. N. Roussopoulos, S. Kelley, and F. Vincent, “Nearest Neighbor Queries,” ACM Sigmod, pp. 71-79, 1995.

    19. G.R. Hjaltason and H. Samet, “Ranking in Spatial Databases,” in Proc. Fourth Symp. Large Spatial Databases (SSD), pp. 83-95, Aug. 1995.

    20. S. Berchtold, C. Bohm, and H.-P. Kriegel, “The Pyramid-Technique: Towards Breaking the Curse of Dimensionality,” ACM SIGMOD, May 1998.

    21. D.A. White and R. Jain, “Similarity Indexing: Algorithms and Performance,” in Proc. Int'l Soc. for Optical Eng. (SPIE), vol. 2670, 1996.

    22. C. Bishop, Neural Networks for Pattern Recognition. Oxford, 1998.

    23. R. P. Lippmann, “A critical overview of neural network pattern classifiers,” in IEEE Workshop on Neural Network for Signal Processing, 1991, pp. 266–275

    24. S. Santoso, E.J. Powers, W.M. Grady, and A.C. Parsons, “Power quality disturbance waveform recognition using wavelet-based neural classifier—Part 2: Application,” IEEE Trans. on Power Delivery, vol. 15, no. 1, Jan. 2000, pp. 229-235.

    25. Daubechies, “Orthonormal bases of compactly supported wavelets,” Communications on Pure and Applied Mathematics, vol. 41, no. 7, pp. 909-996, October 1988.

    26. S. G. Mallat, “A theory for multiresolution signal decomposition: The wavelet representation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 11, no. 7, pp. 67-693, July 1989.

    27. S. G. Mallat and S. Zhong, “Complete signal representation with multiscale edges,’’ NYU Technical Report No. 483, Dec. 1989

    28. S. Santoso, W. M. Grady, E. J. Powers, J. Lamoree, and S. C. Bhatt, “Characterization of distribution power quality events with Fourier and wavelet transforms,” IEEE Trans. Power Delivery, vol. 15, Jan. 2000, pp. 247–254.

    29. M. D. Ganis, C.L. Wilson, and J.L. Blue, "Neural network-based systems for handprint OCR applications," IEEE Trans. Image Processing, vol. 7, no. 8, pp. 1097-1112, Aug. 1998.

    30. S. Osowski, T.H. Linh, and K. Brudzewski, "Neuro-fuzzy network for flavor recognition and classification," IEEE Trans. Instrumentation and Measurement, vol. 53, no. 3, pp. 638-644, June 2004.

    31. P.S. Antonia, Applications in time-frequency signal processing, Boca Raton, CRC Press, 2003.

    32. P. Antognetti, V. Milutinovic, Neural Networks Concepts, Applications, and Implementation, Volume I, Prentice Hall, 1991.

    33. P. Antognetti, V. Milutinovic, Neural Networks Concepts, Applications, and Implementation, Volume II, Prentice Hall, 1991.

    34. D. Zhang and S. K. Pal, Neural Networks and Systolic Array Design, World Scientific, New Jersey, 2002.

    35. R. J. Pasterkamp, J. J. Peschon, M. K. Spriggs, and A. L. Kolodkin, “Semaphorin 7A promotes axon outgrowth through integrins and MAPKs,” Nature 424, 398-405 (24 July 2003).

    36. F. de Castro, “Chemotropic Molecules: Guides for Axonal Pathfinding and Cell Migration During CNS Development” News in Physiological Sciences, Vol. 18, No. 3, 130-136, June 2003

    37. IEEE Std 1159-1995, IEEE recommended practice for monitoring electric power quality, New York, IEEE, 1993.

    38. T. X. Zhu, S. K. Tso, and K. L. Lo, “Wavelet-based fuzzy reasoning approach to power-quality disturbance recognition,” IEEE trans. on Power Delivery, vol. 19, no. 4, Oct. 2004, pp. 1928-1935.

    39. P. K. Dash, M. V. Chilukuri, “Hybrid S-transform and Kalman filtering approach for detection and measurement of short duration disturbances in power networks,” IEEE trans. on Instrumentation and Measurement, vol. 53, no. 2, April 2004, pp. 588-596.

    40. H. He and J.A. Starzyk, “A self-organizing learning array system for power quality classification based on wavelet transform,” IEEE Trans. Power Delivery, vol. 21, no. 1, Jan. 2006, pp. 286-295.

    41. Y. J. Shin, E.J. Powers, M. Grady, and A. Arapostathis, “Power quality indices for transient disturbances,” IEEE Trans. Power Delivery, vol. 21, no. 1, Jan. 2006, pp. 253-261.

    42. O.N. Gerek and D.G. Ece, “Power-quality event analysis using higher order cumulants and quadratic classifiers,” IEEE Trans. Power Delivery, vol. 21, no. 2, April 2006, pp. 883-889.

    43. O.N. Gerek and D.G. Ece, “An adaptive statistical method for power quality analysis,” IEEE Trans. Instrumentation and Measurement, vol. 54, no. 1, Feb. 2005, pp. 184-191.

    44. M.A. Eldery, E.F. El-Saadany, M.M.A Salama, and A. Vannelli, “A novel power quality monitoring allocation algorithm,” IEEE Trans. Power Delivery, vol. 21, no. 2, April 2006, pp. 768-777.

    45. L.H. Tey, P.L. So, and Y.C. Chu, “Improvement of power quality using adaptive shunt active filter”, IEEE Trans. Power Delivery, vol. 20, no. 2, part 2, April 2005, pp. 1558-1568.

    46. D.J. Won, S.J. Ahn, and S.I Moon, “A modified sag characterization using voltage tolerance curve for power quality diagnosis,” IEEE Trans. Power Delivery, vol. 20, no. 4, Oct. 2005, pp. 2638-2643.

    47. B. Wang, S. Omatu, and T. Abe, “Identification of the defective transmission devices using the wavelet transform,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 6, Jun. 2005, pp.919-928.

    48. X. Zhou, C. Zhou, and I.J. Kemp, “An improved methodology for application of wavelet transform to partial discharge measurement denoising,” IEEE Trans. Dielectrics and Electrical Insulation, vol. 12, no.3, June 2005, pp. 586-594.

    49. M. Meade, S.C. Sivakumar, and W.J. Philips, “Comparative performance of principal component analysis, gobor wavelets and discrete wavelet transforms for face recognition,” Canadians Journal of Electrical and Computer Engineering, vol. 30, no. 2, Spring 2005, pp. 93-102.

    50. S. Bhunia and K. Roy, “A novel wavelet transform-based transient current analysis for fault detection and localization,” IEEE Trans. Very Large Scale Integration Systems, vol. 13, no. 4, April 2005, pp. 503-507

    QR CODE