現代的自動化製造系統對於生產能力的要求日益增高，其中設備參數的控制乃是影響製程產出的關鍵因素。各式各樣的參數調校技術開始逐漸取代手動維護的任務，以此在無需人力介入的情況下優化產品品質。但是如何從這些技術中選擇一個最有利的建模方案是非常重要的環節，本文就基於上述這些考量提出了一項結合製程能力指標 (process capability ratio, PCR)的模擬優化 (simulation optimization, SO)框架，為各種以機器學習 (machine learning ,ML)指導的決策代理(decision agent, DA)提供一個標準的訓練流程。在這項工作中，PCR被用於衡量模型在調校參數後所優化的品質表現，以在客觀的量化基準上考慮並選擇具有不同優越性的建模機制。為了展示這個開創性的方法，我們研究了一個包覆電纜絕緣外層的塑膠射出成形 (plastic injection molding, PIM)案例，其中涵蓋串聯製程 (serial process, SP)與並聯製程 (parallel process, PP)兩種類型的資料來源。我們將會嘗試以合成式 (synthetic)的變分自動編碼器 (Variational AutoEncoder, VAE)與生成式 (generative)的確定策略梯度 (Deterministic Policy Gradient, DPG)來對這兩種具有代表性的製程挖掘更好的參數配置。實驗的結果表明，生成式的訓練原則在探索最佳的調校策略上更具有優勢，分別於SP與PP取得優於原始生產表現24%及36%的品質水準。最後，我們以統計檢定探討了這兩種不同模型的適用情境，為後續的研發決策提供強而有力的科學證據。

Modern automated manufacturing systems are increasingly demanding in terms of production capacity, where the control of equipment parameters is a crutial problem affecting process output. A variety of parameter tuning techniques are gradually replacing the manual maintenance tasks to optimize product quality without human intervention. But how to choose a most favorable modeling solution from these techniques is a very important aspect. Based on these considerations, this paper proposed a simulation optimization (SO) framework combining process capability ratio (PCR) to provide a standard training process for various machine learning (ML) guided parameter tuning models. In this work, PCR is used to determine the quality performance of models optimized after tuning parameters to consider and select a modeling mechanism with different superiority on an objective quantitative basis. In order to demonstrate this pioneering approach, a plastic injection molding (PIM) case of covering a cable insulating outer layer was studied, including both serial process (SP) and parallel process (PP) types of data sources. We will try to use both synthetic (VAE) and generative (DPG) Variational AutoEncoder (VAE) and Deterministic Policy Gradient (DPG) to explore better parameter configurations for these two representative processes. The experimental results showed that the generative training principle is more advantageous in exploring the best tuning policies, achieving quality levels 24% and 36% better than the original production performance for SP and PP, respectively. Finally, we also conclude the applicability of these two different models with statistical test, in order to provide strong scientific evidence for subsequent R&D decisions.

[1] S. Ebnesajjad, "7 - Injection Molding," in Melt Processible Fluoroplastics, S. Ebnesajjad, Ed. Norwich, NY: William Andrew Publishing, 2003, pp. 151-193.
[2] J. D. McDonald, "Developing and defining basic SCADA system concepts," in [Proceedings] 1993 Rural Electric Power Conference. Papers Presented at the 37th Annual Conference, 1993, pp. B3/1-B3/5.
[3] B. Lu and X. Zhou, "Quality and reliability oriented maintenance for multistage manufacturing systems subject to condition monitoring," Journal of Manufacturing Systems, vol. 52, pp. 76-85, 2019/07/01/ 2019.
[4] S. L. Lee et al., "Modernizing Pharmaceutical Manufacturing: from Batch to Continuous Production," Journal of Pharmaceutical Innovation, vol. 10, no. 3, pp. 191-199, 2015/09/01 2015.
[5] A. Bandyopadhyay and B. Heer, "Additive manufacturing of multi-material structures," Materials Science and Engineering: R: Reports, vol. 129, pp. 1-16, 2018/07/01/ 2018.
[6] J. Queiroz, P. Leitão, J. Barbosa, E. Oliveira, and G. Garcia, "An agent-based industrial cyber-physical system deployed in an automobile multi-stage production system," in International Workshop on Service Orientation in Holonic and Multi-Agent Manufacturing, 2019, pp. 379-391: Springer.
[7] X.-P. Dang, "General frameworks for optimization of plastic injection molding process parameters," Simulation Modelling Practice and Theory, vol. 41, pp. 15-27, 2014/02/01/ 2014.
[8] F. Guo, X. Zhou, J. Liu, Y. Zhang, D. Li, and H. Zhou, "A reinforcement learning decision model for online process parameters optimization from offline data in injection molding," Applied Soft Computing, vol. 85, p. 105828, 2019/12/01/ 2019.
[9] S. Amaran, N. V. Sahinidis, B. Sharda, and S. J. Bury, "Simulation optimization: a review of algorithms and applications," Annals of Operations Research, vol. 240, no. 1, pp. 351-380, 2016/05/01 2016.
[10] W. C. Van Beers and J. P. Kleijnen, "Kriging interpolation in simulation: a survey," in Proceedings of the 2004 Winter Simulation Conference, 2004., 2004, vol. 1: IEEE.
[11] S. Suthaharan, "Support vector machine," in Machine learning models and algorithms for big data classification: Springer, 2016, pp. 207-235.
[12] Z. Majdisova and V. Skala, "Radial basis function approximations: comparison and applications," vol. 51, pp. 728-743, 2017.
[13] A. I. Khuri and S. Mukhopadhyay, "Response surface methodology," vol. 2, no. 2, pp. 128-149, 2010.
[14] X. Li, "Using" random forest" for classification and regression," vol. 50, no. 4, pp. 1190-1197, 2013.
[15] Y. Gao, L. S. Turng, and X. Wang, "Adaptive geometry and process optimization for injection molding using the Kriging surrogate model trained by numerical simulation," vol. 27, no. 1, pp. 1-16, 2008.
[16] D. Kumar, R. K. Prasad, and S. Mathur, "Optimal design of an in-situ bioremediation system using support vector machine and particle swarm optimization," vol. 151, pp. 105-116, 2013.
[17] X. Li, F. Yan, J. Ma, Z. Chen, X. Wen, and Y. Cao, "RBF and NSGA-II based EDM process parameters optimization with multiple constraints," vol. 16, no. 5, pp. 5788-5803, 2019.
[18] M. Mosayeb Motlagh, P. Azimi, M. Amiri, and G. Madraki, "An efficient simulation optimization methodology to solve a multi-objective problem in unreliable unbalanced production lines," Expert Systems with Applications, vol. 138, p. 112836, 2019/12/30/ 2019.
[19] L. Sun, Y. Ji, X. Zhu, and T. Peng, "Process knowledge-based random forest regression for model predictive control on a nonlinear production process with multiple working conditions," Advanced Engineering Informatics, vol. 52, p. 101561, 2022/04/01/ 2022.
[20] T. L. Chen and C. C. Wang, "Multi-objective simulation optimization for medical capacity allocation in emergency department," Journal of Simulation, vol. 10, no. 1, pp. 50-68, 2016/02/01 2016.
[21] Y.-Y. Feng, I. C. Wu, and T.-L. Chen, "Stochastic resource allocation in emergency departments with a multi-objective simulation optimization algorithm," Health Care Management Science, vol. 20, no. 1, pp. 55-75, 2017/03/01 2017.
[22] A. Bahari and F. Asadi, "A Simulation Optimization Approach for Resource Allocation in an Emergency Department Healthcare Unit," (in eng), Global heart, vol. 15, no. 1, pp. 14-14, 2020.
[23] N. Jian, D. Freund, H. M. Wiberg, and S. G. Henderson, "Simulation optimization for a large-scale bike-sharing system," in 2016 Winter Simulation Conference (WSC), 2016, pp. 602-613.
[24] J. Högdahl, M. Bohlin, and O. Fröidh, "A combined simulation-optimization approach for minimizing travel time and delays in railway timetables," Transportation Research Part B: Methodological, vol. 126, pp. 192-212, 2019/08/01/ 2019.
[25] A. Saif and S. Elhedhli, "Cold supply chain design with environmental considerations: A simulation-optimization approach," European Journal of Operational Research, vol. 251, no. 1, pp. 274-287, 2016/05/16/ 2016.
[26] I. Castilla-Rodríguez, C. Expósito-Izquierdo, B. Melián-Batista, R. M. Aguilar, and J. M. Moreno-Vega, "Simulation-optimization for the management of the transshipment operations at maritime container terminals," Expert Systems with Applications, vol. 139, p. 112852, 2020/01/01/ 2020.
[27] M. F. Yegul, F. S. Erenay, S. Striepe, and M. Yavuz, "Improving configuration of complex production lines via simulation-based optimization," Computers & Industrial Engineering, vol. 109, pp. 295-312, 2017/07/01/ 2017.
[28] L. L. Lim, G. Alpan, and B. Penz, "A simulation-optimization approach for sales and operations planning in build-to-order industries with distant sourcing: Focus on the automotive industry," Computers & Industrial Engineering, vol. 112, pp. 469-482, 2017/10/01/ 2017.
[29] H. Jalota, M. Thakur, and G. Mittal, "A credibilistic decision support system for portfolio optimization," Applied Soft Computing, vol. 59, pp. 512-528, 2017/10/01/ 2017.
[30] N. Metawa, M. Elhoseny, M. K. Hassan, and A. E. Hassanien, "Loan portfolio optimization using Genetic Algorithm: A case of credit constraints," in 2016 12th International Computer Engineering Conference (ICENCO), 2016, pp. 59-64.
[31] S. Chatterjee, S. S. Mahapatra, and K. Abhishek, "Simulation and optimization of machining parameters in drilling of titanium alloys," Simulation Modelling Practice and Theory, vol. 62, pp. 31-48, 2016/03/01/ 2016.
[32] N. Delgarm, B. Sajadi, F. Kowsary, and S. Delgarm, "Multi-objective optimization of the building energy performance: A simulation-based approach by means of particle swarm optimization (PSO)," Applied Energy, vol. 170, pp. 293-303, 2016/05/15/ 2016.
[33] I. Baturynska, O. Semeniuta, and K. Martinsen, "Optimization of Process Parameters for Powder Bed Fusion Additive Manufacturing by Combination of Machine Learning and Finite Element Method: A Conceptual Framework," Procedia CIRP, vol. 67, pp. 227-232, 2018/01/01/ 2018.
[34] E. Lughofer, A.-C. Zavoianu, M. Pratama, and T. Radauer, "Automated Process Optimization in Manufacturing Systems Based on Static and Dynamic Prediction Models," in Predictive Maintenance in Dynamic Systems: Advanced Methods, Decision Support Tools and Real-World Applications, E. Lughofer and M. Sayed-Mouchaweh, Eds. Cham: Springer International Publishing, 2019, pp. 485-531.
[35] A. Slowik and H. Kwasnicka, "Evolutionary algorithms and their applications to engineering problems," Neural Computing and Applications, vol. 32, no. 16, pp. 12363-12379, 2020/08/01 2020.
[36] S.-M. Chen, "Analyzing fuzzy system reliability using vague set theory," vol. 1, no. 1, pp. 82-88, 2003.
[37] C. F. Tan, L. Wahidin, S. Khalil, N. Tamaldin, J. Hu, and G. Rauterberg, "The application of expert system: A review of research and applications," vol. 11, no. 4, pp. 2448-2453, 2016.
[38] S. Yang, J. Wang, Y. Ma, and Y. Tu, "Multi-response online parameter design based on Bayesian vector autoregression model," Computers & Industrial Engineering, vol. 149, p. 106775, 2020/11/01/ 2020.
[39] S. N. Sahu and N. C. Nayak, "Multi-objective optimisation of EDM process using ANN integrated with NSGA-II algorithm," vol. 32, no. 4-5, pp. 381-395, 2018.
[40] S. Sharma and N. Agrawal, "Application of fuzzy techniques in a multistage manufacturing system," The International Journal of Advanced Manufacturing Technology, vol. 60, no. 1, pp. 397-407, 2012/04/01 2012.
[41] R. U. Islam, M. S. Hossain, and K. Andersson, "A Deep Learning Inspired Belief Rule-Based Expert System," IEEE Access, vol. 8, pp. 190637-190651, 2020.
[42] A. A. Hajnoroozi, F. Aminifar, and H. Ayoubzadeh, "Generating Unit Model Validation and Calibration Through Synchrophasor Measurements," IEEE Transactions on Smart Grid, vol. 6, no. 1, pp. 441-449, 2015.
[43] C. C. Tsai et al., "Practical Considerations to Calibrate Generator Model Parameters Using Phasor Measurements," IEEE Transactions on Smart Grid, vol. 8, no. 5, pp. 2228-2238, 2017.
[44] Y. Wehbe and L. Fan, "PMU-based system identification for a modified classic generator model," in 2015 North American Power Symposium (NAPS), 2015, pp. 1-6.
[45] S. R. Khazeiynasab and J. Qi, "PMU Measurement Based Generator Parameter Calibration by Black-Box Optimization with A Stochastic Radial Basis Function Surrogate Model," in 2020 52nd North American Power Symposium (NAPS), 2021, pp. 1-6.
[46] J. Zhang, A. Xue, T. Bi, Z. Wang, and W. Tang, "On-Line Synchronous Generator's Parameters Identification with Dynamic PMU Data," in 2012 Asia-Pacific Power and Energy Engineering Conference, 2012, pp. 1-4.
[47] R. Huang et al., "Calibrating Parameters of Power System Stability Models Using Advanced Ensemble Kalman Filter," IEEE Transactions on Power Systems, vol. 33, no. 3, pp. 2895-2905, 2018.
[48] T. Zhou et al., "Generator Parameter Identification Using Time Series Model and PMU Measurements," in 2021 IEEE 12th International Symposium on Power Electronics for Distributed Generation Systems (PEDG), 2021, pp. 1-6.
[49] R. Huang, R. Fan, T. Yin, S. Wang, and Z. Tan, "Parameters calibration for power grid stability models using deep learning methods," 2019.
[50] X. Zheng, B. Wang, and L. Xie, "Synthetic Dynamic PMU Data Generation: A Generative Adversarial Network Approach," in 2019 International Conference on Smart Grid Synchronized Measurements and Analytics (SGSMA), 2019, pp. 1-6.
[51] S. Wshah et al., "Deep Learning for Model Parameter Calibration in Power Systems," in 2020 IEEE International Conference on Power Systems Technology (POWERCON), 2020, pp. 1-6.
[52] L. Pagnier and M. Chertkov, "Physics-informed graphical neural network for parameter & state estimations in power systems," 2021.
[53] S. M. H. Rizvi, "Time Series Deep learning for Robust Steady-State Load Parameter Estimation using 1D-CNN," Arabian Journal for Science and Engineering, vol. 47, no. 3, pp. 2731-2744, 2022/03/01 2022.
[54] W. J. Murdoch, C. Singh, K. Kumbier, R. Abbasi-Asl, and B. J. a. p. a. Yu, "Interpretable machine learning: definitions, methods, and applications," 2019.
[55] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
[56] X. Qin, Z. Zhang, C. Huang, M. Dehghan, O. R. Zaiane, and M. Jagersand, "U2-Net: Going deeper with nested U-structure for salient object detection," vol. 106, p. 107404, 2020.
[57] A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, "Yolov4: Optimal speed and accuracy of object detection," 2020.
[58] Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie, "A ConvNet for the 2020s," 2022.
[59] W. Fang, Y. Chen, and Q. Xue, "Survey on Research of RNN-Based Spatio-Temporal Sequence Prediction Algorithms," vol. 3, no. 3, p. 97, 2021.
[60] A. Vaswani et al., "Attention is all you need," vol. 30, 2017.
[61] T. Baumeister, S. L. Brunton, and J. Nathan Kutz, "Deep learning and model predictive control for self-tuning mode-locked lasers," Journal of the Optical Society of America B, vol. 35, no. 3, pp. 617-626, 2018/03/01 2018.
[62] R. Kusumoto, L. Palmieri, M. Spies, A. Csiszar, and K. O. Arras, "Informed Information Theoretic Model Predictive Control," in 2019 International Conference on Robotics and Automation (ICRA), 2019, pp. 2047-2053.
[63] J. Gai, J. Shen, H. Wang, and Y. Hu, "A Parameter-Optimized DBN Using GOA and Its Application in Fault Diagnosis of Gearbox," Shock and Vibration, vol. 2020, p. 4294095, 2020/03/17 2020.
[64] G. Wang, Q. S. Jia, J. Qiao, J. Bi, and M. Zhou, "Deep Learning-Based Model Predictive Control for Continuous Stirred-Tank Reactor System," IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 8, pp. 3643-3652, 2021.
[65] S. Kim, J. Jang, and C. O. Kim, "A run-to-run controller for a chemical mechanical planarization process using least squares generative adversarial networks," Journal of Intelligent Manufacturing, vol. 32, no. 8, pp. 2267-2280, 2021/12/01 2021.
[66] S. R. Khazeiynasab, J. Zhao, I. Batarseh, and B. J. I. T. o. P. S. Tan, "Power plant model parameter calibration using conditional variational autoencoder," vol. 37, no. 2, pp. 1642-1652, 2021.
[67] W. Xu, H. Peng, X. Zeng, F. Zhou, X. Tian, and X. Peng, "A Hybrid Modeling Method Based on Linear AR and Nonlinear DBN-AR Model for Time Series Forecasting," Neural Processing Letters, vol. 54, no. 1, pp. 1-20, 2022/02/01 2022.
[68] H. Hassanpour, B. Corbett, P. J. I. Mhaskar, and E. C. Research, "Artificial Neural Network-Based Model Predictive Control Using Correlated Data," vol. 61, no. 8, pp. 3075-3090, 2022.
[69] J. Pfrommer, C. Zimmerling, J. Liu, L. Kärger, F. Henning, and J. Beyerer, "Optimisation of manufacturing process parameters using deep neural networks as surrogate models," Procedia CIRP, vol. 72, pp. 426-431, 2018/01/01/ 2018.
[70] S. Wang et al., "A DRL-Aided Multi-Layer Stability Model Calibration Platform Considering Multiple Events," in 2020 IEEE Power & Energy Society General Meeting (PESGM), 2020, pp. 1-5.
[71] W. Wu, L. Lin, B. Xu, S. Wshah, and R. Elmoudi, Generator Model Parameter Calibration Using Reinforcement Learning. 2020, pp. 1-6.
[72] C. Mazgualdi, T. Masrour, I. Hassani, and A. Khdoudi, "A Deep Reinforcement Learning (DRL) Decision Model for Heating Process Parameters Identification in Automotive Glass Manufacturing," 2021, pp. 77-87.
[73] F. Ogoke and A. B. Farimani, "Thermal control of laser powder bed fusion using deep reinforcement learning," Additive Manufacturing, vol. 46, p. 102033, 2021/10/01/ 2021.
[74] P. Dong, Z.-M. Chen, X.-W. Liao, and W. Yu, "A deep reinforcement learning (DRL) based approach for well-testing interpretation to evaluate reservoir parameters," Petroleum Science, vol. 19, no. 1, pp. 264-278, 2022/02/01/ 2022.
[75] S. R. Khazeiynasab, J. Zhao, I. Batarseh, and B. Tan, "Power Plant Model Parameter Calibration Using Conditional Variational Autoencoder," IEEE Transactions on Power Systems, vol. 37, no. 2, pp. 1642-1652, 2022.
[76] R. J. Williams, "Simple statistical gradient-following algorithms for connectionist reinforcement learning," Machine Learning, vol. 8, no. 3, pp. 229-256, 1992/05/01 1992.
[77] T. P. Lillicrap et al., "Continuous control with deep reinforcement learning," 2015.
[78] J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, "Trust region policy optimization," in International conference on machine learning, 2015, pp. 1889-1897: PMLR.
[79] V. Mnih et al., "Asynchronous methods for deep reinforcement learning," in International conference on machine learning, 2016, pp. 1928-1937: PMLR.
[80] L. Auret and C. Aldrich, "Interpretation of nonlinear relationships between process variables by use of random forests," Minerals Engineering, vol. 35, pp. 27-42, 2012/08/01/ 2012.
[81] H. Bozdogan, "Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions," Psychometrika, vol. 52, no. 3, pp. 345-370, 1987/09/01 1987.
[82] A. Bissel, "How reliable is your capability index?," Quality control applied statistics, vol. 36, no. 7, pp. 385-386, 1991.
[83] V. S. Özsoy, M. G. Ünsal, and H. H. Örkcü, "Use of the heuristic optimization in the parameter estimation of generalized gamma distribution: comparison of GA, DE, PSO and SA methods," vol. 35, no. 4, pp. 1895-1925, 2020.
[84] C.-B. Jin. (2018). VAE-Tensorflow. Available: https://github.com/ChengBinJin/VAE-Tensorflow