隨著大數據雲端運算的需求日漸增加，非揮發性記憶體儲存標準固態硬碟 (NVMe SSD) 和MapReduce框架提供高度平行的資料儲存效能以及平行運算能力。然而此類平行架構中，應用程式之間資源搶奪的干擾和固態硬碟的垃圾收集，造成應用程式無法維持服務層級目標(SLO)。

在本論文中，為了滿足所有應用程式的服務層級目標，我們提出了根據MapReduce型態感知的資源保留策略與相對應的資源分配程序，並提出非同步可搶占之垃圾收集程序，用以減少垃圾收集程序造成的延遲。從實驗結果顯示，我們提出的方法能夠滿足應用程式的服務層級目標。

With the increased number of large-scale data processing, the Non-Volatile Memory Express (NVMe) solid-state disks (SSDs) and the parallel programming model MapReduce are widely used in the cloud servers for providing high data parallelism processing. However, providing the service level objective (SLO) of multiple MapReduce applications on NVMe SSDs becomes complicated from the interference between applications and garbage collection (GC) of the SSD. In this work, we proposed the MapReduce-aware chip reservation, an asynchronous semi-preemptive garbage collection (ASGC), and a request allocation policy. Evaluation results show that our proposed approach delivered a stable throughput while meeting the SLO of applications.

[1] J. Dean and S. Ghemawat, “Mapreduce: simplified data processing on large clusters,” Commun. ACM, vol. 51, no. 1, pp. 107–113, 2008.
[2] M. Bjørling, J. Axboe, D. Nellans, and P. Bonnet, “Linux block io: introducing multi-queue ssd access on multi-core systems,” in Proceedings of the International Systems and Storage Conference, p. 22, ACM, 2013.
[3] “NVM Express.” https://nvmexpress.org, 2019.
[4] C. Gao, L. Shi, M. Zhao, C. J. Xue, K. Wu, and E. H.-M. Sha, “Exploiting parallelism in i/o scheduling for access conflict minimization in flash based solid state drives,” in Proceedings of the Symposium on Mass Storage Systems and Technologies (MSST), pp. 1–11, IEEE, 2014.
[5] N. Elyasi, M. Arjomand, A. Sivasubramaniam, M. T. Kandemir, C. R. Das, and M. Jung, “Exploiting intra-request slack to improve ssd performance,” ACM SIGARCH Computer Architecture News, vol. 45, no. 1, pp. 375–388, 2017.
[6] J. Guo, Y. Hu, B. Mao, and S. Wu, “Parallelism and garbage collection aware i/o scheduler with improved ssd performance,” in IEEE International Parallel and Distributed Processing Symposium (IPDPS), pp. 1184–1193, IEEE, 2017.
[7] J. Lee, Y. Kim, G. M. Shipman, S. Oral, F. Wang, and J. Kim, “A semi-preemptive
garbage collector for solid state drives,” in IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pp. 12–21, IEEE, 2011.
[8] W. Choi, M. Jung, M. Kandemir, and C. Das, “Parallelizing garbage collection with i/o to improve flash resource utilization,” in Proceedings of the International Symposium on High-Performance Parallel and Distributed Computing, pp. 243–254, ACM, 2018.
[9] B. Mao and S. Wu, “Exploiting request characteristics and internal parallelism to improve ssd performance,” in IEEE International Conference on Computer Design (ICCD), pp. 447–450, IEEE, 2015.
[10] Q. Zhang, X. Li, L. Wang, T. Zhang, Y. Wang, and Z. Shao, “Lazy-rtgc: A real-time lazy garbage collection mechanism with jointly optimizing average and worst performance for nand flash memory storage systems,” ACM Transactions on Design Automation of Electronic Systems (TODAES), vol. 20, no. 3, p. 43, 2015.
[11] W. Kang and S. Yoo, “Dynamic management of key states for reinforcement learning-assisted garbage collection to reduce long tail latency in ssd,” in Proceedings of the ACM/ESDA/IEEE Design Automation Conference (DAC), pp. 1–6, IEEE, 2018.
[12] W. Kang, D. Shin, and S. Yoo, “Reinforcement learning-assisted garbage collection to mitigate long-tail latency in ssd,” ACM Transactions on Embedded Computing Systems (TECS), vol. 16, no. 5s, p. 134, 2017.
[13] T. Yang, P. Huang, W. Zhang, H. Wu, and L. Lin, “Cars: A multi-layer conflict-aware request scheduler for nvme ssds,” in Proceedings of the Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1293–1296, IEEE, 2019.
[14] X. Xie, L. Xiao, D.Wei, Q. Li, Z. Song, and X. Ge, “Pinpointing and scheduling access conflicts to improve internal resource utilization in solid-state drives,” Frontiers of Computer Science, vol. 13, no. 1, pp. 35–50, 2019.
[15] S. Wu, W. Zhang, B. Mao, and H. Jiang, “Hotr: Alleviating read/write interference with hot read data replication for flash storage,” in Proceedings of the Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1367–1372, IEEE, 2019.
[16] W. S. Jeong, C. Lee, K. Kim, M. K. Yoon, W. Jeon, M. Jung, and W. W. Ro, “React: Scalable and high-performance regular expression pattern matching accelerator for in-storage processing,” IEEE Transactions on Parallel and Distributed Systems, vol. 31, no. 5, pp. 1137–1151, 2019.
[17] A. Tavakkol, M. Sadrosadati, S. Ghose, J. Kim, Y. Luo, Y.Wang, N. M. Ghiasi, L. Orosa, J. G´omez-Luna, and O. Mutlu, “Flin: Enabling fairness and enhancing performance in modern nvme solid state drives,” in Proceedings of the ACM/IEEE Annual International Symposium on Computer Architecture (ISCA), pp. 397–410, IEEE, 2018.
[18] B. Jun and D. Shin, “Workload-aware budget compensation scheduling for nvme solid state drives,” in IEEE Non-Volatile Memory System and Applications Symposium (NVMSA), pp. 1–6, IEEE, 2015.
[19] S.-M. Huang and L.-P. Chang, “Providing slo compliance on nvme ssds through parallelism reservation,” ACM Transactions on Design Automation of Electronic Systems (TODAES), vol. 23, no. 3, pp. 28:1–28:26, 2018.
[20] J. Shafer, S. Rixner, and A. L. Cox, “The hadoop distributed filesystem: Balancing portability and performance,” in Proceedings of the IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS), pp. 122–133, IEEE, 2010.
[21] J. Hong, L. Li, C. Han, B. Jin, Q. Yang, and Z. Yang, “Optimizing Hadoop framework for solid state drives,” in IEEE International Congress on Big Data (BigData Congress), pp. 9–17, IEEE, 2016.
[22] S. Moon, J. Lee, X. Sun, and Y.-s. Kee, “Optimizing the hadoop mapreduce framework with high-performance storage devices,” The Journal of Supercomputing, vol. 71, no. 9, pp. 3525–3548, 2015.
[23] Y.-S. Lee, L. C. Quero, S.-H. Kim, J.-S. Kim, and S. Maeng, “Activesort: Efficient external sorting using active ssds in the mapreduce framework,” Future Generation Computer Systems, vol. 65, pp. 76–89, 2016.
[24] K. Krish, B. Wadhwa, M. S. Iqbal, M. M. Rafique, and A. R. Butt, “On efficient hierarchical storage for big data processing,” in Proceedings of the IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), pp. 403–408, IEEE, 2016.
[25] F. Alsayoud and A. Miri, “Ssd: Cache or tier an evaluation of ssd cost and efficiency using mapreduce,” in Proceedings of the IEEE/ACS 16th International Conference on Computer Systems and Applications (AICCSA), pp. 1–8, IEEE, 2019.
[26] W. Tan, L. Fong, and Y. Liu, “Effectiveness assessment of solid-state drive used in big data services,” in IEEE International Conference on Web Services (ICWS), pp. 393–400, IEEE, 2014.
[27] X. T. Tran, T. Van Do, N. H. Do, L. Farkas, and C. Rotter, “Provision of disk i/o guarantee for mapreduce applications,” in IEEE Trustcom/BigDataSE/ISPA, vol. 2, pp. 161–166, IEEE, 2015.
[28] H. Jeon, K. El Maghraoui, and G. B. Kandiraju, “Investigating hybrid ssd ftl schemes for hadoop workloads,” in Proceedings of the International Conference on Computing Frontiers, pp. 1–10, 2013.
[29] Y. Hu, H. Jiang, D. Feng, L. Tian, H. Luo, and S. Zhang, “Performance impact and interplay of ssd parallelism through advanced commands, allocation strategy and data granularity,” in Proceedings of the International Conference on Supercomputing, pp. 96–107, 2011.
[30] M. Li, L. Zeng, S. Meng, J. Tan, L. Zhang, A. R. Butt, and N. Fuller, “Mronline: Mapreduce online performance tuning,” in Proceedings of the International Symposium on High-performance parallel and distributed computing, pp. 165–176, 2014.
[31] V. Pandey and P. Saini, “A heuristic method towards deadline-aware energy-efficient mapreduce scheduling problem in hadoop yarn,” Cluster Computing, pp. 1–17, 2020.
[32] “Flexible i/o tester rev. 3.10.” https://github.com/axboe/fio/releases/tag/fio-3.10, 2018.