簡易檢索 / 詳目顯示

回結果列表
研究生： 陳宣穎
Hsuan-Ying Chen
論文名稱： Bayer CFA 影像的壓縮：文獻回顧和性能比較
Compression for Bayer CFA Images: Review and Performance Comparison
指導教授： 鍾國亮
Kuo-Liang Chung
口試委員: 鍾國亮
陳建中
貝蘇章
蔡文祥
李同益
學位類別： 碩士
Master
系所名稱： 電資學院 - 資訊工程系
Department of Computer Science and Information Engineering
論文出版年： 2023
畢業學年度： 111
語文別： 英文
論文頁數： 66
中文關鍵詞： 拜爾濾色鏡 、色彩採樣 、亮度優化 、JPEG-2000 、多功能影像編碼
外文關鍵詞： Bayer color filter array (CFA) images, chroma subsampling, luma modification, JPEG-2000, Versatile Video Coding (VVC)
相關次數： 點閱：190  下載：3 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • Bayer 彩色濾波陣列（CFA）圖像是透過一個覆蓋有 Bayer CFA 模 式的單一晶片影像感測器所拍攝的，這種模式在現今的數位相機中廣泛 使用。過去 20 年來，已經提出許多壓縮方法來壓縮 BayerCFA 圖像。這 些方法可以大略分為壓縮優先（CF-based）和解馬賽克優先（DF-based） 兩種類型。然而，目前還沒有相關的綜述文章詳細介紹這兩種壓縮方法和 它們的壓縮效能。本文首先回顧相關的 CF-based 和 DF-based 壓縮工作， 接著使用 Joint Photographic Experts Group-2000（JPEG-2000）和新推 出的 Versatile Video Coding（VVC）平台 VTM-16.2 來壓縮 Kodak、 IMAX、螢幕內容圖像、影片以及經典圖像資料庫所創建的 Bayer CFA 圖像。根據通常使用的客觀品質、知覺品質指標、知覺效果以及品質-比 特率折衷指標，報告並討論 CF-based 壓縮方法，特別是可逆色彩轉換壓 縮方法和 DF-based 壓縮方法的壓縮效能比較。

    Bayer color filter array (CFA) images are captured by a single-chip image sensor covered with a Bayer CFA pattern which has been widely used in modern digital cameras. In the past two decades, many compression methods have been proposed to compress Bayer CFA images. These compression methods can be roughly divided into the compression-first-based (CFbased) scheme and the demosaicing-first-based (DF-based) scheme. However, in the literature, no review article for the two compression schemes and their compression performance is reported. In this article, the related CF-based and DF-based compression works are reviewed first. Then, the testing Bayer CFA images created from the Kodak, IMAX, screen content images, videos, and classical image datasets are compressed on the Joint Photographic Experts Group-2000 (JPEG-2000) and the newly released Versatile Video Coding (VVC) platform VTM-16.2. In terms of the commonly used objective quality, perceptual quality metrics, the perceptual effect, and the quality–bitrate tradeoff metric, the compression performance comparison of the CF-based compression methods, in particular the reversible color transform-based compression methods and the DF-based compression methods, is reported and discussed.

    Contents Recommendation Letter . . . . . . . . . . . . . . . . . . . . . . . . i Approval Letter . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii Abstract in Chinese . . . . . . . . . . . . . . . . . . . . . . . . . . iii Abstract in English . . . . . . . . . . . . . . . . . . . . . . . . . . iv Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . v Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 The Related CF-Based Compression Methods . . . . . . . 3 1.2 The Related DF-Based Compression Methods . . . . . . . 6 1.3 Motivation and Contribution . . . . . . . . . . . . . . . . 7 2 The Reversible Color Transform-Based (RCT-Based) Compression Works for Bayer CFA Images . . . . . . . . . . . . . . . . 10 2.1 The Y1Cr2Cb3Y4 Method . . . . . . . . . . . . . . . . . . 10 2.2 The YDgCoCg Method . . . . . . . . . . . . . . . . . . . 12 2.3 The YLMN Method . . . . . . . . . . . . . . . . . . . . 14 2.4 The Y∆CbCr Method . . . . . . . . . . . . . . . . . . . . 15 3 The Demosaicing-First-Based (DF-Based) Compression Works for Bayer CFA Images . . . . . . . . . . . . . . . . . . . . . . . 17 3.1 Demosaicing I^{Bayer} to I^{Demo,RGB} and Then Converting I^{Demo,RGB} to I^{YCbCr}. . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.1.1 Demosaicing I^{Bayer} to I^{Demo,RGB} . . . . . . . . . 18 3.1.2 Converting I^{Demo,RGB} to I^{YCbCr}. . . . . . . . . . 20 3.2 Chroma Subsampling . . . . . . . . . . . . . . . . . . . . 20 3.2.1 The Bayer CFA Pattern-Independent Chroma Subsampling Methods . . . . . . . . . . . . . . . . . 21 3.2.2 The Bayer CFA Pattern-Dependent Chroma Subsampling Methods . . . . . . . . . . . . . . . . . 22 3.3 Luma Modification . . . . . . . . . . . . . . . . . . . . . 30 4 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . 33 4.1 Quality Comparison and Discussion . . . . . . . . . . . . 34 4.1.1 Quality Comparison and Discussion . . . . . . . . 34 4.1.2 Execution Time Requirement Comparison and Discussion . . . . . . . . . . . . . . . . . . . . . . . 38 4.2 Quality–Bitrate Tradeoff Comparison and Discussion . . . 39 4.2.1 The Quality–Bitrate Tradeoff Comparison . . . . 40 4.2.2 The Visual Effect Comparison . . . . . . . . . . . 43 5 Conclusions and Future Works . . . . . . . . . . . . . . . . . . 47 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

    [1] H. Chen, M. Sun, and E. Steinbach, “Compression of bayer-pattern video sequences using adjusted chroma subsampling,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 19, no. 12, pp. 1891–1896, 2009.
    [2] B. E. Bayer, “Color imaging array,” United States Patent 3,971,065, 1976.
    [3] S.-Y. Lee and A. Ortega, “A novel approach of image compression in digital cameras with a bayer color filter array,” in Proceedings 2001 International Conference on Image Processing (Cat. No. 01CH37205), vol. 3, pp. 482–485, IEEE, 2001.
    [4] H. S. Malvar and G. J. Sullivan, “Progressive-to-lossless compression of color-filter-array images using macropixel spectral-spatial transformation,” in 2012 Data Compression Conference, pp. 3–12, IEEE, 2012.
    [5] S. K. Mohammed and K. A. Wahid, “Lossless compression in bayer color filter array for capsule endoscopy,” IET Image Processing, vol. 5, pp. 13823 – 13834, 2017.
    [6] S. K. Mohammed and K. A. Wahid, “Lossless and reversible colour space transformation for bayer colour filter array images,” IET Image Processing, vol. 12, no. 8, pp. 1485–1490, 2018.
    [7] K. M. M. Rahman, S. K. Mohammed, S. S. Vedaei, M. R. Mohebbian, F. S. Chafjiri, and K. A. Wahid, “A low complexity lossless bayer cfa image compression,” Signal, Image and Video Processing, vol. 15, pp. 1767–1775, 2021.
    [8] M. Hernández-Cabronero, M. W. Marcellin, I. Blanes, and J. Serra-Sagristà, “Lossless compression of color filter array mosaic images with visualization via jpeg 2000,” IEEE Transactions on Image Processing, vol. 20, no. 2, pp. 257–270, 2018.
    [9] T. Richter and S. Fößel, “Bayer pattern compression with jpeg xs,” 2019 IEEE International Conference on Image Processing (ICIP), pp. 3177–3181, 2019.
    [10] T. Richter, S. Fößel, A. Descampe, and G. Rouvroy, “Bayer cfa pattern compression with jpeg xs,” IEEE Transactions on Image Processing, vol. 30, pp. 6557–6569, 2021.
    [11] N. Zhang and X. Wu, “Lossless compression of color mosaic images,” IEEE Transactions on Image Processing, vol. 15, no. 6, pp. 1379–1388, 2006.
    [12] Y. Lee, K. Hirakawa, and T. Q. Nguyen, “Camera-aware multi-resolution analysis for raw image sensor data compression,” IEEE Transactions on Image Processing, vol. 27, no. 6, pp. 2806–2817, 2018.
    [13] T. Suzuki, “Wavelet-based spectral–spatial transforms for cfa-sampled raw camera image compression,” IEEE Transactions on Image Processing, vol. 29, pp. 433–444, 2019.
    [14] Y. Lee and K. Hirakawa, “Shift-and-decorrelate lifting: Camra for lossless intra frame cfa video compression,” IEEE Signal Processing Letters, vol. 27, pp. 461 – 465, 2020.
    [15] L. Huang and T. Suzuki, “Weighted wavelet-based spectral-spatial transforms for cfa-sampled raw camera image compression considering image features,” IEEE International Conference on Acoustics, Speech and Signal Processing, 2022.
    [16] K.-H. Chung and W. Xiaolin, “A lossless compression scheme for bayer color filter array images,” IEEE Transactions on Image Processing, vol. 17, no. 2, pp. 134–144, 2008.
    [17] W. B. Pennebaker and J. L. Mitchell, JPEG: Still Image Data Compression Standard. Springer Science & Business Media, 1992.
    [18] F. Dufaux, G. J. Sullivan, and T. Ebrahimi, “The jpeg xr image coding standard [standards in a nutshell],” IEEE Signal Processing Magazine, vol. 26, no. 6, pp. 195–204, 2009.
    [19] A. Skodras, C. Christopoulos, and T. Ebrahimi, “The jpeg 2000 still image compression standard,” IEEE Signal Processing Magazine, vol. 18, no. 5, pp. 36–58, 2001.
    [20] M. Rabbani, “Book review: Jpeg2000: Image compression fundamentals, standards and practice,” 2002.
    [21] H. S. Malvar, “Adaptive run-length/golomb-rice rncoding of quantized generalized gaussian sources with unknown statistics,” in Data Compression Conference (DCC’06), pp. 23–32, IEEE, 2006.
    [22] “JPEG XS,” standard, ISO/IEC 21122, 2019.
    [23] S. Mallat, A Wavelet Tour of Signal Processing. Academic Press, 3 ed., 12 2008.
    [24] G. Sullivan and S. Sun, “Spatial scalability filters,” Poznan, Poland, Tech. Rep. JVT-P007, 2005.
    [25] A. Luthra, E. Francois, and W. Husak, “Call for evidence (cfe) for hdr and wcg video coding,” ISO/ IEC JTC1/SC29/WG11 MPEG2014 N, vol. 15083, 2015.
    [26] Y. Zhang, D. Zhao, J. Zhang, R. Xiong, and W. Gao, “Interpolation-dependent image downsampling,” IEEE Transactions on Image Processing, vol. 20, no. 11, pp. 3291–3296, 2011.
    [27] S. Wang, K. Gu, S. Ma, and W. Gao, “Joint chroma downsampling and upsampling for screen content image,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 26, no. 9, pp. 1595– 1609, 2015.
    [28] C.-H. Lin, K.-L. Chung, and C.-W. Yu, “Novel chroma subsampling strategy based on mathematical optimization for compressing mosaic videos with arbitrary rgb color filter arrays in h. 264/avc and hevc,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 26, no. 9, pp. 1722– 1733, 2016.
    [29] K.-L. Chung, Y.-L. Lee, and W.-C. Chien, “Effective gradient descent-based chroma subsampling method for bayer cfa images in hevc,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 29, no. 11, pp. 3281–3290, 2019.
    [30] T.-L. Lin, Y.-C. Yu, K.-H. Jiang, C.-F. Liang, and P.-S. Liaw, “Novel chroma sampling methods for cfa video compression in avc, hevc and vvc,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, no. 9, pp. 3167–3180, 2019.
    [31] K.-L. Chung and S.-N. Chen, “An effective bilinear interpolation-based iterative chroma subsampling method for color images,” Multimedia Tools and Applications, pp. 1–23, 2022.
    [32] K.-L. Chung, T.-C. Leung, T.-Y. Liu, and Y.-C. Tseng, “A cubic convolution interpolation-based chroma subsampling method for bayer and rgbw cfa raw images,” IEEE Access, vol. 10, pp. 22687– 22699, 2022.
    [33] K.-L. Chung, T.-C. Hsu, and C.-C. Huang, “Joint chroma subsampling and distortion-minimizationbased luma modification for rgb color images with application,” IEEE Transactions on Image Processing, vol. 26, no. 10, pp. 4626–4638, 2017.
    [34] Y.-H. Chiu, K.-L. Chung, and C.-H. Lin, “An improved universal subsampling strategy for compressing mosaic videos with arbitrary rgb color filter srrays in h. 264/avc,” Journal of Visual Communication and Image Representation, vol. 25, no. 7, pp. 1791–1799, 2014.
    [35] “Vtm-16.2,” 2022. Available: https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/ -/tree/VTM-16.2.
    [36] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004.
    [37] L. Zhang, L. Zhang, X. Mou, and D. Zhang, “Fsim: A feature similarity index for image quality assessment,” IEEE Transactions on Image Processing, vol. 20, no. 8, pp. 2378–2386, 2011.
    [38] G. Bjontegaard, “Calculation of average psnr differences between rd-curves,” VCEG-M33, 2001.
    [39] R. Stasinski and J. Konrad, “A new class of fast shape-adaptive orthogonal transforms and their application to region-based image compression,” IEEE Transactions on Image Processing, vol. 9, pp. 16– 34, 1999.
    [40] H. S. Malvar and G. J. Sullivan, “Ycocg-r: A color space with rgb reversibility and low dynamic range,” ITU-T/ISO/IEC JVT Document JVT-I014, 2003.
    [41] M. D. Adams, F. Kossentini, and R. K. Ward, “Generalized s transform,” IEEE Transactions on Signal Processing, vol. 50, pp. 2831–2842, 2002.
    [42] H. S. Malvar, G. J. Sullivan, and S. Srinivasan, “Lifting-based reversible color transformations for image compression,” Applications of Digital Image Processing XXXI, vol. 7073, p. 707307, 2008.
    [43] K. T.H. and K. A. Wahid, “Low-complexity colour-space for capsule endoscopy image compression. electron,” Electronics Letters, vol. 47, no. 22, pp. 1217–1218, 2011.
    [44] B. K. Gunturk, J. Glotzbach, Y. Altunbasak, R. W. Schafer, and R. M. Mersereau, “Demosaicking: Color filter array interpolation,” IEEE Signal Processing Magazine, vol. 22, no. 1, pp. 44–54, 2005.
    [45] X. Li, B. Gunturk, and L. Zhang, “Image demosaicing: A systematic survey,” in Visual Communications and Image Processing 2008, vol. 6822, pp. 489–503, SPIE, 2008.
    [46] S. Mihoubi, P.-J. Lapray, and L. Bigué, “Survey of demosaicking methods for polarization filter array images,” Sensors, vol. 18, no. 11, p. 3688, 2018.
    [47] Y. Wang, R. Cao, Y. Guan, T. Liu, and Z. Yu, “A deep survey in the applications of demosaicking,” in 2021 3rd International Academic Exchange Conference on Science and Technology Innovation (IAECST), pp. 596–602, IEEE, 2021.
    [48] T. Sakamoto, C. Nakanishi, and T. Hase, “Software pixel interpolation for digital still cameras suitable for a 32-bit mcu,” IEEE Transactions on Consumer Electronics, vol. 44, no. 4, pp. 1342–1352, 1998.
    [49] R. Kimmel, “Demosaicing: Image reconstruction from color ccd samples,” IEEE Transactions on Image Processing, vol. 8, no. 9, pp. 1221–1228, 1999.
    [50] B. K. Gunturk, Y. Altunbasak, and R. M. Mersereau, “Color plane interpolation using alternating projections,” IEEE Transactions on Image Processing, vol. 11, no. 9, pp. 997–1013, 2002.
    [51] S.-C. Pei and I.-K. Tam, “Effective color interpolation in ccd color filter arrays using signal correlation,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, no. 6, pp. 503–513,
    2003.
    [52] W. Lu and Y.-P. Tan, “Color filter array demosaicking: New method and performance measures,”
    IEEE Transactions on Image Processing, vol. 12, no. 10, pp. 1194–1210, 2003.
    [53] X. Wu and N. Zhang, “Primary-consistent soft-decision color demosaicking for digital cameras (patent pending),” IEEE Transactions on Image Processing, vol. 13, no. 9, pp. 1263–1274, 2004.
    [54] R. Lukac and K. N. Plataniotis, “Normalized color-ratio modeling for cfa interpolation,” IEEE Transactions on Consumer Electronics, vol. 50, no. 2, pp. 737–745, 2004.
    [55] K. Hirakawa and T. W. Parks, “Adaptive homogeneity-directed demosaicing algorithm,” IEEE Transactions on Image Processing, vol. 14, no. 3, pp. 360–369, 2005.
    [56] K.-L. Chung, W.-J. Yang, W.-M. Yan, and C.-C. Wang, “Demosaicing of color filter array captured images using gradient edge detection masks and adaptive heterogeneity-projection,” IEEE Transactions on Image Processing, vol. 17, no. 12, pp. 2356–2367, 2008.
    [57] L. Condat, “A generic variational approach for demosaicking from an arbitrary color filter array,” in 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 1625–1628, IEEE, 2009.
    [58] W.-J. Yang, K.-L. Chung, W.-N. Yang, and L.-C. Lin, “Universal chroma subsampling strategy for compressing mosaic video sequences with arbitrary rgb color filter arrays in h.264/avc,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 23, no. 4, pp. 591–606, 2013.
    [59] L. Zhang, X. Wu, A. Buades, and X. Li, “Color demosaicking by local directional interpolation and nonlocal adaptive thresholding,” Journal of Electronic Imaging, vol. 20, no. 2, p. 023016, 2011.
    [60] D. Kiku, Y. Monno, M. Tanaka, and M. Okutomi, “Residual interpolation for color image demosaicking,” in 2013 IEEE International Conference on Image Processing, pp. 2304–2308, IEEE, 2013.
    [61] W. Ye and K.-K. Ma, “Color image demosaicing using iterative residual interpolation,” IEEE Transactions on Image Processing, vol. 24, no. 12, pp. 5879–5891, 2015.
    [62] D. S. Tan, W.-Y. Chen, and K.-L. Hua, “Deepdemosaicking: Adaptive image demosaicking via multiple deep fully convolutional networks,” IEEE Transactions on Image Processing, vol. 27, no. 5, pp. 2408–2419, 2018.
    [63] N.-S. Syu, Y.-S. Chen, and Y.-Y. Chuang, “Learning deep convolutional networks for demosaicing,” ArXiv Preprint arXiv:1802.03769, 2018.
    [64] Z. Ni, K.-K. Ma, H. Zeng, and B. Zhong, “Color image demosaicing using progressive collaborative representation,” IEEE Transactions on Image Processing, vol. 29, pp. 4952–4964, 2020.
    [65] R. I.-R. BT et al., “Studio encoding parameters of digital television for standard 4: 3 and wide-screen 16: 9 aspect ratios,” Int. Radio Consultative Committee Int. Telecommun. Union, Switzerland, CCIR Rep, pp. 624–4, 2011.
    [66] S. Winkler, M. Kunt, and C. J. Branden Lambrecht, “Vision and video: Models and applications,” in Vision Models and Applications to Image and Video Processing, pp. 201–229, Springer, 2001.
    [67] X. Li and M. T. Orchard, “New edge-directed interpolation,” IEEE Transactions on Image Processing, vol. 10, no. 10, pp. 1521–1527, 2001.
    [68] Y. Lu, S. Li, and H. Shen, “Virtualized screen: A third element for cloud-mobile convergence,” IEEE Multimedia, vol. 18, no. 2, pp. 4–11, 2011.
    [69] G. Bradski and A. Kaehler, Learning OpenCV: Computer Vision with the OpenCV Library. ” O’Reilly Media, Inc.”, 2008. Available: https://books.google.com.tw/books?hl=zh-TW&lr=&id=seAgiOfu2EIC&oi=fnd&pg=PR3&dq=opencv+library&ots=hUL28niGSa&
    sig=2f-MT4LGYlFPnyfkVd6xLIo27VM&redir_esc=y#v=snippet&q=bilinear&f=false.
    [70] J. R. Magnus and H. Neudecker, Matrix Differential Calculus with Applications in Statistics and Econometrics. John Wiley & Sons, 2019.
    [71] Y.-C. Yu, J.-W. Jhang, X. Wei, H.-W. Tseng, Y. Wen, Z. Liu, T.-L. Lin, S.-L. Chen, Y.-S. Chiou, and H.-Y. Lee, “Chroma upsampling for ycbcr 420 videos,” in 2017 IEEE International Conference on Consumer Electronics - Taiwan (ICCE-TW), pp. 163–164, 2017.
    [72] T. Yamagami, T. Sasaki, and A. Suga, “Image signal processing apparatus having a color filter with offset luminance filter elements,” June 21 1994. US Patent 5,323,233.
    [73] M. Tachi, “Image processing device, image processing method, and program pertaining to image correction,” Nov. 20 2012. US Patent 8,314,863.
    [74] J. T. Compton and J. F. Hamilton, “Image sensor with improved light sensitivity,” Mar. 20 2012. U.S. Patent 8 139 130 B2.
    [75] S. ho Lee, P. Oh, and M. G. Kang, “Three dimensional colorization based image/video reconstruction from white-dominant rgbw pattern images,” Digital Signal Processing, vol. 93, pp. 87–101, 2019.
    [76] P.-H. Su, P.-C. Chen, and H. H. Chen, “Compensation of spectral mismatch to enhance wrgb demosaicking,” in 2015 IEEE International Conference on Image Processing (ICIP), pp. 68–72, 2015.
    [77] K.-L. Chung, T.-H. Chan, and S.-N. Chen, “Effective three-stage demosaicking method for rgbw cfa images using the iterative error-compensation based approach,” Sensors, vol. 20, no. 14, p. 3908, 2020.
    [78] I. Bakurov, M. Buzzelli, R. Schettini, M. Castelli, and L. Vanneschi, “Structural similarity index (ssim) revisited: A data-driven approach,” Expert Systems with Applications, vol. 189, p. 116087, 2022.
    [79] A. K. Venkataramanan, C. Wu, A. C. Bovik, I. Katsavounidis, and Z. Shahid, “A hitchhiker’s guide to structural similarity,” IEEE Access, vol. 9, pp. 28872–28896, 2021.

    QR CODE