黃疸在新生兒中很常見，需要幾天的追蹤。如果要得到總血清膽紅素 (TSB)值，需要標準程序血液測試。抽血會造成疼痛以及一定的風險，且不能在醫療環境以外的地方進行。因此，需要一種可以在家中完成的非侵入性方法。然而，以往許多黃疸檢測方法的研究存在以下不足。感興趣區域（Region of Interest, ROI）需要手動分割或使用顏色空間閾值等方法進行分割，但準確度可能不令人滿意。此外，照片的品質、方向和拍攝距離都必須手動確認。此外，許多研究選擇一些顏色空間，然後在不評估它們的情況下使用所有通道進行預測；然而，一個顏色空間中的某些通適合用於檢測黃疸，而其他通道則不適合。為了解決這些問題，我們提出了一個名為 JDCC（Jaundice Detection with Calibration Cards）的新框架來預測新生兒膽紅素。它包括一個我們設計的新穎的皮膚校正卡，可以實現顏色校正和檢測區域定位，從而克服上述所有採樣和顏色校正問題。此外，我們進行色彩空間通道分析和整合，使模型可以使用更少的通道獲得更高的性能。還分析了顏色通道的特徵，幫助我們了解和證明哪些顏色通道適合預測膽紅素。此外，我們為最終輸出開發了一個集成協議，以降低假陰性率。由於錯判真正的高膽紅素血症病例是最不好的，因此假陰性的發生是非常危險的。實驗結果表明我們的方法相較於最具代表性的方法，在真實新生兒黃疸資料集中將斯皮爾曼相關性提高了6.5%，平均絕對誤差降低了5.8%。

Neonatal jaundice is common in newborns, and it requires several days of follow-up. A standard procedure blood test is required to obtain total serum bilirubin (TSB) levels. Blood draws are painful and risky, and should not be performed outside of a medical setting. Therefore, a non-invasive method that can be completed at home is necessary. However, many previous studies of non-invasive jaundice detection methods have the following shortcomings. Regions of interest (ROI) need to be segmented manually or using certain methods such as color space thresholding, but the accuracy may not be satisfying. In addition, the photo’s quality, orientation, and shooting distance have to be manually confirmed. Moreover, many studies select some color spaces and then employ all the channels for prediction without evaluating them; however, some channels in one color space are useful for detecting jaundice while others are not. To address these issues, we propose a new framework called JDCC (Jaundice Detection with Calibration Cards) for newborn bilirubin prediction. It includes a novel skin calibration card, which is able to achieve color calibration and locate the detection region, thereby conquering all the sampling and color correction problems above. In addition, we perform color space channel analysis and integration so that the model may use fewer channels to obtain higher performance. The characteristics of color channels are also analyzed, helping us understand and prove which color channels are suitable for predicting bilirubin levels. Furthermore, missing true cases of hyperbilirubinemia (i.e., the occurrence of false negatives) is very dangerous, since it may delay the medical treatment and cause serious problems. Therefore, we develop an ensemble agreement for the final output to reduce the false negative rate. Extensive experiments show that our method improves Spearman's correlation by 6.5% and mean absolute error by 5.8% compared to the representative baselines in the real-world neonatal jaundice dataset.

[1] Detection of ArUco Markers. shorturl.at/uFNOU. Accessed: 2022.
[2] Belal Ahmad, Mohd Usama, Chuen-Min Huang, Kai Hwang, M Shamim Hossain, and Ghulam
Muhammad. Discriminative feature learning for skin disease classification using deep convolutional
neural network. IEEE Access, 8:39025–39033, 2020.
[3] Alhanoof Althnian, Nada Almanea, and Nourah Aloboud. Neonatal jaundice diagnosis using a smartphone
camera based on eye, skin, and fused features with transfer learning. Sensors, 21(21):7038,
2021.
[4] Richard C Aster, Brian Borchers, and Clifford H Thurber. Parameter estimation and inverse problems.
Elsevier, 2018.
[5] Anders Aune, Gunnar Vartdal, Håkon Bergseng, Lise Lyngsnes Randeberg, and Elisabeth Darj. Bilirubin
estimates from smartphone images of newborn infants’skin correlated highly to serum bilirubin
levels. Acta Paediatrica, 109(12):2532–2538, 2020.
[6] Mustafa Ayd￿n, F￿rat Hardalaç, Berkan Ural, and Serhat Karap. Neonatal jaundice detection system.
Journal of medical systems, 40(7):1–11, 2016.
[7] R Bhatt and A Dhall. Skin segmentation dataset [uci machine learning repository]. retrieved march 6,
2018, 2012.
[8] Vinod K Bhutani, Lois Johnson, and Emidio M Sivieri. Predictive ability of a predischarge hourspecific
serum bilirubin for subsequent significant hyperbilirubinemia in healthy term and near-term
newborns. Pediatrics, 103(1):6–14, 1999.
[9] Vinod K Bhutani, Ann R Stark, Laura C Lazzeroni, Ronald Poland, Glenn R Gourley, Steve Kazmierczak,
Linda Meloy, Anthony E Burgos, Judith Y Hall, David K Stevenson, et al. Predischarge screening
for severe neonatal hyperbilirubinemia identifies infants who need phototherapy. The Journal of
pediatrics, 162(3):477–482, 2013.
[10] Judit Bort-Roig, Nicholas D Gilson, Anna Puig-Ribera, Ruth S Contreras, and Stewart G Trost. Measuring
and influencing physical activity with smartphone technology: a systematic review. Sports
medicine, 44(5):671–686, 2014.
[11] Leo Breiman. Random forests. Machine learning, 45(1):5–32, 2001.
[12] Lilian De Greef, Mayank Goel, Min Joon Seo, Eric C Larson, James W Stout, James A Taylor, and
Shwetak N Patel. Bilicam: using mobile phones to monitor newborn jaundice. In Proceedings of the
2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pages 331–342,
2014.
[13] Graham D Finlayson, Michal Mackiewicz, and Anya Hurlbert. Color correction using root-polynomial
regression. IEEE Transactions on Image Processing, 24(5):1460–1470, 2015.
[14] National Collaborating Centre for Women’s, Children’s Health, et al. Neonatal jaundice: Clinical
guideline. London (UK): National Institute for Health and Clinical Excellence (NICE), 2010.
[15] Paul Geladi and Bruce R Kowalski. Partial least-squares regression: a tutorial. Analytica chimica
acta, 185:1–17, 1986.
[16] María Goñi, Simon B Eickhoff, Mehran Sahandi Far, Kaustubh R Patil, and Juergen Dukart.
Smartphone-based digital biomarkers for parkinson’s disease in a remotely-administered setting.
medRxiv, 2021.
[17] Shubhi Gupta, Sanjeev Thakur, and Ashutosh Gupta. Optimized hybrid machine learning approach
for smartphone based diabetic retinopathy detection. Multimedia Tools and Applications, 81(10):
14475–14501, 2022.
[18] Md Kamrul Hasan, Munirul Haque, Nazmus Sakib, Richard Love, and Sheikh I Ahamed. Smartphonebased
human hemoglobin level measurement analyzing pixel intensity of a fingertip video on different
color spaces. Smart Health, 5:26–39, 2018.
[19] Warqaa Hashim, Ali Al-Naji, Izzat A Al-Rayahi, Makram Alkhaled, and Javaan Chahl. Neonatal
jaundice detection using a computer vision system. Designs, 5(4):63, 2021.
[20] Yangjian Huang, Junkai Xu, Bo Yu, and Peter B Shull. Validity of fitbit, jawbone up, nike+ and other
wearable devices for level and stair walking. Gait & posture, 48:36–41, 2016.
[21] Tae-Eui Kam, Han Zhang, Zhicheng Jiao, and Dinggang Shen. Deep learning of static and dynamic
brain functional networks for early mci detection. IEEE transactions on medical imaging, 39(2):478–
487, 2019.
[22] Michael Kaplan, Ruben Bromiker, and Cathy Hammerman. Hyperbilirubinemia, hemolysis, and increased
bilirubin neurotoxicity. In Seminars in Perinatology, volume 38, pages 429–437. Elsevier,
2014.
[23] Sungjun Kwon, Hyunseok Kim, and Kwang Suk Park. Validation of heart rate extraction using video
imaging on a built-in camera system of a smartphone. In 2012 annual international conference of the
IEEE engineering in medicine and biology society, pages 2174–2177. IEEE, 2012.
[24] Bruno Lecouat, Ken Chang, Chuan-Sheng Foo, Balagopal Unnikrishnan, James M Brown, Houssam
Zenati, Andrew Beers, Vijay Chandrasekhar, Jayashree Kalpathy-Cramer, and Pavitra Krishnaswamy.
Semi-supervised deep learning for abnormality classification in retinal images. arXiv
preprint arXiv:1812.07832, 2018.
[25] Qing Li, Weidong Cai, Xiaogang Wang, Yun Zhou, David Dagan Feng, and Mei Chen. Medical image
classification with convolutional neural network. In 2014 13th international conference on control
automation robotics & vision (ICARCV), pages 844–848. IEEE, 2014.
[26] Amirreza Mahbod, Gerald Schaefer, Chunliang Wang, Rupert Ecker, and Isabella Ellinge. Skin lesion
classification using hybrid deep neural networks. In ICASSP 2019-2019 IEEE International Conference
on Acoustics, Speech and Signal Processing (ICASSP), pages 1229–1233. IEEE, 2019.
[27] Robert G Mannino, David R Myers, Erika A Tyburski, Christina Caruso, Jeanne Boudreaux, Traci
Leong, GD Clifford, and Wilbur A Lam. Smartphone app for non-invasive detection of anemia using
only patient-sourced photos. Nature communications, 9(1):1–10, 2018.
[28] Alex Mariakakis, Megan A Banks, Lauren Phillipi, Lei Yu, James Taylor, and Shwetak N Patel. Biliscreen: smartphone-based scleral jaundice monitoring for liver and pancreatic disorders. Proceedings
of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1(2):1–26, 2017.
[29] Kazuhisa Matsunaga, Akira Hamada, Akane Minagawa, and Hiroshi Koga. Image classification
of melanoma, nevus and seborrheic keratosis by deep neural network ensemble. arXiv preprint
arXiv:1703.03108, 2017.
[30] Sarah B Munkholm, Tobias Krøgholt, Finn Ebbesen, Pal B Szecsi, and Søren R Kristensen. The
smartphone camera as a potential method for transcutaneous bilirubin measurement. PLoS One, 13
(6):e0197938, 2018.
[31] Saritha Murali and VK Govindan. Shadow detection and removal from a single image using lab color
space. Cybern. Inf. Technol, 13(1):95–103, 2013.
[32] Andrea Nemcova, Ivana Jordanova, Martin Varecka, Radovan Smisek, Lucie Marsanova, Lukas Smital,
and Martin Vitek. Monitoring of heart rate, blood oxygen saturation, and blood pressure using a
smartphone. Biomedical Signal Processing and Control, 59:101928, 2020.
[33] Damilola A Okuboyejo, Oludayo O Olugbara, and Solomon A Odunaike. Automating skin disease diagnosis
using image classification. In proceedings of the world congress on engineering and computer
science, volume 2, pages 850–854, 2013.
[34] VA Oliveira and A Conci. Skin detection using hsv color space. In H. Pedrini, & J. Marques de
Carvalho, Workshops of Sibgrapi, pages 1–2. Citeseer, 2009.
[35] Felix Outlaw, Miranda Nixon, Oluwatobiloba Odeyemi, Lindsay W MacDonald, Judith Meek, and
Terence S Leung. Smartphone screening for neonatal jaundice via ambient-subtracted sclera chromaticity.
PloS one, 15(3):e0216970, 2020.
[36] Pouria Padidar, Mohammadamin Shaker, Hamid Amoozgar, Mohammadhossein Khorraminejad-
Shirazi, Fariba Hemmati, Khadijeh Sadat Najib, and Shahnaz Pourarian. Detection of neonatal jaundice
by using an android os-based smartphone application. Iranian Journal of Pediatrics, 29(2), 2019.
[37] Mohammad Fattahi Sani and Ghader Karimian. Automatic navigation and landing of an indoor ar.
drone quadrotor using aruco marker and inertial sensors. In 2017 international conference on computer
and drone applications (IConDA), pages 102–107. IEEE, 2017.
[38] Steven M Shapiro. Definition of the clinical spectrum of kernicterus and bilirubin-induced neurologic
dysfunction (bind). Journal of perinatology, 25(1):54–59, 2005.
[39] Feng Shi, Bojiang Chen, Qiqi Cao, Ying Wei, Qing Zhou, Rui Zhang, Yaojie Zhou, Wenjie Yang,
Xiang Wang, Rongrong Fan, et al. Semi-supervised deep transfer learning for benign-malignant diagnosis
of pulmonary nodules in chest ct images. IEEE Transactions on Medical Imaging, 41(4):
771–781, 2021.
[40] Fabio Alexandre Spanhol, Luiz S Oliveira, Caroline Petitjean, and Laurent Heutte. Breast cancer
histopathological image classification using convolutional neural networks. In 2016 international joint
conference on neural networks (IJCNN), pages 2560–2567. IEEE, 2016.
[41] Peng Tang, Qiaokang Liang, Xintong Yan, Shao Xiang, and Dan Zhang. Gp-cnn-dtel: Global-part
cnn model with data-transformed ensemble learning for skin lesion classification. IEEE journal of
biomedical and health informatics, 24(10):2870–2882, 2020.
[42] James A Taylor, James W Stout, Lilian de Greef, Mayank Goel, Shwetak Patel, Esther K Chung, Aruna
Koduri, Shawn McMahon, Jane Dickerson, Elizabeth A Simpson, et al. Use of a smartphone app to
assess neonatal jaundice. Pediatrics, 140(3), 2017.
[43] JA Stephen Viggiano. Comparison of the accuracy of different white-balancing options as quantified
by their color constancy. In Sensors and Camera Systems for Scientific, Industrial, and Digital
Photography Applications V, volume 5301, pages 323–333. SPIE, 2004.
[44] Zheng Wang, Ying Xiao, Futian Weng, Xiaojun Li, Danhua Zhu, Fanggen Lu, Xiaowei Liu, Muzhou Hou, and Yu Meng. R-jaunlab: automatic multi-class recognition of jaundice on photos of subjects with region annotation networks. Journal of Digital Imaging, 34(2):337–350, 2021.
[45] Samir S Yadav and Shivajirao M Jadhav. Deep convolutional neural network based medical image classification for disease diagnosis. Journal of Big Data, 6(1):1–18, 2019.
[46] Rui Yan, Fei Ren, Zihao Wang, Lihua Wang, Tong Zhang, Yudong Liu, Xiaosong Rao, Chunhou
Zheng, and Fa Zhang. Breast cancer histopathological image classification using a hybrid deep neural
network. Methods, 173:52–60, 2020.
[47] Jialiang Yang, Jie Ju, Lei Guo, Binbin Ji, Shufang Shi, Zixuan Yang, Songlin Gao, Xu Yuan, Geng
Tian, Yuebin Liang, et al. Prediction of her2-positive breast cancer recurrence and metastasis risk from
histopathological images and clinical information via multimodal deep learning. Computational and
structural biotechnology journal, 20:333–342, 2022.
[48] Peng Yao, Shuwei Shen, Mengjuan Xu, Peng Liu, Fan Zhang, Jinyu Xing, Pengfei Shao, Benjamin
Kaffenberger, and Ronald X Xu. Single model deep learning on imbalanced small datasets for skin
lesion classification. IEEE Transactions on Medical Imaging, 41(5):1242–1254, 2021.