Nowadays, sound generation has become one of the directions in multimedia research. People are searching new methods to generate interesting sounds. Therefore, in this research we address the problem of making multimedia system for sound production from a given image through two different approaches including color based segmentation and object detection. We use jQuery audiosynth.js to generate the sound of notes in the color mapping sonification system, while
YOLOv3 is used in object detection for sonification. The system will play suitable sound from a local database that matches with the object detected by the system. We choose to implement the systems in a web-based platform using JavaScript associated by node.js with modern web browsers that support Web Audio APIs. In this case, Mozilla Firefox and Google Chrome have already supported this feature. In addition, the web-based sonification system can still be used in different platforms such Android and Windows because it doesn’t depend on the chosen platform. The purpose of the research is to generate a pleasing sound for an image through two approaches presented. A user study was performed to evaluate the systems by using online programs and questionnaires. The results indicate that most of the users agree that the sonification systems
presented were interesting and unique.

Nowadays, sound generation has become one of the directions in multimedia research. People are searching new methods to generate interesting sounds. Therefore, in this research we address the problem of making multimedia system for sound production from a given image through two different approaches including color based segmentation and object detection. We use jQuery audiosynth.js to generate the sound of notes in the color mapping sonification system, while
YOLOv3 is used in object detection for sonification. The system will play suitable sound from a local database that matches with the object detected by the system. We choose to implement the systems in a web-based platform using JavaScript associated by node.js with modern web browsers that support Web Audio APIs. In this case, Mozilla Firefox and Google Chrome have already supported this feature. In addition, the web-based sonification system can still be used in different platforms such Android and Windows because it doesn’t depend on the chosen platform. The purpose of the research is to generate a pleasing sound for an image through two approaches presented. A user study was performed to evaluate the systems by using online programs and questionnaires. The results indicate that most of the users agree that the sonification systems
presented were interesting and unique.

[1] S. Cavaco, J. T. Henriques, M. Mengucci, N. Correia and F. Medeiros, "Color Sonification for the Visually Impaired," Procedia Technology 9, pp. 1048-1057, 2013.
[2] T. Wörtwein, B. Schauerte, K. E. Mueller and R. Stiefelhagen, "Interactive Web-based Image Sonification for the Blind," ICMI '15 Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 375-376 , 2015.
[3] T. Wörtwein , B. Schauerte, K. Müller and R. Stiefelhagen, "Mobile Interactive Image Sonification for the Blind," International Conference on Computers Helping People with Special Needs, vol. 9758, pp. 212-219, 2016.
[4] J. Redmon, S. Divvala, R. Girshick and A. Farhadi, "You Only Look Once : Unified, Real-Time Object Detection," Computer Vision and Pattern Recognition (CVPR), pp. ISSN 1063-6919, 2016 IEEE.
[5] C. M. Fernandes, D. Migotina and A. C. Rosa, "Sonification of Sleep EEG," ARTECH2017 Proceedings of the 8th International Conference on Digital Arts, pp. 161-164 , 2017.
[6] D. Migotina and A. Rosa, "Segmentation of Sleep EEG Signal by Optimal Thresholds," Proceedings of the BioMed2012, pp. 114-121, 2012.
[7] R. F. Cohen, V. Haven, J. A. Lanzoni and A. Meacham, "Using an Audio Interface to Assist Users Who are Visually Impaired with Steering Tasks," Assets '06 Proceedings of the 8th international ACM SIGACCESS conference on Computers and Accessibility, pp. 119-124 , 2006.
[8] S. Hasegawa, S. Ishijima, F. Kato, H. Mitake and M. Sato, "Realtime Sonification of the Center of Gravity for Skiing," AH '12 Proceedings of the 3rd Augmented Human International Conference, 2012.
[9] S. Serafin and G. Serafin, "Sound Design to Enhance Presence in Photorealistic Virtual Reality," Proceedings of ICAD 04-Tenth Meeting of the International Conference on Auditory Display, 2004.
[10] T. Yoshida, K. M. Kitani, H. Koike, S. Belongie and K. Sclei, "EdgeSonic: Image Feature Sonification for the Visually Impaired," Proceedings of the 2nd Augmented Human International Conference, 2011.
[11] A. C. Marruffo, "Automatic Sonification of Video Sequences Through Object Detection and Physical Modelling," Aalborg University Copenhagen, Denmark, 2017.
[12] M. Banf and V. Blanz, "Sonification of Images for the Visually Impaired using a Muti-Level Approach," Proceedings of the 4th Augmented Human International Conference, pp. 162-169, 2013.
[13] "Node.js," [Online]. Available: https://nodejs.org/.
[14] K. Lei, Y. Ma and Z. Tan, "Performance Comparison and Evaluation of Web Development Technologies in PHP, Python and Node.js," IEEE 17th International Conference on Computational Science and Engineering, 2014.
[15] J. Redmon and . A. Farhadi, "YOLOv3: An Incremental Improvement," 2018.
[16] T.-Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan and S. Belongie, "Feature Pyramid Networks for Object Detection," Proceedings of the IEEE Conference on Computer Vision, p. 2117–2125, 2017.
[17] "YOLO: Real-Time Object Detection," [Online]. Available: https://pjreddie.com/darknet/yolo/.
[18] J. Redmon, S. Divvala, R. Girshick and A. Farhadi, "You Only Look Once: Unified, Real-Time Object Detection".