Recommendation systems are offering users the ability to explore their interests and discover new things that are already popular in e-commerce websites and on online multimedia services. They will become interesting for food and recipe since they give users suggestions to select from the retrieved cooking based on food images or ingredients while possibly including personal preferences. This research proposes a recipe recommendation system that is based on the available ingredients with an additional feature to replace similar ingredients in the retrieved recipe. The system takes the images of the ingredients as input and recognizes them through a convolutional neural network (CNN). This research adopts Tensorflow Object Detection API to train the objects and a fine-tuned Faster R-CNN model with a training dataset for twelve ingredients. The detected ingredients can, in turn, be used to recommend recipes, and for each missing ingredient, our system finds the most suitable substitute ingredient that has large co-occurrence relations with the main ingredients with a higher frequency.

[1] Shota Sasano, Xian-Hua Han, and Y.-W. Chen, "Food Recognition by Combined Bags of Color Features and Texture Features " presented at the International Congress on Image and Signal Processing, BioMedical Engineering and Informatics(CISP-BMEI ), 2016.

[2] A. Yongsheng, C. yu, C. Jingjing, N. Chong-Wah, J. Jia, L. Huanbo, et al., "PIC2DISH:ACustomizedCookingAssistantSystem " presented at the MM'17, Mountain View, CA, USA, 2017.

[3] A. Willis, E. Lin, and B. Zhang, "Forage : Optimizing Food Use With Machine Learning Generated Recipes," 2017.

[4] Fang-FeiKuo, Cheng-TeLi, Man-KwanShan, and Suh-YinLee, "Intelligent menu planning: Recommending set of recipes by ingredients.," presented at the In Proceedings of the ACM multimedia, workshop on Multimedia for cooking and eating activities.ACM, 2012

[5] Chun-Yuen Teng, yu-Ru Lin, and L. A. Adamic, "Recipe recommendation using ingredient networks," arXiv, vol. 3, 2012.

[6] Mezgec, S.; Koroušić Seljak, and B. NutriNet, "A Deep Learning Food and Drink Image Recognition System for Dietary Assessment.," Nutrients., vol. 9, 2017.

[7] Yasuhiro Hayashi, Keisuke Doman, Ichiro Ide, Daisuke Deguchi, and Hiroshi Murase, "Automatic authoring of a domestic cooking video based on the description of cooking instructions," presented at the In Proceedings of the 5th international workshop on Multimedia for cooking & eating activities.ACM, 2013.

[8] KeisukeDoman, ChengYingKuai, TomokazuTakahashi, IchiroIde, and H. Murase, "Smart Video CooKing: a multimedia cooking recipe browsing Video " presented at the MM, Mountain View, CA, USA, 2017.

[9] Angelica Willis , Elbert Lin , and Brian Zhang, " Optimizing Food Use With Machine Learning Generated Recipes," 2017.

[10] Mayumi Ueda, Syungo Asanuma, Yusuke Miyawaki, and S. Nakajima, "Recipe Recommendation Method by Considering the User’s Preference and Ingredient Quantity of Target Recipe " Proceedings of the International MultiConference of Engineers and Computer Scientists vol. 1, 2014.

[11] Naoki SHINO, Ryosuke YAMANISHI, and J. FUKUMOTO, "Recommendation System for Alternative-ingredients Based on Co-occurrence Relation on Recipe Database and the Ingredient Category," presented at the AI International Congress on Advanced Applied Informatics, Japan, 2016.

[12] Yoshiyuki Kawano and K. Yanai, "Real-timeMobileFoodRecognitionSystem," presented at the IEEE Conference on Computer Vision and Pattern Recognition Workshops Tokyo 1-5-1 Chofugaoka, Chofu-shi, Tokyo, 182-8585 Japan 2013.

[13] Minami WAZUMI, Xian-Hua HAN, Danni AI, and Yen-Wei CHEN, "Auto-Recognition of Food Images Using SPIN Feature for Food-Log System."

[14] Micael Carvalho, Remi Cadene, David Picard, Laure Soulier, and M. Cord, "Images & Recipes: Retrieval in the cooking context," SIGIR, 2018.

[15] M.P Nerkar, Preety Vajpayee, Varsha Ganjoo, Vaibhav Suryawanshi, and P. Sonawane, "RECIPE RETRIEVAL USING INGREDIENT RECOGNITION " International Journal of Advance Engineering and Research Development Special Issue on Recent Trends in Data Engineering vol. 4, 2017.

[16] Mart´ın Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, et al., "Tensorflow: a system for large-scale machine learning.," OSDI, , vol. 16, , pp. 265–283, , 2016,.

[17] K. He, X. Zhang, S. Ren, and J. Sun, "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification," presented at the In Proceedings of the IEEE international conference on computer vision 2015.

[18] Derry Alamsyah and M. Fachrurrozi, "Faster R-CNN with Inception V2 for Fingertip Detection in Homogenous Background Image," Journal of Physics: Conference Series, vol. 1196 p. 012017, 2019.

[19] D. P. Kingma and J. A. Ba, " A method for stochastic optimization," arXiv preprint arXiv vol. 1412, p. 6980 2014.

[20] S. Ioffe, Szegedy, and C. B. normalization, "Accelerating deep network training by reducing internal covariate shift," presented at the International conference on machine learning 2015.

[21] S. Ren, K. He, R. Girshick, and J. Sun, " Towards real-time object detection with region proposal networks," 2015.

[22] Md Zahangir Alom, Tarek M Taha, Christopher Yakopcic, Stefan Westberg, Mahmudul Hasan, Brian C Van Esesn, et al., "The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches.," arXiv preprint arXiv, vol. 15, p. 1803.01164, 2018.

[23] P. P. Angara., "Towards a Deeper Understanding of Current Conversational Frameworks through the Design and Development of a Cognitive Agent," 2018.