本研究旨在解決消費者在不同電商平台上進行比價時所面臨的商品匹配問題。各電商平台的渠道多樣化，包括原廠、代理商和個人賣家上架重複商品，且命名慣例不同，導致相同商品的名稱和圖片有所差異。本論文提出了一種基於文字和圖像的深度學習模型，用於跨平台商品匹配。該模型利用產品名稱和照片作為實體提及，目的是在降低計算量的同時提升匹配效能。我們的方法包括兩個階段的深度學習網路架構：首先，通過三元組網路訓練的限縮 (Block) 網路進行初步過濾，然後由孿生網路訓練的匹配分類 (Match) 網路進一步分類，以確定商品提及是否吻合。我們還設計了一個電商競品採集與標註平台，用於提供產品搜尋對比和匹配商品標註，作為訓練匹配模型所需的正負樣本依據。實驗結果顯示，透過持上述訓練所持續微調語意編碼器，本兩階模型能顯著提升商品匹配的精確性和效率，即使在計算資源有限的情況下也能實現高效的商品匹配，特別適用於需要快速更新和適應不斷變化市場的電商平台。此外，我們還提出了一個商品比對塑模與佈署框架，統合了電商競業之商品匹配所需的標註、模型建模、線上查詢以及匹配佈署等功能。

This study addresses the issue of product matching for consumers comparing prices or services across different e-commerce platforms. The diverse channels on e-commerce platforms, including original manufacturers, agents, and individual sellers, lead to variations in product listings. Additionally, differing naming conventions across platforms result in discrepancies in product names and images for the same items. This paper proposes a deep learning-based model that leverages both text and images to match products across platforms. The model uses product names and images as entity mentions to improve matching accuracy while minimizing computational complexity. Our method involves a two-stage deep learning network architecture: initially, a Block network trained with triplet loss filters out less similar products, followed by a Match network trained with a Siamese network to classify whether product mentions match. We also developed a platform for collecting and annotating competitive products across e-commerce platforms, providing the necessary positive and negative samples for training the two-stage model. Experimental results show that continuously fine-tuning the semantic encoder through the aforementioned training significantly improves the accuracy and efficiency of product matching. The model remains effective even with limited computational resources, making it particularly suitable for e-commerce platforms that require rapid updates and adaptation to an ever-changing market. Additionally, we introduce a product comparison modeling and deployment framework, integrating the annotation, model building, online querying, and matching deployment functionalities required for competitive product matching in e-commerce.

[1] J. Pollack, H. Köpcke, and E. Rahm, “Intermediate Fusion for Multimodal Product Matching,” 2024.
[2] "Shopee - Price Match Guarantee." https://www.kaggle.com/competitions/shopee-product-matching (accessed.
[3] W. Zijia, L. Ye, and Z. Zhongkai, "BERT-based knowledge extraction method of unstructured domain text," arXiv preprint arXiv:2103.00728, 2021.
[4] P. M. Alves, P. Geraldo Filho, and V. P. Gonçalves, "Leveraging BERT's Power to Classify TTP from Unstructured Text," in 2022 Workshop on Communication Networks and Power Systems (WCNPS), 2022: IEEE, pp. 1-7.
[5] D. Zhu, M. A. Hedderich, F. Zhai, D. I. Adelani, and D. Klakow, "Is BERT robust to label noise? A study on learning with noisy labels in text classification," arXiv preprint arXiv:2204.09371, 2022.
[6] S. Mudgal et al., "Deep learning for entity matching: A design space exploration," in Proceedings of the 2018 International Conference on Management of Data, 2018, pp. 19-34.
[7] J. Wang, Y. Li, and W. Hirota, "Machamp: A generalized entity matching benchmark," in Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 4633-4642.
[8] A. Naeim abadi, M. T. Nayeem, and D. Rafiei, "Product Entity Matching via Tabular Data," in Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, 2023, pp. 4215-4219.
[9] R. Peeters, R. C. Der, and C. Bizer, "WDC Products: A Multi-Dimensional Entity Matching Benchmark," arXiv preprint arXiv:2301.09521, 2023.
[10] D. Bahdanau, K. Cho, and Y. Bengio, "Neural machine translation by jointly learning to align and translate," arXiv preprint arXiv:1409.0473, 2014.
[11] A. Vaswani et al., "Attention is all you need," Advances in neural information processing systems, vol. 30, 2017.
[12] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” ArXiv Prepr. ArXiv181004805, 2018.
[13] REIMERS, Nils; GUREVYCH, Iryna. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084, 2019.
[14] K. Shah, S. Kopru, and J. D. Ruvini, "Neural network based extreme classification and similarity models for product matching," in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers), 2018, pp. 8-15.
[15] M. Almagro, D. Jiménez, D. Ortego, E. Almazán, and E. Martínez, "Block-SCL: Blocking Matters for Supervised Contrastive Learning in Product Matching," arXiv preprint arXiv:2207.02008, 2022.
[16] R. Peeters and C. Bizer, "Supervised contrastive learning for product matching," in Companion Proceedings of the Web Conference 2022, 2022, pp. 248-251.
[17] U. Ahsan et al., "Visually Compatible Home Decor Recommendations Using Object Detection and Product Matching," in 2021 International Conference on Computational Science and Computational Intelligence (CSCI), 2021: IEEE, pp. 214-220.
[18] L. Huang, W. Shao, F. Wang, W. Xie, and K.-C. Wong, "Metric Learning Based Vision Transformer for Product Matching," in Neural Information Processing: 28th International Conference, ICONIP 2021, Sanur, Bali, Indonesia, December 8–12, 2021, Proceedings, Part I 28, 2021: Springer, pp. 3-13.
[19] X. Wu, A. Magnani, S. Chaidaroon, A. Puthenputhussery, C. Liao, and Y. Fang, "A Multi-task Learning Framework for Product Ranking with BERT," in Proceedings of the ACM Web Conference 2022, 2022, pp. 493-501.
[20] A. Dosovitskiy et al., "An image is worth 16x16 words: Transformers for image recognition at scale," arXiv preprint arXiv:2010.11929, 2020.
[21] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, "End-to-end object detection with transformers," in European conference on computer vision, 2020: Springer, pp. 213-229.
[22] A. Radford et al., "Learning transferable visual models from natural language supervision," in International conference on machine learning, 2021: PMLR, pp. 8748-8763.
[23] Q. Sun, Y. Fang, L. Wu, X. Wang, and Y. Cao, "Eva-clip: Improved training techniques for clip at scale," arXiv preprint arXiv:2303.15389, 2023.
[24] M. Wilke and E. Rahm, "Towards Multi-Modal Entity Resolution for Product Matching," in GvDB, 2021.
[25] K. Hari Krishnan, “E-commerce Product Similarity Match Detection using Product Text and Images,” Dublin, National College of Ireland, 2021.
[26] M. Sortur, P. Rajpoot, Manjunath, Subhanandh, and H. C. Rao, "E-Commerce Product Matching at Internet Scale," in 2022 13th International Conference on E-business, Management and Economics, 2022, pp. 45-51.
[27] M. Tsimpoukelli, J. L. Menick, S. Cabi, S. Eslami, O. Vinyals, and F. Hill, "Multimodal few-shot learning with frozen language models," Advances in Neural Information Processing Systems, vol. 34, pp. 200-212, 2021.
[28] J. Li, D. Li, S. Savarese, and S. Hoi, "Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models," arXiv preprint arXiv:2301.12597, 2023.
[29] J. Pollack, H. Köpcke, and E. Rahm, "Intermediate Fusion for Multimodal Product Matching," 2024.