域適應通常用於解決計算機視覺領域中的域轉移問題。領域適應是具有挑戰性任務，同時對於時間序列數據，任務會變得更加複雜。考慮到數據的依賴性以及時間戳的關係，時間序列數據會更有複雜性。目前大多數方法基於適用於非時間序列數據的域適應的解決方案來解決這個問題。但這些方法仍然不足，因為時間戳之間的依賴關係並沒有被充分利用或考慮。本文提出了一種時間注意機制，能夠更好地捕捉歷史數據中的潛在模式。這也使模型能夠學習重要的來源和目標域特徵。之後，我們利用時間分佈匹配機制來關聯來源域和目標域。對兩個真實世界數據集的實驗驗證了我們提出的模型實現了最先進的結果。

Domain adaptation is commonly used to address the domain shift problem in the computer vision field. Domain adaptation alone is challenging, but the task becomes even more complicated concerning time series data. Time series data involves added complexity considering the dependencies of data with considerations on timestamps. Most approaches address this problem by borrowing solutions from domain adaptation that works with non-time series data. This is insufficient since the dependency between timestamps is not fully utilized or considered. This paper proposes a temporal attention mechanism to better capture latent patterns in historical data. This also enables the model to learn important source and target domain features. Afterwhich we leveraged a temporal distribution matching mechanism to associate between source and target domain. Experiments on two real-world datasets verify that our proposed model achieves state-of-the-art results.

References
[1] L. Chen, H. Zhang, J. Xiao, W. Liu, and S.-F. Chang, “Zero-Shot Visual Recognition Using SemanticsPreserving Adversarial Embedding Networks,” in Intl. Conf. on Computer Vision and Pattern Recognition (CVPR), (Salt Lake City, UT), pp. 1043–1052, June 2018.
[2] B. Gong, K. Grauman, and F. Sha, “Learning Kernels for Unsupervised Domain Adaptation with
Applications to Visual Object Recognition,” in Intl. Journal of Computer Vision (IJCV), vol. 109,
pp. 3–27, Apr. 2014.
[3] H. H. Zhuo and Q. Yang, “Action-Model Acquisition for Planning via Transfer Learning,” in Artificial
Intelligence, vol. 212, pp. 80–103, July 2014.
[4] T. T. Nguyen, T. Silander, Z. Li, and T.-Y. Leong, “Scalable Transfer Learning in Heterogeneous,
Dynamic Environments,” in Artificial Intelligence, vol. 247, pp. 70–94, June 2017.
[5] P. Zhao, S. C. H. Hoi, J. Wang, and B. Li, “Online Transfer Learning,” in Research Collection School
of Information Systems, vol. 216, pp. 76–102, 2014.
[6] F. Liu, G. Zhang, and J. Lu, “Heterogeneous Domain Adaptation: An Unsupervised Approach,” in
Trans. on Neural Networks and Learning Systems, vol. 31(12), pp. 5588–5602, Mar. 2020.
[7] M. Ghifary, W. B. Kleijn, and M. Zhang, “Domain Adaptive Neural Networks for Object Recognition,” in Pacific Rim Intl. Conf. on Artificial Intelligence (PRICAI), (Gold Coast, Australia), Dec.
2014.
[8] S. J. Pan, I. W. Tsang, J. T. Kwok, and Q. Yang, “Domain Adaptation via Transfer Component Analysis,” in Trans. on Neural Networks, vol. 22(2), pp. 199–210, Feb. 2011.
[9] B. Fernando, A. Habrard, M. Sebban, and T. Tuytelaars, “Unsupervised Visual Domain Adaptation
Using Subspace Alignment,” in Intl. Conf. on Computer Vision (ICCV), (Sydney, Australia), pp. 2960–
2967, Dec. 2013.
[10] B. Sun and K. Saenko, “Subspace Distribution Alignment for Unsupervised Domain Adaptation,” in
British Machine Vision Conf. (BMVC), vol. 24, p. 10, 2015.
[11] B. Gong, Y. Shi, F. Sha, and K. Grauman, “Geodesic Flow Kernel for Unsupervised Domain Adaptation,” in Intl. Conf. on Computer Vision and Pattern Recognition (CVPR), (Providence, RI), pp. 2066–
2073, June 2012.
[12] E. Tzeng, J. Hoffman, N. Zhang, K. Saenko, and T. Darrell, “Deep Domain Confusion: Maximizing
for Domain Invariance,” in arXiv preprint arXiv:1412.3474, Dec. 2014.
[13] B. Sun and K. Saenko, “Deep CORAL: Correlation Alignment for Deep Domain Adaptation,” in
Europ. Conf. on Computer Vision (ECCV), vol. 9915, pp. 443–450, Nov. 2016.
[14] F. Zhuang, X. Cheng, P. Luo, S. J. Pan, and Q. He, “Supervised Representation Learning with Double Encoding-Layer Autoencoder for Transfer Learning,” in ACM Trans. on Intelligent Systems and
Technology (TIST), vol. 9(2), Article 16, pp. 1–17, Mar. 2018.
[15] J. Jiang, X. Wang, M. Long, and J. Wang, “Resource Efficient Domain Adaptation,” in ACM Intl.
Conf. on Multimedia (ACMMM), pp. 2220–2228, Oct. 2020.
[16] N. Courty, R. Flamary, D. Tuia, and A. Rakotomamonjy, “Optimal Transport for Domain Adaptation,”
in Trans. on Pattern Analysis and Machine Intelligence, vol. 39(9), pp. 1853–1865, Oct. 2016.
[17] M. Baktashmotlagh, M. Harandi, and M. Salzmann, “Distribution-Matching Embedding for Visual
Domain Adaptation,” in Journal of Machine Learning Research (JMLR), vol. 17, pp. 1–30, July 2016.
[18] R. Cai, J. Chen, Z. Li, W. Chen, K. Zhang, J. Ye, Z. Li, X. Yang, and Z. Zhang, “Time Series Domain Adaptation via Sparse Associative Structure Alignment,” in Association for the Advancement of
Artificial Intelligence (AAAI), vol. 216, pp. 76–102, 2014.
[19] V. Flunkert, D. Salinas, and J. Gasthaus, “DeepAR: Probabilistic Forecasting with Autoregressive
Recurrent Networks,” International Journal of Forecasting, vol. 36, pp. 1181–1191, 2020. arXiv:
1704.04110.
[20] B. N. Oreshkin, N. Chapados, D. Carpov, and Y. Bengio, “N-BEATS: Neural basis expansion analysis for interpretable time series forecasting,” International Conference on Learning Representations,
p. 31, 2020.
[21] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin,
“Attention Is All You Need,” in Advances in neural information processing systems, pp. 5998–6008,
2017.
[22] S. Wu, X. Xiao, Q. Ding, P. Zhao, Y. Wei, and J. Huang, “Adversarial Sparse Transformer for Time
Series Forecasting,” Advances in Neural Information Processing Systems, vol. 33, p. 11, 2020.
[23] H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, “Informer: Beyond Efficient
Transformer for Long Sequence Time-Series Forecasting,” in Proceedings of the AAAI Conference on
Artificial Intelligence, 2021. arXiv: 2012.07436.
[24] R. Wang, D. Maddix, C. Faloutsos, Y. Wang, and R. Yu, “Bridging Physics-based and Data-driven
modeling for Learning Dynamical Systems,” in Learning for Dynamics and Control, pp. 385–398,
2021.
[25] G. Wilson and D. J. Cook, “A Survey of Unsupervised Deep Domain Adaptation,” arXiv:1812.02849
[cs, stat], Feb. 2020. arXiv: 1812.02849.
[26] A. Ramponi and B. Plank, “Neural Unsupervised Domain Adaptation in NLP—A Survey,”
arXiv:2006.00632 [cs], May 2020. arXiv: 2006.00632.
[27] C. Cortes and M. Mohri, “Domain Adaptation in Regression,” in Algorithmic Learning Theory
(J. Kivinen, C. Szepesvári, E. Ukkonen, and T. Zeugmann, eds.), vol. 6925, pp. 308–323, Berlin,
Heidelberg: Springer Berlin Heidelberg, 2011. Series Title: Lecture Notes in Computer Science.
[28] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and
V. Lempitsky, “Domain-Adversarial Training of Neural Networks,” The journal of machine learning research, vol. 17, pp. 2096–2030, 2016.
[29] D. Wright and I. Augenstein, “Transformer Based Multi-Source Domain Adaptation,” in Proceedings
of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), (Online),
pp. 7963–7974, Association for Computational Linguistics, 2020.
[30] W.-Y. Chen, Y.-C. Liu, Z. Kira, Y.-C. F. Wang, and J.-B. Huang, “A Closer Look at Few-shot Classification,” arXiv:1904.04232 [cs], Jan. 2020. arXiv: 1904.04232.
[31] R. Cai, J. Chen, Z. Li, W. Chen, K. Zhang, J. Ye, Z. Li, X. Yang, and Z. Zhang, “Time Series Domain
Adaptation via Sparse Associative Structure Alignment,” Proceedings of the AAAI Conference on
Artificial Intelligence, vol. 35, pp. 6859–6867, 2021.
[32] H. Hu, M. Tang, and C. Bai, “DATSING: Data Augmented Time Series Forecasting with Adversarial
Domain Adaptation,” in Proceedings of the 29th ACM International Conference on Information &
Knowledge Management, (Virtual Event Ireland), pp. 2061–2064, ACM, Oct. 2020.
[33] K. Bousmalis, G. Trigeorgis, N. Silberman, D. Krishnan, and D. Erhan, “Domain Separation Networks,” vol. 29, pp. 345–351, 2016.
[34] G. Shi, C. Feng, L. Huang, B. Zhang, H. Ji, L. Liao, and H. Huang, “Genre Separation Network with
Adversarial Training for Cross-genre Relation Extraction,” in Proceedings of the 2018 Conference on
Empirical Methods in Natural Language Processing, (Brussels, Belgium), pp. 1018–1023, Association for Computational Linguistics, 2018.
[35] G. Lai, W.-C. Chang, Y. Yang, and H. Liu, “Modeling long- and short-term temporal patterns with
deep neural networks,” SIGIR, pp. 95–104, 2018.
[36] Y. Qin, D. Song, H. Cheng, W. Cheng, G. Jiang, and G. W. Cottrell, “A dual-stage attention-based
recurrent neural network for time series prediction,” in IJCAI’17, pp. 2627–2633, 2017.
[37] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and
V. Lempitsky, “Domain-adversarial training of neural networks,” JMLR, vol. 17, pp. 59:1–59:35,
2016.
[38] B. Sun and K. Saenko, “Deep coral: Correlation alignment for deep domain adaptation,” in ECCV,
2016.
[39] R. E. Schapire, “The boosting approach to machine learning: An overview,” Nonlinear estimation
and classification, pp. 149–171, 2003.
[40] A. Gretton, B. K. Sriperumbudur, D. Sejdinovic, H. Strathmann, S. Balakrishnan, M. Pontil, and
K. Fukumizu, “Optimal kernel choice for large-scale two-sample tests,” in NeurIPS, 2012.
[41] Y. Zheng, X. Yi, M. Li, R. Li, Z. Shan, E. Chang, and T. Li, “Forecasting fine-grained air quality
based on big data,” in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge
Discovery and Data Mining, pp. 2267–2276, ACM, 2015.
[42] P. R. d. O. da Costa, A. Akçay, Y. Zhang, and U. Kaymak, “Remaining useful lifetime prediction via
deep domain adaptation,” Reliability Engineering & System Safety, vol. 195, p. 106682, 2020.
[43] Y. Ganin and V. Lempitsky, “Unsupervised domain adaptation by backpropagation,” in International
conference on machine learning, pp. 1180–1189, PMLR, 2015.
[44] E. Tzeng, J. Hoffman, N. Zhang, K. Saenko, and T. Darrell, “Deep domain confusion: Maximizing
for domain invariance,” arXiv preprint arXiv:1412.3474, 2014.
[45] S. Purushotham, W. Carvalho, T. Nilanon, and Y. Liu, “Variational recurrent adversarial deep domain adaptation,” in 5th International Conference on Learning Representations, ICLR 2017, Toulon,
France, April 24-26, 2017, Conference Track Proceedings, OpenReview.net, 2017.
[46] J. Chung, K. Kastner, L. Dinh, K. Goel, A. C. Courville, and Y. Bengio, “A recurrent latent variable
model for sequential data,” in Advances in neural information processing systems, pp. 2980–2988,
2015.
[47] R. Cai, J. Chen, Z. Li, W. Chen, K. Zhang, J. Ye, Z. Li, X. Yang, and Z. Zhang, “Time series domain
adaptation via sparse associative structure alignment,” ArXiv, vol. abs/2205.03554, 2021.