在過去，傳統監督學習高度依賴標記數據，這限制了其應用在各種不同種類的數據集。隨著自監督學習在計算機視覺領域取得了驚人的進展，它顯示了在不依賴標籤的情況下達到與監督學習相媲美的成果的潛力。近年來，這種學習方式及其與數據擴增的結合受到了廣泛關注。然而，序列資料在收集和標注上的挑戰性更大，這也讓序列資料無法在傳統的監督學習有好的結果。

在自監督學習的背景下，並非所有的數據擴增技術都適用於序列資料，尤其是在保持時序依賴性的同時增加數據多樣性的情境中。有鑒於此，我們提出了一種序列資料數據擴增技術，這種技術旨在豐富模型的訓練數據，同時維持數據的時序結構。我們的方法不僅與其他現有方法進行了比較，而且通過一系列實驗證明了其可以增強自監督學習所得到表徵的有效性。

進一步地，為了充分利用這些透過自監督學習獲得的高質量表徵，我們提出了一個新架構。此架構不僅優化了分類器學習表徵的方式，還在多種序列分類任務中證明了其卓越的性能。我們的實驗結果表明，相比傳統自監督學習方法，我們的架構能夠更加高效地利用未標記數據中的信息。

In the past, traditional supervised learning heavily relied on labeled data, limiting its application across various types of datasets. With self-supervised learning making remarkable progress in the field of computer vision, it has demonstrated potential to achieve results comparable to supervised learning without the need for labels. This approach, especially when combined with data augmentation, has gained attention in recent years. However, the challenges of collecting and annotating sequential data have made it difficult to achieve good results with traditional supervised learning in this area.

In the context of self-supervised learning, not all data augmentation techniques are suitable for sequential data, particularly when it comes to maintaining temporal dependencies while increasing data diversity. In light of this, we propose a new data augmentation technique designed specifically for sequential data. This technique aims to enrich the training data while preserving its temporal structure. Our method, compared with other methods, has been proven to enhance the effectiveness of representations obtained through self-supervised learning in a series of experiments.

Furthermore, to fully utilize these high-quality representations obtained through self-supervised learning, we introduce a new framework. This framework not only optimizes the way classifier learns representations but also demonstrates superior performance in various sequential classification tasks. Our experimental results show that our framework can more efficiently utilize information from unlabeled data compared to traditional self-supervised learning methods.

[1] A. Jaiswal, A. R. Babu, M. Z. Zadeh, D. Banerjee, and F. Makedon, “A survey on contrastive selfsupervised learning,” Technologies, vol. 9, no. 1, p. 2, 2020.
[2] X. Liu, F. Zhang, Z. Hou, L. Mian, Z. Wang, J. Zhang, and J. Tang, “Self-supervised learning: Generative or contrastive,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 1, pp. 857–876, 2021.
[3] L. Ericsson, H. Gouk, C. C. Loy, and T. M. Hospedales, “Self-supervised representation learning: Introduction, advances, and challenges,” IEEE Signal Processing Magazine, vol. 39, no. 3, pp. 42–62, 2022.
[4] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in International conference on machine learning, pp. 1597–1607, PMLR, 2020.
[5] X. Chen and K. He, “Exploring simple siamese representation learning,” in Proceedings of the IEEE/ CVF conference on computer vision and pattern recognition, pp. 15750–15758, 2021.
[6] P. Sermanet, C. Lynch, Y. Chebotar, J. Hsu, E. Jang, S. Schaal, S. Levine, and G. Brain, “Timecontrastive networks: Self-supervised learning from video,” in 2018 IEEE international conference
on robotics and automation (ICRA), pp. 1134–1141, IEEE, 2018.
[7] T. Pan, Y. Song, T. Yang, W. Jiang, and W. Liu, “Videomoco: Contrastive video representation learning with temporally adversarial examples,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11205–11214, 2021.
[8] J. Pöppelbaum, G. S. Chadha, and A. Schwung, “Contrastive learning based self-supervised timeseries analysis,” Applied Soft Computing, vol. 117, p. 108397, 2022.
[9] E. Eldele, M. Ragab, Z. Chen, M. Wu, C. K. Kwoh, X. Li, and C. Guan, “Time-series representation learning via temporal and contextual contrasting,” arXiv preprint arXiv:2106.14112, 2021.
[10] Z. Yue, Y. Wang, J. Duan, T. Yang, C. Huang, Y. Tong, and B. Xu, “Ts2vec: Towards universal representation of time series,” in Proceedings of the AAAI Conference on Artificial Intelligence, pp. 8980–8987, 2022.
[11] M. N. Mohsenvand, M. R. Izadi, and P. Maes, “Contrastive representation learning for electroencephalogram classification,” in Machine Learning for Health, pp. 238–253, PMLR, 2020.
[12] P. Shi, W. Ye, and Z. Qin, “Self-supervised pre-training for time series classification,” in 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1–8, IEEE, 2021.
[13] Y. Lin, X. Guo, and Y. Lu, “Self-supervised video representation learning with meta-contrastive network,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8239–8249, 2021.
[14] K. Hu, J. Shao, Y. Liu, B. Raj, M. Savvides, and Z. Shen, “Contrast and order representations for video self-supervised learning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7939–7949, 2021.
[15] Y. Bengio, A. Courville, and P. Vincent, “Representation learning: A review and new perspectives,”IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 8, pp. 1798–1828, 2013.
[16] Q. Wen, L. Sun, F. Yang, X. Song, J. Gao, X. Wang, and H. Xu, “Time series data augmentation for deep learning: A survey,” arXiv preprint arXiv:2002.12478, 2020.
[17] C. Oh, S. Han, and J. Jeong, “Time-series data augmentation based on interpolation,” Procedia Computer Science, vol. 175, pp. 64–71, 2020.
[18] B. Liu, Z. Zhang, and R. Cui, “Efficient time series augmentation methods,” in 2020 13th international congress on image and signal processing, BioMedical engineering and informatics (CISP-BMEI),
pp. 1004–1009, IEEE, 2020.
[19] C. Shorten and T. M. Khoshgoftaar, “A survey on image data augmentation for deep learning,” Journal of big data, vol. 6, no. 1, pp. 1–48, 2019.
[20] L. Perez and J. Wang, “The effectiveness of data augmentation in image classification using deep learning,” arXiv preprint arXiv:1712.04621, 2017.
[21] A. Mikołajczyk and M. Grochowski, “Data augmentation for improving deep learning in image classification problem,” in 2018 international interdisciplinary PhD workshop (IIPhDW), pp. 117–122, IEEE, 2018.
[22] Q. Ma, Z. Zheng, J. Zheng, S. Li, W. Zhuang, and G. W. Cottrell, “Joint-label learning by dual augmentation for time series classification,” in Proceedings of the AAAI Conference on Artificial Intelligence, pp. 8847–8855, 2021.
[23] Z. Wang, W. Yan, and T. Oates, “Time series classification from scratch with deep neural networks: A strong baseline,” in 2017 International joint conference on neural networks (IJCNN), pp. 1578–1585, IEEE, 2017