StatioCL: Contrastive Learning for Time Series via Non-Stationary and Temporal Contrast

Abstract

Contrastive learning (CL) has emerged as a promising approach for unsupervised representation learning in time series data by embed- ding similar input pairs closely while distancing dissimilar ones. However, existing CL methods often introduce false negative pairs (FNPs) by randomly selecting distinct segments as dissimilar pairs. Training with these FNPs can lead to erroneous representation learn- ing, reduced model performance, and overall inefficiency. To mit- igate these issues, in the context of time series, we systematically define and categorize FNPs into semantic false negative pairs and temporal false negative pairs for the first time: the former arising from overlooking similarities in label categories and the latter from neglecting temporal proximity in time series data. Moreover, we introduce a novel CL framework, StatioCL, by utilizing the intrinsic characteristics, non-stationarity and temporal dependency to mitigate both types of FNPs without additional label information. By inter- preting and differentiating non-stationary states, which reflect the correlation between trends or temporal dynamics with underlying data patterns, StatioCL effectively captures the semantic charac- teristics and eliminates semantic FNPs. Simultaneously, StatioCL establishes fine-grained similarity levels based on temporal depen- dencies to capture varying temporal proximity between segments and to mitigate temporal FNPs. Our approach rectifies the inaccu- racies in learned representations and captures time series’ intrinsic semantic and temporal characteristics. Evaluated on 34 real-world benchmark datasets for time series classification, StatioCL demon- strates a substantial improvement over state-of-the-art CL methods, achieving a 2.9% increase in Recall and a 19.2% reduction in FNPs. Most importantly, STC also shows enhanced data efficiency and robustness against label scarcity.