TLDR: A new metric, Signal Dice Similarity Coefficient (SDSC), is proposed for self-supervised learning in time series. Unlike traditional methods that rely on amplitude-sensitive errors like MSE, SDSC focuses on structural agreement between signals. It is shown to improve performance in forecasting and classification tasks, especially in low-resource settings, by enhancing the semantic quality of signal representations. A hybrid loss combining SDSC and MSE is also introduced for balanced performance.
Self-Supervised Learning (SSL) has become a powerful technique for extracting meaningful information from unlabeled data across various fields, including computer vision and natural language processing. In the realm of time series analysis, SSL is also crucial for learning representations that can be used for tasks like forecasting and classification. However, a common challenge in time series SSL is the reliance on traditional distance-based metrics, such as Mean Squared Error (MSE), to evaluate how well a signal is reconstructed.
While MSE is effective at minimizing numerical errors, it has significant limitations when it comes to capturing the true semantic meaning or structural consistency of time series signals. MSE is highly sensitive to amplitude variations, meaning a small change in signal strength can lead to a large error score, even if the overall shape remains similar. More critically, MSE is invariant to waveform polarity (it doesn’t care if a signal is inverted) and can be unbounded, making interpretation difficult. This can lead to situations where semantically very different signals (e.g., a phase-inverted signal, a scaled signal, or even a flat zero signal) might yield similar MSE values, misleadingly suggesting a good reconstruction despite significant structural or semantic deviations.
To address these critical issues, researchers Jeyoung Lee and Hochul Kang from The Catholic University of Korea have proposed a novel metric called the Signal Dice Similarity Coefficient (SDSC). This new metric is designed to be ‘structure-aware,’ focusing on the agreement of temporal signal structures rather than just amplitude differences. SDSC is inspired by the Dice Similarity Coefficient (DSC), a metric widely used in image processing for semantic segmentation, which quantifies the overlap between two sets.
SDSC extends this concept to continuous, signed time-series data. Instead of measuring set overlap, it quantifies structural alignment by looking at the intersection of signed amplitudes at each point in time. This approach allows SDSC to capture important features like polarity, phase, and overall alignment, which are often overlooked by distance-based metrics. A key advantage of SDSC is that its score is bounded between 0 and 1, where 1 indicates perfect structural agreement, making it much easier to interpret and compare across different signals or models. It also demonstrates increased robustness to amplitude scaling and polarity shifts, providing a more accurate assessment of signal quality.
Although SDSC is primarily a metric, it can also be used as a loss function for training machine learning models. This is achieved by subtracting the SDSC score from 1 (so a perfect score of 1 becomes a loss of 0). To enable stable gradient-based optimization, a smooth, differentiable approximation of the Heaviside step function is incorporated into its calculation, typically using a sigmoid-based formulation. This ensures that the training process remains stable even with the inherent discontinuities of the original Heaviside function.
Recognizing that while SDSC excels at capturing structure, it might overlook critical amplitude information in certain tasks, the researchers also propose a hybrid loss function. This hybrid approach combines the structural awareness of SDSC with the amplitude sensitivity of MSE. By carefully weighting these two components, the hybrid loss promotes reconstructions that are both structurally aligned and numerically precise, offering a balanced approach to signal representation learning.
Experiments conducted using the SimMTM baseline model demonstrated the effectiveness of SDSC. In pre-training, models trained with SDSC loss achieved higher SDSC scores, indicating better structural alignment, even if their MSE values were higher. Conversely, MSE-trained models showed a weak correlation with SDSC, suggesting they only partially capture structural features incidentally. For downstream tasks, SDSC-based pre-training showed comparable or even improved performance over MSE, particularly in in-domain and low-resource scenarios for both forecasting and classification. For instance, in classification tasks with frozen encoders, SDSC models consistently outperformed others in settings where waveform structure was crucial, such as the gesture dataset. The hybrid loss consistently provided stable and balanced performance across various datasets.
Also Read:
- Adaptive Margin in Time Series Contrastive Learning: A Closer Look at Clustering vs. Classification Performance
- Boosting Phenotype Prediction: Self-Supervised Learning for Gene Expression
In conclusion, the introduction of the Signal Dice Similarity Coefficient (SDSC) marks a significant step forward in time series representation learning. It challenges the conventional reliance on amplitude-centric metrics like MSE by offering a robust, structure-aware alternative. The findings suggest that focusing on structural fidelity in signal representations can significantly enhance the semantic quality of learned features, positioning SDSC as a promising metric for future self-supervised frameworks in time series domains. For more details, you can refer to the research paper.
