Right on Time: Revising Time Series Models by Constraining their Explanations (2024)

3.1 Explain

Given a pair of input $\bm{x}$ and model output $\hat{\bm{y}}$ for time series classification, the explainer generates an explanation $e_{f}(\bm{x})\in\mathbb{R}^{T}$ in the form of attributions to explain $\hat{\bm{y}}$ w.r.t.$\bm{x}$. For an element of the input, a large attribution value means a large influence on the output. In the remainder of the paper, explanations refer to the model $f$, but we drop $f$ from the notation to declutter it, resulting in $e(\bm{x})$.We use IG(Sundararajan etal., 2017) as an explainer, an established gradient-based attribution method. However, we make some adjustments to the base method to make it more suitable for time series and model revision (Eq.1, further details in SubSec.A.2). In the following, we introduce the modifications to use attributions for forecasting and to obtain explanations in the frequency domain.

Attributions for Forecasting. In a classification setting, attributions are generated by propagating gradients back from the model output (of its highest activated class) to the model inputs. However, there is often no single model output in time series forecasting. Instead, the model generates one output for each timestep of the forecasting window simultaneously. Naively, one could use these $W$ outputs and generate as many explanations $e^{\prime}_{1}(\bm{x}),\dots e^{\prime}_{W}(\bm{x})$. This number of explanations would, however, make it even harder for humans to interpret the results, as the size of the explanation increases with $W$(Miller, 2019). Therefore, we propose to aggregate the individual explanations by averaging in Eq.2.Averaging attributions over the forecasting window provides a simple yet robust aggregation of the explanations. Other means of combining them, potentially even weighted based on distance of the forecast in the future are also imaginable. Overall, this allows attributions for time series classification and forecasting to be generated similarly.

Attributions in the Frequency Domain.Time series data is often given in the frequency representation. Sometimes, this format is more intuitive for humans to understand than the spatial representations. As a result, providing explanations in this domain is essential. Vielhaben etal. (2023) showed how to obtain frequency attributions of the method Layerwise Relevance Propagation (Bach etal., 2015), even if the model does not operate directly on the frequency domain. We transfer this idea to IG: for an input sample $\bm{x}$, we generate attributions with IG, resulting in $e(\bm{x})\in\mathbb{R}^{T}$ (Eq.1 for classification or Eq.2 for forecasting). We then interpret the explanation as a time series, with the attribution scores as values. To obtain the frequency explanation, we perform a Fourier transformation of $e(\bm{x})$, resulting in the frequency explanation $\hat{e}(\bm{x})\in\mathbb{C}^{T}$ with $\hat{E}$ for the entire set.

3.2 Obtain

The next step of RioT is to obtain user feedback on confounding factors. For an input $\bm{x}$, a user can mark parts that are confounded, resulting in a feedback mask $a(\bm{x})\in\{0,1\}^{T}$. In this binary mask, a $1$ signals a potential confounder at this time step. Thereby, it is not necessary to have feedback for every sample of the dataset, as a mask $a(\bm{x})=(0,\dots,0)^{T}$ corresponds to no feedback.Feedback can also be given on the frequency explanation in a similar manner, marking which elements in the frequency domain are potential confounders. The resulting feedback mask $\hat{a}(\bm{x})=(\hat{a}(\bm{x})_{re},\hat{a}(\bm{x})_{im})$ can be different for the real $\hat{a}(\bm{x})_{re}\in\{0,1\}^{T}$ and imaginary part $\hat{a}(\bm{x})_{im}\in\{0,1\}^{T}$. For the whole dataset, we then have spatial annotations $A$ and frequency annotations $\hat{A}$.As the annotated feedback masks have to come from human experts, obtaining them can become costly. In many cases, however, confounders occur systematically and it is therefore possible to apply the same annotation mask to multiple samples. This can drastically reduce the number of annotations required in practice.

3.3 Revise

The last step of RioT is integrating the feedback into the model. We apply the general idea of using a loss-based model revision (Schramowski etal., 2020; Ross etal., 2017; Stammer etal., 2020) based on the explanations and the annotation mask. Given the input data $(\mathcal{X},\mathcal{Y})$, we define the original task (or right-answer) loss as $\mathcal{L}_{RA}(\mathcal{X},\mathcal{Y})$. This loss measures the model performance and is the primary learning objective. To incorporate the feedback, we further use the right-reason loss $\mathcal{L}_{RR}(A,E)$. This loss aligns model explanations $E=\{e(\bm{x})|\bm{x}\in\mathcal{X}\}$ and user feedback $A$ by penalizing the model for explanations in the annotated areas. To achieve model revision and a good task performance, both losses are combined, where $\lambda$ is a hyperparameter to balance both parts of the combined loss $\mathcal{L}(\mathcal{X},\mathcal{Y},A,E)=\mathcal{L}_{\mathrm{RA}}(\mathcal{X}$. Together, the combined loss simultaneously optimizes the primary training objective (e.g. accuracy) and feedback alignment.

It is important to note that giving feedback in the frequency domain allows a new form of model revision through XIL. Even if we effectively still apply masking in the frequency domain, the effect in the original input domain is entirely different. Masking out a single frequency affects all time points without preventing the model from looking at any of them. In general, having an invertible transformation from the input domain to a different representation allows to give feedback more flexible than before. The Fourier transformation is a prominent example but not the only possible choice for this. Using other transformations like wavelets (Graps, 1995), is also possible.

Computational Costs. Including RioT in the training of a model increases the computational cost. Computing the right reason loss term requires the computation of a mixed partial derivative: $\frac{\partial^{2}f_{\theta}(x)}{\partial\theta\partial x}$. Even though this is a second-order derivative, it does not result in any substantial cost increases, as the second-order component of our loss can be formalized as a Hessian-vector product (cf.SubSec.A.4), which is known to be fast to compute (Martens, 2010). We also observed this in our experimental evaluation, as even the naive implementation of our loss in PyTorch scales to large models.

4.1 Experimental Setup

Data. We perform experiments on various datasets. For classification, we focus mainly on the UCR/UEA repository(Dau etal., 2018), which holds a wide variety of datasets for this task. The data originates from different domains, e.g., health records, industrial sensor data, and audio signals. We select all available datasets of a minimal size (cf.SubSec.A.3), which results in Fault Detection A, Ford A, Ford B, and Sleep. We omit experiments on the very small datasets of UCR, as these are generally less suited for deep learning (IsmailFawaz etal., 2020). We use the splits provided by the UCR archive.For time series forecasting, we evaluate on three popular datasets of the Darts repository (Herzen etal., 2022): ETTM1, Energy, and Weather with 70%/30% train/test splits.These datasets are sufficiently large, allowing us to investigate the impact of confounding behavior in isolation without the risk of overfitting.We standardize all datasets as suggested by Wu etal. (2021), i.e., rescaling the distribution of values to zero mean and a standard deviation of one.

Production Press Sensor Data (P2S).RioT aims to mitigate confounders in time series data. To assess our method, we need datasets with annotated real-world confounders. So far, there are no such datasets available. To fill this gap, we introduce Production Press Sensor Data (P2S)²²2https://huggingface.co/datasets/AIML-TUDA/P2S, a dataset of sensor recordings with naturally occurring confounders.The sensor data comes from a high-speed press production line for metal parts, one of the sheet metal working industry’s most economically significant processes. The task is to predict whether a run is defective based on the sensor data. The recordings include different production speeds, which, although not affecting part quality, influence process friction and applied forces. Fig.3 shows samples recorded at different speeds from normal and defect runs, highlighting variations even within the same class.An expert identified regions in the time series that vary with production speed, potentially distracting models from relevant classification indicators, especially when no defect and normal runs of the same speed are in the training data. Thus, the run’s speed is a confounder, challenging models to generalize beyond training. The default P2S setting includes normal and defect runs of different speeds, with an unconfounded setting of runs at the same speed. Further details on the dataset are available in App.B.

Models.For time series classification, we use the FCN model of Ma etal. (2023), with a slightly modified architecture for Sleep to achieve a better unconfounded performance (cf. SubSec.A.2). Additionally, we use the OFA model by Zhou etal. (2023).For forecasting, we use the recently introduced TiDE model (Das etal., 2023), PatchTST (Nie etal., 2023) and NBEATS (Oreshkin etal., 2020) to highlight the applicability of our method to a variety of model classes.

Metrics.In our evaluations, we compare the performance of models on confounded and unconfounded datasets with and without RioT. For classification, we report balanced (multiclass) accuracy (ACC), and for forecasting the mean squared error (MSE). The corresponding mean absolute error (MAE) results can be found inSubSec.A.6. We report average and standard deviation over 5 runs.

Confounders.To evaluate how well RioT can mitigate confounders in a more controlled setting, we add spatial (sp) or frequency (freq) shortcuts to the datasets from the UCR and Darts repositories. These confounders create spurious correlations between patterns and class labels or forecasting signals in the training data, but are absent in validation or test data. We generate an annotation mask based on the confounder area or frequency to simulate human feedback. More details on the confounders can be found in SubSec.A.5.

4.2 Evaluations

Removing Confounders for Time Series Classification.We evaluate the effectiveness of RioT (spatial: RioT_sp, frequency: RioT_freq) in addressing confounders in classification tasks by comparing balanced accuracy with and without RioT.As shown in Tab.1, without RioT, both FCN and OFA overfit to shortcuts, achieving $\approx$100% training accuracy, while having poor test performance. Applying RioT significantly improves test performance for both models across all datasets. In some cases, RioT even reaches the performance of the ideal reference (unconfounded) scenario as if there would be no confounder in the data. Even on FordB, where the drop in training-to-test performance of the reference indicates a distribution shift, RioT_sp is still beneficial.Similarly, RioT_freq enhances performance on frequency-confounded data, though the improvement is less pronounced for FCN on Ford B, suggesting essential frequency information is sometimes obscured by RioT_freq.In summary, RioT (both RioT_sp and RioT_freq) successfully mitigates confounders, enhancing test generalization for FCN and OFA models.

Removing Confounders for Time Series Forecasting.Confounders are not exclusive to time series classification and can significantly impact other tasks, such as forecasting. In Tab.2 we outline that spatial confounders cause models to overfit, but applying RioT_sp reduces MSE across datasets, especially for Energy, where MSE drops by up to 56%.In the frequency-confounded setting, the training data includes a recurring Dirac impulse as a distracting confounder (cf.SubSec.A.5 for details). RioT_freq alleviates this distraction and improves the test performance significantly. For example, TiDE’s test MSE on ETTM1 decreases by 14% compared to the confounded model.

In general, RioT effectively addresses spatial and frequency confounders in forecasting tasks.Interestingly, for TiDE on the Energy dataset, the performance with RioT_freq even surpasses the unconfounded model. Here, the added frequency acts as a form of data augmentation, enhancing model robustness. A similar behavior can also be observed for NBEATS and ETTM1, where the confounded setting actually improves the model slightly, and RioT even improves upon that.

Removing Confounders in the Real-World.So far, our experiments have demonstrated the ability to counteract confounders within controlled environments. However, real-world scenarios often have more complex confounder structures. Our new proposed dataset P2S presents such real-world conditions. The model explanations for a sample in Fig.4 (top) reveal a focus on distinct regions of the sensor curve, specifically the two middle regions. With domain knowledge, it’s clear that these regions shouldn’t affect the model’s output. By applying RioT, we can redirect the model’s attention away from these regions. New model explanations highlight that the model still focuses on incorrect regions, which can be mitigated by extending the annotated area. InTab.4, the model performance (exemplarly with FCN) in these settings is presented. Without RioT, the model overfits to the confounder. the test performance improves already with partial feedback (2) and improves even more with full feedback (4). These results highlight the effectiveness of RioT in real-world scenarios, where not all confounders are initially known.

Removing Multiple Confounders at Once.In the previous experiments, we illustrated that RioT is suitable for addressing individual confounding factors, whether spatial or frequency-based. Real-world time series data, however, often present a blend of multiple confounding factors that simultaneously may influence model performance.

We thus investigate the impact of applying RioT to both spatial and frequency confounders simultaneously (cf.Tab.4), exemplary using FCN and TiDE. When Sleep is confounded in both domains, FCN without RioT overfits and fails to generalize. Addressing only one confounder does not mitigate the effects, as the model adapts to the other. However, combining feedback for both domains (RioT_freq,sp) significantly improves test performance, matching the frequency-confounded scenario (cf.Tab.1).Tab.4 (bottom) shows the impact of multiple confounders on the Energy dataset for forecasting. When faced with both spatial shortcut and noise confounders, the model overfits, indicated by lower training MSE. While applying either spatial or frequency feedback individually already shows some effect, utilizing both types of feedback simultaneously (RioT_freq,sp) results in the largest improvements, as both confounders are addressed.The performance gap between RioT_freq,sp and the non-confounded model is more pronounced than in single confounder cases (cf. Tab.2), suggesting a compounded challenge.Optimize the deconfounding process in highly complex data environments thus remains an important challenge.

Feedback Generalization.As human feedback is an essential aspect of RioT, we investigate the required annotations and the potential to generalize annotations across samples. Our findings indicate that not every sample needs annotation. Fig.5 shows that we can significantly reduce the amount of annotated data for classification and forecasting (cf. App.Tab.6 and Tab.5 for results on the other datasets). Even minimal feedback, such as annotating just 5% of the samples, substantially improves performance compared to no feedback.Furthermore, the results on P2S highlights that annotations can be generalized across multiple samples. Once the confounder on P2S has been identified on a couple of samples, the expert annotations can be used on full dataset.The systematic nature of shortcut confounders suggest that generalizing annotations is an effective possibility to obtain feedback efficiently.While RioT does rely on human annotations, these findings highlight that it can work without extensive manual human interactions, and that obtained annotations can be utilized efficiently.

Limitations.An important aspect of RioT is the human feedback provided in the Obtain step. Integrating human feedback into the model is a key advantage of RioT, but can also be a limitation. While we have shown that a small fraction of samples with annotations can be sufficient, and that annotations can be applied for many samples, they are still necessary for RioT. Additionally, like many other (explanatory) interactive learning methods, RioT assumes correct human feedback. Thus, considering possible repercussions of inaccurate feedback when applying RioT in practice is important.Another potential drawback of RioT are increased training costs. RioT requires computation of a mixed-partial derivative to optimize the model’s explanation when using gradient-based attributions. While this affects training cost, the loss can be formulated as a Hessian-vector product, which is fast to compute in practice, making the additional overhead easy to manage.

A.1 Impact Statement

Our research advances machine learning by enhancing the interpretability and reliability of time series models, significantly impacting human interaction with AI systems. By developing Right on Time (RioT), which guides models to focus on correct reasoning, we improve the transparency and trust in machine learning decisions. While human feedback can provide many benefits, one has also to be aware that it could be incorrect, and evaluate the consequences carefully.

A.2 Implementation and Experimental Details

Adaption of Integrated Gradients (IG).A part of IG is a multiplication of the model gradient with the input itself, improving the explanation’s quality [Shrikumar etal., 2017]. However, this multiplication makes some implicit assumptions about the input format. In particular, it assumes that there are no inputs with negative values. Otherwise, multiplying the attribution score with a negative input would flip the attribution’s sign, which is not desired. For images, this is unproblematic because they are always equal to or larger than zero. In time series, negative values can occur and normalization to make them all positive is not always suitable. To avoid this problem, we use only the input magnitude and not the input sign to compute the IG attributions.

Computing Explanations.To compute explanations with Integrated Gradients, we followed the common practice of using a baseline of zeros. The standard approach worked well in our experiments, so we did not explore other baseline choices in this work. For the implementation, we utilized the widely-used Captum³³3https://github.com/pytorch/captum library, where we patched the captum._utils.gradient.compute_gradients function to allow for the propagation of the gradient with respect to the input to be propagated back into the parameters.

Model Training and Hyperparameters.To find suitable parameters for model training, we performed a hyperparameter search over batch size, learning rate, and the number of training epochs. We then used these parameters for all model trainings and evaluations, with and without RioT. In addition, we selected suitable $\lambda$ values for RioT with a hyperparameter selection on the respective validation sets. The exact values for the model training parameters and the $\lambda$ values can be found in the provided code.

To avoid model overfitting on the forecasting datasets, we performed shifted sampling with a window size of half the lookback window.

All experiments were executed using our Python 3.11 and PyTorch code, which is available in the provided code. To ensure reproducibility and consistency, we utilized Docker. Configurations and Python executables for all experiments are provided in the repository.

Hardware.To conduct our experiments, we utilized single GPUs from Nvidia DGX2 machines equipped with A100-40G and A100-80G graphics processing units.

By maintaining a consistent hardware setup and a controlled software environment, we aimed to ensure the reliability and reproducibility of our experimental results.

A.3 UCR Dataset selection

We focused our evaluation on a subset of UCR datasets with a minimum size. Our selection process was as follows: First, we discarded all multivariate datasets, as we only considered univariate data in this paper. Then we removed all datasets with time series of different length or missing values. We further excluded all datasets of the category SIMULATED, to avoid datasets which were synthetic from the beginning. We furthermore considered only datasets with less than 10 classes, as having a per-class confounder on more than 10 classes would result in a very high number of different confounders, which would probably rarely happen. Besides these criteria, we discarded all datasets with less than 1000 training samples or a per sample length of less than 100, to avoid the small datasets of UCR, which leads to the resulting four datasets: Fault Detection A, Ford A, Ford B and Sleep.

A.4 Computational Costs of RioT

A.5 Details on Confounding Factors

In the datasets which are not P2S, we added synthetic confounders to evaluate the effectiveness of confounders. In the following, we provide details on the nature of these confounders in the four settings:

Forecasting Spatial.For forecasting datasets, spatial confounders are shortcuts that act as the actual solution to the forecasting problem. Periodically, data from the time series is copied back in time. This “back-copy” is a shortcut for the forecast, as it resembles the time steps of the forecasting window. Due to the windowed sampling from the time series, this shortcut occurs at every second sample. The exact confounder formulation is outlined in the sketch below (Fig.7), with an exemplary lookback length of $9$, forecasting horizon of $3$ and window stride of $6$. This results in a shortcut confounder in samples 1 and 3 (marked red) and overlapping in sample 2 (marked orange).

A.6 Additional Experimental Results

This section provides further insights into our experiments, covering both forecasting and classification tasks. Specifically, it showcases performance through various metrics such as MAE, MSE, and accuracy, and explores different feedback configurations.

Feedback Generalization.: Tab.6 and Tab.5 detail provided feedback percentages for forecasting and classification across all datasets, respectively. These tables report the performance of the TIDE and FCN models, highlighting how different levels of feedback impact model outcomes on various datasets. Tab.5 focuses on MAE and MSE for forecasting, while Tab.6 presents ACC for classification.

Removing Confounders for Time Series Forecasting. Tab.7 reports the MAE results for our forecasting experiment across different models, datasets and configurations. It emphasizes how well each model performs on both the confounded training set and after applying RioT, with the Unconfounded configuration representing the ideal scenario unaffected by confounders.

Removing Multiple Confounders at Once. Tab.8 reports the MAE values and illustrates the effectiveness of combining spatial and frequency feedback via RioT for the TiDE model. The results demonstrate significant improvements in forecasting accuracy when integrating both feedback domains compared to using them separately.

B.1 Real-World setting

The production of parts in multiple progressive forming stages using stamping, deep drawing and bending operations with progressive dies is generally one of the most economically significant manufacturing processes in the sheet metal working industry and enables the production of complex parts on short process routes with consistent quality. For the tests, a four-stage progressive die was used on a Bruderer BSTA 810-145 high-speed press with varied stroke speed. The strip material to be processed is fed into the progressive die by a BSV300 servo feed unit, linked to the cycle of the press, in the stroke movement while the tools are not engaged. The part to be produced remains permanently connected to the sheet strip throughout the entire production run. The stroke height of the tool is 63 mm and the material feed per stroke is 60 mm. The experimental setup with the progressive die set up on the high-speed press is shown in Fig.8.

The four stages include a pilot punching stage, a round stamping stage, deep drawing and a cut-out stage. In the first stage, a 3 mm hole is punched in the metal strip. This hole is used by guide pins in the subsequent stages to position the metal strip. During the stroke movement, the pilot pin always engages in the pilot hole first, thus ensuring the positioning accuracy of the components. In the subsequent stage, a circular blank is cut into the sheet metal strip. This is necessary so that the part can be deep-drawn directly from the sheet metal strip. This is a round geometry that forms small arms in the subsequent deep-drawing step that hold the component on the metal strip. In the final stage, the component is then separated from the sheet metal strip and the process cycle is completed. The respective process steps are performed simultaneously so that each stage carries out its respective process with each stroke and therefore a part is produced with each stroke. Fig.9 shows the upper tool unfolded and the forming stages associated with the respective steps on the continuous sheet metal strip.

B.2 Data collection

An indirect piezoelectric force sensor (Kistler 9240A) was integrated into the upper mould mounting plate of the deep-drawing stage for data acquisition. The sensor is located directly above the punch and records not only the indirect process force but also the blank holder forces which are applied by spring assemblies between the upper mounting plate and the blank holder plate. The data is recorded at a sampling frequency of 25 kHz. The material used is DC04 with a width of 50 mm and a thickness of 0.5 mm. The voltage signals from the sensors are digitised using a "CompactRIO" (NI cRIO 9047) with integrated NI 9215 measuring module (analogue voltage input $\pm$ 10 V). Data recording is started via an inductive proximity switch when the press ram passes below a defined stroke height during the stroke movement and is stopped again as it passes the inductive proximity switch during the return stroke movement. Due to the varying process speed caused by the stroke speeds that have been set, the recorded time series have a different number of data points. Further, there are slight variations in the length of the time series withing one experiment. For this reason, all time series are interpolated to a length of 4096 data points and we discard any time series that deviate considerably from the mean length of a run (i.e., threshold of 3). A total of 12 series of experiments, shown in Tab.9, were carried out with production rates from 80 to 225 spm. To simulate a defect, the spring hardness of the blank holder was manipulated in the test series that were marked as defect. The manipulated experiments result in the component bursting and tearing during production. In a real production environment, this would lead directly to the parts being rejected.

B.3 Data characteristics

Fig.10 shows the progression of the time series recorded with the indirect force sensor. The force curve characterises the process cycle during a press stroke. The measurement is started by the trigger which is activated by the ram moving downwards. The downholer plates touch down at point A and press the strip material onto the die. Between point A and point B, the downholder springs are compressed, causing the applied force to increase linearly. The deep drawing process begins at point B. At point C, the press reaches its bottom dead centre and the reverse stroke begins so that the punch moves out of the material again. At point D, the deep-drawing punch is released from the material and now the hold-down springs relax linearly up to point E. At point E, the downholder plate lifts off again, the component is fed to the next process step and the process is complete.

B.4 Confounders

The presented dataset P2S is confounded by the speed with which the progressive tool is operated. The higher the stroke rate of the press, the more friction is occurring and the higher is the impact of the downholder plate. The differences can be observed in Fig.3. Since we are aware of these physics-based confounders, we are able to annotate them in our dataset. As the process speed increases, the friction that occurs between the die and the material in the deep-drawing stage changes, as the frictional force is dependent on the frictional speed. This is particularly evident in the present case, as deep-drawing oils, which can optimize the friction condition, were not used in the experiments. The affected area from friction of the punch are in 1380 to 1600 (start of deep drawing) and 2080 to 2500 (end of deep drawing). In addition, the impulse of the downholder plate affecting the die increases due to the increased process dynamics. If the process speed is increased, the process force also increases in the ranges of the time series from 800 to 950 (downholder plate sets down) and 3250 to 3550 (downholder plate lifts).

In the experiment setting of Tab.4, the training data set is selected in such a way that the stroke rate correlates with the class label, i.e., there are only normal experiments with small stroke rates and defect ones with high stroke rate. Experiment 1, 2, 3, 10, 11, 12 are the training data and the remaining experiments are the test data. To get a unconfounded setting where the model is not affected by any confounder, we use normal and defect experiments with the same speed in training and respectively test data. This results in experiments 1, 3, 5, 7, 9, 11 in the training set and the remaining in the test set.