Deep Reinforced Active Learning for Time Series Anomaly Detection

Abstract

Massive time series data are recorded in various domains. Identifying exceptional data objects within time series can help avoid potential faults, dangers, and accidents, which is significant in maintaining the target system’s health and stability. Recent years have witnessed a long list of unsupervised time-series anomaly detection models that can only work without any control after deployment. This motivates us to consider an intriguing question: can we devise a model that adaptively and iteratively evolves according to the interaction with human analysts. We intend to install a handle on the model, thus realizing a “human-in-the-loop” learning paradigm. However, it is still a non-trivial task due to the difficulty of (i) accurately exploring valuable data from the unlabeled set as query samples and (ii) fully exploiting returned annotated data. To tackle these challenges, this paper proposes a novel reinforced active time series anomaly detection algorithm. We first propose to use the ensemble of unsupervised anomaly scoring, and by leveraging the derived anomaly scores, we devise two reward strategies. The learning process is guided by these reward strategies, during which the agent is encouraged to explore possible anomalies hidden in the unlabeled set. These potential anomalies are submitted as queries for human labeling and further exploited in our reward functions to supervise the agent taking expected actions. Extensive experiments on real-world datasets demonstrate that our method substantially outperforms five state-of-the-art competitors and obtains 12.1%–60.3% ${F}_{1}$ score improvement, and 11.5%–59.1% AUC-PR improvement.