A Novel Anomaly Detection for Streaming Data using LSTM Autoencoders
Raghav Agarwal1, Tanishq Nagpal2, Dibyajyoti Roy3, Aju D4
1Raghav Agarwal*, Department of School of Computer Science and Engineering, Vellore Institute of Technology, Vellore (Tamil Nadu), India.
2Tanishq Nagpal, Department of School of Computer Science and Engineering, Vellore Institute of Technology, Vellore (Tamil Nadu), India.
3Dibyajyoti Roy, Department of School of Electronics and Engineering, Vellore Institute of Technology, Vellore (Tamil Nadu), India.
4Aju D, Professor, Department of School of Computer Science and Engineering, Vellore Institute of Technology, Vellore (Tamil Nadu), India.
Manuscript received on July 18, 2021. | Revised Manuscript received on July 30, 2021. | Manuscript published on July 30, 2021. | PP: 233-241 | Volume-10 Issue-2, July 2021. | Retrieval Number: 100.1/ijrte.B62940710221| DOI: 10.35940/ijrte.B6294.0710221
Open Access | Ethics and Policies | Cite | Mendeley
© The Authors. Blue Eyes Intelligence Engineering and Sciences Publication (BEIESP). This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
Abstract: The high-volume and velocity data stream generated from devices and applications from different domains grows steadily and is valuable for big data research. One of the most important topics is anomaly detection for streaming data, which has attracted attention and investigation in plenty of areas, e.g., the sensor data anomaly detection, predictive maintenance, event detection. Those efforts could potentially avoid large amount of financial costs in the manufacture. However, different from traditional anomaly detection tasks, anomaly detection in streaming data is especially difficult due to that data arrives along with the time with latent distribution changes, so that a single stationary model doesn’t fit streaming data all the time. An anomaly could become normal during the data evolution, therefore it is necessary to maintain a dynamic system to adapt the changes. In this work, we propose a LSTMs-Autoencoder anomaly detection model for streaming data. This is a mini-batch based streaming processing approach. We experimented with streaming data that containing different kinds of anomalies as well as concept drifts, the results suggest that our model can sufficiently detect anomaly from data stream and update model timely to fit the latest data property.. Index Terms: About four key words or phrases in alphabetical order, separated by commas.
Keywords: An Anomaly Could Become Normal During The Data Evolution, Therefore It Is Necessary To Maintain A Dynamic System To Adapt The Changes.