Unsupervised Technique for Anomaly Detection in Qatar Stock Market

Abstract

The aim of anomaly detection is to find patterns or data points that are not confirming the expected behavior inside the dataset. Techniques from a variety of disciplines like machine learning, statistics, information theory and data mining are used to solve this problem. The form of input data from stock market is a non-linear complex time series. Hence, the statistical methods in this case will be ineffective. Using the behavior of similar time series for detecting anomalies in Qatar stock exchange and American stock market index (Standard Poor (SP)) is the main goal of this paper. Supervised learning techniques were used extensively in detecting stock market manipulation. The problem of supervised learning techniques is that they require substantial effort in labeling the data. Having dynamic nature of anomalous behavior causes another problem. In this research, we investigate the use of unsupervised learning for detecting stock market manipulation and we introduce a new preprocessing step for improving the recall of the anomaly detection system without hurting the precision. The Contextual Anomaly Detector (CAD) that is based on unsupervised technique is used to find anomalies by looking at time series that have similar behaviors. The use of our new preprocessing steps with CAD improved the recall significantly compared to other studies.