Evaluation of Deep Neural Network in Directional Prediction of Forex Market

In this paper we apply deep neural network techniques in order to forecast future trend of Forex market. We will utilize deep neural network to predict whether there will be uptrend or downtrend or neutral. The result can be used by traders as a baseline to make further decision about whether to sell or buy exchange rates. Historical exchange rate values are used as input to deep neural network and performance of the model is examined over various size of input data. Besides, for making a better decision system, various threshold values of exchange rate percentage change are examined in order to flag up or down directions. We use the model to predict five commonly traded exchange rates. The accuracy of the model is calculated by both F-Score and percentage of correct classification. Results indicate that deep neural network have acceptable performance in prediction of direction of the market. Besides, results indicate that the model maintains its accuracy over different input length and threshold values.


Introduction
One of earliest explanation of market behaviour was random walk theory. With this explanation, market moves in stochastically random steps with limited possibility of prediction and analysis. On the other hand, two methodologies including technical analysis and fundamental analysis brings further possibility for prediction of market behaviour. Fundamental analysis investigates large scale factors like social, political indices and important economic measures in order to predict and analysis of behaviour of the market. Technical analysis focus on patterns and mathematical indicators derived from stock values in order to explain market behaviour. These two methodologies are the most common methods for market analysis and decision making in currency market.
Forex Market is one of the challenging areas for both market analysis and machine learning techniques. Many early attempts to predict the market was based on time series techniques, mainly known as Auto Regressive Integrated Moving Average (ARIMA) and Generalized Autoregressive Conditional Heteroskedasticity (GARCH) models (Box et al., 2016). As the market has non-stationary nature and these models were based on stationarity, these attempts were not enough satisfying. Later, by start of application of machine learning in time series modelling, compare to early models, these new models show better performance. Studies that compare performance of Neural Network (NN) with traditional ARIMA model indicate superiority of performance of NN (Kamruzzaman and Sarker, 2003a;2003b;Yao and Tan, 2000). Other investigations compare performance of different machine learning techniques. Using technical indicators as input for prediction, Patel et al. (2015b) compares performance of four models including NN, Support Vector Machine (SVM), Random Forest, and Naive Bayes, concluding superiority of Random Forest over the rest. Cao and Tay (2001) compared performance of SVM with NN. Results indicate that SVM has better performance over NN. Nagpure (2019) compares performance of different machine learning techniques including SVR, Long Short-Term Memory (LSTM), and NN. Results show that all models have acceptable performance in prediction of exchange rates. Adetiba et al. (2013) trained NN with fundamental data in order to create an automated platform for trading in real world with promising performance in prediction of exchange rates. Baasher and Fakhr (2011) performs binary (uptrend/downtrend) classification on exchange rate values using and comparing performance of Radial Basis Function (RBF), Multilayer Perceptron (MLP), and SVM method.
Hybrid models were another area of investigation to find acceptable models. Many machine learning architectures and models were combined for finding a better solution.
Mainly, results indicate that combination of different machine learning techniques improves model's performance. Vargas et al. (2018) show superiority of hybrid models over conventional models. It used combination of technical indicators and financial news as input data to a hybrid deep learning model which combines Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) models for market prediction. Zhang (2003) used hybrid combination of NN and ARIMA models in order to capture both linearity and non-linearity nature of exchange market. The results indicate that hybrid model has better performance than single ARIMA or NN models. Patel et al. (2015a) performed two stages machine learning model, where the first stage uses SVR model to predict on day ahead values of exchange rates and technical indicators. Using NN, SVR, and Random Forest models, the second stage predicts the desired exchange rate based on predictions of the first stage SVRs. The results show two stage model has better performance compare to single SVR or NN models. Nayak (2017) used different architectures of Higher Order Neural Network (HONN) in order to predict exchange rate. It utilized Particle Swarm Optimization (PSO) and Genetic Algorithm (GA) to select parameters of HONN architectures. In terms of statistical evaluation, results show that HONN outperforms traditional models like MLP and RBF. Ow et al. (2016) utilized two stage machine learning techniques. In the first stage, Linear Regression Line (LRL) method is utilized to extract general repeated patterns of exchange rate. Then, the extracted patterns are fed to an MLP for classification. Results show that the model is capable of classification of unknown upcoming patterns. Zargany and Ahmadi (2015) performs modular version of NN where each module deals separately with specific recognized pattern on exchange rate data. First, exchange rate data is recognized by five harmonic patterns. Then, each NN get these harmonic points from each harmonic pattern recognition unit. The system shows acceptable performance in harmonic pattern detection. D'Lima and Khan (2016) utilized MLP for predicting direction of Forex movement and a Hybrid Neuro Fuzzy system (ANFIS) for predicting exchange rate values in next days. Both NN and ANFIS models predict future trend and values with acceptable accuracy. Tsantekidis et al. (2018) used advanced deep learning architectures like CNN and LSTM for exchange rate prediction. Results show that combination of these architectures outperforms individual implementation. Using Technical Indicator (TI) as machine learning model inputs, Abreu et al. (2018) utilized Naive Bayes classifier to detect whether prices are overvalued or undervalued. It performs Genetic Algorithm (GA) for optimizing parameters used in calculation of technical indicators. The results show that GA optimization boost performance of the model. Wang (2014) performs Principle Component Analysis (PCA) to illustrate existence of co-movement between stock markets. Then, combined with PCA, SVM and ANN models used for predicting future stocks direction. Results show that performing PCA on dataset enhance both SVM and ANN prediction performances. Tiong et al. (2016) used combination of NN with LRL analysis to identify and classify uptrend/downtrend of Forex market. Features of uptrend/downtrend, which are extracted using LRL, are fed to NN for classification. For increasing accuracy of prediction, Ince and Trafalis (2007) utilized non-linear type of PCA for data pre-processing. It compares SVR and NN in prediction of future price and results indicate that NN outperforms SVM.
Text mining is another area of attempt for market prediction. Having acceptable accuracy in prediction of market direction, Khadjeh Nassirtoussi et al. (2015) performs text mining on financial news headline. Khadjeh Nassirtoussi et al. (2014) surveys studies that used text mining for financial market prediction. For categorizing studies, it considers three aspects including linguistic, machine learning techniques and behavioral-economic. Alostad and Davulcu (2015) implement text mining on financial news to predict direction of stock prices. It shows that using breaking news alongside financial news can boost model accuracy. Onsumran et al. (2015) used text mining for extracting information from economic news articles and three machine learning methods namely SVM, K-Nearest Neighbour (K-NN), and Naive Bayes for classification. Results suggests that SVM outperform other methods in classification. Bollen et al. (2010) investigated possibility of prediction of stock market index from Twitter feeds. They extracted moods from Twitter feeds and used a Neuro-Fuzzy for direction prediction.

Methodology
Our methodology in basically based on Efficient Market Hypothesis (EMH), where it believes that market prices reflects all information about the market. In other words, without need for further investigation of other indices and factors, the prices itself mirror all information about the market. Based on this, we use four Forex candlestick values including close, open, high, and low values in order to build our input vector to the model. It is believed that the deep neural network is capable of understanding relation between existing pattern in the input vector (historical exchange rate values) and future trends. Directional prediction is chosen for evaluation of the model because it can be used for trading strategies. It is a measure which is conceivable for traders, and other more sophisticated strategies can be built up according this measure. As the goal is to predict trend at a time T, the input vector consists of candlestick values from time T-L to T-1, where L is input window length. It is number of days before time T considered for creating input vector.

Data Processing
Data processing is an important step of any deep learning application. As data is usually unclean, it needs process to make data ready for using. Historical data is obtained from Currency market at a glance (2020), consisting of daily data of open, low, high, and close values of different exchange rates from Jun. 1995 to Jul. 2020. For avoiding saturation in deep neural network nodes, applying normalization on input values is necessary. By calculating , input values are maintained in [ 1 2 ] (Basheer and Hajmeer, 2000).

Evaluation Metrics
The objective is to evaluate whether deep neural network is capable of predicting three classes of exchange rate directional movement including up, down, and neutral. These three classes are dependent in a sense that the probability of each class is dependent on probability of other classes, and the classes are mutually exclusive, which means at any time only one class is flagged. With these conditions, the objective of optimization is considered Softmax cross entropy (Gomez, 2018). In the final layer of deep neural network, output ( )is passed through a Softmax unit and probability of each class is evaluated. Error is calculated using cross entropy of target labels and probability of classes : In order to detect up and down movement in exchange rates, we assume that if the percentage of change between closing prices of current day and a day before exceeds certain threshold (δ) in positive (negative) direction, the uptrend (downtrend) flag will be raised. If this threshold doesn't exceed, it is considered as neutral. This threshold is aimed for both removing possible noises for considering some level of certainty in detection of direction of exchange rates, and to consider whether amplitude of the change is enough in order to consider it valuable for trading. The target labels of classification is as follows: The accuracy of the model is defined as percentage of correct classification over number of samples . It is calculated based on indicator function 1{ }. By considering nonstationarity nature of exchange rate values, it is possible that up and down changes of the exchange rate values be skewed and therefore, performance of the model become different in detecting up or down directions. Therefore, by evaluating F-Score measures, possibility of skewed classes will be considered. It is calculated by using two parameters including Precision and Recall. These two parameters are calculated using True Positive (TP), False Positive (FP), and False Negative (FN). Thanks to Pedregosa et al. (2012), these model metrics are calculated and depicted.

Evaluation of Deep Neural Network in Directional Prediction of Forex Market
We implement model calculations using TensorFlow software library. It is widely used for machine learning engineering and has capability of extension of calculation to Graphic Processing Unit (GPU), where the process of evaluation become faster (Abadi et al., 2015). In order to find optimum architecture of deep neural network, for each exchange rate, different deep neural network architectures are evaluated.
3 Results Table 1, Table 2, Table 3, and Table 4 show performance of deep neural network in directional prediction of five exchange rates. Table 1 and Table 2 shows effect of input window length (L) on the performance of deep neural network. As window length increases, more historical data is taken into account for predicting next directional movement. In this case, it is assumed that the threshold (δ) is zero, so any change in exchange rate values are considered whether uptrend or downtrend (no change is classified as neutral). As results show, the model is capable of predicting exchange rate directional movement with best accuracy of 94% and F-Score of 0.93. The results suggest that the model is capable of extracting patterns in exchange rate and draw true conclusion whether the pattern is leading to uptrend or downtrend.   Table 3 and Table 4 show performance of deep neural network over different threshold values (δ) for five exchange rates. As the threshold increases, the more changes in exchange rates are classified as neutral. The results prove robustness in performance of deep neural network over different threshold values. Further, it means one can define this threshold based on trading strategy without having concern about model's performance.

Conclusion
In this paper we investigate capability of deep neural network in predicting direction of Forex exchange rate changes. We introduce window length as time period where data within this period is used for creating input vector, and threshold value for signifying whether uptrend or downtrend (otherwise neutral) has accrued. We probe performance of the model in different time window length and different threshold values in detecting up or down trends. Results indicate that the model has acceptable performance and maintains its performance within different window length and threshold values.