Available online at www.HighTechJournal.org HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 45 ISSN: 2723-9535 Modifying Hidden Layer in Neural Network Models to Improve Prediction Accuracy: A Combined Model for Estimating Stock Price Abbas Mahmoudabadi 1* , Mehdi Kanaani 2, Fatemeh Pourhossein Ghazimahalleh 3 1 Ph.D. in Industrial Engineering, Director of Master Program in Industrial Engineering, MehrAstan University, Guilan, Iran. 2 Graduated Student, Department of Industrial Engineering, MehrAstan University, Guilan, Iran. 3 Graduated Student, Department of E-Business and Information Technology, MehrAstan University, Guilan, Iran. Received 06 November 2021; Revised 02 January 2022; Accepted 12 January 2022; Published 01 March 2022 Abstract Investment experts, who deal with stock price estimation, commonly look for the most accurate and appropriate statistical techniques to make decisions on investment. The aim of this study is to improve the accuracy of stock price prediction models through modifying the structure of a combined neural network model with time-series data, in which the main contribution is to insert the time-series analysis prediction into the hidden layer of the neural network. The proposed structure is made up of neural networks and time-series analysis, with variable reduction used to remove attributes with inter-correlations. Data has been collected over six years (72 months) from the Iranian stock market, including the number of trades, new-coin price, gold-18 price, US Dollar and Euro equivalent currencies, oil-index price, Brent-oil price, industry index, and balanced stock index, followed by developing the prediction models. Comparing the performance criteria of the proposed structure to the traditional ones in terms of the mean square and mean absolute errors revealed that inserting time-series estimated variables into hidden layers would improve the performance of neural network models to estimate stock prices for making investment decisions. Keywords:Artificial Neural Network; Stock Price Estimation; Time-Series Data Analysis; Combined Prediction Modeling. 1. Introduction 1.1. Stock Market Prediction Prediction of stock prices and market situations are essential issues in financing where they are getting more attention from investment experts who are willing to make proper decisions and to ensure that the positive direction of the market is successfully predicted [1]. On the other hand, the existence of the nonlinearity and volatility of the financial market has been identified by many researchers and financial analysts [2]. Therefore, many models have been proposed utilizing a variety of fundamental, technical, and time-series forecasting techniques to gain accurate predictions in this field. One of the main concerns in stock market investment is to gain an overall view of the future and predict the trend of stock prices as well as illustrative graphs to make the right decisions and affordable plans for the future. Although precise forecasting of markets may be impossible, the researchers intend to tackle this problem by proposing methods that are more accurate and comparing the accuracy of the prediction models to select the best method. Some characteristics of financial time-series data, such as non-stationarity, nonlinearity, and high volatility, prompt investment professionals to create more accurate and fitted models [3]. While some of the forecasting models' *Corresponding author: mahmoudabadi@mehrastan.ac.ir http://dx.doi.org/10.28991/HIJ-2022-03-01-05  This is an open access article under the CC-BY license (https://creativecommons.org/licenses/by/4.0/). © Authors retain all copyrights. https://creativecommons.org/licenses/by/4.0/ https://orcid.org/0000-0002-1175-6730 HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 46 and applications' results do not satisfy investors, they can learn the stock market's operation rules to gain a clear context of what happens in various situations. In this regard, forecasting studies are almost always regarded as difficult due to the existing uncertainty in the stock price system and the complexity of external economic and environmental factors. 1.2. Neural Network Models for Prediction Because of the rapid development of artificial intelligence and computer technology, the stock price forecasting models are constantly updated, integrated, and improved. Among the above techniques, neural network modeling is getting more attention because, compared to other traditional methods, it has achieved amazing results in the field of prediction [4]. The basic concept behind the neural network structure is to develop a model to interpret the relationships between variables according to its internal relations, so it is widely used in pattern recognition, intelligent control, signal processing, and other fields in which there are relations between attributes. The artificial neural network models are also used for predicting the stock price. In the modeling of stock prices, mainly in non-linear format, the rationality and applicability of the model construction have their own advantages that can provide the nonlinear prediction model with a wider space for sensitivity analysis [5]. In the case of financial time-series, data is extremely nonlinear and fluctuates. Time-series approaches are usually attributed to dynamic systems, so we need algorithms to interpret the hidden patterns and underlying dynamics behavior of data, so called machine learning techniques. Although machine-learning techniques have been recently improved and applied in various research fields, the estimating methods still need to be improved to be suitable for time-series analysis [6]. To deal with the above challenge, neural network models have been developed. Despite the advantages of the neural networks that have been extensively considered for the prediction of the stock market, sometimes they fail to predict the financial markets accurately [7]. Putrier believed that investment and fund management could be defined as optimal dynamic problems, and that they should predict portfolio dynamical behavior in order to optimize investors' capital structures. A comprehensive study has also been conducted, focusing on the common parameters for designing a back propagation neural network and providing a systematic methodology for forecasting economic time-series data [8, 9]. Komo et al. [10] developed and compared two neural network models, Radial Basis Function (RBF) and back propagation (also known as multilayer perceptron (MLP)), for predicting stock market prices by employing data from the Wall Street Journal's Dow Jones as a benchmark. A notable success of the proposed models was achieving prediction accuracies of over 80% based on the Dow Jones monthly industrial index predictions, and the results demonstrated that RBF neural networks are preferred to MLP networks. 1.3. Stock Market Prediction Methods Let us enumerate some of the available forecasting methods used in the prediction of stock prices. They are not only used for stock market prediction but also in fields that generate numeric measures that can be computed based on time-series data. Fundamental Analysis: Fundamental analysis is a kind of investment analysis adopted by investors for taking decisions [9]. By studying a company’s sales, management proficiency, earnings, dividends, profits, and a host of other economic factors, they basically estimate the intrinsic worth of a company’s share where the above factors have a bearing on the company’s profitability and business prospects. The process leads experts to estimate the price of a particular company’s share and consider the estimated value as the intrinsic or true value of the share, which reflects the inherent worth and value. The estimated intrinsic price would help investors judge whether the shares are currently over-priced or under-priced. A fundamentalist who uses fundamental analysis makes money by purchasing underpriced stocks and selling them when they become overpriced. Fundamental analysis is more useful for long-term investments. Technical Analysis: In technical analysis, a large number of rules and indicators are committed to identifying and explaining the regularity of dynamic historical prices. Technical analysis makes use of patterns in a financial instrument's price history to forecast price behavior in the future [11]. Technical analysts argue that prices gradually adjust to new information, so the moving average method is one of the most common techniques utilized by technical analysis. Although the MA method is easy to use and apply in investment decision-making or empirical tests [12], the research conducted by Dzikevicius [13] showed that moving average methods may generate errors and deviations in forecasting, so they would not be successful in estimating the trend of prices in long-term decisions. Technical analysts, who are also called "chartists" as they use charts and graphs to keep a record of share price movements, believe that an accurate study of share price charts and graphs would reveal regular and recurrent patterns of price behavior that are likely to be repeated in the future [14]. In brief, technical analysis attempts to predict the future price of a particular share based on a study of its price movements in the past, so it is more commonly used for taking "buying" and "selling" decisions in the stock market than the other techniques. HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 47 Time-series Forecasting: Time-series forecasting intends to predict a dependent variable or an attribute for the future based on its past behavior. This is a significant concern in the field of stock market investment, where investors are willing to make the right decisions at the right time to maximize their financial profit. Time-series forecasting usually comes across a specific trend in the past data to predict the future data, so conventional research usually uses time-series analysis techniques like mixed auto regression, moving average, and multiple regression models [15]. Overall, if data is available for a long period but at the same intervals, the easier way is to find a pattern to predict the future, but if the history of a stock is not enough or segregated data is available, the accuracy of the analysis and forecast is getting a little difficult. There are a lot of methods in this field, but the most well-known ones are the moving average, autoregressive moving average, and autoregressive integrated moving average, followed by seasonal prediction methods. Among them, the moving average works such that the average of a fixed number of items in the time-series is attributed to the next time interval. Fluctuating data that moves through the series is smoothly uniformed by dropping the top items and adding the below ones with each successive average [16]. For application and in the modeling of linear and stationary time-series data, researchers usually employ combination models for many purposes due to their superiority, ease of implementation, and robustness [17]. 1.4. Artificial Neural Networks Structure in Stock Market Prediction Investing in the stock market usually involves higher risk due to its uncertainty and volatility [18], so forecasting the stock price behavior is crucial in terms of accuracy. The difficulty arises when nonlinear and complex behavior of stock prices is observed. Due to the dynamic environment together with existing incomplete or noisy data that may be observed, for example in traffic conditions [19], the artificial neural network models are suitable to be utilized because of their proper adaptation to this kind of data [20]. Therefore, in the last two decades, research has constantly attempted to develop neural network models for forecasting stock prices [21]. There are many types of neural network models where Multi-Layer Perceptron (MLP) are feed-forward neural networks with one or more layers between the input and output layers. Feed-forward means that data flows in one direction from an input layer to an output layer (forward) through a transitional layer, usually called a hidden layer. An MLP consists of multiple layers of nodes in a directed graph in which each layer is connected to the next one. Except for the input nodes, each node is a neuron, known as a processing element, with a nonlinear activation function to connect the previous nodes and make another node in its own layer. The above characteristics would improve the prediction model to be able to resolve problems that are not linearly-based structured with one or more hidden layers [22]. In a neural network model, the output Yi of each neuron of the nth layer is mathematically defined by a derivable nonlinear function based on Equation 1, where F is the non-linear activation function, Wji is the weight of the connection between the neuron Nj and Ni, and finally Yi is the output of the neuron of the (n−1)th layer [23]. Y𝑖 = 𝐹(∑ 𝑊𝑗𝑖𝑌𝑖 𝑛 𝑗 ) (1) 1.5. Performances of Neural Network Models In general, neural network modeling is among data processing techniques called data mining. Setty et al. [24] reviewed the applications of data mining techniques to evaluate the performance of stock markets and concluded that there is a rising gap between storage and retrieval systems. Since storage is more powerful, a technological leap needs to be made to prioritize information about end-user problems. When Dase et al. [25] reviewed the literature on the application of artificial neural networks to stock market predictions, they demonstrated that data mining tools are useful in this field. They revealed that predicting stock indexes through performing traditional time-series analysis is too difficult, but the artificial neural network may be more suitable, so they revealed that the artificial neural network is a useful technique for predicting stock markets. In the field of combining prediction techniques, Kumar et al. [26] introduced some basic ideas of time-series data, the need for ANN, the importance of stock indices, and a survey of the previous works, and investigated neural network models' applications for time-series in forecasting, and the result was that existing functional relationships are the main scientific characteristics of the above models. In the preceding study, ANN performance measures such as mean square errors, root mean square errors, mean absolute errors, and others were defined for model comparison, and it was concluded that the ANN model achieved the lowest prediction errors when fitting them to a large amount of stock market data. Attempts and observations in the literature for improving ANN performance include changing the number of nodes in the hidden layer [27], changing the number of network modules and channels [28], and improving accuracy through knowledge distillation [29], where filtering is used as a compression technique. As more studies for more approaches, Li [30] improved the accuracy of ANN models through combing the above networks with intelligent diagnosis in medical treatments where the combined model improved diagnosis efficiency and saved doctors' time. Instead of outputs, the residuals from a two-step combined model are estimated, and the accuracy of the prediction is determined by the set of errors and indices [31]. The prediction accuracy of neural networks can also be improved by composing them with time-series data as well as developing hybrid methods [32]. HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 48 1.6. Vision Statement Following the above mentioned, it is concluded that the combination of prediction techniques to achieve more accurate predictions of stock exchanges by neural network models may be a useful approach, but not in the common way where all attributes are simultaneously considered as input variables. Therefore, the novelty of the present research methodology is to develop another approach different from the previous ones. It is developed to insert a part of a data series into the hidden layer of the neural network instead of the input layer. Therefore, the concept behind this research work is to estimate the desired variable during a specific period in the first stage and predict the stock exchange price in the second stage. The time-series analysis results are inserted as hidden nodes in the hidden layer of the neural network model. 2. Research Methodology and Procedure 2.1. Data Gathering This research work focuses on the stock price forecasting, so Tehran Stock Exchange has been selected as case study because of data availability and Shiraz Petrochemical Company (Iran) stock price is under detail study. Data sets are downloaded from tsetmc.com, investing.com, and mop.ir sites from May 21, 2012 to March 18, 2018. They have been analyzed and summarized to monthly measures in which the number of records has been set to 72. It means that there are 72 months of data for modeling in this research work. Input variables are Number of Trades (TN), Trade Volume (TV), New-Coin (NC), Gold-18 (G18), US-Dollar (USD), Euro (EU), Oil-Index (OIN), Brent-Oil Price (BOP), Oil-Price (OP), Industry-Index (IND), Total Index (TI), and Balanced-Index (BLI), and output variable is the stock price of Shiraz Petrochemical Company (SPC). A classification of collected data is shown in Table 1. Table 1. Classification of collected data fields and their abbreviations used in modeling Type Descriptions of Variables Input variables Number of Trades (TN), Trade Volume (TV), New-Coin (NC), Gold 18 (G18), US-Dollar (USD), Euro (EU), Oil- Index (OIN), Brent-Oil Price (BOP), Oil-Price (OP), Industry-Index (IND), Total Index (TI), Balanced-Index (BLI). Output variable Stock Price of Shiraz Petrochemical Company (SPC) 2.2. Inter-correlation Test Some of so-called independent variables may have inter-correlation with each other or the researchers may intend to reduce the number of variables. Although, many techniques of data reduction are studied and implemented over the financial studies [33], in particular in prediction models [34], but the inter-correlation test is commonly utilized to ensure that the variables those are considered as input in the modeling procedure would not have inter-correlation. Statistically, the correlation between to variables is obtained by Equation 2 followed by utilizing the hypothesis test of t-test where the Equation 3 calculates t-stat and the corresponding P-Value is obtained based on t-Stat. The P-Value is checked to decide if the variables are significantly independent or correlated. The procedure will continue until the variables do not have inter-correlation [35]. The whole procedure is named variable reduction which is needed prior to modeling stage. 𝑟 = ∑ (𝑋1𝑖 − 𝑋1 ̅̅ ̅)(𝑋2𝑖 − 𝑋2 ̅̅ ̅)𝑛 𝑖=1 √∑ (𝑋1𝑖 − 𝑋1 ̅̅ ̅)2 ∑ (𝑋2𝑖 − 𝑋2 ̅̅ ̅)2𝑛 𝑖=1 𝑛 𝑖=1 (2) 𝑡 = 𝑟√ 𝑛 − 2 1 − 𝑟2 (3) 2.3. Neural Network Modeling Developing model is following the variable reduction stage where input, hidden, and output layers are defined based on the purified variables and time-series node is inserted as a hidden node in hidden layer. The number of hidden nodes excluding time-series node which are all depicted in Figure 1 where the time-series node is shown as TS-Node. As shown it is a combined model of neural network and time-series analysis where the results of time-series analysis is a part of neural network model. HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 49 Figure 1. Schematic view of the new combined time-series and neural network model 2.4. Running Model The stock price of Shiraz Petrochemical Company (SPC) is now predicted by inserting time-series analysis into the neural network model and the results are discussed to estimate the output for the future. In addition, the structure of neural network model will be developed in various types to reach the best performances of the model. 2.5. Validation The final stage of the research methodology is validation stage in which the performance criteria of the combined time-series and neural network models will be evaluated. Two well-known criteria of mean square errors (MSE) and mean absolute errors (MAE), respectively formulated by Equation 4 and 5, are compared where 𝑌𝑡 is the observed value for Stock Market Price Shiraz Petroleum Company at time t and �̂�𝑡 is the estimated one at the same time. A schematic view of the research stages is depicted in Figure 2. MSE = 1 𝑛 ∑ 𝑒𝑡 2 𝑛 𝑡=1 = 1 𝑛 ∑(𝑌𝑡 − �̂�𝑡)2 𝑛 𝑡=1 (4) MAE = 1 𝑛 ∑ 𝑒𝑡 2 𝑛 𝑡=1 = 1 𝑛 ∑ |𝑌𝑡 − �̂�𝑡| 𝑛 𝑡=1 (5) Figure 2. Overall view of the research methodology followed in this study 3. Numerical Results 3.1. Descriptive Statistics The first stage of the research methodology is to investigate the data downloaded from the sites mentioned in the previous section. Data has been gathered duration six years (72 months) from May 2012 to March 2018. Regardless to the variables selected for developing the neutral network model, Table 2 summarizes the overall stats calculated for six years in monthly measures. Identifying the attributes have studied and data available Gathering data, pre-analyzing and purification Validating the models’ performances and select the best model Running the structured model and predicting stock price Structuring neural network models by inserting time-series node into hidden layer Input Layer TN NC BLI …. Hidden Layer HN1 HN2 TS-Node …. Output Layer Output HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 50 Table 2. Descriptive statistics for all variables available in stock marketing Stat Count Mean Minimum Maximum Range SD Kurtosis Skewness TN 72 119 11 468 457 114 1.050 1.376 TV 72 2083761 30760 47511261 47480500 5786630 55.371 7.134 NC 72 10331833 6480556 14685882 8205327 1899264 -0.398 0.410 G18 72 1019696 887938 1316429 428491 109579 0.268 1.121 USD 72 32758 17045 42216 25170 5399 2.145 -1.295 EU 72 41613 36223 49943 13720 3189 0.193 0.722 OIN 72 220765 50543 366533 315990 87922 -0.578 -0.231 BOP 72 80 32 124 93 30 -1.813 -0.054 OP 72 70 30 107 77 25 -1.704 0.133 IND 72 57089 19306 87329 68023 18004 -0.204 -0.575 TI 72 66339 24280 98435 74155 19488 -0.170 -0.859 BLI 72 12889 9410 17739 8329 2969 -1.454 0.312 SPC 72 4996 1974 13140 11166 2979 -0.489 0.891 3.2. Inter-correlation Test The second stage is to check the inter-correlation for all candidate variables as inputs. The t-test has been utilized to check whether there is inter-correlation between the independent variables or not. The first step is to calculate pair- wise correlations between all variables by Equation 2 which are eventually tabulated in Table 3. The second step is to calculate t-Stat through Equation 3. The results have been tabulated in Table 4. Finally, the third step is to obtain P- Value for the first type of errors in hypothesis testing followed by tabulating them in Table 5. As shown, the P-Values that are more than 0.05 and highlighted in gray that should be removed from the modeling process because they have inter-correlations with the other variables. Table 3. Correlations between candidate variables (Pair wise correlations) Correlation TN TV NC G18 USD EU OIN BOP OP IND TI BLI TN 1 TV 0.2223 1 NC 0.5381 0.0367 1 G18 0.4873 0.1150 0.7624 1 USD 0.3401 0.0567 0.7202 0.6189 1 EU 0.1445 0.0968 0.1753 0.3582 -0.1382 1 OIN 0.3092 -0.0576 0.3461 0.3484 0.5117 0.0352 1 BOP -0.0156 -0.1490 -0.1162 -0.2947 -0.6364 0.1282 -0.0197 1 OP 0.0800 -0.1151 0.0019 -0.1353 -0.5145 0.2697 0.1044 0.8943 1 IND 0.2606 0.0617 0.3116 0.4751 0.7445 0.0324 0.7655 -0.6038 -0.4655 1 TI 0.1832 0.0576 0.2232 0.3768 0.7068 -0.0458 0.7769 -0.5916 -0.4615 0.9849 1 BLI 0.5207 0.1561 0.4981 0.6556 0.5808 0.3450 0.4805 -0.4919 -0.3033 0.7184 0.6484 1 Table 4. T-Stats for correlation between candidate variables Correlation TN TV NC G18 USD EU OIN BOP OP IND TI BLI TN - TV 1.9079 - NC 5.3407 0.3070 - G18 4.6695 0.9683 9.8561 - USD 3.0256 0.4755 8.6855 6.5932 - EU 1.2214 0.8134 1.4901 3.2104 -1.1671 - OIN 2.7203 -0.4829 3.0860 3.1095 4.9826 0.2949 - BOP -0.1307 -1.2610 -0.9790 -2.5797 -6.9019 1.0813 -0.1651 - OP 0.6718 -0.9695 0.0157 -1.1427 -5.0201 2.3432 0.8784 16.7208 - IND 2.2584 0.5173 2.7437 4.5170 9.3298 0.2714 9.9540 -6.3374 -4.4006 - TI 1.5593 0.4826 1.9158 3.4030 8.3605 -0.3838 10.3241 -6.1392 -4.3527 47.6644 - BLI 5.1024 1.3223 4.8059 7.2639 5.9689 3.0749 4.5836 -4.7275 -2.6635 8.6403 7.1250 - HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 51 Table 5. P-Value for t-test between candidate variables P-Value TN TV NC G18 USD EU OIN BOP OP IND TI BLI TN - TV 0.0605 - NC 0.0000 0.7598 - G18 0.0000 0.3362 0.0000 - USD 0.0035 0.6359 0.0000 0.0000 - EU 0.2260 0.4187 0.1407 0.0020 0.2471 - OIN 0.0082 0.6307 0.0029 0.0027 0.0000 0.7689 - BOP 0.8964 0.2115 0.3310 0.0120 0.0000 0.2833 0.8694 - OP 0.5039 0.3357 0.9875 0.2571 0.0000 0.0220 0.3827 0.0000 - IND 0.0270 0.6066 0.0077 0.0000 0.0000 0.7869 0.0000 0.0000 0.0000 - TI 0.1234 0.6309 0.0595 0.0011 0.0000 0.7023 0.0000 0.0000 0.0000 0.0000 - BLI 0.0000 0.1904 0.0000 0.0000 0.0000 0.0030 0.0000 0.0000 0.0096 0.0000 0.0000 - According to what has been concluded from the inter-correlation test, among all variables which corresponding data are available, the independent variables are Trade Number (TN), New-Coin price(NC), Gold 18 (G18), US-Dollar (USD), Oil-Index (OIN), Industry-Index (IND), and eventually Balanced-Index (BLI). From now on, the model structuring uses the independent variables to develop the neural network model as well as the stock price for Shiraz petroleum company (SPC) that will be considered as dependent variable. 3.3. Network Modeling Following the purification process of candidate variables and extracting independent ones, it is time to develop the neural network models. The stock price for Shiraz Petroleum Company (SPC) is estimated based on the dependent variables. According to the approach followed in this study, the neural network model is constructed in two stages. The first stage is to obtain coefficients for constructing hidden nodes’ values (HNs) based on independent variables, and the second stage is to obtain output values where one of the hidden nodes is the moving average value of stock price for k last periods. Equations 6 and 7 represent the general functions developed in the above mentioned stages. The hidden layer can be also received another node by an exponential smoothing time-series value in which the Equation 8 indicates the exponential time-series where α is the smoothing factor, F denotes the forecasted, and R is the observed value of stock for dependent variable. HNVt = 𝑓(𝑇𝑁𝑡 , 𝑁𝐶𝑡 , 𝐺18𝑡 , 𝑈𝑆𝐷𝑡 , 𝑂𝐼𝑁𝑡 , 𝐼𝑁𝐷𝑡 , 𝐵𝐿𝐼𝑡) (6) SPCt = 𝑔(𝐻𝑁𝑉𝑡 , 1 𝑘 ∑ 𝑆𝑃𝐶𝑗 𝑡 𝑗=𝑡−𝑘+1 ) (7) SPCt = 𝑔(𝐻𝑁𝑉𝑡 , ∝ 𝑅𝑆𝑃𝐶𝑡 + (1−∝)𝑅𝑆𝑃𝐶𝑡−1 ) (8) To illustrate how the model is developed, one of the models, as a sample, is constructed in neural network format and represented by Equations 9 to 12. As shown, the time-series node is used in the second series of equations where a two period moving average for SPC is inserted for formulation. It should be mentioned that the coefficients have been obtained where the target is to minimize mean square errors. The other possible models have been also developed and the results are discussed in the next subsection. HN1t = 1.847 − 107.870𝑇𝑁𝑡 − 1.432𝑁𝐶𝑡 + 6.308𝐺18𝑡 − 36.226𝑈𝑆𝐷𝑡 + 19.879𝑂𝐼𝑁𝑡 + 5.563𝐼𝑁𝐷𝑡 + 94.754𝐵𝐿𝐼𝑡 (9) HN2t = −3.898 − 86.304𝑇𝑁𝑡 − 1.203𝑁𝐶𝑡 + 4.407𝐺18𝑡 − 136.406𝑈𝑆𝐷𝑡 + 12.815𝑂𝐼𝑁𝑡 + 44.937𝐼𝑁𝐷𝑡 + 174.04𝐵𝐿𝐼𝑡 (10) HN3t = 8.347 + 255.646𝑇𝑁𝑡 − 0.446𝑁𝐶𝑡 + 1.674𝐺18𝑡 − 65.759𝑈𝑆𝐷𝑡 + 4.247𝑂𝐼𝑁𝑡 + 22.559𝐼𝑁𝐷𝑡 + 64.524𝐵𝐿𝐼𝑡 (11) SPCt = 37.087 + 0.000318𝐻𝑁1𝑡 + 0.00448𝐻𝑁2𝑡 + 0.01039𝐻𝑁3𝑡 + 1.01877 × 1 2 (𝑆𝑃𝐶𝑡−1 + 𝑆𝑃𝐶𝑡−2) (12) 3.4. Validation The last stage is to validate the model performances in different models. Two criteria of mean square errors (MSE) and mean absolute errors (MAE) have been calculated and tabulated in Table 6. The first column identifies the time- series method combined with neural network. Three different approaches of no time-series, adding moving average, HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 52 and exponential smoothing are studied where the structure of input layers are the same for all models. The third column represents the hidden layer components in which moving average approaches compose of one, two, and three moving average durations and smoothing factor is different model to model for exponential smoothing approaches. The fourth and fifth columns respectively represent the mean square errors (MSE) and mean absolute errors (MAE) all depicted in Figures 3 and 4 as well where they respectively show the performance criteria for moving average and exponential smoothing combined models. In both figures, mean square errors (MSE) is depicted at the left side and mean absolute errors (MAE) is depicted at the right side. Table 6. Summary of different structures for neural network model and performance criteria TS Method Input Layer Hidden Layer Components MSE MAE No Series Input Nodes (TN, ...) Hidden Nodes Only 3559875 1482 Moving Input Nodes (TN, ...) Hidden Nodes + 𝑆𝑃𝐶𝑡−1 365498 392 Average Input Nodes (TN, ...) Hidden Nodes + 1 2 (𝑆𝑃𝐶𝑡−1 + 𝑆𝑃𝐶𝑡−2) 493358 472 Input Nodes (TN, ...) Hidden Nodes + 1 3 (𝑆𝑃𝐶𝑡−1 + 𝑆𝑃𝐶𝑡−2 + 𝑆𝑃𝐶𝑡−3) 705617 558 Exponential Input Nodes (TN, ...) Hidden Nodes + 0.1𝑅𝑆𝑃𝐶𝑡 + 0.9𝑅𝑆𝑃𝐶𝑡−1 1086996 762 Smoothing Input Nodes (TN, ...) Hidden Nodes + 0.3𝑅𝑆𝑃𝐶𝑡 + 0.7𝑅𝑆𝑃𝐶𝑡−1 564520 577 Input Nodes (TN, ...) Hidden Nodes + 0.5𝑅𝑆𝑃𝐶𝑡 + 0.5𝑅𝑆𝑃𝐶𝑡−1 156773 278 Input Nodes (TN, ...) Hidden Nodes + 0.7𝑅𝑆𝑃𝐶𝑡 + 0.3𝑅𝑆𝑃𝐶𝑡−1 81276 224 Input Nodes (TN, ...) Hidden Nodes + 0.9𝑅𝑆𝑃𝐶𝑡 + 0.1𝑅𝑆𝑃𝐶𝑡−1 45130 181 Looking more carefully at what has been derived from Table 6 and Figures 3 and 4, it is observed that the worst performance criteria belong to the model developed in the traditional method in which time-series variables are absent from the model. Inserting time-series values into the hidden layer significantly affects the performance criteria where both MSE and MAE have the same behavior in that they are decreased if time-series values are inserted into the hidden layer. Therefore, it is concluded that modifying the structure of the hidden layer would improve the efficiency of neural network prediction performances by not only adding a moving average node in the hidden layer but also other outputs of time-series analysis, such as exponential smoothing values. Figure 3. The performance criteria for combined neural network models with moving average 0 200 400 600 800 1000 1200 1400 1600 0 500000 1000000 1500000 2000000 2500000 3000000 3500000 4000000 Hidden Nodes Only Hidden Nodes + One- Period Moving Average Hidden Nodes + Two- Period Moving Average Hidden Nodes + Three- Period Moving Average M ea n S q u a re E rr o rs MSE MAE M ea n A b so lu te E rr o rs HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 53 Figure 4. The performance criteria for combined neural network models with exponential smoothing 4. Conclusion In the present research work, the structure of the neural network model has been modified to improve the performance of prediction ability for estimating dependent variables based on predictors. Two methods of time-series analysis, including moving average and exponential smoothing, have been separately inserted as hidden nodes in the structure of the hidden layer. The modified neural network models have been utilized to predict stock exchanges in the Iranian stock market, where Shiraz Petroleum Company (Iran) was under study. Neural network models have been developed by employing experimental data collected over six years, followed by testing inter-correlation coefficients to ensure that dependent variables would not have inter-correlation. All models in which the time-series results are inserted as hidden nodes have been evaluated based on two criteria: mean and absolute square errors. Looking more carefully at the obtained results, it is revealed that the modification of the hidden layer in the neural network models would improve the accuracy of prediction models, at least in stock market predictions. 5. Declarations 5.1. Author Contributions Conceptualization, A.M.; methodology, A.M. and F.P.G.; software, F.P.G. and A.M.; validation, A.M.; formal analysis, M.K.; investigation, M.K.; resources, M.K.; data curation, A.M.; writing—original draft preparation, M.K. and F.P.G.; writing—review and editing, A.M.; visualization, A.M.; supervision, A.M.; project administration, A.M. All authors have read and agreed to the published version of the manuscript. 5.2. Data Availability Statement The data presented in this study are available on request from the corresponding author. 5.3. Funding The authors received no financial support for the research, authorship, and/or publication of this article. 5.4. Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. 0 200 400 600 800 1000 1200 1400 1600 0 500000 1000000 1500000 2000000 2500000 3000000 3500000 4000000 Hidden Nodes Only Hidden Nodes + ES (α=0.1) Hidden Nodes + ES (α=0.3) Hidden Nodes + ES (α=0.5) Hidden Nodes + ES (α=0.7) Hidden Nodes + ES (α=0.9) M ea n S q u a re E rr o rs MSE MAE M ea n A b so lu te E rr o rs HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 54 6. References [1] Kumar, D., Sarangi, P. K., &Verma, R. (2022). A systematic review of stock market prediction using machine learning and statistical techniques. In Materials Today: Proceedings (Vol. 49). Proceedings. doi:10.1016/j.matpr.2020.11.399. [2] Abhyankar, A., Copeland, L. S., & Wong, W. (1997). Uncovering nonlinear structure in real-time stock-market indexes: The S & P 500, the DAX, the Nikkei 225, and the FTSE-100. Journal of Business and Economic Statistics, 15(1), 1–14. doi:10.1080/07350015.1997.10524681. [3] Rezaei, H., Faaljou, H., &Mansourfar, G. (2021). Stock price prediction using deep learning and frequency decomposition. Expert Systems with Applications, 169, 114332. doi:10.1016/j.eswa.2020.114332. [4] Chung, H., & Shin, K. S. (2018). Genetic algorithm-optimized long short-term memory network for stock market prediction. Sustainability (Switzerland), 10(10), 3765. doi:10.3390/su10103765. [5] Atsalakis, G. S., &Valavanis, K. P. (2009). Surveying stock market forecasting techniques - Part II: Soft computing methods. Expert Systems with Applications, 36(3 PART 2), 5932–5941. doi:10.1016/j.eswa.2008.07.006. [6] Hu, M., Ji, Z., Yan, K., Guo, Y., Feng, X., Gong, J., Zhao, X., & Dong, L. (2018). Detecting Anomalies in Time-series Data via a Meta-Feature Based Approach. IEEE Access, 6, 27760–27776. doi:10.1109/ACCESS.2018.2840086. [7] Maier, H. R., & Dandy, G. C. (2000). Neural networks for the prediction and forecasting of water resources variables: A review of modelling issues and applications. Environmental Modelling and Software, 15(1), 101–124. doi:10.1016/S1364- 8152(99)00007-9. [8] Zhang, D., & Lou, S. (2021). The application research of neural network and BP algorithm in stock price pattern classification and prediction. Future Generation Computer Systems, 115, 872–879. doi:10.1016/j.future.2020.10.009. [9] Devadoss, A. V., &Ligori, T. A. A. (2013). Forecasting of stock prices using multi-layer perceptron. International Journal of Computing Algorithm, 2, 440–449. [10] Komo, D., Chang, C. I., &Ko, H. (1994). Neural network technology for stock market index prediction. ISSIPNN 1994 - 1994 International Symposium on Speech, Image Processing and Neural Networks, Proceedings, 543–546. doi:10.1109/SIPNN.1994.344854. [11] Pring, M. J. Martin Pring’s Introduction to Technical Analysis. McGraw Hill Professional, New York, United States. [12] Ben-Zion, U., Klein, P., Shachmurove, Y., & Yagil, Y. (2005). Efficiency Differences between the S&P 500 and the Tel-Aviv 25 Indices: A Moving Average Comparison. In SSRN Electronic Journal. doi:10.2139/ssrn.420243. [13] Dzikevicius, A., Saranda, S., & Kravcionok, A. (2010). The Accuracy of Simple Trading Rules in Stock Markets. Economics and Management, 15, 910–916. [14] Vanstone, B., & Finnie, G. (2006). Combining technical analysis and neural networks in the Australian stockmarket. In Proceedings of the 10th IASTED International Conference on Artificial Intelligence and Soft Computing, ASC 2006, 125–130. [15] Huang, W., Nakamori, Y., & Wang, S. Y. (2005). Forecasting stock market movement direction with support vector machine. Computers and Operations Research, 32(10), 2513–2522. doi:10.1016/j.cor.2004.03.016. [16] Montgomery, D. C., Jennings, C. L., & Kulahci, M. (2015). Introduction to time-series analysis and forecasting. John Wiley & Sons, New York, United States. [17] Wlezien, C. (2000). An essay on “combined” time-series processes. Electoral Studies, 19(1), 77–93. doi:10.1016/S0261- 3794(99)00037-2. [18] Horvath, B., Muguruza, A., & Tomas, M. (2021). Deep learning volatility: a deep neural network perspective on pricing and calibration in (rough) volatility models. Quantitative Finance, 21(1), 11–27. doi:10.1080/14697688.2020.1817974. [19] Mahmoudabadi, A., &Fakharian, S. (2010) Estimating missing traffic data using artificial neural network. Proceedings of Second International Conference on Intelligent Network and Computing (ICINC 2010), 144–147. [20] Chen, Q., Song, Y., & Zhao, J. (2020). Short-term traffic flow prediction based on improved wavelet neural network. Neural Computing and Applications, 33(14), 8181–8190. doi:10.1007/s00521-020-04932-5. [21] Aghbashlo, M., Hosseinpour, S., &Mujumdar, A. S. (2015). Application of Artificial Neural Networks (ANNs) in Drying Technology: A Comprehensive Review. Drying Technology, 33(12), 1397–1462. doi:10.1080/07373937.2015.1036288. [22] Vaferi, B., Eslamloueyan, R., &Ayatollahi, S. (2011). Automatic recognition of oil reservoir models from well testing data by using multi-layer perceptron networks. Journal of Petroleum Science and Engineering, 77(3–4), 254–262. doi:10.1016/j.petrol.2011.03.002. https://en.wikipedia.org/wiki/New_York_City HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 55 [23] Celikoglu, H. B., &Cigizoglu, H. K. (2007). Modelling public transport trips by radial basis function neural networks. Mathematical and Computer Modelling, 45(3–4), 480–489. doi:10.1016/j.mcm.2006.07.002. [24] Setty, D. V., Rangaswamy, T. M., &Subramanya, K. N. (2010). A review on Data Mining Applications to the Performance of Stock Marketing. International Journal of Computer Applications, 1(3), 33–43. doi:10.5120/88-187. [25] RK, D., & DD, P. (2010). Application of Artificial Neural Network for Stock Market Predictions: a Review of Literature. International Journal of Machine Intelligence, 2(2), 14–17. doi:10.9735/0975-2927.2.2.14-17. [26] Kumar, D. A., &Murugan, S. (2013). Performance analysis of Indian stock market index using neural network time series model. Proceedings of the 2013 International Conference on Pattern Recognition, Informatics and Mobile Engineering, PRIME 2013, 72–78. doi:10.1109/ICPRIME.2013.6496450. [27] Abiodun, O. I., Jantan, A., Omolara, A. E., Dada, K. V., Mohamed, N. A. E., & Arshad, H. (2018). State-of-the-art in artificial neural network applications: A survey. Heliyon, 4(11), 938. doi:10.1016/j.heliyon.2018.e00938. [28] Shi, C., Tan, C., Wang, T., & Wang, L. (2021). A waste classification method based on a multilayer hybrid convolution neural network. Applied Sciences (Switzerland), 11(18), 8572. doi:10.3390/app11188572. [29] Prakosa, S. W., Leu, J. S., & Chen, Z. H. (2021). Improving the accuracy of pruned network using knowledge distillation. Pattern Analysis and Applications, 24(2), 819–830. doi:10.1007/s10044-020-00940-2. [30] Li, X. (2021). Artificial intelligence neural network based on intelligent diagnosis. Journal of Ambient Intelligence and Humanized Computing, 12(1), 923–931. doi:10.1007/s12652-020-02108-6. [31] Buevich, A., Sergeev, A., Shichkin, A., &Baglaeva, E. (2021). A two-step combined algorithm based on NARX neural network and the subsequent prediction of the residues improves prediction accuracy of the greenhouse gases concentrations. Neural Computing and Applications, 33(5), 1547–1557. doi:10.1007/s00521-020-04995-4. [32] Büyükşahin, Ü. Ç., &Ertekin, Ş. (2019). Improving forecasting accuracy of time-series data using a new ARIMA-ANN hybrid method and empirical mode decomposition. Neurocomputing, 361, 151–163. doi:10.1016/j.neucom.2019.05.099. [33] Mahmoudabadi, A. , A., Mehrshad, M., &Aminnaseri, M. R. (2017). Credit Risk Assessment Utilizing Data Reduction Technique for Individual Loaning in Financial Institutes (Case Study: Tejarat Bank, Rasht, Iran). American Journal of Theoretical and Applied Business, 3(1), 11. doi:10.11648/j.ajtab.20170301.12. [34] Delima, A. J. P., Sison, A. M., & Medina, R. P. (2019). Variable reduction-based prediction through modified Genetic Algorithm. International Journal of Advanced Computer Science and Applications, 10(5), 356–363. doi:10.14569/ijacsa.2019.0100544. [35] Boddy, R., & Smith, G. (2009). Statistical Methods in Practice: For Scientists and Technologists. In Statistical Methods in Practice: For Scientists and Technologists. John Wiley & Sons. doi:10.1002/9780470749296.