12560 FACTA UNIVERSITATIS Series: Electronics and Energetics Vol. 37, No 1, March 2024, pp. 211 - 227 https://doi.org/10.2298/FUEE2401211U © 2024 by University of Niš, Serbia | Creative Commons License: CC BY-NC-ND Original scientific paper A COMPREHENSIVE COMPARATIVE STUDY OF MACHINE LEARNING MODELS FOR PREDICTING CRYPTOCURRENCY Yüksel Akay Ünvan, Cansu Ergenç Finance and Banking Department, Business School, Ankara Yildirim Beyazit University, Ankara, Turkey ORCID iDs: Yüksel Akay Ünvan https://orcid.org/0000-0002-0983-1455 Cansu Ergenç https://orcid.org/0000-0002-4722-0911 Abstract. This study aims to find the best performing model in predicting cryptocurrencies using different machine learning models. In our study, an analysis was performed on various cryptocurrencies such as Aave, BinanceCoin, Bitcoin, Cardano, Cosmos, Dogecoin, Ethereum, Solana, Tether, Tron, USDCoin and XRP. Decision Trees, Random Forests, K- Nearest Neighbours (KNN), Gradient Boost Machine (GBM), LightGBM, XGBoost, CatBoost, Artificial Neural Networks (ANN), Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs) and Short Term Memory networks in Long Comparisons (LSTM) models were used. The performance of the models is compared with Mean Squared Error (MSE), Root Mean Square Error (RMSE) and Mean Absolute Error (MAE). The study results show that there is no single model that consistently outperforms others for all cryptocurrencies. Models such as XGBoost and Random Forests show consistent and strong performance across different cryptocurrencies, proving their robustness in this particular use case. Deep learning algorithms, including Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs) and Long Short Term Memory Networks (LSTMs), show significant accuracy in predicting some cryptocurrencies. Key words: Cryptocurrencies, Machine Learning Forecasting Model, Prediction methods, Efficiency 1. INTRODUCTION Cryptocurrencies have recently attracted a great deal of attention as a distinctive and ever- changing type of investment. Investors and researchers have turned to cryptocurrencies due to their significant profit potential and market volatility. The ability to predict the prices of cryptocurrencies is crucial for investors, traders and financial analysts who want to make informed decisions in this rapidly growing industry. Machine learning and deep learning techniques have become powerful tools for accurately predicting cryptocurrency prices in Received December 1, 2023; revised January 30, 2024 and February 29, 2024; accepted March 03, 2024 Corresponding author: Yüksel Akay ÜNVAN Finance and Banking Department, Business School, Ankara Yildirim Beyazit University, Ankara, Turkey E-mail: akay.unvan@gmail.com https://orcid.org/0000-0002-0983-1455 https://orcid.org/0000-0002-4722-0911 mailto:akay.unvan@gmail.com 212 Y. A. ÜNVAN, C. ERGENÇ recent years. These approaches are able to effectively capture complex temporal correlations and non-linear patterns in data collected from bitcoin prices. Each cryptocurrency and its underlying platform contain different features and applications that contribute to the diverse environment of the cryptocurrency world. Aave is a decentralised finance (DeFi) protocol that allows users to borrow and lend different cryptocurrencies [1]. In this system, the lender earns interest on digital assets in liquidity pools. The borrower can obtain a loan by pledging collateral. The creators originally designed BinanceCoin as a utility token for the Binance cryptocurrency exchange, and later expanded its use. BinanceCoin's uses include powering Binance's decentralised applications (dApps) and paying transaction fees on the Binance Exchange [2]. Cardano is a blockchain platform that aims to provide a more secure and scalable infrastructure for the development of decentralised applications and smart contracts [3]. Cosmos is a decentralised network of independent blockchains powered by Byzantine Fault Tolerance (BFT) consensus algorithms [4]. Dogecoin, initially created as a meme coin, gained significant value and popularity by experiencing significant increases in its exchange rate [5]. Ethereum is a decentralised platform that enables smart contracts and decentralised applications to be created and run without any downtime, fraud, control or interference from a third party. Solana is a high-performance blockchain that enables fast, secure and scalable decentralised applications and cryptocurrencies [6]. Tether is a type of cryptocurrency, known as a stablecoin, designed to provide a stable value by being pegged to a reserve asset such as the US dollar. Tron is a blockchain-based decentralised platform that aims to create a free, global digital content entertainment system with distributed storage technology. USDCoin is a stablecoin cryptocurrency whose value is directly pegged to the US dollar. XRP is the native digital asset of XRP Ledger, a blockchain created by Ripple. In this study, analyses were conducted on various cryptocurrencies such as Aave, BinanceCoin, Bitcoin, Cardano, Cosmos, Dogecoin, Ethereum, Solana, Tether, Tron, USDCoin and XRP. Decision Trees, Random Forests, K-Nearest Neighbours (KNN), Gradient Boost Machine (GBM), LightGBM, XGBoost, CatBoost, Artificial Neural Networks (ANN), Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Short-Term Memory networks in Long Comparisons (LSTM) models were used. The organisation of the rest of this paper is as follows: Section 2 contains a literature review on forecasting methods and cryptocurrencies. Section 3 presents machine learning forecasting methods. The results of the study are presented in Section 4. Finally, final comments are given in Section 5. 2. LITERATURE REVIEW In the digitalizing world, the use of cryptocurrencies for investment purposes has become widespread in recent years. For this reason, the number of studies on cryptocurrencies in the literature is increasing day by day. There are many studies on Bitcoin and other cryptocurrencies. Bitcoin, the groundbreaking cryptocurrency, has been extensively studied in terms of its preference and function in many areas [7]. Analyzes of Bitcoin often focus on economic policy uncertainty and volatility behavior, with insights into its potential as a financial instrument. Furthermore, the application of machine learning for ransomware classification in Bitcoin transactions has been evaluated, highlighting the importance of robust prediction models in the context of cryptocurrency security [8]. In general, studies have focused on Bitcoin, but other types of cryptocurrencies have also been examined. Gray prediction approach was used to predict the closing price of cryptocurrencies such as Bionic, Cardano, Dogecoin, A Comprehensive Comparative Study OF Machine Learning Models for Predicting Cryptocurrency 213 Ethereum, XRP [9]. Another study examines price distortion and information efficiency before and during the COVID-19 outbreak using Bitcoin, BNB, Cardano, Ethereum and XRP cryptocurrencies [10]. The use of the Cardano formula in data-driven gradient flows highlights the mathematical foundations for cryptocurrencies beyond Bitcoin [11]. The feasibility of including different cryptocurrencies in investment portfolios for diversification was assessed and the potential benefits of diversification were highlighted, especially in the case of Terra [12]. In the field of sentiment analysis, comparative analyses using the Support Vector Machine and Naive Bayes algorithm have been performed on cryptocurrencies, shedding light on the potential applications of cryptocurrencies as digital goods for trading and investment assets. In addition, cryptocurrency price prediction based on ARIMA, Random Forest, and LSTM algorithms has been investigated; LSTM shows superior prediction accuracy compared to Random Forest and ARIMA models [13]. We enhance the existing body of research by doing a thorough examination of several machine learning models applied to a diverse set of cryptocurrencies. In contrast to typical research that concentrate on a solitary digital currency or a restricted range of models, we conduct a comparative analysis of the effectiveness of several models such as Decision Trees, Random Forests, KNN, GBM, LightGBM, XGBoost, CatBoost, ANN, CNNs, RNNs, and LSTMs across twelve different cryptocurrencies. This provides a more holistic understanding of the applicability and robustness of these models in cryptocurrency prediction, addressing a significant gap in the literature. 3. METHODOLOGY In this study, Decision Trees, Random Forests, KNN, ANN, CNNs, RNN, LSTMs, GBM, LightGBM, XGBoost and CatBoost models are used in the comparisons. Table 1 provides a comprehensive list of the cryptocurrencies and prediction models that were used in the research. Table 1 Cryptocurrencies and prediction models used in this paper Cryptocurrencies Models Aave Decision Trees BinanceCoin Random Forests Bitcoin Gradient Boosting Machines (GBM) Cardano XGBoost Cosmos LightGBM Dogecoin CatBoost Ethereum K-Nearest Neighbors (KNN) Solana Neural Networks Tether Convolutional Neural Networks (CNNs) Tron Recurrent Neural Networks (RNNs) USDCoin Long Short-Term Memory Networks (LSTMs) XRP The dataset contains Date, Open, High, Low, Close, Volume and Market Cap information for Aave, BinanceCoin, Bitcoin, Cardano, Cosmos, Dogecoin, Ethereum, Solana, Tether, Tron, 214 Y. A. ÜNVAN, C. ERGENÇ USDCoin and XRP cryptocurrencies. Table 2 provides explanations of the values in the data set. The date range of each cryptocurrency is given in Table 3. Table 2 Characteristics of the dataset Variable Description Date Date of observation Open Opening price on the given day High Highest price on the given day Low Lowest price on the given day Close Closing price on the given day Volume Volume of transactions on the given day Market Cap Market capitalization in USD Table 3 The range of dataset Cryptocurrencies Dataset Cryptocurrencies Dataset Aave 05.10.2020 - 06.07.2021 Ethereum 08.08.2015 - 06.07.2021 BinanceCoin 26.07.2017 - 06.07.2021 Solana 11.04.2020 - 06.07.2021 Bitcoin 29.04.2013 - 06.07.2021 Tether 26.02.2015 - 06.07.2021 Cardano 02.10.2017 - 06.07.2021 Tron 14.09.2017 - 06.07.2021 Cosmos 15.03.2019 - 06.07.2021 USDCoin 09.10.2018 - 06.07.2021 Dogecoin 16.12.2013 - 06.07.2021 XRP 05.08.2013 - 06.07.2021 3.1. Decision Trees Model The theoretical background of decision trees models includes internal nodes that represent attribute tests [14]. The tree structure includes branches that reflect test results and leaf nodes that represent class labels [15]. Decision trees are widely used due to their simplicity and efficiency in knowledge discovery, which are important factors. Decision tree algorithms are used in areas such as comparing Bitcoin and Ethereum [16], predicting the closing price of cryptocurrencies [17]. Figure 1 shows the fit of cryptocurrencies to the Decision Trees model. The practical application of these models to cryptocurrency prediction is diverse, with each sub-figure providing empirical evidence of the model predictive capacity for a specific cryptocurrency. The sub-figures in Figure 1 reflect the adaptability of decision trees across different cryptocurrencies, highlighting the model performance in terms of accuracy and its ability to capture trends within the highly volatile cryptocurrency market. 3.2. Random Forests Model Random Forests models are a robust and adaptive learning approach widely used in many different domains [18]. The theoretical structure of random forests is a machine learning methodology that improves prediction accuracy and reduces overfitting by combining many decision trees [19]. Figure 2 shows the fit of cryptocurrencies to the Random Forests model. Random forest algorithm has applications in areas such as financial fraud detection [20], bank failure prediction [21], and estimating cryptocurrency value [22]. A Comprehensive Comparative Study OF Machine Learning Models for Predicting Cryptocurrency 215 Fig. 1 (a) of constituent Aave. (b) of constituent BinanceCoin. (c) of constituent Bitcoin. (d) of constituent Cardano. (e) of constituent Cosmos. (f) of constituent Dogecoin. (g) of constituent Ethereum. (h) of constituent Solana. (i) of constituent Tether. (j) of constituent Tron. (k) of constituent USDCoin. (l) of constituent XRP. Source: Figure by authors Fig. 2 (a) of constituent Aave. (b) of constituent BinanceCoin. (c) of constituent Bitcoin. (d) of constituent Cardano. (e) of constituent Cosmos. (f) of constituent Dogecoin. (g) of constituent Ethereum. (h) of constituent Solana. (i) of constituent Tether. (j) of constituent Tron. (k) of constituent USDCoin. (l) of constituent XRP. Source: Figure by authors 216 Y. A. ÜNVAN, C. ERGENÇ 3.3. K-Nearest Neighbours Model K-Nearest Neighbours (KNN) models is a widely used and fundamental approach in the field of machine learning and pattern recognition [23]. The KNN method is a non- parametric machine learning technique that does not assume any particular data distribution and uses nearby examples to generate predictions [24]. The KNN method is widely used in stock price prediction [25,26]. Figure 3 shows the fit of cryptocurrencies to the KNN model. Fig. 3 (a) of constituent Aave. (b) of constituent BinanceCoin. (c) of constituent Bitcoin. (d) of constituent Cardano. (e) of constituent Cosmos. (f) of constituent Dogecoin. (g) of constituent Ethereum. (h) of constituent Solana. (i) of constituent Tether. (j) of constituent Tron. (k) of constituent USDCoin. (l) of constituent XRP. Source: Figure by authors 3.4. Artificial Neural Networks Model Artificial Neural Networks (ANN) models are powerful and flexible machine learning models that have attracted great interest in various fields [27]. Artificial neural networks (ANNs) are computational models that replicate the architecture and functioning of the human brain [28]. These systems are specifically designed to process complex data inputs and perform tasks such as classification, regression and pattern recognition. ANNs are highly valuable in a variety of applications including document recognition, reinforcement learning and predictive modelling due to their flexibility and malleability [29]. Figure 4 shows the fit of cryptocurrencies to the ANN model. A Comprehensive Comparative Study OF Machine Learning Models for Predicting Cryptocurrency 217 Fig. 4 (a) of constituent Aave. (b) of constituent BinanceCoin. (c) of constituent Bitcoin. (d) of constituent Cardano. (e) of constituent Cosmos. (f) of constituent Dogecoin. (g) of constituent Ethereum. (h) of constituent Solana. (i) of constituent Tether. (j) of constituent Tron. (k) of constituent USDCoin. (l) of constituent XRP. Source: Figure by authors 3.5. Recurrent Neural Networks (RNNs) Model Recurrent Neural Networks (RNNs) models are highly efficient and widely used deep learning models that have received significant acclaim in various fields [30]. Figure 5 shows the fit of cryptocurrencies to the RNNs model. Fig. 5. (a) of constituent Aave. (b) of constituent BinanceCoin. (c) of constituent Bitcoin. (d) of constituent Cardano. (e) of constituent Cosmos. (f) of constituent Dogecoin. (g) of constituent Ethereum. (h) of constituent Solana. (i) of constituent Tether. (j) of constituent Tron. (k) of constituent USDCoin. (l) of constituent XRP. Source: Figure by authors 218 Y. A. ÜNVAN, C. ERGENÇ RNNs are computer models that efficiently capture the patterns and dynamics of sequences using cyclic connections between nodes in the network. RNNs provide a quality that makes them well suited for tasks involving the manipulation of sequential data [31]. RNNs have shown remarkable effectiveness in a variety of fields, including speech recognition, music classification, and the processing of time series data [32-34]. 3.6. Convolutional Neural Networks Model Convolutional Neural Networks (CNNs) models are highly effective and widely used deep learning models that have attracted great interest in various fields [35]. CNNs are specifically designed to process structured grid data such as photographs and time series data [36]. They have shown outstanding performance in a number of applications, including image identification, medical diagnostics and signal processing. Figure 6 shows the fit of cryptocurrencies to the CNNs model. Fig. 6 (a) of constituent Aave. (b) of constituent BinanceCoin. (c) of constituent Bitcoin. (d) of constituent Cardano. (e) of constituent Cosmos. (f) of constituent Dogecoin. (g) of constituent Ethereum. (h) of constituent Solana. (i) of constituent Tether. (j) of constituent Tron. (k) of constituent USDCoin. (l) of constituent XRP. Source: Figure by authors 3.7. Long Short-Term Memory Networks (LSTMs) Model Long Short-Term Memory Networks (LSMs) models are very effective and widely used deep learning models that have received significant attention in various fields [37]. LSMs, also known as Long Short-Term Memory networks, are a special type of recurrent neural network (RNN) primarily designed to capture and comprehend long-term dependencies in sequential input [38]. LSTM models have performed well for tasks such as the analysis of time series data, natural language processing and speech recognition [39, 40]. Figure 7 shows the fit of cryptocurrencies to the LSTMs model. Furthermore, LSTMs have been used in analysing time series data for various purposes such as increasing load shedding, predicting instantaneous load and predicting deformation in concrete dams. Their ability to collect temporal correlations and model complex time series data has made them very useful models for modelling data in areas such as engineering and energy management. A Comprehensive Comparative Study OF Machine Learning Models for Predicting Cryptocurrency 219 Fig. 7 (a) of constituent Aave. (b) of constituent BinanceCoin. (c) of constituent Bitcoin. (d) of constituent Cardano. (e) of constituent Cosmos. (f) of constituent Dogecoin. (g) of constituent Ethereum. (h) of constituent Solana. (i) of constituent Tether. (j) of constituent Tron. (k) of constituent USDCoin. (l) of constituent XRP. Source: Figure by authors 3.8. Gradient Boosting Machines (GBM) Model Gradient Boosting Machines (GBM) model is defined as a machine learning method that creates robust prediction models by combining the weak prediction model [41]. CPM is used in areas such as estimating credit risks [42], estimating bank failure [43], and estimating the value of cryptocurrency [44]. Figure 8 shows the fit of cryptocurrencies to the GBM model. Fig. 8 (a) of constituent Aave. (b) of constituent BinanceCoin. (c) of constituent Bitcoin. (d) of constituent Cardano. (e) of constituent Cosmos. (f) of constituent Dogecoin. (g) of constituent Ethereum. (h) of constituent Solana. (i) of constituent Tether. (j) of constituent Tron. (k) of constituent USDCoin. (l) of constituent XRP. Source: Figure by authors 220 Y. A. ÜNVAN, C. ERGENÇ 3.9. XGBoost Model The XGBoost models machine learning approach, commonly referred to as Extreme Gradient Boosting, is widely used in various areas of academic literature [45]. The theoretical infrastructure of the XGBoost model is based on the application of boosting techniques to create a robust prediction model. The XGBoost model is used in areas such as credit card fraud detection [46], corporate financial management risk model creation [47], and stock market volatility [48]. Figure 9 shows the fit of cryptocurrencies to the XGBoost model. Fig. 9 (a) of constituent Aave. (b) of constituent BinanceCoin. (c) of constituent Bitcoin. (d) of constituent Cardano. (e) of constituent Cosmos. (f) of constituent Dogecoin. (g) of constituent Ethereum. (h) of constituent Solana. (i) of constituent Tether. (j) of constituent Tron. (k) of constituent USDCoin. (l) of constituent XRP. Source: Figure by authors 3.10. LightGBM Model LightGBM models short for Light Gradient Boosting Machine, is a highly effective machine learning model that can be applied in multiple domains [49]. Figure 10 shows the fit of cryptocurrencies to the LightGBM model. LightGBM is a machine learning technique that uses boosting concepts to create an accurate predictive model, similar to the XGBoost model [50]. LightGBM has applications in predicting stock and cryptocurrency prices [51-53]. 3.11. CatBoost Model The CatBoost models algorithm has attracted much attention due to its ability to efficiently process categorical data and deliver outstanding performance in various applications. CatBoost algorithm uses gradient boosting [54]. According to Prokhorenkova et al. (2018), the CatBoost technique has shown better results in terms of quality compared to existing gradient boosting applications on widely recognised datasets [55]. CatBoost algorithm is used in the field of finance, such as comparing the performance of stock market strategies [56] and predicting the trends of stock indexes [57]. Figure 11 shows the fit of cryptocurrencies to the CatBoost model. A Comprehensive Comparative Study OF Machine Learning Models for Predicting Cryptocurrency 221 Fig. 10 (a) of constituent Aave. (b) of constituent BinanceCoin. (c) of constituent Bitcoin. (d) of constituent Cardano. (e) of constituent Cosmos. (f) of constituent Dogecoin. (g) of constituent Ethereum. (h) of constituent Solana. (i) of constituent Tether. (j) of constituent Tron. (k) of constituent USDCoin. (l) of constituent XRP. Source: Figure by authors CatBoost has shown to be a powerful and versatile gradient boosting method, demonstrating its effectiveness in handling categorical data, achieving outstanding prediction accuracy, and being applicable across several domains. The tool's ability to handle complex data types and provide understandable insights makes it a valuable asset for predictive modeling and interdisciplinary research. Fig. 11 (a) of constituent Aave. (b) of constituent BinanceCoin. (c) of constituent Bitcoin. (d) of constituent Cardano. (e) of constituent Cosmos. (f) of constituent Dogecoin. (g) of constituent Ethereum. (h) of constituent Solana. (i) of constituent Tether. (j) of constituent Tron. (k) of constituent USDCoin. (l) of constituent XRP. Source: Figure by authors 222 Y. A. ÜNVAN, C. ERGENÇ 4. RESULTS In this section, the results obtained using the methods of Decision Trees, Random Forests, K-Nearest Neighbors (KNN), Gradient Boosting Machine (GBM), LightGBM, XGBoost, CatBoost, Artificial Neural Networks (ANN), Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Long Short-Term Memory networks (LSTMs) have been examined. The results have been compared using the MSE, RMSE, and MAE criteria. The mathematical expressions are given below. MSE = 21 ˆ( )i iY Y n − (1) RMSE = 21 ˆ( )i iY Y n − (2) MAE = 1 ˆ iY Y n − (3) The MSE, RMSE, and MAE values obtained for the cryptocurrencies Aave, BinanceCoin, Bitcoin, Cardano, Cosmos, Dogecoin, Ethereum, Solana, Tether, Tron, USDCoin, and XRP are provided in Table 4 and Table 5. Table 4 MSE, RMSE and MAE value for Aave, BinanceCoin, Bitcoin, Cardano, Cosmos, Dogecoin, Ethereum, Solana, Tether, Tron, USDCoin, and XRP Aave BinanceCoin MSE RMSE MAE MSE RMSE MAE Decision Trees 418.708 20.462 13.388 54.428 7.377 1.865 Random Forests 229.562 15.151 9.78 20.659 4.545 1.398 KNN 463.522 21.529 15.641 43.932 6.628 2.309 GBM 275.487 16.597 11.52 17.365 4.167 1.399 LightGBM 285.093 16.884 12.097 24.586 4.958 1.517 XGBoost 33.012 5.745 3.647 5.822 2.412 0.758 CatBoost 1013.619 31.837 21.028 66507.27 257.89 172.51 ANN 316.178 17.781 13.629 14.794 3.846 1.416 CNNs 0.0007 0.027 0.017 0.014 0.121 0.079 RNNs 0.0034 0.058 0.042 0.109 0.33 0.215 LSTMs 0.015 0.125 0.107 0.02 0.144 0.096 Bitcoin Cardano MSE RMSE MAE MSE RMSE MAE Decision Trees 76726.6 276.995 109.151 0.001 0.037 0.012 Random Forests 53947.48 232.265 90.785 0.0004 0.021 0.008 KNN 248067.6 498.063 213.448 0.0011 0.033 0.015 GBM 73052.82 270.282 118.201 0.0004 0.02 0.008 LightGBM 115722.3 340.179 113.87 0.0008 0.029 0.01 XGBoost 81810.14 286.02 88.537 0.0001 0.01 0.004 CatBoost 3.32E+08 18207.52 10177.2 0.328 0.573 0.573 ANN 66318.21 257.523 117.74 0.0004 0.021 0.012 CNNs 0.0001 0.012 0.008 0.0018 0.042 0.034 RNNs 0.002 0.044 0.024 0.008 0.094 0.067 LSTMs 0.0022 0.047 0.024 0.005 0.073 0.051 A Comprehensive Comparative Study OF Machine Learning Models for Predicting Cryptocurrency 223 Cosmos Dogecoin MSE RMSE MAE MSE RMSE MAE Decision Trees 0.469 0.685 0.355 0.00001 0.003 0.0005 Random Forests 0.192 0.438 0.238 0.00006 0.002 0.0004 KNN 0.418 0.646 0.388 0.00004 0.006 0.0009 GBM 0.2145 0.463 0.248 0.000008 0.002 0.0004 LightGBM 0.213 0.461 0.265 0.00006 0.008 0.0011 XGBoost 0.039 0.198 0.112 0.00005 0.002 0.0003 CatBoost 104.81 10.237 8.834 0.019 0.137 0.056 ANN 0.117 0.342 0.201 0.00001 0.004 0.0013 CNNs 0.003 0.052 0.041 0.031 0.177 0.071 RNNs 0.013 0.115 0.099 0.0096 0.098 0.039 LSTMs 0.0022 0.047 0.039 0.0009 0.031 0.012 Table 5 MSE, RMSE and MAE value for Aave, BinanceCoin, Bitcoin, Cardano, Cosmos, Dogecoin, Ethereum, Solana, Tether, Tron, USDCoin, and XRP Ethereum Solana MSE RMSE MAE MSE RMSE MAE Decision Trees 547.753 23.404 9.83 1.101 1.049 0.564 Random Forests 376.747 19.409 8.327 0.626 0.791 0.419 KNN 1379.609 37.143 16.824 1.986 1.409 0.67 GBM 386.734 19.665 8.967 0.906 0.952 0.524 LightGBM 444.698 21.087 8.776 0.929 0.964 0.964 XGBoost 211.86 14.555 6.643 0.917 0.958 0.283 CatBoost 835884.7 914.267 519.195 254.684 15.958 14.328 ANN 309.621 17.596 8.443 0.697 0.835 0.474 CNNs 0.0005 0.023 0.012 0.0021 0.046 0.029 RNNs 0.0002 0.014 0.007 0.004 0.064 0.037 LSTMs 0.0015 0.039 0.019 0.016 0.127 0.121 Tether Tron MSE RMSE MAE MSE RMSE MAE Decision Trees 0.00002 0.005 0.002 0.00002 0.005 0.0016 Random Forests 0.00001 0.003 0.001 0.00001 0.003 0.0012 KNN 0.00001 0.003 0.001 0.00003 0.005 0.0019 GBM 0.00001 0.003 0.001 0.00002 0.004 0.0013 LightGBM 0.00001 0.003 0.001 0.00003 0.005 0.0014 XGBoost 0.00001 0.003 0.001 0.00001 0.003 0.0007 CatBoost 0.000006 0.002 0.001 0.0002 0.015 0.008 ANN 0.0005 0.022 0.016 0.00003 0.006 0.0033 CNNs 0.0002 0.0168 0.0108 0.0005 0.022 0.015 RNNs 0.017 0.132 0.095 0.0002 0.014 0.01 LSTMs 0.008 0.089 0.072 0.0008 0.029 0.017 USDCoin XRP MSE RMSE MAE MSE RMSE MAE Decision Trees 0.00002 0.005 0.002 0.002 0.049 0.011 Random Forests 0.00001 0.003 0.001 0.001 0.033 0.008 KNN 0.00001 0.003 0.001 0.002 0.050 0.014 GBM 0.00001 0.003 0.001 0.001 0.038 0.009 LightGBM 0.00003 0.003 0.001 0.005 0.072 0.012 XGBoost 0.00001 0.003 0.001 0.002 0.051 0.008 CatBoost 0.00002 0.001 0.001 0.005 0.071 0.034 ANN 0.00007 0.027 0.019 0.001 0.038 0.012 CNNs 0.00005 0.022 0.017 0.0003 0.019 0.0115 RNNs 0.002 0.046 0.041 0.0002 0.014 0.008 LSTMs 0.003 0.056 0.054 0.0003 0.018 0.0113 224 Y. A. ÜNVAN, C. ERGENÇ When the results of Aave and BinanceCoin were examined, it was seen that XGBoost showed the best performance. When MSE values are examined, it is seen that the best performing model for Bitcoin is Random Forest and XGBoost. It is seen that the CatBoost model shows the worst performance for Bitcoin. It can be said that all prediction models for Cardano, Cosmos and Dogecoin give good results. However, similar to the Bitcoin results, it can be said that Random Forest and XGBoost performed well in Cardano, Cosmos and Dogecoin. When Ethereum values are examined, it is seen that the best performing model is Random Forest. On the other hand, it is seen that the KNN model shows the worst performance for Ethereum. When Solana values are examined, it is seen that the Random Forest model performs well, similar to other cryptocurrencies. After Random Forest, the best performing models are GBM, XBoost and LightGBM, respectively. It is possible to say that Tether performs well in all models. It can be said that the reason for this is due to the stable nature of Tether as a stablecoin. Additionally, when Tether's graphics are examined, it is seen that it fits all models well. It was concluded that CatBoost, Random Forests, KNN, GBM, LightGBM and XGBoost models fit very well, respectively. When Tron and USDCoin are examined, it is seen that the performance of the models is good. The worst performing model for Tron appears to be CNNs. It is seen that the best performing models are Random Forests and XGBoost. For USDCoin, Random Forests, KNN, GBM and XGBoost models appear to show the best results. It can be seen that the worst performing model is LSTMs. When XRP is examined, it is seen that there are large differences between the performances of the models. CNNs, RNNs and LSTMs appear to perform best. It can be seen that LightGBM and CatBoost perform the worst. Models such as ANN, CNNs, RNNs and LSTMs show different performances in different cryptocurrencies. It can be interpreted that CNNs perform well for Aave, BinanceCoin and Bitcoin. It appears to perform worse for Cardano, Cosmos and Dogecoin. It can be said that RNNs and LSTM models generally perform well, but are not the best performing models in the cryptocurrencies examined. 5. CONCLUSION In this study, Decision Trees, Random Forests, KNN, GBM, LightGBM, XGBoost, CatBoost, ANN were used to predict the closing price of Aave, BinanceCoin, Bitcoin, Cardano, Cosmos, Dogecoin, Ethereum, Solana, Tether, Tron, USDCoin and Models such as CNNs, RNNs and LSTMs have been used. The efficiencies of these models were compared using MSE, RMSE and MAE criteria. The results show that there is no single good model for all cryptocurrencies. Different models appear to fit well in different cryptocurrencies. Stablecoins such as Tether appear to fit well with many models. The accuracy of deep learning models such as ANNs, CNNs, RNNs, and LSTMs appears to be strong on cryptocurrencies such as Aave, BinanceCoin, and Bitcoin. Although classical machine learning models such as XGBoost and Random Forests are mostly reliable, it has been concluded that the choice of model should depend on the different characteristics of each cryptocurrency. While our study provides valuable insights into the performance of predictive models for cryptocurrencies, it is important to acknowledge its limitations. In the beginning, the cryptocurrency market is inherently volatile, and the success of the models may be affected by rapidly evolving external variables. Future research in this area could explore: Investigating the development of real-time predictive models that can adapt to rapidly changing market conditions. Integrating external data sources, such as market sentiment analysis or news sentiment, to enhance predictive accuracy. Assessing the risks A Comprehensive Comparative Study OF Machine Learning Models for Predicting Cryptocurrency 225 associated with cryptocurrency investments, considering the limitations of predictive models. This study builds on and extends the findings of previous research in the field of cryptocurrency prediction, reinforcing the idea that model selection should be tailored to the specific characteristics of each cryptocurrency. Our results are consistent with the broader consensus that cryptocurrency prediction is a multifaceted challenge and that no single model can excel in this complex environment. As a result, this research serves as a valuable reference for investors, policymakers, and researchers who want to delve into the intricacies of cryptocurrency price prediction. While there is no crystal ball in the cryptocurrency world, our study sheds light on various machine learning models that can help make informed decisions in this ever-evolving environment. REFERENCES [1] F. Alonso and M. Á. Sicilia, "Cryptocurrency Curated News Event Database From GDELT," Research Square (Research Square), Oct. 2022. [2] F. D’Amario and Milos Ciganovic, "Forecasting Cryptocurrencies Log-Returns: a LASSO-VAR and Sentiment Approach," arXiv (Cornell University), Sep. 2022. [3] S. A. Manavi, G. Jafari, S. Rouhani, and M. Ausloos, "Demythifying the belief in cryptocurrencies decentralized aspects. A study of cryptocurrencies time cross-correlations with common currencies, commodities and financial indices," Physica A: Statistical Mechanics and its Applications, vol. 556, p. 124759, Oct. 2020. [4] L. J. Tjahyana, "Studi Netnografi Pola Komunikasi Jaringan Komunitas Cryptocurrency Dogecoin Pada Twitter," Jurnal Komunikatif, vol. 10, no. 1, pp. 16–37, Jul. 2021. [5] S. Ichsani and N. S. Mahendra, "Return and Risk Analysis on Cryptocurrency Assets," Kontigensi : Jurnal Ilmiah Manajemen, vol. 10, no. 1, pp. 149–160, Jun. 2022. [6] B. Ebner, L. Eid, and B. Klar, "Cauchy or not Cauchy? New goodness-of-fit tests for the Cauchy distribution," arXiv (Cornell University), Jun. 2021. [7] R. Kher, S. Terjesen, and C. Liu, "Blockchain, Bitcoin, and ICOs: a review and research agenda," Small Business Economics, Jan. 2020. [8] Q. A. Al-Haija and A. A. Alsulami, "High Performance Classification Model to Identify Ransomware Payments for Heterogeneous Bitcoin Networks,” Electronics, vol. 10, no. 17, p. 2113, Aug. 2021. [9] P. K. Singh, A. K. Pandey, and S. C. Bose, “A new grey system approach to forecast closing price of Bitcoin, Bionic, Cardano, Dogecoin, Ethereum, XRP Cryptocurrencies," Quality & Quantity, Jul. 2022. [10] L. H. S. Fernandes, E. Bouri, J. W. L. Silva, L. Bejan, and F. H. A. de Araujo, "The resilience of cryptocurrency market efficiency to COVID-19 shock," Physica A: Statistical Mechanics and its Applications, p. 128218, Oct. 2022. [11] J.-F. Pietschmann and M. Schlottbom, "Data driven gradient flows," arXiv (Cornell University), May 2022. [12] L. Juškaitė and L. Gudelytė-Žilinskienė, "Investigation of the Feasibility of Including Different Cryptocurrencies in the Investment Portfolio for its Diversification," Journal Business, Management and Economics Engineering, vol. 20, no. 01, pp. 172–188, May 2022. [13] K. He, Q. Yang, L. Ji, J. Pan, and Y. Zou, "Financial Time Series Forecasting with the Deep Learning Ensemble Model," Mathematics, vol. 11, no. 4, p. 1054, Jan. 2023. [14] L. Rokach and O. Z. Maimon, Data mining with decision trees: theory and applications. New Jersey Etc.: World Scientific, Cop, 2015. [15] B. Gupta, A. Rawat, A. Jain, A. Arora, and N. Dhami, "Analysis of Various Decision Tree Algorithms for Classification in Data Mining," International Journal of Computer Applications, vol. 163, no. 8, pp. 15– 19, Apr. 2017. [16] A. Singh, "A New Investment Opportunity: Bitcoin & Ethereum Cryptocurrency," International Journal of Scientific Research in Engineering and Management, vol. 06, no. 10, Oct. 2022. [17] A. A. Oyedele, A. O. Ajayi, L. O. Oyedelec, S. A. Bello, and K. O. Jimoh, "Performance evaluation of deep learning and boosted trees for cryptocurrency closing price prediction," Expert Systems with Applications, p. 119233, Nov. 2022. [18] H. Anantharaman, A. Mubarak, and B.T Shobana, "Modelling an Adaptive e-Learning System Using LSTM and Random Forest Classification," Nov. 2018. [19] M. Fratello and R. Tagliaferri, "Decision Trees and Random Forests," pp. 374–383, Jan. 2016. 226 Y. A. ÜNVAN, C. ERGENÇ [20] C. Liu, Y. Chan, S. H. Alam Kazmi, and H. Fu, "Financial Fraud Detection Model: Based on Random Forest," International Journal of Economics and Finance, vol. 7, no. 7, Jun. 2015. [21] Z. Rustam and G. S. Saragih, "Predicting Bank Financial Failures using Random Forest," IEEE Xplore, May 01, 2018. [22] A. Inamdar, A. Bhagtani, S. Bhatt, and P. M. Shetty, "Predicting Cryptocurrency Value using Sentiment Analysis," IEEE Xplore, May 01, 2019. https://ieeexplore.ieee.org/abstract/document/9065838 (accessed Mar. 30, 2022). [23] Z. Zhang, "Introduction to machine learning: k-nearest neighbors," Annals of Translational Medicine, vol. 4, no. 11, pp. 218–218, Jun. 2016. [24] Y. Lee, C.-H. Wei, and K.-C. Chao, "Non-parametric machine learning methods for evaluating the effects of traffic accident duration on freeways," Archives of Transport, vol. 43, no. 3, pp. 91–104, Sep. 2017. [25] Y. A. Ünvan, and C. Ergenç "Stock Market Forecasting with Machine Learning: The case of BIST-100 Index," International Research Journal of Modernization in Engineering Technology and Science, Jun. 2023. [26] J. Behera, A. K. Pasayat, H. Behera, and P. Kumar, "Prediction based mean-value-at-risk portfolio optimization using machine learning regression algorithms for multi-national stock markets," Engineering Applications of Artificial Intelligence, vol. 120, p. 105843, Apr. 2023. [27] M. Chen, U. Challita, W. Saad, C. Yin, and M. Debbah, "Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial, " IEEE Communications Surveys & Tutorials, vol. 21, no. 4, pp. 3039–3071, 2019. [28] M. C. Nwadiugwu, "Neural Networks, Artificial Intelligence and the Computational Brain," arXiv.org, Dec. 25, 2020. https://arxiv.org/abs/2101.08635 [29] V. Uraikul, C. W. Chan, and P. Tontiwachwuthikul, "Artificial intelligence for monitoring and supervisory control of process systems," Engineering Applications of Artificial Intelligence, vol. 20, no. 2, pp. 115–131, Mar. 2007. [30] S. Islam et al., "A comprehensive survey on applications of transformers for deep learning tasks," Expert Systems with Applications, pp. 122666–122666, Nov. 2023. [31] S. Zhang et al., "Architectural Complexity Measures of Recurrent Neural Networks," Neural Information Processing Systems, 2016. [32] P. B. Weerakody, K. W. Wong, G. Wang, and W. Ela, "A review of irregular time series data handling with gated recurrent neural networks," Neurocomputing, vol. 441, pp. 161–178, Jun. 2021. [33] A. Shewalkar, D. Nyavanandi, and S. A. Ludwig, "Performance Evaluation of Deep Neural Networks Applied to Speech Recognition: RNN, LSTM and GRU," Journal of Artificial Intelligence and Soft Computing Research, vol. 9, no. 4, pp. 235–245, Oct. 2019. [34] E. Brophy, Z. Wang, Q. She, and T. Ward, "Generative adversarial networks in time series: A survey and taxonomy," arXiv:2107.11098 [cs], Jul. 2021, Available: https://arxiv.org/abs/2107.11098 [35] A. Khan, A. Sohail, U. Zahoora, and A. S. Qureshi, "A survey of the recent architectures of deep convolutional neural networks,” Artificial Intelligence Review, vol. 53, Apr. 2020. [36] X. Lu et al., "Fault diagnosis for photovoltaic array based on convolutional neural network and electrical time series graph," Energy Conversion and Management, vol. 196, pp. 950–965, Sep. 2019. [37] M. Al-Smadi, B. Talafha, M. Al-Ayyoub, and Y. Jararweh, "Using long short-term memory deep neural networks for aspect-based sentiment analysis of Arabic reviews," International Journal of Machine Learning and Cybernetics, vol. 10, no. 8, pp. 2163–2175, Mar. 2018. [38] G. Liu and J. Guo, "Bidirectional LSTM with attention mechanism and convolutional layer for text classification," Neurocomputing, vol. 337, pp. 325–338, Apr. 2019. [39] L. Yao and Y. Guan, "An Improved LSTM Structure for Natural Language Processing," IEEE Xplore, Dec. 01, 2018. https://ieeexplore.ieee.org/document/8690387 (accessed Dec. 25, 2021). [40] M. K. Nammous and K. Saeed, "Natural Language Processing: Speaker, Language, and Gender Identification with LSTM," Advances in intelligent systems and computing, pp. 143–156, Jan. 2019. [41] N. Aziz, E. A. P. Akhir, I. A. Aziz, J. Jaafar, M. H. Hasan, and A. N. C. Abas, "A Study on Gradient Boosting Algorithms for Development of AI Monitoring and Prediction Systems," IEEE Xplore, Oct. 01, 2020. https://ieeexplore.ieee.org/document/9247843 (accessed Sep. 13, 2022). [42] Y. C. Chang, K.-H. Chang, and G.-J. Wu, "Application of eXtreme gradient boosting trees in the construction of credit risk assessment models for financial institutions," Applied Soft Computing, vol. 73, pp. 914–920, Dec. 2018. [43] P. Carmona, F. Climent, and A. Momparler, "Predicting failure in the U.S. banking sector: An extreme gradient boosting approach," International Review of Economics & Finance, vol. 61, pp. 304–323, May 2019. [44] J. S. Heo, D.-H. Kwon, J.-B. Kim, Y.-H. Han, and C.-H. An, "Prediction of Cryptocurrency Price Trend Using Gradient Boosting," KIPS Transactions on Software and Data Engineering, vol. 7, no. 10, pp. 387–396, Oct. 2018. https://arxiv.org/abs/2101.08635 https://arxiv.org/abs/2107.11098 A Comprehensive Comparative Study OF Machine Learning Models for Predicting Cryptocurrency 227 [45] T. Kavzoglu and A. Teke, "Predictive Performances of Ensemble Machine Learning Algorithms in Landslide Susceptibility Mapping Using Random Forest, Extreme Gradient Boosting (XGBoost) and Natural Gradient Boosting (NGBoost)," Arabian Journal for Science and Engineering, Jan. 2022. [46] T. R. Noviandy, G. M. Idroes, A. Maulana, I. Hardi, E. S. Ringga, and Rinaldi Idroes, "Credit Card Fraud Detection for Contemporary Financial Management Using XGBoost-Driven Machine Learning and Data Augmentation Techniques," Indatu Journal of Management and Accounting, vol. 1, no. 1, pp. 29–35, Sep. 2023. [47] R. Qin, "The Construction of Corporate Financial Management Risk Model Based on XGBoost Algorithm," Journal of Mathematics, vol. 2022, pp. 1–8, Apr. 2022. [48] Y. Wang and Y. Guo, "Forecasting method of stock market volatility in time series data based on mixed model of ARIMA and XGBoost," China Communications, vol. 17, no. 3, pp. 205–221, Mar. 2020. [49] F. Alzamzami, M. Hoda, and A. E. Saddik, "Light Gradient Boosting Machine for General Sentiment Classification on Short Texts: A Comparative Evaluation," IEEE Access, vol. 8, pp. 101840–101858, 2020. [50] A. Shehadeh, O. Alshboul, R. E. Al Mamlook, and O. Hamedat, "Machine learning models for predicting the residual value of heavy construction equipment: An evaluation of modified decision tree, LightGBM, and XGBoost regression," Automation in Construction, vol. 129, p. 103827, Sep. 2021. [51] X. Sun, M. Liu, and Z. Sima, "A novel cryptocurrency price trend forecasting model based on LightGBM," Finance Research Letters, Dec. 2018. [52] R. Xu, Y. Chen, T. Xiao, J. Wang, and X. Wang, "Predicting the trend of stock index based on feature engineering and CatBoost model," International Journal of Financial Engineering, p. 2150027, May 2021. [53] L. Tian, L. Feng, L. Yang, and Y. Guo, "Stock price prediction based on LSTM and LightGBM hybrid model," The Journal of Supercomputing, Feb. 2022. [54] A. A. Ibrahim, R. L., M. M., R. O., and G. A., "Comparison of the CatBoost Classifier with other Machine Learning Methods," International Journal of Advanced Computer Science and Applications, vol. 11, no. 11, 2020. [55] L. Prokhorenkova, G. Gusev, A. Vorobev, A. V. Dorogush, and A. Gulin, "CatBoost: unbiased boosting with categorical features," Neural Information Processing Systems, 2018. https://proceedings.neurips.cc/paper/2018/ hash/14491b756b3a51daac41c24863285549-Abstract.html [56] Mousavi and H. Nikoomaram, "A comparative study of the performance of Stock trading strategies based on LGBM and CatBoost algorithms.," International Journal of Finance & Managerial Accounting, vol. 7, no. 26, pp. 63–75, Jul. 2022 [57] R. Xu, Y. Chen, T. Xiao, J. Wang, and X. Wang, "Predicting the trend of stock index based on feature engineering and CatBoost model," International Journal of Financial Engineering, p. 2150027, May 2021. https://proceedings.neurips.cc/paper/2018/hash/14491b756b3a51daac41c24863285549-Abstract.html https://proceedings.neurips.cc/paper/2018/hash/14491b756b3a51daac41c24863285549-Abstract.html