International Journal of Energetica (IJECA) https://www.ijeca.info ISSN: 2543-3717 Volume 8. Issue 1. 2023 Page 31-37 This open access article is licensed under the CC BY-NC license (https://creativecommons.org/licenses/by-nc/4.0/) Page 31 Developed nonlinear model based on bootstrap aggregated neural networks for predicting global hourly scale horizontal irradiance Abdennasser Dahmani 1,2* , Yamina Ammi 2 , Salah Hanini 2 , Zied Driss 3 1 Department of Mechanical Engineering, Faculty of Science and Technology, GIDD Industrial Engineering and Sustainable Development Laboratory, University of Relizane, Bourmadia, 48000, Relizane, ALGERIA 2 Laboratory of Biomaterials and Transport Phenomena (LBMPT), University of Medea, ALGERIA 3 Laboratory of Electro-Mechanic Systems, National School of Engineers of Sfax, University of Sfax, TUNISIA *Corresponding author: Email: Dahmani.abdennasser@univ-relizane.dz Abstract – This research study examines the use of two models of artificial intelligence based on a single neural network (SNN) and bootstrap aggregated neural networks (BANN) for the prediction value of hourly global horizontal irradiance (GHI) received over one year in Tamanrasset City (Southern Algeria). The SNN and BANN were created using overall data points. To improve the accuracy and durability of neural network models generated with a limited amount of training data, stacked neural networks are developed. To create many subsets of training data, the training dataset is re-sampled using bootstrap re-sampling with replacement. A neural network model is created for each set of training datasets. A stacked neural network is created by combining multiple individual neural networks (INN). For the testing phase, higher correlation coefficients (R = 0.9580) were discovered when experimental global horizontal irradiance (GHI) was compared to predicted global horizontal irradiance (GHI). The performance of the models (INN, BANN, and SNN) demonstrates that models generated with BANN are more accurate and robust than models built with individual neural networks (INN) and (SNN). Keywords: Horizontal irradiance, Single neural networks, Bootstrap aggregated, Prediction. Received: 31/04/2023 – Revised: 25/05/2023 – Accepted: 05/06/2023 I. Introduction Renewable energy sources are universally acknowledged as being essential for any country's long-term energy growth. This strategic focus on energy resource diversification has been spurred by the threats of not having adequate and secure sources of energy at low rates, as well as the environmental harm caused by the usage of fossil fuels [1,2]. Solar energy is important both economically and environmentally since it protects the environment and promotes a healthy society [3]. Algeria is located in the Sunbelt, an area rich in solar energy potential. The Algerian Ministry of Energy and Mining approved Law in 2011, which lays the framework for the country's long-term renewable energy policy. Algeria's geographical location gives it a huge edge in terms of solar energy potential. On the whole of Algeria's national territory, the annual sunlight length surpasses 3,600 hours, and on high plateaus and in the Sahara, it may even approach 3,900 hours [4]. Unfortunately, due to the cost-effectiveness of the measurement equipment (solarimeters/ pyranometers) and systems involved, solar irradiation measurements are difficult to obtain in many locations in Algeria (charge, cost, maintenance, calibration requirement). Despite the presence of a number of weather stations in various sites around Algeria, measurements are not always available due to recording problems caused by significant power outages, particularly during the summer, or because dataset is limited. As a result, it is far more significant to use complicated procedures to accurately predicted solar radiation (SR) using more readily available meteorological data [5]. Recently, artificial intelligence techniques have received much method that offers an alternative approach to modeling as they can deal with difficult and poorly https://www.ijeca.info/ https://creativecommons.org/licenses/by-nc/4.0/ Abdennasser Dahmani et al. / International Journal of Energetica (IJECA) Vol. 8, N°1, 2023, pp. 31-37 Page 32 defined problems in many scientific domains. In the meteorological field, several authors have investigated to apply of artificial neural network models for modeling the prediction of (SR) as can be seen from the following literature: Rehman and Mohandes. Used neural network technology to predict the values of solar radiation falling on the city of Abha in Saudi Arabia from 1998 to 2004 using three models based on temperature and relative humidity, the findings reveal that neural networks can accurately estimate GSR from temperature and relative humidity for sites where temperature and humidity data are available [6]. A research paper applied radial basis function technology for a good and optimal prediction of the hourly solar radiation values in the Saudi Arabian city of Menorah using only two parameters as input, the temperature, and length of sunshine. However, the results were good [7]. The transfer model was constructed using artificial neural networks (ANN) to predict the differences between tilted and horizontal irradiance at three different sites in Taiwan over a one-year. The experimental findings reveal that the suggested ANNs with differential outputs may significantly increase estimation [8]. One study applied the Datasets from the Meteorological Agency and the Turkish state for predicting solar radiation parameters over a four-year period for seven cities in the Anatolian-Mediterranean region of Turkey. The obtained results show that the method can be used by researchers or scientists to design efficient solar devices. The number of input parameters was also found to be the most effective parameter for estimating future solar radiation data [9]. The ability of a multilayer perceptron to provide very short (five-minute) exposure readiness estimates over a two-year period was investigated in Bouzareah –Algeria [10]. The inclined irradiation using data from thirteen places in Algeria's various climate zones, eight parameters were used as input. The current best neural network model has a root mean square error low 6 Wh/m 2 [11]. A research paper applied twenty-two empirical models, artificial neural network techniques, machine learning methods, and tree- based ensemble methods tested to estimate daily global solar irradiance in five cities in Morocco. The results show that the empirical results perform well and are more robust than other intelligent models [12]. Another study Focused on using neural networks in multi-location to predict the GSR over Italy using non-meteorological data such as geographic locations [13]. Another research paper tested twenty empirical models to predict (GSR) in Six meteorological stations on the Fiji Islands [14]. The Bootstrap Aggregated Neural Networks method was suggested by a research group [15]. It is a method of improving the generalization ability of a model by training multiple neural networks and then combining them. This strategy is successful and easy to use, having to be applied in a variety of situations [16]. Stacked neural networks have been shown to generalize better than individual neural networks [17,18]. Several strategies proposed to exploit the strengths of artificial neural network models [19]. The aim of this research work was to improve a bootstrap aggregated-based neural networks (BANN) model to predict the hourly global horizontal irradiance (GHI) received over one year in Tamanrasset City (Southern Algeria) based climatological and meteorological parameters. To the best of our knowledge, there is no reported study of model prediction (GHI) using bootstrap-based ANNs in the literature. II. Material and Methods II.1. Study Area and Data Collection Tamanrasset is the largest Algerian province with an area of 336000 km 2 , about 15 % of the country's area located in the Great South-East. which benefits from the abundant sunlight of more than 3600 hours per year with an average daily irradiation more than 6 kWh/m 2 /day [20]. In this research work, the meteorological database (DB) were collected from Eppley PSP pyranometer for predicting hourly global horizontal irradiance (GHI) in Tamanrasset region over one year situated in the west of Algeria with latitude: +22.783°, longitude: + 5.517°, and altitude of 1377 m. It is characterized by higher radiation than other Algerian ground places. The geographical coordinates of the study site are located on the map of Algeria (see Figure 1). Figure 1. Position of the study area: a) Algeria country Abdennasser Dahmani et al. / International Journal of Energetica (IJECA) Vol. 8, N°1, 2023, pp. 31-37 Page 33 Figure 1. Position of the study area: b) Tamanrasset area. c) Elevation of the city [21] II.2. Single Neural Network Single Neural Networks (SNN) have been developed as a model of how the human brain processes information, learning SNNs, and extracting developmental performance from past operational information. Generally, there are two cases of neural network model learning, supervised learning and unsupervised learning. In the first case of supervised learning, a set of training input vectors is trained with a corresponding set of target vectors to tune the weights in (NN). In the second case, no object vectors are specified in unsupervised learning of one or more hidden layers, whose computation nodes are correspondingly named hidden neurons of hidden layers [22,23]. The structure of the neural network contains three layers called input, hidden, and output layers. The inputs signal received from external sources bias (b) are multiplied by weights (W). Depending on the neural network activation function, if the results of multiplying y surpass the threshold, the signal will be released and sent to the output [5]. Figure 2 shows the architecture of feed forward single neural network. Figure 2. Multi-layer perceptron, feed-forward single neural network II.3. Bootstrap aggregated neural networks (BANN) A most significant method to enhance the strength of neural network models is to improve a set of neural models, many researcher have studied the multiple of a set of neural networks. The creation of the bootstrap aggregated neural networks model is done by sampling the training database using a function Matlab 2020b software [24,25]. Bootstrap aggregated neural networks (BANN) is used in this research study to improve an accurate model for prediction hourly global horizontal irradiance (GHI) in Tamanrasset city, Figure 3 showed a bootstrap aggregated where neural network model are built to the relationship between inputs-outputs and are then aggregated Individual neural networks (INN) are learned using various training datasets. Figure 3. Structure of Bootstrap aggregated neural networks (BANN) model II.4. Modeling the Neural Networks In this research study, a method based on the optimization and improvement of the structural design of neural networks is advanced. It is based, as defined in Figure 4, on the plan of three neural network models: SNN, INN, and BANN (Stacking of 10, 15, 20, 25, and 30 networks)). The neural network models (SNN and INN) were ameliorated, every neural network (SNN and "10, 15, 20, 25, and 30" individual NN models) have three layers: one input layer with eight neurons for NN in the input layer, the number of neurons in hidden was varied between 3 to 25 neurons. The tangent hyperbolic, the log sigmoid, the sin, and the exponential Activation functions were applied in the hidden layer. The pure-linear Activation function was applied in the output layer. BFGS quasi- Newton training algorithm was applied for trained for all neural network (SNN and INNi) models. The average of the INN outputs gives the (BANN) model. The MATLAB 2020b and STATISTICA software applied for the creation of each model. Abdennasser Dahmani et al. / International Journal of Energetica (IJECA) Vol. 8, N°1, 2023, pp. 31-37 Page 34 Figure 4. Flow diagram for development different models (SNN, INN, and BANN (Stacking of 10, 15, 20, 25, and 30 networks) III. Results and Discussion III.1. Effect of the Division of Database Table 1 displays error (root mean squared error "RMSE") values and coefficient of correlation "R" obtained for the global horizontal irradiance under the influence of the division of the database for the SNN model with division 1 (2694 points for training (80%), 336 for validation phase (10%), and 336 for testing phase (10%)), division 2 (2356 points for training (70%), 505 for validation phase (15%) and 505 for testing phase (15%)), and division 3 (2020 points for training (60%),673 for validation phase (20%), and 673 for testing phase (20%)). It has been observed that the first division represents the greatest result. Table 1. Influence of the division of database Phases Percentage Errors RMSE (Wh/m 2 ) R Division 1 Training phase 80% 87.1485 0.9361 Validation phase 10% 93.1570 0.9241 Testing phase 10% 81.7573 0.9446 Division 2 Training phase 70% 91.6322 0.9287 Validation phase 15% 94.3601 0.9225 Testing phase 15% 89.8774 0.9345 Division 3 Training phase 60% 92.9007 0.9277 Validation phase 20% 92.4126 0.9267 Testing phase 20% 84.8200 0.9381 III.2. Comparison between Different Bootstrap Neural Network Models In order to compare five models of BANN were implemented: stacking 10 networks, stacking 15 networks, stacking 20 networks, stacking 25 networks, and stacking 30 networks. We evaluated the model predictive error (MPE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE) and Standard Error of Prediction (SEP): Where n is the total number of data; is the experimental value, represents the calculated value from the neural network model and is the average value of experimental data. Figure 5 shows the comparison between BANN models in terms of MAE, MPE, RMSE, and SEP. It can be seen that the BANN with stacking 30 nets is more robust than the other BANN stacking models, (MAE = 49.9095 (Wh/m 2 ), (MPE = 10.9726 (Wh/m 2 ), (RMSE = 71.4998 (Wh/m 2 ), and (SEP=13.5391 (Wh/m 2 )) for testing phase. Figure 5. Several type of errors Vs. different stacking neural network models for test datasets III.3. Performance Neural Network (INN and SNN) Models The configurations of the 30 optimized individual neural networks "NN" models used to build the BANN (with stacking 30 nets) and the single neural network "SNN" model. Abdennasser Dahmani et al. / International Journal of Energetica (IJECA) Vol. 8, N°1, 2023, pp. 31-37 Page 35 Table 2 shows the model structure of each neural network "INN" and each neural network "SNN". The networks INN and SNN are clearly incoordinated and have different architectures. 15 individual neural networks use a logarithmic sigmoid (logistic) transfer function and 14 individual neural networks use a tangential hyperbolic (tanh) transfer function; it is the SNN model that brings efficiency, reliability, and robustness the same function, and an exponential transfer function is used in the hidden layer of the neural network in INN29. In the hidden layer, the neural network does not use the sin function. We conclude that two transfer functions (sigmoid and tanh) outperform functions (exponential and sinusoidal), and these results are consistent with those of [26]. Thirty individual neural networks INN have enough layer neurons (10 to 25 neurons) to achieve a good approximation. Table 2. Structures of the optimized NN and SNN models III.4. Comparison between BANN(stacking 30 nets) and SNN (Fig. 6a, b, c, and d) show comparisons of experimental and predicted global horizontal solar irradiance with a consistent vector close to the ideal [i.e. = 1 (slope), = 0 (intercept), R = 1 (determination coefficient).)] match the neural network profile of the SNN ([α, β, R] = [0.8532, 83.3870, 0.9242] for the validation phase, [α, β, R] = [0.8964, 60, 8747, 0.9446] for the test phase, for BANN (stack of 30 networks) ([α, β, R] = [1.0002, -8.8450, 0.9387] for validation stage, [α, β, R] = [1.0060, -9.0290, 0.9580] with in the validation and testing phases) in the validation phase of BANN and SNN models, the slope α is close to 1, and in the testing phase of the two neural network models, it is near to 1. The intercept β is far from 0 in the validation phase, and in the testing phase, for these neural networks (SNN and BANN (Stacking of 30 networks)), the model SNN and BANN regression coefficients are generally considered to be excellent (0.90 ≤ R ≤ 1.00). (a) (b) (c) NNi Neurons number in hidden layer Activation function in hidden layer NNi Neurons number in hidden layer Activation function in hidden layer INN1 13 Tanh INN16 14 Logistic INN2 16 Logistic INN17 25 Logistic INN3 23 Tanh INN18 16 Tanh INN4 10 Tanh INN19 21 Tanh INN5 17 Logistic INN20 21 Logistic INN6 23 Logistic INN21 21 Logistic INN7 22 Tanh INN22 23 Logistic INN8 25 Tanh INN23 25 Logistic INN9 20 Logistic INN24 24 Logistic INN10 21 Logistic INN25 22 Logistic INN11 25 Tanh INN26 25 Tanh INN12 23 Tanh INN27 20 Tanh INN13 21 Tanh INN28 20 Tanh INN14 10 Logistic INN29 22 Expo INN15 18 Logistic INN30 19 Tanh Abdennasser Dahmani et al. / International Journal of Energetica (IJECA) Vol. 8, N°1, 2023, pp. 31-37 Page 36 (d) Figure. 6 Comparison between experimental Vs. calculated global horizontal irradiance(GHI): (a) SNN "validation phase", (b) SNN "testing phase", (c) BANN(Stacking of 30 networks) "validation phase", (d) BANN(Stacking of 30 networks) "testing phase". The two models were compared in terms mentions in section 3 to fit the developed BANN (stacked 30 network) model as a reasonable alternative to the SNN model (RPD). The equation for this parameter (RPD) is as follows: SD is the standard deviation of the experimental data. The comparison between BANN (Stacking of 30 networks) and SNN models is shown in Figure 7. In this comparison, the advantage of the BANN (Stacking of 30 networks) model is proved. It demonstrates the stability and reliability of neural network models (BANN (Stacking of 30 networks) and SNN). The BANN model (Stacking of 30 networks) is more accurate than the SNN model and can calculate hourly Global Horizontal Radiation (GHI). When applied to an experimental database, this comparison confirms the robust predictions of the bootstrap aggregated neural network. Figure. 7 Comparison between SNN Vs. BANN(Stacking of 30 networks) models in terms of the MAE, MPE, RMSE, SEP, and RPD for testing data IV. Conclusion In this research study, two t neural network models (BANN(Stacking of 30 networks) and SNN) have been developed to predict hourly global horizontal irradiance (GHI) in Tamanrasset- Algeria. The comparison between BANN and SNN models revealed the superiority of the BANN model in predicting the hourly global horizontal irradiance (GHI) as it gave the best performances (the RMSE for the test dataset were 71.4998 Wh/m 2 for BANN, 81.7573 Wh/m 2 for SNN and MPE for the test dataset were 10.9726 Wh/m 2 for BANN and 15.4304 Wh/m 2 for SNN). The BANN model has higher precision and can describe the prediction of hourly global horizontal irradiance more accurately compared with the SNN model. The BANN model will be suitable for predicting global horizontal irradiance for other locations and can also be used to install solar-energy systems. In addition, this model can be applied by researchers in terms of site selection and techno-economic performance evaluation of solar energy applications particularly those relying on photovoltaic technologies. Declaration  The authors declare that they have no known financial or non-financial competing interests in any material discussed in this paper.  The authors declare that this article has not been published before and is not in the process of being published in any other journal.  The authors confirmed that the paper was free of plagiarism Acknowledgments The authors gratefully acknowledge the help the University of Relizane throughout this project, and the group of Laboratory of Biomaterials and Transport Phenomena in the University of Medea. References [1] R. T. Nand and A. Raturi, "Feasibility study of a grid connected photovoltaic system for the central region of Fiji," Appl. Sol. Energy, vol. 49, no. 2, 2013, pp. 110. [2] C. A. Agostini, S. Nasirov, and C. Silva, "Solar PV planning toward sustainable development in Chile: challenges and recommendations," J. Environ. Dev., vol. 25, no. 1, 2016, pp. 25–46. Abdennasser Dahmani et al. / International Journal of Energetica (IJECA) Vol. 8, N°1, 2023, pp. 31-37 Page 37 [3] M. Ouria and H. Sevinc, "Evaluation of the potential of solar energy utilization in Famagusta, Cyprus," Sustain. cities Soc., vol. 37, 2018, pp. 189–202. [4] A. B. Stambouli, Z. Khiat, S. Flazi, and Y. Kitamura, "A review on the renewable energy development in Algeria: Current perspective, energy scenario and sustainability issues," Renewable and Sustainable Energy Reviews, vol. 16, no. 7, 2012, pp. 4445–4460. doi: 10.1016/j.rser.2012.04.031. [5] A. Dahmani, Y. Ammi, and S. Hanini, "Neural network for prediction solar radiation in Relizane region (Algeria)-Analysis study." [6] S. Rehman and M. Mohandes, "Artificial neural network estimation of global solar radiation using air temperature and relative humidity," Energy Policy, vol. 36, no. 2, 2008, pp. 571–576. [7] M. Benghanem and A. Mellit, "Radial Basis Function Network-based prediction of global solar radiation data: Application for sizing of a stand-alone photovoltaic system at Al-Madinah, Saudi Arabia," Energy, vol. 35, no. 9, 2010, pp. 3751–3762. doi: 10.1016/j.energy.2010.05.024. [8] H.-Y. Cheng, C.-C. Yu, K.-C. Hsu, C.-C. Chan, M.-H. Tseng, and C.-L. Lin, "Estimating solar irradiance on tilted surface with arbitrary orientations and tilt angles," Energies, vol. 12, no. 8, 2019, pp. 1427. [9] A. Koca, H. F. Oztop, Y. Varol, and G. O. Koca, "Estimation of solar radiation using artificial neural networks with different input parameters for Mediterranean region of Anatolia in Turkey," Expert Syst. Appl., vol. 38, no. 7, 2011, pp. 8756–8762. [10] K. Dahmani, R. Dizene, G. Notton, C. Paoli, C. Voyant, and M. L. Nivet, "Estimation of 5-min time-step data of tilted solar global irradiation using ANN (Artificial Neural Network) model," Energy, vol. 70, 2014, pp. 374–381, doi: 10.1016/j.energy.2014.04.011. [11] M. Laidi, S. Hanini, A. Rezrazi, M. R. Yaiche, A. A. El Hadj, and F. Chellali, "Supervised artificial neural network-based method for conversion of solar radiation data (case study: Algeria)," Theor. Appl. Climatol., vol. 128, no. 1–2, 2016, pp. 439–451, doi: 10.1007/s00704- 015-1720-7. [12] Z. Bounoua, L. O. Chahidi, and A. Mechaqrane, "Estimation of daily global solar radiation using empirical and machine-learning methods: A case study of five Moroccan locations," Sustain. Mater. Technol., vol. 28, 2021, p. e00261. [13] E. F. Alsina, M. Bortolini, M. Gamberi, and A. Regattieri, "Artificial neural network optimization for monthly average daily global solar radiation prediction," Energy Convers. Manag., vol. 120, 2016, pp. 320–329. [14] O. M. Oyewola, T. E. Patchali, O. O. Ajide, S. Singh, and O. J. Matthew, "Global solar radiation predictions in Fiji Islands based on empirical models," Alexandria Eng. J., vol. 61, no. 11, Nov. 2022, pp. 8555–8571, doi: 10.1016/j.aej.2022.01.065. [15] L. K. Hansen and P. Salamon, "Neural network ensembles," IEEE Trans. Pattern Anal. Mach. Intell., vol. 12, no. 10, 1990, pp. 993–1001. [16] J. Zhou, Y. Wu, G. Yan, and Z. Ma, "Solar radiation estimation using artificial neural networks," Taiyangneng Xuebao/Acta Energiae Solaris Sin., vol. 26, no. 4, 2005, pp. 509–512. [17] D. V Sridhar, R. C. Seagrave, and E. B. Bartlett, "Process modeling using stacked neural networks," AIChE J., vol. 42, no. 9, 1996, pp. 2529–2539. [18] J. Zhang, E. B. Martin, A. J. Morris, and C. Kiparissides, "Inferential estimation of polymer quality using stacked neural networks," Comput. Chem. Eng., vol. 21, 1997, pp. S1025–S1030. [19] D. H. Wolpert, "Stacked generalization," Neural networks, vol. 5, no. 2, 1992, pp. 241–259. [20] N. Bailek et al., "A new empirical model for forecasting the diffuse solar radiation over Sahara in the Algerian Big South," Renew. Energy, vol. 117, 2018, pp. 530– 537, doi: 10.1016/j.renene.2017.10.081. [21] L. Achour, M. Bouharkat, O. Assas, and O. Behar, "Hybrid model for estimating monthly global solar radiation for the Southern of Algeria: (Case study: Tamanrasset, Algeria)," Energy, vol. 135, 2017, pp. 526–539, doi: 10.1016/j.energy.2017.06.155. [22] Y. Ammi, L. Khaouane, and S. Hanini, "Stacked neural networks for predicting the membranes performance by treating the pharmaceutical active compounds," Neural Comput. Appl., vol. 33, no. 19, 2021, pp. 12429–12444, doi: 10.1007/s00521-021-05876-0. [23] M. E. Emiroglu, O. Bilhan, and O. Kisi, "Neural networks for estimation of discharge capacity of triangular labyrinth side-weir located on a straight channel," Expert Syst. Appl., vol. 38, no. 1, 2011, pp. 867–874. [24] R. J. Tibshirani and B. Efron, "An introduction to the bootstrap," Monogr. Stat. Appl. Probab., vol. 57, 1993, pp. 1–436. [25] F. N. Osuolale and J. Zhang, "Exergetic optimisation of atmospheric and vacuum distillation system based on bootstrap aggregated neural network models," Exergy A Better Environ. Improv. Sustain. 1 Fundam., 2018, pp. 1033–1046. [26] Y. Ammi, L. Khaouane, and S. Hanini, "A Model Based on Bootstrapped Neural Networks for Modeling the Removal of Organic Compounds by Nanofiltration and Reverse Osmosis Membranes," Arab. J. Sci. Eng., vol. 43, no. 11, 2018, pp. 6271–6284, doi: 10.1007/s13369-018-3484-8.