The Journal of Engineering Research (TJER), Vol. 16, No. 2 (2019) 77-86 MODELLING OF ELECTRICAL SUBMERSIBLE PUMPS FOR PETROLEUM MULTIPHASE FLUIDS, AN INTELLIGENT APPROACH SUPPORTED BY A CRITICAL REVIEW AND EXPERIMENTAL RESULTS M. Mohammadzaheria, * , R. Tafreshi b , Z. Khan c , H. Ziaiefar a , M. Ghodsi a , M. Franchek c , and K. Grigoriadis c a Sultan Qaboos University, Muscat, Oman b Texas A&M University at Qatar, Education City, Doha, Qatar c University of Houston, 4800 Calhoun Road, Houston, USA. ABSTRACT: This paper initially reviews existing empirical models which predict head or pressure increase of two-phase petroleum fluids in electrical submersible pumps (ESPs), then, proposes an alternative model, a fully connected cascade (FCC in short) artificial neural network to serve the same purpose. Empirical models of ESP are extensively in use; while analytical models are yet to be vastly employed in practice due to their complexity, reliance on over-simplified assumptions or lack of accuracy. The proposed FCC is trained and cross-validated with the same data used in developing a number of empirical models; however, the developed model presents higher accuracy than the aforementioned empirical models. The mean of absolute prediction error of the FCC for the experimental data not used in its training, is 68% less than the most accurate existing empirical model. Keywords: Electrical submersible pumps; Empirical models; Cascade artificial neural networks; Multiphase petroleum fluid. الغاطسة الكهربائية للسوائل البترولية المتعددة ، نهج نمذجة المضخات ذكي مدعوم بمراجعة نقدية ونتائج تجريبية ري اهظمرتضى دمحم أ * رضا تفريشي ، ب ، زوروا خان جـ ، حميد رضا زيايفر أ ، موجاتبا غودسي أ ، ماثيو فرانشيتك جـ ، كارولوس غريغورياديس جـ الورقة مبدئيًا النماذج التجريبية الحالية التي تتنبأ بزيادة في ارتفاع منسوب أو زيادة ضغط تستعرض هذه : الملخص السوائل البترولية ثنائية الطور في المضخات الغاطسة الكهربائية ؛ ثم تقترح نموذًجا بديًًل ، وهو عبارة عن شبكة عصبية أن النماذج التجريبية للمضخات الغاطسة الكهربائية صناعية متتالية متصلة اتصال كامل لخدمة نفس الغرض. علما ب مستخدمة على نطاق واسع ؛ في حين أن النماذج التحليلية لم تستخدم بعد على نطاق واسع في الممارسة العملية بسبب من مدى تعقيدها أو اعتمادها على افتراضات مبسطة للغاية أو غير دقيقة. ويتم تجريب الشبكة المتصلة بالكامل و التحقق صحتها باستعمال نفس البيانات المستخدمة في تطوير عدد من النماذج التجريبية ؛ ومع ذلك ، فإن النماذج المطورة تعطي دقة أعلى من النماذج التجريبية المذكورة أعًله. فمتوسط خطأ التنبؤ المطلق للمتتالية المتصلة بالكامل ، و تبين أن متوسط ٪ من النموذج الحالي التجريبي 86المتصلة اتصال كامل غير المستخدمة في التجربة ، أقل بنسبة البيانات للشبكة التجريبية األكثر دقة . سوائل ؛الشبكات العصبية االصطناعية المتتالية ؛النماذج التجريبية ؛لمضخات الغاطسة الكهربائية : االمفتاحية الكلمات .البترول متعددة األطوار * Corresponding author’s e-mail: mmzahery@gmail.com DOI: 10.24200/tjer.vol16iss2pp77-86 mailto:mmzahery@gmail.com The Journal of Engineering Research (TJER), Vol. 16, No. 2 (2019) 77-86 NOMENCLATURE a Fixed value of a variable b,B Bias in the FCC C Weight matrix of FCC cascade connection E Training Error Ei Model Parameter f Activation function g Gravity acceleration G Error gradient vector H Head (m) H Hamiltonian Matrix J Model Parameter n Number N Rotational Speed (rpm) p Pressure (MPa) q Flow rate (lpm, litres per minute) T,W FCC weight matrices v Variable x Unknown parameter Greek Letters α Gas void ratio β Flow parameter of the affinity law δ Head parameter of the affinity law η A Coefficient in FCC training θ A vector of FCC parameters λ A coefficient in FCC training φ An indicator of surging Indices d Data g Gas in Input l Liquid m Mixture of gas and liquid p Parameters Abbreviations ANN Artificial Neural Network ESP Electrical Submersible Pump FCC Fully Connected Cascade ANN LSE Least Square of Errors 1. INTRODUCTION Electrical Submersible Pumps (ESPs), invented in 1927, are the second prevalent artificial lift methods in the petroleum industry (Machado et al. 2019) and are employed for downhole drilling operations at various flow rates (Mohammadzaheri et al. 2016; Zhu et al. 2018). Selection of ESP size is a crucial task, because over- or under-sizing leads to premature equipment failure or inadequate oil recovery, respectively. ESP manufacturers provide curves to be used in choosing the right size of ESPs. However, these curves are valid when only liquid is pumped. In some reservoirs, ESPs should pump two-phase flow with high gas content. In this case, instead of the curves, the designer needs models which predict the head of ESPs with fluids comprising of gas and liquid. These models are known as ‘head predicting’ models, which may accompany supplementary models (widely known as ‘critical’ models) providing flow information, e.g. border of surging, within the ESP ( Barrios and Prado 2011). Critical models are largely used to define the validity area of a head-predicting model. Head-predicting models with no accompanying critical model are considered to be valid for the entire operating area. Except for head-predicting and critical models, some other models have also been developed for side purposes such as estimation of gas bubble size by Zhu and Zhang (2017), or in-situ gas volume fraction by Zhu and Zhang ( 2016), in ESPs pumping two-phase flows; these models are outside the scope of this paper. Analytical and empirical methods have been employed to develop head predicting models for two-phase flows in ESPs. Analytical head predict- ing models have been derived based on mass and momentum balances (Sachdeva 1988; Sun and Prado 2005). However, they are based on unrealistic assumptions and/or oversimplification of complex physics of two-phase flows within the ESP. Therefore, analytical models are not considered an ideal option to model two-phase flow in ESPs, and empirical models are widely trusted alternatively (Zhou and Sachdeva 2010; Mohammadzaheri et al. 2018). Section 2 of this article reviews prevailing empirical models. Section 3 reports the development of a cascade artificial neural network to predict ESP head for two-phase fluids. The proposed model is shown to be valid for a range of operating conditions. 2. REVIEW OF ESP EMPIRICAL MODELS In the literature, six empirical models have been reported to estimate ESPs head or pressure for two- phase fluids in steady state situation. Some have their own critical models. Two major sources have supplied the experimental data for development of empirical models (Lea and Bearden 1982; Cirilo 1998). In this section, the models are classified on the basis of data source. Group 1: the models developed with the data collected from diesel fuel/carbon dioxide mixtures. Group 2: the models developed with the data of experiments carried out on water/air mixture. Group 1 models are more impor- tant for the oil industry, as the diesel fuel/ carbon dioxide mixture is more similar to petroleum fluids. 2.1 Group 1 Group 1 models are based on the data presented in Lea and Bearden ( 1982). These data cover an extensive range of gas void fractions (0 to 50%) and intake pressures (0.345 to 2.758 MPa). These data 78 M. Mohammadzaheri, R. Tafreshi, Z. Khan, H. Ziaiefar, M. Ghodsi, M. Franchek, and K. Grigoriadis are the result of 12 series of experiments; each with a particular pressure and flow rate. At each series, gas void fraction has changed and head generated by a number of ESP pumps (based on ESP type) has been recorded. Both the mixture of air and water and the mixture of carbon dioxide and diesel fuel have been used in these experiments. In this group, Models 1 and 3 directly predict head; whereas, Model 2 predicts pressure increase and needs (1) to estimate head, Hm, out of pressure increase,  p: where ρ and g represent fluid density and gravity acceleration, respectively. Index m refers to mixture of gas and liquid. Model 1 This model was developed by Turpin et al in 1986 (Turpin et al. 1986) and is claimed to be valid both for air/water and CO2/diesel fuel mixtures: where ql and qg are liquid and gas flow rates, pin is input pressure in MPa, and Hl is the head from manufacturer’s catalogue for a liquid flow rate of ql + qg. To find unknown parameters (x1 and x2, identified as 16.47 and -2.84), (2) can be re-written as or with use of head, flow rate and pressure information from experiments, (3) would be a linear equation with two unknowns. Least square of error (LSE) method can straightforwardly solve such equations (Mohammadzaheri et al. 2009). All the empirical models of group1 are convertible to a linear equation similarly. Model 1 is valid where the pump is stable or ϕ<1, where Model 2 Sachdeva et al. proposed this model in 1992 Sachdeva et al. (1992): where is pressure increases in MPa for 6- or 8- stage pumps (depending on the pump type) (Zhou and Sachdeva 2010). α is gas void fraction, ie. the ratio of free gas volumetric rate (qg) to the total volumetric rate (qm): Similar to Model 1, (5) can be converted to a linear equation solvable by LSE method. Parameters of (5) are listed in Zhou and Sachdeva ( 2010) for multiple stages of ESPs. Model 2 has no accompanying critical model; thus, it is assumed to be valid for the whole operating area. Model 3 This model was offered by Zhou and Sachdeva in 2010 (Zhou and Sachdeva 2010) where C is pressure unit coefficient, eg. 1, 1000, 0.145 or 145 for psi, ksi, kPa or MPa. Hmax and qmax are nominal maximum head and flow rate of the pump. Advantageously, use of Hmax and qmax can make head dimensionless in empirical models of ESPs, as originally proposed in Romero (1999). Moreover, use of aforementioned parameters allows the model to utilize catalogue information. Model 3 is indeed a revised version of Model 2, which estimates head instead of pressure, alike other empirical models developed in 1999 onwards. The output of (7) is evidently the maximum head when flow rate is zero. Parameters of (7) are listed in Zhou and Sachdeva (2010). K2 and K3 in Models 2 and 3 merely depend on ESP type as to Zhou and Sachdeva (2010); however, can be considered as the gain of Model 3 as demonstrated in (8): This gain ( ) would depend on the pressure unit. As an alternative to remove this dependency, dimensionless pressure could be used. Model 3 has been validated only for stable operating areas before surging. (9) defines maximum stable flow rate for a given pair of pressure and gas void ratio as to Zhou and Sachdeva (2010): For 8 stages of I-42B radial ESP, K3c=3.7453, E7=- 0.07244, E8=0.318544.The unit for flow is lpm (litres per minute). 2.2 Group 2 The data (used to develop these models are presented in Cirilo (1998), where air is the only gas and water is the sole liquid. In this paper, Cirilio 79 The Journal of Engineering Research (TJER), Vol. 16, No. 2 (2019) 77-86 claimed that the gas void fraction at the border of surging solely depends on intake pressure: Two of group 2 models are valid at particular flow regimes: bubbly regime and elongated bubble regime (Estevam 2002; Gamboa and Prado 2010). Model 4 Romero(1999) analyzed the data of Cirilo (1998), and suggested a new model to predict head (Model 4) in which its validity area is defined by (12): Oddly, gas void ratio, gas volumetric flow rate or intake pressure play no role in this model. Critical model of (12) determines both maximum gas void ratio and minimum liquid flow rate: Model 5 Duran and Prado (2003) suggested long-known homogenous model of (13) or (14) for bubbly regime: It is worth mentioning that (14) is twin of (13). The only difficulty to use (13) is the need for density. To address this weakness and ease calculations, Duran and Prado (2003) presented the following closure correlation: (6), (14) and (15) can find any of ql, qg, qm, ρl , ρg , ρm and α based on others. (16) is the critical model of the homogenous model: The set of (16) and (6) determines maximum gas flow rate in bubbly regime; that is, at any higher flow rate, the homogenous model is invalid. Model 6 Duran and Prado suggested a head-predicting model peculiar to the elongated bubbly regime (Duran and Prado 2003): Both pressure and density are absent in this model. LSE technique has been employed to find constants -0.0224 and -0.0103. Model 6 is accompanied by a critical model (18). The set of (18) and (6) provides with the minimum gas flow rate or gas void fraction in elongated bubble regime: ( ( ) ) ( ( ) ) ( ) 2.3 Unconsidered Inputs to Empirical Models The first overlooked input of Models 1-6 is rotational speed. All these models have been developed with the data collected at a single rotational speed of 3500 rpm. Hence, these empirical models are valid only at this rotational speed. Affinity laws of (19) are used to extend the validity area of empirical models. These laws are developed with the data collected at different rotational speeds (Zhou and Sachdeva 2010): where N is rotational speed in rpm, β=1 and 0.8 have been suggested for liquid and two-phase fluids respectively, and δ=2 has been suggested for both (Zhou and Sachdeva 2010). Temperature is the other overlooked input to Models 1-6. The reason is nonexistence of experimental data which reflect temperature effect. Experiments on ESPs pumping liquid have shown that that temperature has a significant effect only if numerous (>100) pump stages are used (Kirvelis and Davies 2003). 2.4 Review Summary Two groups of empirical models (Models 1-3 and Models 4-6) for two-phase fluids in ESPs were reviewed in this section. The parameters of these models have been identified using experimental data of air/water (both groups) and carbon dioxide/diesel fuel (group 1) mixtures. These models estimate head (Models 1, 3-6) or pressure increase (Model 2) generated by ESP systems, and are supplemented by five critical models defining their validity area. The validity area of an empirical model without a critical model is the operating area where its associated data have been gathered. Head/pressure predicting models, except for Model 80 M. Mohammadzaheri, R. Tafreshi, Z. Khan, H. Ziaiefar, M. Ghodsi, M. Franchek, and K. Grigoriadis 2, have parameters which can be found in the catalogue. Majority of models have three inputs: the first is either intake pressure or density and the second and third are two of ql, qg,, qm or α. Pump rotational speed and temperature have been overlooked in empirical modelling so far. All empirical models, except for Model 5, are linear equations with few unknowns or can be converted to such linear equations through logarithm. As a result, straightforward technique of LSE can be employed to identify handful unknown parameters of models. However, this research shows that the cost of this simplicity is loss of accuracy. In this research, a complex mathematical architecture, namely fully connected cascade (FCC) architecture, is used, which is a type of artificial neural networks (ANNs) with the most powerful architecture for system identification (Hunter et al. 2012). 3. FCC DEVELOPMENT An FCC was designed and developed to model mixtures of carbon dioxide and diesel fuel pumped by eight stages of an I-42B radial ESP up to head of 16.76 m, detailed in Lea and Bearden (1982). The FCC was developed with the same data as Models 1, 2 and 3 and is also comparable with 5. Motivated by a comparative review of section 2, the FCC has one output, Hm , and three inputs: pin, qm and α. As depicted in Fig.1, the FCC has a hidden layer with sigmoid activation functions of f, presented in (20), and weight matrix of W: Five neurons have been considered for the hidden layer. This choice is based on a multilayer perceptron developed for a similar purpose (Mohammadzaheri et al. 2015) and the fact that the number of hidden layer neurons in FCCs should be less than this number in multilayer perceptron’s (Hunter et al. 2012). Fig. 1 and Eq. (21) illustrate the proposed FCC: where W and T are the matrices of the first and the second layer weights, C is the matrix of cascade connection weights. B and b represent biases of the first and the second layers, respectively. FCC model of (21) has 29 parameters in total. 111 data sets have been used to develop and cross-validate the FCC model: 69 data sets for initialization and identification of parameters, 25 sets for over- fitting prevention (also known as validation) and 17 sets for cross-validation or test. The process of parameter identification (known as training) is iterative as detailed in Appendix A. At each iteration of FCC training, the real and model outputs for 69 sets of training data and 25 sets of validation data were calculated; mean of square of their discrepancies were considered as the training and validation error. Inconsistency in trend of these dual errors is as a sign of over-fitting and should trigger to end training. Over-fitting diminishes the generality of ANNs (Mohammadzaheri et al. 2007). Afterwards, the error was similarly calculated for 17 sets of test data, used neither in training nor in validation. A reasonably small test error ensured cross-validation. 4. EXPERIMENTAL ASSESSMENT OF MODELS This section compares head prediction error in different models and operating areas. Mean and maximum absolute test error of the FCC are 2.73% and 7.46% of maximum fluid head (16.76 m), respectively. Figure 2 presents head prediction error of different models/operating areas for test data, Table 1 presents these information for the whole experimental data. Models 4 and 6 are not presented because their parameters have been identified using experimental data of air-water mixture; thus, these models are not comparable with the developed FCC. Models 1 and 3 have critical models defining their validity or stability area, where surging absents. Surprisingly, stable areas of these models do not match. Models 2 and 5 have no validity area in terms of input pressure and gas void fraction. The background of cells in Table 1 reflects models validity, cells with white, light grey and dark grey background show the operating areas where the model is valid, partly valid (in some flow rates) or invalid, respectively. FCC is considered valid for the whole operating area due to cross-validation. Figures 3-5 show the average head prediction error for test data at each operation condition or «intake pressure, gas void fraction» pair. Figures 3-5 exhibit the following results:  In 9 out of 11 operating conditions, FCC provides the most accurate head prediction. In two other operating conditions, FCC stands second after Models 3 and 1, respectively. Parameter identification of an FCC (alike any other empirical model) is an off-line processes and should be performed only once, unless mechanical/geometrical properties of the system considerably change. Eqs. (20 and 21) can be solved using any programming languages used by computer or even engineering calculators.  The homogenous model of 5 performs better at higher intake pressures.  Fig. 3-5 (presenting the test data) are consistent with Table 1 (presenting the whole data) in Model 2 has the weakest performance even weaker than homogenous model of 5. 81 The Journal of Engineering Research (TJER), Vol. 16, No. 2 (2019) 77-86 Figure 1. A schematic of the employed cascade artificial neural network. Figure 2. Mean of absolute test error of different models. Table 1. Absolute prediction error in m for different models/operating areas, with use of all experimental data. Pin α Model 1 Model 2 Model 3 Model 5 FCC 0.34 10% 4.4794 7.3481 1.5249 1.4201 0.3301 0.34 15% 3.6546 3.5022 2.2303 2.6305 3.6848 0.2761 0.34 20% 2.6291 2.0887 4.8159 0.2893 0.34 30% 2.0193 1.5408 1.3218 5.0083 0.4958 0.34 40% 0.7112 0.3429 0.9351 5.3848 0.1243 0.69 10% 0.9770 6.8847 1.2913 1.7243 0.2952 0.69 15% 1.4056 4.0781 1.8101 1.9155 0.4395 0.69 20% 1.5108 3.1795 2.0356 2.5146 0.2699 0.69 30% 3.1539 2.3640 1.4171 3.0655 0.7127 0.69 40% 1.9507 1.1623 0.8797 3.5723 0.4273 2.76 30% 1.1361 3.0234 1.7786 1.6668 0.3354 2.76 40% 0.8983 2.5197 1.3117 1.3563 0.2370 2.76 50% 2.7563 2.2885 1.7650 1.7355 0.3938 average 2.0986 3.1018 1.6101 2.9127 0.3559 5. CONCLUSION Head estimation of two-phase petroleum fluids in ESPs is widely performed using empirical models, and analytical and numerical models have yet to be trusted for this task. Analytical models are based on idealistic assumptions and demand hardly accessible information of fluid. This article first critically reviewed the existing popular empirical models, then proposed a fully connected cascade artificial neural network as a substitute of the reviewed empirical models. The developed FCC, provided a superior estimation accuracy. 82 M. Mohammadzaheri, R. Tafreshi, Z. Khan, H. Ziaiefar, M. Ghodsi, M. Franchek, and K. Grigoriadis Figure 3. The trend of average head prediction error respect to gas void fraction for different models at intake pressure of 0.34 MPa, for test data. Figure 4. The trend of average head prediction error respect to gas void fraction for different models at intake pressure of 0.69 MPa, for test data. Figure 5. The trend of average head prediction error respect to gas void fraction for different models at intake pressure of 2.78 MPa, for test data. CONFLICT OF INTEREST The authors declare no conflicts of interest. FUNDING This work was supported by NPRP grant from the Qatar National Research Fund (a member of Qatar Foundation), grant number is 7-1114-2-415. REFERENCES Barrios L. Prado M . G . (2011), Modeling two- phase flow inside an electrical submersible pump stage. Journal of Energy Resources Technology, 133(4): 1-10. Cirilo, R. (1998), Air-water flow through electric submersible pumps, a Master dissertation at the Department of Petroleum Engineering, University of Tulsa, USA. Duran J. M. Prado M.G. (2003), ESP stages air-water two-phase performance-modeling and experiment- tal data. SPE Report, No. 87626. Estevam V. (2002), A phenomenological analysis about centrifugal pump in two-phase flow operation. A PhD thesis at Faculdade de Engenharia Mecânica, Universidade Estadual de Campinas, Brazil. Gambo J. Prado M.G. (2010), Visualization study of performance breakdown in two-phase performance of an electrical submersible pump, International Pump Users Symposium, Houston, USA. Hunter D.H. Yu, M.S. Pukish III, J. Kolbusz and B. M. Wilamowski (2012), Selection of proper neural network sizes and architectures - A comparative study. IEEE Transactions on Industrial Informatics, 8(2): 228-240. Kirvelis R. Davies D. (2003), Enthalpy balance model leads to more accurate modelling of heavy oil production with an electric submersible pump. Chemical Engineering Research and Design, 81(3): 342-351. Lea J. F. Bearden J. (1982), Effect of gaseous fluids on submersible pump performance. Journal of Petroleum Technology 34(12): 922-930. Machado, A. P. F. Resende C.Z. Cavalieri D.C. (2019), Estimation and prediction of motor load torque applied to electrical submersible pumps. Control Engineering Practice 84: 284-296. Mohammadzaheri M.A. AlQallaf Ghodsi M. Ziaiefar H. (2018), Development of a fuzzy model to estimate the head of gaseous petroleum fluids driven by electrical submersible pumps. Fuzzy Information and Engineering 10(1): 99-106. Mohammadzaheri M. Chen L. (2008), Intelligent modeling of MIMO nonlinear dynamic process plants for predictive control purposes. The 17th World Congress of The International Federation of 83 The Journal of Engineering Research (TJER), Vol. 16, No. 2 (2019) 77-86 Automatic Control, Seoul, Korea. Mohammadzaheri M. Chen L. (2010), Intelligent predictive control of model helicopters' yaw angle. Asian Journal of Control. 12(6): 1-13. Mohammadzaheri M. Chen L. A. Ghaffari and J. Willison (2009), A combination of linear and nonlinear activation functions in neural networks for modeling a de-superheater. Simulation Modelling Practice and Theory. 17(2): 398-407. Mohammadzaheri M. Mirsepahi A. Asef-afshar O. and Koohi H. (2007), Neuro-fuzzy modeling of superheating system of a steam power plant. Applied Mathematical Sciences. 1: 2091-2099. Mohammadzaheri M. Tafreshi R. Khan Z, Franchek M. and Grigoriadis K. (2015), Modeling of petroleum multiphase fluids in ESPs, an intelligent approach. Offshore Mediterranean Conference, Ravenna, Italy. Mohammadzaheri, M., R. Tafreshi, Z. Khan, M. Franchek and K. Grigoriadis (2016), An intelligent approach to optimize multiphase subsea oil fields lifted by electrical submersible pumps. Journal of Computational Science 15: 50-59. Moré J.J. (1978), The Levenberg-Marquardt algorithm: implementation and theory. Lecture Notes in Mathematics, vol. 630, Springer, Berlin, Heidelberg: 105-116. Nguyen D. Widrow B. (1990), Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights. International Joint Conference on Neural Networks. San Diego, USA, 17-21 June, 1990. Romero M (1999), An evaluation of an electrical submersible pumping system for high GOR wells, a Master dissertation at University of Tulsa, USA. Sachdeva R (1988), Two-phase flow through electric submersible pumps, a PhD thesis at University of Tulsa, USA. Sachdeva R. Doty D. and Schmidt Z. (1992), Performance of axial electric submersible pumps in a gassy well, SPE Rocky Mountain Regional Meeting. Casper, USA. Sun D. Prado M. (2005), Modeling gas-liquid head performance of electrical sub mersible pump s. Journal of Pressure Vessel Technology, 127(1): 31- 38. Turpin J.L. Lea J.F. Bearden J.L. (1986), Gas-liquid flow through centrifugal pumps—correlation of data, The Third International Pump Symposium, College Station, Texas, USA. Wilamowski (2012), Selection of proper neural network sizes and architectures—a comparative study. IEEE Transactions on Industrial Informatics. 8(2): 228-240. Zhou D. Sachdeva R . (2010), Simple model of electric submersible pump in gassy well. Journal of Petroleum Science and Engineering. 70(3): 204- 213. Zhu J. Zhang H.Q. (2016), Mechanistic modeling and numerical simulation of in-situ gas void fraction inside ESP impeller. Journal of Natural Gas Science and Engineering. 36: 144-154. Zhu J. Zhang H.Q. (2017), Numerical study on electrical-submersible-pump two-phase performance and bubble-size modeling. SPE Production & Operations. 32(3): 267-278. Zhu J. Zhu H, Wang Z.J. Zhang, R. Cuamatzi- Melendez, J. A. M. Farfan and H.Q. Zhang (2018), Surfactant effect on air/water flow in a multistage electrical submersible pump (ESP). Experimental Thermal and Fluid Science 98: 95- 111. 84 84 M. Mohammadzaheri, R. Tafreshi, Z. Khan, H. Ziaiefar, M. Ghodsi, M. Franchek, and K. Grigoriadis Appendix A Training of the FCC was composed of four major tasks: 1. Data normalization 2. Error function definition 3. Parameters initialization 4. Parameter tuning A.1 Data Normalization The difference of magnitude scale among input or output columns of data negatively affects the parameter identification (training) process and should be avoided through normalization, a process to assure input or output data columns have almost same scale of magnitude (Mohammadzaheri and Chen 2008). In this research, for the purpose of normalization, the input and output data columns were mapped into the range of [-1 1]. In practice, the input(s)/outputs(s) of an FCC, trained with the mapped data, should be mapped/de- mapped into the real range. A.2 Error Function Definition The error function represents the discrepancy between the FCC and real system outputs for identical inputs. Mean of squared errors was used as the error function in this research, presented as (A.1): where hat (^) shows the estimated value by the FCC and Hm is the measured head of the mixture. n is the number of data sets. A.3 Parameters Initialization The FCC shown in Fig. 1 has a sigmoid activation function in the hidden layer. As depicted in Fig.A.1, a sigmoid function is nearly linear for an interval; outside this interval, the output is nearly fixed or saturated (Mohammadzaheri et al. 2016). Figure A.1. The sigmoid function presented by (6) and its linear interval. In practice, any input to a sigmoid activation function, with a value outside the linear interval (shown in Fig. A.1), has trivial influence on the output. Within the structure of an FCC, it means the input has trivial impact on the error and training process. Nguyen-Widrow algorithm suggests initial values for FCC parameters so that inputs to the sigmoid functions lie within the linear interval. Random functions are also used in this algorithm to avoid repeated initial weights (Nguyen and Widrow, 1990). A.4 Parameter Tuning After initialization, the parameters are tuned iteratively to minimize the error (detailed in section A.2). The error is influenced by 29 parameters of (21). Hence, E can be presented as E(θ), where θ is a vector of 85 85 The Journal of Engineering Research (TJER), Vol. 16, No. 2 (2019) 77-86 all parameters of the FCC. The first step is to analytically derive the first and second derivatives of E(θ), error back propagation. In this research, Levenberg-Marquardt method, Eq.(A.2), detailed in (Moré 1978, Mohammadzaheri and Chen 2010), was used to minimize E(θ), calculated with 69 sets of the training data and (A.1). Levenberg-Marquardt method is based on approximation of the error function using Taylor series up to the second order derivatives: where G and J are the vector and matrix of partial derivatives of error respect to θ elements. Algorithms to find η and λ have been detailed in(Jang et al. 2006, Mohammadzaheri and Chen 2010). Appendix B Figures 2-5 and Table 1 are only concerned with the models’ error. They provide no information about real values of head. Figures B.1-2 tend to compensate this shortcoming through depicting measured/predicted head versus flow rate for two operating areas. Figure B.1. Experimentally measured and predicted head by different models versus flow rate at intake pressure of 2.76 MPa and gas void fraction of 50%. Figure B.2. Experimentally measured and predicted head by different models versus flow rate at intake pressure of 0.69 MPa and gas void fraction of 15%. 86